
{"id":127,"date":"2008-08-26T14:07:59","date_gmt":"2008-08-26T21:07:59","guid":{"rendered":"http:\/\/briantroy.com\/blog\/2008\/08\/26\/the-great-imap-migration\/"},"modified":"2008-08-26T14:07:59","modified_gmt":"2008-08-26T21:07:59","slug":"the-great-imap-migration","status":"publish","type":"post","link":"https:\/\/blogarchive.briantroy.com\/index.php\/2008\/08\/26\/the-great-imap-migration\/","title":{"rendered":"The great IMAP migration&#8230;"},"content":{"rendered":"<p>Late last week GoDaddy finally pushed me over the edge&#8230; I just couldn&#8217;t take it anymore. I&#8217;m all for combatting spam&#8230; I really am, but when 60% plus of the emails sent to me bounce as spam\/virus something has gone dreadfully awry.<\/p>\n<p>So, I had to search for a new email hosting provider &#8211; and while I was at it I could right one of the major (non-email bounce) issues I had with GoDaddy &#8211; no IMAP support.<\/p>\n<p>I reviewed several options&#8230; and came to the conclusion that Google Apps (for MY domain) would be the right choice. I only had one serious issue with this. I wanted gmail to provide my IMAP store. The challenge? How to migrate 20 accounts with all of the associated IMAP folders and emails to gmail.<\/p>\n<p>Turns out there is no simple answer unless you opt for a premier account with Google &#8211; which will run you $50\/user\/year. The premier account has an IMAP migration tool (but no backup&#8230; keep reading). But here is what I settled on:<\/p>\n<p><a href=\"http:\/\/www.linux-france.org\/prj\/imapsync\/\">imapsync<\/a> would handle moving my current IMAP store to Gmail. To save you some time here is the full command I used:<\/p>\n<p>\/usr\/local\/imapsync\/imapsync &#8211;syncinternaldates &#8211;host1 &lt;my_imap_server&gt; &#8211;user1 &lt;username&gt; &#8211;ssl1 &#8211;password1 &lt;pwd&gt; &#8211;host2 imap.gmail.com &#8211;port1 993 &#8211;port2 993 &#8211;ssl2 &#8211;user2 &lt;gmail user&gt; &#8211;password2 &lt;gmail pwd&gt; &#8211;prefix2 &#8216;[Gmail]\/&#8217;<\/p>\n<p>Now that I had all of the old email moved to Gmail I only had one other issue&#8230; What if Gmail &#8220;went away&#8221;.<\/p>\n<p>Minor Tanget:<\/p>\n<p>An associate of mine said &#8220;You don&#8217;t have to worry about it&#8230; Google will never go away. Backing up Google is silly.&#8221; &#8211; to which I respond:<\/p>\n<p>Isn&#8217;t that what we thought about aol?<\/p>\n<p>End of Tangent.<\/p>\n<p>So I needed to find a solution that would allow me to keep all of my Gmail IMAP folders and emails copied in a reasonably secure way &#8211; ensuring that if I ever needed to move away from Gmail I could without losing anything.<\/p>\n<p>Initially, I tried imapsync again to sync my Gmail data back down to my IMAP server. This, however, was not practical. imapsync&#8217;s primary attribute is not speed. It is not a high speed backup\/sync solution&#8230; it is a bullet-proof solution. That made it:<\/p>\n<ol>\n<li>Way too much overhead for my needs<\/li>\n<li>Way too slow for my needs<\/li>\n<li>it is written in Perl &#8211; personal thing here but ICK.<\/li>\n<\/ol>\n<p>I also looked at <a href=\"http:\/\/software.complete.org\/software\/projects\/show\/offlineimap\">offlineimap<\/a> (which I had looked at for the initial migration as well&#8230; but it has no ability to &#8220;dry run&#8221;- which was too big a chance to take). The advantage of offlineimap is that it &#8220;syncs&#8221; your IMAP to a Maildir (which &#8211; conveniently is my dovecot IMAP server&#8217;s file structure). This makes it much faster than doing IMAP to IMAP synchronization. Here are some other advantages:<\/p>\n<ol>\n<li>Runs as a daemon with defined intervals at which to sync accounts<\/li>\n<li>Supports multiple accounts<\/li>\n<li>Supports &#8220;quick sync&#8221; to only sync if a maildir has changed<\/li>\n<li>Python&#8230; giant step up from perl<\/li>\n<\/ol>\n<p>NOTE: Why wasn&#8217;t I concerned about no &#8220;dry run&#8221;? Because I was creating a backup of the GMail IMAP folders and emails in a simple directory on my server. If the run went bad&#8230; just delete the directory and try again. I was also certain I had told offlineimap NOT to delete or expunge anything from my GMail account.<\/p>\n<p>Again, to save you some time, here is my offlineimap config file:<\/p>\n<blockquote>\n<p>[general]<\/p>\n<p>accounts = brian.cosinity, brian.brianandkelly, brian.briantroy<\/p>\n<p>metadata = \/etc\/offlineimap\/.offlineimap<\/p>\n<p>maxsyncaccounts = 3<\/p>\n<p>ui = Noninteractive.Basic<\/p>\n<p>[Account brian.cosinity]<\/p>\n<p>localrepository = Local.brian.cosinity<\/p>\n<p>remoterepository = brian.cosinity.gmail<\/p>\n<p>autorefresh = 120<\/p>\n<p>quick = 10<\/p>\n<p>[Repository Local.brian.cosinity]<\/p>\n<p>type = Maildir<\/p>\n<p>localfolders = \/u02\/gmail-imap-backups\/brian.cosinity<\/p>\n<p>[Repository brian.cosinity.gmail]<\/p>\n<p>folderfilter = lambda foldername: foldername not in [&#8216;[Gmail]\/All Mail&#8217;]<\/p>\n<p>type = IMAP<\/p>\n<p>remotehost = imap.gmail.com<\/p>\n<p>remoteuser = &lt;gmail_account&gt;<\/p>\n<p>remoteport = 993<\/p>\n<p>ssl = yes<\/p>\n<p>remotepass = &lt;gmail_password&gt;<\/p>\n<p>[Account brian.briantroy]\n<\/p><\/blockquote>\n<blockquote><p>\n  localrepository = Local.brian.briantroy\n<\/p><\/blockquote>\n<blockquote><p>\n  remoterepository = brian.briantroy.gmail\n<\/p><\/blockquote>\n<blockquote><p>\n  autorefresh = 120quick = 10\n<\/p><\/blockquote>\n<blockquote><p>\n  [Repository Local.brian.briantroy]\n<\/p><\/blockquote>\n<blockquote><p>\n  type = Maildir\n<\/p><\/blockquote>\n<blockquote><p>\n  localfolders = \/u02\/gmail-imap-backups\/brian.briantroy\n<\/p><\/blockquote>\n<blockquote><p>\n  <br \/>\n  [Repository brian.briantroy.gmail]\n<\/p><\/blockquote>\n<blockquote><p>\n  folderfilter = lambda foldername: foldername not in [&#8216;[Gmail]\/All Mail&#8217;]\n<\/p><\/blockquote>\n<blockquote><p>\n  type = IMAP\n<\/p><\/blockquote>\n<blockquote><p>\n  remotehost = imap.gmail.com\n<\/p><\/blockquote>\n<blockquote><p>\n  remoteuser = &lt;gmail_account&gt;\n<\/p><\/blockquote>\n<blockquote><p>\n  remoteport = 993\n<\/p><\/blockquote>\n<blockquote><p>\n  ssl = yes\n<\/p><\/blockquote>\n<blockquote><p>\n  remotepass = &lt;gmail_password&gt;\n<\/p><\/blockquote>\n<blockquote><p>\n  [Account brian.brianandkelly]\n<\/p><\/blockquote>\n<blockquote><p>\n  localrepository = Local.brian.brianandkelly\n<\/p><\/blockquote>\n<blockquote><p>\n  remoterepository = brian.brianandkelly.gmail\n<\/p><\/blockquote>\n<blockquote><p>\n  autorefresh = 120\n<\/p><\/blockquote>\n<blockquote><p>\n  quick = 10\n<\/p><\/blockquote>\n<blockquote><p>\n  [Repository Local.brian.brianandkelly]\n<\/p><\/blockquote>\n<blockquote><p>\n  type = Maildir\n<\/p><\/blockquote>\n<blockquote><p>\n  localfolders = \/u02\/gmail-imap-backups\/brian.brianandkelly\n<\/p><\/blockquote>\n<blockquote><p>\n  [Repository brian.brianandkelly.gmail]\n<\/p><\/blockquote>\n<blockquote><p>\n  folderfilter = lambda foldername: foldername not in [&#8216;[Gmail]\/All Mail&#8217;, &#8216;[Gmail]\/System Email&#8217;]\n<\/p><\/blockquote>\n<blockquote><p>\n  type = IMAP\n<\/p><\/blockquote>\n<blockquote><p>\n  remotehost = imap.gmail.com\n<\/p><\/blockquote>\n<blockquote><p>\n  remoteuser = &lt;gmail_account&gt;\n<\/p><\/blockquote>\n<blockquote><p>\n  remoteport = 993\n<\/p><\/blockquote>\n<blockquote><p>\n  ssl = yes\n<\/p><\/blockquote>\n<blockquote><p>\n  remotepass = &lt;gmail_password&gt;\n<\/p><\/blockquote>\n<p>What this does is backup 3 gmail accounts (brian.cosinity, brian.briantroy, and brian.brianandkelly) to \/u02\/gmail-imap-backup\/&lt;folder&gt;. These accounts are backed up every 120 minutes. Also, every 10th sync is a full sync &#8211; so only once per day (every 20 hours) does offlineimap fully sync the accounts &#8211; the other syncs are &#8220;quick&#8221;.<br \/>\nYou can check out the <a href=\"http:\/\/software.complete.org\/software\/wiki\/offlineimap\">offlineimap wiki<\/a> for more information about each directive in this config file.<br \/>\nThis solution works for me. If you are looking for true dual synchronized IMAP servers you would have to go a bit further that what I&#8217;ve done here.<br \/>\nNow &#8211; with any luck &#8211; Google Gmail will never go down and I&#8217;ll never need any of this&#8230; but I&#8217;ll sleep better knowing it is there.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Late last week GoDaddy finally pushed me over the edge&#8230; I just couldn&#8217;t take it anymore. I&#8217;m all for combatting spam&#8230; I really am, but when 60% plus of the emails sent to me bounce as spam\/virus something has gone dreadfully awry. So, I had to search for a new email hosting provider &#8211; and&hellip; <a class=\"more-link\" href=\"https:\/\/blogarchive.briantroy.com\/index.php\/2008\/08\/26\/the-great-imap-migration\/\">Continue reading <span class=\"screen-reader-text\">The great IMAP migration&#8230;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12,31],"tags":[142,143],"_links":{"self":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/posts\/127"}],"collection":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/comments?post=127"}],"version-history":[{"count":0,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/posts\/127\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/media?parent=127"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/categories?post=127"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogarchive.briantroy.com\/index.php\/wp-json\/wp\/v2\/tags?post=127"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}