[Arm-netbook] offlineimap syncing off of gmail...

[Philip Hands]

Luke Kenneth Casson Leighton <lkcl at lkcl.net> writes:

> after trying for 90 minutes to connect to accounts.google.com last
> night, with over twenty refreshes, finally obtaining just the HTML of
> a page then having to spend ANOTHER hour just to get the matching
> CSS... only to run into HTTP proxy problems and having to clear the
> cache and DO IT ALL OVER AGAIN....
>
> ... i'm absolutely sick and tired of the fucking fucked internet here,
> how it's interfering with my ability to communicate, get source code
> and much more.
>
> so, i'm converting to offline imap, setting up the "prayer webmail"
> client, a cyrus imap server, and syncing everything to my laptop now i
> have the spare space on the 500GB SSD.

If you can deal with emacs, have a look at notmuch

Even if you cannot -- you should probably have a look at notmuch
(there's a vim-based thingumy, and there are other front ends too)

> the only thing is... well... look at the numbers:
>
> Folder INBOX [acc: lkcl]:
>  Copy message UID 143 (98/155030) Remote:INBOX -> Local
> Folder INBOX.bak.sent [remote name [Gmail]/Sent Mail] [acc: lkcl]:
>  Copy message UID 307 (116/55244) Remote:[Gmail]/Sent Mail -> Local
> Folder INBOX.bak.flagged [remote name [Gmail]/Starred] [acc: lkcl]:
>  Copy message UID 508 (161/3185) Remote:[Gmail]/Starred -> Local
> Folder INBOX.bak.important [remote name [Gmail]/Important] [acc: lkcl]:
>  Copy message UID 296 (114/51381) Remote:[Gmail]/Important -> Local
>
> that's nearly 10 GIGABYTES of mail over a ten year period (i created
> the gmail account in 2006 when i was running out of space on my laptop
> to hold all the email i had in mutt Maildirs).
>
> 55 *thousand* messages in the "sent mail" folder.  a hundred and fifty
> five thousand in the inbox.
>
> after hacking offlineimap so that it would only use HTTPS proxy for
> connecting to accounts.google.com but would connect direct to
> imap.gmail.com (using a fixed IP address because the china government
> MODIFIES the DNS entries to point to the wrong locations...)....
>
> ... i was able to initiate the download / sync process.  connectivity
> is so fucking slow it's about one message every... two to five
> seconds.  at this rate it's going to take TWO HUNDRED hours to sync
> the entire mailbox.  that's ten DAYS for god's sake.
>
> we may actually have to go stay in a hotel in hong kong (where they
> have a 100 mbytes / sec internet connection) just so i can sync all
> the email.

If you point notmuch at that it'll say something like:

   523,567 mails, that's not much mail...

and you'll be able to do similar searchy things to what you probably got
used to with google.

I tend to shunt my inbox into an archive directory so that I can use
dumb imap on my phone and not need to fill the phone with 110k of mails,
but I know I've got more mail than you're talking about, and searches
return useful results in sub-second mostly.

% find ./Maildir -type f | wc -l
559007
% du -hs Maildir
12G	Maildir

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/    http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,    GERMANY
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.phcomp.co.uk/pipermail/arm-netbook/attachments/20161216/c387f8f5/attachment-0003.sig>