Slacy’s Blog

I’m running my own hand-crafted remote backup (rsync++) script to back up data from one machine to another.  But, it has some serious drawbacks, and I’m still looking for something better.  Here are my requirements:

• Runs on Ubuntu
• Is either:
  • Available in the standard distro (i.e. via apt-get)
  • A simple self-contained script (Python, Perl, Bash, etc.)
• Uses ssh/scp/rsync to transport files
• Is incremental (i.e. does “what’s changed from last time”, and only backs up that much)
• Handles file renames & moves without re-transmitting the underlying data.
• Puts files in a simple “directory by date” format.  No fancy databases, no fancy metadata required for restoring.
• Preserves file permissions & modification dates.
• Correctly handles both hard & soft symbolic links.
• Is client-side only.  In other words, I don’t want to have to maintain 2 copies of this script, one for the server, and one for the client.

Is there anything out there that meets these requirements?

Thanks Paul and Greg for the advice on backing up my GMail account.  Although fetchmail seems to work for some users, there are a couple of easier to use scripts out there.

Paul pointed me at Matt Cutts’ great blog post about using the getmail script to archive GMail.

Greg pointed me at libgmail which comes with some simple driver scripts to download all your content.  You can get it from the sourceforge.net page, and read more on the libgmail author’s website.

I’m running a big batch getmail run now, and putting all the messages in MailDir format.  The thing that I’m actually looking for is a way to preserve GMail’s label structure using hard or soft links in a MailDir-like format, for easy searching.   I suspect that libgmail will allow this, but I haven’t dug into it enough to know for sure.

There are many guides to using fetchmail to backup your gmail account, but I use Google Apps For Your Domain to host my mail.  So, I needed to craft up a slightly modified version of the fetchmail commandline:

$ fetchmail –ssl -p pop3 -vk –user slacy@slacy.com pop.gmail.com

The funny thing is that within a few minutes of running this script, I got e-mails from my friends saying “Hey, did you mean to send this to me” and “I got this weird message about delivery failure from you”.

Reading the fetchmail man page in more detail, it says that after fetching the mail, it delivers it via the local delivery agent.  In my case, I suspect that what’s happening is that it’s going to my local SMTP server, which is set up to forward back to GMail, and I think that somehow created the re-sending.

If you have any other ideas how or why a fetchmail run would re-send old messages to other recipeients, let me know.  I’m going to have to do a fair amount of digging before I re-run the script.