Thursday, October 4, 2012

ReadyNAS NV+ backup to Amazon Glacier Part 1

My previous posts about planning to back up my ReadyNAS NV+ to DreamHost all came unstuck when I read the DH page explaining what they mean by "Unlimited storage". I'm sure this page was added after I agreed for my account to be moved to one of their new servers in exchange for "unlimited storage and bandwidth" back in 2007. Anyway, I'm not in a position to argue.

DH does not allow me to use my account for storing "non-website-related" files, such as copies of ebooks, videos and music files I've purchased, or in my case, created myself. For "personal files", they offer up to 50GB of free but not-backed up storage and then charge 10c/month for each GB over 50.

My current storage needs are around 1TB so this make DH's "personal files" storage an expensive $95/month.

Amazon's announcement of Glacier in August didn't register at the time as a cheap form of backup for anyone. I didn't read the fine print and simply assumed it was only for S3 users of which I am not. As I read further today I realised it's ideal for my requirements as part of a "belts and braces" backup strategy. 1TB would cost me $10/mth (about the same as I'm paying for my "unlimited" DH account). It's not important if it can take up to 4 hours to retrieve files. I will already have the local backup on my Drobo. It's only if I get a double disaster/failure hitting my home systems will I need to retrieve from Glacier. It'll take me more than four hours to replace my home hardware if there's a fire etc.

During the installation I learned that Duplicity can backup to Amazon S3 as one of its backends. It seems a pretty obvious addition to allow it to backup to Glacier. But when I Googled, I found lots of requests for such an addition but no note of it happening.

However I did discover glacier-cli which is built on an equally intriguing project called git-annex. Git-annex uses the git filesystem and commands without actually having to check the files into git. Git is really fast at identifying file differences and there's a lot of user command line knowledge that annex can take advantage of.

Work in progress and worth watching.

4 comments:

Unknown said...

I'm in the same boat as you are. Keep me posted!

Dennis J said...

Nice nice nice! Really longing for this one!

Adrian said...

Nice, looking forward to that solution. Having my Netgear ReadyNAS Duo directly backing-up data to cheap and reliable Glacier would be fantastic feature. Thanks for the work.

Rob said...

Hi. Did you make any progress with this? I've not got quite as much data as you, but 500GB of mostly family photos and videos is still quite enough that I want to protect it.