Mystery of why scp stalled solved!
But it's taken me many days to understand the solution. I've had to learn how to use WireShark and tcpdump (both useful to know at any time). Then I ran tcpdump on the NV+ while running scp to upload to DreamHost. Then I did the same thing on my MacBook Air and the only difference I could see when I loaded the pcap files into WireShark was that the NV+ was being forced to packet split and eventually the ACK response from DH got lost. But why was it packet splitting? The MTU of the modem router is 1492. The MBA has no problem working this out and sends the correct sized packets. Why can't the NV+?
Still didn't make any sense to me. I tried a number of times switching off Jumbo packets/frames on the NV+ and setting the MTU to 1492 but scp still was forced to packet split. In my frustration I started looking for other backup solutions and discovered Amazon Glacier (more on that in a later blog).
So I started closing all the TABs I had open in Chrome concerning scp stalling and noticed a link I hadn't seen before on this page. I presume I didn't understand it when I saw it and ignored it. The link is to an FAQ for the "snail book". And as I read through it I now understood what I was seeing in the tcpdump file.
So I tried to change the NV+ MTU to 576 as suggested but Frontview limits me to 1400 upto 1500, so I entered 1400 and re-ran scp and it works! Seems the default 1492 in the NV+ Frontview input box is simply wrong for ADSL, but OK for internal LAN transfers (1492+8byte header = 1500 (default TCP packet size)).
Updated command to run Duplicity (The following should be one line.)
duplicity --exclude-globbing-filelist=$HOME/exclude.list
--archive-dir=/c/temp/cache/duplicity
--tempdir=/c/temp/tempsig
--use-scp /media/Music scp://dh/backup/nas
Note that I've specified the archive and temp directories to be on /c/. It is critical not to use the default directories on root filesystem for these. Trying to squeeze a backup of 50GB into a 2GB filesystem is sure to lead to disaster.
1 comment:
I'm currently backing up to DreamHost as well. While DH requires you to have all of your files web accessible, it doesn't mean that they need to be publicly accessible. All of my photos (60 GB+) are up on DH in ~/data/photos/, a non-web accessable dir, but then I pull the photos into Gallery (http://gallery.menalto.com/) and have all the permission locked down. So it doesn't volate their terms, and I'm a happy camper.
Post a Comment