git annex get performance issues with rsync
joey at kitenet.net
Wed Jan 18 17:09:50 CET 2012
Adam Spiers wrote:
> One of my USB drives just died, so I'm doing a 'git annex get --not
> --copies 1' to re-attain data redundancy. It seems that a new rsync
> instance is invoked for each file? In my case, I have thousands of
> photos which are big enough to be worth annexing but still not
> individually huge, so it seems that the overhead of each rsync
> invocation is significantly impacting throughput. A quick empirical
> test showed in 20 seconds, that 'git annex get' managed to transfer 11
> photos, whereas a single (manual) rsync run transferred 33. Is this
> easily fixable?
No, it's on the todo list but very far down it.
You can enable ssh's connection sharing though. (ControlMaster)
see shy jo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 828 bytes
Desc: Digital signature
More information about the vcs-home