Python script for automatic synchronization based on inotify

[Rene Mayrhofer]

Dear Antonio,

On 11.03.2011 10:16, Antonio Ospite wrote:
>> What does it do?
>> ------------------------
>> Automatically keep DVCS repositories in sync whenever changes happen by automatically committing and pushing/pulling.
> I like this distributed sync approach, however I can see cases where
> automatic pushes are not wanted, but that should be easy to handle as
> the problems of monitoring changes (your 1.-4. below) and the one of
> synchronizing repositories (your 5. below) are quite independent. Do you
> think synchronization could be triggered manually using some hook on
> the central repository or on the VCS clients?
>
Yes, I think this is possible, and I fully agree that a future version 
should allow to split these two tasks. My current (TODO) plan is to make 
it possible for the user to interact with the automatic commit process 
(e.g. by specifying a commit message, even in hindsight, and/or to 
"veto" committing any new files that were picked up by inotify).

Push/pull synchronization can easily re-use my current XMPP bot, and the 
inotify part is indeed half separated from the sync logic.

> Maybe you could try to separate those problems more clearly in the
> program.
>
> I can see the vcs-home problem partitioned like this:
>    a. Track changes (keeping host/user settings separate from sharable
>       ones)
>    b. Monitor/Store changes (automatic commits vs. manual commits)
>    c. Distribute/Synchronize changes
>    d. Notify changes to physical users
>
> I'd say your solution for c. is my preferred one for the moment, are
> you using XMMP resources to differentiate logins for the same account?
> Naively I'd use some combination of user/host/repository as a resource
> id (http://wiki.xmpp.org/web/Jabber_Resources).
Yes, I use different XMPP resources for each login (the machine/host 
name, mostly) to distinguish them within the same XMPP account. It took 
a bit of trial&error with the Python XMPP and Jabberbot modules to get 
that message-to-self part right (and an extension to Jabberbot along the 
way), but it is still my preferred solution as well. Should work across 
any NAT etc. and can be set up privately for closed repo synchronization.

Concerning structuring the application, a) and b) will need to be 
tightly integrated, but c) and d) can be separated well. I am currently 
unsure how best to achieve that in terms of Python components, though. 
If you have any suggestions (Python classes vs. Python modules vs. 
multiple processes and IPC), I am more than happy to discuss them.


> Just a note about coding style you could try to follow PEP-8 guidelines
> for python code http://www.python.org/dev/peps/pep-0008/
> There are automatic style checkers to help you with that.
Do you happen to know of any integration with the Eclipse pydev plugin 
(yes, I do most of my work in Java/Scala these days, so Eclipse is my 
standard IDE at the moment...)? I fully admit that I am new to Python 
coding and therefore welcome improvements concerning style, structure, 
and documentation best practices.


PS: I still see the Python script as a proof-of-concept and for rapid 
prototyping of new ideas. All the concepts that have matured 
sufficiently in this code should hopefully make it into more complete 
and end-user ready tools like SparkleShare for the nice GUI parts. I 
also plan to - at some point in the future when I will have more time 
for new projects - redo the tool in Java (with JGit) so that it can 
eventually run on Android. My final goal is to have a complete Dropbox 
replacement between all my machines including my mobile phone. Sharing 
file could e.g. be done via any of the git web interfaces, although the 
required ACL part is still completely unspecified at this point.

PPS: Would anybody actually be interested in using this with a DVCS 
other than git? I use git myself for performance reasons, but have done 
some testing with mercurial as well (as you can see in the config file).

best regards,
Rene