Python script for automatic synchronization based on inotify
rene at mayrhofer.eu.org
Mon Mar 14 19:49:24 CET 2011
On 11.03.2011 10:16, Antonio Ospite wrote:
>> What does it do?
>> Automatically keep DVCS repositories in sync whenever changes happen by automatically committing and pushing/pulling.
> I like this distributed sync approach, however I can see cases where
> automatic pushes are not wanted, but that should be easy to handle as
> the problems of monitoring changes (your 1.-4. below) and the one of
> synchronizing repositories (your 5. below) are quite independent. Do you
> think synchronization could be triggered manually using some hook on
> the central repository or on the VCS clients?
Yes, I think this is possible, and I fully agree that a future version
should allow to split these two tasks. My current (TODO) plan is to make
it possible for the user to interact with the automatic commit process
(e.g. by specifying a commit message, even in hindsight, and/or to
"veto" committing any new files that were picked up by inotify).
Push/pull synchronization can easily re-use my current XMPP bot, and the
inotify part is indeed half separated from the sync logic.
> Maybe you could try to separate those problems more clearly in the
> I can see the vcs-home problem partitioned like this:
> a. Track changes (keeping host/user settings separate from sharable
> b. Monitor/Store changes (automatic commits vs. manual commits)
> c. Distribute/Synchronize changes
> d. Notify changes to physical users
> I'd say your solution for c. is my preferred one for the moment, are
> you using XMMP resources to differentiate logins for the same account?
> Naively I'd use some combination of user/host/repository as a resource
> id (http://wiki.xmpp.org/web/Jabber_Resources).
Yes, I use different XMPP resources for each login (the machine/host
name, mostly) to distinguish them within the same XMPP account. It took
a bit of trial&error with the Python XMPP and Jabberbot modules to get
that message-to-self part right (and an extension to Jabberbot along the
way), but it is still my preferred solution as well. Should work across
any NAT etc. and can be set up privately for closed repo synchronization.
Concerning structuring the application, a) and b) will need to be
tightly integrated, but c) and d) can be separated well. I am currently
unsure how best to achieve that in terms of Python components, though.
If you have any suggestions (Python classes vs. Python modules vs.
multiple processes and IPC), I am more than happy to discuss them.
> Just a note about coding style you could try to follow PEP-8 guidelines
> for python code http://www.python.org/dev/peps/pep-0008/
> There are automatic style checkers to help you with that.
Do you happen to know of any integration with the Eclipse pydev plugin
(yes, I do most of my work in Java/Scala these days, so Eclipse is my
standard IDE at the moment...)? I fully admit that I am new to Python
coding and therefore welcome improvements concerning style, structure,
and documentation best practices.
PS: I still see the Python script as a proof-of-concept and for rapid
prototyping of new ideas. All the concepts that have matured
sufficiently in this code should hopefully make it into more complete
and end-user ready tools like SparkleShare for the nice GUI parts. I
also plan to - at some point in the future when I will have more time
for new projects - redo the tool in Java (with JGit) so that it can
eventually run on Android. My final goal is to have a complete Dropbox
replacement between all my machines including my mobile phone. Sharing
file could e.g. be done via any of the git web interfaces, although the
required ACL part is still completely unspecified at this point.
PPS: Would anybody actually be interested in using this with a DVCS
other than git? I use git myself for performance reasons, but have done
some testing with mercurial as well (as you can see in the config file).
More information about the vcs-home