automating git-annex relationships

[martin f krafft]

also sprach Adam Spiers <vcs-home at adamspiers.org> [2014-09-09 15:59 +0200]:
> This is how I do it:
> 
> http://lists.madduck.net/pipermail/vcs-home/2012-May/000806.html

Right, so you seem to have a central mr config repo with all the
knowledge of who has which repo checked out, so that you can then
say that repo XYZ should have remotes A, B, and C, because you are
pushing that repo to A, B, and C.

This is one way of doing it, and I'll probably look very closely
into your code soon (it's cloned).

However, my question was more about ways to reuse the knowledge
git-annex accumulates anyway.

Say there's ~/photos and it's currently checked out on machineA and
machineB as A and B. Now I bring machineC online and pull ~/photos
there too (into C)

With your approach, I'd then change the global mr config and add
C to the remotes for ~/photos, and wait until this percolates to all
clients.

However, by time time I clone into C, git-annex knows whether A/B
have copies.

What's needed now is a way for me to specify mappings or something
similar, globally for git-annex. Such a mapping would tell git-annex
how each machine can be reached from each other machine, e.g.

  machineA → machineB  machineB.local
  machineB → machineA  machineA.local
  machineA → machineC  -
  machineC → machineA  machineA.rack.example.org
  machineB → machineC  -
  machineC → machineB  machineB.rack.example.org

and maybe some path mangling stuff, since A on machineA might not be
in the same place on the filesystem as B on machineB, etc.

With this knowledge, at the time of cloning, git-annex could set up
all the remotes and bootstrap the annex content from the fastest
machine.

Here's a bit of demo:

  % GIT_DIR=repo.git git --bare init && ( cd repo.git && git annex init centre )
  Initialized empty Git repository in /home/madduck/.tmp/cdt.qu3lXa/repo.git/
  init centre ok
  (Recording state in git...)

  % git clone ./repo.git one
  Cloning into 'one'...
  done.

  % cd one

  % git annex init one
  init one ok
  (Recording state in git...)

  % date > a && git annex add a
  add a ok
  (Recording state in git...)

  % git annex sync --content
  (merging origin/git-annex into git-annex...)
  (Recording state in git...)
  commit  ok
  pull origin
  ok
  copy a copy a (to origin...) ok
  pull origin
  ok
  (Recording state in git...)
  push origin
  Counting objects: 21, done.
  Delta compression using up to 8 threads.
  Compressing objects: 100% (16/16), done.
  Writing objects: 100% (21/21), 1.81 KiB | 0 bytes/s, done.
  Total 21 (delta 3), reused 0 (delta 0)
  To /home/madduck/.tmp/cdt.qu3lXa/./repo.git
  * [new branch]      git-annex -> synced/git-annex
  * [new branch]      master -> synced/master
  ok

  % git annex whereis
  whereis a (2 copies) 
      2ec5ef5a-4ad1-4b21-b391-9667666ec014 -- centre [origin]
      653ff939-5433-44cc-b32e-3d6a2b18d117 -- one [here]
  ok

So far, so good. Now let's clone "two":

  % cd ../

  % git clone ./repo.git two && cd two && git annex init two
  Cloning into 'two'...
  done.
  init two ok
  (Recording state in git...)

  % git annex whereis
  (merging origin/git-annex origin/synced/git-annex into git-annex...)
  (Recording state in git...)
  whereis a (2 copies)
      2ec5ef5a-4ad1-4b21-b391-9667666ec014 -- centre [origin]
      653ff939-5433-44cc-b32e-3d6a2b18d117 -- one
  ok

If git-annex knew at this point that "one" was accessible as ../one
(or other-machine:/path/to/one), then it could set up the remote:

  % git remote add one ../one && git remote update
  Fetching origin
  Fetching one
  From ../one
   * [new branch]      git-annex  -> one/git-annex
   * [new branch]      master     -> one/master
   * [new branch]      synced/master -> one/synced/master

  % git annex sync --content
  […]

  % cat a
  Tue Sep  9 16:43:51 CEST 2014

And the next time that "one" is sync'd, "two" appears, and if
git-annex knows that "two" is accessible as ../two, then it could
set up the remote:

  % git annex sync
  […]

  % git annex whereis
  whereis a (3 copies)
      2ec5ef5a-4ad1-4b21-b391-9667666ec014 -- centre [origin]
      653ff939-5433-44cc-b32e-3d6a2b18d117 -- one [here]
      8e720e4f-e90e-4c63-b2cc-20a50a2598cf -- two
  ok

  % git remote add two ../two && git remote update
  […]

My point is that rather than having to maintain which repos should
know about which remotes in some central place, git-annex provides
these data.

Now we just need to find a way to make it happen.

Joey, have you considered this?

-- 
@martinkrafft | http://madduck.net/ | http://two.sentenc.es/
 
"the surest way to corrupt a youth is to instruct him to hold in
 higher esteem those who think alike than those who think
 differently."
                                              -- friedrich nietzsche
 
spamtraps: madduck.bogus at madduck.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: digital_signature_gpg.asc
Type: application/pgp-signature
Size: 1107 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
URL: <http://lists.madduck.net/pipermail/vcs-home/attachments/20140909/a0fd9f5e/attachment.sig>