shlomif_tech [shlomif]

Many months ago, I noticed that SVK, a version control system based on Subversion, stopped propagating changesets to the remote Subversion repositories. Furthermore, I was unable to mirror other remote repositories. I wrote about it, asking for help:

• Post to the svk-users mailing list
• My "Problems with SVK" entry on this blog

All of this has been left unsolved until yesterday when I decided that I had enough and wanted to finally resolve this problem. My first idea was to try to write a script that will propagate the changesets using the Subversion APIs, so I looked at the SVK code trying to see where exactly it happens. I found the SVK code hard-to-follow, so I first contemplated fixing what caused it to report an error, but was also unable to.

Then I joined the Freenode IRC network and went to the #svk channel. There was only one person there who tried to help me, and he was intermittent in helping. Then I joined #perl and casually mentioned that I thought SVK sucked (just to lose some steam), which prompted mst to try to help me. After interrogating me he suggested that in order to recover the changesets from SVK, I should follow the following steps:

1. Find out what is the last SVK revision which still propagated changesets to the Subversion repository. This can be found using the commit message in the svn log.
2. Revert (using a selective svnadmin dump and then an svnadmin create ; and svnadmin load) to the older version of ~/.svk/local that corresponds with the revision number of the commit. (While naturally keeping a copy of ~/.svk/local/ and its dump in a safe place.)
3. See that a simple change in the svk checkout (like svk mkdir to-delete-`date +%s`) will propagate to the remote repository.
4. Revert that change using the Subversion client directly.
5. Revert to the working-but-old copy of the SVK repository, and apply a filtered stream of the dumps (using svndumpfilter) that will exclude changes from the //mirror part of the SVK repository.
6. After all that is done, checkout the trunk and do svk push from there.

Of course, the devil is in the details, and it took me a lot of experimentation until I got to something working. Facts I've discovered:

1. svndumpfilter needs a dump that was generated without the --deltas flag (and possibly without --incremental either. Otherwise it will complain.
2. I couldn't seem to get svndumpfilter exclude mirror to work and instead ended up using svndumpfilter include local
3. The --drop-empty-revs and --renumber-revs flags to svndumpfilter which seemed like a good idea confused svnadmin load, and everything worked after I removed them.
4. Using tar -czf and tar -xf can really speed up restoring older versions of a Subversion repository because svnadmin load is pretty slow.
5. I ended up taking a range of repositories from the pristine revision to "HEAD" and then manually editing out the first revision. Maybe it was unnecessary.

After all that, "svk push" worked and propagated 800 changesets to my homepage's trunk in 8:20 hours (must be a record). Since I only have an ADSL connection with 25 KBps upstream, and the Subversion service is located in Taiwan and I'm living in Israel, it took a while and I left it overnight to run.

Today, after I woke up, I saw that the operation ended up successfully. I was able to build an up-to-date version of my homesite from the Subversion sources, and am mostly ready to stop using SVK.

Again, I'd like to thank mst for suggesting this ultra-cool idea for recovering my data from SVK. It took me the whole of one day, but now I'm finally free, and my data is out there and safe. I'm going to use plain-old-Subversion to work against it, until and assuming I'll get the hang of hg/bzr/git/whatever. But at least I no longer have to use a mostly-unusable SVK.

Hackfully and happily yours, Shlomi Fish.