Distributed Software Development in Freenet

Original site (in-Freenet-link!)
Mirror1, Mirror2 (plain old web)

Basically, this project is an adaption of the arch revision control system for use with freenet. For a more in-depth explanation of the subject, check this page.


2003-07-16 - Ed. 9
Phew - six weeks gone in an instant... :-(
The software hasn't changed since the previous edition. That is, I have made a few small steps towards splitfile support, but nothing workable yet.
The LinuxTag was a great event, although there were quite a few empty seats during my talk. Here's the paper and the slides. Someone from the audience suggested creating a "black list" of tools to avoid in anonymous projects. E. g. tools that create proprietary binary file formats should be avoided because you don't know if the files contain hidden information about their creator. I think that's a good idea. Once I find the time to set up NIMble I'll integrate the blacklist on this site.
Someone else has asked me if my Arch version also works with Entropy. I think it should, mostly - but I haven't managed to get Entropy running yet, so I don't know for sure.
Arch has made incredible progress during the last weeks, i. e. it's been completely rewritten in C. That means it's much faster now, but unfortunately the new C version can't make use of my modifications. So this is what I'll be working on in the "near" future...
Freenet also seems to have made some progress with the latest release, although I'm not quite sure... some things definitely are faster, some seem much slower. Time will tell, I guess.
2003-06-03 - Ed. 8
This new release includes the following changes:
  • Merged with arch1.0pre25, including some bugfixes
  • Finished replacing fcptools with calls to ezFCPlib
The file containing your SSK keys now has a different format. To upgrade from the previous release, create a new keyfile (readable only by yourself), starting with two lines in the following format:
Public key:  ...keydata...
Private key: ...keydata...
That's two spaces in the first line!
2003-05-30 - Ed. 7
These changes have not yet been included in the tarball below. They're only available through the archive and its mirrors.
  • Merged with arch1.0pre24 including an important bugfix
  • Improved publish-freenet-mirror: added DBR options, support for manifest files > 32k, support for index templates
  • Improved the freesite: added index tags, activelinks; some doc fixes
  • Started replacing fcptools with calls to ezFCPlib


Freenet sucks. I have two nodes running in independent locations. When I first inserted the project through one node I couldn't retrieve the files through the other. :-( When I later inserted it through the other node, I only got a handful of key collisions on 118 files.

ezFCPlib sucks, too. The documentation doesn't match the implementation, and the implementation is broken.

After using ezFCPlib for a while it doesn't seem to suck as much as I feared. Actually it's pretty useful if you work around the buggy documentation as well as the buggy implementation. Yes, that does sound silly.

Has it always been that difficult to create (and maintain) a freesite? I've been inserting and re-inserting the stuff many times during the past week, and I've tried to get listed on TFE every fucking day. I just don't seem to get through.

Quick Start Guide

  1. Download the source tarball
  2. Unpack it, then type
        cd arch-freenet-0.2/src
        mkdir build
        cd build
        ../configure --prefix=/usr
        make CFLAGS="-O -Wl,-s"
        make test
        make install
    configure --prefix sets the install location for the software. Use a different path if you like.
    CFLAGS=-Wl,-s instructs GNU ld to strip the binaries when linking. This will create significantly smaller executables.
  3. Include <prefix>/bin in your PATH.
  4. Work through the arch tutorial.
  5. To access other people's in-freenet-archives, register them with a freenet: URL, like e. g.
    larch register-archive DSDiF@LCgWj0qabAqxUNKbfI93PCTH9RU.freenet--DSDiF freenet:SSK@LCgWj0qabAqxUNKbfI93PCTH9RUPAgM/DSDiF
    Then, you can use standard arch commands, like e. g.
    larch get DSDiF@LCgWj0qabAqxUNKbfI93PCTH9RU.freenet--DSDiF/dists--freenet--0.1 arch-freenet
    Optionally, you can use the environment variables FCP_HOST, FCP_PORT, FCP_HTL, FCP_HTL_INCR and FCP_RETRIES to configure access to your freenet node.
  6. To create a slightly anonymized local work environment, use the command
    larch new-freenet-project --help
    This will explain how to set up a few things, like e. g. an SSK key pair.
  7. After commiting your work into a local archive, you can mirror it into freenet with the following steps:
    1. Create an "index.html" in the top-level directory of your local archive (like the page you're looking at).
    2. Register the target freenet: URL as another archive, like
      larch register-archive project@pubkey.freenet--mirror freenet:SSK@pubkeyPAgM/project
    3. Use "publish-freenet-mirror" to publish a local archive into freenet:
      larch publish-freenet-mirror --new-edition --sskfile ssk-secret-file local-archive-name project@pubkey.freenet--mirror
      This will insert a copy of your archive as "freenet:SSK@pubkeyPAgM/project/edition". Edition numbers are automatically maintained in your local archive in the file "=meta-info/freenet-edition".
      You may want to set the INSERT_HTL environment variable to override fcpput's default.
    4. (Optional:) Maintain a DBR site redirecting to the current edition at the same freenet URL, but without the edition number. You can use the --dbr* options of publish-freenet-mirror for this, if you have GNU date installed.

If you want to get the arch-in-freenet sources "the arch way", you must register two archives:

  1. larch register-archive lordemf.net--2003 http://regexps.srparish.net/{archives}/lord@emf.net--2003/
  2. One of
    • larch register-archive DSDiF@LCgWj0qabAqxUNKbfI93PCTH9RU.freenet--DSDiF http://www.unix-ag.uni-kl.de/~conrad/Archives/DSDiF/
    • larch register-archive DSDiF@LCgWj0qabAqxUNKbfI93PCTH9RU.freenet--DSDiF
    • larch register-archive DSDiF@LCgWj0qabAqxUNKbfI93PCTH9RU.freenet--DSDiF freenet:SSK@LCgWj0qabAqxUNKbfI93PCTH9RUPAgM/DSDiF
    (For the last option you need the patched arch version, of course. Don't use this for bootstrapping - it won't work. :-)

Then, check out the dist project:

larch get DSDiF@LCgWj0qabAqxUNKbfI93PCTH9RU.freenet--DSDiF/dists--freenet--0.1 arch-freenet-0.2

Finally, get all modules with

cd arch-freenet-0.2; larch build-config dsdif/freenet.0.2

Known Problems

  • Splitfiles are not yet supported, which limits the maximum size of a file in the archive to slightly less than 4 MB.
  • Always make a backup copy of your local archive before you "publish-freenet-mirror"! The tar files in the archive are modified to help preserve your anonymity. This has (probably) destroyed at least one file during my tests. You have been warned!
  • Everything is veeeeeery sloooooow. You can, however, run publish-freenet-mirror as well as library-add in the background while you work on your project. Just don't expect rapid development cycles.
  • Everything is very unreliable as well. When I first inserted the project through one node I couldn't retrieve it through another. :-(
  • The suggested naming convention sucks. You'd probably better use an IIP mail address or something like that.

Contact Information

For arch-specific problems and / or questions, contact Tom Lord <lord@emf.net>, or check the arch mailinglist archive at http://lists.fifthvision.net/mailman/listinfo/arch-users/.

For arch-in-freenet specific questions, you may contact me (Peter Conrad) at <conrad@unix-ag.uni-kl.de>. This is my PGP-Key. Also, sometimes (rarely, actually) I'm hanging out on IIP as "dsdif".


I held a talk on the subject at LinuxTag 2003 in Karlsruhe, Germany. Here's the paper and the slides.

Tom Lord, the inventor of arch, has been unemployed for quite a while now. Despite the great work he is doing on the arch project, he is chronically low on money. If you can afford it, send him some via paypal to lord@emf.net. Or even better, if you have a job to offer, contact him at the given email address.