CTM for OpenBSD

What is ctm?

Ctm is a system which was designed by Poul-Henning Kamp for making changes to a source tree available on a daily basis by email. This is a good way to stay up to date with the current source- or CVS tree if you have a bad Internet connection, for instance via modem.

Ctm uses very low bandwidth for distributing the changes to the tree, and compresses them using gzip -9. In contrast to CVSup, ctm is written in plain C and therefore available for every platform supported by OpenBSD.

How does it work?

The basic idea of ctm is that you subscribe to a special mailing list for a particular source tree. Each day you receive an email message containing all the changes to that tree during that period. One set of differences is called a "delta".

You begin with a base set, against which later deltas are patched. To begin using ctm you download the latest base set and all deltas generated after that. Once you have processed those you can process later deltas you receive from the mailing list. The base sets are split into pieces of 1400kB each for easier downloading over bad lines, and you can even put them onto floppies this way.

Base sets are generated once every 50 relative deltas, so you never have to grab more than 50 deltas to catch up.

There are currently two kinds of base delta. For quite a while we were making tar.gz files manually, but we are switching back to real ctm base deltas. The latter contain md5 checksums and are generated automatically, thus are a bit safer. You can distinguish these two kinds of base delta by their filename. An example for a tar.gz file is OpenBSD-cvs.1450.tar.gz, whereas a ctm base delta would be named OpenBSD-cvs.1500A.gz. See below for detailed instructions.

How much bandwidth does ctm use?

You obtain the base set and all relative deltas up to the recent one via ftp from the sites listed below. Sizes are:

Larger ctm updates are rare, though, usually occurring only after a major import of something like binutils, perl, gcc, etc.

How do I use ctm?

Note: The examples below just cover OpenBSD-cvs for clarity. The other source sets are handled in the same fashion.

The addresses for the deltas are:

NOTE: I'm very interested in finding other sites around the world that would like to mirror these deltas.

Should I choose the source or CVS tree?

This depends on your diskspace, among other factors. Since ctm doesn't deal with files which have been modified locally, the "better" choice is probably the CVS tree. You then can check out your working source tree from your CVS tree, and keep your local modifications in your source tree, because cvs is smart enough to merge in your local changes.

The drawback, however, is the amount of diskspace it takes. A checked-out source tree takes about 350MB or so, but if you have your own cvs tree you need about 1Gb for it, plus another 350MB for the checked out tree.

This doesn't count the 100MB or more (depending on the architecture) you will need to do a build, i.e. compile the sources.

If you just get the source tree, you will need to deal with local modifications. One way to do this is to use the union filesystem, although there have been reports that the union filesystem can be unstable if both the upper and lower layers reside on the same physical filesystem. If you place your upper and lower layers on separate filesystems you should have no problems. A sample of this is as follows:

Suppose your ctm-updated tree is in /usr/src-ctm, and your real source tree, the one you make your modifications to and do your builds from, is in /usr/src. /usr/src should be initially an empty directory. The following command will set up the union mount:

	mount -t union -o -b /usr/src-ctm /usr/src

Modifications made to files in /usr/src will become a file contained within /usr/src, hiding the one in src-ctm. If changes are made via ctm to the underlying src-ctm tree, those changes will not be seen if there is a file in the upper layer hiding it.

You should periodically unmount the union and search for files which are local to the union filesystem.

	umount /usr/src
	find /usr/src -type f

The commands ls -W and rm -W will be useful too, as objects named "whiteouts" in /usr/src will also hide files in src-ctm.

If you get the CVS tree, you can use the cvs checkout command to check out a source tree from it, and each time you update with ctm you can use the cvs update command to update your source tree.

To get an initial src tree:

	cd /usr
	cvs -qd YOUR_CVS_TREE checkout -A src

and after each ctm update:

	cd /usr/src
	cvs -q update -PAd

How stable is ctm for OpenBSD?

Ctm has been in use for over four years for OpenBSD, and for FreeBSD for an even longer period of time. Ctm is very reliable and stable.

More information

If you want to learn more about ctm, a good place to start with is the FreeBSD Handbook: There are also man pages for all of the ctm utilities.


Important notes and announcements about ctm will be posted to:


For any problems, suggestions, reports and questions regarding ctm contact the ctm maintainer Hans Günter Weigand.

OpenBSD/CTM logo designed for the OpenBSD Project by Phillip F Knaack.

OpenBSD www@openbsd.org
$OpenBSD: ctm.html,v 1.13 2001/01/01 22:01:21 naddy Exp $