Dotlocking support for qmail

WARNING: (added 2013-11-16) I have not been involved in running any sort of mail server for several years now. In particular, I don't follow what is going on the the qmail world. This patch never came with any sort of warranty, but that is triply true now. Even if a serious bug or security vulnerability came to my attention, I would not be in a position to fix it. If you do need to use this patch, please read the code and form your own understanding of how it works.

There will probably never be any support for dotlocking in the official release of qmail, for the reason that it is not sufficiently reliable. Still, dotlocking is popular in a world filled with NFS file systems (a problem in its own right), because ordinary file locking tends to create all sorts of problems with hung stat and lock daemons and stuff. So I have made a patch to allow qmail to use dotlocking on mailbox files. Of course, the long range solution is to use maildir instead, and then all this will be moot.

A patch for qmail 1.01

This patch will work with the original source code for qmail 1.01. It patches the following files:

Makefile

TARGETS

dot-qmail.9

dotlock.c

dotlock.h

maildir2mbox.c

mbox.5

qmail-local.c

Updated on 1998-02-27. The earlier version was buggy and should not be used.

Want to use qmail 1.02?: Sorry, a patch for 1.02 will not be available, since 1.03 was out before I got around to it.
A patch for qmail 1.03: Updated on 1998-10-26: Various minor build glitches fixed, and a bug in dotlock fixed (it would not run a program with no arguments, which does not affect the usual usage when run by qmail-local.); This patch will work with the original source code for qmail 1.03. It patches the following files:

Makefile
TARGETS
dot-qmail.9
dotlock.8	New file
dotlock.c	New file
hier.c
mboxappend.8	New file
mboxappend.c	New file
qmail-local.c

New in this version: I no longer add the dotlocking capabilities to qmail-local, but instead move it out into the program dotlock, which simply dotlocks a given file and calls a specified program, namely mboxappend in the case where it is being run from qmail-local. dotlock is responsible for locking and running a timer, while mboxappend is responsible for actually appending text to the mailbox, as well as for cleaning up its act upon an ALRM signal received from dotlock (see below for a further explanation of these issues). I hope that this version of the patch will be easier to maintain in the face of new qmail releases, and also it is less likely to interfere with the other duties of qmail-local. Also, dotlock is useful to have as a separate program for the manipulation of mailboxes.

A word on the implementation of dotlocking: Basically, dotlocking works as follows. In order to gain the lock on a file FILE, a program first generates a unique filename, typically of the form FILE.lock.something-unique. Then that file is linked to FILE.lock. If this fails, it is presumed that somebody else holds the lock. If it succeeds, the unique file name is removed, the program performs whatever actions it wanted to perform on FILE, and then it releases the lock by deleting FILE.lock.

The good news is that this works on all kinds of UNIX systems, even over NFS almost no matter how broken the implementation. Otherwise, file locking over NFS is a tricky business at best.

The bad news is that dotlocking is prone to leaving FILE.lock lying around, in case the machine or program crashed at the wrong time. This is sufficiently much of a problem that the algorithm needs to be changed. The policy at our site, and what this patch implements, is the following: No program is allowed to hold a lock for more than 60 seconds, and any lock older than that may be stolen. With this patch, qmail-local or maildir2mbox will detect an approaching 60 second deadline, clean up, and commit suicide. Any message(s) not added to the mbox will be left in the queue to be delivered later (qmail-local) or in the maildir (maildir2mbox).

The bad, bad news is that, particularly on a heavily loaded machine, the cleanup and suicide may not complete in time, leading to a corrupted mailbox.

Finally, a piece of good news (and good advice): If you use maildirs instead of mboxes, none of these problems apply. By all means, do so if you can. This patch is for those who cannot, and who have implemented a dotlocking policy at their site.

Harald Hanche-Olsen

2013-11-16 (minor update 2019-10-15: Add css and meta tags. Yes, the layout is horrible, but I am not going to fix it. So there!)