wiki:LancelotFaq

Project Lancelot FAQ

(or, rather, questions likely to be frequently asked if anybody would actually use the software except myself)

Project Lancelot Meta-Information

Q. Where do I get Project Lancelot?

The official Project Lancelot web site is at http://code.anselms.net/projects/lancelot/ . This contains the software (both ready-made releases as well as the master revision control repository, available through Mercurial), documentation, a timeline, roadmap and bug-tracking system.

We shall eventually try to get Project Lancelot integrated into the Debian GNU/Linux distribution but there is no timeframe for this yet.

Q. Why the funny name?

A. If you think »Project Lancelot« is funny you must not have heard of that other mailing list manager which is called »Enemies of Carlotta« :)

The name came up when we were talking about starting the project. I had recently been reading T. H. White's »The Once and Future King« to my SO, and it occurred to me that mailing list management as a field of endeavour was in dire need of a knight riding in to slay all the dragons and free all the damsels in distress. Of course for all the nobility this is ultimately futile (we're not going to replace Mailman anytime soon), hence Project Lancelot. I could have called it Project Don Quixote but that name is already spoken for.

Q. Isn't there some software inside KDE that is called Lancelot?

A. Yes. So what? We came first, so if anyone ought to change their project name it should by rights be them. It also does something completely different from what Project Lancelot does. (We try to refer to our program as »Project Lancelot« to reduce the chance of confusion.)

Project Lancelot Configuration

Q. How do I conveniently transfer settings from one list to another?

A. You can initialise a new list from the settings of an existing list using

  $ pl-init --clone=oldlist@example.com newlist@example.com admin@example.com

This will use the configuration settings from oldlist@… except for the list.name, list.address, and list.owneraddress parameters, which are set as for a new list. The details are in pl-init(1). It is also possible to override individual items when cloning a list, by giving them on the command line as usual.

Once a list has been created, a nifty way of transferring (groups of) configuration items is by doing

  $ pl-conf -q oldlist@example.com PATTERN | pl-conf -i newlist@example.com

where PATTERN is something like digest.* (to transfer all digest settings).

Project Lancelot and MTAs

Q. How do I let users manage their own mailing lists without involving the mail administrator?

Project Lancelot stores mailing list meta-information in a user's home directory, and therefore there is no problem with this in principle. The main problem is actually getting mail for a new mailing list into pl-incoming(1) without having the mail administrator edit an aliases file (or similar). How to do this depends on the system's MTA.

It's probably easiest if you're running Qmail; for a list called, say jdoe-list@…, the user jdoe simply puts a .qmail-list-default file into their home directory that redirects all mail into pl-incoming(1) (some translation of return codes may be appropriate). Of course Qmail sucks in many other respects, so don't convert your mail system to Qmail just for Project Lancelot's sake.

With Postfix, the problem is that while one can use a recipient_delimiter like + to handle addresses like jdoe+list@… by means of a .forward+list file in jdoe 's home directory, it is impossible to foresee all adresses of the form jdoe+list-bounce-4711-anselm=anselms.net@… which might arise through normal mailing list operations and would have to be handled through their own .forward files. It is probably best to use an MDA on steroids like Procmail or Mailfilter to funnel all messages whose address extensions start with a list name to the pl-incoming(1) command, e.g. (for Procmail),

  :0 H : list.lock
  ^TOjdoe+list
  |/usr/bin/pl-incoming list@example.com

If you're handling mail for many different domains, a more convenient way may be to use Postfix's virtual_mailbox_domains together with an MDA like Mailfilter (Postfix's virtual(8) delivery method is insufficient because it does not allow delivery to user-specified programs).

Q. What is this VERP thing and why would I want to use it?

A. VERP, or Variable Envelope Return Path, is a method of finding out which bounce messages came from which subscribing address. While the subscriber address is often obvious from a bounce message, this is not always the case -- people move to new addresses and get their mail forwarded from their old accounts, but do not bother to change all their mailing list subscription. Hence Project Lancelot may be sending a message to jdoe@…, whose owner may long since have moved to john.doe@…, getting all his mail sent on from example.com to there. If the elsewhere.net address develops a problem, a bounce will be sent to Project Lancelot from the john.doe@… address, which Project Lancelot will not be able to find in the subscriber database, since the subscription is still in the name of jdoe@….

VERP works by encoding the original subscriber address into the SMTP envelope return path, which is the address that a receiving MTA uses for bounce messages. Instead of putting something like

  MAIL FROM:<lancelot+bounce@anselms.net>

in a message for jdoe@… (which would direct all bounces to lancelot+bounce@…, with no obvious sign of which subscriber address caused the bounce), Project Lancelot uses

  MAIL FROM:<lancelot+bounce-1234-jdoe=example.com@anselms.net>

Thus, bounce messages are sent to this address, which Project Lancelot can pick apart to reconstruct that this bounce was caused by a message to jdoe. (The 1234 is the message number, which for now is duly recorded when the bounce is processed, but not used further.)

Q. Isn't VERP terribly expensive?

It depends. If you have lots of subscribers at the same domain (say, aol.com), then without VERP you could send one copy of your message to that domain's mail server, with instructions to deliver that copy to all of the subscribers there. With VERP, you have to send one message per subscriber, which of course seems wasteful. However, for many of us robustness is preferable to speed, so with the costs of Internet bandwidth being what they are, being able to trace errors is often worth the extra traffic.

Project Lancelot supports two different flavours of VERP, according to the value of the smtp.verp parameter. The first (called lancelot) lets Project Lancelot do all the work by itself. This means that, for a mailing list with N subscribers, it sends N copies of every message to the local MTA, each with the correct VERP address. The second (called smtp) makes use of the XVERP ESMTP extension to let the local MTA do the VERPing. This means that Project Lancelot sends the message to the local MTA with a bunch of up to smtp.maxrecipients subscriber addresses at one go (thus for N subscribers, only N/(smtp.maxrecipients+1) messages are submitted locally), and the local MTA passes them on with the correct VERP-encoded addresses. This is preferable because it puts less load on the local MTA. If Project Lancelot is set up for smtp-style VERP and it finds that the local MTA does not support the XVERP extension, it falls back to lancelot-style VERP. Therefore it is usually safe to set smtp.verp = smtp even if you do not know for sure whether your local MTA supports it or not.

Q. How do I find out whether my MTA supports XVERP?

Read the documentation :) If you're more of a hands-on type person, you could try something along the lines of

  $ telnet localhost smtp              <<< contact the local MTA
  Trying 127.0.0.1...
  Connected to mail.example.com.
  Escape character is '^]'.
  220 mail.example.com ESMTP
  EHLO mail.example.com                <<< introduce yourself
  250-mail.example.com
  ... (a bunch of "250-" lines) ...
  250-XVERP                            <<< this line must be there
  ... (possibly more "250-" lines) ...
  250 8BITMIME                         <<< or whatever

If the output of your MTA in answer to the EHLO command does not contain a line saying XVERP, then your MTA does not support the extension.

Note for Postfix users: Postfix does support the XVERP extension but, beginning with Postfix 2.1, it must be specially enabled. In the Postfix main.cf file, set the smtpd_authorized_verp_clients parameter to include the address and/or network of the machine running Project Lancelot. In the usual case of Project Lancelot running on the same machine as your MTA, something like

  smtpd_authorized_verp_clients = 127.0.0.1/8

is fine.

Troubleshooting

Q. Whatever I do, I get the following message:

  /usr/bin/pl-list: error accessing list list@example.com

I've been using this list for a while without problems, and I just upgraded Project Lancelot to a new version.

A. If you have had the list for quite some time, the most likely explanation is that it was created when Project Lancelot used SQLite 2 for its database. Recent versions of Project Lancelot use SQLite 3, and the on-disk format of the database files is not directly compatible. You can check whether this is your problem by invoking any Project Lancelot command (pl-list is a good candidate) with the --debug command line option. This should give a message like

  pl-list[24983]: db: /home/alfred/.pl/list@example.com/list.db is an SQLite 2 database, please upgrade

(among others).

Now, about getting this sorted out: If you still have SQLite 2 around, an easy way to upgrade your database is

  $ cd $HOME/.pl/list@example.com    # go to the list directory
  $ cp list.db list.db.old           # make a backup copy (we love backup copies!)
  $ sqlite list.db .dump | sqlite3 list.db.new && mv list.db.new list.db

This uses the SQLite 2 command line tool to dump the old database in SQL and feeds that SQL code to the SQLite 3 command line tool to create a new database, which is then copied over the old one. (Without SQLite 2, you're essentially hosed. Your best bet would be to install SQLite 2, which isn't a big deal; your Linux distribution should have it, and if not, it is still available from the SQLite download page. Check the bottom of that page for instructions on how to get SQLite 2.8.)

Note for Our-ISP users. Remember that your list directory is something along the lines of $HOME/domains/example.com/list.

Q. I'm using ask_confirm(cmd=foo) to have my users confirm a workflow but the module does not appear to do anything (I can see from the log that it is being invoked but no confirmation messages are sent out, and Project Lancelot just proceeds with the workflow as if the module wasn't there).

A. Remember that you must set confirm.foo = 1 in the list configuration for your ask_confirm to actually do anything. This is so we can ship things like the unsubscribe workflow that contain ask_confirm but do not by default require confirmation.

Last modified 5 years ago Last modified on Jul 26, 2012, 2:53:57 PM