HOWTO - Using exim and Mailman 2.1 together

Mailman is a list manager with web front end and built in archiving functions. Details can be found at http://www.list.org/. This documentation describes the configuration of Exim (version 3 or 4) to work with Mailman version 2.1

Mailman configuration

There is no Mailman configuration needed other than the standard options detailed in the Mailman install documentation. The Exim configuration is transparent to Mailman. The user/group settings for Mailman must match those in the config fragments given below.

Exim configuration

The Exim configuration is built so that a list created within Mailman automatically appears to Exim without the need for defining any additional aliases.

The drawback of this configuration is that it will work poorly on systems supporting lists in several different mail domains. While Mailman handles virtual domains, it does not yet support having two distinct lists with the same name in different virtual domains, using the same Mailman installation. This will eventually change. (But see below for a variation on this scheme that should accomodate virtual domains better.)

The configuration file excerpts below are for use in an already functional Exim configuration, which accepts mail for the domain in which the list resides. If this domain is separate from the others handled by your Exim configuration, then you'll need to:

[Note: the instructions in this document should work with either Exim 3 or Exim 4. In Exim 3, you must have a 'local_domains' configuration setting; in Exim 4, you most likely have a 'local_domains' domainlist. If you don't, you probably know what you're doing and can adjust accordingly. Similarly, in Exim 4 the concept of "directors" has disappeared -- there are only routers now. So if you're using Exim 4, whenever this document says "director", read "router".]

Whether you are using Exim 3 or Exim 4, you will need to add some macros to the main section of your Exim config file. You will also need to define one new transport. With Exim 3, you'll need to add a new director; with Exim 4, a new router plays the same role.

Finally, the configuration supplied here should allow co-habiting Mailman 2.0 and 2.1 installations, with the proviso that you'll probably want to use "mm21" in place of "mailman" -- e.g., MM21_HOME, mm21_transport, etc.

Main configuration settings

First, you need to add some macros to the top of your Exim config file. These just make the directors (routers) and transport below a bit cleaner. Obviously, you'll need to edit these based on how you configured and installed Mailman.

  # Home dir for your Mailman installation -- aka Mailman's prefix
  # directory.
  MAILMAN_HOME=/usr/local/mailman
  MAILMAN_WRAP=MAILMAN_HOME/mail/mailman

  # User and group for Mailman, should match your --with-mail-gid
  # switch to Mailman's configure script.
  MAILMAN_UID=mailman
  MAILMAN_GID=mailman

Transport for Exim 3

Add this to the transports section of your Exim config file, i.e. somewhere between the first and second "end" line:

  mailman_transport:
    driver = pipe
    command = MAILMAN_WRAP \
              '${if def:local_part_suffix \
                    {${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
                    {post}}' \
              $local_part
    current_directory = MAILMAN_HOME
    home_directory = MAILMAN_HOME
    user = MAILMAN_UID
    group = MAILMAN_GID

(XXX this is untested by me under Exim 3! Can someone using Exim 3 please let me know if it works?)

Director for Exim 3

If you're using Exim 3, you'll need to add the following director to your config file (directors go between the second and third "end" lines). Also, don't forget that order matters -- eg. you can make Mailman lists take precedence over system aliases by putting this directors in front of your aliasfile director, or vice-versa.

  # Handle all addresses related to a list 'foo': the posting address.
  # Automatically detects list existence by looking
  # for lists/$local_part/config.pck under MAILMAN_HOME.
  mailman_director:
    driver = smartuser
    require_files = MAILMAN_HOME/lists/$local_part/config.pck
    suffix_optional
    suffix = -bounces : -bounces+* : \
             -confirm+* : -join : -leave : \
             -owner : -request : -admin
    transport = mailman_transport

Router for Exim 4

In Exim 4, there are no such things as directors -- you instead need to add a router. Also, the canonical order of the configuration file was changed so routers come before transports, so the routers for Exim 4 come first here. Put these two routers somewhere after the "begin routers" line of your config file, and remember that order matters.

  mailman_router:
    driver = accept
    require_files = MAILMAN_HOME/lists/$local_part/config.pck
    local_part_suffix_optional
    local_part_suffix = -bounces : -bounces+* : \
                        -confirm+* : -join : -leave : \
                        -owner : -request : -admin
    transport = mailman_transport

Transports for Exim 4

The transport for Exim 4 is the same as for Exim 3; just copy the transport given above to somewhere under the "begin transports" line of your Exim config file.

Notes

Exim should be configured to allow reasonable volume -- e.g. don't set max_recipients down to a silly value -- and with normal degrees of security -- specifically, be sure to allow relaying from 127.0.0.1, but pretty much nothing else. Parallel deliveries and other tweaks can also be used if you like; experiment with your setup to see what works. Delay warning messages should be switched off or configured to only happen for non-list mail, unless you like receiving tons of mail when some random host is down.

Problems

Receiver Verification

Exim's receiver verification feature is very useful -- it lets Exim reject unrouteable addresses at SMTP time. However, this is most useful for externally-originating mail that is addresses to mail in one of your local domains. For Mailman list traffic, mail originates on your server, and is addressed to random external domains that are not under your control. Furthermore, each message is addressed to many recipients -- up to 500 if you use Mailman's default configuration, and don't tweak SMTP_MAX_RCPTS.

Doing receiver verification on Mailman list traffic is a recipe for trouble. In particular, Exim will attempt to route every recipient addresses in outgoing Mailman list posts. Even though this requires nothing more than a few DNS lookups for each address, it can still introduce significant delays. Therefore, you should disable recipient verification for Mailman traffic.

Under Exim 3, put this in your main configuration section:

  receiver_verify_hosts = !127.0.0.1

Under Exim 4, this is probably already taken care of for you by the default recipient verification ACL statement (in the "RCPT TO" ACL):

  accept  domains       = +local_domains
          endpass
          message       = unknown user
          verify        = recipient

which only does recipient verification on addresses in your domain. (That's not exactly the same as doing recipient verification only on messages coming from non-127.0.0.1 hosts, but it should do the trick for Mailman.)

SMTP Callback

Exim's SMTP callback feature is an even more powerful way to detect bogus sender addresses than normal sender verification. Unfortunately, lots of servers send bounce messages with a bogus address in the header, and there are plenty that sound bounces with bogus envelope senders (even though they're supposed to just use an epmty envelope sender for bounces).

In order to ensure that Mailman can disable/remove bouncing addresses, you generally want to receive bounces for Mailman lists, even if those bounces are themselves not bouncable. Thus, you might want to disable SMTP callback on bounce messages.

If you do header and envelope callbacks, you can disable them for bounces to mailman lists (it is quite common for internal hosts to bounce with a non reachable internal address). The idea is that you typically don't want non bounceable Email, but you'd better accept bounces to mailman lists so that you can unsubscribe the people who are bouncing.

With Exim 4, you can accomplish this using something like the following in your "RCPT TO" ACL:

  # Accept bounces to lists even if callbacks or other checks would fail
  warn     message      = X-WhitelistedRCPT-nohdrfromcallback: Yes
           condition    = \
           ${if and {{match{$local_part}{(.*)-bounces\+.*}}
                     {exists {MAILMAN_HOME/lists/$1/config.pck}}} \
                {yes}{no}}
                {yes}{no}}

  accept   condition    = \
           ${if and {{match{$local_part}{(.*)-bounces\+.*}}
                     {exists {MAILMAN_HOME/lists/$1/config.pck}}} \
                {yes}{no}}
                {yes}{no}}

  # Now, check sender address with SMTP callback.
  deny   !verify = sender/callout=90s/check_postmaster

If you also do SMTP callbacks on header addresses, you'll want something like this in your "DATA" ACL:

  deny   !condition = $header_X-WhitelistedRCPT-nohdrfromcallback:
         !verify = header_sender/callout=90s/check_postmaster

[XXX all this stuff is completely untested by me! -Greg]

Doing VERP with Exim and Mailman

VERP will send one email, with a separate envelope sender (return path), for each of your subscribers -- read the information in ~mailman/Mailman/Default.py for the options that start with VERP. In a nutshell, all you need to do to enable VERP with Exim is to add these lines to ~mailman/Mailman/mm_cfg.py:

  VERP_PASSWORD_REMINDERS = 1
  VERP_PERSONALIZED_DELIVERIES = 1
  VERP_DELIVERY_INTERVAL = 1
  VERP_CONFIRMATIONS = 1

(The directors/routers above are smart enough to deal with VERP bounces.)

Virtual Domains

One approach to handling virtual domains is to use a separate Mailman installation for each virtual domain. (Currently, this is the only way to have lists with the same name in different virtual domains handled by the same machine.)

In this case, the MAILMAN_HOME and MAILMAN_WRAP macros are useless -- you can remove them. Change your director (router) to something like this:

  require_files = /virtual/${domain}/mailman/lists/${lc:$local_part}/config.pck

and change your transport like this:

  command = /virtual/${domain}/mailman/mail/mailman \
            ${if def:local_part_suffix \
                 {${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}}
                 {post}} \
              $local_part
  current_directory = /virtual/${domain}/mailman
  home_directory = /virtual/${domain}/mailman

List Verification

This is how a set of address tests for the Exim lists look on a working system. The list in question is quixote-users@mems-exchange.org, and these commands were run on the mems-exchange.org mail server ("% " indicates the Unix shell prompt):

  % exim -bt quixote-users
  quixote-users@mems-exchange.org
    router = mailman_main_router, transport = mailman_transport

  % exim -bt quixote-users-request
  quixote-users-request@mems-exchange.org
    router = mailman_router, transport = mailman_transport

  % exim -bt quixote-users-bounces
  quixote-users-bounces@mems-exchange.org
    router = mailman_router, transport = mailman_transport

  % exim -bt quixote-users-bounces+luser=example.com
  quixote-users-bounces+luser=example.com@mems-exchange.org
    router = mailman_router, transport = mailman_transport

If your "exim -bt" output looks something like this, that's a start: at least it means Exim will pass the right messages to the right Mailman commands. It by no means guarantees that your Exim/Mailman installation is functioning perfectly, though!

Document History

Originally written by Nigel Metheringham. Updated by Marc Merlin for Mailman 2.1, Exim 4. Overhauled/reformatted/clarified/simplified by Greg Ward.


$Id: mailman.html,v 1.3 2002/06/14 10:42:18 nigel Exp $