X-Git-Url: https://git.exim.org/exim.git/blobdiff_plain/27d0d9e6e002b2a9ea9a053e8163523592786ab5..cb45303cf2a8d9922702f13db42b3285c48f6aa7:/doc/doc-docbook/spec.xfpt diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt index d8cf6e73c..7f96768f7 100644 --- a/doc/doc-docbook/spec.xfpt +++ b/doc/doc-docbook/spec.xfpt @@ -45,14 +45,16 @@ . Update the Copyright year (only) when changing content. . ///////////////////////////////////////////////////////////////////////////// -.set previousversion "4.92" +.set previousversion "4.95" .include ./local_params .set ACL "access control lists (ACLs)" .set I "    " +.set drivernamemax "64" + .macro copyyear -2018 +2021 .endmacro . ///////////////////////////////////////////////////////////////////////////// @@ -161,6 +163,13 @@ .macro index .echo "** Don't use .index; use .cindex or .oindex or .vindex" .endmacro + + +. use this for a concept-index entry for a header line +.macro chindex +.cindex "&'$1'& header line" +.cindex "header lines" $1 +.endmacro . //////////////////////////////////////////////////////////////////////////// @@ -184,7 +193,7 @@ .copyyear - University of Cambridge + The Exim Maintainers .literal off @@ -193,6 +202,8 @@ . This chunk of literal XML implements index entries of the form "x, see y" and . "x, see also y". However, the DocBook DTD doesn't allow entries . at the top level, so we have to put the .chapter directive first. + +. These do not turn up in the HTML output, unfortunately. The PDF does get them. . ///////////////////////////////////////////////////////////////////////////// .chapter "Introduction" "CHID1" @@ -318,6 +329,10 @@ zero, binary binary zero + + headers + header lines + .literal off @@ -371,11 +386,13 @@ contributors. .section "Exim documentation" "SECID1" . Keep this example change bar when updating the documentation! +.new .cindex "documentation" This edition of the Exim specification applies to version &version() of Exim. Substantive changes from the &previousversion; edition are marked in some renditions of this document; this paragraph is so marked if the rendition is capable of showing a change indicator. +.wen This document is very much a reference manual; it is not a tutorial. The reader is expected to have some familiarity with the SMTP mail transfer protocol and @@ -741,17 +758,17 @@ the Exim documentation, &"spool"& is always used in the first sense. .chapter "Incorporated code" "CHID2" .cindex "incorporated code" .cindex "regular expressions" "library" -.cindex "PCRE" +.cindex "PCRE2" .cindex "OpenDMARC" A number of pieces of external code are included in the Exim distribution. .ilist Regular expressions are supported in the main Exim program and in the -Exim monitor using the freely-distributable PCRE library, copyright -© University of Cambridge. The source to PCRE is no longer shipped with -Exim, so you will need to use the version of PCRE shipped with your system, +Exim monitor using the freely-distributable PCRE2 library, copyright +© University of Cambridge. The source to PCRE2 is not longer shipped with +Exim, so you will need to use the version of PCRE2 shipped with your system, or obtain and install the full version of the library from -&url(ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre). +&url(https://github.com/PhilipHazel/pcre2/releases). .next .cindex "cdb" "acknowledgment" Support for the cdb (Constant DataBase) lookup method is provided by code @@ -1392,21 +1409,46 @@ Again, cutthrough delivery counts as a verification. .next Individual routers can be explicitly skipped when running the routers to check an address given in the SMTP EXPN command (see the &%expn%& option). + .next If the &%domains%& option is set, the domain of the address must be in the set of domains that it defines. +.cindex "tainted data" "de-tainting" +A match verifies the variable &$domain$& (which carries tainted data) +and assigns an untainted value to the &$domain_data$& variable. +Such an untainted value is often needed in the transport. +For specifics of the matching operation and the resulting untainted value, +refer to section &<>&. + +When an untainted value is wanted, use this option +rather than the generic &%condition%& option. + .next .vindex "&$local_part_prefix$&" +.vindex "&$local_part_prefix_v$&" .vindex "&$local_part$&" .vindex "&$local_part_suffix$&" +.vindex "&$local_part_suffix_v$&" .cindex affix "router precondition" If the &%local_parts%& option is set, the local part of the address must be in -the set of local parts that it defines. If &%local_part_prefix%& or +the set of local parts that it defines. +A match verifies the variable &$local_part$& (which carries tainted data) +and assigns an untainted value to the &$local_part_data$& variable. +Such an untainted value is often needed in the transport. +For specifics of the matching operation and the resulting untainted value, +refer to section &<>&. + +When an untainted value is wanted, use this option +rather than the generic &%condition%& option. + +If &%local_part_prefix%& or &%local_part_suffix%& is in use, the prefix or suffix is removed from the local part before this check. If you want to do precondition tests on local parts that include affixes, you can do so by using a &%condition%& option (see below) -that uses the variables &$local_part$&, &$local_part_prefix$&, and -&$local_part_suffix$& as necessary. +that uses the variables &$local_part$&, &$local_part_prefix$&, +&$local_part_prefix_v$&, &$local_part_suffix$& +and &$local_part_suffix_v$& as necessary. + .next .vindex "&$local_user_uid$&" .vindex "&$local_user_gid$&" @@ -1416,23 +1458,35 @@ an account on the local host. If this check succeeds, the uid and gid of the local user are placed in &$local_user_uid$& and &$local_user_gid$& and the user's home directory is placed in &$home$&; these values can be used in the remaining preconditions. + .next If the &%router_home_directory%& option is set, it is expanded at this point, because it overrides the value of &$home$&. If this expansion were left till later, the value of &$home$& as set by &%check_local_user%& would be used in subsequent tests. Having two different values of &$home$& in the same router could lead to confusion. + .next If the &%senders%& option is set, the envelope sender address must be in the set of addresses that it defines. + .next If the &%require_files%& option is set, the existence or non-existence of specified files is tested. + .next .cindex "customizing" "precondition" If the &%condition%& option is set, it is evaluated and tested. This option uses an expanded string to allow you to set up your own custom preconditions. Expanded strings are described in chapter &<>&. + +Note that while using +this option for address matching technically works, +it does not set any de-tainted values. +Such values are often needed, either for router-specific options or +for transport options. +Using the &%domains%& and &%local_parts%& options is usually the most +convenient way to obtain them. .endlist @@ -1683,20 +1737,20 @@ overridden if necessary. A C99-capable compiler will be required for the build. -.section "PCRE library" "SECTpcre" -.cindex "PCRE library" -Exim no longer has an embedded PCRE library as the vast majority of -modern systems include PCRE as a system library, although you may need to -install the PCRE package or the PCRE development package for your operating -system. If your system has a normal PCRE installation the Exim build +.section "PCRE2 library" "SECTpcre" +.cindex "PCRE2 library" +Exim no longer has an embedded regular-expression library as the vast majority of +modern systems include PCRE2 as a system library, although you may need to +install the PCRE2 package or the PCRE2 development package for your operating +system. If your system has a normal PCRE2 installation the Exim build process will need no further configuration. If the library or the -headers are in an unusual location you will need to either set the PCRE_LIBS +headers are in an unusual location you will need to either set the PCRE2_LIBS and INCLUDE directives appropriately, -or set PCRE_CONFIG=yes to use the installed &(pcre-config)& command. +or set PCRE2_CONFIG=yes to use the installed &(pcre-config)& command. If your operating system has no -PCRE support then you will need to obtain and build the current PCRE -from &url(ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/). -More information on PCRE is available at &url(https://www.pcre.org/). +PCRE2 support then you will need to obtain and build the current PCRE2 +from &url(https://github.com/PhilipHazel/pcre2/releases). +More information on PCRE2 is available at &url(https://www.pcre.org/). .section "DBM libraries" "SECTdb" .cindex "DBM libraries" "discussion of" @@ -1749,9 +1803,13 @@ the traditional &'ndbm'& interface. .next To complicate things further, there are several very different versions of the Berkeley DB package. Version 1.85 was stable for a very long time, releases -2.&'x'& and 3.&'x'& were current for a while, but the latest versions when Exim last revamped support were numbered 4.&'x'&. -Maintenance of some of the earlier releases has ceased. All versions of -Berkeley DB could be obtained from +2.&'x'& and 3.&'x'& were current for a while, +.new +but the latest versions when Exim last revamped support were numbered 5.&'x'&. +Maintenance of some of the earlier releases has ceased, +and Exim no longer supports versions before 3.&'x'&. +.wen +All versions of Berkeley DB could be obtained from &url(http://www.sleepycat.com/), which is now a redirect to their new owner's page with far newer versions listed. It is probably wise to plan to move your storage configurations away from @@ -1775,6 +1833,9 @@ USE_DB=yes .endd Similarly, for gdbm you set USE_GDBM, and for tdb you set USE_TDB. An error is diagnosed if you set more than one of these. +.new +You can set USE_NDBM if needed to override an operating system default. +.wen At the lowest level, the build-time configuration sets none of these options, thereby assuming an interface of type (1). However, some operating system @@ -1789,7 +1850,11 @@ in one of these lines: .code DBMLIB = -ldb DBMLIB = -ltdb +DBMLIB = -lgdbm -lgdbm_compat .endd +.new +The last of those was for a Linux having GDBM provide emulated NDBM facilities. +.wen Settings like that will work if the DBM library is installed in the standard place. Sometimes it is not, and the library's header file may also not be in the default path. You may need to set INCLUDE to specify where the header @@ -1886,11 +1951,10 @@ to your &_Local/Makefile_& and rebuild Exim. .section "Including TLS/SSL encryption support" "SECTinctlsssl" .cindex "TLS" "including support for TLS" .cindex "encryption" "including support for" -.cindex "SUPPORT_TLS" .cindex "OpenSSL" "building Exim with" .cindex "GnuTLS" "building Exim with" -Exim can be built to support encrypted SMTP connections, using the STARTTLS -command as per RFC 2487. It can also support legacy clients that expect to +Exim is usually built to support encrypted SMTP connections, using the STARTTLS +command as per RFC 2487. It can also support clients that expect to start a TLS session immediately on connection to a non-standard port (see the &%tls_on_connect_ports%& runtime option and the &%-tls-on-connect%& command line option). @@ -1899,35 +1963,39 @@ If you want to build Exim with TLS support, you must first install either the OpenSSL or GnuTLS library. There is no cryptographic code in Exim itself for implementing SSL. +If you do not want TLS support you should set +.code +DISABLE_TLS=yes +.endd +in &_Local/Makefile_&. + If OpenSSL is installed, you should set .code -SUPPORT_TLS=yes +USE_OPENSL=yes TLS_LIBS=-lssl -lcrypto .endd in &_Local/Makefile_&. You may also need to specify the locations of the OpenSSL library and include files. For example: .code -SUPPORT_TLS=yes +USE_OPENSSL=yes TLS_LIBS=-L/usr/local/openssl/lib -lssl -lcrypto TLS_INCLUDE=-I/usr/local/openssl/include/ .endd .cindex "pkg-config" "OpenSSL" If you have &'pkg-config'& available, then instead you can just use: .code -SUPPORT_TLS=yes +USE_OPENSSL=yes USE_OPENSSL_PC=openssl .endd .cindex "USE_GNUTLS" If GnuTLS is installed, you should set .code -SUPPORT_TLS=yes USE_GNUTLS=yes TLS_LIBS=-lgnutls -ltasn1 -lgcrypt .endd in &_Local/Makefile_&, and again you may need to specify the locations of the library and include files. For example: .code -SUPPORT_TLS=yes USE_GNUTLS=yes TLS_LIBS=-L/usr/gnu/lib -lgnutls -ltasn1 -lgcrypt TLS_INCLUDE=-I/usr/gnu/include @@ -1935,7 +2003,6 @@ TLS_INCLUDE=-I/usr/gnu/include .cindex "pkg-config" "GnuTLS" If you have &'pkg-config'& available, then instead you can just use: .code -SUPPORT_TLS=yes USE_GNUTLS=yes USE_GNUTLS_PC=gnutls .endd @@ -2523,11 +2590,32 @@ use of Exim's filtering capabilities, you should make the document entitled +.section "Running the daemon" SECTdaemonLaunch +The most common command line for launching the Exim daemon looks like +.code +exim -bd -q5m +.endd +This starts a daemon which +.ilist +listens for incoming smtp connections, launching handler processes for +each new one +.next +starts a queue-runner process every five minutes, to inspect queued messages +and run delivery attempts on any that have arrived at their retry time +.endlist +Should a queue run take longer than the time between queue-runner starts, +they will run in parallel. +Numbers of jobs of the various types are subject to policy controls +defined in the configuration. + + .section "Upgrading Exim" "SECID36" .cindex "upgrading Exim" If you are already running Exim on your host, building and installing a new version automatically makes it available to MUAs, or any other programs that call the MTA directly. However, if you are running an Exim daemon, you do need +.cindex restart "on HUP signal" +.cindex signal "HUP, to restart" to send it a HUP signal, to make it re-execute itself, and thereby pick up the new binary. You do not need to stop processing mail in order to install a new version of Exim. The install script does not modify an existing runtime @@ -2626,6 +2714,7 @@ supplementary group is one of those listed in the &%trusted_groups%& configuration option. Note that the Exim group is not automatically trusted. .cindex '&"From"& line' +.cindex "envelope from" .cindex "envelope sender" Trusted users are always permitted to use the &%-f%& option or a leading &"From&~"& line to specify the envelope sender of a message that is passed to @@ -2633,10 +2722,8 @@ Exim through the local interface (see the &%-bm%& and &%-f%& options below). See the &%untrusted_set_sender%& option for a way of permitting non-trusted users to set envelope senders. -.cindex "&'From:'& header line" -.cindex "&'Sender:'& header line" -.cindex "header lines" "From:" -.cindex "header lines" "Sender:" +.chindex From: +.chindex Sender: For a trusted user, there is never any check on the contents of the &'From:'& header line, and a &'Sender:'& line is never added. Furthermore, any existing &'Sender:'& line in incoming local (non-TCP/IP) messages is not removed. @@ -2765,7 +2852,12 @@ used to specify a path on the command line if a pid file is required. The SIGHUP signal .cindex "SIGHUP" +.cindex restart "on HUP signal" +.cindex signal "HUP, to restart" .cindex "daemon" "restarting" +.cindex signal "to reload configuration" +.cindex daemon "reload configuration" +.cindex reload configuration can be used to cause the daemon to re-execute itself. This should be done whenever Exim's configuration file, or any file that is incorporated into it by means of the &%.include%& facility, is changed, and also whenever a new version @@ -2875,6 +2967,7 @@ separate document entitled &'Exim's interfaces to mail filtering'&. When testing a filter file, .cindex "&""From""& line" +.cindex "envelope from" .cindex "envelope sender" .oindex "&%-f%&" "for filter testing" the envelope sender can be set by the &%-f%& option, @@ -3715,6 +3808,7 @@ between &%-F%& and the <&'string'&> is optional. .cindex "sender" "address" .cindex "address" "sender" .cindex "trusted users" +.cindex "envelope from" .cindex "envelope sender" .cindex "user" "trusted" This option sets the address of the envelope sender of a locally-generated @@ -3780,9 +3874,11 @@ headers.) .cindex "Solaris" "&'mail'& command" .cindex "dot" "in incoming non-SMTP message" This option, which has the same effect as &%-oi%&, specifies that a dot on a -line by itself should not terminate an incoming, non-SMTP message. I can find -no documentation for this option in Solaris 2.4 Sendmail, but the &'mailx'& -command in Solaris 2.4 uses it. See also &%-ti%&. +line by itself should not terminate an incoming, non-SMTP message. +Solaris 2.4 (SunOS 5.4) Sendmail has a similar &%-i%& processing option +&url(https://docs.oracle.com/cd/E19457-01/801-6680-1M/801-6680-1M.pdf), +p. 1M-529), and therefore a &%-oi%& command line option, which both are used +by its &'mailx'& command. .vitem &%-L%&&~<&'tag'&> .oindex "&%-L%&" @@ -3828,7 +3924,9 @@ id, and the remaining ones must be email addresses. However, if the message is active (in the middle of a delivery attempt), it is not altered. This option can be used only by an admin user. -.vitem "&%-MC%&&~<&'transport'&>&~<&'hostname'&>&~<&'sequence&~number'&>&&& +.vitem "&%-MC%&&~<&'transport'&>&~<&'hostname'&>&&& + &~<&'host&~IP'&>&&& + &~<&'sequence&~number'&>&&& &~<&'message&~id'&>" .oindex "&%-MC%&" .cindex "SMTP" "passed connection" @@ -3852,6 +3950,12 @@ This option is not intended for use by external callers. It is used internally by Exim in conjunction with the &%-MC%& option. It signifies that the remote host supports the ESMTP &_DSN_& extension. +.vitem &%-MCd%& +.oindex "&%-MCd%&" +This option is not intended for use by external callers. It is used internally +by Exim in conjunction with the &%-d%& option +to pass on an information string on the purpose of the process. + .vitem &%-MCG%&&~<&'queue&~name'&> .oindex "&%-MCG%&" This option is not intended for use by external callers. It is used internally @@ -3864,12 +3968,27 @@ This option is not intended for use by external callers. It is used internally by Exim in conjunction with the &%-MC%& option. It signifies that a remote host supports the ESMTP &_CHUNKING_& extension. +.vitem &%-MCL%& +.oindex "&%-MCL%&" +This option is not intended for use by external callers. It is used internally +by Exim in conjunction with the &%-MC%& option. It signifies that the server to +which Exim is connected advertised limits on numbers of mails, recipients or +recipient domains. +The limits are given by the following three arguments. + .vitem &%-MCP%& .oindex "&%-MCP%&" This option is not intended for use by external callers. It is used internally by Exim in conjunction with the &%-MC%& option. It signifies that the server to which Exim is connected supports pipelining. +.vitem &%-MCp%& +.oindex "&%-MCp%&" +This option is not intended for use by external callers. It is used internally +by Exim in conjunction with the &%-MC%& option. It signifies that the connection +t a remote server is via a SOCKS proxy, using addresses and ports given by +the following four arguments. + .vitem &%-MCQ%&&~<&'process&~id'&>&~<&'pipe&~fd'&> .oindex "&%-MCQ%&" This option is not intended for use by external callers. It is used internally @@ -3879,11 +3998,16 @@ together with the file descriptor number of an open pipe. Closure of the pipe signals the final completion of the sequence of processes that are passing messages through the same SMTP connection. +.vitem &%-MCq%&&~<&'recipient&~address'&>&~<&'size'&> +.oindex "&%-MCq%&" +This option is not intended for use by external callers. It is used internally +by Exim to implement quota checking for local users. + .vitem &%-MCS%& .oindex "&%-MCS%&" This option is not intended for use by external callers. It is used internally by Exim in conjunction with the &%-MC%& option, and passes on the fact that the -SMTP SIZE option should be used on messages delivered down the existing +ESMTP SIZE option should be used on messages delivered down the existing connection. .vitem &%-MCT%& @@ -3892,6 +4016,16 @@ This option is not intended for use by external callers. It is used internally by Exim in conjunction with the &%-MC%& option, and passes on the fact that the host to which Exim is connected supports TLS encryption. +.vitem &%-MCr%&&~<&'SNI'&> &&& + &%-MCs%&&~<&'SNI'&> +.oindex "&%-MCs%&" +.oindex "&%-MCr%&" +These options are not intended for use by external callers. It is used internally +by Exim in conjunction with the &%-MCt%& option, and passes on the fact that +a TLS Server Name Indication was sent as part of the channel establishment. +The argument gives the SNI string. +The "r" variant indicates a DANE-verified connection. + .vitem &%-MCt%&&~<&'IP&~address'&>&~<&'port'&>&~<&'cipher'&> .oindex "&%-MCt%&" This option is not intended for use by external callers. It is used internally @@ -3949,6 +4083,18 @@ is sent to the sender, containing the text &"cancelled by administrator"&. Bounce messages are just discarded. This option can be used only by an admin user. +.vitem &%-MG%&&~<&'queue&~name'&>&~<&'message&~id'&>&~<&'message&~id'&>&~... +.oindex "&%-MG%&" +.cindex queue named +.cindex "named queues" "moving messages" +.cindex "queue" "moving messages" +This option requests that each listed message be moved from its current +queue to the given named queue. +The destination queue name argument is required, but can be an empty +string to define the default queue. +If the messages are not currently located in the default queue, +a &%-qG%& option will be required to define the source queue. + .vitem &%-Mmad%&&~<&'message&~id'&>&~<&'message&~id'&>&~... .oindex "&%-Mmad%&" .cindex "delivery" "cancelling all" @@ -4044,8 +4190,9 @@ the standard output. This option can be used only by an admin user. .vitem &%-m%& .oindex "&%-m%&" -This is apparently a synonym for &%-om%& that is accepted by Sendmail, so Exim -treats it that way too. +This is a synonym for &%-om%& that is accepted by Sendmail +(&url(https://docs.oracle.com/cd/E19457-01/801-6680-1M/801-6680-1M.pdf) +p. 1M-258), so Exim treats it that way too. .vitem &%-N%& .oindex "&%-N%&" @@ -4159,6 +4306,7 @@ forces queueing. .vitem &%-odqs%& .oindex "&%-odqs%&" .cindex "SMTP" "delaying delivery" +.cindex "first pass routing" This option is a hybrid between &%-odb%&/&%-odi%& and &%-odq%&. However, like &%-odb%& and &%-odi%&, this option has no effect if &%queue_only_override%& is false and one of the queueing options in the @@ -4357,6 +4505,15 @@ written. When &%-oX%& is used with &%-bd%&, or when &%-q%& with a time is used without &%-bd%&, this is the only way of causing Exim to write a pid file, because in those cases, the normal pid file is not used. +.vitem &%-oPX%& +.oindex "&%-oPX%&" +.cindex "pid (process id)" "of daemon" +.cindex "daemon" "process id (pid)" +This option is not intended for general use. +The daemon uses it when terminating due to a SIGTEM, possibly in +combination with &%-oP%&&~<&'path'&>. +It causes the pid file to be removed. + .vitem &%-or%&&~<&'time'&> .oindex "&%-or%&" .cindex "timeout" "for non-SMTP input" @@ -4389,6 +4546,25 @@ of the syntax, and how it interacts with configuration file options, are given in chapter &<>&. When &%-oX%& is used to start a daemon, no pid file is written unless &%-oP%& is also present to specify a pid filename. +.vitem &%-oY%& +.oindex &%-oY%& +.cindex "daemon notifier socket" +This option controls the creation of an inter-process communications endpoint +by the Exim daemon. +It is only relevant when the &%-bd%& (start listening daemon) option is also +given. +Normally the daemon creates this socket, unless a &%-oX%& and &*no*& &%-oP%& +option is also present. +If this option is given then the socket will not be created. This could be +required if the system is running multiple daemons. + +The socket is currently used for +.ilist +fast ramp-up of queue runner processes +.next +obtaining a current queue size +.endlist + .vitem &%-pd%& .oindex "&%-pd%&" .cindex "Perl" "starting the interpreter" @@ -4466,11 +4642,19 @@ appear in the correct order. Each flag is described in a separate item below. .cindex "queue" "double scanning" .cindex "queue" "routing" .cindex "routing" "whole queue before delivery" +.cindex "first pass routing" +.cindex "queue runner" "two phase" An option starting with &%-qq%& requests a two-stage queue run. In the first stage, the queue is scanned as if the &%queue_smtp_domains%& option matched every domain. Addresses are routed, local deliveries happen, but no remote transports are run. +Performance will be best if the &%queue_run_in_order%& option is false. +If that is so and the &%queue_fast_ramp%& option is true then +in the first phase of the run, +once a threshold number of messages are routed for a given host, +a delivery process is forked in parallel with the rest of the scan. + .cindex "hints database" "remembering routing" The hints database that remembers which messages are waiting for specific hosts is updated, as if delivery to those hosts had been deferred. After this is @@ -4516,7 +4700,7 @@ for later delivery. .vitem &%-q[q][i][f[f]][l][G[/