X-Git-Url: https://git.exim.org/users/jgh/exim.git/blobdiff_plain/be24b950ae0db88b1c9811b3a028e95133c55efa..3db72f4b639a64cacf152e4f7718a18581426b10:/doc/doc-docbook/spec.xfpt diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt index bf042ac2f..a1f361339 100644 --- a/doc/doc-docbook/spec.xfpt +++ b/doc/doc-docbook/spec.xfpt @@ -52,7 +52,7 @@ .set I "    " .macro copyyear -2019 +2020 .endmacro . ///////////////////////////////////////////////////////////////////////////// @@ -3910,7 +3910,7 @@ messages through the same SMTP connection. .oindex "&%-MCS%&" This option is not intended for use by external callers. It is used internally by Exim in conjunction with the &%-MC%& option, and passes on the fact that the -SMTP SIZE option should be used on messages delivered down the existing +ESMTP SIZE option should be used on messages delivered down the existing connection. .vitem &%-MCT%& @@ -4518,6 +4518,7 @@ appear in the correct order. Each flag is described in a separate item below. .cindex "queue" "routing" .cindex "routing" "whole queue before delivery" .cindex "first pass routing" +.cindex "queue runner" "two phase" An option starting with &%-qq%& requests a two-stage queue run. In the first stage, the queue is scanned as if the &%queue_smtp_domains%& option matched every domain. Addresses are routed, local deliveries happen, but no remote @@ -4525,6 +4526,10 @@ transports are run. .new Performance will be best if the &%queue_run_in_order%& option is false. +If that is so and the &%queue_fast_ramp%& option is true then +in the first phase of the run, +once a threshold number of messages are routed for a given host, +a delivery process is forked in parallel with the rest of the scan. .wen .cindex "hints database" "remembering routing" @@ -6381,7 +6386,7 @@ All other options are defaulted. .code local_delivery: driver = appendfile - file = /var/mail/$local_part_verified + file = /var/mail/$local_part_data delivery_date_add envelope_to_add return_path_add @@ -6394,7 +6399,7 @@ traditional BSD mailbox format. .new We prefer to avoid using &$local_part$& directly to define the mailbox filename, as it is provided by a potential bad actor. -Instead we use &$local_part_verified$&, +Instead we use &$local_part_data$&, the result of looking up &$local_part$& in the user database (done by using &%check_local_user%& in the the router). .wen @@ -6641,6 +6646,8 @@ file that is searched could contain lines like this: .endd When the lookup succeeds, the result of the expansion is a list of domains (and possibly other types of item that are allowed in domain lists). +.cindex "tainted data" "de-tainting" +The result of the expansion is not tainted. In the second example, the lookup is a single item in a domain list. It causes Exim to use a lookup to see if the domain that is being processed can be found @@ -6661,6 +6668,18 @@ If the value of &$sender_host_address$& is 192.168.5.6, expansion of the first &%domains%& setting above generates the second setting, which therefore causes a second lookup to occur. +.new +The lookup type may optionally be followed by a comma +and a comma-separated list of options. +Each option is a &"name=value"& pair. +Whether an option is meaningful depands on the lookup type. + +All lookups support the option &"cache=no_rd"&. +If this is given then the cache that Exim manages for lookup results +is not checked before diong the lookup. +The result of the lookup is still written to the cache. +.wen + The rest of this chapter describes the different lookup types that are available. Any of them can be used in any part of the configuration where a lookup is permitted. @@ -6678,6 +6697,13 @@ lookup to succeed. The lookup type determines how the file is searched. .new .cindex "tainted data" "single-key lookups" The file string may not be tainted + +All single-key lookups support the option &"ret=key"&. +If this is given and the lookup +(either underlying implementation or cached value) +returns data, the result is replaced with a non-tainted +version of the lookup key. +.cindex "tainted data" "de-tainting" .wen .next .cindex "query-style lookup" "definition of" @@ -6853,9 +6879,32 @@ If a selector is numeric, it must apply to a JSON array; the (zero-based) nunbered array element is selected. Otherwise it must apply to a JSON object; the named element is selected. The final resulting element can be a simple JSON type or a JSON object -or array; for the latter two a string-representation os the JSON +or array; for the latter two a string-representation of the JSON is returned. For elements of type string, the returned value is de-quoted. + + +.new +.next +.cindex LMDB +.cindex lookup lmdb +.cindex database lmdb +&(lmdb)&: The given file is an LMDB database. +LMDB is a memory-mapped key-value store, +with API modeled loosely on that of BerkeleyDB. +See &url(https://symas.com/products/lightning-memory-mapped-database/) +for the feature set and operation modes. + +Exim provides read-only access via the LMDB C library. +The library can be obtained from &url(https://github.com/LMDB/lmdb) +or your operating system package repository. +To enable LMDB support in Exim set LOOKUP_LMDB=yes in &_Local/Makefile_&. + +You will need to separately create the LMDB database file, +possibly using the &"mdb_load"& utility. +.wen + + .next .cindex "linear search" .cindex "lookup" "lsearch" @@ -6977,9 +7026,10 @@ be followed by optional colons. lookup types support only literal keys. .next +.cindex "spf lookup type" .cindex "lookup" "spf" -If Exim is built with SPF support, manual lookups can be done -(as opposed to the standard ACL condition method. +&(spf)&: If Exim is built with SPF support, manual lookups can be done +(as opposed to the standard ACL condition method). For details see section &<>&. .endlist ilist @@ -8334,6 +8384,35 @@ in the previous section. You could also use the &(wildlsearch)& or +.new +.section "Results of list checking" SECTlistresults +The primary result of doing a list check is a truth value. +In some contexts additional information is stored +about the list element that matched: +.vlist +.vitem hosts +A &%hosts%& ACL condition +will store a result in the &$host_data$& variable. +.vitem local_parts +A &%local_parts%& router option or &%local_parts%& ACL condition +will store a result in the &$local_part_data$& variable. +.vitem domains +A &%domains%& router option or &%domains%& ACL condition +.vitem senders +A &%senders%& router option or &%senders%& ACL condition +will store a result in the &$sender_data$& variable. +.vitem recipients +A &%recipients%& ACL condition +will store a result in the &$recipient_data$& variable. +.endlist + +The detail of the additional information depends on the +type of match and is given below as the &*value*& information. +.wen + + + + .section "Named lists" "SECTnamedlists" .cindex "named lists" .cindex "list" "named" @@ -8496,6 +8575,12 @@ If a pattern consists of a single @ character, it matches the local host name, as set by the &%primary_hostname%& option (or defaulted). This makes it possible to use the same configuration file on several different hosts that differ only in their names. + +.new +The value for a match will be the primary host name. +.wen + + .next .cindex "@[] in a domain list" .cindex "domain list" "matching local IP interfaces" @@ -8505,7 +8590,14 @@ in square brackets (as in an email address that contains a domain literal), but only if that IP address is recognized as local for email routing purposes. The &%local_interfaces%& and &%extra_local_interfaces%& options can be used to control which of a host's several IP addresses are treated as local. -In today's Internet, the use of domain literals is controversial. +In today's Internet, the use of domain literals is controversial; +see the &%allow_domain_literals%& main option. + +.new +The value for a match will be the string &`@[]`&. +.wen + + .next .cindex "@mx_any" .cindex "@mx_primary" @@ -8554,6 +8646,11 @@ involved, it is easiest to change the delimiter for the main list as well: domains = >&) to specify that it is not to be expanded (unless you really do want to build a regular expression by expansion, of course). + +.new +The value for a match will be the list element string (starting with the circumflex). +Additionally, &$0$& will be set to the string matching the regular expression, +and &$1$& (onwards) to any submatches identified by parentheses. +.wen + + + .next .cindex "lookup" "in domain list" .cindex "domain list" "matching by lookup" @@ -8593,12 +8705,15 @@ must be a filename in a suitable format for the lookup type. For example, for domains = cdb;/etc/mail/local_domains.cdb .endd The appropriate type of lookup is done on the file using the domain name as the -key. In most cases, the data that is looked up is not used; Exim is interested +key. In most cases, the value resulting from the lookup is not used; Exim is interested only in whether or not the key is present in the file. However, when a lookup is used for the &%domains%& option on a router -or a &%domains%& condition in an ACL statement, the data is preserved in the +or a &%domains%& condition in an ACL statement, the value is preserved in the &$domain_data$& variable and can be referred to in other router options or other statements in the same ACL. +.cindex "tainted data" "de-tainting" +The value will be untainted. + .next Any of the single-key lookup type names may be preceded by @@ -8617,6 +8732,7 @@ original lookup fails. This is not a useful feature when using a domain list to select particular domains (because any domain would match), but it might have value if the result of the lookup is being used via the &$domain_data$& expansion variable. + .next If the pattern starts with the name of a query-style lookup type followed by a semicolon (for example, &"nisplus;"& or &"ldap;"&), the remainder of the @@ -8626,25 +8742,37 @@ chapter &<>&. For example: hold_domains = mysql;select domain from holdlist \ where domain = '${quote_mysql:$domain}'; .endd -In most cases, the data that is looked up is not used (so for an SQL query, for +In most cases, the value resulting from the lookup is not used (so for an SQL query, for example, it doesn't matter what field you select). Exim is interested only in whether or not the query succeeds. However, when a lookup is used for the -&%domains%& option on a router, the data is preserved in the &$domain_data$& +&%domains%& option on a router, the value is preserved in the &$domain_data$& variable and can be referred to in other options. +.cindex "tainted data" "de-tainting" +The value will be untainted. + .next .new If the pattern starts with the name of a lookup type of either kind (single-key or query-style) it may be -followed by a command and options, +followed by a comma and options, The options are lookup-type specific and consist of a comma-separated list. Each item starts with a tag and and equals "=". .wen + .next .cindex "domain list" "matching literal domain name" If none of the above cases apply, a caseless textual comparison is made between the pattern and the domain. + +The value for a match will be the list element string. +.cindex "tainted data" "de-tainting" +Note that this is commonly untainted +(depending on the way the list was created). +This is a useful way of obtaining an untainted equivalent to +the domain, for later operations. .endlist + Here is an example that uses several different kinds of pattern: .code domainlist funny_domains = \ @@ -10516,6 +10644,14 @@ ${sort {${lookup dnsdb{>:,,mx=example.com}}} {<} {${listextract{1}{<,$item}}}} will sort an MX lookup into priority order. + +.new +.vitem &*${srs_encode&~{*&<&'secret'&>&*}{*&<&'return&~path'&>&*}{*&<&'original&~domain'&>&*}}*& +SRS encoding. See SECT &<>& for details. +.wen + + + .vitem &*${substr{*&<&'string1'&>&*}{*&<&'string2'&>&*}{*&<&'string3'&>&*}}*& .cindex "&%substr%& expansion item" .cindex "substring extraction" @@ -11578,6 +11714,13 @@ includes the case of letters, whereas for &%gti%& the comparison is case-independent. Case and collation order are defined per the system C locale. + +.new +.vitem &*inbound_srs&~{*&<&'local&~part'&>&*}{*&<&'secret'&>&*}*& +SRS decode. See SECT &<>& for details. +.wen + + .vitem &*inlist&~{*&<&'string1'&>&*}{*&<&'string2'&>&*}*& &&& &*inlisti&~{*&<&'string1'&>&*}{*&<&'string2'&>&*}*& .cindex "string" "comparison" @@ -12341,7 +12484,9 @@ the complete argument of the ETRN command (see section &<>&). .cindex "tainted data" If the origin of the data is an incoming message, the result of expanding this variable is tainted. -See also &$domain_verified$&. +When un untainted version is needed, one should be obtained from +looking up the value in a local (therefore trusted) database. +Often &$domain_data$& is usable in this role. .wen @@ -12548,29 +12693,15 @@ Consider carefully the implications of using it unvalidated as a name for file access. This presents issues for users' &_.forward_& and filter files. For traditional full user accounts, use &%check_local_users%& and the -&$local_part_verified$& variable rather than this one. +&$local_part_data$& variable rather than this one. For virtual users, store a suitable pathname component in the database which is used for account name validation, and use that retrieved value rather than this variable. +Often &$local_part_data$& is usable in this role. If needed, use a router &%address_data%& or &%set%& option for the retrieved data. .wen -.vindex "&$local_part_prefix$&" -.vindex "&$local_part_prefix_v$&" -.vindex "&$local_part_suffix$&" -.vindex "&$local_part_suffix_v$&" -.cindex affix variables -If a local part prefix or suffix has been recognized, it is not included in the -value of &$local_part$& during routing and subsequent delivery. The values of -any prefix or suffix are in &$local_part_prefix$& and -&$local_part_suffix$&, respectively. -.new -If the affix specification included a wildcard then the portion of -the affix matched by the wildcard is in -&$local_part_prefix_v$& or &$local_part_suffix_v$& as appropriate. -.wen - When a message is being delivered to a file, pipe, or autoreply transport as a result of aliasing or forwarding, &$local_part$& is set to the local part of the parent address, not to the filename or command (see &$address_file$& and @@ -12611,44 +12742,33 @@ router as &$local_part_data$&. In addition, if the driver routes the address to a transport, the value is available in that transport. If the transport is handling multiple addresses, the value from the first address is used. +.new +The &%check_local_user%& router option also sets this variable. +.wen + &$local_part_data$& is also set when the &%local_parts%& condition in an ACL matches a local part by means of a lookup. The data read by the lookup is available during the rest of the ACL statement. In all other situations, this variable expands to nothing. -.vitem &$local_part_prefix$& -.vindex "&$local_part_prefix$&" +.vindex &$local_part_prefix$& &&& + &$local_part_prefix_v$& &&& + &$local_part_suffix$& &&& + &$local_part_suffix_v$& .cindex affix variables -When an address is being routed or delivered, and a -specific prefix for the local part was recognized, it is available in this -variable, having been removed from &$local_part$&. - -.new -.vitem &$local_part_prefix_v$& -.vindex "&$local_part_prefix_v$&" -When &$local_part_prefix$& is valid and the prefix match used a wildcard, -the portion matching the wildcard is available in this variable. -.wen - -.vitem &$local_part_suffix$& -.vindex "&$local_part_suffix$&" -When an address is being routed or delivered, and a -specific suffix for the local part was recognized, it is available in this -variable, having been removed from &$local_part$&. - +If a local part prefix or suffix has been recognized, it is not included in the +value of &$local_part$& during routing and subsequent delivery. The values of +any prefix or suffix are in &$local_part_prefix$& and +&$local_part_suffix$&, respectively. .new -.vitem &$local_part_suffix_v$& -.vindex "&$local_part_suffix_v$&" -When &$local_part_suffix$& is valid and the suffix match used a wildcard, -the portion matching the wildcard is available in this variable. -.wen +.cindex "tainted data" +If the specification did not include a wildcard then +the affix variable value is not tainted. -.new -.vitem &$local_part_verified$& -.vindex "&$local_part_verified$&" -If the router generic option &%check_local_part%& has run successfully, -this variable has the user database version of &$local_part$&. -Such values are not tainted and hence usable for building file names. +If the affix specification included a wildcard then the portion of +the affix matched by the wildcard is in +&$local_part_prefix_v$& or &$local_part_suffix_v$& as appropriate, +and both the whole and varying values are tainted. .wen .vitem &$local_scan_data$& @@ -13693,6 +13813,17 @@ the value of the Distinguished Name of the certificate is made available in the If certificate verification fails it may refer to a failing chain element which is not the leaf. + +.new +.vitem &$tls_in_resumption$& &&& + &$tls_out_resumption$& +.vindex &$tls_in_resumption$& +.vindex &$tls_out_resumption$& +.cindex TLS resumption +Observability for TLS session resumption. See &<>& for details. +.wen + + .vitem &$tls_in_sni$& .vindex "&$tls_in_sni$&" .vindex "&$tls_sni$&" @@ -14693,6 +14824,7 @@ See also the &'Policy controls'& section above. .row &%hold_domains%& "hold delivery for these domains" .row &%local_interfaces%& "for routing checks" .row &%queue_domains%& "no immediate delivery for these" +.row &%queue_fast_ramp%& "parallel delivery with 2-phase queue run" .row &%queue_only%& "no immediate delivery at all" .row &%queue_only_file%& "no immediate delivery if file exists" .row &%queue_only_load%& "no immediate delivery if load is high" @@ -14739,6 +14871,7 @@ Those options that undergo string expansion before use are marked with .cindex "8-bit characters" .cindex "log" "selectors" .cindex "log" "8BITMIME" +.cindex "ESMTP extensions" 8BITMIME This option causes Exim to send 8BITMIME in its response to an SMTP EHLO command, and to accept the BODY= parameter on MAIL commands. However, though Exim is 8-bit clean, it is not a protocol converter, and it @@ -14952,6 +15085,7 @@ That is, set the option to an empty string so that no check is done. .option auth_advertise_hosts main "host list&!!" * .cindex "authentication" "advertising" .cindex "AUTH" "advertising" +.cindex "ESMTP extensions" AUTH If any server authentication mechanisms are configured, Exim advertises them in response to an EHLO command only if the calling host matches this list. Otherwise, Exim does not advertise AUTH. @@ -15013,12 +15147,18 @@ just the command name, it is not a complete command line. If an argument is required, it must come from the &%-oA%& command line option. -.option bounce_message_file main string unset +.option bounce_message_file main string&!! unset .cindex "bounce message" "customizing" .cindex "customizing" "bounce message" This option defines a template file containing paragraphs of text to be used for constructing bounce messages. Details of the file's contents are given in -chapter &<>&. See also &%warn_message_file%&. +chapter &<>&. +.new +.cindex bounce_message_file "tainted data" +The option is expanded to give the file path, which must be +absolute and untainted. +.wen +See also &%warn_message_file%&. .option bounce_message_text main string unset @@ -15207,6 +15347,7 @@ may wish to deliberately disable them. .option chunking_advertise_hosts main "host list&!!" * .cindex CHUNKING advertisement .cindex "RFC 3030" "CHUNKING" +.cindex "ESMTP extensions" CHUNKING The CHUNKING extension (RFC3030) will be advertised in the EHLO message to these hosts. Hosts may use the BDAT command as an alternate to DATA. @@ -15558,6 +15699,7 @@ described in section &<>&. .cindex "bounce messages" "success" .cindex "DSN" "success" .cindex "Delivery Status Notification" "success" +.cindex "ESMTP extensions" DSN DSN extensions (RFC3461) will be advertised in the EHLO message to, and accepted from, these hosts. Hosts may use the NOTIFY and ENVID options on RCPT TO commands, @@ -15565,6 +15707,10 @@ and RET and ORCPT options on MAIL FROM commands. A NOTIFY=SUCCESS option requests success-DSN messages. A NOTIFY= option with no argument requests that no delay or failure DSNs are sent. +.new +&*Note*&: Supplying success-DSN messages has been criticised +on privacy grounds; it can leak details of internal forwarding. +.wen .option dsn_from main "string&!!" "see below" .cindex "&'From:'& header line" "in bounces" @@ -16446,6 +16592,11 @@ to depend on the IP address of the remote host for messages arriving via TCP/IP. After expansion, the value must be a sequence of decimal digits, optionally followed by K or M. +.cindex "SIZE" "ESMTP extension, advertising" +.cindex "ESMTP extensions" SIZE +If nonzero the value will be advertised as a parameter to the ESMTP SIZE +service extension keyword. + &*Note*&: This limit cannot be made to depend on a message's sender or any other properties of an individual message, because it has to be advertised in the server's response to EHLO. String expansion failure causes a temporary @@ -16713,6 +16864,7 @@ of the &%-oX%& option, unless a path is explicitly supplied by &%-oP%&. .option pipelining_advertise_hosts main "host list&!!" * .cindex "PIPELINING" "suppressing advertising" +.cindex "ESMTP extensions" PIPELINING This option can be used to suppress the advertisement of the SMTP PIPELINING extension to specific hosts. See also the &*no_pipelining*& control in section &<>&. When PIPELINING is not advertised and @@ -16724,6 +16876,7 @@ not count as protocol errors (see &%smtp_max_synprot_errors%&). .option pipelining_connect_advertise_hosts main "host list&!!" * .cindex "pipelining" "early connection" .cindex "pipelining" PIPE_CONNECT +.cindex "ESMTP extensions" X_PIPE_CONNECT If Exim is built with the SUPPORT_PIPE_CONNECT build option this option controls which hosts the facility is advertised to and from which pipeline early-connection (before MAIL) SMTP @@ -16737,6 +16890,7 @@ Currently the option name &"X_PIPE_CONNECT"& is used. .option prdr_enable main boolean false .cindex "PRDR" "enabling on server" +.cindex "ESMTP extensions" PRDR This option can be used to enable the Per-Recipient Data Response extension to SMTP, defined by Eric Hall. If the option is set, PRDR is advertised by Exim when operating as a server. @@ -16847,6 +17001,17 @@ domains that do not match are processed. All other deliveries wait until the next queue run. See also &%hold_domains%& and &%queue_smtp_domains%&. +.new +.option queue_fast_ramp main boolean false +.cindex "queue runner" "two phase" +.cindex "queue" "double scanning" +If set to true, two-phase queue runs, initiated using &%-qq%& on the +command line, may start parallel delivery processes during their first +phase. This will be done when a threshold number of messages have been +routed for a single host. +.wen + + .option queue_list_requires_admin main boolean true .cindex "restricting access to features" .oindex "&%-bp%&" @@ -17456,6 +17621,7 @@ hosts), you can do so by an appropriate use of a &%control%& modifier in an ACL .option smtp_etrn_command main string&!! unset .cindex "ETRN" "command to be run" +.cindex "ESMTP extensions" ETRN .vindex "&$domain$&" If this option is set, the given command is run whenever an SMTP ETRN command is received from a host that is permitted to issue such commands (see @@ -17636,7 +17802,8 @@ example, instead of &"Administrative prohibition"&, it might give: .option smtputf8_advertise_hosts main "host list&!!" * -.cindex "SMTPUTF8" "advertising" +.cindex "SMTPUTF8" "ESMTP extension, advertising" +.cindex "ESMTP extensions" SMTPUTF8 When Exim is built with support for internationalised mail names, the availability thereof is advertised in response to EHLO only to those client hosts that match this option. See @@ -17927,6 +18094,7 @@ unfortunately not all, operating systems. .cindex "TLS" "advertising" .cindex "encryption" "on SMTP connection" .cindex "SMTP" "encrypted connection" +.cindex "ESMTP extensions" STARTTLS When Exim is built with support for TLS encrypted connections, the availability of the STARTTLS command to set up an encrypted session is advertised in response to EHLO only to those client hosts that match this option. See @@ -18176,6 +18344,14 @@ preference order of the available ciphers. Details are given in sections &<>& and &<>&. +.new +.option tls_resumption_hosts main "host list&!!" unset +.cindex TLS resumption +This option controls which connections to offer the TLS resumption feature. +See &<>& for details. +.wen + + .option tls_try_verify_hosts main "host list&!!" unset .cindex "TLS" "client certificate verification" .cindex "certificate" "verification of client" @@ -18364,14 +18540,20 @@ regular expression by a parenthesized subpattern. The default value for See &%uucp_from_pattern%& above. -.option warn_message_file main string unset +.option warn_message_file main string&!! unset .cindex "warning of delay" "customizing the message" .cindex "customizing" "warning message" This option defines a template file containing paragraphs of text to be used for constructing the warning message which is sent by Exim when a message has been in the queue for a specified amount of time, as specified by &%delay_warning%&. Details of the file's contents are given in chapter -&<>&. See also &%bounce_message_file%&. +&<>&. +.new +.cindex warn_message_file "tainted data" +The option is expanded to give the file path, which must be +absolute and untainted. +.wen +See also &%bounce_message_file%&. .option write_rejectlog main boolean true @@ -19182,7 +19364,7 @@ but the user is specified symbolically, the gid associated with the uid is used. For example: .code require_files = mail:/some/file -require_files = $local_part:$home/.procmailrc +require_files = $local_part_data:$home/.procmailrc .endd If a user or group name in a &%require_files%& list does not exist, the &%require_files%& condition fails. @@ -21813,7 +21995,7 @@ local_users: # This transport overrides the group group_delivery: driver = appendfile - file = /var/spool/mail/$local_part + file = /var/spool/mail/$local_part_data group = mail .endd If &%user%& is set for a transport, its value overrides what is set in the @@ -22328,7 +22510,7 @@ message. For example, a content scan could insert a new header line containing a spam score. This could be interpreted by a filter in the user's MUA. It is not possible to discard a message at this stage. -.cindex "SMTP" "SIZE" +.cindex "SIZE" "ESMTP extension" A problem might arise if the filter increases the size of a message that is being sent down an SMTP connection. If the receiving SMTP server has indicated support for the SIZE parameter, Exim will have sent the size of the message @@ -22648,7 +22830,7 @@ is used as a result of a &"keep"& action in the filter. This example shows one way of handling this requirement: .code file = ${if eq{$address_file}{inbox} \ - {/var/mail/$local_part} \ + {/var/mail/$local_part_data} \ {${if eq{${substr_0_1:$address_file}}{/} \ {$address_file} \ {$home/mail/$address_file} \ @@ -22829,8 +23011,8 @@ The string value is expanded for each delivery, and must yield an absolute path. The most common settings of this option are variations on one of these examples: .code -file = /var/spool/mail/$local_part -file = /home/$local_part/inbox +file = /var/spool/mail/$local_part_data +file = /home/$local_part_data/inbox file = $home/inbox .endd .cindex "&""sticky""& bit" @@ -23586,7 +23768,7 @@ and directories in a maildir mailbox, including subdirectories for maildir++ folders. Consider this example: .code maildir_format = true -directory = /var/mail/$local_part\ +directory = /var/mail/$local_part_data\ ${if eq{$local_part_suffix}{}{}\ {/.${substr_1:$local_part_suffix}}} maildirfolder_create_regex = /\.[^/]+$ @@ -24576,14 +24758,14 @@ configuration for &%procmail%&: # transport procmail_pipe: driver = pipe - command = /usr/local/bin/procmail -d $local_part + command = /usr/local/bin/procmail -d $local_part_data return_path_add delivery_date_add envelope_to_add check_string = "From " escape_string = ">From " umask = 077 - user = $local_part + user = $local_part_data group = mail # router @@ -25041,7 +25223,8 @@ facilities such as AUTH, PIPELINING, SIZE, and STARTTLS. .option hosts_avoid_pipelining smtp "host list&!!" unset .cindex "PIPELINING" "avoiding the use of" -Exim will not use the SMTP PIPELINING extension when delivering to any host +.cindex "ESMTP extensions" PIPELINING +Exim will not use the ESMTP PIPELINING extension when delivering to any host that matches this list, even if the server host advertises PIPELINING support. .option hosts_pipe_connect smtp "host list&!!" unset @@ -25245,6 +25428,7 @@ such as DNSBL lookups, will still delay the emission of the SMTP banner. .option hosts_try_prdr smtp "host list&!!" * .cindex "PRDR" "enabling, optional in client" +.cindex "ESMTP extensions" PRDR This option provides a list of servers to which, provided they announce PRDR support, Exim will attempt to negotiate PRDR for multi-recipient messages. @@ -25307,6 +25491,20 @@ so can cause parallel connections to the same host if &%remote_max_parallel%& permits this. +.new +.option message_linelength_limit smtp integer 998 +.cindex "line length" limit +This option sets the maximum line length, in bytes, that the transport +will send. Any messages with lines exceeding the given value +will fail and a failure-DSN ("bounce") message will if possible be returned +to the sender. +The default value is that defined by the SMTP standards. + +It is generally wise to also check in the data ACL so that messages +received via SMTP can be refused without producing a bounce. +.wen + + .option multi_domain smtp boolean&!! true .vindex "&$domain$&" When this option is set, the &(smtp)& transport can handle a number of @@ -25402,7 +25600,7 @@ See also the &%max_parallel%& generic transport option. .option size_addition smtp integer 1024 -.cindex "SMTP" "SIZE" +.cindex "SIZE" "ESMTP extension" .cindex "message" "size issue for transport filter" .cindex "size" "of message" .cindex "transport" "filter" @@ -25488,6 +25686,14 @@ is used in different ways by OpenSSL and GnuTLS (see sections ciphers is a preference order. +.new +.option tls_resumption_hosts smtp "host list&!!" unset +.cindex TLS resumption +This option controls which connections to use the TLS resumption feature. +See &<>& for details. +.wen + + .option tls_sni smtp string&!! unset .cindex "TLS" "Server Name Indication" @@ -25587,13 +25793,17 @@ The &%tls_verify_certificates%& option must also be set. If both this option and &%tls_try_verify_hosts%& are unset operation is as if this option selected all hosts. -.option utf8_downconvert smtp integer!! unset +.option utf8_downconvert smtp integer&!! -1 .cindex utf8 "address downconversion" .cindex i18n "utf8 address downconversion" If built with internationalization support, -this option controls conversion of UTF-8 in message addresses +this option controls conversion of UTF-8 in message envelope addresses to a-label form. -For details see section &<>&. +If, after expansion, the value is 1, 0, or -1 then this value overrides +any value previously set for the message. Otherwise, any previously +set value is used. To permit use of a previous value, +set this option to an empty string. +For details on the values see section &<>&. @@ -26701,6 +26911,7 @@ transfer of mail between servers that have no managerial connection with each other. .cindex "AUTH" "description of" +.cindex "ESMTP extensions" AUTH Very briefly, the way SMTP authentication works is as follows: .ilist @@ -28732,6 +28943,7 @@ tls_require_ciphers = ${if =={$received_port}{25}\ .section "Configuring an Exim server to use TLS" "SECID182" .cindex "TLS" "configuring an Exim server" +.cindex "ESMTP extensions" STARTTLS When Exim has been built with TLS support, it advertises the availability of the STARTTLS command to client hosts that match &%tls_advertise_hosts%&, but not to any others. The default value of this option is *, which means @@ -28991,6 +29203,7 @@ deliveries as well as to incoming, the latter one causing logging of the server certificate's DN. The remaining client configuration for TLS is all within the &(smtp)& transport. +.cindex "ESMTP extensions" STARTTLS It is not necessary to set any options to have TLS work in the &(smtp)& transport. If Exim is built with TLS support, and TLS is advertised by a server, the &(smtp)& transport always tries to start a TLS session. However, @@ -29311,6 +29524,100 @@ Open-source PKI book, available online at .ecindex IIDencsmtp2 +.new +.section "TLS Resumption" "SECTresumption" +.cindex TLS resumption +TLS Session Resumption for TLS 1.2 and TLS 1.3 connections can be used (defined +in RFC 5077 for 1.2). The support for this requires GnuTLS 3.6.3 or OpenSSL 1.1.1 +(or later). + +Session resumption (this is the "stateless" variant) involves the server sending +a "session ticket" to the client on one connection, which can be stored by the +client and used for a later session. The ticket contains sufficient state for +the server to reconstruct the TLS session, avoiding some expensive crypto +calculation and (on TLS1.2) one full packet roundtrip time. + +.ilist +Operational cost/benefit: + + The extra data being transmitted costs a minor amount, and the client has + extra costs in storing and retrieving the data. + + In the Exim/Gnutls implementation the extra cost on an initial connection + which is TLS1.2 over a loopback path is about 6ms on 2017-laptop class hardware. + The saved cost on a subsequent connection is about 4ms; three or more + connections become a net win. On longer network paths, two or more + connections will have an average lower startup time thanks to the one + saved packet roundtrip. TLS1.3 will save the crypto cpu costs but not any + packet roundtrips. + +.cindex "hints database" tls + Since a new hints DB is used on the TLS client, + the hints DB maintenance should be updated to additionally handle "tls". + +.next +Security aspects: + + The session ticket is encrypted, but is obviously an additional security + vulnarability surface. An attacker able to decrypt it would have access + all connections using the resumed session. + The session ticket encryption key is not committed to storage by the server + and is rotated regularly (OpenSSL: 1hr, and one previous key is used for + overlap; GnuTLS 6hr but does not specify any overlap). + Tickets have limited lifetime (2hr, and new ones issued after 1hr under + OpenSSL. GnuTLS 2hr, appears to not do overlap). + + There is a question-mark over the security of the Diffie-Helman parameters + used for session negotiation. + +.next +Observability: + + The &%log_selector%& "tls_resumption" appends an asterisk to the tls_cipher "X=" + element. + + The variables &$tls_in_resumption$& and &$tls_out_resumption$& + have bits 0-4 indicating respectively + support built, client requested ticket, client offered session, + server issued ticket, resume used. A suitable decode list is provided + in the builtin macro _RESUME_DECODE for in &%listextract%& expansions. + +.next +Control: + +The &%tls_resumption_hosts%& main option specifies a hostlist for which +exim, operating as a server, will offer resumption to clients. +Current best practice is to not offer the feature to MUA connection. +Commonly this can be done like this: +.code +tls_resumption_hosts = ${if inlist {$received_port}{587:465} {:}{*}} +.endd +If the peer host matches the list after expansion then resumption +is offered and/or accepted. + +The &%tls_resumption_hosts%& smtp transport option performs the +equivalent function for operation as a client. +If the peer host matches the list after expansion then resumption +is attempted (if a stored session is available) or the information +stored (if supplied by the peer). + + +.next +Issues: + + In a resumed session: +.ilist + The variables &$tls_{in,out}_cipher$& will have values different + to the original (under GnuTLS). +.next + The variables &$tls_{in,out}_ocsp$& will be "not requested" or "no response", + and the &%hosts_require_ocsp%& smtp trasnport option will fail. +. XXX need to do something with that hosts_require_ocsp +.endlist + +.endlist +.wen + .section DANE "SECDANE" .cindex DANE @@ -30933,6 +31240,7 @@ calling host. Its effect lasts until the end of the SMTP connection. .vitem &*control&~=&~no_pipelining*& .cindex "PIPELINING" "suppressing advertising" +.cindex "ESMTP extensions" PIPELINING This control turns off the advertising of the PIPELINING extension to SMTP in the current session. To be useful, it must be obeyed before Exim sends its response to an EHLO command. Therefore, it should normally appear in an ACL @@ -31008,7 +31316,7 @@ data is read. that are being submitted at the same time using &%-bs%& or &%-bS%&. .vitem &*control&~=&~utf8_downconvert*& -This control enables conversion of UTF-8 in message addresses +This control enables conversion of UTF-8 in message envelope addresses to a-label form. For details see section &<>&. .endlist vlist @@ -35138,7 +35446,7 @@ central_filter: check_local_user driver = redirect domains = +local_domains - file = /central/filters/$local_part + file = /central/filters/$local_part_data no_verify allow_filter allow_freeze @@ -35896,13 +36204,14 @@ used to contain the envelope information. .cindex "outgoing LMTP over TCP/IP" .cindex "EHLO" .cindex "HELO" -.cindex "SIZE option on MAIL command" +.cindex "SIZE" "option on MAIL command" Outgoing SMTP and LMTP over TCP/IP is implemented by the &(smtp)& transport. The &%protocol%& option selects which protocol is to be used, but the actual processing is the same in both cases. +.cindex "ESMTP extensions" SIZE If, in response to its EHLO command, Exim is told that the SIZE -parameter is supported, it adds SIZE=<&'n'&> to each subsequent MAIL +extension is supported, it adds SIZE=<&'n'&> to each subsequent MAIL command. The value of <&'n'&> is the message size plus the value of the &%size_addition%& option (default 1024) to allow for additions to the message such as per-transport header lines, or changes made in a @@ -36298,7 +36607,8 @@ RCPT failures. .section "The ETRN command" "SECTETRN" .cindex "ETRN" "processing" -RFC 1985 describes an SMTP command called ETRN that is designed to +.cindex "ESMTP extensions" ETRN +RFC 1985 describes an ESMTP command called ETRN that is designed to overcome the security problems of the TURN command (which has fallen into disuse). When Exim receives an ETRN command on a TCP/IP connection, it runs the ACL specified by &%acl_smtp_etrn%& in order to decide whether the command @@ -36653,10 +36963,10 @@ lists in a separate domain from normal mail. For example: lists: driver = redirect domains = lists.example - file = /usr/lists/$local_part + file = ${lookup {$local_part} dsearch,ret=full {/usr/lists}} forbid_pipe forbid_file - errors_to = $local_part-request@lists.example + errors_to = ${quote_local_part:$local_part-request}@lists.example no_more .endd This router is skipped for domains other than &'lists.example'&. For addresses @@ -36744,7 +37054,8 @@ lists_request: driver = redirect domains = lists.example local_part_suffix = -request - file = /usr/lists/$local_part$local_part_suffix + local_parts = ${lookup {$local_part} dsearch,filter=file {/usr/lists}} + file = /usr/lists/${local_part_data}-request no_more lists_post: @@ -36752,10 +37063,10 @@ lists_post: domains = lists.example senders = ${if exists {/usr/lists/$local_part}\ {lsearch;/usr/lists/$local_part}{*}} - file = /usr/lists/$local_part + file = ${lookup {$local_part} dsearch,ret=full {/usr/lists}} forbid_pipe forbid_file - errors_to = $local_part-request@lists.example + errors_to = ${quote_local_part:$local_part-request}@lists.example no_more lists_closed: @@ -36813,7 +37124,7 @@ verp_smtp: max_rcpt = 1 return_path = \ ${if match {$return_path}{^(.+?)-request@your.dom.example\$}\ - {$1-request+$local_part=$domain@your.dom.example}fail} + {${quote_local_part:$1-request+$local_part=$domain}@your.dom.example}fail} .endd This has the effect of rewriting the return path (envelope sender) on outgoing SMTP messages, if the local part of the original return path ends in @@ -36864,7 +37175,7 @@ verp_dnslookup: transport = remote_smtp errors_to = \ ${if match {$return_path}{^(.+?)-request@your.dom.example\$}} - {$1-request+$local_part=$domain@your.dom.example}fail} + {${quote_local_part:$1-request+$local_part=$domain}@your.dom.example}fail} no_more .endd Before you start sending out messages with VERPed return paths, you must also @@ -36952,7 +37263,7 @@ follows: .code my_mailboxes: driver = appendfile - file = /var/mail/$domain/$local_part + file = /var/mail/$domain/$local_part_data user = mail .endd This uses a directory of mailboxes for each domain. The &%user%& setting is @@ -37007,9 +37318,9 @@ another MTA: userforward: driver = redirect check_local_user - file = $home/.forward$local_part_suffix local_part_suffix = -* local_part_suffix_optional + file = ${lookup {.forward$local_part_suffix} dsearch,ret=full {$home} {$value}fail} allow_filter .endd If there is no suffix, &_.forward_& is used; if the suffix is &'-special'&, for @@ -37926,6 +38237,7 @@ selection marked by asterisks: &`*tls_certificate_verified `& certificate verification status &`*tls_cipher `& TLS cipher suite on <= and => lines &` tls_peerdn `& TLS peer DN on <= and => lines +&` tls_resumption `& append * to cipher field &` tls_sni `& TLS SNI on <= lines &` unknown_in_list `& DNS lookup failed in list match @@ -38327,6 +38639,14 @@ connection, the cipher suite used is added to the log line, preceded by X=. connection, and a certificate is supplied by the remote host, the peer DN is added to the log line, preceded by DN=. .next +.cindex "log" "TLS resumption" +.cindex "TLS" "logging session resumption" +.new +&%tls_resumption%&: When a message is sent or received over an encrypted +connection and the TLS session resumed one used on a previous TCP connection, +an asterisk is appended to the X= cipher field in the log line. +.wen +.next .cindex "log" "TLS SNI" .cindex "TLS" "logging SNI" &%tls_sni%&: When a message is received over an encrypted connection, and @@ -38872,6 +39192,10 @@ for remote hosts .next &'ratelimit'&: the data for implementing the ratelimit ACL condition .next +.new +&'tls'&: TLS session resumption data +.wen +.next &'misc'&: other hints data .endlist @@ -40285,8 +40609,8 @@ There is no dot-stuffing (and no dot-termination). . //////////////////////////////////////////////////////////////////////////// . //////////////////////////////////////////////////////////////////////////// -.chapter "DKIM, SPF and DMARC" "CHAPdkim" &&& - "DKIM, SPF and DMARC Support" +.chapter "DKIM, SPF, SRS and DMARC" "CHAPdkim" &&& + "DKIM, SPF, SRS and DMARC Support" .section "DKIM (DomainKeys Identified Mail)" SECDKIM .cindex "DKIM" @@ -40976,6 +41300,108 @@ The lookup will return the same result strings as can appear in +.section "SRS (Sender Rewriting Scheme)" SECTSRS +.cindex SRS "sender rewriting scheme" + +.new +SRS can be used to modify sender addresses when forwarding so that +SPF verification does not object to them. +It operates by encoding the original envelope sender in a new +sender local part and using a domain run by the forwarding site +as the new domain for the sender. Any DSN message should be returned +to this new sender at the forwarding site, which can extract the +original sender from the coded local part and forward the DSN to +the originator. + +This is a way of avoiding the breakage that SPF does to forwarding. +The constructed local-part will be longer than the original, +leading to possible problems with very long addresses. +The changing of the sender address also hinders the tracing of mail +problems. + +Exim can be built to include native SRS support. To do this +SUPPORT_SRS=yes must be defined in &_Local/Makefile_&. +If this has been done, the macros _HAVE_SRS and _HAVE_NATIVE_SRS +will be defined. +The support is limited to SRS0-encoding; SRS1 is not supported. + +.cindex SRS excoding +To encode an address use this expansion item: +.vlist +.vitem &*${srs_encode&~{*&<&'secret'&>&*}{*&<&'return&~path'&>&*}{*&<&'original&~domain'&>&*}}*& +.cindex "&%srs_encode%& expansion item" +.cindex SRS "expansion item" +The first argument should be a secret known and used by all systems +handling the recipient domain for the original message. +There is no need to periodically change this key; a timestamp is also +encoded. +The second argument should be given as the envelope sender address before this +encoding operation. +The third argument should be the recipient domain of the message when +it arrived at this system. +.endlist + +.cindex SRS decoding +To decode an address use this expansion condition: +.vlist +.vitem &*inbound_srs&~{*&<&'local&~part'&>&*}{*&<&'secret'&>&*}*& +The first argument should be the recipient local prt as is was received. +The second argument is the site secret. + +If the messages is not for an SRS-encoded recipient the condition will +return false. If it is, the condition will return true and the variable +&$srs_recipient$& will be set to the decoded (original) value. +.endlist + +Example usage: +.code + #macro + SRS_SECRET = + + #routers + + outbound: + driver = dnslookup + # if outbound, and forwarding has been done, use an alternate transport + domains = ! +my_domains + transport = ${if eq {$local_part@$domain} \ + {$original_local_part@$original_domain} \ + {remote_smtp} {remote_forwarded_smtp}} + + inbound_srs: + driver = redirect + senders = : + domains = +my_domains + # detect inbound bounces which are SRS'd, and decode them + condition = ${if inbound_srs {$local_part} {SRS_SECRET}} + data = $srs_recipient + + inbound_srs_failure: + driver = redirect + senders = : + domains = +my_domains + # detect inbound bounces which look SRS'd but are invalid + condition = ${if inbound_srs {$local_part} {}} + allow_fail + data = :fail: Invalid SRS recipient address + + #... further routers here + + + # transport; should look like the non-forward outbound + # one, plus the max_rcpt and return_path options + remote_forwarded_smtp: + driver = smtp + # modify the envelope from, for mails that we forward + max_rcpt = 1 + return_path = ${srs_encode {SRS_SECRET} {$return_path} {$original_domain}} +.endd + + +.wen + + + .section DMARC SECDMARC .cindex DMARC verification @@ -41385,6 +41811,7 @@ requirement, upon libidn2. .section "MTA operations" SECTi18nMTA .cindex SMTPUTF8 "ESMTP option" +.cindex "ESMTP extensions" SMTPUTF8 The main configuration option &%smtputf8_advertise_hosts%& specifies a host list. If this matches the sending host and accept_8bitmime is true (the default) then the ESMTP option @@ -41431,22 +41858,27 @@ may use the following modifier: control = utf8_downconvert control = utf8_downconvert/ .endd -This sets a flag requiring that addresses are converted to -a-label form before smtp delivery, for use in a -Message Submission Agent context. +This sets a flag requiring that envelope addresses are converted to +a-label form before smtp delivery. +This is usually for use in a Message Submission Agent context, +but could be used for any message. + If a value is appended it may be: .display -&`1 `& (default) mandatory downconversion +&`1 `& mandatory downconversion &`0 `& no downconversion &`-1 `& if SMTPUTF8 not supported by destination host .endd +If no value is given, 1 is used. If mua_wrapper is set, the utf8_downconvert control is initially set to -1. The smtp transport has an option &%utf8_downconvert%&. If set it must expand to one of the three values described above, -and it overrides any previously set value. +or an empty string. +If non-empty it overrides value previously set +(due to mua_wrapper or by an ACL control). There is no explicit support for VRFY and EXPN.