&<<CHAPlocalscan>>&)
&`lookup `& general lookup code and all lookups
&`memory `& memory handling
-&`pid `& add pid to debug output lines
+&`noutf8 `& modifier: avoid UTF-8 line-drawing
+&`pid `& modifier: add pid to debug output lines
&`process_info `& setting info for the process log
&`queue_run `& queue runs
&`receive `& general message reception logic
&`retry `& retry handling
&`rewrite `& address rewriting
&`route `& address routing
-&`timestamp `& add timestamp to debug output lines
+&`timestamp `& modifier: add timestamp to debug output lines
&`tls `& TLS logic
&`transport `& transports
&`uid `& changes of uid/gid and looking up uid/gid
of all debug output lines. This can be useful when trying to track down delays
in processing.
+.new
+.cindex debugging "UTF-8 in"
+.cindex UTF-8 "in debug output"
+The &`noutf8`& selector disables the use of
+UTF-8 line-drawing characters to group related information.
+When disabled. ascii-art is used instead.
+Using the &`+all`& option does not set this modifier,
+.wen
+
If the &%debug_print%& option is set in any driver, it produces output whenever
any debugging is selected, or if &%-v%& is used.
the implicit key is the host's IP address rather than its name (see section
&<<SECThoslispatsikey>>&).
.next
+.new
+.cindex lookup json
+.cindex json "lookup type"
+.cindex JSON expansions
+&(json)&: The given file is a text file with a JSON structure.
+An element of the structure is extracted, defined by the search key.
+The key is a list of subelement selectors
+(colon-separated by default but changeable in the usual way)
+which are applied in turn to select smaller and smaller portions
+of the JSON structure.
+If a selector is numeric, it must apply to a JSON array; the (zero-based)
+nunbered array element is selected.
+Otherwise it must apply to a JSON object; the named element is selected.
+The final resulting element can be a simple JSON type or a JSON object
+or array; for the latter two a string-representation os the JSON
+is returned.
+For elements of type string, the returned value is de-quoted.
+.wen
+.next
.cindex "linear search"
.cindex "lookup" "lsearch"
.cindex "lsearch lookup type"
&*Warning*&: Unlike most other single-key lookup types, a file of data for
&((n)wildlsearch)& can &'not'& be turned into a DBM or cdb file, because those
lookup types support only literal keys.
+
+.next
+.cindex "lookup" "spf"
+If Exim is built with SPF support, manual lookups can be done
+(as opposed to the standard ACL condition method.
+For details see section &<<SECSPF>>&.
.endlist ilist
This forces an expansion failure (see section &<<SECTforexpfai>>&);
{<&'string2'&>} must be present for &"fail"& to be recognized.
+.new
+.vitem "&*${extract json {*&<&'key'&>&*}{*&<&'string1'&>&*}{*&<&'string2'&>&*}&&&
+ {*&<&'string3'&>&*}}*&" &&&
+ "&*${extract jsons{*&<&'key'&>&*}{*&<&'string1'&>&*}{*&<&'string2'&>&*}&&&
+ {*&<&'string3'&>&*}}*&"
+.cindex "expansion" "extracting from JSON object"
+.cindex JSON expansions
+The key and <&'string1'&> are first expanded separately. Leading and trailing
+white space is removed from the key (but not from any of the strings). The key
+must not be empty and must not consist entirely of digits.
+The expanded <&'string1'&> must be of the form:
+.display
+{ <&'"key1"'&> : <&'value1'&> , <&'"key2"'&> , <&'value2'&> ... }
+.endd
+.vindex "&$value$&"
+The braces, commas and colons, and the quoting of the member name are required;
+the spaces are optional.
+Matching of the key against the member names is done case-sensitively.
+For the &"json"& variant,
+if a returned value is a JSON string, it retains its leading and
+trailing quotes.
+For the &"jsons"& variant, which is intended for use with JSON strings, the
+leading and trailing quotes are removed from the returned value.
+. XXX should be a UTF-8 compare
+
+The results of matching are handled as above.
+.wen
+
.vitem "&*${extract{*&<&'number'&>&*}{*&<&'separators'&>&*}&&&
{*&<&'string1'&>&*}{*&<&'string2'&>&*}{*&<&'string3'&>&*}}*&"
empty (for example, the fifth field above).
+.new
+.vitem "&*${extract json {*&<&'number'&>&*}}&&&
+ {*&<&'string1'&>&*}{*&<&'string2'&>&*}{*&<&'string3'&>&*}}*&" &&&
+ "&*${extract jsons{*&<&'number'&>&*}}&&&
+ {*&<&'string1'&>&*}{*&<&'string2'&>&*}{*&<&'string3'&>&*}}*&"
+.cindex "expansion" "extracting from JSON array"
+.cindex JSON expansions
+The <&'number'&> argument must consist entirely of decimal digits,
+apart from leading and trailing white space, which is ignored.
+
+Field selection and result handling is as above;
+there is no choice of field separator.
+For the &"json"& variant,
+if a returned value is a JSON string, it retains its leading and
+trailing quotes.
+For the &"jsons"& variant, which is intended for use with JSON strings, the
+leading and trailing quotes are removed from the returned value.
+.wen
+
+
.vitem &*${filter{*&<&'string'&>&*}{*&<&'condition'&>&*}}*&
.cindex "list" "selecting by condition"
.cindex "expansion" "selecting from list by condition"
.vitem &*$rheader_*&<&'header&~name'&>&*:*&&~or&~&*$rh_*&<&'header&~name'&>&*:*&
This item inserts &"raw"& header lines. It is described with the &%header%&
-expansion item above.
+expansion item in section &<<SECTexpansionitems>>& above.
.vitem "&*${run{*&<&'command'&>&*&~*&<&'args'&>&*}{*&<&'string1'&>&*}&&&
{*&<&'string2'&>&*}}*&"
To scan a named list, expand it with the &*listnamed*& operator.
+.new
+.vitem "&*forall_json{*&<&'a JSON array'&>&*}{*&<&'a condition'&>&*}*&" &&&
+ "&*forany_json{*&<&'a JSON array'&>&*}{*&<&'a condition'&>&*}*&" &&&
+ "&*forall_jsons{*&<&'a JSON array'&>&*}{*&<&'a condition'&>&*}*&" &&&
+ "&*forany_jsons{*&<&'a JSON array'&>&*}{*&<&'a condition'&>&*}*&"
+.cindex JSON "iterative conditions"
+.cindex JSON expansions
+.cindex expansion "&*forall_json*& condition"
+.cindex expansion "&*forany_json*& condition"
+.cindex expansion "&*forall_jsons*& condition"
+.cindex expansion "&*forany_jsons*& condition"
+As for the above, except that the first argument must, after expansion,
+be a JSON array.
+The array separator is not changeable.
+For the &"jsons"& variants the elements are expected to be JSON strings
+and have their quotes removed before the evaluation of the condition.
+.wen
+
+
.vitem &*ge&~{*&<&'string1'&>&*}{*&<&'string2'&>&*}*& &&&
&*gei&~{*&<&'string1'&>&*}{*&<&'string2'&>&*}*&
the value of &$authenticated_id$& is normally the login name of the calling
process. However, a trusted user can override this by means of the &%-oMai%&
command line option.
-This second case also sets up inforamtion used by the
+This second case also sets up information used by the
&$authresults$& expansion item.
.vitem &$authenticated_fail_id$&
inserting the message header line with the given name. Note that the name must
be terminated by colon or white space, because it may contain a wide variety of
characters. Note also that braces must &'not'& be used.
+See the full description in section &<<SECTexpansionitems>>& above.
.vitem &$headers_added$&
.vindex "&$headers_added$&"
A list of hosts, whether obtained via &%route_data%& or &%route_list%&, is
always separately expanded before use. If the expansion fails, the router
declines. The result of the expansion must be a colon-separated list of names
-and/or IP addresses, optionally also including ports. The format of each item
+and/or IP addresses, optionally also including ports.
+If the list is written with spaces, it must be protected with quotes.
+The format of each item
in the list is described in the next section. The list separator can be changed
as described in section &<<SECTlistconstruct>>&.
.option delay_after_cutoff smtp boolean true
+.cindex "final cutoff" "retries, controlling"
+.cindex retry "final cutoff"
This option controls what happens when all remote IP addresses for a given
domain have been inaccessible for so long that they have passed their retry
cutoff times.
If the value of this option begins with a digit it is taken as a port number;
otherwise it is looked up using &[getservbyname()]&. The default value is
-normally &"smtp"&, but if &%protocol%& is set to &"lmtp"&, the default is
-&"lmtp"&. If the expansion fails, or if a port number cannot be found, delivery
+normally &"smtp"&,
+but if &%protocol%& is set to &"lmtp"& the default is &"lmtp"&
+and if &%protocol%& is set to &"smtps"& the default is &"smtps"&.
+If the expansion fails, or if a port number cannot be found, delivery
is deferred.
+.new
+Note that at least one Linux distribution has been seen failing
+to put &"smtps"& in its &"/etc/services"& file, resulting is such deferrals.
+.wen
+
.option protocol smtp string smtp
If this option is set to &"smtps"&, the default value for the &%port%& option
changes to &"smtps"&, and the transport initiates TLS immediately after
connecting, as an outbound SSL-on-connect, instead of using STARTTLS to upgrade.
-The Internet standards bodies strongly discourage use of this mode.
+.new
+The Internet standards bodies used to strongly discourage use of this mode,
+but as of RFC 8314 it is perferred over STARTTLS for message submission
+(as distinct from MTA-MTA communication).
+.wen
.option retry_include_ip_address smtp boolean&!! true
messages. If this delivery fails, the address fails immediately. The
post-cutoff retry time is not used.
+.cindex "final cutoff" "retries, controlling"
+.cindex retry "final cutoff"
If the delivery is remote, there are two possibilities, controlled by the
.oindex "&%delay_after_cutoff%&"
&%delay_after_cutoff%& option of the &(smtp)& transport. The option is true by
-default. Until the post-cutoff retry time for one of the IP addresses is
+default. Until the post-cutoff retry time for one of the IP addresses,
+as set by the &%retry_data_expire%& option, is
reached, the failing email address is bounced immediately, without a delivery
attempt taking place. After that time, one new delivery attempt is made to
those IP addresses that are past their retry times, and if that still fails,
AUTH_CRAM_MD5=yes
AUTH_CYRUS_SASL=yes
AUTH_DOVECOT=yes
+AUTH_EXTERNAL=yes
AUTH_GSASL=yes
AUTH_HEIMDAL_GSSAPI=yes
AUTH_PLAINTEXT=yes
the Cyrus SASL authentication library.
The third is an interface to Dovecot's authentication system, delegating the
work via a socket interface.
-The fourth provides an interface to the GNU SASL authentication library, which
+.new
+The fourth provides for negotiation of authentication done via non-SMTP means,
+as defined by RFC 4422 Appendix A.
+.wen
+The fifth provides an interface to the GNU SASL authentication library, which
provides mechanisms but typically not data sources.
-The fifth provides direct access to Heimdal GSSAPI, geared for Kerberos, but
+The sixth provides direct access to Heimdal GSSAPI, geared for Kerberos, but
supporting setting a server keytab.
-The sixth can be configured to support
+The seventh can be configured to support
the PLAIN authentication mechanism (RFC 2595) or the LOGIN mechanism, which is
-not formally documented, but used by several MUAs. The seventh authenticator
+not formally documented, but used by several MUAs.
+The eighth authenticator
supports Microsoft's &'Secure Password Authentication'& mechanism.
-The eighth is an Exim authenticator but not an SMTP one;
+The last is an Exim authenticator but not an SMTP one;
instead it can use information from a TLS negotiation.
The authenticators are configured using the same syntax as other drivers (see
.option server_set_id authenticators string&!! unset
.vindex "&$authenticated_id$&"
+.vindex "&$authenticated_fail_id$&"
When an Exim server successfully authenticates a client, this string is
expanded using data from the authentication, and preserved for any incoming
messages in the variable &$authenticated_id$&. It is also included in the log
lines for incoming messages. For example, a user/password authenticator
configuration might preserve the user name that was used to authenticate, and
refer to it subsequently during delivery of the message.
+On a failing authentication the expansion result is instead saved in
+the &$authenticated_fail_id$& variable.
If expansion fails, the option is ignored.
expansion is &"1"&, &"yes"&, or &"true"&, authentication succeeds and the
generic &%server_set_id%& option is expanded and saved in &$authenticated_id$&.
For any other result, a temporary error code is returned, with the expanded
-string as the error text
+string as the error text.
&*Warning*&: If you use a lookup in the expansion to find the user's
password, be sure to make the authentication fail if the user is unknown.
+. ////////////////////////////////////////////////////////////////////////////
+. ////////////////////////////////////////////////////////////////////////////
+
+.chapter "The external authenticator" "CHAPexternauth"
+.scindex IIDexternauth1 "&(external)& authenticator"
+.scindex IIDexternauth2 "authenticators" "&(external)&"
+.cindex "authentication" "Client Certificate"
+.cindex "authentication" "X509"
+.cindex "Certificate-based authentication"
+The &(external)& authenticator provides support for
+authentication based on non-SMTP information.
+The specification is in RFC 4422 Appendix A
+(&url(https://tools.ietf.org/html/rfc4422)).
+It is only a transport and negotiation mechanism;
+the process of authentication is entirely controlled
+by the server configuration.
+
+The client presents an identity in-clear.
+It is probably wise for a server to only advertise,
+and for clients to only attempt,
+this authentication method on a secure (eg. under TLS) connection.
+
+One possible use, compatible with the
+K-9 Mail Andoid client (&url(https://k9mail.github.io/)),
+is for using X509 client certificates.
+
+It thus overlaps in function with the TLS authenticator
+(see &<<CHAPtlsauth>>&)
+but is a full SMTP SASL authenticator
+rather than being implicit for TLS-connection carried
+client certificates only.
+
+The examples and discussion in this chapter assume that
+client-certificate authentication is being done.
+
+The client must present a certificate,
+for which it must have been requested via the
+&%tls_verify_hosts%& or &%tls_try_verify_hosts%& main options
+(see &<<CHAPTLS>>&).
+For authentication to be effective the certificate should be
+verifiable against a trust-anchor certificate known to the server.
+
+.section "External options" "SECTexternsoptions"
+.cindex "options" "&(external)& authenticator (server)"
+The &(external)& authenticator has two server options:
+
+.option server_param2 external string&!! unset
+.option server_param3 external string&!! unset
+.cindex "variables (&$auth1$& &$auth2$& etc)" "in &(external)& authenticator"
+These options are expanded before the &%server_condition%& option
+and the result are placed in &$auth2$& and &$auth3$& resectively.
+If the expansion is forced to fail, authentication fails. Any other expansion
+failure causes a temporary error code to be returned.
+
+They can be used to clarify the coding of a complex &%server_condition%&.
+
+.section "Using external in a server" "SECTexternserver"
+.cindex "AUTH" "in &(external)& authenticator"
+.cindex "numerical variables (&$1$& &$2$& etc)" &&&
+ "in &(external)& authenticator"
+.vindex "&$auth1$&, &$auth2$&, etc"
+.cindex "base64 encoding" "in &(external)& authenticator"
+
+When running as a server, &(external)& performs the authentication test by
+expanding a string. The data sent by the client with the AUTH command, or in
+response to subsequent prompts, is base64 encoded, and so may contain any byte
+values when decoded. The decoded value is treated as
+an identity for authentication and
+placed in the expansion variable &$auth1$&.
+
+For compatibility with previous releases of Exim, the value is also placed in
+the expansion variable &$1$&. However, the use of this
+variable for this purpose is now deprecated, as it can lead to confusion in
+string expansions that also use them for other things.
+
+.vindex "&$authenticated_id$&"
+Once an identity has been received,
+&%server_condition%& is expanded. If the expansion is forced to fail,
+authentication fails. Any other expansion failure causes a temporary error code
+to be returned. If the result of a successful expansion is an empty string,
+&"0"&, &"no"&, or &"false"&, authentication fails. If the result of the
+expansion is &"1"&, &"yes"&, or &"true"&, authentication succeeds and the
+generic &%server_set_id%& option is expanded and saved in &$authenticated_id$&.
+For any other result, a temporary error code is returned, with the expanded
+string as the error text.
+
+Example:
+.code
+ext_ccert_san_mail:
+ driver = external
+ public_name = EXTERNAL
+
+ server_advertise_condition = $tls_in_certificate_verified
+ server_param2 = ${certextract {subj_altname,mail,>:} \
+ {$tls_in_peercert}}
+ server_condition = ${if forany {$auth2} \
+ {eq {$item}{$auth1}}}
+ server_set_id = $auth1
+.endd
+This accepts a client certificate that is verifiable against any
+of your configured trust-anchors
+(which usually means the full set of public CAs)
+and which has a mail-SAN matching the claimed identity sent by the client.
+
+Note that, up to TLS1.2, the client cert is on the wire in-clear, including the SAN,
+The account name is therefore guessable by an opponent.
+TLS 1.3 protects both server and client certificates, and is not vulnerable
+in this way.
+Likewise, a traditional plaintext SMTP AUTH done inside TLS is not.
+
+
+.section "Using external in a client" "SECTexternclient"
+.cindex "options" "&(external)& authenticator (client)"
+The &(external)& authenticator has one client option:
+
+.option client_send external string&!! unset
+This option is expanded and sent with the AUTH command as the
+identity being asserted.
+
+Example:
+.code
+ext_ccert:
+ driver = external
+ public_name = EXTERNAL
+
+ client_condition = ${if !eq{$tls_out_cipher}{}}
+ client_send = myaccount@smarthost.example.net
+.endd
+
+
+.ecindex IIDexternauth1
+.ecindex IIDexternauth2
+
+
+
+
+
. ////////////////////////////////////////////////////////////////////////////
. ////////////////////////////////////////////////////////////////////////////
driver = tls
server_param1 = ${certextract {subj_altname,mail,>:} \
{$tls_in_peercert}}
- server_condition = ${if forany {$auth1} \
+ server_condition = ${if and { {eq{$tls_in_certificate_verified}{1}} \
+ {forany {$auth1} \
{!= {0} \
{${lookup ldap{ldap:///\
mailname=${quote_ldap_dn:${lc:$item}},\
ou=users,LDAP_DC?mailid} {$value}{0} \
- } } } }
+ } } } }}}
server_set_id = ${if = {1}{${listcount:$auth1}} {$auth1}{}}
.endd
This accepts a client certificate that is verifiable against any
of your configured trust-anchors
(which usually means the full set of public CAs)
and which has a SAN with a good account name.
-Note that the client cert is on the wire in-clear, including the SAN,
-whereas a plaintext SMTP AUTH done inside TLS is not.
+
+Note that, up to TLS1.2, the client cert is on the wire in-clear, including the SAN,
+The account name is therefore guessable by an opponent.
+TLS 1.3 protects both server and client certificates, and is not vulnerable
+in this way.
+Likewise, a traditional plaintext SMTP AUTH done inside TLS is not.
. An alternative might use
. .code
apply to all TLS connections. For any host that matches one of these options,
Exim requests a certificate as part of the setup of the TLS session. The
contents of the certificate are verified by comparing it with a list of
-expected certificates.
+expected trust-anchors or certificates.
These may be the system default set (depending on library version),
an explicit file or,
depending on library version, a directory, identified by
.endd
where &_/cert/file_& contains a single certificate.
+There is no checking of names of the client against the certificate
+Subject Name or Subject Alternate Names.
+
The difference between &%tls_verify_hosts%& and &%tls_try_verify_hosts%& is
what happens if the client does not supply a certificate, or if the certificate
does not match any of the certificates in the collection named by
certificate verification to the listed servers. Verification either must
or need not succeed respectively.
+The &%tls_verify_cert_hostnames%& option lists hosts for which additional
+checks are made: that the host name (the one in the DNS A record)
+is valid for the certificate.
+The option defaults to always checking.
+
+.new
+Do not use a client certificate that contains an "OCSP Must-Staple" extension.
+TLS 1.2 and below does not support client-side OCSP stapling, and
+(as of writing) the TLS libraries do not provide for it even with
+TLS 1.3.
+Be careful when using the same certificate for server- and
+client-certificate for this reason.
+.wen
+
The &(smtp)& transport has two OCSP-related options:
&%hosts_require_ocsp%&; a host-list for which a Certificate Status
is requested and required for the connection to proceed. The default
DANE-TA usage is effectively declaring a specific CA to be used; this might be a private CA or a public,
well-known one.
A private CA at simplest is just a self-signed certificate (with certain
-attributes) which is used to sign cerver certificates, but running one securely
+attributes) which is used to sign server certificates, but running one securely
does require careful arrangement.
With DANE-TA, as implemented in Exim and commonly in other MTAs,
the server TLS handshake must transmit the entire certificate chain from CA to server-certificate.
&%pipelining%&: A field is added to delivery and accept
log lines when the ESMTP PIPELINING extension was used.
The field is a single "L".
+
On accept lines, where PIPELINING was offered but not used by the client,
the field has a minus appended.
.next
git://git.exim.org / exim.git / blobdiff