Removed doc references to relay-test.mail-abuse.org

[users/jgh/exim.git] / doc / doc-docbook / spec.xfpt

diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt
index 33f35bf89e83f8d4752ea9b04eba73271754b206..673cdf250fcfbc3df28164bad82b9f8530d83d85 100644 (file)
--- a/doc/doc-docbook/spec.xfpt
+++ b/doc/doc-docbook/spec.xfpt
@@ -9677,7 +9677,8 @@ ${readsocket{inet:[::1]:1234}{request string}}
 Only a single host name may be given, but if looking it up yields more than
 one IP address, they are each tried in turn until a connection is made. For
 both kinds of socket, Exim makes a connection, writes the request string
 Only a single host name may be given, but if looking it up yields more than
 one IP address, they are each tried in turn until a connection is made. For
 both kinds of socket, Exim makes a connection, writes the request string

-(unless it is an empty string) and reads from the socket until an end-of-file
+unless it is an empty string; and no terminating NUL is ever sent)
+and reads from the socket until an end-of-file

 is read. A timeout of 5 seconds is applied. Additional, optional arguments
 extend what can be done. Firstly, you can vary the timeout. For example:
 .code
 is read. A timeout of 5 seconds is applied. Additional, optional arguments
 extend what can be done. Firstly, you can vary the timeout. For example:
 .code


@@ -10033,6 +10034,25 @@ environments where Exim uses base 36 instead of base 62 for its message
 identifiers, base-36 digits. The number is converted to decimal and output as a
 string.
 
 identifiers, base-36 digits. The number is converted to decimal and output as a
 string.
 

+.new
+.vitem &*${base64:*&<&'string'&>&*}*&
+.cindex "expansion" "base64 encoding"
+.cindex "base64 encoding" "in string expansion"
+.cindex "&%base64%& expansion item"
+.cindex certificate "base64 of DER"
+This operator converts a string into one that is base64 encoded.
+
+If the string is a single variable of type certificate,
+returns the base64 encoding of the DER form of the certificate.
+
+
+.vitem &*${base64d:*&<&'string'&>&*}*&
+.cindex "expansion" "base64 decoding"
+.cindex "base64 decoding" "in string expansion"
+.cindex "&%base64d%& expansion item"
+This operator converts a base64-encoded string into the un-coded form.
+.wen
+

 
 .vitem &*${domain:*&<&'string'&>&*}*&
 .cindex "domain" "extraction"
 
 .vitem &*${domain:*&<&'string'&>&*}*&
 .cindex "domain" "extraction"


@@ -10282,11 +10302,14 @@ Letters in IPv6 addresses are always output in lower case.
 .vitem &*${md5:*&<&'string'&>&*}*&
 .cindex "MD5 hash"
 .cindex "expansion" "MD5 hash"
 .vitem &*${md5:*&<&'string'&>&*}*&
 .cindex "MD5 hash"
 .cindex "expansion" "MD5 hash"

-.cindex "certificate fingerprint"
+.cindex certificate fingerprint

 .cindex "&%md5%& expansion item"
 The &%md5%& operator computes the MD5 hash value of the string, and returns it
 as a 32-digit hexadecimal number, in which any letters are in lower case.
 
 .cindex "&%md5%& expansion item"
 The &%md5%& operator computes the MD5 hash value of the string, and returns it
 as a 32-digit hexadecimal number, in which any letters are in lower case.
 

+If the string is a single variable of type certificate,
+returns the MD5 hash fingerprint of the certificate.
+

 
 .vitem &*${nhash_*&<&'n'&>&*_*&<&'m'&>&*:*&<&'string'&>&*}*&
 .cindex "expansion" "numeric hash"
 
 .vitem &*${nhash_*&<&'n'&>&*_*&<&'m'&>&*:*&<&'string'&>&*}*&
 .cindex "expansion" "numeric hash"


@@ -10420,15 +10443,18 @@ variables or headers inside regular expressions.
 .vitem &*${sha1:*&<&'string'&>&*}*&
 .cindex "SHA-1 hash"
 .cindex "expansion" "SHA-1 hashing"
 .vitem &*${sha1:*&<&'string'&>&*}*&
 .cindex "SHA-1 hash"
 .cindex "expansion" "SHA-1 hashing"

-.cindex "certificate fingerprint"
+.cindex certificate fingerprint

 .cindex "&%sha2%& expansion item"
 The &%sha1%& operator computes the SHA-1 hash value of the string, and returns
 it as a 40-digit hexadecimal number, in which any letters are in upper case.
 
 .cindex "&%sha2%& expansion item"
 The &%sha1%& operator computes the SHA-1 hash value of the string, and returns
 it as a 40-digit hexadecimal number, in which any letters are in upper case.
 

+If the string is a single variable of type certificate,
+returns the SHA-1 hash fingerprint of the certificate.
+

 
 .vitem &*${sha256:*&<&'certificate'&>&*}*&
 .cindex "SHA-256 hash"
 
 .vitem &*${sha256:*&<&'certificate'&>&*}*&
 .cindex "SHA-256 hash"

-.cindex "certificate fingerprint"
+.cindex certificate fingerprint

 .cindex "expansion" "SHA-256 hashing"
 .cindex "&%sha256%& expansion item"
 The &%sha256%& operator computes the SHA-256 hash fingerprint of the
 .cindex "expansion" "SHA-256 hashing"
 .cindex "&%sha256%& expansion item"
 The &%sha256%& operator computes the SHA-256 hash fingerprint of the


@@ -10457,10 +10483,10 @@ the system administrator. &*Warning*&: The file size may be incorrect on 32-bit
 systems for files larger than 2GB.
 
 .vitem &*${str2b64:*&<&'string'&>&*}*&
 systems for files larger than 2GB.
 
 .vitem &*${str2b64:*&<&'string'&>&*}*&

-.cindex "expansion" "base64 encoding"
-.cindex "base64 encoding" "in string expansion"

 .cindex "&%str2b64%& expansion item"
 .cindex "&%str2b64%& expansion item"

-This operator converts a string into one that is base64 encoded.
+.new
+Now deprecated, a synonym for the &%base64%& expansion operator.
+.wen

 
 
 
 
 
 


@@ -11327,7 +11353,7 @@ this variable holds the pipe command when the transport is running.
 .vitem "&$auth1$& &-- &$auth3$&"
 .vindex "&$auth1$&, &$auth2$&, etc"
 These variables are used in SMTP authenticators (see chapters
 .vitem "&$auth1$& &-- &$auth3$&"
 .vindex "&$auth1$&, &$auth2$&, etc"
 These variables are used in SMTP authenticators (see chapters

-&<<CHAPplaintext>>&&--&<<CHAPspa>>&). Elsewhere, they are empty.
+&<<CHAPplaintext>>&&--&<<CHAPtlsauth>>&). Elsewhere, they are empty.

 
 .vitem &$authenticated_id$&
 .cindex "authentication" "id"
 
 .vitem &$authenticated_id$&
 .cindex "authentication" "id"


@@ -12674,6 +12700,7 @@ If TLS has not been negotiated, the value will be 0.
 
 .vitem &$tls_in_ourcert$&
 .vindex "&$tls_in_ourcert$&"
 
 .vitem &$tls_in_ourcert$&
 .vindex "&$tls_in_ourcert$&"

+.cindex certificate veriables

 This variable refers to the certificate presented to the peer of an
 inbound connection when the message was received.
 It is only useful as the argument of a
 This variable refers to the certificate presented to the peer of an
 inbound connection when the message was received.
 It is only useful as the argument of a


@@ -12764,6 +12791,7 @@ See &$tls_in_ocsp$& for values.
 .vitem &$tls_in_peerdn$&
 .vindex "&$tls_in_peerdn$&"
 .vindex "&$tls_peerdn$&"
 .vitem &$tls_in_peerdn$&
 .vindex "&$tls_in_peerdn$&"
 .vindex "&$tls_peerdn$&"

+.cindex certificate "extracting fields"

 When a message is received from a remote host over an encrypted SMTP
 connection, and Exim is configured to request a certificate from the client,
 the value of the Distinguished Name of the certificate is made available in the
 When a message is received from a remote host over an encrypted SMTP
 connection, and Exim is configured to request a certificate from the client,
 the value of the Distinguished Name of the certificate is made available in the


@@ -13759,6 +13787,7 @@ See also the &'Policy controls'& section above.
 .row &%bounce_message_file%&         "content of bounce"
 .row &%bounce_message_text%&         "content of bounce"
 .row &%bounce_return_body%&          "include body if returning message"
 .row &%bounce_message_file%&         "content of bounce"
 .row &%bounce_message_text%&         "content of bounce"
 .row &%bounce_return_body%&          "include body if returning message"

+.row &%bounce_return_linesize_limit%& "limit on returned message line length"

 .row &%bounce_return_message%&       "include original message in bounce"
 .row &%bounce_return_size_limit%&    "limit on returned message"
 .row &%bounce_sender_authentication%& "send authenticated sender with bounce"
 .row &%bounce_return_message%&       "include original message in bounce"
 .row &%bounce_return_size_limit%&    "limit on returned message"
 .row &%bounce_sender_authentication%& "send authenticated sender with bounce"


@@ -14067,6 +14096,24 @@ error that is detected during reception, only those header lines preceding the
 point at which the error was detected are returned.
 .cindex "bounce message" "including original"
 
 point at which the error was detected are returned.
 .cindex "bounce message" "including original"
 

+.option bounce_return_linesize_limit main integer 998
+.cindex "size" "of bounce lines, limit"
+.cindex "bounce message" "line length limit"
+.cindex "limit" "bounce message line length"
+This option sets a limit in bytes on the line length of messages
+that are returned to senders due to delivery problems,
+when &%bounce_return_message%& is true.
+The default value corresponds to RFC limits.
+If the message being returned has lines longer than this value it is
+treated as if the &%bounce_return_size_limit%& (below) restriction was exceeded.
+
+The option also applies to bounces returned when an error is detected
+during reception of a messsage.
+In this case lines from the original are truncated.
+
+The option does not apply to messages generated by an &(autoreply)& transport.
+
+

 .option bounce_return_message main boolean true
 If this option is set false, none of the original message is included in
 bounce messages generated by Exim. See also &%bounce_return_size_limit%& and
 .option bounce_return_message main boolean true
 If this option is set false, none of the original message is included in
 bounce messages generated by Exim. See also &%bounce_return_size_limit%& and


@@ -14997,6 +15044,7 @@ logged.
 
 .option ldap_ca_cert_dir main string unset
 .cindex "LDAP", "TLS CA certificate directory"
 
 .option ldap_ca_cert_dir main string unset
 .cindex "LDAP", "TLS CA certificate directory"

+.cindex certificate "directory for LDAP"

 This option indicates which directory contains CA certificates for verifying
 a TLS certificate presented by an LDAP server.
 While Exim does not provide a default value, your SSL library may.
 This option indicates which directory contains CA certificates for verifying
 a TLS certificate presented by an LDAP server.
 While Exim does not provide a default value, your SSL library may.


@@ -15006,6 +15054,7 @@ and constrained to be a directory.
 
 .option ldap_ca_cert_file main string unset
 .cindex "LDAP", "TLS CA certificate file"
 
 .option ldap_ca_cert_file main string unset
 .cindex "LDAP", "TLS CA certificate file"

+.cindex certificate "file for LDAP"

 This option indicates which file contains CA certificates for verifying
 a TLS certificate presented by an LDAP server.
 While Exim does not provide a default value, your SSL library may.
 This option indicates which file contains CA certificates for verifying
 a TLS certificate presented by an LDAP server.
 While Exim does not provide a default value, your SSL library may.


@@ -15015,6 +15064,7 @@ and constrained to be a file.
 
 .option ldap_cert_file main string unset
 .cindex "LDAP" "TLS client certificate file"
 
 .option ldap_cert_file main string unset
 .cindex "LDAP" "TLS client certificate file"

+.cindex certificate "file for LDAP"

 This option indicates which file contains an TLS client certificate which
 Exim should present to the LDAP server during TLS negotiation.
 Should be used together with &%ldap_cert_key%&.
 This option indicates which file contains an TLS client certificate which
 Exim should present to the LDAP server during TLS negotiation.
 Should be used together with &%ldap_cert_key%&.


@@ -15022,6 +15072,7 @@ Should be used together with &%ldap_cert_key%&.
 
 .option ldap_cert_key main string unset
 .cindex "LDAP" "TLS client key file"
 
 .option ldap_cert_key main string unset
 .cindex "LDAP" "TLS client key file"

+.cindex certificate "key for LDAP"

 This option indicates which file contains the secret/private key to use
 to prove identity to the LDAP server during TLS negotiation.
 Should be used together with &%ldap_cert_file%&, which contains the
 This option indicates which file contains the secret/private key to use
 to prove identity to the LDAP server during TLS negotiation.
 Should be used together with &%ldap_cert_file%&, which contains the


@@ -15400,7 +15451,7 @@ harm. This option overrides the &%pipe_as_creator%& option of the &(pipe)&
 transport driver.
 
 
 transport driver.
 
 

-.option openssl_options main "string list" "+no_sslv2"
+.option openssl_options main "string list" "+no_sslv2 +single_dh_use"

 .cindex "OpenSSL "compatibility options"
 This option allows an administrator to adjust the SSL options applied
 by OpenSSL to connections.  It is given as a space-separated list of items,
 .cindex "OpenSSL "compatibility options"
 This option allows an administrator to adjust the SSL options applied
 by OpenSSL to connections.  It is given as a space-separated list of items,


@@ -23367,6 +23418,15 @@ the message. As a result, the overall timeout for a message depends on the size
 of the message. Its value must not be zero. See also &%final_timeout%&.
 
 
 of the message. Its value must not be zero. See also &%final_timeout%&.
 
 

+.option dkim_domain smtp string&!! unset
+.option dkim_selector smtp string&!! unset
+.option dkim_private_key smtp string&!! unset
+.option dkim_canon smtp string&!! unset
+.option dkim_strict smtp string&!! unset
+.option dkim_sign_headers smtp string&!! unset
+DKIM signing options.  For details see &<<SECDKIMSIGN>>&.
+
+

 .option delay_after_cutoff smtp boolean true
 This option controls what happens when all remote IP addresses for a given
 domain have been inaccessible for so long that they have passed their retry
 .option delay_after_cutoff smtp boolean true
 This option controls what happens when all remote IP addresses for a given
 domain have been inaccessible for so long that they have passed their retry


@@ -24009,7 +24069,7 @@ and certificate verification fails the TLS connection is closed.
 .option tls_verify_hosts smtp "host list&!!" unset
 .cindex "TLS" "server certificate verification"
 .cindex "certificate" "verification of server"
 .option tls_verify_hosts smtp "host list&!!" unset
 .cindex "TLS" "server certificate verification"
 .cindex "certificate" "verification of server"

-This option gives a list of hosts for which. on encrypted connections,
+This option gives a list of hosts for which, on encrypted connections,

 certificate verification must succeed.
 The &%tls_verify_certificates%& option must also be set.
 If both this option and &%tls_try_verify_hosts%& are unset
 certificate verification must succeed.
 The &%tls_verify_certificates%& option must also be set.
 If both this option and &%tls_try_verify_hosts%& are unset


@@ -26461,6 +26521,24 @@ tls:
                        }    }   } }
   server_set_id =     ${if = {1}{${listcount:$auth1}} {$auth1}{}}
 .endd
                        }    }   } }
   server_set_id =     ${if = {1}{${listcount:$auth1}} {$auth1}{}}
 .endd

+This accepts a client certificate that is verifiable against any
+of your configured trust-anchors
+which usually means the full set of public CAs)
+and which has a SAN with a good account name.
+Note that the client cert is on the wire in-clear, including the SAN,
+whereas a plaintext SMTP AUTH done inside TLS is not.
+
+. An alternative might use
+. .code
+.   server_param1 = ${sha256:$tls_in_peercert}
+. .endd
+. to require one of a set of specific certs that define a given account
+. (the verification is still required, but mostly irrelevant).
+. This would help for per-device use.
+. 
+. However, for the future we really need support for checking a
+. user cert in LDAP - which probably wants a base-64 DER.
+

 .ecindex IIDtlsauth1
 .ecindex IIDtlsauth2
 
 .ecindex IIDtlsauth1
 .ecindex IIDtlsauth2
 


@@ -27382,9 +27460,6 @@ a realistic ACL for checking RCPT commands. This is discussed in chapter
 .section "Testing ACLs" "SECID188"
 The &%-bh%& command line option provides a way of testing your ACL
 configuration locally by running a fake SMTP session with which you interact.
 .section "Testing ACLs" "SECID188"
 The &%-bh%& command line option provides a way of testing your ACL
 configuration locally by running a fake SMTP session with which you interact.

-The host &'relay-test.mail-abuse.org'& provides a service for checking your
-relaying configuration (see section &<<SECTcheralcon>>& for more details).
-

 
 
 .section "Specifying when ACLs are used" "SECID189"
 
 
 .section "Specifying when ACLs are used" "SECID189"


@@ -27499,6 +27574,12 @@ Note that a client may issue more than one EHLO or HELO command in an SMTP
 session, and indeed is required to issue a new EHLO or HELO after successfully
 setting up encryption following a STARTTLS command.
 
 session, and indeed is required to issue a new EHLO or HELO after successfully
 setting up encryption following a STARTTLS command.
 

+.new
+Note also that a deny neither forces the client to go away nor means that
+mail will be refused on the connection.  Consider checking for
+&$sender_helo_name$& being defined in a MAIL or RCPT ACL to do that.
+.wen
+

 If the command is accepted by an &%accept%& verb that has a &%message%&
 modifier, the message may not contain more than one line (it will be truncated
 at the first newline and a panic logged if it does). Such a message cannot
 If the command is accepted by an &%accept%& verb that has a &%message%&
 modifier, the message may not contain more than one line (it will be truncated
 at the first newline and a panic logged if it does). Such a message cannot


@@ -28533,6 +28614,14 @@ Note also that headers cannot be
 modified by any of the post-data ACLs (DATA, MIME and DKIM).
 Headers may be modified by routers (subject to the above) and transports.
 
 modified by any of the post-data ACLs (DATA, MIME and DKIM).
 Headers may be modified by routers (subject to the above) and transports.
 

+.new
+All the usual ACLs are called; if one results in the message being
+rejected, all effort spent in delivery (including the costs on
+the ultimate destination) will be wasted.
+Note that in the case of data-time ACLs this includes the entire
+message body.
+.wen
+

 Cutthrough delivery is not supported via transport-filters or when DKIM signing
 of outgoing messages is done, because it sends data to the ultimate destination
 before the entire message has been received from the source.
 Cutthrough delivery is not supported via transport-filters or when DKIM signing
 of outgoing messages is done, because it sends data to the ultimate destination
 before the entire message has been received from the source.


@@ -30821,14 +30910,6 @@ in chapter &<<CHAPdefconfil>>&.
 You can check the relay characteristics of your configuration in the same way
 that you can test any ACL behaviour for an incoming SMTP connection, by using
 the &%-bh%& option to run a fake SMTP session with which you interact.
 You can check the relay characteristics of your configuration in the same way
 that you can test any ACL behaviour for an incoming SMTP connection, by using
 the &%-bh%& option to run a fake SMTP session with which you interact.

-
-For specifically testing for unwanted relaying, the host
-&'relay-test.mail-abuse.org'& provides a useful service. If you telnet to this
-host from the host on which Exim is running, using the normal telnet port, you
-will see a normal telnet connection message and then quite a long delay. Be
-patient. The remote host is making an SMTP connection back to your host, and
-trying a number of common probes to test for open relay vulnerability. The
-results of the tests will eventually appear on your terminal.

 .ecindex IIDacl
 
 
 .ecindex IIDacl
 
 


@@ -33991,13 +34072,20 @@ specific badly-behaved hosts that you have to live with.
 When Exim receives a VRFY or EXPN command on a TCP/IP connection, it
 runs the ACL specified by &%acl_smtp_vrfy%& or &%acl_smtp_expn%& (as
 appropriate) in order to decide whether the command should be accepted or not.
 When Exim receives a VRFY or EXPN command on a TCP/IP connection, it
 runs the ACL specified by &%acl_smtp_vrfy%& or &%acl_smtp_expn%& (as
 appropriate) in order to decide whether the command should be accepted or not.

-If no ACL is defined, the command is rejected.

 
 

+.new

 .cindex "VRFY" "processing"
 .cindex "VRFY" "processing"

+When no ACL is defined for VRFY, or if it rejects without
+setting an explicit response code, the command is accepted
+(with a 252 SMTP response code)
+in order to support awkward clients that do a VRFY before every RCPT.
+.wen

 When VRFY is accepted, it runs exactly the same code as when Exim is
 When VRFY is accepted, it runs exactly the same code as when Exim is

-called with the &%-bv%& option.
+called with the &%-bv%& option, and returns 250/451/550
+SMTP response codes.

 
 .cindex "EXPN" "processing"
 
 .cindex "EXPN" "processing"

+If no ACL for EXPN is defined, the command is rejected.

 When EXPN is accepted, a single-level expansion of the address is done.
 EXPN is treated as an &"address test"& (similar to the &%-bt%& option) rather
 than a verification (the &%-bv%& option). If an unqualified local part is given
 When EXPN is accepted, a single-level expansion of the address is done.
 EXPN is treated as an &"address test"& (similar to the &%-bt%& option) rather
 than a verification (the &%-bv%& option). If an unqualified local part is given


@@ -35493,7 +35581,7 @@ the following table:
 &`    `&        on &"Completed"& lines: time spent on queue
 &`R   `&        on &`<=`& lines: reference for local bounce
 &`    `&        on &`=>`&  &`**`& and &`==`& lines: router name
 &`    `&        on &"Completed"& lines: time spent on queue
 &`R   `&        on &`<=`& lines: reference for local bounce
 &`    `&        on &`=>`&  &`**`& and &`==`& lines: router name

-&`S   `&        size of message
+&`S   `&        size of message in bytes

 &`SNI `&        server name indication from TLS client hello
 &`ST  `&        shadow transport name
 &`T   `&        on &`<=`& lines: message subject (topic)
 &`SNI `&        server name indication from TLS client hello
 &`ST  `&        shadow transport name
 &`T   `&        on &`<=`& lines: message subject (topic)


@@ -37886,8 +37974,10 @@ linked to a domain which that entity controls.  It permits reputation to
 be tracked on a per-domain basis, rather than merely upon source IP address.
 DKIM is documented in RFC 4871.
 
 be tracked on a per-domain basis, rather than merely upon source IP address.
 DKIM is documented in RFC 4871.
 

-Since version 4.70, DKIM support is compiled into Exim by default. It can be
-disabled by setting DISABLE_DKIM=yes in &_Local/Makefile_&.
+.new
+DKIM support is compiled into Exim by default if TLS support is present.
+.wen
+It can be disabled by setting DISABLE_DKIM=yes in &_Local/Makefile_&.

 
 Exim's DKIM implementation allows to
 .olist
 
 Exim's DKIM implementation allows to
 .olist


@@ -37922,7 +38012,7 @@ where you accept mail from relay sources (internal hosts or authenticated
 senders).
 
 
 senders).
 
 

-.section "Signing outgoing messages" "SECID513"
+.section "Signing outgoing messages" "SECDKIMSIGN"

 .cindex "DKIM" "signing"
 
 Signing is implemented by setting private options on the SMTP transport.
 .cindex "DKIM" "signing"
 
 Signing is implemented by setting private options on the SMTP transport.


@@ -37937,7 +38027,7 @@ option is put into the &%$dkim_domain%& expansion variable.
 MANDATORY:
 This sets the key selector string. You can use the &%$dkim_domain%& expansion
 variable to look up a matching selector. The result is put in the expansion
 MANDATORY:
 This sets the key selector string. You can use the &%$dkim_domain%& expansion
 variable to look up a matching selector. The result is put in the expansion

-variable &%$dkim_selector%& which should be used in the &%dkim_private_key%&
+variable &%$dkim_selector%& which may be used in the &%dkim_private_key%&

 option along with &%$dkim_domain%&.
 
 .option dkim_private_key smtp string&!! unset
 option along with &%$dkim_domain%&.
 
 .option dkim_private_key smtp string&!! unset


@@ -38335,7 +38425,7 @@ UTF-8 form internally; any comparison or regular-expression use will
 require appropriate care.  Filenames created, eg. by
 the appendfile transport, will have UTF-8 names.
 
 require appropriate care.  Filenames created, eg. by
 the appendfile transport, will have UTF-8 names.
 

-Helo names sent by the smtp transport will have any UTF-8
+HELO names sent by the smtp transport will have any UTF-8

 components expanded to a-label form,
 and any certificate name checks will be done using the a-label
 form of the name.
 components expanded to a-label form,
 and any certificate name checks will be done using the a-label
 form of the name.