I18N: add a utf8_downconvert option to the smtp transport. Bug 2248

[users/heiko/exim.git] / doc / doc-docbook / spec.xfpt

diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt
index 7ae09cfbe8419f54964271f198fcdfee580b6679..fb5944d959f93261b061a3a8e32f70f88380e744 100644 (file)
--- a/doc/doc-docbook/spec.xfpt
+++ b/doc/doc-docbook/spec.xfpt
@@ -3966,8 +3966,17 @@ the messages are active, their status is not altered. This option can be used
 only by an admin user or by the user who originally caused the message to be
 placed on the queue.
 
 only by an admin user or by the user who originally caused the message to be
 placed on the queue.
 

+. .new
+. .vitem &%-MS%&
+. .oindex "&%-MS%&"
+. .cindex REQUIRETLS
+. This option is used to request REQUIRETLS processing on the message.
+. It is used internally by Exim in conjunction with -E when generating
+. a bounce message.
+. .wen
+

 .vitem &%-Mset%&&~<&'message&~id'&>
 .vitem &%-Mset%&&~<&'message&~id'&>

-.oindex "&%-Mset%&
+.oindex "&%-Mset%&"

 .cindex "testing" "string expansion"
 .cindex "expansion" "testing"
 This option is useful only in conjunction with &%-be%& (that is, when testing
 .cindex "testing" "string expansion"
 .cindex "expansion" "testing"
 This option is useful only in conjunction with &%-be%& (that is, when testing


@@ -9878,15 +9887,26 @@ extend what can be done. Firstly, you can vary the timeout. For example:
 .code
 ${readsocket{/socket/name}{request string}{3s}}
 .endd
 .code
 ${readsocket{/socket/name}{request string}{3s}}
 .endd

+

 The third argument is a list of options, of which the first element is the timeout
 and must be present if the argument is given.
 Further elements are options of form &'name=value'&.
 The third argument is a list of options, of which the first element is the timeout
 and must be present if the argument is given.
 Further elements are options of form &'name=value'&.

-One option type is currently recognised, defining whether (the default)
+Two option types is currently recognised: shutdown and tls.
+The first defines whether (the default)

 or not a shutdown is done on the connection after sending the request.
 Example, to not do so (preferred, eg. by some webservers):
 .code
 ${readsocket{/socket/name}{request string}{3s:shutdown=no}}
 .endd
 or not a shutdown is done on the connection after sending the request.
 Example, to not do so (preferred, eg. by some webservers):
 .code
 ${readsocket{/socket/name}{request string}{3s:shutdown=no}}
 .endd

+.new
+The second, tls, controls the use of TLS on the connection.  Example:
+.code
+${readsocket{/socket/name}{request string}{3s:tls=yes}}
+.endd
+The default is to not use TLS.
+If it is enabled, a shutdown as descripbed above is never done.
+.wen
+

 A fourth argument allows you to change any newlines that are in the data
 that is read, in the same way as for &%readfile%& (see above). This example
 turns them into spaces:
 A fourth argument allows you to change any newlines that are in the data
 that is read, in the same way as for &%readfile%& (see above). This example
 turns them into spaces:


@@ -10414,7 +10434,7 @@ abbreviation &%h%& can be used when &%hash%& is used as an operator.
 .cindex "expansion" "hex to base64"
 .cindex "&%hex2b64%& expansion item"
 This operator converts a hex string into one that is base64 encoded. This can
 .cindex "expansion" "hex to base64"
 .cindex "&%hex2b64%& expansion item"
 This operator converts a hex string into one that is base64 encoded. This can

-be useful for processing the output of the MD5 and SHA-1 hashing functions.
+be useful for processing the output of the various hashing functions.

 
 
 
 
 
 


@@ -24696,6 +24716,16 @@ The &%tls_verify_certificates%& option must also be set.
 If both this option and &%tls_try_verify_hosts%& are unset
 operation is as if this option selected all hosts.
 
 If both this option and &%tls_try_verify_hosts%& are unset
 operation is as if this option selected all hosts.
 

+.new
+.option utf8_downconvert smtp integer!! unset
+.cindex utf8 "address downconversion"
+.cindex i18n "utf8 address downconversion"
+If built with internationalization support,
+this option controls conversion of UTF-8 in message addresses
+to a-label form.
+For details see section &<<SECTi18nMTA>>&.
+.wen
+

 
 
 
 
 
 


@@ -26819,15 +26849,17 @@ without code changes in Exim.
 
 
 .option server_channelbinding gsasl boolean false
 
 
 .option server_channelbinding gsasl boolean false

+Do not set this true without consulting a cryptographic engineer.
+

 Some authentication mechanisms are able to use external context at both ends
 of the session to bind the authentication to that context, and fail the
 authentication process if that context differs.  Specifically, some TLS
 ciphersuites can provide identifying information about the cryptographic
 context.
 
 Some authentication mechanisms are able to use external context at both ends
 of the session to bind the authentication to that context, and fail the
 authentication process if that context differs.  Specifically, some TLS
 ciphersuites can provide identifying information about the cryptographic
 context.
 

-This means that certificate identity and verification becomes a non-issue,
-as a man-in-the-middle attack will cause the correct client and server to
-see different identifiers and authentication will fail.
+This should have meant that certificate identity and verification becomes a
+non-issue, as a man-in-the-middle attack will cause the correct client and
+server to see different identifiers and authentication will fail.

 
 This is currently only supported when using the GnuTLS library.  This is
 only usable by mechanisms which support "channel binding"; at time of
 
 This is currently only supported when using the GnuTLS library.  This is
 only usable by mechanisms which support "channel binding"; at time of


@@ -26835,7 +26867,11 @@ writing, that's the SCRAM family.
 
 This defaults off to ensure smooth upgrade across Exim releases, in case
 this option causes some clients to start failing.  Some future release
 
 This defaults off to ensure smooth upgrade across Exim releases, in case
 this option causes some clients to start failing.  Some future release

-of Exim may switch the default to be true.
+of Exim might have switched the default to be true.
+
+However, Channel Binding in TLS has proven to be broken in current versions.
+Do not plan to rely upon this feature for security, ever, without consulting
+with a subject matter expert (a cryptographic engineer).

 
 
 .option server_hostname gsasl string&!! "see below"
 
 
 .option server_hostname gsasl string&!! "see below"


@@ -27417,7 +27453,10 @@ the size of the generated prime, so it might still be too large.
 .oindex "&%tls_require_ciphers%&" "OpenSSL"
 There is a function in the OpenSSL library that can be passed a list of cipher
 suites before the cipher negotiation takes place. This specifies which ciphers
 .oindex "&%tls_require_ciphers%&" "OpenSSL"
 There is a function in the OpenSSL library that can be passed a list of cipher
 suites before the cipher negotiation takes place. This specifies which ciphers

-are acceptable. The list is colon separated and may contain names like
+.new
+are acceptable for TLS versions prior to 1.3.
+.wen
+The list is colon separated and may contain names like

 DES-CBC3-SHA. Exim passes the expanded value of &%tls_require_ciphers%&
 directly to this function call.
 Many systems will install the OpenSSL manual-pages, so you may have
 DES-CBC3-SHA. Exim passes the expanded value of &%tls_require_ciphers%&
 directly to this function call.
 Many systems will install the OpenSSL manual-pages, so you may have


@@ -27482,6 +27521,18 @@ This example will prefer ECDSA-authenticated ciphers over RSA ones:
 tls_require_ciphers = ECDSA:RSA:!COMPLEMENTOFDEFAULT
 .endd
 
 tls_require_ciphers = ECDSA:RSA:!COMPLEMENTOFDEFAULT
 .endd
 

+.new
+For TLS version 1.3 the control available is less fine-grained
+and Exim does not provide access to it at present.
+The value of the &%tls_require_ciphers%& option is ignored when
+TLS version 1.3 is negotiated.
+
+As of writing the library default cipher suite list for TLSv1.3 is
+.code
+TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
+.endd
+.wen
+

 
 .section "Requiring specific ciphers or other parameters in GnuTLS" &&&
          "SECTreqciphgnu"
 
 .section "Requiring specific ciphers or other parameters in GnuTLS" &&&
          "SECTreqciphgnu"


@@ -28132,20 +28183,48 @@ Support for client-side operation of DANE can be included at compile time by def
 in &_Local/Makefile_&.
 If it has been included, the macro "_HAVE_DANE" will be defined.
 
 in &_Local/Makefile_&.
 If it has been included, the macro "_HAVE_DANE" will be defined.
 

-The TLSA record for the server may have "certificate usage" of DANE-TA(2) or DANE-EE(3).  The latter specifies
-the End Entity directly, i.e. the certificate involved is that of the server (and should be the sole one transmitted
-during the TLS handshake); this is appropriate for a single system, using a self-signed certificate.
+The TLSA record for the server may have "certificate usage" of DANE-TA(2) or DANE-EE(3).
+These are the "Trust Anchor" and "End Entity" variants.
+The latter specifies the End Entity directly, i.e. the certificate involved is that of the server
+(and if only DANE-EE is used then it should be the sole one transmitted during the TLS handshake);
+this is appropriate for a single system, using a self-signed certificate.

 DANE-TA usage is effectively declaring a specific CA to be used; this might be a private CA or a public,
 DANE-TA usage is effectively declaring a specific CA to be used; this might be a private CA or a public,

-well-known one.  A private CA at simplest is just a self-signed certificate which is used to sign
-cerver certificates, but running one securely does require careful arrangement.  If a private CA is used
-then either all clients must be primed with it, or (probably simpler) the server TLS handshake must transmit
-the entire certificate chain from CA to server-certificate.  If a public CA is used then all clients must be primed with it
-(losing one advantage of DANE) - but the attack surface is reduced from all public CAs to that single CA.
+well-known one.
+A private CA at simplest is just a self-signed certificate (with certain
+attributes) which is used to sign cerver certificates, but running one securely
+does require careful arrangement.
+With DANE-TA, as implemented in Exim and commonly in other MTAs,
+the server TLS handshake must transmit the entire certificate chain from CA to server-certificate.

 DANE-TA is commonly used for several services and/or servers, each having a TLSA query-domain CNAME record,
 all of which point to a single TLSA record.
 DANE-TA is commonly used for several services and/or servers, each having a TLSA query-domain CNAME record,
 all of which point to a single TLSA record.

+DANE-TA and DANE-EE can both be used together.

 
 

-Another approach which should be seriously considered is to use DANE with a certificate
-from a public CA, because of another technology, "MTA-STS", described below.
+.new
+Our recommendation is to use DANE with a certificate from a public CA,
+because this enables a variety of strategies for remote clients to verify
+your certificate.
+You can then publish information both via DANE and another technology,
+"MTA-STS", described below.
+
+When you use DANE-TA to publish trust anchor information, you ask entities
+outside your administrative control to trust the Certificate Authority for
+connections to you.
+If using a private CA then you should expect others to still apply the
+technical criteria they'd use for a public CA to your certificates.
+In particular, you should probably try to follow current best practices for CA
+operation around hash algorithms and key sizes.
+Do not expect other organizations to lower their security expectations just
+because a particular profile might be reasonable for your own internal use.
+
+When this text was last updated, this in practice means to avoid use of SHA-1
+and MD5; if using RSA to use key sizes of at least 2048 bits (and no larger
+than 4096, for interoperability); to use keyUsage fields correctly; to use
+random serial numbers.
+The list of requirements is subject to change as best practices evolve.
+If you're not already using a private CA, or it doesn't meet these
+requirements, then we encourage you to avoid all these issues and use a public
+CA such as &url(https://letsencrypt.org/,Let's Encrypt) instead.
+.wen

 
 The TLSA record should have a Selector field of SPKI(1) and a Matching Type field of SHA2-512(2).
 
 
 The TLSA record should have a Selector field of SPKI(1) and a Matching Type field of SHA2-512(2).
 


@@ -28163,6 +28242,16 @@ are workable for 4th-field hashes.
 
 For use with the DANE-TA model, server certificates must have a correct name (SubjectName or SubjectAltName).
 
 
 For use with the DANE-TA model, server certificates must have a correct name (SubjectName or SubjectAltName).
 

+.new
+The Certificate issued by the CA published in the DANE-TA model should be
+issued using a strong hash algorithm.
+Exim, and importantly various other MTAs sending to you, will not
+re-enable hash algorithms which have been disabled by default in TLS
+libraries.
+This means no MD5 and no SHA-1.  SHA2-256 is the minimum for reliable
+interoperability (and probably the maximum too, in 2018).
+.wen
+

 The use of OCSP-stapling should be considered, allowing for fast revocation of certificates (which would otherwise
 be limited by the DNS TTL on the TLSA records).  However, this is likely to only be usable with DANE-TA.  NOTE: the
 default of requesting OCSP for all hosts is modified iff DANE is in use, to:
 The use of OCSP-stapling should be considered, allowing for fast revocation of certificates (which would otherwise
 be limited by the DNS TTL on the TLSA records).  However, this is likely to only be usable with DANE-TA.  NOTE: the
 default of requesting OCSP for all hosts is modified iff DANE is in use, to:


@@ -28245,8 +28334,8 @@ MTA-STS to let those clients who do use that protocol derive trust
 information.
 
 The MTA-STS design requires a certificate from a public Certificate Authority
 information.
 
 The MTA-STS design requires a certificate from a public Certificate Authority

-which is recognized by clients sending to you.  That selection is outside your
-control.
+which is recognized by clients sending to you.
+That selection of which CAs are trusted by others is outside your control.

 
 The most interoperable course of action is probably to use
 &url(https://letsencrypt.org/,Let's Encrypt), with automated certificate
 
 The most interoperable course of action is probably to use
 &url(https://letsencrypt.org/,Let's Encrypt), with automated certificate


@@ -29877,9 +29966,10 @@ warn   hosts           = +internal_hosts
 warn   message         = Remove internal headers
        remove_header   = $acl_c_ihdrs
 .endd
 warn   message         = Remove internal headers
        remove_header   = $acl_c_ihdrs
 .endd

-Removed header lines are accumulated during the MAIL, RCPT, and predata ACLs.
-They are removed from the message before processing the DATA and MIME ACLs.
-There is no harm in attempting to remove the same header twice nor is removing
+Header names for removal are accumulated during the MAIL, RCPT, and predata ACLs.
+Matching header lines are removed from the message before processing the DATA and MIME ACLs.
+If multiple header lines match, all are removed.
+There is no harm in attempting to remove the same header twice nor in removing

 a non-existent header. Further header lines to be removed may be accumulated
 during the DATA and MIME ACLs, after which they are removed from the message,
 if present. In the case of non-SMTP messages, headers to be removed are
 a non-existent header. Further header lines to be removed may be accumulated
 during the DATA and MIME ACLs, after which they are removed from the message,
 if present. In the case of non-SMTP messages, headers to be removed are


@@ -39215,8 +39305,10 @@ hash-method or key-size:
        set dkim_verify_reason = hash too weak or key too short
 .endd
 
        set dkim_verify_reason = hash too weak or key too short
 .endd
 

-After all the DKIM ACL runs have completed, the value becomes a
+So long as a DKIM ACL is defined (it need do no more than accept),
+after all the DKIM ACL runs have completed, the value becomes a

 colon-separated list of the values after each run.
 colon-separated list of the values after each run.

+This is maintained for the mime, prdr and data ACLs.

 
 .vitem &%$dkim_verify_reason%&
 A string giving a little bit more detail when &%$dkim_verify_status%& is either
 
 .vitem &%$dkim_verify_reason%&
 A string giving a little bit more detail when &%$dkim_verify_status%& is either


@@ -39781,6 +39873,12 @@ If a value is appended it may be:
 If mua_wrapper is set, the utf8_downconvert control
 is initially set to -1.
 
 If mua_wrapper is set, the utf8_downconvert control
 is initially set to -1.
 

+.new
+The smtp transport has an option &%utf8_downconvert%&.
+If set it must expand to one of the three values described above,
+and it overrides any previously set value.
+.wen
+

 
 There is no explicit support for VRFY and EXPN.
 Configurations supporting these should inspect
 
 There is no explicit support for VRFY and EXPN.
 Configurations supporting these should inspect