Docs: describe operation of continued-connection TLS

[users/heiko/exim.git] / doc / doc-docbook / spec.xfpt

diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt
index be93cf67075ebd93f4493b621ecd1c49f2a216a2..9737aed05a29fc9de557a6c3ffd3fc639acf8e38 100644 (file)
--- a/doc/doc-docbook/spec.xfpt
+++ b/doc/doc-docbook/spec.xfpt
@@ -464,7 +464,7 @@ which contains what used to be a separate FAQ, as well as various other
 examples, tips, and know-how that have been contributed by Exim users.
 
 .cindex Bugzilla
 examples, tips, and know-how that have been contributed by Exim users.
 
 .cindex Bugzilla

-An Exim Bugzilla exists at &url(http://bugs.exim.org). You can use
+An Exim Bugzilla exists at &url(https://bugs.exim.org). You can use

 this to report bugs, and also to add items to the wish list. Please search
 first to check that you are not duplicating a previous entry.
 
 this to report bugs, and also to add items to the wish list. Please search
 first to check that you are not duplicating a previous entry.
 


@@ -493,18 +493,11 @@ via this web page:
 Please ask Debian-specific questions on this list and not on the general Exim
 lists.
 
 Please ask Debian-specific questions on this list and not on the general Exim
 lists.
 

-.section "Exim training" "SECID4"
-.cindex "training courses"
-Training courses in Cambridge (UK) used to be run annually by the author of
-Exim, before he retired. At the time of writing, there are no plans to run
-further Exim courses in Cambridge. However, if that changes, relevant
-information will be posted at &url(http://www-tus.csx.cam.ac.uk/courses/exim/).
-

 .section "Bug reports" "SECID5"
 .cindex "bug reports"
 .cindex "reporting bugs"
 Reports of obvious bugs can be emailed to &'bugs@exim.org'& or reported
 .section "Bug reports" "SECID5"
 .cindex "bug reports"
 .cindex "reporting bugs"
 Reports of obvious bugs can be emailed to &'bugs@exim.org'& or reported

-via the Bugzilla (&url(http://bugs.exim.org)). However, if you are unsure
+via the Bugzilla (&url(https://bugs.exim.org)). However, if you are unsure

 whether some behaviour is a bug or not, the best thing to do is to post a
 message to the &'exim-dev'& mailing list and have it discussed.
 
 whether some behaviour is a bug or not, the best thing to do is to post a
 message to the &'exim-dev'& mailing list and have it discussed.
 


@@ -515,10 +508,6 @@ message to the &'exim-dev'& mailing list and have it discussed.
 .cindex "distribution" "ftp site"
 The master ftp site for the Exim distribution is
 .display
 .cindex "distribution" "ftp site"
 The master ftp site for the Exim distribution is
 .display

-&*ftp://ftp.csx.cam.ac.uk/pub/software/email/exim*&
-.endd
-This is mirrored by
-.display

 &*ftp://ftp.exim.org/pub/exim*&
 .endd
 The file references that follow are relative to the &_exim_& directories at
 &*ftp://ftp.exim.org/pub/exim*&
 .endd
 The file references that follow are relative to the &_exim_& directories at


@@ -3869,12 +3858,12 @@ by Exim in conjunction with the &%-MC%& option, and passes on the fact that the
 host to which Exim is connected supports TLS encryption.
 
 .new
 host to which Exim is connected supports TLS encryption.
 
 .new

-.vitem &%-MCt%&&~<&'IP&~address'&>&~<&'port'&>
+.vitem &%-MCt%&&~<&'IP&~address'&>&~<&'port'&>&~<&'cipher'&>

 .oindex "&%-MCt%&"
 This option is not intended for use by external callers. It is used internally
 by Exim in conjunction with the &%-MC%& option, and passes on the fact that the
 connection is being proxied by a parent process for handling TLS encryption.
 .oindex "&%-MCt%&"
 This option is not intended for use by external callers. It is used internally
 by Exim in conjunction with the &%-MC%& option, and passes on the fact that the
 connection is being proxied by a parent process for handling TLS encryption.

-The pair of arguments give the local address and port being proxied.
+The arguments give the local address and port being proxied, and the TLS cipher.

 .wen
 
 .vitem &%-Mc%&&~<&'message&~id'&>&~<&'message&~id'&>&~...
 .wen
 
 .vitem &%-Mc%&&~<&'message&~id'&>&~<&'message&~id'&>&~...


@@ -11042,9 +11031,14 @@ colon-separated components are permitted, each containing from one to four
 hexadecimal digits. There may be fewer than eight components if an empty
 component (adjacent colons) is present. Only one empty component is permitted.
 
 hexadecimal digits. There may be fewer than eight components if an empty
 component (adjacent colons) is present. Only one empty component is permitted.
 

-&*Note*&: The checks are just on the form of the address; actual numerical
-values are not considered. Thus, for example, 999.999.999.999 passes the IPv4
-check. The main use of these tests is to distinguish between IP addresses and
+.new
+&*Note*&: The checks used to be just on the form of the address; actual numerical
+values were not considered. Thus, for example, 999.999.999.999 passed the IPv4
+check.
+This is no longer the case.
+.wen
+
+The main use of these tests is to distinguish between IP addresses and

 host names, or between IPv4 and IPv6 addresses. For example, you could use
 .code
 ${if isip4{$sender_host_address}...
 host names, or between IPv4 and IPv6 addresses. For example, you could use
 .code
 ${if isip4{$sender_host_address}...


@@ -13572,6 +13566,7 @@ listed in more than one group.
 .row &%message_body_visible%&        "how much to show in &$message_body$&"
 .row &%mua_wrapper%&                 "run in &""MUA wrapper""& mode"
 .row &%print_topbitchars%&           "top-bit characters are printing"
 .row &%message_body_visible%&        "how much to show in &$message_body$&"
 .row &%mua_wrapper%&                 "run in &""MUA wrapper""& mode"
 .row &%print_topbitchars%&           "top-bit characters are printing"

+.row &%spool_wireformat%&            "use wire-format spool data files when possible"

 .row &%timezone%&                    "force time zone"
 .endtable
 
 .row &%timezone%&                    "force time zone"
 .endtable
 


@@ -15685,7 +15680,7 @@ harm. This option overrides the &%pipe_as_creator%& option of the &(pipe)&
 transport driver.
 
 
 transport driver.
 
 

-.option openssl_options main "string list" "+no_sslv2 +single_dh_use"
+.option openssl_options main "string list" "+no_sslv2 +single_dh_use +no_ticket"

 .cindex "OpenSSL "compatibility options"
 This option allows an administrator to adjust the SSL options applied
 by OpenSSL to connections.  It is given as a space-separated list of items,
 .cindex "OpenSSL "compatibility options"
 This option allows an administrator to adjust the SSL options applied
 by OpenSSL to connections.  It is given as a space-separated list of items,


@@ -16827,6 +16822,25 @@ as failures in the configuration file.
 By using this option to override the compiled-in path, it is possible to run
 tests of Exim without using the standard spool.
 
 By using this option to override the compiled-in path, it is possible to run
 tests of Exim without using the standard spool.
 

+.new
+.option spool_wireformat main boolean false
+.cindex "spool directory" "file formats"
+If this option is set, Exim may for some messages use an alternate format
+for data-files in the spool which matches the wire format.
+Doing this permits more efficient message reception and transmission.
+Currently it is only done for messages received using the EMSTP CHUNKING
+option.
+
+Users of the local_scan() API (see &<<CHAPlocalscan>>&),
+and any external programs which are passed a reference to a message data file
+(except via the &"regex"&, &"malware"& or &"spam"&) ACL conditions)
+will need to be aware of the potential different format.
+
+Using any of the ACL conditions noted will negate the reception benefit
+(as a Unix-mbox-format file is contructed for them).
+The transimssion benefit is maintained.
+.wen
+

 .option sqlite_lock_timeout main time 5s
 .cindex "sqlite lookup type" "lock timeout"
 This option controls the timeout that the &(sqlite)& lookup uses when trying to
 .option sqlite_lock_timeout main time 5s
 .cindex "sqlite lookup type" "lock timeout"
 This option controls the timeout that the &(sqlite)& lookup uses when trying to


@@ -23695,7 +23709,7 @@ the message. As a result, the overall timeout for a message depends on the size
 of the message. Its value must not be zero. See also &%final_timeout%&.
 
 
 of the message. Its value must not be zero. See also &%final_timeout%&.
 
 

-.option dkim_domain smtp string&!! unset
+.option dkim_domain smtp string list&!! unset

 .option dkim_selector smtp string&!! unset
 .option dkim_private_key smtp string&!! unset
 .option dkim_canon smtp string&!! unset
 .option dkim_selector smtp string&!! unset
 .option dkim_private_key smtp string&!! unset
 .option dkim_canon smtp string&!! unset


@@ -27637,13 +27651,22 @@ built, then you have SNI support).
          "SECTmulmessam"
 .cindex "multiple SMTP deliveries with TLS"
 .cindex "TLS" "multiple message deliveries"
          "SECTmulmessam"
 .cindex "multiple SMTP deliveries with TLS"
 .cindex "TLS" "multiple message deliveries"

+.new

 Exim sends multiple messages down the same TCP/IP connection by starting up
 an entirely new delivery process for each message, passing the socket from
 one process to the next. This implementation does not fit well with the use
 of TLS, because there is quite a lot of state information associated with a TLS
 connection, not just a socket identification. Passing all the state information
 Exim sends multiple messages down the same TCP/IP connection by starting up
 an entirely new delivery process for each message, passing the socket from
 one process to the next. This implementation does not fit well with the use
 of TLS, because there is quite a lot of state information associated with a TLS
 connection, not just a socket identification. Passing all the state information

-to a new process is not feasible. Consequently, Exim shuts down an existing TLS
-session before passing the socket to a new process. The new process may then
+to a new process is not feasible. Consequently, for sending using TLS Exim
+starts an additional proxy process for handling the encryption, piping the
+unencrypted data stream from and to the delivery processes.
+
+An older mode of operation can be enabled on a per-host basis by the
+&%hosts_noproxy_tls%& option on the &(smtp)& transport.  If the host matches
+this list the proxy process descibed above is not used; instead Exim
+.wen
+shuts down an existing TLS session being run by the delivery process
+before passing the socket to a new process. The new process may then

 try to start a new TLS session, and if successful, may try to re-authenticate
 if AUTH is in use, before sending the next message.
 
 try to start a new TLS session, and if successful, may try to re-authenticate
 if AUTH is in use, before sending the next message.
 


@@ -29008,12 +29031,14 @@ sender when the destination system is doing content-scan based rejection.
 .cindex "&ACL;" "enabling debug logging"
 .cindex "debugging" "enabling from an ACL"
 This control turns on debug logging, almost as though Exim had been invoked
 .cindex "&ACL;" "enabling debug logging"
 .cindex "debugging" "enabling from an ACL"
 This control turns on debug logging, almost as though Exim had been invoked

-with &`-d`&, with the output going to a new logfile, by default called
-&'debuglog'&.  The filename can be adjusted with the &'tag'& option, which
+with &`-d`&, with the output going to a new logfile in the usual logs directory,
+by default called &'debuglog'&.
+The filename can be adjusted with the &'tag'& option, which

 may access any variables already defined.  The logging may be adjusted with
 the &'opts'& option, which takes the same values as the &`-d`& command-line
 option.
 may access any variables already defined.  The logging may be adjusted with
 the &'opts'& option, which takes the same values as the &`-d`& command-line
 option.

-Logging may be stopped, and the file removed, with the &'kill'& option.
+Logging started this way may be stopped, and the file removed,
+with the &'kill'& option.

 Some examples (which depend on variables that don't exist in all
 contexts):
 .code
 Some examples (which depend on variables that don't exist in all
 contexts):
 .code


@@ -30903,6 +30928,23 @@ command when performing the callout, instead of an empty address. There is no
 need to use this option unless you know that the called hosts make use of the
 sender when checking recipients. If used indiscriminately, it reduces the
 usefulness of callout caching.
 need to use this option unless you know that the called hosts make use of the
 sender when checking recipients. If used indiscriminately, it reduces the
 usefulness of callout caching.

+
+.new
+.vitem &*hold*&
+This option applies to recipient callouts only. For example:
+.code
+require  verify = recipient/callout=use_sender,hold
+.endd
+It causes the connection to be helod open and used for any further recipients
+and for eventual delivery (should that be done quickly).
+Doing this saves on TCP and SMTP startup costs, and TLS costs also
+when that is used for the connections.
+The advantage is only gained if there are no callout cache hits
+(which could be enforced by the no_cache option),
+if the use_sender option is used,
+if neither the random nor the use_postmaster option is used,
+and if no other callouts intervene.
+.wen

 .endlist
 
 If you use any of the parameters that set a non-empty sender for the MAIL
 .endlist
 
 If you use any of the parameters that set a non-empty sender for the MAIL


@@ -35769,9 +35811,9 @@ down a single SMTP connection, an asterisk follows the IP address in the log
 lines for the second and subsequent messages.
 .new
 When two or more messages are delivered down a single TLS connection, the
 lines for the second and subsequent messages.
 .new
 When two or more messages are delivered down a single TLS connection, the

-TLS-related information logged for the first message delivered
-(which may not be the earliest line in the log)
+DNS and some TLS-related information logged for the first message delivered

 will not be present in the log lines for the second and subsequent messages.
 will not be present in the log lines for the second and subsequent messages.

+TLS cipher information is still available.

 .wen
 
 .cindex "delivery" "cutthrough; logging"
 .wen
 
 .cindex "delivery" "cutthrough; logging"


@@ -35885,6 +35927,7 @@ the following table:
 &`I   `&        local interface used
 &`K   `&        CHUNKING extension used
 &`id  `&        message id for incoming message
 &`I   `&        local interface used
 &`K   `&        CHUNKING extension used
 &`id  `&        message id for incoming message

+&`M8S `&        8BITMIME status for incoming message

 &`P   `&        on &`<=`& lines: protocol used
 &`    `&        on &`=>`& and &`**`& lines: return path
 &`PRDR`&        PRDR extension used
 &`P   `&        on &`<=`& lines: protocol used
 &`    `&        on &`=>`& and &`**`& lines: return path
 &`PRDR`&        PRDR extension used


@@ -36633,8 +36676,7 @@ autodetection of some well known compression extensions.
 .cindex "&'exipick'&"
 John Jetmore's &'exipick'& utility is included in the Exim distribution. It
 lists messages from the queue according to a variety of criteria. For details
 .cindex "&'exipick'&"
 John Jetmore's &'exipick'& utility is included in the Exim distribution. It
 lists messages from the queue according to a variety of criteria. For details

-of &'exipick'&'s facilities, visit the web page at
-&url(http://www.exim.org/eximwiki/ToolExipickManPage) or run &'exipick'& with
+of &'exipick'&'s facilities, run &'exipick'& with

 the &%--help%& option.
 
 
 the &%--help%& option.
 
 


@@ -38081,8 +38123,8 @@ The address of an authenticated sender &-- the value of the
 &$authenticated_sender$& variable.
 
 .vitem "&%-body_linecount%&&~<&'number'&>"
 &$authenticated_sender$& variable.
 
 .vitem "&%-body_linecount%&&~<&'number'&>"

-This records the number of lines in the body of the message, and is always
-present.
+This records the number of lines in the body of the message, and is
+present unless &%-spool_file_wireformat%& is.

 
 .vitem "&%-body_zerocount%&&~<&'number'&>"
 This records the number of binary zero bytes in the body of the message, and is
 
 .vitem "&%-body_zerocount%&&~<&'number'&>"
 This records the number of binary zero bytes in the body of the message, and is


@@ -38164,6 +38206,12 @@ to ensure that the caller is displayed in queue listings).
 If a message was scanned by SpamAssassin, this is present. It records the value
 of &$spam_score_int$&.
 
 If a message was scanned by SpamAssassin, this is present. It records the value
 of &$spam_score_int$&.
 

+.vitem &%-spool_file_wireformat%&
+The -D file for this message is in wire-format (for ESMTP CHUNKING)
+rather than Unix-format.
+The line-ending is CRLF rather than newline.
+There is still, however, no leading-dot-stuffing.
+

 .vitem &%-tls_certificate_verified%&
 A TLS certificate was received from the client that sent this message, and the
 certificate was verified by the server.
 .vitem &%-tls_certificate_verified%&
 A TLS certificate was received from the client that sent this message, and the
 certificate was verified by the server.


@@ -38325,22 +38373,26 @@ senders).
 Signing is enabled by setting private options on the SMTP transport.
 These options take (expandable) strings as arguments.
 
 Signing is enabled by setting private options on the SMTP transport.
 These options take (expandable) strings as arguments.
 

-.option dkim_domain smtp string&!! unset
-MANDATORY:
-The domain you want to sign with. The result of this expanded
-option is put into the &%$dkim_domain%& expansion variable.
+.option dkim_domain smtp string list&!! unset
+The domain(s) you want to sign with.
+.new
+After expansion, this can be a list.
+Each element in turn is put into the &%$dkim_domain%& expansion variable
+while expanding the remaining signing options.
+.wen

 If it is empty after expansion, DKIM signing is not done.
 
 .option dkim_selector smtp string&!! unset
 If it is empty after expansion, DKIM signing is not done.
 
 .option dkim_selector smtp string&!! unset

-MANDATORY:
-This sets the key selector string. You can use the &%$dkim_domain%& expansion
-variable to look up a matching selector. The result is put in the expansion
+This sets the key selector string.
+You can use the &%$dkim_domain%& expansion variable to look up a matching selector.
+The result is put in the expansion

 variable &%$dkim_selector%& which may be used in the &%dkim_private_key%&
 option along with &%$dkim_domain%&.
 variable &%$dkim_selector%& which may be used in the &%dkim_private_key%&
 option along with &%$dkim_domain%&.

+If the option is empty after expansion, DKIM signing is not done.

 
 .option dkim_private_key smtp string&!! unset
 
 .option dkim_private_key smtp string&!! unset

-MANDATORY:
-This sets the private key to use. You can use the &%$dkim_domain%& and
+This sets the private key to use.
+You can use the &%$dkim_domain%& and

 &%$dkim_selector%& expansion variables to determine the private key to use.
 The result can either
 .ilist
 &%$dkim_selector%& expansion variables to determine the private key to use.
 The result can either
 .ilist


@@ -38353,16 +38405,15 @@ be "0", "false" or the empty string, in which case the message will not
 be signed. This case will not result in an error, even if &%dkim_strict%&
 is set.
 .endlist
 be signed. This case will not result in an error, even if &%dkim_strict%&
 is set.
 .endlist

+If the option is empty after expansion, DKIM signing is not done.

 
 .option dkim_canon smtp string&!! unset
 
 .option dkim_canon smtp string&!! unset

-OPTIONAL:

 This option sets the canonicalization method used when signing a message.
 The DKIM RFC currently supports two methods: "simple" and "relaxed".
 The option defaults to "relaxed" when unset. Note: the current implementation
 only supports using the same canonicalization method for both headers and body.
 
 .option dkim_strict smtp string&!! unset
 This option sets the canonicalization method used when signing a message.
 The DKIM RFC currently supports two methods: "simple" and "relaxed".
 The option defaults to "relaxed" when unset. Note: the current implementation
 only supports using the same canonicalization method for both headers and body.
 
 .option dkim_strict smtp string&!! unset

-OPTIONAL:

 This  option  defines  how  Exim  behaves  when  signing a message that
 should be signed fails for some reason.  When the expansion evaluates to
 either "1" or "true", Exim will defer. Otherwise Exim will send the message
 This  option  defines  how  Exim  behaves  when  signing a message that
 should be signed fails for some reason.  When the expansion evaluates to
 either "1" or "true", Exim will defer. Otherwise Exim will send the message


@@ -38370,11 +38421,10 @@ unsigned. You can use the &%$dkim_domain%& and &%$dkim_selector%& expansion
 variables here.
 
 .option dkim_sign_headers smtp string&!! unset
 variables here.
 
 .option dkim_sign_headers smtp string&!! unset

-OPTIONAL:
-When set, this option must expand to (or be specified as) a colon-separated
+If set, this option must expand to (or be specified as) a colon-separated

 list of header names. Headers with these names will be included in the message
 list of header names. Headers with these names will be included in the message

-signature. When unspecified, the header names recommended in RFC4871 will be
-used.
+signature.
+When unspecified, the header names recommended in RFC4871 will be used.

 
 
 .section "Verifying DKIM signatures in incoming mail" "SECID514"
 
 
 .section "Verifying DKIM signatures in incoming mail" "SECID514"


@@ -38897,7 +38947,7 @@ The current list of events is:
 &`msg:rcpt:host:defer    after    transport  `& per recipient per host
 &`msg:rcpt:defer         after    transport  `& per recipient
 &`msg:host:defer         after    transport  `& per attempt
 &`msg:rcpt:host:defer    after    transport  `& per recipient per host
 &`msg:rcpt:defer         after    transport  `& per recipient
 &`msg:host:defer         after    transport  `& per attempt

-&`msg:fail:delivery      after    main       `& per recipient
+&`msg:fail:delivery      after    transport  `& per recipient

 &`msg:fail:internal      after    main       `& per recipient
 &`tcp:connect            before   transport  `& per connection
 &`tcp:close              after    transport  `& per connection
 &`msg:fail:internal      after    main       `& per recipient
 &`tcp:connect            before   transport  `& per connection
 &`tcp:close              after    transport  `& per connection


@@ -38914,6 +38964,11 @@ The second column in the table above describes whether the event fires
 before or after the action is associates with.  Those which fire before
 can be used to affect that action (more on this below).
 
 before or after the action is associates with.  Those which fire before
 can be used to affect that action (more on this below).
 

+.new
+The third column in the table above says what section of the configumration
+should define the event action.
+.wen
+

 An additional variable, &$event_data$&, is filled with information varying
 with the event type:
 .display
 An additional variable, &$event_data$&, is filled with information varying
 with the event type:
 .display