DNS: time-limit cached returns, using TTL. Bug 1395

[users/heiko/exim.git] / doc / doc-docbook / spec.xfpt

diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt
index b086aee3e6fd32750d54791ef89ba9180755d9f7..ed3533a0c8bfd05007cc3e753c774217c146f9e4 100644 (file)
--- a/doc/doc-docbook/spec.xfpt
+++ b/doc/doc-docbook/spec.xfpt
@@ -6060,8 +6060,8 @@ address_pipe:
 .endd
 This transport is used for handling deliveries to pipes that are generated by
 redirection (aliasing or users' &_.forward_& files). The &%return_output%&
 .endd
 This transport is used for handling deliveries to pipes that are generated by
 redirection (aliasing or users' &_.forward_& files). The &%return_output%&

-option specifies that any output generated by the pipe is to be returned to the
-sender.
+option specifies that any output on stdout or stderr generated by the pipe is to
+be returned to the sender.

 .code
 address_file:
   driver = appendfile
 .code
 address_file:
   driver = appendfile


@@ -6993,6 +6993,15 @@ Retries for the dnsdb lookup can be controlled by a retry modifier.
 The form if &"retry_VAL"& where VAL is an integer.
 The default count is set by the main configuration option &%dns_retry%&.
 
 The form if &"retry_VAL"& where VAL is an integer.
 The default count is set by the main configuration option &%dns_retry%&.
 

+.new
+.cindex cacheing "of dns lookup"
+.cindex TTL "of dns lookup"
+.cindex DNS TTL
+Dnsdb lookup results are cached within a single process (and its children).
+The cache entry lifetime is limited to the smallest time-to-live (TTL)
+value of the set of returned DNS records.
+.wen
+

 
 .section "Pseudo dnsdb record types" "SECID66"
 .cindex "MX record" "in &(dnsdb)& lookup"
 
 .section "Pseudo dnsdb record types" "SECID66"
 .cindex "MX record" "in &(dnsdb)& lookup"


@@ -9264,10 +9273,11 @@ by earlier ACLs are visible.
 Upper case and lower case letters are synonymous in header names. If the
 following character is white space, the terminating colon may be omitted, but
 this is not recommended, because you may then forget it when it is needed. When
 Upper case and lower case letters are synonymous in header names. If the
 following character is white space, the terminating colon may be omitted, but
 this is not recommended, because you may then forget it when it is needed. When

-white space terminates the header name, it is included in the expanded string.
-If the message does not contain the given header, the expansion item is
-replaced by an empty string. (See the &%def%& condition in section
-&<<SECTexpcond>>& for a means of testing for the existence of a header.)
+white space terminates the header name, this white space is included in the
+expanded string.  If the message does not contain the given header, the
+expansion item is replaced by an empty string. (See the &%def%& condition in
+section &<<SECTexpcond>>& for a means of testing for the existence of a
+header.)

 
 If there is more than one header with the same name, they are all concatenated
 to form the substitution string, up to a maximum length of 64K. Unless
 
 If there is more than one header with the same name, they are all concatenated
 to form the substitution string, up to a maximum length of 64K. Unless


@@ -9389,7 +9399,7 @@ yields &"42"&, and
 .code
 ${listextract{-3}{<, x,42,99,& Mailer,,/bin/bash}{result: $value}}
 .endd
 .code
 ${listextract{-3}{<, x,42,99,& Mailer,,/bin/bash}{result: $value}}
 .endd

-yields &"result: 99"&.
+yields &"result: 42"&.

 
 If {<&'string3'&>} is omitted, an empty string is used for string3.
 If {<&'string2'&>} is also omitted, the value that was
 
 If {<&'string3'&>} is omitted, an empty string is used for string3.
 If {<&'string2'&>} is also omitted, the value that was


@@ -11109,7 +11119,8 @@ support for TLS or the content scanning extension.
 When a &%match%& expansion condition succeeds, these variables contain the
 captured substrings identified by the regular expression during subsequent
 processing of the success string of the containing &%if%& expansion item.
 When a &%match%& expansion condition succeeds, these variables contain the
 captured substrings identified by the regular expression during subsequent
 processing of the success string of the containing &%if%& expansion item.

-However, they do not retain their values afterwards; in fact, their previous
+In the expansion condition case
+they do not retain their values afterwards; in fact, their previous

 values are restored at the end of processing an &%if%& item. The numerical
 variables may also be set externally by some other matching process which
 precedes the expansion of the string. For example, the commands available in
 values are restored at the end of processing an &%if%& item. The numerical
 variables may also be set externally by some other matching process which
 precedes the expansion of the string. For example, the commands available in


@@ -11317,9 +11328,10 @@ not the same as the user id of the originator of a message (see
 &$originator_uid$&). If Exim re-execs itself, this variable in the new
 incarnation normally contains the Exim uid.
 
 &$originator_uid$&). If Exim re-execs itself, this variable in the new
 incarnation normally contains the Exim uid.
 

-.vitem &$compile_date$&
-.vindex "&$compile_date$&"
-The date on which the Exim binary was compiled.
+.vitem &$callout_address$&
+.vindex "&$callout_address$&"
+After a callout for verification, spamd or malware daemon service, the
+address that was connected to.

 
 .vitem &$compile_number$&
 .vindex "&$compile_number$&"
 
 .vitem &$compile_number$&
 .vindex "&$compile_number$&"


@@ -12156,6 +12168,12 @@ increases for each accepted recipient. It can be referenced in an ACL.
 This variable is set to contain the matching regular expression after a
 &%regex%& ACL condition has matched (see section &<<SECTscanregex>>&).
 
 This variable is set to contain the matching regular expression after a
 &%regex%& ACL condition has matched (see section &<<SECTscanregex>>&).
 

+.vitem "&$regex1$&, &$regex2$&, etc"
+.cindex "regex submatch variables (&$1regex$& &$2regex$& etc)"
+When a &%regex%& or &%mime_regex%& ACL condition succeeds,
+these variables contain the
+captured substrings identified by the regular expression.
+

 
 .vitem &$reply_address$&
 .vindex "&$reply_address$&"
 
 .vitem &$reply_address$&
 .vindex "&$reply_address$&"


@@ -14459,7 +14477,7 @@ routing, but which are not used for listening by the daemon. See section
 . Allow this long option name to split; give it unsplit as a fifth argument
 . for the automatic .oindex that is generated by .option.
 
 . Allow this long option name to split; give it unsplit as a fifth argument
 . for the automatic .oindex that is generated by .option.
 

-.option "extract_addresses_remove_ &~&~arguments" main boolean true &&&
+.option "extract_addresses_remove_arguments" main boolean true &&&

          extract_addresses_remove_arguments
 .oindex "&%-t%&"
 .cindex "command line" "addresses with &%-t%&"
          extract_addresses_remove_arguments
 .oindex "&%-t%&"
 .cindex "command line" "addresses with &%-t%&"


@@ -16668,6 +16686,10 @@ must if set expand to the absolute path to a file which contains a current
 status proof for the server's certificate, as obtained from the
 Certificate Authority.
 
 status proof for the server's certificate, as obtained from the
 Certificate Authority.
 

+.new
+Usable for GnuTLS 3.4.4 or 3.3.17 or OpenSSL 1.1.0 (or later).
+.wen
+

 
 .option tls_on_connect_ports main "string list" unset
 .cindex SSMTP
 
 .option tls_on_connect_ports main "string list" unset
 .cindex SSMTP


@@ -22678,6 +22700,7 @@ Note that &$address_pipe$& is handled specially in &%command%& when
 &%force_command%& is set, expanding out to the original argument vector as
 separate items, similarly to a Unix shell &`"$@"`& construct.
 
 &%force_command%& is set, expanding out to the original argument vector as
 separate items, similarly to a Unix shell &`"$@"`& construct.
 

+

 .option ignore_status pipe boolean false
 If this option is true, the status returned by the subprocess that is set up to
 run the command is ignored, and Exim behaves as if zero had been returned.
 .option ignore_status pipe boolean false
 If this option is true, the status returned by the subprocess that is set up to
 run the command is ignored, and Exim behaves as if zero had been returned.


@@ -22688,27 +22711,29 @@ from the transport unless the status value is one of those listed in
 &*Note*&: This option does not apply to timeouts, which do not return a status.
 See the &%timeout_defer%& option for how timeouts are handled.
 
 &*Note*&: This option does not apply to timeouts, which do not return a status.
 See the &%timeout_defer%& option for how timeouts are handled.
 

+

 .option log_defer_output pipe boolean false
 .cindex "&(pipe)& transport" "logging output"
 If this option is set, and the status returned by the command is
 one of the codes listed in &%temp_errors%& (that is, delivery was deferred),
 .option log_defer_output pipe boolean false
 .cindex "&(pipe)& transport" "logging output"
 If this option is set, and the status returned by the command is
 one of the codes listed in &%temp_errors%& (that is, delivery was deferred),

-and any output was produced, the first line of it is written to the main log.
+and any output was produced on stdout or stderr, the first line of it is
+written to the main log.

 
 
 .option log_fail_output pipe boolean false
 
 
 .option log_fail_output pipe boolean false

-If this option is set, and the command returns any output, and also ends with a
-return code that is neither zero nor one of the return codes listed in
-&%temp_errors%& (that is, the delivery failed), the first line of output is
-written to the main log. This option and &%log_output%& are mutually exclusive.
-Only one of them may be set.
-
+If this option is set, and the command returns any output on stdout or
+stderr, and also ends with a return code that is neither zero nor one of
+the return codes listed in &%temp_errors%& (that is, the delivery
+failed), the first line of output is written to the main log. This
+option and &%log_output%& are mutually exclusive.  Only one of them may
+be set.

 
 
 .option log_output pipe boolean false
 
 
 .option log_output pipe boolean false

-If this option is set and the command returns any output, the first line of
-output is written to the main log, whatever the return code. This option and
-&%log_fail_output%& are mutually exclusive. Only one of them may be set.
-
+If this option is set and the command returns any output on stdout or
+stderr, the first line of output is written to the main log, whatever
+the return code. This option and &%log_fail_output%& are mutually
+exclusive. Only one of them may be set.

 
 
 .option max_output pipe integer 20K
 
 
 .option max_output pipe integer 20K


@@ -26751,7 +26776,9 @@ starts retrying to fetch an OCSP proof some time before its current
 proof expires.  The downside is that it requires server support.
 
 Unless Exim is built with the support disabled,
 proof expires.  The downside is that it requires server support.
 
 Unless Exim is built with the support disabled,

-or with GnuTLS earlier than version 3.1.3,
+.new
+or with GnuTLS earlier than version 3.3.16 / 3.4.8
+.wen

 support for OCSP stapling is included.
 
 There is a global option called &%tls_ocsp_file%&.
 support for OCSP stapling is included.
 
 There is a global option called &%tls_ocsp_file%&.


@@ -29027,7 +29054,8 @@ This condition is relevant only in an ACL that is run after a message has been
 received, that is, in an ACL specified by &%acl_smtp_data%& or
 &%acl_not_smtp%&. It checks the syntax of all header lines that can contain
 lists of addresses (&'Sender:'&, &'From:'&, &'Reply-To:'&, &'To:'&, &'Cc:'&,
 received, that is, in an ACL specified by &%acl_smtp_data%& or
 &%acl_not_smtp%&. It checks the syntax of all header lines that can contain
 lists of addresses (&'Sender:'&, &'From:'&, &'Reply-To:'&, &'To:'&, &'Cc:'&,

-and &'Bcc:'&). Unqualified addresses (local parts without domains) are
+and &'Bcc:'&), returning true if there are no problems.
+Unqualified addresses (local parts without domains) are

 permitted only in locally generated messages and from hosts that match
 &%sender_unqualified_hosts%& or &%recipient_unqualified_hosts%&, as
 appropriate.
 permitted only in locally generated messages and from hosts that match
 &%sender_unqualified_hosts%& or &%recipient_unqualified_hosts%&, as
 appropriate.


@@ -29189,9 +29217,15 @@ deny  dnslists = blackholes.mail-abuse.org
 warn  message  = X-Warn: sending host is on dialups list
       dnslists = dialups.mail-abuse.org
 .endd
 warn  message  = X-Warn: sending host is on dialups list
       dnslists = dialups.mail-abuse.org
 .endd

-DNS list lookups are cached by Exim for the duration of the SMTP session,
+.cindex cacheing "of dns lookup"
+.cindex DNS TTL
+DNS list lookups are cached by Exim for the duration of the SMTP session
+.new
+(but limited by the DNS return TTL value),
+.wen

 so a lookup based on the IP address is done at most once for any incoming
 so a lookup based on the IP address is done at most once for any incoming

-connection. Exim does not share information between multiple incoming
+connection (assuming long-enough TTL).
+Exim does not share information between multiple incoming

 connections (but your local name server cache should be active).
 
 
 connections (but your local name server cache should be active).
 
 


@@ -30944,6 +30978,10 @@ malware = * / defer_ok / tmo=10s
 .endd
 A timeout causes the ACL to defer.
 
 .endd
 A timeout causes the ACL to defer.
 

+.vindex "&$callout_address$&"
+When a connection is made to the scanner the expansion variable &$callout_address$&
+is set to record the actual address used.
+

 .vindex "&$malware_name$&"
 When a virus is found, the condition sets up an expansion variable called
 &$malware_name$& that contains the name of the virus. You can use it in a
 .vindex "&$malware_name$&"
 When a virus is found, the condition sets up an expansion variable called
 &$malware_name$& that contains the name of the virus. You can use it in a


@@ -31094,6 +31132,10 @@ a dollar sign. In this case, the expansion may return a string that is
 used as the list so that multiple spamd servers can be the result of an
 expansion.
 
 used as the list so that multiple spamd servers can be the result of an
 expansion.
 

+.vindex "&$callout_address$&"
+When a connection is made to the server the expansion variable &$callout_address$&
+is set to record the actual address used.
+

 .section "Calling SpamAssassin from an Exim ACL" "SECID206"
 Here is a simple example of the use of the &%spam%& condition in a DATA ACL:
 .code
 .section "Calling SpamAssassin from an Exim ACL" "SECID206"
 Here is a simple example of the use of the &%spam%& condition in a DATA ACL:
 .code


@@ -31162,7 +31204,8 @@ The integer value is useful for numeric comparisons in conditions.
 A string consisting of a number of &"+"& or &"-"& characters, representing the
 integer part of the spam score value. A spam score of 4.4 would have a
 &$spam_bar$& value of &"++++"&. This is useful for inclusion in warning
 A string consisting of a number of &"+"& or &"-"& characters, representing the
 integer part of the spam score value. A spam score of 4.4 would have a
 &$spam_bar$& value of &"++++"&. This is useful for inclusion in warning

-headers, since MUAs can match on such strings.
+headers, since MUAs can match on such strings. The maximum length of the
+spam bar is 50 characters.

 
 .vitem &$spam_report$&
 A multiline text table, containing the full SpamAssassin report for the
 
 .vitem &$spam_report$&
 A multiline text table, containing the full SpamAssassin report for the


@@ -31451,6 +31494,8 @@ deny message = contains blacklisted regex ($regex_match_string)
 The conditions returns true if any one of the regular expressions matches. The
 &$regex_match_string$& expansion variable is then set up and contains the
 matching regular expression.
 The conditions returns true if any one of the regular expressions matches. The
 &$regex_match_string$& expansion variable is then set up and contains the
 matching regular expression.

+The expansion variables &$regex1$& &$regex2$& etc
+are set to any substrings captured by the regular expression.

 
 &*Warning*&: With large messages, these conditions can be fairly
 CPU-intensive.
 
 &*Warning*&: With large messages, these conditions can be fairly
 CPU-intensive.


@@ -35303,6 +35348,9 @@ selection marked by asterisks:
 &` incoming_interface         `&  local interface on <= and => lines
 &` incoming_port              `&  remote port on <= lines
 &`*lost_incoming_connection   `&  as it says (includes timeouts)
 &` incoming_interface         `&  local interface on <= and => lines
 &` incoming_port              `&  remote port on <= lines
 &`*lost_incoming_connection   `&  as it says (includes timeouts)

+.new
+&` outgoing_interface         `&  local interface on => lines
+.wen

 &` outgoing_port              `&  add remote port to => lines
 &`*queue_run                  `&  start and end queue runs
 &` queue_time                 `&  time on queue for one recipient
 &` outgoing_port              `&  add remote port to => lines
 &`*queue_run                  `&  start and end queue runs
 &` queue_time                 `&  time on queue for one recipient


@@ -35428,9 +35476,11 @@ client's ident port times out.
 &%incoming_interface%&: The interface on which a message was received is added
 to the &"<="& line as an IP address in square brackets, tagged by I= and
 followed by a colon and the port number. The local interface and port are also
 &%incoming_interface%&: The interface on which a message was received is added
 to the &"<="& line as an IP address in square brackets, tagged by I= and
 followed by a colon and the port number. The local interface and port are also

-added to other SMTP log lines, for example &"SMTP connection from"& and to
-rejection lines
-and (despite the name) the local interface is added to &"=>"& lines..
+added to other SMTP log lines, for example &"SMTP connection from"&, to
+rejection lines, and (despite the name) to outgoing &"=>"& and &"->"& lines.
+.new
+The latter can be disabled by turning off the &%outgoing_interface%& option.
+.wen

 .next
 .cindex "log" "incoming remote port"
 .cindex "port" "logging remote"
 .next
 .cindex "log" "incoming remote port"
 .cindex "port" "logging remote"


@@ -35448,13 +35498,30 @@ important with the widening use of NAT (see RFC 2505).
 &%lost_incoming_connection%&: A log line is written when an incoming SMTP
 connection is unexpectedly dropped.
 .next
 &%lost_incoming_connection%&: A log line is written when an incoming SMTP
 connection is unexpectedly dropped.
 .next

+.cindex "log" "outgoing interface"
+.cindex "log" "local interface"
+.cindex "log" "local address and port"
+.cindex "TCP/IP" "logging local address and port"
+.cindex "interface" "logging"
+.new
+&%outgoing_interface%&: If &%incoming_interface%& is turned on, then the
+interface on which a message was sent is added to delivery lines as an I= tag
+followed by IP address in square brackets. You can disable this by turning
+off the &%outgoing_interface%& option.
+.wen
+.next

 .cindex "log" "outgoing remote port"
 .cindex "port" "logging outgoint remote"
 .cindex "TCP/IP" "logging ougtoing remote port"
 &%outgoing_port%&: The remote port number is added to delivery log lines (those
 .cindex "log" "outgoing remote port"
 .cindex "port" "logging outgoint remote"
 .cindex "TCP/IP" "logging ougtoing remote port"
 &%outgoing_port%&: The remote port number is added to delivery log lines (those

-containing => tags) following the IP address. This option is not included in
-the default setting, because for most ordinary configurations, the remote port
-number is always 25 (the SMTP port).
+containing => tags) following the IP address.
+.new
+The local port is also added if &%incoming_interface%& and
+&%outgoing_interface%& are both enabled.
+.wen
+This option is not included in the default setting, because for most ordinary
+configurations, the remote port number is always 25 (the SMTP port), and the
+local port is a random ephemeral port.

 .next
 .cindex "log" "process ids in"
 .cindex "pid (process id)" "in log lines"
 .next
 .cindex "log" "process ids in"
 .cindex "pid (process id)" "in log lines"