Docs: tidying

[exim.git] / doc / doc-docbook / spec.xfpt

diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt
index 8674cfa430cd97a4f42bfadc9279a05e04419d48..1b0dd8832900bf53272ea40691b1a3087b7ddfd0 100644 (file)
--- a/doc/doc-docbook/spec.xfpt
+++ b/doc/doc-docbook/spec.xfpt
@@ -2837,6 +2837,12 @@ displayed.
 These options are used by Sendmail for selecting configuration files and are
 ignored by Exim.
 
 These options are used by Sendmail for selecting configuration files and are
 ignored by Exim.
 

+.new
+.cmdopt -atrn <&'host'&> <&'domainlist'&>
+This option requests an ODMR customer connection.
+See &<<SECTODMRCUST>>& for details.
+.wen
+

 .cmdopt -B <&'type'&>
 .oindex "&%-B%&"
 .cindex "8-bit characters"
 .cmdopt -B <&'type'&>
 .oindex "&%-B%&"
 .cindex "8-bit characters"


@@ -5210,7 +5216,7 @@ This can then be used in a &(redirect)& router setting like this:
 data = ${lookup mysql{ALIAS_QUERY}}
 .endd
 In earlier versions of Exim macros were sometimes used for domain, host, or
 data = ${lookup mysql{ALIAS_QUERY}}
 .endd
 In earlier versions of Exim macros were sometimes used for domain, host, or

-address lists. In Exim 4 these are handled better by named lists &-- see
+address lists. In Exim version 4 these are handled better by named lists &-- see

 section &<<SECTnamedlists>>&.
 
 
 section &<<SECTnamedlists>>&.
 
 


@@ -8441,10 +8447,18 @@ type of match and is given below as the &*value*& information.
 
 .section "Expansion of lists" "SECTlistexpand"
 .cindex "expansion" "of lists"
 
 .section "Expansion of lists" "SECTlistexpand"
 .cindex "expansion" "of lists"

-Each list is expanded as a single string before it is used.
+.new
+Each list, after any leading change-of-separator specification
+(see &<<SECTlistsepchange>>&) is expanded as a single string,

 .cindex "tainted data" tracking
 .cindex "tainted data" tracking

-&*Note*&: As a result, if any componend was tainted then the
-entire result string becomes tainted.
+&*Note*&: As a result, if any component was tainted then the
+entire expansion result string becomes tainted.
+
+Splitting out a leading explicit change-of-separator permits
+one being safely used on a list that has tainted components
+while still detecting the use of a tainted setting.
+The latter is not permitted.
+.wen

 
 &'Exception: the router headers_remove option, where list-item
 splitting is done before string-expansion.'&
 
 &'Exception: the router headers_remove option, where list-item
 splitting is done before string-expansion.'&


@@ -8627,6 +8641,11 @@ domainlist  dom2 = !a.b : *.b
 where &'x.y'& does not match. It's best to avoid negation altogether in
 referenced lists if you can.
 
 where &'x.y'& does not match. It's best to avoid negation altogether in
 referenced lists if you can.
 

+.new
+The list item which references a named list (&"+<listname>"&)
+may not be tainted.
+.wen
+

 .cindex "hiding named list values"
 .cindex "named lists" "hiding value of"
 Some named list definitions may contain sensitive data, for example, passwords for
 .cindex "hiding named list values"
 .cindex "named lists" "hiding value of"
 Some named list definitions may contain sensitive data, for example, passwords for


@@ -8731,6 +8750,9 @@ possible to use the same configuration file on several different hosts that
 differ only in their names.
 
 The value for a match will be the primary host name.
 differ only in their names.
 
 The value for a match will be the primary host name.

+.new
+The pattern may not be tainted.
+.wen

 
 
 .next
 
 
 .next


@@ -8746,6 +8768,9 @@ In today's Internet, the use of domain literals is controversial;
 see the &%allow_domain_literals%& main option.
 
 The value for a match will be the string &`@[]`&.
 see the &%allow_domain_literals%& main option.
 
 The value for a match will be the string &`@[]`&.

+.new
+The pattern may not be tainted.
+.wen

 
 
 .next
 
 
 .next


@@ -8762,6 +8787,10 @@ local host, and the second only when no primary MX target is the local host,
 but a secondary MX target is. &"Primary"& means an MX record with the lowest
 preference value &-- there may of course be more than one of them.
 
 but a secondary MX target is. &"Primary"& means an MX record with the lowest
 preference value &-- there may of course be more than one of them.
 

+.new
+The pattern may not be tainted.
+.wen
+

 The MX lookup that takes place when matching a pattern of this type is
 performed with the resolver options for widening names turned off. Thus, for
 example, a single-component domain will &'not'& be expanded by adding the
 The MX lookup that takes place when matching a pattern of this type is
 performed with the resolver options for widening names turned off. Thus, for
 example, a single-component domain will &'not'& be expanded by adding the


@@ -9230,6 +9259,8 @@ does not match the list. This may not always be what you want to happen.
 To change Exim's behaviour, the special items &`+include_unknown`& or
 &`+ignore_unknown`& may appear in the list (at top level &-- they are
 not recognized in an indirected file).
 To change Exim's behaviour, the special items &`+include_unknown`& or
 &`+ignore_unknown`& may appear in the list (at top level &-- they are
 not recognized in an indirected file).

+The effects of these special items do not propagate into referenced
+named lists.

 
 .ilist
 If any item that follows &`+include_unknown`& requires information that
 
 .ilist
 If any item that follows &`+include_unknown`& requires information that


@@ -9596,6 +9627,13 @@ longer case-independent. This does not affect the domain, which remains in
 lower case. However, although independent matches on the domain alone are still
 performed caselessly, regular expressions that match against an entire address
 become case-sensitive after &"+caseful"& has been seen.
 lower case. However, although independent matches on the domain alone are still
 performed caselessly, regular expressions that match against an entire address
 become case-sensitive after &"+caseful"& has been seen.

+The effects of &"+caseful"& propagate into any referenced named lists.
+
+.new
+This string may not be tainted.
+To do caseful matching on list elements whic are tainted,
+place them in a named list.
+.wen

 
 
 
 
 
 


@@ -9614,6 +9652,12 @@ matching in the local part list, but not elsewhere in the router. If
 &%caseful_local_part%& is set true in a router, matching in the &%local_parts%&
 option is case-sensitive from the start.
 
 &%caseful_local_part%& is set true in a router, matching in the &%local_parts%&
 option is case-sensitive from the start.
 

+.new
+This string may not be tainted.
+To do caseful matching on list elements whic are tainted,
+place them in a named list.
+.wen
+

 If a local part list is indirected to a file (see section &<<SECTfilnamlis>>&),
 comments are handled in the same way as address lists &-- they are recognized
 only if the # is preceded by white space or the start of the line.
 If a local part list is indirected to a file (see section &<<SECTfilnamlis>>&),
 comments are handled in the same way as address lists &-- they are recognized
 only if the # is preceded by white space or the start of the line.


@@ -10096,11 +10140,15 @@ leading and trailing quotes are removed from the returned value.
 .cindex "list" "selecting by condition"
 .cindex "expansion" "selecting from list by condition"
 .vindex "&$item$&"
 .cindex "list" "selecting by condition"
 .cindex "expansion" "selecting from list by condition"
 .vindex "&$item$&"

-After expansion, <&'string'&> is interpreted as a list, colon-separated by
-default, but the separator can be changed in the usual way (&<<SECTlistsepchange>>&).
-For each item
-in this list, its value is placed in &$item$&, and then the condition is
-evaluated.
+.new
+<&'string1'&> first has the part after any change-of-list-separator
+(see &<<SECTlistsepchange>>&) expanded,
+then the whole is taken as a list.
+.wen
+The default separator for the list is a colon.
+
+For each item in this list,
+its value is placed in &$item$&, and then the condition is evaluated.

 Any modification of &$value$& by this evaluation is discarded.
 If the condition is true, &$item$& is added to the output as an
 item in a new list; if the condition is false, the item is discarded. The
 Any modification of &$value$& by this evaluation is discarded.
 If the condition is true, &$item$& is added to the output as an
 item in a new list; if the condition is false, the item is discarded. The


@@ -10364,8 +10412,12 @@ The <&'number'&> argument must consist entirely of decimal digits,
 apart from an optional leading minus,
 and leading and trailing white space (which is ignored).
 
 apart from an optional leading minus,
 and leading and trailing white space (which is ignored).
 

-After expansion, <&'string1'&> is interpreted as a list, colon-separated by
-default, but the separator can be changed in the usual way (&<<SECTlistsepchange>>&).
+.new
+The <&'string1'&> argument, after any leading change-of-separator
+(see &<<SECTlistsepchange>>&),
+is expanded and the whole forms the list.
+.wen
+By default, the list separator is a colon.

 
 The first field of the list is numbered one.
 If the number is negative, the fields are
 
 The first field of the list is numbered one.
 If the number is negative, the fields are


@@ -10465,10 +10517,15 @@ ${lookup nisplus {[name=$local_part],passwd.org_dir:gcos} \
 .vitem &*${map{*&<&'string1'&>&*}{*&<&'string2'&>&*}}*&
 .cindex "expansion" "list creation"
 .vindex "&$item$&"
 .vitem &*${map{*&<&'string1'&>&*}{*&<&'string2'&>&*}}*&
 .cindex "expansion" "list creation"
 .vindex "&$item$&"

-After expansion, <&'string1'&> is interpreted as a list, colon-separated by
-default, but the separator can be changed in the usual way (&<<SECTlistsepchange>>&).
-For each item
-in this list, its value is place in &$item$&, and then <&'string2'&> is
+.new
+<&'string1'&> first has the part after any change-of-list-separator
+(see &<<SECTlistsepchange>>&) expanded,
+then the whole is taken as a list.
+.wen
+The default separator for the list is a colon.
+
+For each item in this list,
+its value is place in &$item$&, and then <&'string2'&> is

 expanded and added to the output as an item in a new list. The separator used
 for the output list is the same as the one used for the input, but a separator
 setting is not included in the output. For example:
 expanded and added to the output as an item in a new list. The separator used
 for the output list is the same as the one used for the input, but a separator
 setting is not included in the output. For example:


@@ -10688,9 +10745,15 @@ locks out the use of this expansion item in filter files.
 .cindex "list" "reducing to a scalar"
 .vindex "&$value$&"
 .vindex "&$item$&"
 .cindex "list" "reducing to a scalar"
 .vindex "&$value$&"
 .vindex "&$item$&"

-This operation reduces a list to a single, scalar string. After expansion,
-<&'string1'&> is interpreted as a list, colon-separated by default, but the
-separator can be changed in the usual way (&<<SECTlistsepchange>>&).
+This operation reduces a list to a single, scalar string.
+
+.new
+<&'string1'&> first has the part after any change-of-list-separator
+(see &<<SECTlistsepchange>>&) expanded,
+then the whole is taken as a list.
+.wen
+The default separator for the list is a colon.
+

 Then <&'string2'&> is expanded and
 assigned to the &$value$& variable. After this, each item in the <&'string1'&>
 list is assigned to &$item$&, in turn, and <&'string3'&> is expanded for each of
 Then <&'string2'&> is expanded and
 assigned to the &$value$& variable. After this, each item in the <&'string1'&>
 list is assigned to &$item$&, in turn, and <&'string3'&> is expanded for each of


@@ -10847,8 +10910,13 @@ rather than any Unicode-aware character handling.
 .cindex sorting "a list"
 .cindex list sorting
 .cindex expansion "list sorting"
 .cindex sorting "a list"
 .cindex list sorting
 .cindex expansion "list sorting"

-After expansion, <&'string'&> is interpreted as a list, colon-separated by
-default, but the separator can be changed in the usual way (&<<SECTlistsepchange>>&).
+.new
+<&'string'&> first has the part after any change-of-list-separator
+(see &<<SECTlistsepchange>>&) expanded,
+then the whole is taken as a list.
+.wen
+The default separator for the list is a colon.
+

 The <&'comparator'&> argument is interpreted as the operator
 of a two-argument expansion condition.
 The numeric operators plus ge, gt, le, lt (and ~i variants) are supported.
 The <&'comparator'&> argument is interpreted as the operator
 of a two-argument expansion condition.
 The numeric operators plus ge, gt, le, lt (and ~i variants) are supported.


@@ -11304,7 +11372,8 @@ All measurement is done in bytes and is not UTF-8 aware.
 .cindex "list" "item count"
 .cindex "list" "count of items"
 .cindex "&%listcount%& expansion item"
 .cindex "list" "item count"
 .cindex "list" "count of items"
 .cindex "&%listcount%& expansion item"

-The string is interpreted as a list and the number of items is returned.
+The part of the string after any leading change-of-separator is expanded,
+then the whole is interpreted as a list and the number of items is returned.

 
 
 .vitem &*${listnamed:*&<&'name'&>&*}*&&~and&~&*${listnamed_*&<&'type'&>&*:*&<&'name'&>&*}*&
 
 
 .vitem &*${listnamed:*&<&'name'&>&*}*&&~and&~&*${listnamed_*&<&'type'&>&*:*&<&'name'&>&*}*&


@@ -11925,9 +11994,14 @@ attempt. It is false during any subsequent delivery attempts.
 .cindex "expansion" "&*forall*& condition"
 .cindex "expansion" "&*forany*& condition"
 .vindex "&$item$&"
 .cindex "expansion" "&*forall*& condition"
 .cindex "expansion" "&*forany*& condition"
 .vindex "&$item$&"

-These conditions iterate over a list. The first argument is expanded to form
-the list. By default, the list separator is a colon, but it can be changed by
-the normal method (&<<SECTlistsepchange>>&).
+These conditions iterate over a list.
+.new
+The first argument, after any leading change-of-separator
+(see &<<SECTlistsepchange>>&),
+is expanded and the whole forms the list.
+.wen
+By default, the list separator is a colon.
+

 The second argument is interpreted as a condition that is to
 be applied to each item in the list in turn. During the interpretation of the
 condition, the current list item is placed in a variable called &$item$&.
 The second argument is interpreted as a condition that is to
 be applied to each item in the list in turn. During the interpretation of the
 condition, the current list item is placed in a variable called &$item$&.


@@ -12004,9 +12078,14 @@ SRS decode.  See SECT &<<SECTSRS>>& for details.
        &*inlisti&~{*&<&'subject'&>&*}{*&<&'list'&>&*}*&
 .cindex "string" "comparison"
 .cindex "list" "iterative conditions"
        &*inlisti&~{*&<&'subject'&>&*}{*&<&'list'&>&*}*&
 .cindex "string" "comparison"
 .cindex "list" "iterative conditions"

-Both strings are expanded; the second string is treated as a list of simple
-strings; if the first string is a member of the second, then the condition
-is true.
+The <&'subject'&> string is expanded.
+.new
+The <&'list'&> first has any change-of-list-separator
++(see &<<SECTlistsepchange>>&) retained verbatim,
++then the remainder is expanded.
++.wen
+The whole is treated as a list of simple strings;
+if the subject string is a member of that list, then the condition is true.

 For the case-independent &%inlisti%& condition, case is defined per the system C locale.
 
 These are simpler to use versions of the more powerful &*forany*& condition.
 For the case-independent &%inlisti%& condition, case is defined per the system C locale.
 
 These are simpler to use versions of the more powerful &*forany*& condition.


@@ -12199,6 +12278,10 @@ just as easy to use the fact that a lookup is itself a condition, and write:
 
 Note that <&'string2'&> is not itself subject to string expansion, unless
 Exim was built with the EXPAND_LISTMATCH_RHS option.
 
 Note that <&'string2'&> is not itself subject to string expansion, unless
 Exim was built with the EXPAND_LISTMATCH_RHS option.

+.new
+For the latter case, only the part after any leading
+change-of-separator specification is expanded.
+.wen

 
 Consult section &<<SECThoslispatip>>& for further details of these patterns.
 
 
 Consult section &<<SECThoslispatip>>& for further details of these patterns.
 


@@ -12227,9 +12310,10 @@ ${if match_domain{$domain}{+local_domains}{...
 .endd
 .cindex "&`+caseful`&"
 For address lists, the matching starts off caselessly, but the &`+caseful`&
 .endd
 .cindex "&`+caseful`&"
 For address lists, the matching starts off caselessly, but the &`+caseful`&

-item can be used, as in all address lists, to cause subsequent items to
-have their local parts matched casefully. Domains are always matched
-caselessly.
+item can be used, as in all address lists, to cause subsequent items
+(including those of referenced named lists)
+to have their local parts matched casefully.
+Domains are always matched caselessly.

 
 The variable &$value$& will be set for a successful match and can be
 used in the success clause of an &%if%& expansion item using the condition.
 
 The variable &$value$& will be set for a successful match and can be
 used in the success clause of an &%if%& expansion item using the condition.


@@ -12244,6 +12328,10 @@ Any previous &$value$& is restored after the if.
 
 Note that <&'string2'&> is not itself subject to string expansion, unless
 Exim was built with the EXPAND_LISTMATCH_RHS option.
 
 Note that <&'string2'&> is not itself subject to string expansion, unless
 Exim was built with the EXPAND_LISTMATCH_RHS option.

+.new
+For the latter case, only the part after any leading
+change-of-separator specification is expanded.
+.wen

 
 &*Note*&: Host lists are &'not'& supported in this way. This is because
 hosts have two identities: a name and an IP address, and it is not clear
 
 &*Note*&: Host lists are &'not'& supported in this way. This is because
 hosts have two identities: a name and an IP address, and it is not clear


@@ -12558,6 +12646,13 @@ this variable holds the pipe command when the transport is running.
 .vindex ATRN "data for routing"
 When an ATRN command is accepted, this variable is filled in with the client
 IP and port, for use in a manualroute router.
 .vindex ATRN "data for routing"
 When an ATRN command is accepted, this variable is filled in with the client
 IP and port, for use in a manualroute router.

+
+.vitem &$atrn_mode$&
+.vindex ATRN mode
+.vindex ODMR mode
+When in provider mode this variable will contain &"P"&.
+When in customer mode it will contain &"C"&.
+Otherwise, it will be empty.

 .wen
 
 .vitem "&$auth1$& &-- &$auth4$&"
 .wen
 
 .vitem "&$auth1$& &-- &$auth4$&"


@@ -15264,12 +15359,13 @@ non-SMTP message. See section &<<SECnonSMTP>>& for further details.
 .cindex ATRN "ACL for"
 .cindex ATRN advertisement
 .cindex "ESMTP extensions" ATRN
 .cindex ATRN "ACL for"
 .cindex ATRN advertisement
 .cindex "ESMTP extensions" ATRN

+.cindex ODMR provider

 This option defines the ACL that is run when an SMTP ATRN command is
 received.
 If no value is set, or the result after expansion is an empty string,
 then the ATRN facility is not advertised.
 See chapter &<<CHAPACL>>& for general information on ACLs,
 This option defines the ACL that is run when an SMTP ATRN command is
 received.
 If no value is set, or the result after expansion is an empty string,
 then the ATRN facility is not advertised.
 See chapter &<<CHAPACL>>& for general information on ACLs,

-and section &<<SECTATRN>>& for description of ATRN.
+and section &<<SECTODMRPRDVR>>& for description of ATRN.

 .wen
 
 .option acl_smtp_auth main string&!! unset
 .wen
 
 .option acl_smtp_auth main string&!! unset


@@ -19724,8 +19820,8 @@ real_localuser:
 For security, it would probably be a good idea to restrict the use of this
 router to locally-generated messages, using a condition such as this:
 .code
 For security, it would probably be a good idea to restrict the use of this
 router to locally-generated messages, using a condition such as this:
 .code

-  condition = ${if match {$sender_host_address}\
-                         {\N^(|127\.0\.0\.1)$\N}}
+  condition = ${if match_ip {$sender_host_address} \
+                       {<; ; 127.0.0.1 ; ::1}}

 .endd
 
 If both &%local_part_prefix%& and &%local_part_suffix%& are set for a router,
 .endd
 
 If both &%local_part_prefix%& and &%local_part_suffix%& are set for a router,


@@ -22467,8 +22563,8 @@ real_localuser:
 For security, it would probably be a good idea to restrict the use of this
 router to locally-generated messages, using a condition such as this:
 .code
 For security, it would probably be a good idea to restrict the use of this
 router to locally-generated messages, using a condition such as this:
 .code

-  condition = ${if match {$sender_host_address}\
-                         {\N^(|127\.0\.0\.1)$\N}}
+  condition = ${if match_ip {$sender_host_address} \
+                       {<; ; 127.0.0.1 ; ::1}}

 .endd
 
 
 .endd
 
 


@@ -37980,22 +38076,29 @@ for it to change them before running the command.
 
 
 .new
 
 
 .new

-.subsection "The ATRN command" SECTATRN
+.subsection "The ATRN command, and ODMR" SECTODMR

 .cindex ATRN processing
 .cindex "ESMTP extensions" ATRN
 .cindex ATRN processing
 .cindex "ESMTP extensions" ATRN

-A second method for intermittently-connecting destinations
-is specified by
+.cindex ODMR provider
+A second method for handling
+On-Demand Message Reception (ODMR)
+for intermittently-connecting destinations is specified by

 &url(https://www.rfc-editor.org/rfc/rfc2645.html,RFC 2645).
 
 This describes an ESMTP command called ATRN which requests
 &url(https://www.rfc-editor.org/rfc/rfc2645.html,RFC 2645).
 
 This describes an ESMTP command called ATRN which requests

-a swap in server/client roles of the communicating endpoints, and delivery
-of queued messages.
+a swap in server/client roles of the communicating SMTP endpoints,
+and delivery of queued messages.

 Note that this supports customers having IP addresses that
 change frequently.
 
 Note that this supports customers having IP addresses that
 change frequently.
 

-Exim supports the &"provider"& side of ATRN, using the terms
-of that specification:
-initially as an SMTP server, then transferring to an SMTP client
+Exim supports both the &"provider"& and &"customer"& sides of ODMR,
+to use the terms of that specification.
+
+. need a sub-subsection here
+.subsection "ODMR provider connection" SECTODMRPRDVR
+
+In the &"provider"& use case Exim is
+initially an SMTP server, then transferring to an SMTP client

 role if an ATRN command is accepted.
 
 .oindex "&%acl_smtp_atrn%&"
 role if an ATRN command is accepted.
 
 .oindex "&%acl_smtp_atrn%&"


@@ -38062,13 +38165,13 @@ other situations so can be safely placed in a general router chain.
 For example:
 .code
 begin routers
 For example:
 .code
 begin routers

-odmr_client:
+to_odmr_customer:

   driver =      manualroute
   route_data =  <;$atrn_host
   driver =      manualroute
   route_data =  <;$atrn_host

-  transport =   client_smtp
+  transport =   call_customer

 
 begin transports
 
 begin transports

-client_smtp:
+call_customer:

   driver =      smtp
 .endd
 
   driver =      smtp
 .endd
 


@@ -38084,6 +38187,46 @@ method be supported.
 Exim does not enforce this, but leaves it up to the configuration;
 see chapter &<<CHID9>>&.
 
 Exim does not enforce this, but leaves it up to the configuration;
 see chapter &<<CHID9>>&.
 

+
+.subsection "ODMR customer connection" SECTODMRCUST
+.cindex ODMR customer
+Exim supports the &"customer"& side of ODMR,
+with a command-line option &"-atrn"&& that requests a connection
+to a given host, issuance of an ATRN command then operation
+in SMTP server mode.
+The option must be followed by two arguments.
+
+The first is the name or IP of the provider to be contacted.
+
+The second, which may be empty, should be a comma-separated list
+of domains for which mail is to be requested.
+Interpretation of the list is up to the provider;
+an empty list is expected to result in some default being returned.
+
+The provider host is placed in &$domain$" for routing;
+router and transport must be configured suitably to make the connection.
+For example:
+.code
+begin routers
+to_odmr_provider:
+  driver =     manualroute
+  condition =  ${if eq {$atrn_mode}{C}}
+  route_data = <;$domain
+  transport =  call_provider
+
+begin transports
+call_provider:
+  driver =              smtp
+  port =                odmr
+  hosts_try_auth =     *
+  command_timeout =     10m
+.endd
+
+Note that the specification requires a long timeout for the ATRN
+command, to allow for scanning of queued messages.
+
+Configuration should also include client-side authentication
+and processing for receiving messages.

 .wen
 
 
 .wen