X-Git-Url: https://git.exim.org/users/jgh/exim.git/blobdiff_plain/d36a05018e1ac918951d551450acc53137ecf6e0..6c4545562d5b520ad23d9470382dde4b2af9671f:/doc/doc-txt/experimental-spec.txt?ds=inline diff --git a/doc/doc-txt/experimental-spec.txt b/doc/doc-txt/experimental-spec.txt index 7bb17883c..1e826aee1 100644 --- a/doc/doc-txt/experimental-spec.txt +++ b/doc/doc-txt/experimental-spec.txt @@ -6,67 +6,7 @@ about experimental features, all of which are unstable and liable to incompatible change. -OCSP Stapling support --------------------------------------------------------------- - -X.509 PKI certificates expire and can be revoked; to handle this, the -clients need some way to determine if a particular certificate, from a -particular Certificate Authority (CA), is still valid. There are three -main ways to do so. - -The simplest way is to serve up a Certificate Revocation List (CRL) with -an ordinary web-server, regenerating the CRL before it expires. The -downside is that clients have to periodically re-download a potentially -huge file from every certificate authority it knows of. - -The way with most moving parts at query time is Online Certificate -Status Protocol (OCSP), where the client verifies the certificate -against an OCSP server run by the CA. This lets the CA track all -usage of the certs. This requires running software with access to the -private key of the CA, to sign the responses to the OCSP queries. OCSP -is based on HTTP and can be proxied accordingly. - -The only widespread OCSP server implementation (known to this writer) -comes as part of OpenSSL and aborts on an invalid request, such as -connecting to the port and then disconnecting. This requires -re-entering the passphrase each time some random client does this. - -The third way is OCSP Stapling; in this, the server using a certificate -issued by the CA periodically requests an OCSP proof of validity from -the OCSP server, then serves it up inline as part of the TLS -negotiation. This approach adds no extra round trips, does not let the -CA track users, scales well with number of certs issued by the CA and is -resilient to temporary OCSP server failures, as long as the server -starts retrying to fetch an OCSP proof some time before its current -proof expires. The downside is that it requires server support. - -If Exim is built with EXPERIMENTAL_OCSP and it was built with OpenSSL, -then it gains one new option: "tls_ocsp_file". - -The file specified therein is expected to be in DER format, and contain -an OCSP proof. Exim will serve it as part of the TLS handshake. This -option will be re-expanded for SNI, if the tls_certificate option -contains $tls_sni, as per other TLS options. - -Exim does not at this time implement any support for fetching a new OCSP -proof. The burden is on the administrator to handle this, outside of -Exim. The file specified should be replaced atomically, so that the -contents are always valid. Exim will expand the "tls_ocsp_file" option -on each connection, so a new file will be handled transparently on the -next connection. - -Exim will check for a valid next update timestamp in the OCSP proof; -if not present, or if the proof has expired, it will be ignored. - -At this point in time, we're gathering feedback on use, to determine if -it's worth adding complexity to the Exim daemon to periodically re-fetch -OCSP files and somehow handling multiple files. There is no client support -for OCSP in Exim, this is feature expected to be used by mail clients. - - - - -Brightmail AntiSpam (BMI) suppport +Brightmail AntiSpam (BMI) support -------------------------------------------------------------- Brightmail AntiSpam is a commercial package. Please see @@ -102,7 +42,7 @@ These four steps are explained in more details below. 1) Adding support for BMI at compile time To compile with BMI support, you need to link Exim against - the Brighmail client SDK, consisting of a library + the Brightmail client SDK, consisting of a library (libbmiclient_single.so) and a header file (bmi_api.h). You'll also need to explicitly set a flag in the Makefile to include BMI support in the Exim binary. Both can be achieved @@ -404,15 +344,21 @@ which the spf condition should succeed. Valid strings are: This means the queried domain has published a SPF record, but wants to allow outside servers to send mail under its domain as well. - o err_perm This indicates a syntax error in the SPF - record of the queried domain. This should be - treated like "none". - o err_temp This indicates a temporary error during all + This should be treated like "none". + o permerror This indicates a syntax error in the SPF + record of the queried domain. You may deny + messages when this occurs. (Changed in 4.83) + o temperror This indicates a temporary error during all processing, including Exim's SPF processing. You may defer messages when this occurs. + (Changed in 4.83) + o err_temp Same as permerror, deprecated in 4.83, will be + removed in a future release. + o err_perm Same as temperror, deprecated in 4.83, will be + removed in a future release. You can prefix each string with an exclamation mark to invert -is meaning, for example "!fail" will match all results but +its meaning, for example "!fail" will match all results but "fail". The string list is evaluated left-to-right, in a short-circuit fashion. When a string matches the outcome of the SPF check, the condition succeeds. If none of the listed @@ -462,8 +408,8 @@ variables. $spf_result This contains the outcome of the SPF check in string form, - one of pass, fail, softfail, none, neutral, err_perm or - err_temp. + one of pass, fail, softfail, none, neutral, permerror or + temperror. $spf_smtp_comment This contains a string that can be used in a SMTP response @@ -502,16 +448,29 @@ spf_guess = v=spf1 a/16 mx/16 ptr ?all would relax host matching rules to a broader network range. +A lookup expansion is also available. It takes an email +address as the key and an IP address as the database: + + ${lookup {username@domain} spf {ip.ip.ip.ip}} + +The lookup will return the same result strings as they can appear in +$spf_result (pass,fail,softfail,neutral,none,err_perm,err_temp). +Currently, only IPv4 addresses are supported. + + + SRS (Sender Rewriting Scheme) Support -------------------------------------------------------------- Exiscan currently includes SRS support via Miles Wilton's libsrs_alt library. The current version of the supported -library is 0.5. +library is 0.5, there are reports of 1.0 working. In order to use SRS, you must get a copy of libsrs_alt from -http://srs.mirtol.com/ +https://opsec.eu/src/srs/ + +(not the original source, which has disappeared.) Unpack the tarball, then refer to MTAs/README.EXIM to proceed. You need to set @@ -521,8 +480,10 @@ EXPERIMENTAL_SRS=yes in your Local/Makefile. + DCC Support -------------------------------------------------------------- +Distributed Checksum Clearinghouse; http://www.rhyolite.com/dcc/ *) Building exim @@ -570,10 +531,549 @@ through to eg. SpamAssassin. If you want to pass even more headers in the middle of the DATA stage you can set $acl_m_dcc_add_header -to tell the DCC routines add more information; eg, you might set +to tell the DCC routines to add more information; eg, you might set this to some results from ClamAV. Be careful. Header syntax is not checked and is added "as is". +In case you've troubles with sites sending the same queue items from several +hosts and fail to get through greylisting you can use +$acl_m_dcc_override_client_ip + +Setting $acl_m_dcc_override_client_ip to an IP address overrides the default +of $sender_host_address. eg. use the following ACL in DATA stage: + + warn set acl_m_dcc_override_client_ip = \ + ${lookup{$sender_helo_name}nwildlsearch{/etc/mail/multipleip_sites}{$value}{}} + condition = ${if def:acl_m_dcc_override_client_ip} + log_message = dbg: acl_m_dcc_override_client_ip set to \ + $acl_m_dcc_override_client_ip + +Then set something like +# cat /etc/mail/multipleip_sites +mout-xforward.gmx.net 82.165.159.12 +mout.gmx.net 212.227.15.16 + +Use a reasonable IP. eg. one the sending cluster actually uses. + +DMARC Support +-------------------------------------------------------------- + +DMARC combines feedback from SPF, DKIM, and header From: in order +to attempt to provide better indicators of the authenticity of an +email. This document does not explain the fundamentals, you +should read and understand how it works by visiting the website at +http://www.dmarc.org/. + +DMARC support is added via the libopendmarc library. Visit: + + http://sourceforge.net/projects/opendmarc/ + +to obtain a copy, or find it in your favorite rpm package +repository. If building from source, this description assumes +that headers will be in /usr/local/include, and that the libraries +are in /usr/local/lib. + +1. To compile Exim with DMARC support, you must first enable SPF. +Please read the above section on enabling the EXPERIMENTAL_SPF +feature. You must also have DKIM support, so you cannot set the +DISABLE_DKIM feature. Once both of those conditions have been met +you can enable DMARC in Local/Makefile: + +EXPERIMENTAL_DMARC=yes +LDFLAGS += -lopendmarc +# CFLAGS += -I/usr/local/include +# LDFLAGS += -L/usr/local/lib + +The first line sets the feature to include the correct code, and +the second line says to link the libopendmarc libraries into the +exim binary. The commented out lines should be uncommented if you +built opendmarc from source and installed in the default location. +Adjust the paths if you installed them elsewhere, but you do not +need to uncomment them if an rpm (or you) installed them in the +package controlled locations (/usr/include and /usr/lib). + + +2. Use the following global settings to configure DMARC: + +Optional: +dmarc_tld_file Defines the location of a text file of valid + top level domains the opendmarc library uses + during domain parsing. Maintained by Mozilla, + the most current version can be downloaded + from a link at http://publicsuffix.org/list/. + If unset, "/etc/exim/opendmarc.tlds" (hardcoded) + is used. + +Optional: +dmarc_history_file Defines the location of a file to log results + of dmarc verification on inbound emails. The + contents are importable by the opendmarc tools + which will manage the data, send out DMARC + reports, and expire the data. Make sure the + directory of this file is writable by the user + exim runs as. + +dmarc_forensic_sender The email address to use when sending a + forensic report detailing alignment failures + if a sender domain's dmarc record specifies it + and you have configured Exim to send them. + Default: do-not-reply@$default_hostname + + +3. By default, the DMARC processing will run for any remote, +non-authenticated user. It makes sense to only verify DMARC +status of messages coming from remote, untrusted sources. You can +use standard conditions such as hosts, senders, etc, to decide that +DMARC verification should *not* be performed for them and disable +DMARC with a control setting: + + control = dmarc_disable_verify + +A DMARC record can also specify a "forensic address", which gives +exim an email address to submit reports about failed alignment. +Exim does not do this by default because in certain conditions it +results in unintended information leakage (what lists a user might +be subscribed to, etc). You must configure exim to submit forensic +reports to the owner of the domain. If the DMARC record contains a +forensic address and you specify the control statement below, then +exim will send these forensic emails. It's also advised that you +configure a dmarc_forensic_sender because the default sender address +construction might be inadequate. + + control = dmarc_enable_forensic + +(AGAIN: You can choose not to send these forensic reports by simply +not putting the dmarc_enable_forensic control line at any point in +your exim config. If you don't tell it to send them, it will not +send them.) + +There are no options to either control. Both must appear before +the DATA acl. + + +4. You can now run DMARC checks in incoming SMTP by using the +"dmarc_status" ACL condition in the DATA ACL. You are required to +call the spf condition first in the ACLs, then the "dmarc_status" +condition. Putting this condition in the ACLs is required in order +for a DMARC check to actually occur. All of the variables are set +up before the DATA ACL, but there is no actual DMARC check that +occurs until a "dmarc_status" condition is encountered in the ACLs. + +The dmarc_status condition takes a list of strings on its +right-hand side. These strings describe recommended action based +on the DMARC check. To understand what the policy recommendations +mean, refer to the DMARC website above. Valid strings are: + + o accept The DMARC check passed and the library recommends + accepting the email. + o reject The DMARC check failed and the library recommends + rejecting the email. + o quarantine The DMARC check failed and the library recommends + keeping it for further inspection. + o none The DMARC check passed and the library recommends + no specific action, neutral. + o norecord No policy section in the DMARC record for this + sender domain. + o nofrom Unable to determine the domain of the sender. + o temperror Library error or dns error. + o off The DMARC check was disabled for this email. + +You can prefix each string with an exclamation mark to invert its +meaning, for example "!accept" will match all results but +"accept". The string list is evaluated left-to-right in a +short-circuit fashion. When a string matches the outcome of the +DMARC check, the condition succeeds. If none of the listed +strings matches the outcome of the DMARC check, the condition +fails. + +Of course, you can also use any other lookup method that Exim +supports, including LDAP, Postgres, MySQL, etc, as long as the +result is a list of colon-separated strings. + +Several expansion variables are set before the DATA ACL is +processed, and you can use them in this ACL. The following +expansion variables are available: + + o $dmarc_status + This is a one word status indicating what the DMARC library + thinks of the email. It is a combination of the results of + DMARC record lookup and the SPF/DKIM/DMARC processing results + (if a DMARC record was found). The actual policy declared + in the DMARC record is in a separate expansion variable. + + o $dmarc_status_text + This is a slightly longer, human readable status. + + o $dmarc_used_domain + This is the domain which DMARC used to look up the DMARC + policy record. + + o $dmarc_domain_policy + This is the policy declared in the DMARC record. Valid values + are "none", "reject" and "quarantine". It is blank when there + is any error, including no DMARC record. + + o $dmarc_ar_header + This is the entire Authentication-Results header which you can + add using an add_header modifier. + + +5. How to enable DMARC advanced operation: +By default, Exim's DMARC configuration is intended to be +non-intrusive and conservative. To facilitate this, Exim will not +create any type of logging files without explicit configuration by +you, the admin. Nor will Exim send out any emails/reports about +DMARC issues without explicit configuration by you, the admin (other +than typical bounce messages that may come about due to ACL +processing or failure delivery issues). + +In order to log statistics suitable to be imported by the opendmarc +tools, you need to: +a. Configure the global setting dmarc_history_file. +b. Configure cron jobs to call the appropriate opendmarc history + import scripts and truncating the dmarc_history_file. + +In order to send forensic reports, you need to: +a. Configure the global setting dmarc_forensic_sender. +b. Configure, somewhere before the DATA ACL, the control option to + enable sending DMARC forensic reports. + + +6. Example usage: +(RCPT ACL) + warn domains = +local_domains + hosts = +local_hosts + control = dmarc_disable_verify + + warn !domains = +screwed_up_dmarc_records + control = dmarc_enable_forensic + + warn condition = (lookup if destined to mailing list) + set acl_m_mailing_list = 1 + +(DATA ACL) + warn dmarc_status = accept : none : off + !authenticated = * + log_message = DMARC DEBUG: $dmarc_status $dmarc_used_domain + add_header = $dmarc_ar_header + + warn dmarc_status = !accept + !authenticated = * + log_message = DMARC DEBUG: '$dmarc_status' for $dmarc_used_domain + + warn dmarc_status = quarantine + !authenticated = * + set $acl_m_quarantine = 1 + # Do something in a transport with this flag variable + + deny condition = ${if eq{$dmarc_domain_policy}{reject}} + condition = ${if eq{$acl_m_mailing_list}{1}} + message = Messages from $dmarc_used_domain break mailing lists + + deny dmarc_status = reject + !authenticated = * + message = Message from $dmarc_used_domain failed sender's DMARC policy, REJECT + + + +DANE +------------------------------------------------------------ +DNS-based Authentication of Named Entities, as applied +to SMTP over TLS, provides assurance to a client that +it is actually talking to the server it wants to rather +than some attacker operating a Man In The Middle (MITM) +operation. The latter can terminate the TLS connection +you make, and make another one to the server (so both +you and the server still think you have an encrypted +connection) and, if one of the "well known" set of +Certificate Authorities has been suborned - something +which *has* been seen already (2014), a verifiable +certificate (if you're using normal root CAs, eg. the +Mozilla set, as your trust anchors). + +What DANE does is replace the CAs with the DNS as the +trust anchor. The assurance is limited to a) the possibility +that the DNS has been suborned, b) mistakes made by the +admins of the target server. The attack surface presented +by (a) is thought to be smaller than that of the set +of root CAs. + +It also allows the server to declare (implicitly) that +connections to it should use TLS. An MITM could simply +fail to pass on a server's STARTTLS. + +DANE scales better than having to maintain (and +side-channel communicate) copies of server certificates +for every possible target server. It also scales +(slightly) better than having to maintain on an SMTP +client a copy of the standard CAs bundle. It also +means not having to pay a CA for certificates. + +DANE requires a server operator to do three things: +1) run DNSSEC. This provides assurance to clients +that DNS lookups they do for the server have not +been tampered with. The domain MX record applying +to this server, its A record, its TLSA record and +any associated CNAME records must all be covered by +DNSSEC. +2) add TLSA DNS records. These say what the server +certificate for a TLS connection should be. +3) offer a server certificate, or certificate chain, +in TLS connections which is traceable to the one +defined by (one of?) the TSLA records + +There are no changes to Exim specific to server-side +operation of DANE. + +The TLSA record for the server may have "certificate +usage" of DANE-TA(2) or DANE-EE(3). The latter specifies +the End Entity directly, i.e. the certificate involved +is that of the server (and should be the sole one transmitted +during the TLS handshake); this is appropriate for a +single system, using a self-signed certificate. + DANE-TA usage is effectively declaring a specific CA +to be used; this might be a private CA or a public, +well-known one. A private CA at simplest is just +a self-signed certificate which is used to sign +cerver certificates, but running one securely does +require careful arrangement. If a private CA is used +then either all clients must be primed with it, or +(probably simpler) the server TLS handshake must transmit +the entire certificate chain from CA to server-certificate. +If a public CA is used then all clients must be primed with it +(losing one advantage of DANE) - but the attack surface is +reduced from all public CAs to that single CA. +DANE-TA is commonly used for several services and/or +servers, each having a TLSA query-domain CNAME record, +all of which point to a single TLSA record. + +The TLSA record should have a Selector field of SPKI(1) +and a Matching Type field of SHA2-512(2). + +At the time of writing, https://www.huque.com/bin/gen_tlsa +is useful for quickly generating TLSA records; and commands like + + openssl x509 -in -pubkey -noout /dev/null \ + | openssl sha512 \ + | awk '{print $2}' + +are workable for 4th-field hashes. + +For use with the DANE-TA model, server certificates +must have a correct name (SubjectName or SubjectAltName). + +The use of OCSP-stapling should be considered, allowing +for fast revocation of certificates (which would otherwise +be limited by the DNS TTL on the TLSA records). However, +this is likely to only be usable with DANE-TA. NOTE: the +default of requesting OCSP for all hosts is modified iff +DANE is in use, to: + + hosts_request_ocsp = ${if or { {= {0}{$tls_out_tlsa_usage}} \ + {= {4}{$tls_out_tlsa_usage}} } \ + {*}{}} + +The (new) variable $tls_out_tlsa_usage is a bitfield with +numbered bits set for TLSA record usage codes. +The zero above means DANE was not in use, +the four means that only DANE-TA usage TLSA records were +found. If the definition of hosts_request_ocsp includes the +string "tls_out_tlsa_usage", they are re-expanded in time to +control the OCSP request. + +This modification of hosts_request_ocsp is only done if +it has the default value of "*". Admins who change it, and +those who use hosts_require_ocsp, should consider the interaction +with DANE in their OCSP settings. + + +For client-side DANE there are two new smtp transport options, +hosts_try_dane and hosts_require_dane. +[ should they be domain-based rather than host-based? ] + +Hosts_require_dane will result in failure if the target host +is not DNSSEC-secured. + +DANE will only be usable if the target host has DNSSEC-secured +MX, A and TLSA records. + +A TLSA lookup will be done if either of the above options match +and the host-lookup succeeded using dnssec. +If a TLSA lookup is done and succeeds, a DANE-verified TLS connection +will be required for the host. If it does not, the host will not +be used; there is no fallback to non-DANE or non-TLS. + +If DANE is requested and useable (see above) the following transport +options are ignored: + hosts_require_tls + tls_verify_hosts + tls_try_verify_hosts + tls_verify_certificates + tls_crl + tls_verify_cert_hostnames + +If DANE is not usable, whether requested or not, and CA-anchored +verification evaluation is wanted, the above variables should be set +appropriately. + +Currently dnssec_request_domains must be active (need to think about that) +and dnssec_require_domains is ignored. + +If verification was successful using DANE then the "CV" item +in the delivery log line will show as "CV=dane". + +There is a new variable $tls_out_dane which will have "yes" if +verification succeeded using DANE and "no" otherwise (only useful +in combination with EXPERIMENTAL_EVENT), and a new variable +$tls_out_tlsa_usage (detailed above). + + + +DSN extra information +--------------------- +If compiled with EXPERIMENTAL_DSN_INFO extra information will be added +to DSN fail messages ("bounces"), when available. The intent is to aid +tracing of specific failing messages, when presented with a "bounce" +complaint and needing to search logs. + + +The remote MTA IP address, with port number if nonstandard. +Example: + Remote-MTA: X-ip; [127.0.0.1]:587 +Rationale: + Several addresses may correspond to the (already available) + dns name for the remote MTA. + +The remote MTA connect-time greeting. +Example: + X-Remote-MTA-smtp-greeting: X-str; 220 the.local.host.name ESMTP Exim x.yz Tue, 2 Mar 1999 09:44:33 +0000 +Rationale: + This string sometimes presents the remote MTA's idea of its + own name, and sometimes identifies the MTA software. + +The remote MTA response to HELO or EHLO. +Example: + X-Remote-MTA-helo-response: X-str; 250-the.local.host.name Hello localhost [127.0.0.1] +Limitations: + Only the first line of a multiline response is recorded. +Rationale: + This string sometimes presents the remote MTA's view of + the peer IP connecting to it. + +The reporting MTA detailed diagnostic. +Example: + X-Exim-Diagnostic: X-str; SMTP error from remote mail server after RCPT TO:: 550 hard error +Rationale: + This string sometimes give extra information over the + existing (already available) Diagnostic-Code field. + + +Note that non-RFC-documented field names and data types are used. + + +LMDB Lookup support +------------------- +LMDB is an ultra-fast, ultra-compact, crash-proof key-value embedded data store. +It is modeled loosely on the BerkeleyDB API. You should read about the feature +set as well as operation modes at https://symas.com/products/lightning-memory-mapped-database/ + +LMDB single key lookup support is provided by linking to the LMDB C library. +The current implementation does not support writing to the LMDB database. + +Visit https://github.com/LMDB/lmdb to download the library or find it in your +operating systems package repository. + +If building from source, this description assumes that headers will be in +/usr/local/include, and that the libraries are in /usr/local/lib. + +1. In order to build exim with LMDB lookup support add or uncomment + +EXPERIMENTAL_LMDB=yes + +to your Local/Makefile. (Re-)build/install exim. exim -d should show +Experimental_LMDB in the line "Support for:". + +EXPERIMENTAL_LMDB=yes +LDFLAGS += -llmdb +# CFLAGS += -I/usr/local/include +# LDFLAGS += -L/usr/local/lib + +The first line sets the feature to include the correct code, and +the second line says to link the LMDB libraries into the +exim binary. The commented out lines should be uncommented if you +built LMDB from source and installed in the default location. +Adjust the paths if you installed them elsewhere, but you do not +need to uncomment them if an rpm (or you) installed them in the +package controlled locations (/usr/include and /usr/lib). + +2. Create your LMDB files, you can use the mdb_load utility which is +part of the LMDB distribution our your favourite language bindings. + +3. Add the single key lookups to your exim.conf file, example lookups +are below. + +${lookup{$sender_address_domain}lmdb{/var/lib/baruwa/data/db/relaydomains.mdb}{$value}} +${lookup{$sender_address_domain}lmdb{/var/lib/baruwa/data/db/relaydomains.mdb}{$value}fail} +${lookup{$sender_address_domain}lmdb{/var/lib/baruwa/data/db/relaydomains.mdb}} + + +Queuefile transport +------------------- +Queuefile is a pseudo transport which does not perform final delivery. +It simply copies the exim spool files out of the spool directory into +an external directory retaining the exim spool format. + +The spool files can then be processed by external processes and then +requeued into exim spool directories for final delivery. + +The motivation/inspiration for the transport is to allow external +processes to access email queued by exim and have access to all the +information which would not be available if the messages were delivered +to the process in the standard email formats. + +The mailscanner package is one of the processes that can take advantage +of this transport to filter email. + +The transport can be used in the same way as the other existing transports, +i.e by configuring a router to route mail to a transport configured with +the queuefile driver. + +The transport only takes one option: + +* directory - This is used to specify the directory messages should be +copied to + +The generic transport options (body_only, current_directory, disable_logging, +debug_print, delivery_date_add, envelope_to_add, event_action, group, +headers_add, headers_only, headers_remove, headers_rewrite, home_directory, +initgroups, max_parallel, message_size_limit, rcpt_include_affixes, +retry_use_local_part, return_path, return_path_add, shadow_condition, +shadow_transport, transport_filter, transport_filter_timeout, user) are +ignored. + +Sample configuration: + +(Router) + +scan: + driver = accept + transport = scan + +(Transport) + +scan: + driver = queuefile + directory = /var/spool/baruwa-scanner/input + + +In order to build exim with Queuefile transport support add or uncomment + +EXPERIMENTAL_QUEUEFILE=yes + +to your Local/Makefile. (Re-)build/install exim. exim -d should show +Experimental_QUEUEFILE in the line "Support for:". + -------------------------------------------------------------- End of file