X-Git-Url: https://git.exim.org/users/heiko/exim.git/blobdiff_plain/6c1c3d1dbe1a62ffd24ad9b3cd9efdfe275c74c5..14a465c3f0ae93b383f57f12b2ac8709ac0d55b6:/doc/doc-txt/experimental-spec.txt diff --git a/doc/doc-txt/experimental-spec.txt b/doc/doc-txt/experimental-spec.txt index 031c5f4c1..d8bd0bf46 100644 --- a/doc/doc-txt/experimental-spec.txt +++ b/doc/doc-txt/experimental-spec.txt @@ -6,97 +6,6 @@ about experimental features, all of which are unstable and liable to incompatible change. -OCSP Stapling support --------------------------------------------------------------- - -X.509 PKI certificates expire and can be revoked; to handle this, the -clients need some way to determine if a particular certificate, from a -particular Certificate Authority (CA), is still valid. There are three -main ways to do so. - -The simplest way is to serve up a Certificate Revocation List (CRL) with -an ordinary web-server, regenerating the CRL before it expires. The -downside is that clients have to periodically re-download a potentially -huge file from every certificate authority it knows of. - -The way with most moving parts at query time is Online Certificate -Status Protocol (OCSP), where the client verifies the certificate -against an OCSP server run by the CA. This lets the CA track all -usage of the certs. This requires running software with access to the -private key of the CA, to sign the responses to the OCSP queries. OCSP -is based on HTTP and can be proxied accordingly. - -The only widespread OCSP server implementation (known to this writer) -comes as part of OpenSSL and aborts on an invalid request, such as -connecting to the port and then disconnecting. This requires -re-entering the passphrase each time some random client does this. - -The third way is OCSP Stapling; in this, the server using a certificate -issued by the CA periodically requests an OCSP proof of validity from -the OCSP server, then serves it up inline as part of the TLS -negotiation. This approach adds no extra round trips, does not let the -CA track users, scales well with number of certs issued by the CA and is -resilient to temporary OCSP server failures, as long as the server -starts retrying to fetch an OCSP proof some time before its current -proof expires. The downside is that it requires server support. - -If Exim is built with EXPERIMENTAL_OCSP and it was built with OpenSSL, -or with GnuTLS 3.1.3 or later, then it gains a new global option: -"tls_ocsp_file". - -The file specified therein is expected to be in DER format, and contain -an OCSP proof. Exim will serve it as part of the TLS handshake. This -option will be re-expanded for SNI, if the tls_certificate option -contains $tls_sni, as per other TLS options. - -Exim does not at this time implement any support for fetching a new OCSP -proof. The burden is on the administrator to handle this, outside of -Exim. The file specified should be replaced atomically, so that the -contents are always valid. Exim will expand the "tls_ocsp_file" option -on each connection, so a new file will be handled transparently on the -next connection. - -Under OpenSSL Exim will check for a valid next update timestamp in the -OCSP proof; if not present, or if the proof has expired, it will be -ignored. - -Also, given EXPERIMENTAL_OCSP, the smtp transport gains two options: -- "hosts_require_ocsp"; a host-list for which an OCSP Stapling -is requested and required for the connection to proceed. The default -value is empty. -- "hosts_request_ocsp"; a host-list for which (additionally) an OCSP -Stapling is requested (but not necessarily verified). The default -value is "*" meaning that requests are made unless configured -otherwise. - -The host(s) should also be in "hosts_require_tls", and -"tls_verify_certificates" configured for the transport. - -For the client to be able to verify the stapled OCSP the server must -also supply, in its stapled information, any intermediate -certificates for the chain leading to the OCSP proof from the signer -of the server certificate. There may be zero or one such. These -intermediate certificates should be added to the server OCSP stapling -file (named by tls_ocsp_file). - -Note that the proof only covers the terminal server certificate, -not any of the chain from CA to it. - -At this point in time, we're gathering feedback on use, to determine if -it's worth adding complexity to the Exim daemon to periodically re-fetch -OCSP files and somehow handling multiple files. - - A helper script "ocsp_fetch.pl" for fetching a proof from a CA - OCSP server is supplied. The server URL may be included in the - server certificate, if the CA is helpful. - - One failure mode seen was the OCSP Signer cert expiring before the end - of validity of the OCSP proof. The checking done by Exim/OpenSSL - noted this as invalid overall, but the re-fetch script did not. - - - - Brightmail AntiSpam (BMI) suppport -------------------------------------------------------------- @@ -853,87 +762,94 @@ b. Configure, somewhere before the DATA ACL, the control option to Transport post-delivery actions -------------------------------------------------------------- -An arbitrary per-transport string can be expanded on successful delivery, +An arbitrary per-transport string can be expanded upon various transport events and (for SMTP transports) a second string on deferrals caused by a host error. +Additionally a main-section configuration option can be expanded on some +per-message events. This feature may be used, for example, to write exim internal log information (not available otherwise) into a database. -In order to use the feature, you must set +In order to use the feature, you must compile with EXPERIMENTAL_TPDA=yes in your Local/Makefile -and define the expandable strings in the runtime config file, to -be executed at end of delivery. +and define one or both of +- the tpda_event_action option in the transport +- the delivery_event_action +to be expanded when the event fires. + +A new variable, $tpda_event, is set to the event type when the +expansion is done. The current list of events is: -Additionally, there are 6 more variables, available at end of -delivery: + msg:complete main per message + msg:delivery transport per recipient + msg:host:defer transport per attempt + msg:fail:delivery main per recipient + msg:fail:internal main per recipient + tcp:connect transport per connection + tcp:close transport per connection + tls:cert transport per certificate in verification chain + smtp:connect transport per connection -tpda_delivery_ip IP of host, which has accepted delivery -tpda_delivery_port Port of remote host which has accepted delivery -tpda_delivery_fqdn FQDN of host, which has accepted delivery -tpda_delivery_local_part local part of address being delivered -tpda_delivery_domain domain part of address being delivered -tpda_delivery_confirmation SMTP confirmation message +The expansion is called for all event types, and should use the $tpda_event +value to decide when to act. The variable data is a colon-separated +list, describing an event tree. -In case of a deferral caused by a host-error: -tpda_defer_errno Error number -tpda_defer_errstr Error string possibly containing more details +There is an auxilary variable, $tpda_data, for which the +content is event_dependent: -The $router_name and $transport_name variables are also usable. + msg:delivery smtp confirmation mssage + msg:host:defer error string + tls:cert verification chain depth + smtp:connect smtp banner +The msg:host:defer event populates one extra variable, $tpda_defer_errno. -To take action after successful deliveries, set the following option -on any transport of interest. +The following variables are likely to be useful depending on the event type: + + router_name, transport_name + local_part, domain + host, host_address, host_port + tls_out_peercert + lookup_dnssec_authenticated, tls_out_dane + sending_ip_address, sending_port + message_exim_id -tpda_delivery_action An example might look like: -tpda_delivery_action = \ -${lookup pgsql {SELECT * FROM record_Delivery( \ +tpda_event_action = ${if = {msg:delivery}{$tpda_event} \ +{${lookup pgsql {SELECT * FROM record_Delivery( \ '${quote_pgsql:$sender_address_domain}',\ '${quote_pgsql:${lc:$sender_address_local_part}}', \ - '${quote_pgsql:$tpda_delivery_domain}', \ - '${quote_pgsql:${lc:$tpda_delivery_local_part}}', \ - '${quote_pgsql:$tpda_delivery_ip}', \ - '${quote_pgsql:${lc:$tpda_delivery_fqdn}}', \ - '${quote_pgsql:$message_exim_id}')}} - -The string is expanded after the delivery completes and any + '${quote_pgsql:$domain}', \ + '${quote_pgsql:${lc:$local_part}}', \ + '${quote_pgsql:$host_address}', \ + '${quote_pgsql:${lc:$host}}', \ + '${quote_pgsql:$message_exim_id}')}} \ +} {}} + +The string is expanded for each of the supported events and any side-effects will happen. The result is then discarded. Note that for complex operations an ACL expansion can be used. -In order to log host deferrals, add the following option to an SMTP -transport: +The expansion of the tpda_event_action option should normally +return an empty string. Should it return anything else the +following will be forced: -tpda_host_defer_action + msg:delivery (ignored) + msg:host:defer (ignored) + msg:fail:delivery (ignored) + tcp:connect do not connect + tcp:close (ignored) + tls:cert refuse verification + smtp:connect close connection -This is a private option of the SMTP transport. It is intended to -log failures of remote hosts. It is executed only when exim has -attempted to deliver a message to a remote host and failed due to -an error which doesn't seem to be related to the individual -message, sender, or recipient address. -See section 47.2 of the exim documentation for more details on how -this is determined. -Example: -tpda_host_defer_action = \ -${lookup mysql {insert into delivlog set \ - msgid = '${quote_mysql:$message_exim_id}', \ - senderlp = '${quote_mysql:${lc:$sender_address_local_part}}', \ - senderdom = '${quote_mysql:$sender_address_domain}', \ - delivlp = '${quote_mysql:${lc:$tpda_delivery_local_part}}', \ - delivdom = '${quote_mysql:$tpda_delivery_domain}', \ - delivip = '${quote_mysql:$tpda_delivery_ip}', \ - delivport = '${quote_mysql:$tpda_delivery_port}', \ - delivfqdn = '${quote_mysql:$tpda_delivery_fqdn}', \ - deliverrno = '${quote_mysql:$tpda_defer_errno}', \ - deliverrstr = '${quote_mysql:$tpda_defer_errstr}' \ - }} Redis Lookup @@ -1108,11 +1024,27 @@ an example, in my connect ACL, I have: logwrite = Internal Server Address: $received_ip_address:$received_port -4. Runtime issues to be aware of: +4. Recommended ACL additions: - Since the real connections are all coming from your proxy, and the per host connection tracking is done before Proxy Protocol is evaluated, smtp_accept_max_per_host must be set high enough to handle all of the parallel volume you expect per inbound proxy. + - With the smtp_accept_max_per_host set so high, you lose the ability + to protect your server from massive numbers of inbound connections + from one IP. In order to prevent your server from being DOS'd, you + need to add a per connection ratelimit to your connect ACL. I + suggest something like this: + + # Set max number of connections per host + LIMIT = 5 + # Or do some kind of IP lookup in a flat file or database + # LIMIT = ${lookup{$sender_host_address}iplsearch{/etc/exim/proxy_limits}} + + defer message = Too many connections from this IP right now + ratelimit = LIMIT / 5s / per_conn / strict + + +5. Runtime issues to be aware of: - The proxy has 3 seconds (hard-coded in the source code) to send the required Proxy Protocol header after it connects. If it does not, the response to any commands will be: @@ -1131,7 +1063,7 @@ an example, in my connect ACL, I have: mail programs from working because that would require mail from localhost to use Proxy Protocol. Again, not advised! -5. Example of a refused connection because the Proxy Protocol header was +6. Example of a refused connection because the Proxy Protocol header was not sent from a host configured to use Proxy Protocol. In the example, the 3 second timeout occurred (when a Proxy Protocol banner should have been sent), the banner was displayed to the user, but all commands are