options, and new features, see the NewStuff file next to this ChangeLog.
+Exim version 4.92
+-----------------
+
+JH/01 Remove code calling the customisable local_scan function, unless a new
+ definition "HAVE_LOCAL_SCAN=yes" is present in the Local/Makefile.
+
+JH/02 Bug 1007: Avoid doing logging from signal-handlers, as that can result in
+ non-signal-safe functions being used.
+
+JH/03 Bug 2269: When presented with a received message having a stupidly large
+ number of DKIM-Signature headers, disable DKIM verification to avoid
+ a resource-consumption attack. The limit is set at twenty.
+
+JH/04 Add variables $arc_domains, $arc_oldest_pass for ARC verify. Fix the
+ report of oldest_pass in ${authres } in consequence, and separate out
+ some descriptions of reasons for verification fail.
+
+JH/05 Bug 2273: Cutthrough delivery left a window where the received messsage
+ files in the spool were present and unlocked. A queue-runner could spot
+ them, resulting in a duplicate delivery. Fix that by doing the unlock
+ after the unlink. Investigation by Tim Stewart. Take the opportunity to
+ add more error-checking on spoolfile handling while that code is being
+ messed with.
+
+PP/01 Refuse to open a spool data file (*-D) if it's a symlink.
+ No known attacks, no CVE, this is defensive hardening.
+
+JH/06 Bug 2275: The MIME ACL unlocked the received message files early, and
+ a queue-runner could start a delivery while other operations were ongoing.
+ Cutthrough delivery was a common victim, resulting in duplicate delivery.
+ Found and investigated by Tim Stewart. Fix by using the open message data
+ file handle rather than opening another, and not locally closing it (which
+ releases a lock) for that case, while creating the temporary .eml format
+ file for the MIME ACL. Also applies to "regex" and "spam" ACL conditions.
+
+JH/07 Bug 177: Make a random-recipient callout success visible in ACL, by setting
+ $sender_verify_failure/$recipient_verify_failure to "random".
+
+JH/08 When generating a selfsigned cert, use serial number 1 since zero is not
+ legitimate.
+
+JH/09 Bug 2274: Fix logging of cmdline args when starting in an unlinked cwd.
+ Previously this would segfault.
+
+JH/10 Fix ARC signing for case when DKIM signing failed. Previously this would
+ segfault.
+
+JH/11 Bug 2264: Exim now only follows CNAME chains one step by default. We'd
+ like zero, since the resolver should be doing this for us, But we need one
+ as a CNAME but no MX presence gets the CNAME returned; we need to check
+ that doesn't point to an MX to declare it "no MX returned" rather than
+ "error, loop". A new main option is added so the older capability of
+ following some limited number of chain links is maintained.
+
+JH/12 Add client-ip info to non-pass iprev ${authres } lines.
+
+JH/13 For receent Openssl versions (1.1 onward) use modern generic protocol
+ methods. These should support TLS 1.3; they arrived with TLS 1.3 and the
+ now-deprecated earlier definitions used only specified the range up to TLS
+ 1.2 (in the older-version library docs).
+
+JH/14 Bug 2284: Fix DKIM signing for body lines starting with a pair of dots.
+
+JH/15 Rework TLS client-side context management. Stop using a global, and
+ explicitly pass a context around. This enables future use of TLS for
+ connections to service-daemons (eg. malware scanning) while a client smtp
+ connection is using TLS; with cutthrough connections this is quite likely.
+
+JH/16 Fix ARC verification to do AS checks in reverse order.
+
+JH/17 Support a "tls" option on the ${readsocket } expansion item.
+
+JH/18 Bug 2287: Fix the protocol name (eg utf8esmtp) for multiple messages
+ using the SMTPUTF8 option on their MAIL FROM commands, in one connection.
+ Previously the "utf8" would be re-prepended for every additional message.
+
+JH/19 Reject MAIL FROM commands with SMTPUTF8 when the facility was not advertised.
+ Previously thery were accepted, resulting in issues when attempting to
+ forward messages to a non-supporting MTA.
+
+PP/02 Let -n work with printing macros too, not just options.
+
+JH/20 Bug 2296: Fix cutthrough for >1 address redirection. Previously only
+ one parent address was copied, and bogus data was used at delivery-logging
+ time. Either a crash (after delivery) or bogus log data could result.
+ Discovery and analysis by Tim Stewart.
+
+PP/03 Make ${utf8clean:} expansion operator detect incomplete final character.
+ Previously if the string ended mid-character, we did not insert the
+ promised '?' replacement.
+
+PP/04 Documentation: current string operators work on bytes, not codepoints.
+
+JH/21 Change as many as possible of the global flags into one-bit bitfields; these
+ should pack well giving a smaller memory footprint so better caching and
+ therefore performance. Group the declarations where this can't be done so
+ that the byte-sized flag variables are not interspersed among pointer
+ variables, giving a better chance of good packing by the compiler.
+
+JH/22 Bug 1896: Fix the envelope from for DMARC forensic reports to be possibly
+ non-null, to avoid issues with sites running BATV. Previously reports were
+ sent with an empty envelope sender so looked like bounces.
+
+JH/23 Bug 2318: Fix the noerror command within filters. It wasn't working.
+ The ignore_error flag wasn't being returned from the filter subprocess so
+ was not set for later routers. Investigation and fix by Matthias Kurz.
+
+JH/24 Bug 2310: Raise a msg:fail:internal event for each undelivered recipient,
+ and a msg:complete for the whole, when a message is manually removed using
+ -Mrm. Developement by Matthias Kurz, hacked on by JH.
+
+JH/25 Avoid fixed-size buffers for pathnames in DB access. This required using
+ a "Gnu special" function, asprintf() in the DB utility binary builds; I
+ hope that is portable enough.
+
+JH/26 Bug 2311: Fix DANE-TA verification under GnuTLS. Previously it was also
+ requiring a known-CA anchor certificate; make it now rely entirely on the
+ TLSA as an anchor. Checking the name on the leaf cert against the name
+ on the A-record for the host is still done for TA (but not for EE mode).
+
+JH/27 Fix logging of proxy address. Previously, a pointless "PRX=[]:0" would be
+ included in delivery lines for non-proxied connections, when compiled with
+ SUPPORT_SOCKS and running with proxy logging enabled.
+
+JH/28 Bug 2314: Fire msg:fail:delivery event even when error is being ignored.
+ Developement by Matthias Kurz, tweaked by JH. While in that bit of code,
+ move the existing event to fire before the normal logging of message
+ failure so that custom logging is bracketed by normal logging.
+
+JH/29 Bug 2322: A "fail" command in a non-system filter (file) now fires the
+ msg:fail:internal event. Developement by Matthias Kurz.
+
+JH/30 Bug 2329: Increase buffer size used for dns lookup from 2k, which was
+ far too small for todays use of crypto signatures stored there. Go all
+ the way to the max DNS message size of 64kB, even though this might be
+ overmuch for IOT constrained device use.
+
+JH/31 Fix a bad use of a copy function, which could be used to pointlessly
+ copy a string over itself. The library routine is documented as not
+ supporting overlapping copies, and on MacOS it actually raised a SIGABRT.
+
+JH/32 For main options check_spool_space and check_inode_space, where the
+ platform supports 64b integers, support more than the previous 2^31 kB
+ (i.e. more than 2 TB). Accept E, P and T multipliers in addition to
+ the previous G, M, k.
+
+JH/33 Bug 2338: Fix the cyrus-sasl authenticator to fill in the
+ $authenticated_fail_id variable on authentication failure. Previously
+ it was unset.
+
+JH/34 Increase RSA keysize of autogen selfsign cert from 1024 to 2048. RHEL 8.0
+ OpenSSL didn't want to use such a weak key. Do for GnuTLS also, and for
+ more-modern GnuTLS move from GNUTLS_SEC_PARAM_LOW to
+ GNUTLS_SEC_PARAM_MEDIUM.
+
+JH/35 OpenSSL: fail the handshake when SNI processing hits a problem, server
+ side. Previously we would continue as if no SNI had been received.
+
+JH/36 Harden the handling of string-lists. When a list consisted of a sole
+ "<" character, which should be a list-separator specification, we walked
+ off past the nul-terimation.
+
+JH/37 Bug 2341: Send "message delayed" warning MDNs (restricted to external
+ causes) even when the retry time is not yet met. Previously they were
+ not, meaning that when (say) an account was over-quota and temp-rejecting,
+ and multiple senders' messages were queued, only one sender would get
+ notified on each configured delay_warning cycle.
+
+JH/01 Bug 2351: Log failures to extract envelope addresses from message headers.
+
+
+Exim version 4.91
+-----------------
+
+GF/01 DEFER rather than ERROR on redis cluster MOVED response.
+ When redis_servers is set to a list of > 1 element, and the Redis servers
+ in that list are in cluster configuration, convert the REDIS_REPLY_ERROR
+ case of MOVED into a DEFER case instead, thus moving the query onto the
+ next server in the list. For a cluster of N elements, all N servers must
+ be defined in redis_servers.
+
+GF/02 Catch and remove uninitialized value warning in exiqsumm
+ Check for existence of @ARGV before looking at $ARGV[0]
+
+JH/01 Replace the store_release() internal interface with store_newblock(),
+ which internalises the check required to safely use the old one, plus
+ the allocate and data copy operations duplicated in both (!) of the
+ extant use locations.
+
+JH/02 Disallow '/' characters in queue names specified for the "queue=" ACL
+ modifier. This matches the restriction on the commandline.
+
+JH/03 Fix pgsql lookup for multiple result-tuples with a single column.
+ Previously only the last row was returned.
+
+JH/04 Bug 2217: Tighten up the parsing of DKIM signature headers. Previously
+ we assumed that tags in the header were well-formed, and parsed the
+ element content after inspecting only the first char of the tag.
+ Assumptions at that stage could crash the receive process on malformed
+ input.
+
+JH/05 Bug 2215: Fix crash associated with dnsdb lookup done from DKIM ACL.
+ While running the DKIM ACL we operate on the Permanent memory pool so that
+ variables created with "set" persist to the DATA ACL. Also (at any time)
+ DNS lookups that fail create cache records using the Permanent pool. But
+ expansions release any allocations made on the current pool - so a dnsdb
+ lookup expansion done in the DKIM ACL releases the memory used for the
+ DNS negative-cache, and bad things result. Solution is to switch to the
+ Main pool for expansions.
+ While we're in that code, add checks on the DNS cache during store_reset,
+ active in the testsuite.
+ Problem spotted, and debugging aided, by Wolfgang Breyha.
+
+JH/06 Fix issue with continued-connections when the DNS shifts unreliably.
+ When none of the hosts presented to a transport match an already-open
+ connection, close it and proceed with the list. Previously we would
+ queue the message. Spotted by Lena with Yahoo, probably involving
+ round-robin DNS.
+
+JH/07 Bug 2214: Fix SMTP responses resulting from non-accept result of MIME ACL.
+ Previously a spurious "250 OK id=" response was appended to the proper
+ failure response.
+
+JH/08 The "support for" informational output now, which built with Content
+ Scanning support, has a line for the malware scanner interfaces compiled
+ in. Interface can be individually included or not at build time.
+
+JH/09 The "aveserver", "kavdaemon" and "mksd" interfaces are now not included
+ by the template makefile "src/EDITME". The "STREAM" support for an older
+ ClamAV interface method is removed.
+
+JH/10 Bug 2223: Fix mysql lookup returns for the no-data case (when the number of
+ rows affected is given instead).
+
+JH/11 The runtime Berkeley DB library version is now additionally output by
+ "exim -d -bV". Previously only the compile-time version was shown.
+
+JH/12 Bug 2230: Fix cutthrough routing for nonfirst messages in an initiating
+ SMTP connection. Previously, when one had more receipients than the
+ first, an abortive onward connection was made. Move to full support for
+ multiple onward connections in sequence, handling cutthrough connection
+ for all multi-message initiating connections.
+
+JH/13 Bug 2229: Fix cutthrough routing for nonstandard port numbers defined by
+ routers. Previously, a multi-recipient message would fail to match the
+ onward-connection opened for the first recipient, and cause its closure.
+
+JH/14 Bug 2174: A timeout on connect for a callout was also erroneously seen as
+ a timeout on read on a GnuTLS initiating connection, resulting in the
+ initiating connection being dropped. This mattered most when the callout
+ was marked defer_ok. Fix to keep the two timeout-detection methods
+ separate.
+
+JH/15 Relax results from ACL control request to enable cutthrough, in
+ unsupported situations, from error to silently (except under debug)
+ ignoring. This covers use with PRDR, frozen messages, queue-only and
+ fake-reject.
+
+HS/01 Fix Buffer overflow in base64d() (CVE-2018-6789)
+
+JH/16 Fix bug in DKIM verify: a buffer overflow could corrupt the malloc
+ metadata, resulting in a crash in free().
+
+PP/01 Fix broken Heimdal GSSAPI authenticator integration.
+ Broken in f2ed27cf5, missing an equals sign for specified-initialisers.
+ Broken also in d185889f4, with init system revamp.
+
+JH/17 Bug 2113: Fix conversation closedown with the Avast malware scanner.
+ Previously we abruptly closed the connection after reading a malware-
+ found indication; now we go on to read the "scan ok" response line,
+ and send a quit.
+
+JH/18 Bug 2239: Enforce non-usability of control=utf8_downconvert in the mail
+ ACL. Previously, a crash would result.
+
+JH/19 Speed up macro lookups during configuration file read, by skipping non-
+ macro text after a replacement (previously it was only once per line) and
+ by skipping builtin macros when searching for an uppercase lead character.
+
+JH/20 DANE support moved from Experimental to mainline. The Makefile control
+ for the build is renamed.
+
+JH/21 Fix memory leak during multi-message connections using STARTTLS. A buffer
+ was allocated for every new TLS startup, meaning one per message. Fix
+ by only allocating once (OpenSSL) or freeing on TLS-close (GnuTLS).
+
+JH/22 Bug 2236: When a DKIM verification result is overridden by ACL, DMARC
+ reported the original. Fix to report (as far as possible) the ACL
+ result replacing the original.
+
+JH/23 Fix memory leak during multi-message connections using STARTTLS under
+ OpenSSL. Certificate information is loaded for every new TLS startup,
+ and the resources needed to be freed.
+
+JH/24 Bug 2242: Fix exim_dbmbuild to permit directoryless filenames.
+
+JH/25 Fix utf8_downconvert propagation through a redirect router. Previously it
+ was not propagated.
+
+JH/26 Bug 2253: For logging delivery lines under PRDR, append the overall
+ DATA response info to the (existing) per-recipient response info for
+ the "C=" log element. It can have useful tracking info from the
+ destination system. Patch from Simon Arlott.
+
+JH/27 Bug 2251: Fix ldap lookups that return a single attribute having zero-
+ length value. Previously this would segfault.
+
+HS/02 Support Avast multiline protoocol, this allows passing flags to
+ newer versions of the scanner.
+
+JH/28 Ensure that variables possibly set during message acceptance are marked
+ dead before release of memory in the daemon loop. This stops complaints
+ about them when the debug_store option is enabled. Discovered specifically
+ for sender_rate_period, but applies to a whole set of variables.
+ Do the same for the queue-runner and queue-list loops, for variables set
+ from spool message files. Do the same for the SMTP per-message loop, for
+ certain variables indirectly set in ACL operations.
+
+JH/29 Bug 2250: Fix a longstanding bug in heavily-pipelined SMTP input (such
+ as a multi-recipient message from a mailinglist manager). The coding had
+ an arbitrary cutoff number of characters while checking for more input;
+ enforced by writing a NUL into the buffer. This corrupted long / fast
+ input. The problem was exposed more widely when more pipelineing of SMTP
+ responses was introduced, and one Exim system was feeding another.
+ The symptom is log complaints of SMTP syntax error (NUL chars) on the
+ receiving system, and refused recipients seen by the sending system
+ (propating to people being dropped from mailing lists).
+ Discovered and pinpointed by David Carter.
+
+JH/30 The (EXPERIMENTAL_DMARC) variable $dmarc_ar_header is withdrawn, being
+ replaced by the ${authresults } expansion.
+
+JH/31 Bug 2257: Fix pipe transport to not use a socket-only syscall.
+
+HS/03 Set a handler for SIGTERM and call exit(3) if running as PID 1. This
+ allows proper process termination in container environments.
+
+JH/32 Bug 2258: Fix spool_wireformat in combination with LMTP transport.
+ Previously the "final dot" had a newline after it; ensure it is CR,LF.
+
+JH/33 SPF: remove support for the "spf" ACL condition outcome values "err_temp"
+ and "err_perm", deprecated since 4.83 when the RFC-defined words
+ "temperror" and "permerror" were introduced.
+
+JH/34 Re-introduce enforcement of no cutthrough delivery on transports having
+ transport-filters or DKIM-signing. The restriction was lost in the
+ consolidation of verify-callout and delivery SMTP handling.
+ Extend the restriction to also cover ARC-signing.
+
+JH/35 Cutthrough: for a final-dot response timeout (and nonunderstood responses)
+ in defer=pass mode supply a 450 to the initiator. Previously the message
+ would be spooled.
+
+PP/02 DANE: add dane_require_tls_ciphers SMTP Transport option; if unset,
+ tls_require_ciphers is used as before.
+
+HS/03 Malware Avast: Better match the Avast multiline protocol. Add
+ "pass_unscanned". Only tmpfails from the scanner are written to
+ the paniclog, as they may require admin intervention (permission
+ denied, license issues). Other scanner errors (like decompression
+ bombs) do not cause a paniclog entry.
+
+JH/36 Fix reinitialisation of DKIM logging variable between messages.
+ Previously it was possible to log spurious information in receive log
+ lines.
+
+JH/37 Bug 2255: Revert the disable of the OpenSSL session caching. This
+ triggered odd behaviour from Outlook Express clients.
+
+PP/03 Add util/renew-opendmarc-tlds.sh script for safe renewal of public
+ suffix list.
+
+JH/38 DKIM: accept Ed25519 pubkeys in SubjectPublicKeyInfo-wrapped form,
+ since the IETF WG has not yet settled on that versus the original
+ "bare" representation.
+
+JH/39 Fix syslog logging for syslog_timestamp=no and log_selector +millisec.
+ Previously the millisecond value corrupted the output.
+ Fix also for syslog_pid=no and log_selector +pid, for which the pid
+ corrupted the output.
+
+
Exim version 4.90
-----------------
JH/10 Enable use of sendfile in Linux builds as default. It was disabled in
4.77 as the kernel support then wasn't solid, having issues in 64bit
- mode. Now, it's been long enough.
+ mode. Now, it's been long enough. Add support for FreeBSD also.
JH/11 Bug 2104: Fix continued use of a transport connection with TLS. In the
case where the routing stage had gathered several addresses to send to
which naturally failed, giving a failed delivery and bloating the retry
database. Investigation and fix prototype from Wolfgang Breyha.
+JH/12 Fix check on SMTP command input synchronisation. Previously there were
+ false-negatives in the check that the sender had not preempted a response
+ or prompt from Exim (running as a server), due to that code's lack of
+ awareness of the SMTP input buffering.
+
+PP/04 Add commandline_checks_require_admin option.
+ Exim drops privileges sanely, various checks such as -be aren't a
+ security problem, as long as you trust local users with access to their
+ own account. When invoked by services which pass untrusted data to
+ Exim, this might be an issue. Set this option in main configuration
+ AND make fixes to the calling application, such as using `--` to stop
+ processing options.
+
+JH/13 Do pipelining under TLS. Previously, although safe, no advantage was
+ taken. Now take care to pack both (client) MAIL,RCPT,DATA, and (server)
+ responses to those, into a single TLS record each way (this usually means
+ a single packet). As a side issue, smtp_enforce_sync now works on TLS
+ connections.
+
+PP/05 OpenSSL/1.1: use DH_bits() for more accurate DH param sizes. This
+ affects you only if you're dancing at the edge of the param size limits.
+ If you are, and this message makes sense to you, then: raise the
+ configured limit or use OpenSSL 1.1. Nothing we can do for older
+ versions.
+
+JH/14 For the "sock" variant of the malware scanner interface, accept an empty
+ cmdline element to get the documented default one. Previously it was
+ inaccessible.
+
+JH/15 Fix a crash in the smtp transport caused when two hosts in succession
+ are unsuable for non-message-specific reasons - eg. connection timeout,
+ banner-time rejection.
+
+JH/16 Fix logging of delivery remote port, when specified by router, under
+ callout/hold.
+
+PP/06 Repair manualroute's ability to take options in any order, even if one
+ is the name of a transport.
+ Fixes bug 2140.
+
+HS/01 Cleanup, prevent repeated use of -p/-oMr (CVE-2017-1000369)
+
+JH/17 Change the list-building routines interface to use the expanding-string
+ triplet model, for better allocation and copying behaviour.
+
+JH/18 Prebuild the data-structure for "builtin" macros, for faster startup.
+ Previously it was constructed the first time a possibly-matching string
+ was met in the configuration file input during startup; now it is done
+ during compilation.
+
+JH/19 Bug 2141: Use the full-complex API for Berkeley DB rather than the legacy-
+ compatible one, to avoid the (poorly documented) possibility of a config
+ file in the working directory redirecting the DB files, possibly correpting
+ some existing file. CVE-2017-10140 assigned for BDB.
+
+JH/20 Bug 2147: Do not defer for a verify-with-callout-and-random which is not
+ cache-hot. Previously, although the result was properly cached, the
+ initial verify call returned a defer.
+
+JH/21 Bug 2151: Avoid using SIZE on the MAIL for a callout verify, on any but
+ the main verify for receipient in uncached-mode.
+
+JH/22 Retire historical build files to an "unsupported" subdir. These are
+ defined as "ones for which we have no current evidence of testing".
+
+JH/23 DKIM: enforce the DNS pubkey record "h" permitted-hashes optional field,
+ if present. Previously it was ignored.
+
+JH/24 Start using specified-initialisers in C structure init coding. This is
+ a C99 feature (it's 2017, so now considered safe).
+
+JH/25 Use one-bit bitfields for flags in the "addr" data structure. Previously
+ if was a fixed-sized field and bitmask ops via macros; it is now more
+ extensible.
+
+PP/07 GitHub PR 56: Apply MariaDB build fix.
+ Patch provided by Jaroslav Škarvada.
+
+PP/08 Bug 2161: Fix regression in sieve quoted-printable handling introduced
+ during Coverity cleanups [4.87 JH/47]
+ Diagnosis and fix provided by Michael Fischer v. Mollard.
+
+JH/26 Fix DKIM bug: when the pseudoheader generated for signing was exactly
+ the right size to place the terminating semicolon on its own folded
+ line, the header hash was calculated to an incorrect value thanks to
+ the (relaxed) space the fold became.
+
+HS/02 Fix Bug 2130: large writes from the transport subprocess were chunked
+ and confused the parent.
+
+JH/27 Fix SOCKS bug: an unitialized pointer was deref'd by the transport process
+ which could crash as a result. This could lead to undeliverable messages.
+
+JH/28 Logging: "next input sent too soon" now shows where input was truncated
+ for log purposes.
+
+JH/29 Fix queue_run_in_order to ignore the PID portion of the message ID. This
+ matters on fast-turnover and PID-randomising systems, which were getting
+ out-of-order delivery.
+
+JH/30 Fix a logging bug on aarch64: an unsafe routine was previously used for
+ a possibly-overlapping copy. The symptom was that "Remote host closed
+ connection in response to HELO" was logged instead of the actual 4xx
+ error for the HELO.
+
+JH/31 Fix CHUNKING code to properly flush the unwanted chunk after an error.
+ Previously only that bufferd was discarded, resulting in SYMTP command
+ desynchronisation.
+
+JH/32 DKIM: when a message has multiple signatures matching an identity given
+ in dkim_verify_signers, run the dkim acl once for each. Previously only
+ one run was done. Bug 2189.
+
+JH/33 Downgrade an unfound-list name (usually a typo in the config file) from
+ "panic the current process" to "deliberately defer". The panic log is
+ still written with the problem list name; the mail and reject logs now
+ get a temp-reject line for the message that was being handled, saying
+ something like "domains check lookup or other defer". The SMTP 451
+ message is still "Temporary local problem".
+
+JH/34 Bug 2199: Fix a use-after-free while reading smtp input for header lines.
+ A crafted sequence of BDAT commands could result in in-use memory beeing
+ freed. CVE-2017-16943.
+
+HS/03 Bug 2201: Fix checking for leading-dot on a line during headers reading
+ from SMTP input. Previously it was always done; now only done for DATA
+ and not BDAT commands. CVE-2017-16944.
+
+JH/35 Bug 2201: Flush received data in BDAT mode after detecting an error fatal
+ to the message (such as an overlong header line). Previously this was
+ not done and we did not exit BDAT mode. Followon from the previous item
+ though a different problem.
+
Exim version 4.89
-----------------
git://git.exim.org / users / heiko / exim.git / blobdiff