From: Jeremy Harris Date: Mon, 2 Oct 2023 12:24:29 +0000 (+0100) Subject: Merge branch 'exim-4.96+security' X-Git-Tag: exim-4.97-RC2~6 X-Git-Url: https://git.exim.org/exim.git/commitdiff_plain/18eacc1185bea956bd9c793ab34e582c08941799?hp=-c Merge branch 'exim-4.96+security' --- 18eacc1185bea956bd9c793ab34e582c08941799 diff --combined doc/doc-txt/ChangeLog index efa79de22,8c5b7fd9c..a78ec386f --- a/doc/doc-txt/ChangeLog +++ b/doc/doc-txt/ChangeLog @@@ -2,202 -2,28 +2,214 @@@ This document describes *changes* to pr affect Exim's operation, with an unchanged configuration file. For new options, and new features, see the NewStuff file next to this ChangeLog. -Exim version 4.96.1 -------------------- +Exim version 4.97 +----------------- + +JH/01 The hosts_connection_nolog main option now also controls "no MAIL in + SMTP connection" log lines. + +JH/02 Option default value updates: + - queue_fast_ramp (main) true (was false) + - remote_max_parallel (main) 4 (was 2) + +JH/03 Cache static regex pattern compilations, for use by ACLs. + +JH/04 Bug 2903: avoid exit on an attempt to rewrite a malformed address. + Make the rewrite never match and keep the logging. Trust the + admin to be using verify=header-syntax (to actually reject the message). + +JH/05 Follow symlinks for placing a watch on TLS creds files. This means + (under Linux) we watch the dir containing the final file; previously + it would be the dir with the first symlink. We still do not monitor + the entire path. + +JH/06 Check for bad chars in rDNS for sender_host_name. The OpenBSD (at least) + dn_expand() is happy to pass them through. + +JH/07 OpenSSL Fix auto-reload of changed server OCSP proof. Previously, if + the file with the proof had an unchanged name, the new proof(s) were + loaded on top of the old ones (and nover used; the old ones were stapled). + +JH/08 Bug 2915: Fix use-after-free for $regex variables. Previously when + more than one message arrived in a single connection a reference from + the earlier message could be re-used. Often a sigsegv resulted. + These variables were introduced in Exim 4.87. + Debug help from Graeme Fowler. + +JH/09 Fix ${filter } for conditions that modify $value. Previously the + modified version would be used in construction the result, and a memory + error would occur. + +JH/10 GnuTLS: fix for (IOT?) clients offering no TLS extensions at all. + Find and fix by Jasen Betts. + +JH/11 OpenSSL: fix for ancient clients needing TLS support for versions earlier + than TLSv1,2, Previously, more-recent versions of OpenSSL were permitting + the systemwide configuration to override the Exim config. + +HS/01 Bug 2728: Introduce EDITME option "DMARC_API" to work around incompatible + API changes in libopendmarc. + +JH/12 Bug 2930: Fix daemon startup. When started from any process apart from + pid 1, in the normal "background daemon" mode, having to drop process- + group leadership also lost track of needing to create listener sockets. + +JH/13 Bug 2929: Fix using $recipients after ${run...}. A change made for 4.96 + resulted in the variable appearing empty. Find and fix by Ruben Jenster. + +JH/14 Bug 2933: Fix regex substring match variables for null matches. Since 4.96 + a capture group which obtained no text (eg. "(abc)*" matching zero + occurrences) could cause a segfault if the corresponding $ was + expanded. + +JH/15 Fix argument parsing for ${run } expansion. Previously, when an argument + included a close-brace character (eg. it itself used an expansion) an + error occurred. + +JH/16 Move running the smtp connect ACL to before, for TLS-on-connect ports, + starting TLS. Previously it was after, meaning that attackers on such + ports had to be screened using the host_reject_connection main config + option. The new sequence aligns better with the STARTTLS behaviour, and + permits defences against crypto-processing load attacks, even though it + is strictly an incompatible change. + Also, avoid sending any SMTP fail response for either the connect ACL + or host_reject_connection, for TLS-on-connect ports. + +JH/17 Permit the ACL "encrypted" condition to be used in a HELO/EHLO ACL, + Previously this was not permitted, but it makes reasonable sense. + While there, restore a restriction on using it from a connect ACL; given + the change JH/16 it could only return false (and before 4.91 was not + permitted). + +JH/18 Fix a fencepost error in logging. Previously (since 4.92) when a log line + was exactly sized compared to the log buffer, a crash occurred with the + misleading message "bad memory reference; pool not found". + Found and traced by Jasen Betts. + +JH/19 Bug 2911: Fix a recursion in DNS lookups. Previously, if the main option + dns_again_means_nonexist included an element causing a DNS lookup which + itself returned DNS_AGAIN, unbounded recursion occurred. Possible results + included (though probably not limited to) a process crash from stack + memory limit, or from excessive open files. Replace this with a paniclog + whine (as this is likely a configuration error), and returning + DNS_NOMATCH. + +JH/20 Bug 2954: (OpenSSL) Fix setting of explicit EC curve/group. Previously + this always failed, probably leading to the usual downgrade to in-clear + connections. + +JH/21 Fix TLSA lookups. Previously dns_again_means_nonexist would affect + SERVFAIL results, which breaks the downgrade resistance of DANE. Change + to not checking that list for these lookups. + +JH/22 Bug 2434: Add connection-elapsed "D=" element to more connection + closure log lines. + +JH/23 Fix crash in string expansions. Previously, if an empty variable was + immediately followed by an expansion operator, a null-indirection read + was done, killing the process. + +JH/24 Bug 2997: When built with EXPERIMENTAL_DSN_INFO, bounce messages can + include an SMTP response string which is longer than that supported + by the delivering transport. Alleviate by wrapping such lines before + column 80. + +JH/25 Bug 2827: Restrict size of References: header in bounce messages to 998 + chars (RFC limit). Previously a limit of 12 items was made, which with + a not-impossible References: in the message being bounced could still + be over-large and get stopped in the transport. + +JH/26 For a ${readsocket } in TLS mode, send a TLS Close Alert before the TCP + close. Previously a bare socket close was done. + +JH/27 Fix ${srs_encode ..}. Previously it would give a bad result for one day + every 1024 days. + +JH/28 Bug 2996: Fix a crash in the smtp transport. When finding that the + message being considered for delivery was already being handled by + another process, and having an SMTP connection already open, the function + to close it tried to use an uninitialized variable. This would afftect + high-volume sites more, especially when running mailing-list-style loads. + Pollution of logs was the major effect, as the other process delivered + the message. Found and partly investigated by Graeme Fowler. + +JH/29 Change format of the internal ID used for message identification. The old + version only supported 31 bits for a PID element; the new 64 (on systems + which can use Base-62 encoding, which is all currently supported ones + but not Darwin (MacOS) or Cygwin, which have case-insensitive filesystems + and must use Base-36). The new ID is 23 characters rather than 16, and is + visible in various places - notably logs, message headers, and spool file + names. Various of the ancillary utilities also have to know the format. + As well as the expanded PID portion, the sub-second part of the time + recorded in the ID is expanded to support finer precision. Theoretically + this permits a receive rate from a single comms channel of better than the + previous 2000/sec. + The major timestamp part of the ID is not changed; at 6 characters it is + usable until about year 3700. + Updating from previously releases is fully supported: old-format spool + files are still usable, and the utilities support both formats. New + message will use the new format. The one hints-DB file type which uses + message-IDs (the transport wait- DB) will be discarded if an old-format ID + is seen; new ones will be built with only new-format IDs. + Optionally, a utility can be used to convert spool files from old to new, + but this is only an efficiency measure not a requirement for operation + Downgrading from new to old requires running a provided utility, having + first stopped all operations. This will convert any spool files from new + back to old (losing time-precision and PID information) and remove any + wait- hints databases. + +JH/30 Bug 3006: Fix handling of JSON strings having embedded commas. Previously + we treated them as item separators when parsing for a list item, but they + need to be protected by the doublequotes. While there, add handling for + backslashes. + +JH/31 Bug 2998: Fix ${utf8clean:...} to disallow UTF-16 surrogate codepoints. + Found and fixed by Jasen Betts. No testcase for this as my usual text + editor insists on emitting only valid UTF-8. + +JH/32 Fix "tls_dhparam = none" under GnuTLS. At least with 3.7.9 this gave + a null-indirection SIGSEGV for the receive process. + +JH/33 Fix free for live variable $value created by a ${run ...} expansion during + -bh use. Internal checking would spot this and take a panic. + +JH/34 Bug 3013: Fix use of $recipients within arguments for ${run...}. + In 4.96 this would expand to empty. + +JH/35 Bug 3014: GnuTLS: fix expiry date for an auto-generated server + certificate. Find and fix by Andreas Metzler. + +JH/36 Add ARC info to DMARC hostory records. + +JH/37 Bug 3016: Avoid sending DSN when message was accepted under fakereject + or fakedefer. Previously the sender could discover that the message + had in fact been accepted. + +JH/38 Taint-track intermediate values from the peer in multi-stage authentation + sequences. Previously the input was not noted as being tainted; notably + this resulted in behaviour of LOGIN vs. PLAIN being inconsistent under + bad coding of authenticators. -This is a security release. +JH/39 Bug 3023: Fix crash induced by some combinations of zero-length strings + and ${tr...}. Found and diagnosed by Heiko Schlichting. -JH/01 Bug 2999: Fix a possible OOB write in the external authenticator, which ++JH/40 Bug 2999: Fix a possible OOB write in the external authenticator, which + could be triggered by externally-supplied input. Found by Trend Micro. + CVE-2023-42115 + -JH/02 Bug 3000: Fix a possible OOB write in the SPA authenticator, which could ++JH/41 Bug 3000: Fix a possible OOB write in the SPA authenticator, which could + be triggered by externally-controlled input. Found by Trend Micro. + CVE-2023-42116 + -JH/03 Bug 3001: Fix a possible OOB read in the SPA authenticator, which could ++JH/42 Bug 3001: Fix a possible OOB read in the SPA authenticator, which could + be triggered by externally-controlled input. Found by Trend Micro. + CVE-2023-42114 + Exim version 4.96 ----------------- -JH/01 Move the wait-for-next-tick (needed for unique messmage IDs) from +JH/01 Move the wait-for-next-tick (needed for unique message IDs) from after reception to before a subsequent reception. This should mean slightly faster delivery, and also confirmation of reception to senders. diff --combined src/src/auths/auth-spa.c index bcf88c84d,bb3d327d1..8e3146b35 --- a/src/src/auths/auth-spa.c +++ b/src/src/auths/auth-spa.c @@@ -10,7 -10,6 +10,7 @@@ * Samba project (by Andrew Tridgell, Jeremy Allison, and others). * * Copyright (c) The Exim Maintainers 2021 + * SPDX-License-Identifier: GPL-2.0-or-later * Tom Kistner provided additional code, adding spa_build_auth_challenge() to * support server authentication mode. @@@ -156,6 -155,9 +156,9 @@@ int main (int argc, char ** argv up with a different answer to the one above) */ + #ifndef MACRO_PREDEF + + #define DEBUG_X(a,b) ; extern int DEBUGLEVEL; @@@ -1212,7 -1214,9 +1215,9 @@@ char versionString[] = "libntlm versio #define spa_bytes_add(ptr, header, buf, count) \ { \ - if (buf && (count) != 0) /* we hate -Wint-in-bool-contex */ \ + if ( buf && (count) != 0 /* we hate -Wint-in-bool-contex */ \ + && ptr->bufIndex + count < sizeof(ptr->buffer) \ + ) \ { \ SSVAL(&ptr->header.len,0,count); \ SSVAL(&ptr->header.maxlen,0,count); \ @@@ -1230,35 -1234,30 +1235,30 @@@ else #define spa_string_add(ptr, header, string) \ { \ - char *p = string; \ + uschar * p = string; \ int len = 0; \ - if (p) len = strlen(p); \ - spa_bytes_add(ptr, header, (US p), len); \ + if (p) len = Ustrlen(p); \ + spa_bytes_add(ptr, header, p, len); \ } #define spa_unicode_add_string(ptr, header, string) \ { \ - char *p = string; \ - uschar *b = NULL; \ + uschar * p = string; \ + uschar * b = NULL; \ int len = 0; \ if (p) \ { \ - len = strlen(p); \ - b = strToUnicode(p); \ + len = Ustrlen(p); \ + b = US strToUnicode(CS p); \ } \ spa_bytes_add(ptr, header, b, len*2); \ } - #define GetUnicodeString(structPtr, header) \ - unicodeToString(((char*)structPtr) + IVAL(&structPtr->header.offset,0) , SVAL(&structPtr->header.len,0)/2) - #define GetString(structPtr, header) \ - toString(((CS structPtr) + IVAL(&structPtr->header.offset,0)), SVAL(&structPtr->header.len,0)) - #ifdef notdef #define DumpBuffer(fp, structPtr, header) \ - dumpRaw(fp,(US structPtr)+IVAL(&structPtr->header.offset,0),SVAL(&structPtr->header.len,0)) + dumpRaw(fp,(US structPtr)+IVAL(&structPtr->header.offset,0),SVAL(&structPtr->header.len,0)) static void @@@ -1322,8 -1321,33 +1322,33 @@@ buf[len] = 0 return buf; } + static inline uschar * + get_challenge_unistr(SPAAuthChallenge * challenge, SPAStrHeader * hdr) + { + int off = IVAL(&hdr->offset, 0); + int len = SVAL(&hdr->len, 0); + return off + len < sizeof(SPAAuthChallenge) + ? US unicodeToString(CS challenge + off, len/2) : US""; + } + + static inline uschar * + get_challenge_str(SPAAuthChallenge * challenge, SPAStrHeader * hdr) + { + int off = IVAL(&hdr->offset, 0); + int len = SVAL(&hdr->len, 0); + return off + len < sizeof(SPAAuthChallenge) + ? US toString(CS challenge + off, len) : US""; + } + #ifdef notdef + #define GetUnicodeString(structPtr, header) \ + unicodeToString(((char*)structPtr) + IVAL(&structPtr->header.offset,0) , SVAL(&structPtr->header.len,0)/2) + + #define GetString(structPtr, header) \ + toString(((CS structPtr) + IVAL(&structPtr->header.offset,0)), SVAL(&structPtr->header.len,0)) + + void dumpSmbNtlmAuthRequest (FILE * fp, SPAAuthRequest * request) { @@@ -1367,15 -1391,15 +1392,15 @@@ fprintf (fp, " Flags = %08x\n", IV #endif void - spa_build_auth_request (SPAAuthRequest * request, char *user, char *domain) + spa_build_auth_request (SPAAuthRequest * request, uschar * user, uschar * domain) { - char *u = strdup (user); - char *p = strchr (u, '@'); + uschar * u = string_copy(user); + uschar * p = Ustrchr(u, '@'); if (p) { if (!domain) - domain = p + 1; + domain = p + 1; *p = '\0'; } @@@ -1385,7 -1409,6 +1410,6 @@@ SIVAL (&request->msgType, 0, 1) SIVAL (&request->flags, 0, 0x0000b207); /* have to figure out what these mean */ spa_string_add (request, user, u); spa_string_add (request, domain, domain); - free (u); } @@@ -1476,16 -1499,16 +1500,16 @@@ free (u) void spa_build_auth_response (SPAAuthChallenge * challenge, - SPAAuthResponse * response, char *user, - char *password) + SPAAuthResponse * response, uschar * user, + uschar * password) { uint8x lmRespData[24]; uint8x ntRespData[24]; uint32x cf = IVAL(&challenge->flags, 0); - char *u = strdup (user); - char *p = strchr (u, '@'); - char *d = NULL; - char *domain; + uschar * u = string_copy(user); + uschar * p = Ustrchr(u, '@'); + uschar * d = NULL; + uschar * domain; if (p) { @@@ -1493,33 -1516,33 +1517,33 @@@ *p = '\0'; } - else domain = d = strdup((cf & 0x1)? - CCS GetUnicodeString(challenge, uDomain) : - CCS GetString(challenge, uDomain)); + else domain = d = string_copy(cf & 0x1 + ? CUS get_challenge_unistr(challenge, &challenge->uDomain) + : CUS get_challenge_str(challenge, &challenge->uDomain)); - spa_smb_encrypt (US password, challenge->challengeData, lmRespData); - spa_smb_nt_encrypt (US password, challenge->challengeData, ntRespData); + spa_smb_encrypt(password, challenge->challengeData, lmRespData); + spa_smb_nt_encrypt(password, challenge->challengeData, ntRespData); response->bufIndex = 0; memcpy (response->ident, "NTLMSSP\0\0\0", 8); SIVAL (&response->msgType, 0, 3); - spa_bytes_add (response, lmResponse, lmRespData, (cf & 0x200) ? 24 : 0); - spa_bytes_add (response, ntResponse, ntRespData, (cf & 0x8000) ? 24 : 0); + spa_bytes_add(response, lmResponse, lmRespData, cf & 0x200 ? 24 : 0); + spa_bytes_add(response, ntResponse, ntRespData, cf & 0x8000 ? 24 : 0); if (cf & 0x1) { /* Unicode Text */ - spa_unicode_add_string (response, uDomain, domain); - spa_unicode_add_string (response, uUser, u); - spa_unicode_add_string (response, uWks, u); + spa_unicode_add_string(response, uDomain, domain); + spa_unicode_add_string(response, uUser, u); + spa_unicode_add_string(response, uWks, u); } else { /* OEM Text */ - spa_string_add (response, uDomain, domain); - spa_string_add (response, uUser, u); - spa_string_add (response, uWks, u); + spa_string_add(response, uDomain, domain); + spa_string_add(response, uUser, u); + spa_string_add(response, uWks, u); } - spa_string_add (response, sessionKey, NULL); + spa_string_add(response, sessionKey, NULL); response->flags = challenge->flags; - - if (d != NULL) free (d); - free (u); } + + + #endif /*!MACRO_PREDEF*/ diff --combined src/src/auths/auth-spa.h index db93891ab,3b0b3a9e3..64874ca66 --- a/src/src/auths/auth-spa.h +++ b/src/src/auths/auth-spa.h @@@ -9,7 -9,6 +9,7 @@@ * All the code used here was torn by Marc Prud'hommeaux out of the * Samba project (by Andrew Tridgell, Jeremy Allison, and others). */ +/* SPDX-License-Identifier: GPL-2.0-or-later */ /* December 2004: The spa_base64_to_bits() function has no length checking in it. I have added a check. PH */ @@@ -80,10 -79,10 +80,10 @@@ typedef struc void spa_bits_to_base64 (unsigned char *, const unsigned char *, int); int spa_base64_to_bits(char *, int, const char *); - void spa_build_auth_response (SPAAuthChallenge *challenge, - SPAAuthResponse *response, char *user, char *password); - void spa_build_auth_request (SPAAuthRequest *request, char *user, - char *domain); + void spa_build_auth_response (SPAAuthChallenge * challenge, + SPAAuthResponse * response, uschar * user, uschar * password); + void spa_build_auth_request (SPAAuthRequest * request, uschar * user, + uschar * domain); extern void spa_smb_encrypt (unsigned char * passwd, unsigned char * c8, unsigned char * p24); extern void spa_smb_nt_encrypt (unsigned char * passwd, unsigned char * c8, diff --combined src/src/auths/external.c index 078aad0fa,790b98159..f6aa527f2 --- a/src/src/auths/external.c +++ b/src/src/auths/external.c @@@ -4,7 -4,6 +4,7 @@@ /* Copyright (c) Jeremy Harris 2019-2020 */ /* See the file NOTICE for conditions of use and distribution. */ +/* SPDX-License-Identifier: GPL-2.0-or-later */ /* This file provides an Exim authenticator driver for a server to verify a client SSL certificate, using the EXTERNAL @@@ -104,7 -103,7 +104,7 @@@ if (expand_nmax == 0) /* skip if rxd d if (ob->server_param2) { uschar * s = expand_string(ob->server_param2); - auth_vars[expand_nmax] = s; + auth_vars[expand_nmax = 1] = s; expand_nstring[++expand_nmax] = s; expand_nlength[expand_nmax] = Ustrlen(s); if (ob->server_param3) diff --combined src/src/auths/spa.c index 222ccea86,bfaccefda..3cea845b8 --- a/src/src/auths/spa.c +++ b/src/src/auths/spa.c @@@ -5,7 -5,6 +5,7 @@@ /* Copyright (c) University of Cambridge 1995 - 2018 */ /* Copyright (c) The Exim Maintainers 2020 */ /* See the file NOTICE for conditions of use and distribution. */ +/* SPDX-License-Identifier: GPL-2.0-or-later */ /* This file, which provides support for Microsoft's Secure Password Authentication, was contributed by Marc Prud'hommeaux. Tom Kistner added SPA @@@ -285,14 -284,13 +285,13 @@@ SPAAuthRequest request SPAAuthChallenge challenge; SPAAuthResponse response; char msgbuf[2048]; - char *domain = NULL; - char *username, *password; + uschar * domain = NULL, * username, * password; /* Code added by PH to expand the options */ *buffer = 0; /* Default no message when cancelled */ - if (!(username = CS expand_string(ob->spa_username))) + if (!(username = expand_string(ob->spa_username))) { if (f.expand_string_forcedfail) return CANCELLED; string_format(buffer, buffsize, "expansion of \"%s\" failed in %s " @@@ -301,7 -299,7 +300,7 @@@ return ERROR; } - if (!(password = CS expand_string(ob->spa_password))) + if (!(password = expand_string(ob->spa_password))) { if (f.expand_string_forcedfail) return CANCELLED; string_format(buffer, buffsize, "expansion of \"%s\" failed in %s " @@@ -311,7 -309,7 +310,7 @@@ } if (ob->spa_domain) - if (!(domain = CS expand_string(ob->spa_domain))) + if (!(domain = expand_string(ob->spa_domain))) { if (f.expand_string_forcedfail) return CANCELLED; string_format(buffer, buffsize, "expansion of \"%s\" failed in %s " @@@ -331,7 -329,7 +330,7 @@@ if (!smtp_read_response(sx, US buffer, DSPA("\n\n%s authenticator: using domain %s\n\n", ablock->name, domain); - spa_build_auth_request(&request, CS username, domain); + spa_build_auth_request(&request, username, domain); spa_bits_to_base64(US msgbuf, US &request, spa_request_length(&request)); DSPA("\n\n%s authenticator: sending request (%s)\n\n", ablock->name, msgbuf); @@@ -348,7 -346,7 +347,7 @@@ if (!smtp_read_response(sx, US buffer, DSPA("\n\n%s authenticator: challenge (%s)\n\n", ablock->name, buffer + 4); spa_base64_to_bits(CS (&challenge), sizeof(challenge), CCS (buffer + 4)); - spa_build_auth_response(&challenge, &response, CS username, CS password); + spa_build_auth_response(&challenge, &response, username, password); spa_bits_to_base64(US msgbuf, US &response, spa_request_length(&response)); DSPA("\n\n%s authenticator: challenge response (%s)\n\n", ablock->name, msgbuf);