From: Jeremy Harris Date: Sat, 3 Oct 2020 19:59:15 +0000 (+0100) Subject: TLS: preload configuration items X-Git-Url: https://git.exim.org/users/heiko/exim.git/commitdiff_plain/6a9cf7f890226aa085842cd3d94b13e78ea31637 TLS: preload configuration items --- diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt index 2e4df803f..d0c3e7846 100644 --- a/doc/doc-docbook/spec.xfpt +++ b/doc/doc-docbook/spec.xfpt @@ -29271,6 +29271,61 @@ There is no current way to staple a proof for a client certificate. .endd +.new +.section "Caching of static server configuration items" "SECTserverTLScache" +.cindex certificate caching +.cindex privatekey caching +.cindex crl caching +.cindex ocsp caching +.cindex ciphers caching +.cindex "CA bundle" caching +.cindex "certificate authorities" caching +.cindex tls_certificate caching +.cindex tls_privatekey caching +.cindex tls_crl caching +.cindex tls_ocsp_file caching +.cindex tls_require_ciphers caching +.cindex tls_verify_certificate caching +.cindex caching certificate +.cindex caching privatekey +.cindex caching crl +.cindex caching ocsp +.cindex caching ciphers +.cindex caching "certificate authorities +If any of the main configuration options &%tls_certificate%&, &%tls_privatekey%&, +&%tls_crl%& and &%tls_ocsp_file%& have values with no +expandable elements, +then the associated information is loaded at daemon startup. +It is made available +to child processes forked for handling received SMTP connections. + +This caching is currently only supported under Linux. + +If caching is not possible, for example if an item has to be dependent +on the peer host so contains a &$sender_host_name$& expansion, the load +of the associated information is done at the startup of the TLS connection. + +The cache is invalidated and reloaded after any changes to the directories +containing files specified by these options. + +The information specified by the main option &%tls_verify_certificates%& +is similarly cached so long as it specifies files explicitly +or (under GnuTLS) is the string &"system,cache"&. +The latter case is not automatically invaludated; +it is the operator's responsibility to arrange for a daemon restart +any time the system certificate authority bundle is updated. +A HUP signal is sufficient for this. +The value &"system"& results in no caching under GnuTLS. + +The macro _HAVE_TLS_CA_CACHE will be defined if the suffix for "system" +is acceptable in configurations for the Exim executavble. + +Caching of the system Certificate Authorities bundle can +save siginificant time and processing on every TLS connection +accepted by Exim. +.wen + + .section "Configuring an Exim client to use TLS" "SECTclientTLS" @@ -29311,7 +29366,10 @@ unencrypted. The &%tls_certificate%& and &%tls_privatekey%& options of the &(smtp)& transport provide the client with a certificate, which is passed to the server -if it requests it. If the server is Exim, it will request a certificate only if +if it requests it. +This is an optional thing for TLS connections, although either end +may insist on it. +If the server is Exim, it will request a certificate only if &%tls_verify_hosts%& or &%tls_try_verify_hosts%& matches the client. &*Note*&: Do not use a certificate which has the OCSP-must-staple extension, @@ -29391,6 +29449,62 @@ outgoing connection. +.new +.section "Caching of static client configuration items" "SECTclientTLScache" +.cindex certificate caching +.cindex privatekey caching +.cindex crl caching +.cindex ciphers caching +.cindex "CA bundle" caching +.cindex "certificate authorities" caching +.cindex tls_certificate caching +.cindex tls_privatekey caching +.cindex tls_crl caching +.cindex tls_require_ciphers caching +.cindex tls_verify_certificate caching +.cindex caching certificate +.cindex caching privatekey +.cindex caching crl +.cindex caching ciphers +.cindex caching "certificate authorities +If any of the transport configuration options &%tls_certificate%&, &%tls_privatekey%& +and &%tls_crl%& have values with no +expandable elements, +then the associated information is loaded per smtp transport +at daemon startup, at the start of a queue run, or on a +command-line specified message delivery. +It is made available +to child processes forked for handling making SMTP connections. + +This caching is currently only supported under Linux. + +If caching is not possible, the load +of the associated information is done at the startup of the TLS connection. + +The cache is invalidated in the daemon +and reloaded after any changes to the directories +containing files specified by these options. + +The information specified by the main option &%tls_verify_certificates%& +is similarly cached so long as it specifies files explicitly +or (under GnuTLS) is the string &"system,cache"&. +The latter case is not automatically invaludated; +it is the operator's responsibility to arrange for a daemon restart +any time the system certificate authority bundle is updated. +A HUP signal is sufficient for this. +The value &"system"& results in no caching under GnuTLS. + +The macro _HAVE_TLS_CA_CACHE will be defined if the suffix for "system" +is acceptable in configurations for the Exim executavble. + +Caching of the system Certificate Authorities bundle can +save siginificant time and processing on every TLS connection +initiated by Exim. +.wen + + + + .section "Use of TLS Server Name Indication" "SECTtlssni" .cindex "TLS" "Server Name Indication" .cindex "TLS" SNI diff --git a/doc/doc-txt/NewStuff b/doc/doc-txt/NewStuff index abbcf4c6c..acbbc15fd 100644 --- a/doc/doc-txt/NewStuff +++ b/doc/doc-txt/NewStuff @@ -38,6 +38,10 @@ Version 4.95 10. A command-line option to have a daemon not create a notifier socket. +11. Faster TLS startup. When various configuration options contain no + expandable elements, the information can be preloaded and cached rather + than the provious behaviour of always loading at startup time for every + connection. This helps particularly for the CA bundle. Version 4.94 ------------ diff --git a/src/OS/os.h-Linux b/src/OS/os.h-Linux index 4a141346d..2fa1b0b82 100644 --- a/src/OS/os.h-Linux +++ b/src/OS/os.h-Linux @@ -91,5 +91,8 @@ then change the 0 to 1 in the next block. */ /* "Abstract" Unix-socket names */ #define EXIM_HAVE_ABSTRACT_UNIX_SOCKETS +/* inotify(7) etc syscalls */ +#define EXIM_HAVE_INOTIFY + /* End */ diff --git a/src/src/daemon.c b/src/src/daemon.c index f56e36a10..4e90799e6 100644 --- a/src/src/daemon.c +++ b/src/src/daemon.c @@ -963,6 +963,11 @@ daemon_die(void) { int pid; +#ifndef DISABLE_TLS +if (tls_watch_fd >= 0) + { close(tls_watch_fd); tls_watch_fd = -1; } +#endif + if (daemon_notifier_fd >= 0) { close(daemon_notifier_fd); @@ -2039,6 +2044,9 @@ malware_init(); #ifdef SUPPORT_SPF spf_init(); #endif +#ifndef DISABLE_TLS +tls_daemon_init(); +#endif /* Close the log so it can be renamed and moved. In the few cases below where this long-running process writes to the log (always exceptional conditions), it @@ -2277,8 +2285,18 @@ for (;;) fd_set select_listen; FD_ZERO(&select_listen); +#ifndef DISABLE_TLS + if (tls_watch_fd >= 0) + { + FD_SET(tls_watch_fd, &select_listen); + if (tls_watch_fd > max_socket) max_socket = tls_watch_fd; + } +#endif if (daemon_notifier_fd >= 0) + { FD_SET(daemon_notifier_fd, &select_listen); + if (daemon_notifier_fd > max_socket) max_socket = daemon_notifier_fd; + } for (int sk = 0; sk < listen_socket_count; sk++) { FD_SET(listen_sockets[sk], &select_listen); @@ -2321,8 +2339,8 @@ for (;;) errno = select_errno; #ifndef DISABLE_TLS - /* Create or rotate any required keys */ - tls_daemon_init(); + /* Create or rotate any required keys; handle (delayed) filewatch event */ + tls_daemon_tick(); #endif /* Loop for all the sockets that are currently ready to go. If select @@ -2335,6 +2353,15 @@ for (;;) if (!select_failed) { +#if defined(EXIM_HAVE_INOTIFY) && !defined(DISABLE_TLS) + if (tls_watch_fd >= 0 && FD_ISSET(tls_watch_fd, &select_listen)) + { + FD_CLR(tls_watch_fd, &select_listen); + tls_watch_trigger_time = time(NULL); /* Set up delayed event */ + (void) read(tls_watch_fd, big_buffer, big_buffer_size); + break; /* to top of daemon loop */ + } +#endif if ( daemon_notifier_fd >= 0 && FD_ISSET(daemon_notifier_fd, &select_listen)) { diff --git a/src/src/exim.h b/src/src/exim.h index 1ddba187b..6669e809e 100644 --- a/src/src/exim.h +++ b/src/src/exim.h @@ -87,6 +87,10 @@ making unique names. */ # include #endif +#ifdef EXIM_HAVE_INOTIFY +# include +#endif + /* C99 integer types, figure out how to undo this if needed for older systems */ #include diff --git a/src/src/functions.h b/src/src/functions.h index cc9137c8b..c69851962 100644 --- a/src/src/functions.h +++ b/src/src/functions.h @@ -53,10 +53,12 @@ extern uschar * tls_cert_fprt_sha256(void *); extern void tls_clean_env(void); extern BOOL tls_client_start(client_conn_ctx *, smtp_connect_args *, void *, tls_support *, uschar **); +extern void tls_client_creds_reload(BOOL); extern void tls_close(void *, int); extern BOOL tls_could_read(void); extern void tls_daemon_init(void); +extern void tls_daemon_tick(void); extern BOOL tls_dropprivs_validate_require_cipher(BOOL); extern BOOL tls_export_cert(uschar *, size_t, void *); extern int tls_feof(void); @@ -67,7 +69,7 @@ extern uschar *tls_getbuf(unsigned *); extern void tls_get_cache(void); extern BOOL tls_import_cert(const uschar *, void **); extern int tls_read(void *, uschar *, size_t); -extern int tls_server_start(const uschar *, uschar **); +extern int tls_server_start(uschar **); extern BOOL tls_smtp_buffered(void); extern int tls_ungetc(int); extern int tls_write(void *, const uschar *, size_t, BOOL); diff --git a/src/src/globals.c b/src/src/globals.c index fb0abb8fc..240c2eb80 100644 --- a/src/src/globals.c +++ b/src/src/globals.c @@ -143,6 +143,8 @@ uschar *tls_resumption_hosts = NULL; uschar *tls_try_verify_hosts = NULL; uschar *tls_verify_certificates= US"system"; uschar *tls_verify_hosts = NULL; +int tls_watch_fd = -1; +time_t tls_watch_trigger_time = (time_t)0; #else /*DISABLE_TLS*/ uschar *tls_advertise_hosts = NULL; #endif @@ -1560,60 +1562,16 @@ struct timeval timestamp_startup; transport_instance *transports = NULL; transport_instance transport_defaults = { - .next = NULL, - .name = NULL, - .info = NULL, - .options_block = NULL, - .driver_name = NULL, - .setup = NULL, + /* All non-mentioned elements zero/NULL/FALSE */ .batch_max = 1, - .batch_id = NULL, - .home_dir = NULL, - .current_dir = NULL, - .expand_multi_domain = NULL, .multi_domain = TRUE, - .overrides_hosts = FALSE, .max_addresses = 100, .connection_max_messages = 500, - .deliver_as_creator = FALSE, - .disable_logging = FALSE, - .initgroups = FALSE, - .uid_set = FALSE, - .gid_set = FALSE, .uid = (uid_t)(-1), .gid = (gid_t)(-1), - .expand_uid = NULL, - .expand_gid = NULL, - .warn_message = NULL, - .shadow = NULL, - .shadow_condition = NULL, - .filter_command = NULL, - .add_headers = NULL, - .remove_headers = NULL, - .return_path = NULL, - .debug_string = NULL, - .max_parallel = NULL, - .message_size_limit = NULL, - .headers_rewrite = NULL, - .rewrite_rules = NULL, - .rewrite_existflags = 0, .filter_timeout = 300, - .body_only = FALSE, - .delivery_date_add = FALSE, - .envelope_to_add = FALSE, - .headers_only = FALSE, - .rcpt_include_affixes = FALSE, - .return_path_add = FALSE, - .return_output = FALSE, - .return_fail_output = FALSE, - .log_output = FALSE, - .log_fail_output = FALSE, - .log_defer_output = FALSE, .retry_use_local_part = TRUE_UNSET, /* retry_use_local_part: BOOL, but set neither 1 nor 0 so can detect unset */ -#ifndef DISABLE_EVENT - .event_action = NULL -#endif }; int transport_count; diff --git a/src/src/globals.h b/src/src/globals.h index 954a0a3dc..8fbb14136 100644 --- a/src/src/globals.h +++ b/src/src/globals.h @@ -140,6 +140,8 @@ extern uschar *tls_resumption_hosts; /* TLS session resumption */ extern uschar *tls_try_verify_hosts; /* Optional client verification */ extern uschar *tls_verify_certificates;/* Path for certificates to check */ extern uschar *tls_verify_hosts; /* Mandatory client verification */ +extern int tls_watch_fd; /* for inotify of creds files */ +extern time_t tls_watch_trigger_time; /* non-0: triggered */ #endif extern uschar *tls_advertise_hosts; /* host for which TLS is advertised */ diff --git a/src/src/queue.c b/src/src/queue.c index 6748afd5d..4c93c1d7f 100644 --- a/src/src/queue.c +++ b/src/src/queue.c @@ -26,6 +26,9 @@ Michael Haardt. */ #define LOG2_MAXNODES 32 +#ifndef DISABLE_TLS +static BOOL queue_tls_init = FALSE; +#endif /************************************************* * Helper sort function for queue_get_spool_list * @@ -648,6 +651,16 @@ for (int i = queue_run_in_order ? -1 : 0; report_time_since(×tamp_startup, US"queue msg selected"); #endif +#ifndef DISABLE_TLS + if (!queue_tls_init) + { + queue_tls_init = TRUE; + /* Preload TLS library info for smtp transports. Once, and only if we + have a delivery to do. */ + tls_client_creds_reload(FALSE); + } +#endif + single_item_retry: if ((pid = exim_fork(US"qrun-delivery")) == 0) { diff --git a/src/src/smtp_in.c b/src/src/smtp_in.c index aa1d5b09c..c0b6b2ac1 100644 --- a/src/src/smtp_in.c +++ b/src/src/smtp_in.c @@ -2934,7 +2934,7 @@ if (check_proxy_protocol_host()) #ifndef DISABLE_TLS if (tls_in.on_connect) { - if (tls_server_start(tls_require_ciphers, &user_msg) != OK) + if (tls_server_start(&user_msg) != OK) return smtp_log_tls_fail(user_msg); cmd_list[CMD_LIST_TLS_AUTH].is_mail_cmd = TRUE; } @@ -5490,7 +5490,7 @@ while (done <= 0) STARTTLS that don't add to the nonmail command count. */ s = NULL; - if ((rc = tls_server_start(tls_require_ciphers, &s)) == OK) + if ((rc = tls_server_start(&s)) == OK) { if (!tls_remember_esmtp) fl.helo_seen = fl.esmtp = fl.auth_advertised = f.smtp_in_pipelining_advertised = FALSE; diff --git a/src/src/smtp_out.c b/src/src/smtp_out.c index c4c409677..911dd537e 100644 --- a/src/src/smtp_out.c +++ b/src/src/smtp_out.c @@ -609,7 +609,7 @@ if (format) while (!isspace(*p)) p++; while (isspace(*p)) p++; } - while (*p != 0) *p++ = '*'; + while (*p) *p++ = '*'; } HDEBUG(D_transport|D_acl|D_v) debug_printf_indent(" SMTP>> %s\n", big_buffer); diff --git a/src/src/tls-gnu.c b/src/src/tls-gnu.c index 03af7d7dc..f5c6a8bd6 100644 --- a/src/src/tls-gnu.c +++ b/src/src/tls-gnu.c @@ -145,6 +145,9 @@ builtin_macro_create(US"_HAVE_TLS_OCSP"); # ifdef SUPPORT_SRV_OCSP_STACK builtin_macro_create(US"_HAVE_TLS_OCSP_LIST"); # endif +#ifdef EXIM_HAVE_INOTIFY +builtin_macro_create(US"_HAVE_TLS_CA_CACHE"); +# endif } #else @@ -178,8 +181,11 @@ Not handled here: global tlsp->tls_channelbinding. typedef struct exim_gnutls_state { gnutls_session_t session; - gnutls_certificate_credentials_t x509_cred; - gnutls_priority_t priority_cache; + + exim_tlslib_state lib_state; +#define x509_cred libdata0 +#define pri_cache libdata1 + enum peer_verify_requirement verify_requirement; int fd_in; int fd_out; @@ -245,7 +251,11 @@ second connection. XXX But see gnutls_session_get_ptr() */ -static exim_gnutls_state_st state_server; +static exim_gnutls_state_st state_server = { + /* all elements not explicitly intialised here get 0/NULL/FALSE */ + .fd_in = -1, + .fd_out = -1, +}; #ifndef GNUTLS_AUTO_DHPARAMS /* dh_params are initialised once within the lifetime of a process using TLS; @@ -298,7 +308,7 @@ before, for now. */ # define EXIM_SERVER_DH_BITS_PRE2_12 1024 #endif -#define expand_check_tlsvar(Varname, errstr) \ +#define Expand_check_tlsvar(Varname, errstr) \ expand_check(state->Varname, US #Varname, &state->exp_##Varname, errstr) #if GNUTLS_VERSION_NUMBER >= 0x020c00 @@ -335,22 +345,114 @@ tls_server_ticket_cb(gnutls_session_t sess, u_int htype, unsigned when, #endif +/* ------------------------------------------------------------------------ */ +/* Initialisation */ + +#ifndef DISABLE_OCSP + +static BOOL +tls_is_buggy_ocsp(void) +{ +const uschar * s; +uschar maj, mid, mic; + +s = CUS gnutls_check_version(NULL); +maj = atoi(CCS s); +if (maj == 3) + { + while (*s && *s != '.') s++; + mid = atoi(CCS ++s); + if (mid <= 2) + return TRUE; + else if (mid >= 5) + return FALSE; + else + { + while (*s && *s != '.') s++; + mic = atoi(CCS ++s); + return mic <= (mid == 3 ? 16 : 3); + } + } +return FALSE; +} + +#endif + + +static void +tls_g_init(void) +{ +DEBUG(D_tls) debug_printf("GnuTLS global init required\n"); + +#if defined(HAVE_GNUTLS_PKCS11) && !defined(GNUTLS_AUTO_PKCS11_MANUAL) +/* By default, gnutls_global_init will init PKCS11 support in auto mode, +which loads modules from a config file, which sounds good and may be wanted +by some sysadmin, but also means in common configurations that GNOME keyring +environment variables are used and so breaks for users calling mailq. +To prevent this, we init PKCS11 first, which is the documented approach. */ + +if (!gnutls_allow_auto_pkcs11) + if ((rc = gnutls_pkcs11_init(GNUTLS_PKCS11_FLAG_MANUAL, NULL))) + return tls_error_gnu(US"gnutls_pkcs11_init", rc, host, errstr); +#endif + +#ifndef GNUTLS_AUTO_GLOBAL_INIT +if ((rc = gnutls_global_init())) + return tls_error_gnu(US"gnutls_global_init", rc, host, errstr); +#endif + +#if EXIM_GNUTLS_LIBRARY_LOG_LEVEL >= 0 +DEBUG(D_tls) + { + gnutls_global_set_log_function(exim_gnutls_logger_cb); + /* arbitrarily chosen level; bump up to 9 for more */ + gnutls_global_set_log_level(EXIM_GNUTLS_LIBRARY_LOG_LEVEL); + } +#endif + +#ifndef DISABLE_OCSP +if (tls_ocsp_file && (gnutls_buggy_ocsp = tls_is_buggy_ocsp())) + log_write(0, LOG_MAIN, "OCSP unusable with this GnuTLS library version"); +#endif + +exim_gnutls_base_init_done = TRUE; +} + + + +/* Daemon-call before each connection. Nothing to do for GnuTLS. */ + +static void +tls_per_lib_daemon_tick(void) +{ +} + /* Daemon one-time initialisation */ -void -tls_daemon_init(void) + +static void +tls_per_lib_daemon_init(void) { +static BOOL once = FALSE; + +if (!exim_gnutls_base_init_done) + tls_g_init(); + +if (!once) + { + once = TRUE; + #ifdef EXIM_HAVE_TLS_RESUME -/* We are dependent on the GnuTLS implementation of the Session Ticket -encryption; both the strength and the key rotation period. We hope that -the strength at least matches that of the ciphersuite (but GnuTLS does not -document this). */ + /* We are dependent on the GnuTLS implementation of the Session Ticket + encryption; both the strength and the key rotation period. We hope that + the strength at least matches that of the ciphersuite (but GnuTLS does not + document this). */ -static BOOL once = FALSE; -if (once) return; -once = TRUE; -gnutls_session_ticket_key_generate(&server_sessticket_key); /* >= 2.10.0 */ -if (f.running_in_test_harness) ssl_session_timeout = 6; + gnutls_session_ticket_key_generate(&server_sessticket_key); /* >= 2.10.0 */ + if (f.running_in_test_harness) ssl_session_timeout = 6; #endif + + tls_daemon_creds_reload(); + } } /* ------------------------------------------------------------------------ */ @@ -598,7 +700,7 @@ uschar *exp_tls_dhparam; BOOL use_file_in_spool = FALSE; host_item *host = NULL; /* dummy for macros */ -DEBUG(D_tls) debug_printf("Initialising GnuTLS server params.\n"); +DEBUG(D_tls) debug_printf("Initialising GnuTLS server params\n"); if ((rc = gnutls_dh_params_init(&dh_server_params))) return tls_error_gnu(US"gnutls_dh_params_init", rc, host, errstr); @@ -616,7 +718,7 @@ else if (Ustrcmp(exp_tls_dhparam, "historic") == 0) use_file_in_spool = TRUE; else if (Ustrcmp(exp_tls_dhparam, "none") == 0) { - DEBUG(D_tls) debug_printf("Requested no DH parameters.\n"); + DEBUG(D_tls) debug_printf("Requested no DH parameters\n"); return OK; } else if (exp_tls_dhparam[0] != '/') @@ -643,12 +745,12 @@ different filename and ensure we have sufficient bits. */ if (!(dh_bits = gnutls_sec_param_to_pk_bits(GNUTLS_PK_DH, GNUTLS_SEC_PARAM_NORMAL))) return tls_error(US"gnutls_sec_param_to_pk_bits() failed", NULL, NULL, errstr); DEBUG(D_tls) - debug_printf("GnuTLS tells us that for D-H PK, NORMAL is %d bits.\n", + debug_printf("GnuTLS tells us that for D-H PK, NORMAL is %d bits\n", dh_bits); #else dh_bits = EXIM_SERVER_DH_BITS_PRE2_12; DEBUG(D_tls) - debug_printf("GnuTLS lacks gnutls_sec_param_to_pk_bits(), using %d bits.\n", + debug_printf("GnuTLS lacks gnutls_sec_param_to_pk_bits(), using %d bits\n", dh_bits); #endif @@ -656,7 +758,7 @@ DEBUG(D_tls) if (dh_bits > tls_dh_max_bits) { DEBUG(D_tls) - debug_printf("tls_dh_max_bits clamping override, using %d bits instead.\n", + debug_printf("tls_dh_max_bits clamping override, using %d bits instead\n", tls_dh_max_bits); dh_bits = tls_dh_max_bits; } @@ -884,7 +986,8 @@ if ((rc = gnutls_x509_crt_sign(cert, cert, pkey))) goto err; where = US"installing selfsign cert"; /* Since: 2.4.0 */ -if ((rc = gnutls_certificate_set_x509_key(state->x509_cred, &cert, 1, pkey))) +if ((rc = gnutls_certificate_set_x509_key(state->lib_state.x509_cred, + &cert, 1, pkey))) goto err; rc = OK; @@ -911,10 +1014,10 @@ Return: static int tls_add_certfile(exim_gnutls_state_st * state, const host_item * host, - uschar * certfile, uschar * keyfile, uschar ** errstr) + const uschar * certfile, const uschar * keyfile, uschar ** errstr) { -int rc = gnutls_certificate_set_x509_key_file(state->x509_cred, - CS certfile, CS keyfile, GNUTLS_X509_FMT_PEM); +int rc = gnutls_certificate_set_x509_key_file(state->lib_state.x509_cred, + CCS certfile, CCS keyfile, GNUTLS_X509_FMT_PEM); if (rc < 0) return tls_error_gnu( string_sprintf("cert/key setup: cert=%s key=%s", certfile, keyfile), @@ -1024,50 +1127,508 @@ tls_in.ocsp = exim_testharness_disable_ocsp_validity_check #else tls_in.ocsp = OCSP_VFY_NOT_TRIED; #endif -return 0; +return 0; +} + +/* Callback for handshake messages, on server */ +static int +tls_server_hook_cb(gnutls_session_t sess, u_int htype, unsigned when, + unsigned incoming, const gnutls_datum_t * msg) +{ +/* debug_printf("%s: htype %u\n", __FUNCTION__, htype); */ +switch (htype) + { +# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE + case GNUTLS_HANDSHAKE_CLIENT_HELLO: + return tls_server_clienthello_cb(sess, htype, when, incoming, msg); + case GNUTLS_HANDSHAKE_CERTIFICATE_PKT: + return tls_server_servercerts_cb(sess, htype, when, incoming, msg); +# endif + case GNUTLS_HANDSHAKE_CERTIFICATE_STATUS: + return tls_server_certstatus_cb(sess, htype, when, incoming, msg); +# ifdef EXIM_HAVE_TLS_RESUME + case GNUTLS_HANDSHAKE_NEW_SESSION_TICKET: + return tls_server_ticket_cb(sess, htype, when, incoming, msg); +# endif + default: + return 0; + } +} +#endif + + +#if !defined(DISABLE_OCSP) && defined(SUPPORT_GNUTLS_EXT_RAW_PARSE) +static void +tls_server_testharness_ocsp_fiddle(void) +{ +extern char ** environ; +if (environ) for (uschar ** p = USS environ; *p; p++) + if (Ustrncmp(*p, "EXIM_TESTHARNESS_DISABLE_OCSPVALIDITYCHECK", 42) == 0) + { + DEBUG(D_tls) debug_printf("Permitting known bad OCSP response\n"); + exim_testharness_disable_ocsp_validity_check = TRUE; + } +} +#endif + +/************************************************** +* One-time init credentials for server and client * +**************************************************/ + +static void +creds_basic_init(gnutls_certificate_credentials_t x509_cred, BOOL server) +{ +#ifdef SUPPORT_SRV_OCSP_STACK +gnutls_certificate_set_flags(x509_cred, GNUTLS_CERTIFICATE_API_V2); + +# if !defined(DISABLE_OCSP) && defined(SUPPORT_GNUTLS_EXT_RAW_PARSE) +if (server && tls_ocsp_file) + { + if (f.running_in_test_harness) + tls_server_testharness_ocsp_fiddle(); + + if (exim_testharness_disable_ocsp_validity_check) + gnutls_certificate_set_flags(x509_cred, + GNUTLS_CERTIFICATE_API_V2 | GNUTLS_CERTIFICATE_SKIP_OCSP_RESPONSE_CHECK); + } +# endif +#endif +DEBUG(D_tls) + debug_printf("TLS: basic cred init, %s\n", server ? "server" : "client"); +} + +static int +creds_load_server_certs(exim_gnutls_state_st * state, const uschar * cert, + const uschar * pkey, const uschar * ocsp, uschar ** errstr) +{ +const uschar * clist = cert; +const uschar * klist = pkey; +const uschar * olist; +int csep = 0, ksep = 0, osep = 0, cnt = 0, rc; +uschar * cfile, * kfile, * ofile; +#ifndef DISABLE_OCSP +# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE +gnutls_x509_crt_fmt_t ocsp_fmt = GNUTLS_X509_FMT_DER; +# endif + +if (!expand_check(ocsp, US"tls_ocsp_file", &ofile, errstr)) + return DEFER; +olist = ofile; +#endif + +while (cfile = string_nextinlist(&clist, &csep, NULL, 0)) + + if (!(kfile = string_nextinlist(&klist, &ksep, NULL, 0))) + return tls_error(US"cert/key setup: out of keys", NULL, NULL, errstr); + else if ((rc = tls_add_certfile(state, NULL, cfile, kfile, errstr)) > 0) + return rc; + else + { + int gnutls_cert_index = -rc; + DEBUG(D_tls) debug_printf("TLS: cert/key %d %s registered\n", + gnutls_cert_index, cfile); + +#ifndef DISABLE_OCSP + if (ocsp) + { + /* Set the OCSP stapling server info */ + if (gnutls_buggy_ocsp) + { + DEBUG(D_tls) + debug_printf("GnuTLS library is buggy for OCSP; avoiding\n"); + } + else if ((ofile = string_nextinlist(&olist, &osep, NULL, 0))) + { + DEBUG(D_tls) debug_printf("OCSP response file %d = %s\n", + gnutls_cert_index, ofile); +# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE + if (Ustrncmp(ofile, US"PEM ", 4) == 0) + { + ocsp_fmt = GNUTLS_X509_FMT_PEM; + ofile += 4; + } + else if (Ustrncmp(ofile, US"DER ", 4) == 0) + { + ocsp_fmt = GNUTLS_X509_FMT_DER; + ofile += 4; + } + + if ((rc = gnutls_certificate_set_ocsp_status_request_file2( + state->lib_state.x509_cred, CCS ofile, gnutls_cert_index, + ocsp_fmt)) < 0) + return tls_error_gnu( + US"gnutls_certificate_set_ocsp_status_request_file2", + rc, NULL, errstr); + DEBUG(D_tls) + debug_printf(" %d response%s loaded\n", rc, rc>1 ? "s":""); + + /* Arrange callbacks for OCSP request observability */ + + if (state->session) + gnutls_handshake_set_hook_function(state->session, + GNUTLS_HANDSHAKE_ANY, GNUTLS_HOOK_POST, tls_server_hook_cb); + else + state->lib_state.ocsp_hook = TRUE; + + +# else +# if defined(SUPPORT_SRV_OCSP_STACK) + if ((rc = gnutls_certificate_set_ocsp_status_request_function2( + state->lib_state.x509_cred, gnutls_cert_index, + server_ocsp_stapling_cb, ofile))) + return tls_error_gnu( + US"gnutls_certificate_set_ocsp_status_request_function2", + rc, NULL, errstr); + else +# endif + { + if (cnt++ > 0) + { + DEBUG(D_tls) + debug_printf("oops; multiple OCSP files not supported\n"); + break; + } + gnutls_certificate_set_ocsp_status_request_function( + state->lib_state.x509_cred, server_ocsp_stapling_cb, ofile); + } +# endif /* SUPPORT_GNUTLS_EXT_RAW_PARSE */ + } + else + DEBUG(D_tls) debug_printf("ran out of OCSP response files in list\n"); + } +#endif /* DISABLE_OCSP */ + } +return 0; +} + +static int +creds_load_client_certs(exim_gnutls_state_st * state, const host_item * host, + const uschar * cert, const uschar * pkey, uschar ** errstr) +{ +int rc = tls_add_certfile(state, host, cert, pkey, errstr); +if (rc > 0) return rc; +DEBUG(D_tls) debug_printf("TLS: cert/key registered\n"); +return 0; +} + +static int +creds_load_cabundle(exim_gnutls_state_st * state, const uschar * bundle, + const host_item * host, uschar ** errstr) +{ +int cert_count; +struct stat statbuf; + +#ifdef SUPPORT_SYSDEFAULT_CABUNDLE +if (Ustrcmp(bundle, "system") == 0 || Ustrncmp(bundle, "system,", 7) == 0) + cert_count = gnutls_certificate_set_x509_system_trust(state->lib_state.x509_cred); +else +#endif + { + if (Ustat(bundle, &statbuf) < 0) + { + log_write(0, LOG_MAIN|LOG_PANIC, "could not stat '%s' " + "(tls_verify_certificates): %s", bundle, strerror(errno)); + return DEFER; + } + +#ifndef SUPPORT_CA_DIR + /* The test suite passes in /dev/null; we could check for that path explicitly, + but who knows if someone has some weird FIFO which always dumps some certs, or + other weirdness. The thing we really want to check is that it's not a + directory, since while OpenSSL supports that, GnuTLS does not. + So s/!S_ISREG/S_ISDIR/ and change some messaging ... */ + if (S_ISDIR(statbuf.st_mode)) + { + log_write(0, LOG_MAIN|LOG_PANIC, + "tls_verify_certificates \"%s\" is a directory", bundle); + return DEFER; + } +#endif + + DEBUG(D_tls) debug_printf("verify certificates = %s size=" OFF_T_FMT "\n", + bundle, statbuf.st_size); + + if (statbuf.st_size == 0) + { + DEBUG(D_tls) + debug_printf("cert file empty, no certs, no verification, ignoring any CRL\n"); + return OK; + } + + cert_count = + +#ifdef SUPPORT_CA_DIR + (statbuf.st_mode & S_IFMT) == S_IFDIR + ? + gnutls_certificate_set_x509_trust_dir(state->lib_state.x509_cred, + CS bundle, GNUTLS_X509_FMT_PEM) + : +#endif + gnutls_certificate_set_x509_trust_file(state->lib_state.x509_cred, + CS bundle, GNUTLS_X509_FMT_PEM); + +#ifdef SUPPORT_CA_DIR + /* Mimic the behaviour with OpenSSL of not advertising a usable-cert list + when using the directory-of-certs config model. */ + + if ((statbuf.st_mode & S_IFMT) == S_IFDIR) + if (state->session) + gnutls_certificate_send_x509_rdn_sequence(state->session, 1); + else + state->lib_state.ca_rdn_emulate = TRUE; +#endif + } + +if (cert_count < 0) + return tls_error_gnu(US"setting certificate trust", cert_count, host, errstr); +DEBUG(D_tls) + debug_printf("Added %d certificate authorities\n", cert_count); + +return OK; +} + + +static int +creds_load_crl(exim_gnutls_state_st * state, const uschar * crl, uschar ** errstr) +{ +int cert_count; +DEBUG(D_tls) debug_printf("loading CRL file = %s\n", crl); +if ((cert_count = gnutls_certificate_set_x509_crl_file(state->lib_state.x509_cred, + CS crl, GNUTLS_X509_FMT_PEM)) < 0) + return tls_error_gnu(US"gnutls_certificate_set_x509_crl_file", + cert_count, state->host, errstr); + +DEBUG(D_tls) debug_printf("Processed %d CRLs\n", cert_count); +return OK; +} + + +static int +creds_load_pristring(exim_gnutls_state_st * state, const uschar * p, + const char ** errpos) +{ +if (!p) + { + p = exim_default_gnutls_priority; + DEBUG(D_tls) + debug_printf("GnuTLS using default session cipher/priority \"%s\"\n", p); + } +return gnutls_priority_init( (gnutls_priority_t *) &state->lib_state.pri_cache, + CCS p, errpos); +} + +static void +tls_server_creds_init(void) +{ +uschar * dummy_errstr; + +state_server.lib_state = null_tls_preload; +if (gnutls_certificate_allocate_credentials( + (gnutls_certificate_credentials_t *) &state_server.lib_state.x509_cred)) + { + state_server.lib_state.x509_cred = NULL; + return; + } +creds_basic_init(state_server.lib_state.x509_cred, TRUE); + +#ifdef EXIM_HAVE_INOTIFY +/* If tls_certificate has any $ indicating expansions, it is not good. +If tls_privatekey is set but has $, not good. Likewise for tls_ocsp_file. +If all good (and tls_certificate set), load the cert(s). Do not try +to handle selfsign generation for now (tls_certificate null/empty; +XXX will want to do that later though) due to the lifetime/expiry issue. */ + +if ( opt_set_and_noexpand(tls_certificate) + && opt_unset_or_noexpand(tls_privatekey) + && opt_unset_or_noexpand(tls_ocsp_file)) + { + /* Set watches on the filenames. The implementation does de-duplication + so we can just blindly do them all. + */ + + if ( tls_set_watch(tls_certificate, TRUE) + && tls_set_watch(tls_privatekey, TRUE) + && tls_set_watch(tls_ocsp_file, TRUE) + ) + { + DEBUG(D_tls) debug_printf("TLS: preloading server certs\n"); + if (creds_load_server_certs(&state_server, tls_certificate, + tls_privatekey && *tls_privatekey ? tls_privatekey : tls_certificate, + tls_ocsp_file, &dummy_errstr) == 0) + state_server.lib_state.conn_certs = TRUE; + } + } +else + DEBUG(D_tls) debug_printf("TLS: not preloading server certs\n"); + +/* If tls_verify_certificates is non-empty and has no $, load CAs */ + +if (opt_set_and_noexpand(tls_verify_certificates)) + { + if (tls_set_watch(tls_verify_certificates, FALSE)) + { + DEBUG(D_tls) debug_printf("TLS: preloading CA bundle for server\n"); + if (creds_load_cabundle(&state_server, tls_verify_certificates, + NULL, &dummy_errstr) != OK) + return; + state_server.lib_state.cabundle = TRUE; + + /* If CAs loaded and tls_crl is non-empty and has no $, load it */ + + if (opt_set_and_noexpand(tls_crl)) + { + if (tls_set_watch(tls_crl, FALSE)) + { + DEBUG(D_tls) debug_printf("TLS: preloading CRL for server\n"); + if (creds_load_crl(&state_server, tls_crl, &dummy_errstr) != OK) + return; + state_server.lib_state.crl = TRUE; + } + } + else + DEBUG(D_tls) debug_printf("TLS: not preloading CRL for server\n"); + } + } +else + DEBUG(D_tls) debug_printf("TLS: not preloading CA bundle for server\n"); +#endif /* EXIM_HAVE_INOTIFY */ + +/* If tls_require_ciphers is non-empty and has no $, load the +ciphers priority cache. If unset, load with the default. +(server-only as the client one depends on non/DANE) */ + +if (!tls_require_ciphers || opt_set_and_noexpand(tls_require_ciphers)) + { + const char * dummy_errpos; + DEBUG(D_tls) debug_printf("TLS: preloading cipher list for server: %s\n", + tls_require_ciphers); + if ( creds_load_pristring(&state_server, tls_require_ciphers, &dummy_errpos) + == OK) + state_server.lib_state.pri_string = TRUE; + } +else + DEBUG(D_tls) debug_printf("TLS: not preloading cipher list for server\n"); +} + + +/* Preload whatever creds are static, onto a transport. The client can then +just copy the pointer as it starts up. */ + +static void +tls_client_creds_init(transport_instance * t, BOOL watch) +{ +smtp_transport_options_block * ob = t->options_block; +exim_gnutls_state_st tpt_dummy_state; +host_item * dummy_host = (host_item *)1; +uschar * dummy_errstr; + +if (!exim_gnutls_base_init_done) + tls_g_init(); + +ob->tls_preload = null_tls_preload; +if (gnutls_certificate_allocate_credentials( + (struct gnutls_certificate_credentials_st **)&ob->tls_preload.x509_cred)) + { + ob->tls_preload.x509_cred = NULL; + return; + } +creds_basic_init(ob->tls_preload.x509_cred, FALSE); + +tpt_dummy_state.session = NULL; +tpt_dummy_state.lib_state = ob->tls_preload; + +#ifdef EXIM_HAVE_INOTIFY +if ( opt_set_and_noexpand(ob->tls_certificate) + && opt_unset_or_noexpand(ob->tls_privatekey)) + { + if ( !watch + || ( tls_set_watch(ob->tls_certificate, FALSE) + && tls_set_watch(ob->tls_privatekey, FALSE) + ) ) + { + const uschar * pkey = ob->tls_privatekey; + + DEBUG(D_tls) + debug_printf("TLS: preloading client certs for transport '%s'\n", t->name); + + /* The state->lib_state.x509_cred is used for the certs load, and is the sole + structure element used. So we can set up a dummy. The hoat arg only + selects a retcode in case of fail, so any value */ + + if (creds_load_client_certs(&tpt_dummy_state, dummy_host, + ob->tls_certificate, pkey ? pkey : ob->tls_certificate, + &dummy_errstr) == OK) + ob->tls_preload.conn_certs = TRUE; + } + } +else + DEBUG(D_tls) + debug_printf("TLS: not preloading client certs, for transport '%s'\n", t->name); + +if (opt_set_and_noexpand(ob->tls_verify_certificates)) + { + if (!watch || tls_set_watch(ob->tls_verify_certificates, FALSE)) + { + DEBUG(D_tls) + debug_printf("TLS: preloading CA bundle for transport '%s'\n", t->name); + if (creds_load_cabundle(&tpt_dummy_state, ob->tls_verify_certificates, + dummy_host, &dummy_errstr) != OK) + return; + ob->tls_preload.cabundle = TRUE; + + if (opt_set_and_noexpand(ob->tls_crl)) + { + if (!watch || tls_set_watch(ob->tls_crl, FALSE)) + { + DEBUG(D_tls) debug_printf("TLS: preloading CRL for transport '%s'\n", t->name); + if (creds_load_crl(&tpt_dummy_state, ob->tls_crl, &dummy_errstr) != OK) + return; + ob->tls_preload.crl = TRUE; + } + } + else + DEBUG(D_tls) debug_printf("TLS: not preloading CRL, for transport '%s'\n", t->name); + } + } +else + DEBUG(D_tls) + debug_printf("TLS: not preloading CA bundle, for transport '%s'\n", t->name); + +/* We do not preload tls_require_ciphers to to the transport as it implicitly +depends on DANE or plain usage. */ + +#endif } -/* Callback for handshake messages, on server */ -static int -tls_server_hook_cb(gnutls_session_t sess, u_int htype, unsigned when, - unsigned incoming, const gnutls_datum_t * msg) + +#ifdef EXIM_HAVE_INOTIFY +/* Invalidate the creds cached, by dropping the current ones. +Call when we notice one of the source files has changed. */ + +static void +tls_server_creds_invalidate(void) { -/* debug_printf("%s: htype %u\n", __FUNCTION__, htype); */ -switch (htype) - { -# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE - case GNUTLS_HANDSHAKE_CLIENT_HELLO: - return tls_server_clienthello_cb(sess, htype, when, incoming, msg); - case GNUTLS_HANDSHAKE_CERTIFICATE_PKT: - return tls_server_servercerts_cb(sess, htype, when, incoming, msg); -# endif - case GNUTLS_HANDSHAKE_CERTIFICATE_STATUS: - return tls_server_certstatus_cb(sess, htype, when, incoming, msg); -# ifdef EXIM_HAVE_TLS_RESUME - case GNUTLS_HANDSHAKE_NEW_SESSION_TICKET: - return tls_server_ticket_cb(sess, htype, when, incoming, msg); -# endif - default: - return 0; - } +if (state_server.lib_state.pri_cache) + gnutls_priority_deinit(state_server.lib_state.pri_cache); +state_server.lib_state.pri_cache = NULL; + +if (state_server.lib_state.x509_cred) + gnutls_certificate_free_credentials(state_server.lib_state.x509_cred); +state_server.lib_state = null_tls_preload; } -#endif -#if !defined(DISABLE_OCSP) && defined(SUPPORT_GNUTLS_EXT_RAW_PARSE) static void -tls_server_testharness_ocsp_fiddle(void) +tls_client_creds_invalidate(transport_instance * t) { -extern char ** environ; -if (environ) for (uschar ** p = USS environ; *p; p++) - if (Ustrncmp(*p, "EXIM_TESTHARNESS_DISABLE_OCSPVALIDITYCHECK", 42) == 0) - { - DEBUG(D_tls) debug_printf("Permitting known bad OCSP response\n"); - exim_testharness_disable_ocsp_validity_check = TRUE; - } +smtp_transport_options_block * ob = t->options_block; +if (ob->tls_preload.x509_cred) + gnutls_certificate_free_credentials(ob->tls_preload.x509_cred); +ob->tls_preload = null_tls_preload; } #endif + /************************************************* * Variables re-expanded post-SNI * *************************************************/ @@ -1090,13 +1651,12 @@ Returns: OK/DEFER/FAIL static int tls_expand_session_files(exim_gnutls_state_st * state, uschar ** errstr) { -struct stat statbuf; int rc; const host_item *host = state->host; /* macro should be reconsidered? */ -uschar *saved_tls_certificate = NULL; -uschar *saved_tls_privatekey = NULL; -uschar *saved_tls_verify_certificates = NULL; -uschar *saved_tls_crl = NULL; +const uschar *saved_tls_certificate = NULL; +const uschar *saved_tls_privatekey = NULL; +const uschar *saved_tls_verify_certificates = NULL; +const uschar *saved_tls_crl = NULL; int cert_count; /* We check for tls_sni *before* expansion. */ @@ -1109,11 +1669,11 @@ if (!host) /* server */ || Ustrstr(state->tls_certificate, US"tls_out_sni") ) ) { - DEBUG(D_tls) debug_printf("We will re-expand TLS session files if we receive SNI.\n"); + DEBUG(D_tls) debug_printf("We will re-expand TLS session files if we receive SNI\n"); state->trigger_sni_changes = TRUE; } } - else + else /* SNI callback case */ { /* useful for debugging */ saved_tls_certificate = state->exp_tls_certificate; @@ -1122,180 +1682,91 @@ if (!host) /* server */ saved_tls_crl = state->exp_tls_crl; } -if ((rc = gnutls_certificate_allocate_credentials(&state->x509_cred))) - return tls_error_gnu(US"gnutls_certificate_allocate_credentials", - rc, host, errstr); - -#ifdef SUPPORT_SRV_OCSP_STACK -gnutls_certificate_set_flags(state->x509_cred, GNUTLS_CERTIFICATE_API_V2); - -# if !defined(DISABLE_OCSP) && defined(SUPPORT_GNUTLS_EXT_RAW_PARSE) -if (!host && tls_ocsp_file) +if (!state->lib_state.x509_cred) { - if (f.running_in_test_harness) - tls_server_testharness_ocsp_fiddle(); - - if (exim_testharness_disable_ocsp_validity_check) - gnutls_certificate_set_flags(state->x509_cred, - GNUTLS_CERTIFICATE_API_V2 | GNUTLS_CERTIFICATE_SKIP_OCSP_RESPONSE_CHECK); + if ((rc = gnutls_certificate_allocate_credentials( + (gnutls_certificate_credentials_t *) &state->lib_state.x509_cred))) + return tls_error_gnu(US"gnutls_certificate_allocate_credentials", + rc, host, errstr); + creds_basic_init(state->lib_state.x509_cred, !host); } -# endif -#endif -/* remember: expand_check_tlsvar() is expand_check() but fiddling with + +/* remember: Expand_check_tlsvar() is expand_check() but fiddling with state members, assuming consistent naming; and expand_check() returns false if expansion failed, unless expansion was forced to fail. */ /* check if we at least have a certificate, before doing expensive D-H generation. */ -if (!expand_check_tlsvar(tls_certificate, errstr)) - return DEFER; - -/* certificate is mandatory in server, optional in client */ - -if ( !state->exp_tls_certificate - || !*state->exp_tls_certificate - ) - if (!host) - return tls_install_selfsign(state, errstr); - else - DEBUG(D_tls) debug_printf("TLS: no client certificate specified; okay\n"); - -if (state->tls_privatekey && !expand_check_tlsvar(tls_privatekey, errstr)) - return DEFER; - -/* tls_privatekey is optional, defaulting to same file as certificate */ - -if (!state->tls_privatekey || !*state->tls_privatekey) +if (!state->lib_state.conn_certs) { - state->tls_privatekey = state->tls_certificate; - state->exp_tls_privatekey = state->exp_tls_certificate; - } - + if (!Expand_check_tlsvar(tls_certificate, errstr)) + return DEFER; -if (state->exp_tls_certificate && *state->exp_tls_certificate) - { - DEBUG(D_tls) debug_printf("certificate file = %s\nkey file = %s\n", - state->exp_tls_certificate, state->exp_tls_privatekey); + /* certificate is mandatory in server, optional in client */ - if (state->received_sni) - if ( Ustrcmp(state->exp_tls_certificate, saved_tls_certificate) == 0 - && Ustrcmp(state->exp_tls_privatekey, saved_tls_privatekey) == 0 - ) - { - DEBUG(D_tls) debug_printf("TLS SNI: cert and key unchanged\n"); - } + if ( !state->exp_tls_certificate + || !*state->exp_tls_certificate + ) + if (!host) + return tls_install_selfsign(state, errstr); else - { - DEBUG(D_tls) debug_printf("TLS SNI: have a changed cert/key pair.\n"); - } - - if (!host) /* server */ - { - const uschar * clist = state->exp_tls_certificate; - const uschar * klist = state->exp_tls_privatekey; - const uschar * olist; - int csep = 0, ksep = 0, osep = 0, cnt = 0; - uschar * cfile, * kfile, * ofile; -#ifndef DISABLE_OCSP -# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE - gnutls_x509_crt_fmt_t ocsp_fmt = GNUTLS_X509_FMT_DER; -# endif - - if (!expand_check(tls_ocsp_file, US"tls_ocsp_file", &ofile, errstr)) - return DEFER; - olist = ofile; -#endif - - while (cfile = string_nextinlist(&clist, &csep, NULL, 0)) + DEBUG(D_tls) debug_printf("TLS: no client certificate specified; okay\n"); - if (!(kfile = string_nextinlist(&klist, &ksep, NULL, 0))) - return tls_error(US"cert/key setup: out of keys", NULL, host, errstr); - else if (0 < (rc = tls_add_certfile(state, host, cfile, kfile, errstr))) - return rc; - else - { - int gnutls_cert_index = -rc; - DEBUG(D_tls) debug_printf("TLS: cert/key %d %s registered\n", - gnutls_cert_index, cfile); + if (state->tls_privatekey && !Expand_check_tlsvar(tls_privatekey, errstr)) + return DEFER; -#ifndef DISABLE_OCSP - if (tls_ocsp_file) - { - /* Set the OCSP stapling server info */ - if (gnutls_buggy_ocsp) - { - DEBUG(D_tls) - debug_printf("GnuTLS library is buggy for OCSP; avoiding\n"); - } - else if ((ofile = string_nextinlist(&olist, &osep, NULL, 0))) - { - DEBUG(D_tls) debug_printf("OCSP response file %d = %s\n", - gnutls_cert_index, ofile); -# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE - if (Ustrncmp(ofile, US"PEM ", 4) == 0) - { - ocsp_fmt = GNUTLS_X509_FMT_PEM; - ofile += 4; - } - else if (Ustrncmp(ofile, US"DER ", 4) == 0) - { - ocsp_fmt = GNUTLS_X509_FMT_DER; - ofile += 4; - } - - if ((rc = gnutls_certificate_set_ocsp_status_request_file2( - state->x509_cred, CCS ofile, gnutls_cert_index, - ocsp_fmt)) < 0) - return tls_error_gnu( - US"gnutls_certificate_set_ocsp_status_request_file2", - rc, host, errstr); - DEBUG(D_tls) - debug_printf(" %d response%s loaded\n", rc, rc>1 ? "s":""); + /* tls_privatekey is optional, defaulting to same file as certificate */ - /* Arrange callbacks for OCSP request observability */ + if (!state->tls_privatekey || !*state->tls_privatekey) + { + state->tls_privatekey = state->tls_certificate; + state->exp_tls_privatekey = state->exp_tls_certificate; + } - gnutls_handshake_set_hook_function(state->session, - GNUTLS_HANDSHAKE_ANY, GNUTLS_HOOK_POST, tls_server_hook_cb); + if (state->exp_tls_certificate && *state->exp_tls_certificate) + { + BOOL load = TRUE; + DEBUG(D_tls) debug_printf("certificate file = %s\nkey file = %s\n", + state->exp_tls_certificate, state->exp_tls_privatekey); -# else -# if defined(SUPPORT_SRV_OCSP_STACK) - if ((rc = gnutls_certificate_set_ocsp_status_request_function2( - state->x509_cred, gnutls_cert_index, - server_ocsp_stapling_cb, ofile))) - return tls_error_gnu( - US"gnutls_certificate_set_ocsp_status_request_function2", - rc, host, errstr); - else -# endif - { - if (cnt++ > 0) - { - DEBUG(D_tls) - debug_printf("oops; multiple OCSP files not supported\n"); - break; - } - gnutls_certificate_set_ocsp_status_request_function( - state->x509_cred, server_ocsp_stapling_cb, ofile); - } -# endif /* SUPPORT_GNUTLS_EXT_RAW_PARSE */ - } - else - DEBUG(D_tls) debug_printf("ran out of OCSP response files in list\n"); - } -#endif /* DISABLE_OCSP */ + if (state->received_sni) + if ( Ustrcmp(state->exp_tls_certificate, saved_tls_certificate) == 0 + && Ustrcmp(state->exp_tls_privatekey, saved_tls_privatekey) == 0 + ) + { + DEBUG(D_tls) debug_printf("TLS SNI: cert and key unchanged\n"); + load = FALSE; /* avoid re-loading the same certs */ } + else /* unload the pre-SNI certs before loading new ones */ + { + DEBUG(D_tls) debug_printf("TLS SNI: have a changed cert/key pair\n"); + gnutls_certificate_free_keys(state->lib_state.x509_cred); + } + + if ( load + && (rc = host + ? creds_load_client_certs(state, host, state->exp_tls_certificate, + state->exp_tls_privatekey, errstr) + : creds_load_server_certs(state, state->exp_tls_certificate, + state->exp_tls_privatekey, tls_ocsp_file, errstr) + ) ) return rc; } - else /* client */ - { - if (0 < (rc = tls_add_certfile(state, host, - state->exp_tls_certificate, state->exp_tls_privatekey, errstr))) - return rc; - DEBUG(D_tls) debug_printf("TLS: cert/key registered\n"); - } + } +else + { + DEBUG(D_tls) + debug_printf("%s certs were preloaded\n", host ? "client" : "server"); + + if (!state->tls_privatekey) state->tls_privatekey = state->tls_certificate; + state->exp_tls_certificate = US state->tls_certificate; + state->exp_tls_privatekey = US state->tls_privatekey; - } /* tls_certificate */ + if (state->lib_state.ocsp_hook) + gnutls_handshake_set_hook_function(state->session, + GNUTLS_HANDSHAKE_ANY, GNUTLS_HOOK_POST, tls_server_hook_cb); + } /* Set the trusted CAs file if one is provided, and then add the CRL if one is @@ -1304,112 +1775,64 @@ error message is provided. However, if we just refrain from setting anything up in that case, certificate verification fails, which seems to be the correct behaviour. */ -if (state->tls_verify_certificates && *state->tls_verify_certificates) +if (!state->lib_state.cabundle) { - if (!expand_check_tlsvar(tls_verify_certificates, errstr)) - return DEFER; + if (state->tls_verify_certificates && *state->tls_verify_certificates) + { + if (!Expand_check_tlsvar(tls_verify_certificates, errstr)) + return DEFER; #ifndef SUPPORT_SYSDEFAULT_CABUNDLE - if (Ustrcmp(state->exp_tls_verify_certificates, "system") == 0) - state->exp_tls_verify_certificates = NULL; + if (Ustrcmp(state->exp_tls_verify_certificates, "system") == 0) + state->exp_tls_verify_certificates = NULL; #endif - if (state->tls_crl && *state->tls_crl) - if (!expand_check_tlsvar(tls_crl, errstr)) - return DEFER; + if (state->tls_crl && *state->tls_crl) + if (!Expand_check_tlsvar(tls_crl, errstr)) + return DEFER; - if (!(state->exp_tls_verify_certificates && - *state->exp_tls_verify_certificates)) + if (!(state->exp_tls_verify_certificates && + *state->exp_tls_verify_certificates)) + { + DEBUG(D_tls) + debug_printf("TLS: tls_verify_certificates expanded empty, ignoring\n"); + /* With no tls_verify_certificates, we ignore tls_crl too */ + return OK; + } + } + else { DEBUG(D_tls) - debug_printf("TLS: tls_verify_certificates expanded empty, ignoring\n"); - /* With no tls_verify_certificates, we ignore tls_crl too */ + debug_printf("TLS: tls_verify_certificates not set or empty, ignoring\n"); return OK; } + rc = creds_load_cabundle(state, state->exp_tls_verify_certificates, host, errstr); + if (rc != OK) return rc; } else { DEBUG(D_tls) - debug_printf("TLS: tls_verify_certificates not set or empty, ignoring\n"); - return OK; - } - -#ifdef SUPPORT_SYSDEFAULT_CABUNDLE -if (Ustrcmp(state->exp_tls_verify_certificates, "system") == 0) - cert_count = gnutls_certificate_set_x509_system_trust(state->x509_cred); -else -#endif - { - if (Ustat(state->exp_tls_verify_certificates, &statbuf) < 0) - { - log_write(0, LOG_MAIN|LOG_PANIC, "could not stat '%s' " - "(tls_verify_certificates): %s", state->exp_tls_verify_certificates, - strerror(errno)); - return DEFER; - } - -#ifndef SUPPORT_CA_DIR - /* The test suite passes in /dev/null; we could check for that path explicitly, - but who knows if someone has some weird FIFO which always dumps some certs, or - other weirdness. The thing we really want to check is that it's not a - directory, since while OpenSSL supports that, GnuTLS does not. - So s/!S_ISREG/S_ISDIR/ and change some messaging ... */ - if (S_ISDIR(statbuf.st_mode)) - { - DEBUG(D_tls) - debug_printf("verify certificates path is a dir: \"%s\"\n", - state->exp_tls_verify_certificates); - log_write(0, LOG_MAIN|LOG_PANIC, - "tls_verify_certificates \"%s\" is a directory", - state->exp_tls_verify_certificates); - return DEFER; - } -#endif - - DEBUG(D_tls) debug_printf("verify certificates = %s size=" OFF_T_FMT "\n", - state->exp_tls_verify_certificates, statbuf.st_size); - - if (statbuf.st_size == 0) - { - DEBUG(D_tls) - debug_printf("cert file empty, no certs, no verification, ignoring any CRL\n"); - return OK; - } - - cert_count = - -#ifdef SUPPORT_CA_DIR - (statbuf.st_mode & S_IFMT) == S_IFDIR - ? - gnutls_certificate_set_x509_trust_dir(state->x509_cred, - CS state->exp_tls_verify_certificates, GNUTLS_X509_FMT_PEM) - : -#endif - gnutls_certificate_set_x509_trust_file(state->x509_cred, - CS state->exp_tls_verify_certificates, GNUTLS_X509_FMT_PEM); + debug_printf("%s CA bundle was preloaded\n", host ? "client" : "server"); + state->exp_tls_verify_certificates = US state->tls_verify_certificates; #ifdef SUPPORT_CA_DIR - /* Mimic the behaviour with OpenSSL of not advertising a usable-cert list - when using the directory-of-certs config model. */ - - if ((statbuf.st_mode & S_IFMT) == S_IFDIR) - gnutls_certificate_send_x509_rdn_sequence(state->session, 1); +/* Mimic the behaviour with OpenSSL of not advertising a usable-cert list +when using the directory-of-certs config model. */ + if (state->lib_state.ca_rdn_emulate) + gnutls_certificate_send_x509_rdn_sequence(state->session, 1); #endif } -if (cert_count < 0) - return tls_error_gnu(US"setting certificate trust", cert_count, host, errstr); -DEBUG(D_tls) - debug_printf("Added %d certificate authorities.\n", cert_count); -if (state->tls_crl && *state->tls_crl && - state->exp_tls_crl && *state->exp_tls_crl) +if (!state->lib_state.crl) { - DEBUG(D_tls) debug_printf("loading CRL file = %s\n", state->exp_tls_crl); - if ((cert_count = gnutls_certificate_set_x509_crl_file(state->x509_cred, - CS state->exp_tls_crl, GNUTLS_X509_FMT_PEM)) < 0) - return tls_error_gnu(US"gnutls_certificate_set_x509_crl_file", - cert_count, host, errstr); - - DEBUG(D_tls) debug_printf("Processed %d CRLs.\n", cert_count); + if ( state->tls_crl && *state->tls_crl + && state->exp_tls_crl && *state->exp_tls_crl) + return creds_load_crl(state, state->exp_tls_crl, errstr); + } +else + { + DEBUG(D_tls) + debug_printf("%s CRL was preloaded\n", host ? "client" : "server"); + state->exp_tls_crl = US state->tls_crl; } return OK; @@ -1448,18 +1871,19 @@ client-side params. */ if (!state->host) { +/*XXX DDD done-bit */ if (!dh_server_params) if ((rc = init_server_dh(errstr)) != OK) return rc; /* Unnecessary & discouraged with 3.6.0 or later */ - gnutls_certificate_set_dh_params(state->x509_cred, dh_server_params); + gnutls_certificate_set_dh_params(state->.lib_statex509_cred, dh_server_params); } #endif /* Link the credentials to the session. */ if ((rc = gnutls_credentials_set(state->session, - GNUTLS_CRD_CERTIFICATE, state->x509_cred))) + GNUTLS_CRD_CERTIFICATE, state->lib_state.x509_cred))) return tls_error_gnu(US"gnutls_credentials_set", rc, host, errstr); return OK; @@ -1470,47 +1894,12 @@ return OK; *************************************************/ -#ifndef DISABLE_OCSP - -static BOOL -tls_is_buggy_ocsp(void) -{ -const uschar * s; -uschar maj, mid, mic; - -s = CUS gnutls_check_version(NULL); -maj = atoi(CCS s); -if (maj == 3) - { - while (*s && *s != '.') s++; - mid = atoi(CCS ++s); - if (mid <= 2) - return TRUE; - else if (mid >= 5) - return FALSE; - else - { - while (*s && *s != '.') s++; - mic = atoi(CCS ++s); - return mic <= (mid == 3 ? 16 : 3); - } - } -return FALSE; -} - -#endif - - /* Called from both server and client code. In the case of a server, errors before actual TLS negotiation return DEFER. Arguments: host connected host, if client; NULL if server - certificate certificate file - privatekey private key file - sni TLS SNI to send, sometimes when client; else NULL - cas CA certs file - crl CRL file + ob tranport options block, if client; NULL if server require_ciphers tls_require_ciphers setting caller_state returned state-info structure errstr error string pointer @@ -1521,12 +1910,8 @@ Returns: OK/DEFER/FAIL static int tls_init( const host_item *host, - const uschar *certificate, - const uschar *privatekey, - const uschar *sni, - const uschar *cas, - const uschar *crl, - const uschar *require_ciphers, + smtp_transport_options_block * ob, + const uschar * require_ciphers, exim_gnutls_state_st **caller_state, tls_support * tlsp, uschar ** errstr) @@ -1534,85 +1919,60 @@ tls_init( exim_gnutls_state_st * state; int rc; size_t sz; -const char * errpos; -const uschar * p; if (!exim_gnutls_base_init_done) - { - DEBUG(D_tls) debug_printf("GnuTLS global init required.\n"); - -#if defined(HAVE_GNUTLS_PKCS11) && !defined(GNUTLS_AUTO_PKCS11_MANUAL) - /* By default, gnutls_global_init will init PKCS11 support in auto mode, - which loads modules from a config file, which sounds good and may be wanted - by some sysadmin, but also means in common configurations that GNOME keyring - environment variables are used and so breaks for users calling mailq. - To prevent this, we init PKCS11 first, which is the documented approach. */ - if (!gnutls_allow_auto_pkcs11) - if ((rc = gnutls_pkcs11_init(GNUTLS_PKCS11_FLAG_MANUAL, NULL))) - return tls_error_gnu(US"gnutls_pkcs11_init", rc, host, errstr); -#endif - -#ifndef GNUTLS_AUTO_GLOBAL_INIT - if ((rc = gnutls_global_init())) - return tls_error_gnu(US"gnutls_global_init", rc, host, errstr); -#endif - -#if EXIM_GNUTLS_LIBRARY_LOG_LEVEL >= 0 - DEBUG(D_tls) - { - gnutls_global_set_log_function(exim_gnutls_logger_cb); - /* arbitrarily chosen level; bump up to 9 for more */ - gnutls_global_set_log_level(EXIM_GNUTLS_LIBRARY_LOG_LEVEL); - } -#endif - -#ifndef DISABLE_OCSP - if (tls_ocsp_file && (gnutls_buggy_ocsp = tls_is_buggy_ocsp())) - log_write(0, LOG_MAIN, "OCSP unusable with this GnuTLS library version"); -#endif - - exim_gnutls_base_init_done = TRUE; - } + tls_g_init(); if (host) { /* For client-side sessions we allocate a context. This lets us run several in parallel. */ + int old_pool = store_pool; store_pool = POOL_PERM; state = store_get(sizeof(exim_gnutls_state_st), FALSE); store_pool = old_pool; memcpy(state, &exim_gnutls_state_init, sizeof(exim_gnutls_state_init)); + state->lib_state = ob->tls_preload; state->tlsp = tlsp; DEBUG(D_tls) debug_printf("initialising GnuTLS client session\n"); rc = gnutls_init(&state->session, GNUTLS_CLIENT); + + state->tls_certificate = ob->tls_certificate; + state->tls_privatekey = ob->tls_privatekey; + state->tls_sni = ob->tls_sni; + state->tls_verify_certificates = ob->tls_verify_certificates; + state->tls_crl = ob->tls_crl; } else { + /* Server operations always use the one state_server context. It is not + shared because we have forked a fresh process for every receive. However it + can get re-used for successive TLS sessions on a single TCP connection. */ + state = &state_server; - memcpy(state, &exim_gnutls_state_init, sizeof(exim_gnutls_state_init)); state->tlsp = tlsp; DEBUG(D_tls) debug_printf("initialising GnuTLS server session\n"); rc = gnutls_init(&state->session, GNUTLS_SERVER); + + state->tls_certificate = tls_certificate; + state->tls_privatekey = tls_privatekey; + state->tls_sni = NULL; + state->tls_verify_certificates = tls_verify_certificates; + state->tls_crl = tls_crl; } if (rc) return tls_error_gnu(US"gnutls_init", rc, host, errstr); +state->tls_require_ciphers = require_ciphers; state->host = host; -state->tls_certificate = certificate; -state->tls_privatekey = privatekey; -state->tls_require_ciphers = require_ciphers; -state->tls_sni = sni; -state->tls_verify_certificates = cas; -state->tls_crl = crl; - /* This handles the variables that might get re-expanded after TLS SNI; that's tls_certificate, tls_privatekey, tls_verify_certificates, tls_crl */ DEBUG(D_tls) - debug_printf("Expanding various TLS configuration options for session credentials.\n"); + debug_printf("Expanding various TLS configuration options for session credentials\n"); if ((rc = tls_expand_session_files(state, errstr)) != OK) return rc; /* These are all other parts of the x509_cred handling, since SNI in GnuTLS @@ -1623,7 +1983,7 @@ if ((rc = tls_set_remaining_x509(state, errstr)) != OK) return rc; /* set SNI in client, only */ if (host) { - if (!expand_check(sni, US"tls_out_sni", &state->tlsp->sni, errstr)) + if (!expand_check(state->tls_sni, US"tls_out_sni", &state->tlsp->sni, errstr)) return DEFER; if (state->tlsp->sni && *state->tlsp->sni) { @@ -1639,37 +1999,42 @@ else if (state->tls_sni) DEBUG(D_tls) debug_printf("*** PROBABLY A BUG *** " \ "have an SNI set for a server [%s]\n", state->tls_sni); -/* This is the priority string support, -http://www.gnutls.org/manual/html_node/Priority-Strings.html -and replaces gnutls_require_kx, gnutls_require_mac & gnutls_require_protocols. -This was backwards incompatible, but means Exim no longer needs to track -all algorithms and provide string forms for them. */ - -p = NULL; -if (state->tls_require_ciphers && *state->tls_require_ciphers) +if (!state->lib_state.pri_string) { - if (!expand_check_tlsvar(tls_require_ciphers, errstr)) - return DEFER; - if (state->exp_tls_require_ciphers && *state->exp_tls_require_ciphers) + const uschar * p = NULL; + const char * errpos; + + /* This is the priority string support, + http://www.gnutls.org/manual/html_node/Priority-Strings.html + and replaces gnutls_require_kx, gnutls_require_mac & gnutls_require_protocols. + This was backwards incompatible, but means Exim no longer needs to track + all algorithms and provide string forms for them. */ + + if (state->tls_require_ciphers && *state->tls_require_ciphers) { - p = state->exp_tls_require_ciphers; - DEBUG(D_tls) debug_printf("GnuTLS session cipher/priority \"%s\"\n", p); + if (!Expand_check_tlsvar(tls_require_ciphers, errstr)) + return DEFER; + if (state->exp_tls_require_ciphers && *state->exp_tls_require_ciphers) + { + p = state->exp_tls_require_ciphers; + DEBUG(D_tls) debug_printf("GnuTLS session cipher/priority \"%s\"\n", p); + } } + + if ((rc = creds_load_pristring(state, p, &errpos))) + return tls_error_gnu(string_sprintf( + "gnutls_priority_init(%s) failed at offset %ld, \"%.6s..\"", + p, errpos - CS p, errpos), + rc, host, errstr); } -if (!p) +else { - p = exim_default_gnutls_priority; - DEBUG(D_tls) - debug_printf("GnuTLS using default session cipher/priority \"%s\"\n", p); + DEBUG(D_tls) debug_printf("cipher list preloaded\n"); + state->exp_tls_require_ciphers = US state->tls_require_ciphers; } -if ((rc = gnutls_priority_init(&state->priority_cache, CCS p, &errpos))) - return tls_error_gnu(string_sprintf( - "gnutls_priority_init(%s) failed at offset %ld, \"%.6s..\"", - p, errpos - CS p, errpos), - rc, host, errstr); -if ((rc = gnutls_priority_set(state->session, state->priority_cache))) +if ((rc = gnutls_priority_set(state->session, state->lib_state.pri_cache))) return tls_error_gnu(US"gnutls_priority_set", rc, host, errstr); /* This also sets the server ticket expiration time to the same, and @@ -2196,7 +2561,7 @@ if (rc != GNUTLS_E_SUCCESS) { DEBUG(D_tls) if (rc == GNUTLS_E_REQUESTED_DATA_NOT_AVAILABLE) - debug_printf("TLS: no SNI presented in handshake.\n"); + debug_printf("TLS: no SNI presented in handshake\n"); else debug_printf("TLS failure: gnutls_server_name_get(): %s [%d]\n", gnutls_strerror(rc), rc); @@ -2407,7 +2772,6 @@ the STARTTLS command. It must respond to that command, and then negotiate a TLS session. Arguments: - require_ciphers list of allowed ciphers or NULL errstr pointer to error string Returns: OK on success @@ -2417,7 +2781,7 @@ Returns: OK on success */ int -tls_server_start(const uschar * require_ciphers, uschar ** errstr) +tls_server_start(uschar ** errstr) { int rc; exim_gnutls_state_st * state = NULL; @@ -2441,9 +2805,8 @@ DEBUG(D_tls) debug_printf("initialising GnuTLS as a server\n"); gettimeofday(&t0, NULL); #endif - if ((rc = tls_init(NULL, tls_certificate, tls_privatekey, - NULL, tls_verify_certificates, tls_crl, - require_ciphers, &state, &tls_in, errstr)) != OK) return rc; + if ((rc = tls_init(NULL, NULL, + tls_require_ciphers, &state, &tls_in, errstr)) != OK) return rc; #ifdef MEASURE_TIMING report_time_since(&t0, US"server tls_init (delta)"); @@ -2460,21 +2823,21 @@ optional, set up appropriately. */ if (verify_check_host(&tls_verify_hosts) == OK) { DEBUG(D_tls) - debug_printf("TLS: a client certificate will be required.\n"); + debug_printf("TLS: a client certificate will be required\n"); state->verify_requirement = VERIFY_REQUIRED; gnutls_certificate_server_set_request(state->session, GNUTLS_CERT_REQUIRE); } else if (verify_check_host(&tls_try_verify_hosts) == OK) { DEBUG(D_tls) - debug_printf("TLS: a client certificate will be requested but not required.\n"); + debug_printf("TLS: a client certificate will be requested but not required\n"); state->verify_requirement = VERIFY_OPTIONAL; gnutls_certificate_server_set_request(state->session, GNUTLS_CERT_REQUEST); } else { DEBUG(D_tls) - debug_printf("TLS: a client certificate will not be requested.\n"); + debug_printf("TLS: a client certificate will not be requested\n"); state->verify_requirement = VERIFY_NONE; gnutls_certificate_server_set_request(state->session, GNUTLS_CERT_IGNORE); } @@ -2484,7 +2847,7 @@ if (event_action) { state->event_action = event_action; gnutls_session_set_ptr(state->session, state); - gnutls_certificate_set_verify_function(state->x509_cred, verify_cb); + gnutls_certificate_set_verify_function(state->lib_state.x509_cred, verify_cb); } #endif @@ -2541,7 +2904,8 @@ if (rc != GNUTLS_E_SUCCESS) tls_error_gnu(US"gnutls_handshake", rc, NULL, errstr); (void) gnutls_alert_send_appropriate(state->session, rc); gnutls_deinit(state->session); - gnutls_certificate_free_credentials(state->x509_cred); + gnutls_certificate_free_credentials(state->lib_state.x509_cred); + state->lib_state = null_tls_preload; millisleep(500); shutdown(state->fd_out, SHUT_WR); for (int i = 1024; fgetc(smtp_in) != EOF && i > 0; ) i--; /* drain skt */ @@ -2614,7 +2978,7 @@ if (verify_check_given_host(CUSS &ob->tls_verify_cert_hostnames, host) == OK) host->certname; #endif DEBUG(D_tls) - debug_printf("TLS: server cert verification includes hostname: \"%s\".\n", + debug_printf("TLS: server cert verification includes hostname: \"%s\"\n", state->exp_tls_verify_cert_hostnames); } } @@ -2875,7 +3239,7 @@ be requested and supplied, dane verify must pass, and cert verify irrelevant if (conn_args->dane && ob->dane_require_tls_ciphers) { - /* not using expand_check_tlsvar because not yet in state */ + /* not using Expand_check_tlsvar because not yet in state */ if (!expand_check(ob->dane_require_tls_ciphers, US"dane_require_tls_ciphers", &cipher_list, errstr)) return FALSE; @@ -2893,12 +3257,9 @@ if (!cipher_list) gettimeofday(&t0, NULL); #endif - if (tls_init(host, ob->tls_certificate, ob->tls_privatekey, - ob->tls_sni, ob->tls_verify_certificates, ob->tls_crl, - cipher_list, &state, tlsp, errstr) != OK) + if (tls_init(host, ob, cipher_list, &state, tlsp, errstr) != OK) return FALSE; - #ifdef MEASURE_TIMING report_time_since(&t0, US"client tls_init (delta)"); #endif @@ -2929,7 +3290,7 @@ the specified host patterns if one of them is defined */ if (conn_args->dane && dane_tlsa_load(state, &conn_args->tlsa_dnsa)) { DEBUG(D_tls) - debug_printf("TLS: server certificate DANE required.\n"); + debug_printf("TLS: server certificate DANE required\n"); state->verify_requirement = VERIFY_DANE; gnutls_certificate_server_set_request(state->session, GNUTLS_CERT_REQUIRE); } @@ -2944,7 +3305,7 @@ else { tls_client_setup_hostname_checks(host, state, ob); DEBUG(D_tls) - debug_printf("TLS: server certificate verification required.\n"); + debug_printf("TLS: server certificate verification required\n"); state->verify_requirement = VERIFY_REQUIRED; gnutls_certificate_server_set_request(state->session, GNUTLS_CERT_REQUIRE); } @@ -2952,14 +3313,14 @@ else if (verify_check_given_host(CUSS &ob->tls_try_verify_hosts, host) == OK) { tls_client_setup_hostname_checks(host, state, ob); DEBUG(D_tls) - debug_printf("TLS: server certificate verification optional.\n"); + debug_printf("TLS: server certificate verification optional\n"); state->verify_requirement = VERIFY_OPTIONAL; gnutls_certificate_server_set_request(state->session, GNUTLS_CERT_REQUEST); } else { DEBUG(D_tls) - debug_printf("TLS: server certificate verification not required.\n"); + debug_printf("TLS: server certificate verification not required\n"); state->verify_requirement = VERIFY_NONE; gnutls_certificate_server_set_request(state->session, GNUTLS_CERT_IGNORE); } @@ -2988,7 +3349,7 @@ if (tb && tb->event_action) { state->event_action = tb->event_action; gnutls_session_set_ptr(state->session, state); - gnutls_certificate_set_verify_function(state->x509_cred, verify_cb); + gnutls_certificate_set_verify_function(state->lib_state.x509_cred, verify_cb); } #endif @@ -3140,7 +3501,8 @@ if (!ct_ctx) /* server */ } gnutls_deinit(state->session); -gnutls_certificate_free_credentials(state->x509_cred); +gnutls_certificate_free_credentials(state->lib_state.x509_cred); +state->lib_state = null_tls_preload; tlsp->active.sock = -1; tlsp->active.tls_ctx = NULL; @@ -3149,7 +3511,6 @@ tlsp->channelbinding = NULL; if (state->xfer_buffer) store_free(state->xfer_buffer); -memcpy(state, &exim_gnutls_state_init, sizeof(exim_gnutls_state_init)); } @@ -3507,7 +3868,7 @@ if (i < needed_len) i = gnutls_rnd(GNUTLS_RND_NONCE, smallbuf, needed_len); if (i < 0) { - DEBUG(D_all) debug_printf("gnutls_rnd() failed, using fallback.\n"); + DEBUG(D_all) debug_printf("gnutls_rnd() failed, using fallback\n"); return vaguely_random_number_fallback(max); } r = 0; diff --git a/src/src/tls-openssl.c b/src/src/tls-openssl.c index 6ce20f143..80485a44f 100644 --- a/src/src/tls-openssl.c +++ b/src/src/tls-openssl.c @@ -274,6 +274,7 @@ static exim_openssl_option exim_openssl_options[] = { #ifndef MACRO_PREDEF static int exim_openssl_options_size = nelem(exim_openssl_options); +static long init_options = 0; #endif #ifdef MACRO_PREDEF @@ -350,8 +351,9 @@ typedef struct { gstring * corked; } exim_openssl_client_tls_ctx; -static SSL_CTX *server_ctx = NULL; -static SSL *server_ssl = NULL; + +/* static SSL_CTX *server_ctx = NULL; */ +/* static SSL *server_ssl = NULL; */ #ifdef EXIM_HAVE_OPENSSL_TLSEXT static SSL_CTX *server_sni = NULL; @@ -371,7 +373,11 @@ typedef struct ocsp_resp { OCSP_RESPONSE * resp; } ocsp_resplist; -typedef struct tls_ext_ctx_cb { +typedef struct exim_openssl_state { + exim_tlslib_state lib_state; +#define lib_ctx libdata0 +#define lib_ssl libdata1 + tls_support * tlsp; uschar * certificate; uschar * privatekey; @@ -399,17 +405,17 @@ typedef struct tls_ext_ctx_cb { #ifndef DISABLE_EVENT uschar * event_action; #endif -} tls_ext_ctx_cb; +} exim_openssl_state_st; /* should figure out a cleanup of API to handle state preserved per implementation, for various reasons, which can be void * in the APIs. For now, we hack around it. */ -tls_ext_ctx_cb *client_static_cbinfo = NULL; /*XXX should not use static; multiple concurrent clients! */ -tls_ext_ctx_cb *server_static_cbinfo = NULL; +exim_openssl_state_st *client_static_state = NULL; /*XXX should not use static; multiple concurrent clients! */ +exim_openssl_state_st state_server = {.is_server = TRUE}; static int -setup_certs(SSL_CTX *sctx, uschar *certs, uschar *crl, host_item *host, BOOL optional, - int (*cert_vfy_cb)(int, X509_STORE_CTX *), uschar ** errstr ); +setup_certs(SSL_CTX *sctx, uschar *certs, uschar *crl, host_item *host, + uschar ** errstr ); /* Callbacks */ #ifdef EXIM_HAVE_OPENSSL_TLSEXT @@ -427,13 +433,20 @@ static void tk_init(void); static int tls_exdata_idx = -1; #endif -void -tls_daemon_init(void) +static void +tls_per_lib_daemon_tick(void) { #ifndef DISABLE_TLS_RESUME tk_init(); #endif -return; +} + +/* Called once at daemon startup */ + +static void +tls_per_lib_daemon_init(void) +{ +tls_daemon_creds_reload(); } @@ -475,8 +488,270 @@ return host ? FAIL : DEFER; +/************************************************** +* General library initalisation * +**************************************************/ + +static BOOL +lib_rand_init(void * addr) +{ +randstuff r; +if (!RAND_status()) return TRUE; + +gettimeofday(&r.tv, NULL); +r.p = getpid(); +RAND_seed(US (&r), sizeof(r)); +RAND_seed(US big_buffer, big_buffer_size); +if (addr) RAND_seed(US addr, sizeof(addr)); + +return RAND_status(); +} + + +static void +tls_openssl_init(void) +{ +static BOOL once = FALSE; +if (once) return; +once = TRUE; + +#ifdef EXIM_NEED_OPENSSL_INIT +SSL_load_error_strings(); /* basic set up */ +OpenSSL_add_ssl_algorithms(); +#endif + +#if defined(EXIM_HAVE_SHA256) && !defined(OPENSSL_AUTO_SHA256) +/* SHA256 is becoming ever more popular. This makes sure it gets added to the +list of available digests. */ +EVP_add_digest(EVP_sha256()); +#endif + +(void) lib_rand_init(NULL); +(void) tls_openssl_options_parse(openssl_options, &init_options); +} + + + +/************************************************* +* Initialize for DH * +*************************************************/ + +/* If dhparam is set, expand it, and load up the parameters for DH encryption. + +Arguments: + sctx The current SSL CTX (inbound or outbound) + dhparam DH parameter file or fixed parameter identity string + host connected host, if client; NULL if server + errstr error string pointer + +Returns: TRUE if OK (nothing to set up, or setup worked) +*/ + +static BOOL +init_dh(SSL_CTX *sctx, uschar *dhparam, const host_item *host, uschar ** errstr) +{ +BIO *bio; +DH *dh; +uschar *dhexpanded; +const char *pem; +int dh_bitsize; + +if (!expand_check(dhparam, US"tls_dhparam", &dhexpanded, errstr)) + return FALSE; + +if (!dhexpanded || !*dhexpanded) + bio = BIO_new_mem_buf(CS std_dh_prime_default(), -1); +else if (dhexpanded[0] == '/') + { + if (!(bio = BIO_new_file(CS dhexpanded, "r"))) + { + tls_error(string_sprintf("could not read dhparams file %s", dhexpanded), + host, US strerror(errno), errstr); + return FALSE; + } + } +else + { + if (Ustrcmp(dhexpanded, "none") == 0) + { + DEBUG(D_tls) debug_printf("Requested no DH parameters.\n"); + return TRUE; + } + + if (!(pem = std_dh_prime_named(dhexpanded))) + { + tls_error(string_sprintf("Unknown standard DH prime \"%s\"", dhexpanded), + host, US strerror(errno), errstr); + return FALSE; + } + bio = BIO_new_mem_buf(CS pem, -1); + } + +if (!(dh = PEM_read_bio_DHparams(bio, NULL, NULL, NULL))) + { + BIO_free(bio); + tls_error(string_sprintf("Could not read tls_dhparams \"%s\"", dhexpanded), + host, NULL, errstr); + return FALSE; + } + +/* note: our default limit of 2236 is not a multiple of 8; the limit comes from + * an NSS limit, and the GnuTLS APIs handle bit-sizes fine, so we went with + * 2236. But older OpenSSL can only report in bytes (octets), not bits. + * If someone wants to dance at the edge, then they can raise the limit or use + * current libraries. */ +#ifdef EXIM_HAVE_OPENSSL_DH_BITS +/* Added in commit 26c79d5641d; `git describe --contains` says OpenSSL_1_1_0-pre1~1022 + * This predates OpenSSL_1_1_0 (before a, b, ...) so is in all 1.1.0 */ +dh_bitsize = DH_bits(dh); +#else +dh_bitsize = 8 * DH_size(dh); +#endif + +/* Even if it is larger, we silently return success rather than cause things + * to fail out, so that a too-large DH will not knock out all TLS; it's a + * debatable choice. */ +if (dh_bitsize > tls_dh_max_bits) + { + DEBUG(D_tls) + debug_printf("dhparams file %d bits, is > tls_dh_max_bits limit of %d\n", + dh_bitsize, tls_dh_max_bits); + } +else + { + SSL_CTX_set_tmp_dh(sctx, dh); + DEBUG(D_tls) + debug_printf("Diffie-Hellman initialized from %s with %d-bit prime\n", + dhexpanded ? dhexpanded : US"default", dh_bitsize); + } + +DH_free(dh); +BIO_free(bio); + +return TRUE; +} + + + + +/************************************************* +* Initialize for ECDH * +*************************************************/ + +/* Load parameters for ECDH encryption. + +For now, we stick to NIST P-256 because: it's simple and easy to configure; +it avoids any patent issues that might bite redistributors; despite events in +the news and concerns over curve choices, we're not cryptographers, we're not +pretending to be, and this is "good enough" to be better than no support, +protecting against most adversaries. Given another year or two, there might +be sufficient clarity about a "right" way forward to let us make an informed +decision, instead of a knee-jerk reaction. + +Longer-term, we should look at supporting both various named curves and +external files generated with "openssl ecparam", much as we do for init_dh(). +We should also support "none" as a value, to explicitly avoid initialisation. + +Patches welcome. + +Arguments: + sctx The current SSL CTX (inbound or outbound) + host connected host, if client; NULL if server + errstr error string pointer + +Returns: TRUE if OK (nothing to set up, or setup worked) +*/ + +static BOOL +init_ecdh(SSL_CTX * sctx, host_item * host, uschar ** errstr) +{ +#ifdef OPENSSL_NO_ECDH +return TRUE; +#else + +EC_KEY * ecdh; +uschar * exp_curve; +int nid; +BOOL rv; + +if (host) /* No ECDH setup for clients, only for servers */ + return TRUE; + +# ifndef EXIM_HAVE_ECDH +DEBUG(D_tls) + debug_printf("No OpenSSL API to define ECDH parameters, skipping\n"); +return TRUE; +# else + +if (!expand_check(tls_eccurve, US"tls_eccurve", &exp_curve, errstr)) + return FALSE; +if (!exp_curve || !*exp_curve) + return TRUE; + +/* "auto" needs to be handled carefully. + * OpenSSL < 1.0.2: we do not select anything, but fallback to prime256v1 + * OpenSSL < 1.1.0: we have to call SSL_CTX_set_ecdh_auto + * (openssl/ssl.h defines SSL_CTRL_SET_ECDH_AUTO) + * OpenSSL >= 1.1.0: we do not set anything, the libray does autoselection + * https://github.com/openssl/openssl/commit/fe6ef2472db933f01b59cad82aa925736935984b + */ +if (Ustrcmp(exp_curve, "auto") == 0) + { +#if OPENSSL_VERSION_NUMBER < 0x10002000L + DEBUG(D_tls) debug_printf( + "ECDH OpenSSL < 1.0.2: temp key parameter settings: overriding \"auto\" with \"prime256v1\"\n"); + exp_curve = US"prime256v1"; +#else +# if defined SSL_CTRL_SET_ECDH_AUTO + DEBUG(D_tls) debug_printf( + "ECDH OpenSSL 1.0.2+ temp key parameter settings: autoselection\n"); + SSL_CTX_set_ecdh_auto(sctx, 1); + return TRUE; +# else + DEBUG(D_tls) debug_printf( + "ECDH OpenSSL 1.1.0+ temp key parameter settings: default selection\n"); + return TRUE; +# endif +#endif + } + +DEBUG(D_tls) debug_printf("ECDH: curve '%s'\n", exp_curve); +if ( (nid = OBJ_sn2nid (CCS exp_curve)) == NID_undef +# ifdef EXIM_HAVE_OPENSSL_EC_NIST2NID + && (nid = EC_curve_nist2nid(CCS exp_curve)) == NID_undef +# endif + ) + { + tls_error(string_sprintf("Unknown curve name tls_eccurve '%s'", exp_curve), + host, NULL, errstr); + return FALSE; + } + +if (!(ecdh = EC_KEY_new_by_curve_name(nid))) + { + tls_error(US"Unable to create ec curve", host, NULL, errstr); + return FALSE; + } + +/* The "tmp" in the name here refers to setting a temporary key +not to the stability of the interface. */ + +if ((rv = SSL_CTX_set_tmp_ecdh(sctx, ecdh) == 0)) + tls_error(string_sprintf("Error enabling '%s' curve", exp_curve), host, NULL, errstr); +else + DEBUG(D_tls) debug_printf("ECDH: enabled '%s' curve\n", exp_curve); + +EC_KEY_free(ecdh); +return !rv; + +# endif /*EXIM_HAVE_ECDH*/ +#endif /*OPENSSL_NO_ECDH*/ +} + + + /************************************************* -* Callback to generate RSA key * +* Expand key and cert file specs * *************************************************/ /* @@ -518,30 +793,139 @@ return rsa_key; -/* Extreme debug -#ifndef DISABLE_OCSP -void -x509_store_dump_cert_s_names(X509_STORE * store) +/* Create and install a selfsigned certificate, for use in server mode */ + +static int +tls_install_selfsign(SSL_CTX * sctx, uschar ** errstr) { -STACK_OF(X509_OBJECT) * roots= store->objs; -static uschar name[256]; +X509 * x509 = NULL; +EVP_PKEY * pkey; +RSA * rsa; +X509_NAME * name; +uschar * where; -for (int i= 0; i < sk_X509_OBJECT_num(roots); i++) +where = US"allocating pkey"; +if (!(pkey = EVP_PKEY_new())) + goto err; + +where = US"allocating cert"; +if (!(x509 = X509_new())) + goto err; + +where = US"generating pkey"; +if (!(rsa = rsa_callback(NULL, 0, 2048))) + goto err; + +where = US"assigning pkey"; +if (!EVP_PKEY_assign_RSA(pkey, rsa)) + goto err; + +X509_set_version(x509, 2); /* N+1 - version 3 */ +ASN1_INTEGER_set(X509_get_serialNumber(x509), 1); +X509_gmtime_adj(X509_get_notBefore(x509), 0); +X509_gmtime_adj(X509_get_notAfter(x509), (long)60 * 60); /* 1 hour */ +X509_set_pubkey(x509, pkey); + +name = X509_get_subject_name(x509); +X509_NAME_add_entry_by_txt(name, "C", + MBSTRING_ASC, CUS "UK", -1, -1, 0); +X509_NAME_add_entry_by_txt(name, "O", + MBSTRING_ASC, CUS "Exim Developers", -1, -1, 0); +X509_NAME_add_entry_by_txt(name, "CN", + MBSTRING_ASC, CUS smtp_active_hostname, -1, -1, 0); +X509_set_issuer_name(x509, name); + +where = US"signing cert"; +if (!X509_sign(x509, pkey, EVP_md5())) + goto err; + +where = US"installing selfsign cert"; +if (!SSL_CTX_use_certificate(sctx, x509)) + goto err; + +where = US"installing selfsign key"; +if (!SSL_CTX_use_PrivateKey(sctx, pkey)) + goto err; + +return OK; + +err: + (void) tls_error(where, NULL, NULL, errstr); + if (x509) X509_free(x509); + if (pkey) EVP_PKEY_free(pkey); + return DEFER; +} + + + + + + + +/************************************************* +* Information callback * +*************************************************/ + +/* The SSL library functions call this from time to time to indicate what they +are doing. We copy the string to the debugging output when TLS debugging has +been requested. + +Arguments: + s the SSL connection + where + ret + +Returns: nothing +*/ + +static void +info_callback(SSL *s, int where, int ret) +{ +DEBUG(D_tls) { - X509_OBJECT * tmp_obj= sk_X509_OBJECT_value(roots, i); - if(tmp_obj->type == X509_LU_X509) - { - X509_NAME * sn = X509_get_subject_name(tmp_obj->data.x509); - if (X509_NAME_oneline(sn, CS name, sizeof(name))) - { - name[sizeof(name)-1] = '\0'; - debug_printf(" %s\n", name); - } - } + const uschar * str; + + if (where & SSL_ST_CONNECT) + str = US"SSL_connect"; + else if (where & SSL_ST_ACCEPT) + str = US"SSL_accept"; + else + str = US"SSL info (undefined)"; + + if (where & SSL_CB_LOOP) + debug_printf("%s: %s\n", str, SSL_state_string_long(s)); + else if (where & SSL_CB_ALERT) + debug_printf("SSL3 alert %s:%s:%s\n", + str = where & SSL_CB_READ ? US"read" : US"write", + SSL_alert_type_string_long(ret), SSL_alert_desc_string_long(ret)); + else if (where & SSL_CB_EXIT) + if (ret == 0) + debug_printf("%s: failed in %s\n", str, SSL_state_string_long(s)); + else if (ret < 0) + debug_printf("%s: error in %s\n", str, SSL_state_string_long(s)); + else if (where & SSL_CB_HANDSHAKE_START) + debug_printf("%s: hshake start: %s\n", str, SSL_state_string_long(s)); + else if (where & SSL_CB_HANDSHAKE_DONE) + debug_printf("%s: hshake done: %s\n", str, SSL_state_string_long(s)); } } + +#ifdef OPENSSL_HAVE_KEYLOG_CB +static void +keylog_callback(const SSL *ssl, const char *line) +{ +char * filename; +FILE * fp; +DEBUG(D_tls) debug_printf("%.200s\n", line); +if (!(filename = getenv("SSLKEYLOGFILE"))) return; +if (!(fp = fopen(filename, "a"))) return; +fprintf(fp, "%s\n", line); +fclose(fp); +} #endif -*/ + + + #ifndef DISABLE_EVENT @@ -553,7 +937,7 @@ uschar * ev; uschar * yield; X509 * old_cert; -ev = tlsp == &tls_out ? client_static_cbinfo->event_action : event_action; +ev = tlsp == &tls_out ? client_static_state->event_action : event_action; if (ev) { DEBUG(D_tls) debug_printf("verify_event: %s %d\n", what, depth); @@ -660,15 +1044,15 @@ else if (depth != 0) { DEBUG(D_tls) debug_printf("SSL verify ok: depth=%d SN=%s\n", depth, dn); #ifndef DISABLE_OCSP - if (tlsp == &tls_out && client_static_cbinfo->u_ocsp.client.verify_store) + if (tlsp == &tls_out && client_static_state->u_ocsp.client.verify_store) { /* client, wanting stapling */ /* Add the server cert's signing chain as the one for the verification of the OCSP stapled information. */ - if (!X509_STORE_add_cert(client_static_cbinfo->u_ocsp.client.verify_store, + if (!X509_STORE_add_cert(client_static_state->u_ocsp.client.verify_store, cert)) ERR_clear_error(); - sk_X509_push(client_static_cbinfo->verify_stack, cert); + sk_X509_push(client_static_state->verify_stack, cert); } #endif #ifndef DISABLE_EVENT @@ -681,7 +1065,7 @@ else const uschar * verify_cert_hostnames; if ( tlsp == &tls_out - && ((verify_cert_hostnames = client_static_cbinfo->verify_cert_hostnames))) + && ((verify_cert_hostnames = client_static_state->verify_cert_hostnames))) /* client, wanting hostname check */ { @@ -801,15 +1185,15 @@ if (preverify_ok == 1) { tls_out.dane_verified = TRUE; #ifndef DISABLE_OCSP - if (client_static_cbinfo->u_ocsp.client.verify_store) + if (client_static_state->u_ocsp.client.verify_store) { /* client, wanting stapling */ /* Add the server cert's signing chain as the one for the verification of the OCSP stapled information. */ - if (!X509_STORE_add_cert(client_static_cbinfo->u_ocsp.client.verify_store, + if (!X509_STORE_add_cert(client_static_state->u_ocsp.client.verify_store, cert)) ERR_clear_error(); - sk_X509_push(client_static_cbinfo->verify_stack, cert); + sk_X509_push(client_static_state->verify_stack, cert); } #endif } @@ -827,836 +1211,999 @@ return preverify_ok; #endif /*SUPPORT_DANE*/ +#ifndef DISABLE_OCSP /************************************************* -* Information callback * +* Load OCSP information into state * *************************************************/ +/* Called to load the server OCSP response from the given file into memory, once +caller has determined this is needed. Checks validity. Debugs a message +if invalid. -/* The SSL library functions call this from time to time to indicate what they -are doing. We copy the string to the debugging output when TLS debugging has -been requested. +ASSUMES: single response, for single cert. Arguments: - s the SSL connection - where - ret - -Returns: nothing + state various parts of session state + filename the filename putatively holding an OCSP response + is_pem file is PEM format; otherwise is DER */ static void -info_callback(SSL *s, int where, int ret) +ocsp_load_response(exim_openssl_state_st * state, const uschar * filename, + BOOL is_pem) { +BIO * bio; +OCSP_RESPONSE * resp; +OCSP_BASICRESP * basic_response; +OCSP_SINGLERESP * single_response; +ASN1_GENERALIZEDTIME * rev, * thisupd, * nextupd; +STACK_OF(X509) * sk; +unsigned long verify_flags; +int status, reason, i; + DEBUG(D_tls) + debug_printf("tls_ocsp_file (%s) '%s'\n", is_pem ? "PEM" : "DER", filename); + +if (!(bio = BIO_new_file(CS filename, "rb"))) { - const uschar * str; + DEBUG(D_tls) debug_printf("Failed to open OCSP response file \"%s\"\n", + filename); + return; + } - if (where & SSL_ST_CONNECT) - str = US"SSL_connect"; - else if (where & SSL_ST_ACCEPT) - str = US"SSL_accept"; - else - str = US"SSL info (undefined)"; +if (is_pem) + { + uschar * data, * freep; + char * dummy; + long len; + if (!PEM_read_bio(bio, &dummy, &dummy, &data, &len)) + { + DEBUG(D_tls) debug_printf("Failed to read PEM file \"%s\"\n", + filename); + return; + } + freep = data; + resp = d2i_OCSP_RESPONSE(NULL, CUSS &data, len); + OPENSSL_free(freep); + } +else + resp = d2i_OCSP_RESPONSE_bio(bio, NULL); +BIO_free(bio); - if (where & SSL_CB_LOOP) - debug_printf("%s: %s\n", str, SSL_state_string_long(s)); - else if (where & SSL_CB_ALERT) - debug_printf("SSL3 alert %s:%s:%s\n", - str = where & SSL_CB_READ ? US"read" : US"write", - SSL_alert_type_string_long(ret), SSL_alert_desc_string_long(ret)); - else if (where & SSL_CB_EXIT) - if (ret == 0) - debug_printf("%s: failed in %s\n", str, SSL_state_string_long(s)); - else if (ret < 0) - debug_printf("%s: error in %s\n", str, SSL_state_string_long(s)); - else if (where & SSL_CB_HANDSHAKE_START) - debug_printf("%s: hshake start: %s\n", str, SSL_state_string_long(s)); - else if (where & SSL_CB_HANDSHAKE_DONE) - debug_printf("%s: hshake done: %s\n", str, SSL_state_string_long(s)); +if (!resp) + { + DEBUG(D_tls) debug_printf("Error reading OCSP response.\n"); + return; } -} -#ifdef OPENSSL_HAVE_KEYLOG_CB -static void -keylog_callback(const SSL *ssl, const char *line) -{ -char * filename; -FILE * fp; -DEBUG(D_tls) debug_printf("%.200s\n", line); -if (!(filename = getenv("SSLKEYLOGFILE"))) return; -if (!(fp = fopen(filename, "a"))) return; -fprintf(fp, "%s\n", line); -fclose(fp); -} +if ((status = OCSP_response_status(resp)) != OCSP_RESPONSE_STATUS_SUCCESSFUL) + { + DEBUG(D_tls) debug_printf("OCSP response not valid: %s (%d)\n", + OCSP_response_status_str(status), status); + goto bad; + } + +#ifdef notdef + { + BIO * bp = BIO_new_fp(debug_file, BIO_NOCLOSE); + OCSP_RESPONSE_print(bp, resp, 0); /* extreme debug: stapling content */ + BIO_free(bp); + } #endif +if (!(basic_response = OCSP_response_get1_basic(resp))) + { + DEBUG(D_tls) + debug_printf("OCSP response parse error: unable to extract basic response.\n"); + goto bad; + } -#ifndef DISABLE_TLS_RESUME -/* Manage the keysets used for encrypting the session tickets, on the server. */ +sk = state->verify_stack; +verify_flags = OCSP_NOVERIFY; /* check sigs, but not purpose */ -typedef struct { /* Session ticket encryption key */ - uschar name[16]; +/* May need to expose ability to adjust those flags? +OCSP_NOSIGS OCSP_NOVERIFY OCSP_NOCHAIN OCSP_NOCHECKS OCSP_NOEXPLICIT +OCSP_TRUSTOTHER OCSP_NOINTERN */ - const EVP_CIPHER * aes_cipher; - uschar aes_key[32]; /* size needed depends on cipher. aes_128 implies 128/8 = 16? */ - const EVP_MD * hmac_hash; - uschar hmac_key[16]; - time_t renew; - time_t expire; -} exim_stek; +/* This does a full verify on the OCSP proof before we load it for serving +up; possibly overkill - just date-checks might be nice enough. -static exim_stek exim_tk; /* current key */ -static exim_stek exim_tk_old; /* previous key */ +OCSP_basic_verify takes a "store" arg, but does not +use it for the chain verification, which is all we do +when OCSP_NOVERIFY is set. The content from the wire +"basic_response" and a cert-stack "sk" are all that is used. -static void -tk_init(void) -{ -time_t t = time(NULL); +We have a stack, loaded in setup_certs() if tls_verify_certificates +was a file (not a directory, or "system"). It is unfortunate we +cannot used the connection context store, as that would neatly +handle the "system" case too, but there seems to be no library +function for getting a stack from a store. +[ In OpenSSL 1.1 - ? X509_STORE_CTX_get0_chain(ctx) ? ] +We do not free the stack since it could be needed a second time for +SNI handling. -if (exim_tk.name[0]) +Separately we might try to replace using OCSP_basic_verify() - which seems to not +be a public interface into the OpenSSL library (there's no manual entry) - +But what with? We also use OCSP_basic_verify in the client stapling callback. +And there we NEED it; we must verify that status... unless the +library does it for us anyway? */ + +if ((i = OCSP_basic_verify(basic_response, sk, NULL, verify_flags)) < 0) { - if (exim_tk.renew >= t) return; - exim_tk_old = exim_tk; + DEBUG(D_tls) + { + ERR_error_string_n(ERR_get_error(), ssl_errstring, sizeof(ssl_errstring)); + debug_printf("OCSP response verify failure: %s\n", US ssl_errstring); + } + goto bad; } -if (f.running_in_test_harness) ssl_session_timeout = 6; +/* Here's the simplifying assumption: there's only one response, for the +one certificate we use, and nothing for anything else in a chain. If this +proves false, we need to extract a cert id from our issued cert +(tls_certificate) and use that for OCSP_resp_find_status() (which finds the +right cert in the stack and then calls OCSP_single_get0_status()). -DEBUG(D_tls) debug_printf("OpenSSL: %s STEK\n", exim_tk.name[0] ? "rotating" : "creating"); -if (RAND_bytes(exim_tk.aes_key, sizeof(exim_tk.aes_key)) <= 0) return; -if (RAND_bytes(exim_tk.hmac_key, sizeof(exim_tk.hmac_key)) <= 0) return; -if (RAND_bytes(exim_tk.name+1, sizeof(exim_tk.name)-1) <= 0) return; +I'm hoping to avoid reworking a bunch more of how we handle state here. -exim_tk.name[0] = 'E'; -exim_tk.aes_cipher = EVP_aes_256_cbc(); -exim_tk.hmac_hash = EVP_sha256(); -exim_tk.expire = t + ssl_session_timeout; -exim_tk.renew = t + ssl_session_timeout/2; +XXX that will change when we add support for (TLS1.3) whole-chain stapling +*/ + +if (!(single_response = OCSP_resp_get0(basic_response, 0))) + { + DEBUG(D_tls) + debug_printf("Unable to get first response from OCSP basic response.\n"); + goto bad; + } + +status = OCSP_single_get0_status(single_response, &reason, &rev, &thisupd, &nextupd); +if (status != V_OCSP_CERTSTATUS_GOOD) + { + DEBUG(D_tls) debug_printf("OCSP response bad cert status: %s (%d) %s (%d)\n", + OCSP_cert_status_str(status), status, + OCSP_crl_reason_str(reason), reason); + goto bad; + } + +if (!OCSP_check_validity(thisupd, nextupd, EXIM_OCSP_SKEW_SECONDS, EXIM_OCSP_MAX_AGE)) + { + DEBUG(D_tls) debug_printf("OCSP status invalid times.\n"); + goto bad; + } + +supply_response: + /* Add the resp to the list used by tls_server_stapling_cb() */ + { + ocsp_resplist ** op = &state->u_ocsp.server.olist, * oentry; + while (oentry = *op) + op = &oentry->next; + *op = oentry = store_get(sizeof(ocsp_resplist), FALSE); + oentry->next = NULL; + oentry->resp = resp; + } +return; + +bad: + if (f.running_in_test_harness) + { + extern char ** environ; + if (environ) for (uschar ** p = USS environ; *p; p++) + if (Ustrncmp(*p, "EXIM_TESTHARNESS_DISABLE_OCSPVALIDITYCHECK", 42) == 0) + { + DEBUG(D_tls) debug_printf("Supplying known bad OCSP response\n"); + goto supply_response; + } + } +return; } -static exim_stek * -tk_current(void) + +static void +ocsp_free_response_list(exim_openssl_state_st * cbinfo) { -if (!exim_tk.name[0]) return NULL; -return &exim_tk; +for (ocsp_resplist * olist = cbinfo->u_ocsp.server.olist; olist; + olist = olist->next) + OCSP_RESPONSE_free(olist->resp); +cbinfo->u_ocsp.server.olist = NULL; } +#endif /*!DISABLE_OCSP*/ -static exim_stek * -tk_find(const uschar * name) + + + + +static int +tls_add_certfile(SSL_CTX * sctx, exim_openssl_state_st * cbinfo, uschar * file, + uschar ** errstr) { -return memcmp(name, exim_tk.name, sizeof(exim_tk.name)) == 0 ? &exim_tk - : memcmp(name, exim_tk_old.name, sizeof(exim_tk_old.name)) == 0 ? &exim_tk_old - : NULL; +DEBUG(D_tls) debug_printf("tls_certificate file '%s'\n", file); +if (!SSL_CTX_use_certificate_chain_file(sctx, CS file)) + return tls_error(string_sprintf( + "SSL_CTX_use_certificate_chain_file file=%s", file), + cbinfo->host, NULL, errstr); +return 0; } -/* Callback for session tickets, on server */ static int -ticket_key_callback(SSL * ssl, uschar key_name[16], - uschar * iv, EVP_CIPHER_CTX * ctx, HMAC_CTX * hctx, int enc) +tls_add_pkeyfile(SSL_CTX * sctx, exim_openssl_state_st * cbinfo, uschar * file, + uschar ** errstr) { -tls_support * tlsp = server_static_cbinfo->tlsp; -exim_stek * key; +DEBUG(D_tls) debug_printf("tls_privatekey file '%s'\n", file); +if (!SSL_CTX_use_PrivateKey_file(sctx, CS file, SSL_FILETYPE_PEM)) + return tls_error(string_sprintf( + "SSL_CTX_use_PrivateKey_file file=%s", file), cbinfo->host, NULL, errstr); +return 0; +} -if (enc) - { - DEBUG(D_tls) debug_printf("ticket_key_callback: create new session\n"); - tlsp->resumption |= RESUME_CLIENT_REQUESTED; - if (RAND_bytes(iv, EVP_MAX_IV_LENGTH) <= 0) - return -1; /* insufficient random */ - if (!(key = tk_current())) /* current key doesn't exist or isn't valid */ - return 0; /* key couldn't be created */ - memcpy(key_name, key->name, 16); - DEBUG(D_tls) debug_printf("STEK expire " TIME_T_FMT "\n", key->expire - time(NULL)); - /*XXX will want these dependent on the ssl session strength */ - HMAC_Init_ex(hctx, key->hmac_key, sizeof(key->hmac_key), - key->hmac_hash, NULL); - EVP_EncryptInit_ex(ctx, key->aes_cipher, NULL, key->aes_key, iv); +/* Called once during tls_init and possibly again during TLS setup, for a +new context, if Server Name Indication was used and tls_sni was seen in +the certificate string. - DEBUG(D_tls) debug_printf("ticket created\n"); - return 1; +Arguments: + sctx the SSL_CTX* to update + state various parts of session state + errstr error string pointer + +Returns: OK/DEFER/FAIL +*/ + +static int +tls_expand_session_files(SSL_CTX * sctx, exim_openssl_state_st * state, + uschar ** errstr) +{ +uschar * expanded; + +if (!state->certificate) + { + if (!state->is_server) /* client */ + return OK; + /* server */ + if (tls_install_selfsign(sctx, errstr) != OK) + return DEFER; } else { - time_t now = time(NULL); + int err; - DEBUG(D_tls) debug_printf("ticket_key_callback: retrieve session\n"); - tlsp->resumption |= RESUME_CLIENT_SUGGESTED; + if ( !reexpand_tls_files_for_sni + && ( Ustrstr(state->certificate, US"tls_sni") + || Ustrstr(state->certificate, US"tls_in_sni") + || Ustrstr(state->certificate, US"tls_out_sni") + ) ) + reexpand_tls_files_for_sni = TRUE; - if (!(key = tk_find(key_name)) || key->expire < now) - { - DEBUG(D_tls) + if (!expand_check(state->certificate, US"tls_certificate", &expanded, errstr)) + return DEFER; + + if (expanded) + if (state->is_server) { - debug_printf("ticket not usable (%s)\n", key ? "expired" : "not found"); - if (key) debug_printf("STEK expire " TIME_T_FMT "\n", key->expire - now); + const uschar * file_list = expanded; + int sep = 0; + uschar * file; +#ifndef DISABLE_OCSP + const uschar * olist = state->u_ocsp.server.file; + int osep = 0; + uschar * ofile; + BOOL fmt_pem = FALSE; + + if (olist) + if (!expand_check(olist, US"tls_ocsp_file", USS &olist, errstr)) + return DEFER; + if (olist && !*olist) + olist = NULL; + + if ( state->u_ocsp.server.file_expanded && olist + && (Ustrcmp(olist, state->u_ocsp.server.file_expanded) == 0)) + { + DEBUG(D_tls) debug_printf(" - value unchanged, using existing values\n"); + olist = NULL; + } + else + { + ocsp_free_response_list(state); + state->u_ocsp.server.file_expanded = olist; + } +#endif + + while (file = string_nextinlist(&file_list, &sep, NULL, 0)) + { + if ((err = tls_add_certfile(sctx, state, file, errstr))) + return err; + +#ifndef DISABLE_OCSP + if (olist) + if ((ofile = string_nextinlist(&olist, &osep, NULL, 0))) + { + if (Ustrncmp(ofile, US"PEM ", 4) == 0) + { + fmt_pem = TRUE; + ofile += 4; + } + else if (Ustrncmp(ofile, US"DER ", 4) == 0) + { + fmt_pem = FALSE; + ofile += 4; + } + ocsp_load_response(state, ofile, fmt_pem); + } + else + DEBUG(D_tls) debug_printf("ran out of ocsp file list\n"); +#endif + } } - return 0; - } + else /* would there ever be a need for multiple client certs? */ + if ((err = tls_add_certfile(sctx, state, expanded, errstr))) + return err; - HMAC_Init_ex(hctx, key->hmac_key, sizeof(key->hmac_key), - key->hmac_hash, NULL); - EVP_DecryptInit_ex(ctx, key->aes_cipher, NULL, key->aes_key, iv); + if ( state->privatekey + && !expand_check(state->privatekey, US"tls_privatekey", &expanded, errstr)) + return DEFER; - DEBUG(D_tls) debug_printf("ticket usable, STEK expire " TIME_T_FMT "\n", key->expire - now); + /* If expansion was forced to fail, key_expanded will be NULL. If the result + of the expansion is an empty string, ignore it also, and assume the private + key is in the same file as the certificate. */ - /* The ticket lifetime and renewal are the same as the STEK lifetime and - renewal, which is overenthusiastic. A factor of, say, 3x longer STEK would - be better. To do that we'd have to encode ticket lifetime in the name as - we don't yet see the restored session. Could check posthandshake for TLS1.3 - and trigger a new ticket then, but cannot do that for TLS1.2 */ - return key->renew < now ? 2 : 1; + if (expanded && *expanded) + if (state->is_server) + { + const uschar * file_list = expanded; + int sep = 0; + uschar * file; + + while (file = string_nextinlist(&file_list, &sep, NULL, 0)) + if ((err = tls_add_pkeyfile(sctx, state, file, errstr))) + return err; + } + else /* would there ever be a need for multiple client certs? */ + if ((err = tls_add_pkeyfile(sctx, state, expanded, errstr))) + return err; } + +return OK; } -#endif -/************************************************* -* Initialize for DH * -*************************************************/ -/* If dhparam is set, expand it, and load up the parameters for DH encryption. +/************************************************** +* One-time init credentials for server and client * +**************************************************/ -Arguments: - sctx The current SSL CTX (inbound or outbound) - dhparam DH parameter file or fixed parameter identity string - host connected host, if client; NULL if server - errstr error string pointer -Returns: TRUE if OK (nothing to set up, or setup worked) -*/ +#ifdef gnutls +static void +creds_basic_init(gnutls_certificate_credentials_t x509_cred, BOOL server) +{ +} +#endif -static BOOL -init_dh(SSL_CTX *sctx, uschar *dhparam, const host_item *host, uschar ** errstr) +static int +creds_load_server_certs(/*exim_gnutls_state_st * state,*/ const uschar * cert, + const uschar * pkey, const uschar * ocsp, uschar ** errstr) { -BIO *bio; -DH *dh; -uschar *dhexpanded; -const char *pem; -int dh_bitsize; +#ifdef gnutls +const uschar * clist = cert; +const uschar * klist = pkey; +const uschar * olist; +int csep = 0, ksep = 0, osep = 0, cnt = 0, rc; +uschar * cfile, * kfile, * ofile; +#ifndef DISABLE_OCSP +# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE +gnutls_x509_crt_fmt_t ocsp_fmt = GNUTLS_X509_FMT_DER; +# endif -if (!expand_check(dhparam, US"tls_dhparam", &dhexpanded, errstr)) - return FALSE; +if (!expand_check(ocsp, US"tls_ocsp_file", &ofile, errstr)) + return DEFER; +olist = ofile; +#endif -if (!dhexpanded || !*dhexpanded) - bio = BIO_new_mem_buf(CS std_dh_prime_default(), -1); -else if (dhexpanded[0] == '/') - { - if (!(bio = BIO_new_file(CS dhexpanded, "r"))) - { - tls_error(string_sprintf("could not read dhparams file %s", dhexpanded), - host, US strerror(errno), errstr); - return FALSE; - } - } -else - { - if (Ustrcmp(dhexpanded, "none") == 0) - { - DEBUG(D_tls) debug_printf("Requested no DH parameters.\n"); - return TRUE; - } +while (cfile = string_nextinlist(&clist, &csep, NULL, 0)) - if (!(pem = std_dh_prime_named(dhexpanded))) + if (!(kfile = string_nextinlist(&klist, &ksep, NULL, 0))) + return tls_error(US"cert/key setup: out of keys", NULL, NULL, errstr); + else if ((rc = tls_add_certfile(state, NULL, cfile, kfile, errstr)) > 0) + return rc; + else { - tls_error(string_sprintf("Unknown standard DH prime \"%s\"", dhexpanded), - host, US strerror(errno), errstr); - return FALSE; + int gnutls_cert_index = -rc; + DEBUG(D_tls) debug_printf("TLS: cert/key %d %s registered\n", + gnutls_cert_index, cfile); + +#ifndef DISABLE_OCSP + if (ocsp) + { + /* Set the OCSP stapling server info */ + if (gnutls_buggy_ocsp) + { + DEBUG(D_tls) + debug_printf("GnuTLS library is buggy for OCSP; avoiding\n"); + } + else if ((ofile = string_nextinlist(&olist, &osep, NULL, 0))) + { + DEBUG(D_tls) debug_printf("OCSP response file %d = %s\n", + gnutls_cert_index, ofile); +# ifdef SUPPORT_GNUTLS_EXT_RAW_PARSE + if (Ustrncmp(ofile, US"PEM ", 4) == 0) + { + ocsp_fmt = GNUTLS_X509_FMT_PEM; + ofile += 4; + } + else if (Ustrncmp(ofile, US"DER ", 4) == 0) + { + ocsp_fmt = GNUTLS_X509_FMT_DER; + ofile += 4; + } + + if ((rc = gnutls_certificate_set_ocsp_status_request_file2( + state->lib_state.x509_cred, CCS ofile, gnutls_cert_index, + ocsp_fmt)) < 0) + return tls_error_gnu( + US"gnutls_certificate_set_ocsp_status_request_file2", + rc, NULL, errstr); + DEBUG(D_tls) + debug_printf(" %d response%s loaded\n", rc, rc>1 ? "s":""); + + /* Arrange callbacks for OCSP request observability */ + + if (state->session) + gnutls_handshake_set_hook_function(state->session, + GNUTLS_HANDSHAKE_ANY, GNUTLS_HOOK_POST, tls_server_hook_cb); + else + state->lib_state.ocsp_hook = TRUE; + + +# else +# if defined(SUPPORT_SRV_OCSP_STACK) + if ((rc = gnutls_certificate_set_ocsp_status_request_function2( + state->lib_state.x509_cred, gnutls_cert_index, + server_ocsp_stapling_cb, ofile))) + return tls_error_gnu( + US"gnutls_certificate_set_ocsp_status_request_function2", + rc, NULL, errstr); + else +# endif + { + if (cnt++ > 0) + { + DEBUG(D_tls) + debug_printf("oops; multiple OCSP files not supported\n"); + break; + } + gnutls_certificate_set_ocsp_status_request_function( + state->lib_state.x509_cred, server_ocsp_stapling_cb, ofile); + } +# endif /* SUPPORT_GNUTLS_EXT_RAW_PARSE */ + } + else + DEBUG(D_tls) debug_printf("ran out of OCSP response files in list\n"); + } +#endif /* DISABLE_OCSP */ } - bio = BIO_new_mem_buf(CS pem, -1); - } +return 0; +#endif /*gnutls*/ +} -if (!(dh = PEM_read_bio_DHparams(bio, NULL, NULL, NULL))) - { - BIO_free(bio); - tls_error(string_sprintf("Could not read tls_dhparams \"%s\"", dhexpanded), - host, NULL, errstr); - return FALSE; - } +static int +creds_load_client_certs(/*exim_gnutls_state_st * state,*/ const host_item * host, + const uschar * cert, const uschar * pkey, uschar ** errstr) +{ +return 0; +} -/* note: our default limit of 2236 is not a multiple of 8; the limit comes from - * an NSS limit, and the GnuTLS APIs handle bit-sizes fine, so we went with - * 2236. But older OpenSSL can only report in bytes (octets), not bits. - * If someone wants to dance at the edge, then they can raise the limit or use - * current libraries. */ -#ifdef EXIM_HAVE_OPENSSL_DH_BITS -/* Added in commit 26c79d5641d; `git describe --contains` says OpenSSL_1_1_0-pre1~1022 - * This predates OpenSSL_1_1_0 (before a, b, ...) so is in all 1.1.0 */ -dh_bitsize = DH_bits(dh); -#else -dh_bitsize = 8 * DH_size(dh); -#endif +static int +creds_load_cabundle(/*exim_gnutls_state_st * state,*/ const uschar * bundle, + const host_item * host, uschar ** errstr) +{ +#ifdef gnutls +int cert_count; +struct stat statbuf; -/* Even if it is larger, we silently return success rather than cause things - * to fail out, so that a too-large DH will not knock out all TLS; it's a - * debatable choice. */ -if (dh_bitsize > tls_dh_max_bits) - { - DEBUG(D_tls) - debug_printf("dhparams file %d bits, is > tls_dh_max_bits limit of %d\n", - dh_bitsize, tls_dh_max_bits); - } +#ifdef SUPPORT_SYSDEFAULT_CABUNDLE +if (Ustrcmp(bundle, "system") == 0 || Ustrncmp(bundle, "system,", 7) == 0) + cert_count = gnutls_certificate_set_x509_system_trust(state->lib_state.x509_cred); else +#endif { - SSL_CTX_set_tmp_dh(sctx, dh); - DEBUG(D_tls) - debug_printf("Diffie-Hellman initialized from %s with %d-bit prime\n", - dhexpanded ? dhexpanded : US"default", dh_bitsize); - } + if (Ustat(bundle, &statbuf) < 0) + { + log_write(0, LOG_MAIN|LOG_PANIC, "could not stat '%s' " + "(tls_verify_certificates): %s", bundle, strerror(errno)); + return DEFER; + } -DH_free(dh); -BIO_free(bio); +#ifndef SUPPORT_CA_DIR + /* The test suite passes in /dev/null; we could check for that path explicitly, + but who knows if someone has some weird FIFO which always dumps some certs, or + other weirdness. The thing we really want to check is that it's not a + directory, since while OpenSSL supports that, GnuTLS does not. + So s/!S_ISREG/S_ISDIR/ and change some messaging ... */ + if (S_ISDIR(statbuf.st_mode)) + { + log_write(0, LOG_MAIN|LOG_PANIC, + "tls_verify_certificates \"%s\" is a directory", bundle); + return DEFER; + } +#endif -return TRUE; -} + DEBUG(D_tls) debug_printf("verify certificates = %s size=" OFF_T_FMT "\n", + bundle, statbuf.st_size); + if (statbuf.st_size == 0) + { + DEBUG(D_tls) + debug_printf("cert file empty, no certs, no verification, ignoring any CRL\n"); + return OK; + } + cert_count = +#ifdef SUPPORT_CA_DIR + (statbuf.st_mode & S_IFMT) == S_IFDIR + ? + gnutls_certificate_set_x509_trust_dir(state->lib_state.x509_cred, + CS bundle, GNUTLS_X509_FMT_PEM) + : +#endif + gnutls_certificate_set_x509_trust_file(state->lib_state.x509_cred, + CS bundle, GNUTLS_X509_FMT_PEM); -/************************************************* -* Initialize for ECDH * -*************************************************/ +#ifdef SUPPORT_CA_DIR + /* Mimic the behaviour with OpenSSL of not advertising a usable-cert list + when using the directory-of-certs config model. */ -/* Load parameters for ECDH encryption. + if ((statbuf.st_mode & S_IFMT) == S_IFDIR) + if (state->session) + gnutls_certificate_send_x509_rdn_sequence(state->session, 1); + else + state->lib_state.ca_rdn_emulate = TRUE; +#endif + } -For now, we stick to NIST P-256 because: it's simple and easy to configure; -it avoids any patent issues that might bite redistributors; despite events in -the news and concerns over curve choices, we're not cryptographers, we're not -pretending to be, and this is "good enough" to be better than no support, -protecting against most adversaries. Given another year or two, there might -be sufficient clarity about a "right" way forward to let us make an informed -decision, instead of a knee-jerk reaction. +if (cert_count < 0) + return tls_error_gnu(US"setting certificate trust", cert_count, host, errstr); +DEBUG(D_tls) + debug_printf("Added %d certificate authorities\n", cert_count); -Longer-term, we should look at supporting both various named curves and -external files generated with "openssl ecparam", much as we do for init_dh(). -We should also support "none" as a value, to explicitly avoid initialisation. +#endif /*gnutls*/ +return OK; +} -Patches welcome. -Arguments: - sctx The current SSL CTX (inbound or outbound) - host connected host, if client; NULL if server - errstr error string pointer +static int +creds_load_crl(/*exim_gnutls_state_st * state,*/ const uschar * crl, uschar ** errstr) +{ +return FAIL; +} -Returns: TRUE if OK (nothing to set up, or setup worked) -*/ -static BOOL -init_ecdh(SSL_CTX * sctx, host_item * host, uschar ** errstr) +static int +creds_load_pristring(/*exim_gnutls_state_st * state,*/ const uschar * p, + const char ** errpos) { -#ifdef OPENSSL_NO_ECDH -return TRUE; -#else - -EC_KEY * ecdh; -uschar * exp_curve; -int nid; -BOOL rv; +return FAIL; +} -if (host) /* No ECDH setup for clients, only for servers */ - return TRUE; +static int +server_load_ciphers(SSL_CTX * ctx, exim_openssl_state_st * state, + uschar * ciphers, uschar ** errstr) +{ +for (uschar * s = ciphers; *s; s++ ) if (*s == '_') *s = '-'; +DEBUG(D_tls) debug_printf("required ciphers: %s\n", ciphers); +if (!SSL_CTX_set_cipher_list(ctx, CS ciphers)) + return tls_error(US"SSL_CTX_set_cipher_list", NULL, NULL, errstr); +state->server_cipher_list = ciphers; +return OK; +} -# ifndef EXIM_HAVE_ECDH -DEBUG(D_tls) - debug_printf("No OpenSSL API to define ECDH parameters, skipping\n"); -return TRUE; -# else -if (!expand_check(tls_eccurve, US"tls_eccurve", &exp_curve, errstr)) - return FALSE; -if (!exp_curve || !*exp_curve) - return TRUE; -/* "auto" needs to be handled carefully. - * OpenSSL < 1.0.2: we do not select anything, but fallback to prime256v1 - * OpenSSL < 1.1.0: we have to call SSL_CTX_set_ecdh_auto - * (openssl/ssl.h defines SSL_CTRL_SET_ECDH_AUTO) - * OpenSSL >= 1.1.0: we do not set anything, the libray does autoselection - * https://github.com/openssl/openssl/commit/fe6ef2472db933f01b59cad82aa925736935984b - */ -if (Ustrcmp(exp_curve, "auto") == 0) - { -#if OPENSSL_VERSION_NUMBER < 0x10002000L - DEBUG(D_tls) debug_printf( - "ECDH OpenSSL < 1.0.2: temp key parameter settings: overriding \"auto\" with \"prime256v1\"\n"); - exp_curve = US"prime256v1"; +static int +lib_ctx_new(SSL_CTX ** ctxp, host_item * host, uschar ** errstr) +{ +SSL_CTX * ctx; +#ifdef EXIM_HAVE_OPENSSL_TLS_METHOD +if (!(ctx = SSL_CTX_new(host ? TLS_client_method() : TLS_server_method()))) #else -# if defined SSL_CTRL_SET_ECDH_AUTO - DEBUG(D_tls) debug_printf( - "ECDH OpenSSL 1.0.2+ temp key parameter settings: autoselection\n"); - SSL_CTX_set_ecdh_auto(sctx, 1); - return TRUE; -# else - DEBUG(D_tls) debug_printf( - "ECDH OpenSSL 1.1.0+ temp key parameter settings: default selection\n"); - return TRUE; -# endif +if (!(ctx = SSL_CTX_new(host ? SSLv23_client_method() : SSLv23_server_method()))) #endif - } + return tls_error(US"SSL_CTX_new", host, NULL, errstr); -DEBUG(D_tls) debug_printf("ECDH: curve '%s'\n", exp_curve); -if ( (nid = OBJ_sn2nid (CCS exp_curve)) == NID_undef -# ifdef EXIM_HAVE_OPENSSL_EC_NIST2NID - && (nid = EC_curve_nist2nid(CCS exp_curve)) == NID_undef -# endif - ) - { - tls_error(string_sprintf("Unknown curve name tls_eccurve '%s'", exp_curve), - host, NULL, errstr); - return FALSE; - } +/* Set up the information callback, which outputs if debugging is at a suitable +level. */ -if (!(ecdh = EC_KEY_new_by_curve_name(nid))) +DEBUG(D_tls) { - tls_error(US"Unable to create ec curve", host, NULL, errstr); - return FALSE; + SSL_CTX_set_info_callback(ctx, (void (*)())info_callback); +#if defined(EXIM_HAVE_OPESSL_TRACE) && !defined(OPENSSL_NO_SSL_TRACE) + /* this needs a debug build of OpenSSL */ + SSL_CTX_set_msg_callback(ctx, (void (*)())SSL_trace); +#endif +#ifdef OPENSSL_HAVE_KEYLOG_CB + SSL_CTX_set_keylog_callback(ctx, (void (*)())keylog_callback); +#endif } -/* The "tmp" in the name here refers to setting a temporary key -not to the stability of the interface. */ - -if ((rv = SSL_CTX_set_tmp_ecdh(sctx, ecdh) == 0)) - tls_error(string_sprintf("Error enabling '%s' curve", exp_curve), host, NULL, errstr); -else - DEBUG(D_tls) debug_printf("ECDH: enabled '%s' curve\n", exp_curve); - -EC_KEY_free(ecdh); -return !rv; - -# endif /*EXIM_HAVE_ECDH*/ -#endif /*OPENSSL_NO_ECDH*/ +/* Automatically re-try reads/writes after renegotiation. */ +(void) SSL_CTX_set_mode(ctx, SSL_MODE_AUTO_RETRY); +*ctxp = ctx; +return OK; } - - -#ifndef DISABLE_OCSP -/************************************************* -* Load OCSP information into state * -*************************************************/ -/* Called to load the server OCSP response from the given file into memory, once -caller has determined this is needed. Checks validity. Debugs a message -if invalid. - -ASSUMES: single response, for single cert. - -Arguments: - sctx the SSL_CTX* to update - cbinfo various parts of session state - filename the filename putatively holding an OCSP response - is_pem file is PEM format; otherwise is DER - -*/ - static void -ocsp_load_response(SSL_CTX * sctx, tls_ext_ctx_cb * cbinfo, - const uschar * filename, BOOL is_pem) +tls_server_creds_init(void) { -BIO * bio; -OCSP_RESPONSE * resp; -OCSP_BASICRESP * basic_response; -OCSP_SINGLERESP * single_response; -ASN1_GENERALIZEDTIME * rev, * thisupd, * nextupd; -STACK_OF(X509) * sk; -unsigned long verify_flags; -int status, reason, i; - -DEBUG(D_tls) - debug_printf("tls_ocsp_file (%s) '%s'\n", is_pem ? "PEM" : "DER", filename); +SSL_CTX * ctx; +uschar * dummy_errstr; -if (!(bio = BIO_new_file(CS filename, "rb"))) - { - DEBUG(D_tls) debug_printf("Failed to open OCSP response file \"%s\"\n", - filename); - return; - } +tls_openssl_init(); -if (is_pem) - { - uschar * data, * freep; - char * dummy; - long len; - if (!PEM_read_bio(bio, &dummy, &dummy, &data, &len)) - { - DEBUG(D_tls) debug_printf("Failed to read PEM file \"%s\"\n", - filename); - return; - } -debug_printf("read pem file\n"); - freep = data; - resp = d2i_OCSP_RESPONSE(NULL, CUSS &data, len); - OPENSSL_free(freep); - } -else - resp = d2i_OCSP_RESPONSE_bio(bio, NULL); -BIO_free(bio); +state_server.lib_state = null_tls_preload; -if (!resp) - { - DEBUG(D_tls) debug_printf("Error reading OCSP response.\n"); +if (lib_ctx_new(&ctx, NULL, &dummy_errstr) != OK) return; - } +state_server.lib_state.lib_ctx = ctx; -if ((status = OCSP_response_status(resp)) != OCSP_RESPONSE_STATUS_SUCCESSFUL) - { - DEBUG(D_tls) debug_printf("OCSP response not valid: %s (%d)\n", - OCSP_response_status_str(status), status); - goto bad; - } +/* Preload DH params and EC curve */ -#ifdef notdef +if (opt_unset_or_noexpand(tls_dhparam)) { - BIO * bp = BIO_new_fp(debug_file, BIO_NOCLOSE); - OCSP_RESPONSE_print(bp, resp, 0); /* extreme debug: stapling content */ - BIO_free(bp); + DEBUG(D_tls) debug_printf("TLS: preloading DH params for server\n"); + if (init_dh(ctx, tls_dhparam, NULL, &dummy_errstr)) + state_server.lib_state.dh = TRUE; } -#endif - -if (!(basic_response = OCSP_response_get1_basic(resp))) +if (opt_unset_or_noexpand(tls_eccurve)) { - DEBUG(D_tls) - debug_printf("OCSP response parse error: unable to extract basic response.\n"); - goto bad; + DEBUG(D_tls) debug_printf("TLS: preloading ECDH curve for server\n"); + if (init_ecdh(ctx, NULL, &dummy_errstr)) + state_server.lib_state.ecdh = TRUE; } -sk = cbinfo->verify_stack; -verify_flags = OCSP_NOVERIFY; /* check sigs, but not purpose */ +#ifdef EXIM_HAVE_INOTIFY +/* If we can, preload the server-side cert, key and ocsp */ -/* May need to expose ability to adjust those flags? -OCSP_NOSIGS OCSP_NOVERIFY OCSP_NOCHAIN OCSP_NOCHECKS OCSP_NOEXPLICIT -OCSP_TRUSTOTHER OCSP_NOINTERN */ +if ( opt_set_and_noexpand(tls_certificate) + && opt_unset_or_noexpand(tls_privatekey) + && opt_unset_or_noexpand(tls_ocsp_file)) + { + /* Set watches on the filenames. The implementation does de-duplication + so we can just blindly do them all. + */ -/* This does a full verify on the OCSP proof before we load it for serving -up; possibly overkill - just date-checks might be nice enough. + if ( tls_set_watch(tls_certificate, TRUE) + && tls_set_watch(tls_privatekey, TRUE) + && tls_set_watch(tls_ocsp_file, TRUE) + ) + { + state_server.certificate = tls_certificate; + state_server.privatekey = tls_privatekey; +#ifndef DISABLE_OCSP + state_server.u_ocsp.server.file = tls_ocsp_file; +#endif -OCSP_basic_verify takes a "store" arg, but does not -use it for the chain verification, which is all we do -when OCSP_NOVERIFY is set. The content from the wire -"basic_response" and a cert-stack "sk" are all that is used. + DEBUG(D_tls) debug_printf("TLS: preloading server certs\n"); + if (tls_expand_session_files(ctx, &state_server, &dummy_errstr) == OK) + state_server.lib_state.conn_certs = TRUE; + } + } +else + DEBUG(D_tls) debug_printf("TLS: not preloading server certs\n"); -We have a stack, loaded in setup_certs() if tls_verify_certificates -was a file (not a directory, or "system"). It is unfortunate we -cannot used the connection context store, as that would neatly -handle the "system" case too, but there seems to be no library -function for getting a stack from a store. -[ In OpenSSL 1.1 - ? X509_STORE_CTX_get0_chain(ctx) ? ] -We do not free the stack since it could be needed a second time for -SNI handling. -Separately we might try to replace using OCSP_basic_verify() - which seems to not -be a public interface into the OpenSSL library (there's no manual entry) - -But what with? We also use OCSP_basic_verify in the client stapling callback. -And there we NEED it; we must verify that status... unless the -library does it for us anyway? */ +/* If we can, preload the Authorities for checking client certs against. +Actual choice to do verify is made (tls_{,try_}verify_hosts) +at TLS conn startup */ -if ((i = OCSP_basic_verify(basic_response, sk, NULL, verify_flags)) < 0) +if ( opt_set_and_noexpand(tls_verify_certificates) + && opt_unset_or_noexpand(tls_crl)) { - DEBUG(D_tls) + /* Watch the default dir also as they are always included */ + + if ( tls_set_watch(CUS X509_get_default_cert_file(), FALSE) + && tls_set_watch(tls_verify_certificates, FALSE) + && tls_set_watch(tls_crl, FALSE)) { - ERR_error_string_n(ERR_get_error(), ssl_errstring, sizeof(ssl_errstring)); - debug_printf("OCSP response verify failure: %s\n", US ssl_errstring); + DEBUG(D_tls) debug_printf("TLS: preloading CA bundle for server\n"); + + if (setup_certs(ctx, tls_verify_certificates, tls_crl, NULL, &dummy_errstr) + == OK) + state_server.lib_state.cabundle = TRUE; } - goto bad; } +else + DEBUG(D_tls) debug_printf("TLS: not preloading CA bundle for server\n"); +#endif /* EXIM_HAVE_INOTIFY */ -/* Here's the simplifying assumption: there's only one response, for the -one certificate we use, and nothing for anything else in a chain. If this -proves false, we need to extract a cert id from our issued cert -(tls_certificate) and use that for OCSP_resp_find_status() (which finds the -right cert in the stack and then calls OCSP_single_get0_status()). - -I'm hoping to avoid reworking a bunch more of how we handle state here. -XXX that will change when we add support for (TLS1.3) whole-chain stapling -*/ +/* If we can, preload the ciphers control string */ -if (!(single_response = OCSP_resp_get0(basic_response, 0))) +if (opt_set_and_noexpand(tls_require_ciphers)) { - DEBUG(D_tls) - debug_printf("Unable to get first response from OCSP basic response.\n"); - goto bad; + DEBUG(D_tls) debug_printf("TLS: preloading cipher list for server\n"); + if (server_load_ciphers(ctx, &state_server, tls_require_ciphers, + &dummy_errstr) == OK) + state_server.lib_state.pri_string = TRUE; } +else + DEBUG(D_tls) debug_printf("TLS: not preloading cipher list for server\n"); +} -status = OCSP_single_get0_status(single_response, &reason, &rev, &thisupd, &nextupd); -if (status != V_OCSP_CERTSTATUS_GOOD) + + + +/* Preload whatever creds are static, onto a transport. The client can then +just copy the pointer as it starts up. +Called from the daemon after a cache-invalidate with watch set; called from +a queue-run startup with watch clear. */ + +static void +tls_client_creds_init(transport_instance * t, BOOL watch) +{ +smtp_transport_options_block * ob = t->options_block; +exim_openssl_state_st tpt_dummy_state; +host_item * dummy_host = (host_item *)1; +uschar * dummy_errstr; +SSL_CTX * ctx; + +tls_openssl_init(); + +ob->tls_preload = null_tls_preload; +if (lib_ctx_new(&ctx, dummy_host, &dummy_errstr) != OK) + return; +ob->tls_preload.lib_ctx = ctx; + +tpt_dummy_state.lib_state = ob->tls_preload; + +if (opt_unset_or_noexpand(tls_dhparam)) { - DEBUG(D_tls) debug_printf("OCSP response bad cert status: %s (%d) %s (%d)\n", - OCSP_cert_status_str(status), status, - OCSP_crl_reason_str(reason), reason); - goto bad; + DEBUG(D_tls) debug_printf("TLS: preloading DH params for transport '%s'\n", t->name); + if (init_dh(ctx, tls_dhparam, NULL, &dummy_errstr)) + ob->tls_preload.dh = TRUE; } - -if (!OCSP_check_validity(thisupd, nextupd, EXIM_OCSP_SKEW_SECONDS, EXIM_OCSP_MAX_AGE)) +if (opt_unset_or_noexpand(tls_eccurve)) { - DEBUG(D_tls) debug_printf("OCSP status invalid times.\n"); - goto bad; + DEBUG(D_tls) debug_printf("TLS: preloading ECDH curve for transport '%s'\n", t->name); + if (init_ecdh(ctx, NULL, &dummy_errstr)) + ob->tls_preload.ecdh = TRUE; } -supply_response: - /* Add the resp to the list used by tls_server_stapling_cb() */ +#ifdef EXIM_HAVE_INOTIFY +if ( opt_set_and_noexpand(ob->tls_certificate) + && opt_unset_or_noexpand(ob->tls_privatekey)) { - ocsp_resplist ** op = &cbinfo->u_ocsp.server.olist, * oentry; - while (oentry = *op) - op = &oentry->next; - *op = oentry = store_get(sizeof(ocsp_resplist), FALSE); - oentry->next = NULL; - oentry->resp = resp; + if ( !watch + || ( tls_set_watch(ob->tls_certificate, FALSE) + && tls_set_watch(ob->tls_privatekey, FALSE) + ) ) + { + uschar * pkey = ob->tls_privatekey; + + DEBUG(D_tls) + debug_printf("TLS: preloading client certs for transport '%s'\n",t->name); + + if ( tls_add_certfile(ctx, &tpt_dummy_state, ob->tls_certificate, + &dummy_errstr) == 0 + && tls_add_pkeyfile(ctx, &tpt_dummy_state, + pkey ? pkey : ob->tls_certificate, + &dummy_errstr) == 0 + ) + ob->tls_preload.conn_certs = TRUE; + } } -return; +else + DEBUG(D_tls) + debug_printf("TLS: not preloading client certs, for transport '%s'\n", t->name); -bad: - if (f.running_in_test_harness) + +if ( opt_set_and_noexpand(ob->tls_verify_certificates) + && opt_unset_or_noexpand(ob->tls_crl)) + { + if ( !watch + || tls_set_watch(CUS X509_get_default_cert_file(), FALSE) + && tls_set_watch(ob->tls_verify_certificates, FALSE) + && tls_set_watch(ob->tls_crl, FALSE) + ) { - extern char ** environ; - if (environ) for (uschar ** p = USS environ; *p; p++) - if (Ustrncmp(*p, "EXIM_TESTHARNESS_DISABLE_OCSPVALIDITYCHECK", 42) == 0) - { - DEBUG(D_tls) debug_printf("Supplying known bad OCSP response\n"); - goto supply_response; - } + DEBUG(D_tls) + debug_printf("TLS: preloading CA bundle for transport '%s'\n", t->name); + + if (setup_certs(ctx, ob->tls_verify_certificates, + ob->tls_crl, dummy_host, &dummy_errstr) == OK) + ob->tls_preload.cabundle = TRUE; } -return; + } +else + DEBUG(D_tls) + debug_printf("TLS: not preloading CA bundle, for transport '%s'\n", t->name); + +#endif /*EXIM_HAVE_INOTIFY*/ } +#ifdef EXIM_HAVE_INOTIFY +/* Invalidate the creds cached, by dropping the current ones. +Call when we notice one of the source files has changed. */ + static void -ocsp_free_response_list(tls_ext_ctx_cb * cbinfo) +tls_server_creds_invalidate(void) { -for (ocsp_resplist * olist = cbinfo->u_ocsp.server.olist; olist; - olist = olist->next) - OCSP_RESPONSE_free(olist->resp); -cbinfo->u_ocsp.server.olist = NULL; +SSL_CTX_free(state_server.lib_state.lib_ctx); +state_server.lib_state = null_tls_preload; } -#endif /*!DISABLE_OCSP*/ - +static void +tls_client_creds_invalidate(transport_instance * t) +{ +smtp_transport_options_block * ob = t->options_block; +SSL_CTX_free(ob->tls_preload.lib_ctx); +ob->tls_preload = null_tls_preload; +} +#endif /*EXIM_HAVE_INOTIFY*/ -/* Create and install a selfsigned certificate, for use in server mode */ -static int -tls_install_selfsign(SSL_CTX * sctx, uschar ** errstr) +/* Extreme debug +#ifndef DISABLE_OCSP +void +x509_store_dump_cert_s_names(X509_STORE * store) { -X509 * x509 = NULL; -EVP_PKEY * pkey; -RSA * rsa; -X509_NAME * name; -uschar * where; +STACK_OF(X509_OBJECT) * roots= store->objs; +static uschar name[256]; -where = US"allocating pkey"; -if (!(pkey = EVP_PKEY_new())) - goto err; +for (int i= 0; i < sk_X509_OBJECT_num(roots); i++) + { + X509_OBJECT * tmp_obj= sk_X509_OBJECT_value(roots, i); + if(tmp_obj->type == X509_LU_X509) + { + X509_NAME * sn = X509_get_subject_name(tmp_obj->data.x509); + if (X509_NAME_oneline(sn, CS name, sizeof(name))) + { + name[sizeof(name)-1] = '\0'; + debug_printf(" %s\n", name); + } + } + } +} +#endif +*/ -where = US"allocating cert"; -if (!(x509 = X509_new())) - goto err; -where = US"generating pkey"; -if (!(rsa = rsa_callback(NULL, 0, 2048))) - goto err; +#ifndef DISABLE_TLS_RESUME +/* Manage the keysets used for encrypting the session tickets, on the server. */ -where = US"assigning pkey"; -if (!EVP_PKEY_assign_RSA(pkey, rsa)) - goto err; +typedef struct { /* Session ticket encryption key */ + uschar name[16]; -X509_set_version(x509, 2); /* N+1 - version 3 */ -ASN1_INTEGER_set(X509_get_serialNumber(x509), 1); -X509_gmtime_adj(X509_get_notBefore(x509), 0); -X509_gmtime_adj(X509_get_notAfter(x509), (long)60 * 60); /* 1 hour */ -X509_set_pubkey(x509, pkey); + const EVP_CIPHER * aes_cipher; + uschar aes_key[32]; /* size needed depends on cipher. aes_128 implies 128/8 = 16? */ + const EVP_MD * hmac_hash; + uschar hmac_key[16]; + time_t renew; + time_t expire; +} exim_stek; -name = X509_get_subject_name(x509); -X509_NAME_add_entry_by_txt(name, "C", - MBSTRING_ASC, CUS "UK", -1, -1, 0); -X509_NAME_add_entry_by_txt(name, "O", - MBSTRING_ASC, CUS "Exim Developers", -1, -1, 0); -X509_NAME_add_entry_by_txt(name, "CN", - MBSTRING_ASC, CUS smtp_active_hostname, -1, -1, 0); -X509_set_issuer_name(x509, name); +static exim_stek exim_tk; /* current key */ +static exim_stek exim_tk_old; /* previous key */ -where = US"signing cert"; -if (!X509_sign(x509, pkey, EVP_md5())) - goto err; +static void +tk_init(void) +{ +time_t t = time(NULL); -where = US"installing selfsign cert"; -if (!SSL_CTX_use_certificate(sctx, x509)) - goto err; +if (exim_tk.name[0]) + { + if (exim_tk.renew >= t) return; + exim_tk_old = exim_tk; + } -where = US"installing selfsign key"; -if (!SSL_CTX_use_PrivateKey(sctx, pkey)) - goto err; +if (f.running_in_test_harness) ssl_session_timeout = 6; -return OK; +DEBUG(D_tls) debug_printf("OpenSSL: %s STEK\n", exim_tk.name[0] ? "rotating" : "creating"); +if (RAND_bytes(exim_tk.aes_key, sizeof(exim_tk.aes_key)) <= 0) return; +if (RAND_bytes(exim_tk.hmac_key, sizeof(exim_tk.hmac_key)) <= 0) return; +if (RAND_bytes(exim_tk.name+1, sizeof(exim_tk.name)-1) <= 0) return; -err: - (void) tls_error(where, NULL, NULL, errstr); - if (x509) X509_free(x509); - if (pkey) EVP_PKEY_free(pkey); - return DEFER; +exim_tk.name[0] = 'E'; +exim_tk.aes_cipher = EVP_aes_256_cbc(); +exim_tk.hmac_hash = EVP_sha256(); +exim_tk.expire = t + ssl_session_timeout; +exim_tk.renew = t + ssl_session_timeout/2; } - - - -static int -tls_add_certfile(SSL_CTX * sctx, tls_ext_ctx_cb * cbinfo, uschar * file, - uschar ** errstr) +static exim_stek * +tk_current(void) { -DEBUG(D_tls) debug_printf("tls_certificate file '%s'\n", file); -if (!SSL_CTX_use_certificate_chain_file(sctx, CS file)) - return tls_error(string_sprintf( - "SSL_CTX_use_certificate_chain_file file=%s", file), - cbinfo->host, NULL, errstr); -return 0; +if (!exim_tk.name[0]) return NULL; +return &exim_tk; } -static int -tls_add_pkeyfile(SSL_CTX * sctx, tls_ext_ctx_cb * cbinfo, uschar * file, - uschar ** errstr) +static exim_stek * +tk_find(const uschar * name) { -DEBUG(D_tls) debug_printf("tls_privatekey file '%s'\n", file); -if (!SSL_CTX_use_PrivateKey_file(sctx, CS file, SSL_FILETYPE_PEM)) - return tls_error(string_sprintf( - "SSL_CTX_use_PrivateKey_file file=%s", file), cbinfo->host, NULL, errstr); -return 0; +return memcmp(name, exim_tk.name, sizeof(exim_tk.name)) == 0 ? &exim_tk + : memcmp(name, exim_tk_old.name, sizeof(exim_tk_old.name)) == 0 ? &exim_tk_old + : NULL; } - -/************************************************* -* Expand key and cert file specs * -*************************************************/ - -/* Called once during tls_init and possibly again during TLS setup, for a -new context, if Server Name Indication was used and tls_sni was seen in -the certificate string. - -Arguments: - sctx the SSL_CTX* to update - cbinfo various parts of session state - errstr error string pointer - -Returns: OK/DEFER/FAIL -*/ - +/* Callback for session tickets, on server */ static int -tls_expand_session_files(SSL_CTX * sctx, tls_ext_ctx_cb * cbinfo, - uschar ** errstr) +ticket_key_callback(SSL * ssl, uschar key_name[16], + uschar * iv, EVP_CIPHER_CTX * c_ctx, HMAC_CTX * hctx, int enc) { -uschar * expanded; +tls_support * tlsp = state_server.tlsp; +exim_stek * key; -if (!cbinfo->certificate) +if (enc) { - if (!cbinfo->is_server) /* client */ - return OK; - /* server */ - if (tls_install_selfsign(sctx, errstr) != OK) - return DEFER; + DEBUG(D_tls) debug_printf("ticket_key_callback: create new session\n"); + tlsp->resumption |= RESUME_CLIENT_REQUESTED; + + if (RAND_bytes(iv, EVP_MAX_IV_LENGTH) <= 0) + return -1; /* insufficient random */ + + if (!(key = tk_current())) /* current key doesn't exist or isn't valid */ + return 0; /* key couldn't be created */ + memcpy(key_name, key->name, 16); + DEBUG(D_tls) debug_printf("STEK expire " TIME_T_FMT "\n", key->expire - time(NULL)); + + /*XXX will want these dependent on the ssl session strength */ + HMAC_Init_ex(hctx, key->hmac_key, sizeof(key->hmac_key), + key->hmac_hash, NULL); + EVP_EncryptInit_ex(c_ctx, key->aes_cipher, NULL, key->aes_key, iv); + + DEBUG(D_tls) debug_printf("ticket created\n"); + return 1; } else { - int err; - - if ( !reexpand_tls_files_for_sni - && ( Ustrstr(cbinfo->certificate, US"tls_sni") - || Ustrstr(cbinfo->certificate, US"tls_in_sni") - || Ustrstr(cbinfo->certificate, US"tls_out_sni") - ) ) - reexpand_tls_files_for_sni = TRUE; + time_t now = time(NULL); - if (!expand_check(cbinfo->certificate, US"tls_certificate", &expanded, errstr)) - return DEFER; + DEBUG(D_tls) debug_printf("ticket_key_callback: retrieve session\n"); + tlsp->resumption |= RESUME_CLIENT_SUGGESTED; - if (expanded) - if (cbinfo->is_server) + if (!(key = tk_find(key_name)) || key->expire < now) + { + DEBUG(D_tls) { - const uschar * file_list = expanded; - int sep = 0; - uschar * file; -#ifndef DISABLE_OCSP - const uschar * olist = cbinfo->u_ocsp.server.file; - int osep = 0; - uschar * ofile; - BOOL fmt_pem = FALSE; - - if (olist) - if (!expand_check(olist, US"tls_ocsp_file", USS &olist, errstr)) - return DEFER; - if (olist && !*olist) - olist = NULL; + debug_printf("ticket not usable (%s)\n", key ? "expired" : "not found"); + if (key) debug_printf("STEK expire " TIME_T_FMT "\n", key->expire - now); + } + return 0; + } - if ( cbinfo->u_ocsp.server.file_expanded && olist - && (Ustrcmp(olist, cbinfo->u_ocsp.server.file_expanded) == 0)) - { - DEBUG(D_tls) debug_printf(" - value unchanged, using existing values\n"); - olist = NULL; - } - else - { - ocsp_free_response_list(cbinfo); - cbinfo->u_ocsp.server.file_expanded = olist; - } -#endif + HMAC_Init_ex(hctx, key->hmac_key, sizeof(key->hmac_key), + key->hmac_hash, NULL); + EVP_DecryptInit_ex(c_ctx, key->aes_cipher, NULL, key->aes_key, iv); - while (file = string_nextinlist(&file_list, &sep, NULL, 0)) - { - if ((err = tls_add_certfile(sctx, cbinfo, file, errstr))) - return err; + DEBUG(D_tls) debug_printf("ticket usable, STEK expire " TIME_T_FMT "\n", key->expire - now); -#ifndef DISABLE_OCSP - if (olist) - if ((ofile = string_nextinlist(&olist, &osep, NULL, 0))) - { - if (Ustrncmp(ofile, US"PEM ", 4) == 0) - { - fmt_pem = TRUE; - ofile += 4; - } - else if (Ustrncmp(ofile, US"DER ", 4) == 0) - { - fmt_pem = FALSE; - ofile += 4; - } - ocsp_load_response(sctx, cbinfo, ofile, fmt_pem); - } - else - DEBUG(D_tls) debug_printf("ran out of ocsp file list\n"); + /* The ticket lifetime and renewal are the same as the STEK lifetime and + renewal, which is overenthusiastic. A factor of, say, 3x longer STEK would + be better. To do that we'd have to encode ticket lifetime in the name as + we don't yet see the restored session. Could check posthandshake for TLS1.3 + and trigger a new ticket then, but cannot do that for TLS1.2 */ + return key->renew < now ? 2 : 1; + } +} #endif - } - } - else /* would there ever be a need for multiple client certs? */ - if ((err = tls_add_certfile(sctx, cbinfo, expanded, errstr))) - return err; - - if ( cbinfo->privatekey - && !expand_check(cbinfo->privatekey, US"tls_privatekey", &expanded, errstr)) - return DEFER; - /* If expansion was forced to fail, key_expanded will be NULL. If the result - of the expansion is an empty string, ignore it also, and assume the private - key is in the same file as the certificate. */ - if (expanded && *expanded) - if (cbinfo->is_server) - { - const uschar * file_list = expanded; - int sep = 0; - uschar * file; - while (file = string_nextinlist(&file_list, &sep, NULL, 0)) - if ((err = tls_add_pkeyfile(sctx, cbinfo, file, errstr))) - return err; - } - else /* would there ever be a need for multiple client certs? */ - if ((err = tls_add_pkeyfile(sctx, cbinfo, expanded, errstr))) - return err; - } +static void +setup_cert_verify(SSL_CTX * ctx, BOOL optional, + int (*cert_vfy_cb)(int, X509_STORE_CTX *)) +{ +/* If verification is optional, don't fail if no certificate */ -return OK; +SSL_CTX_set_verify(ctx, + SSL_VERIFY_PEER | (optional ? 0 : SSL_VERIFY_FAIL_IF_NO_PEER_CERT), + cert_vfy_cb); } - - /************************************************* * Callback to handle SNI * *************************************************/ @@ -1682,7 +2229,7 @@ static int tls_servername_cb(SSL *s, int *ad ARG_UNUSED, void *arg) { const char *servername = SSL_get_servername(s, TLSEXT_NAMETYPE_host_name); -tls_ext_ctx_cb *cbinfo = (tls_ext_ctx_cb *) arg; +exim_openssl_state_st *state = (exim_openssl_state_st *) arg; int rc; int old_pool = store_pool; uschar * dummy_errstr; @@ -1705,51 +2252,54 @@ if (!reexpand_tls_files_for_sni) not confident that memcpy wouldn't break some internal reference counting. Especially since there's a references struct member, which would be off. */ -#ifdef EXIM_HAVE_OPENSSL_TLS_METHOD -if (!(server_sni = SSL_CTX_new(TLS_server_method()))) -#else -if (!(server_sni = SSL_CTX_new(SSLv23_server_method()))) -#endif - { - ERR_error_string_n(ERR_get_error(), ssl_errstring, sizeof(ssl_errstring)); - DEBUG(D_tls) debug_printf("SSL_CTX_new() failed: %s\n", ssl_errstring); +if (lib_ctx_new(&server_sni, NULL, &dummy_errstr) != OK) goto bad; - } /* Not sure how many of these are actually needed, since SSL object already exists. Might even need this selfsame callback, for reneg? */ -SSL_CTX_set_info_callback(server_sni, SSL_CTX_get_info_callback(server_ctx)); -SSL_CTX_set_mode(server_sni, SSL_CTX_get_mode(server_ctx)); -SSL_CTX_set_options(server_sni, SSL_CTX_get_options(server_ctx)); -SSL_CTX_set_timeout(server_sni, SSL_CTX_get_timeout(server_ctx)); -SSL_CTX_set_tlsext_servername_callback(server_sni, tls_servername_cb); -SSL_CTX_set_tlsext_servername_arg(server_sni, cbinfo); + { + SSL_CTX * ctx = state_server.lib_state.lib_ctx; + SSL_CTX_set_info_callback(server_sni, SSL_CTX_get_info_callback(ctx)); + SSL_CTX_set_mode(server_sni, SSL_CTX_get_mode(ctx)); + SSL_CTX_set_options(server_sni, SSL_CTX_get_options(ctx)); + SSL_CTX_set_timeout(server_sni, SSL_CTX_get_timeout(ctx)); + SSL_CTX_set_tlsext_servername_callback(server_sni, tls_servername_cb); + SSL_CTX_set_tlsext_servername_arg(server_sni, state); + } -if ( !init_dh(server_sni, cbinfo->dhparam, NULL, &dummy_errstr) +if ( !init_dh(server_sni, state->dhparam, NULL, &dummy_errstr) || !init_ecdh(server_sni, NULL, &dummy_errstr) ) goto bad; -if ( cbinfo->server_cipher_list - && !SSL_CTX_set_cipher_list(server_sni, CS cbinfo->server_cipher_list)) +if ( state->server_cipher_list + && !SSL_CTX_set_cipher_list(server_sni, CS state->server_cipher_list)) goto bad; #ifndef DISABLE_OCSP -if (cbinfo->u_ocsp.server.file) +if (state->u_ocsp.server.file) { SSL_CTX_set_tlsext_status_cb(server_sni, tls_server_stapling_cb); - SSL_CTX_set_tlsext_status_arg(server_sni, cbinfo); + SSL_CTX_set_tlsext_status_arg(server_sni, state); } #endif -if ((rc = setup_certs(server_sni, tls_verify_certificates, tls_crl, NULL, FALSE, - verify_callback_server, &dummy_errstr)) != OK) - goto bad; + { + uschar * expcerts; + if ( !expand_check(tls_verify_certificates, US"tls_verify_certificates", + &expcerts, &dummy_errstr) + || (rc = setup_certs(server_sni, expcerts, tls_crl, NULL, + &dummy_errstr)) != OK) + goto bad; + + if (expcerts && *expcerts) + setup_cert_verify(server_sni, FALSE, verify_callback_server); + } /* do this after setup_certs, because this can require the certs for verifying OCSP information. */ -if ((rc = tls_expand_session_files(server_sni, cbinfo, &dummy_errstr)) != OK) +if ((rc = tls_expand_session_files(server_sni, state, &dummy_errstr)) != OK) goto bad; DEBUG(D_tls) debug_printf("Switching SSL context.\n"); @@ -1780,8 +2330,8 @@ project. static int tls_server_stapling_cb(SSL *s, void *arg) { -const tls_ext_ctx_cb * cbinfo = (tls_ext_ctx_cb *) arg; -ocsp_resplist * olist = cbinfo->u_ocsp.server.olist; +const exim_openssl_state_st * state = arg; +ocsp_resplist * olist = state->u_ocsp.server.olist; uschar * response_der; /*XXX blob */ int response_der_len; @@ -1855,7 +2405,8 @@ response_der_len = i2d_OCSP_RESPONSE(olist->resp, &response_der); if (response_der_len <= 0) return SSL_TLSEXT_ERR_NOACK; -SSL_set_tlsext_status_ocsp_resp(server_ssl, response_der, response_der_len); +SSL_set_tlsext_status_ocsp_resp(state_server.lib_state.lib_ssl, + response_der, response_der_len); tls_in.ocsp = OCSP_VFIED; return SSL_TLSEXT_ERR_OK; } @@ -1872,7 +2423,7 @@ BIO_puts(bp, "\n"); static int tls_client_stapling_cb(SSL *s, void *arg) { -tls_ext_ctx_cb * cbinfo = arg; +exim_openssl_state_st * cbinfo = arg; const unsigned char * p; int len; OCSP_RESPONSE * rsp; @@ -2032,95 +2583,64 @@ return i; /************************************************* * Initialize for TLS * *************************************************/ - -static void -tls_openssl_init(void) -{ -#ifdef EXIM_NEED_OPENSSL_INIT -SSL_load_error_strings(); /* basic set up */ -OpenSSL_add_ssl_algorithms(); -#endif - -#if defined(EXIM_HAVE_SHA256) && !defined(OPENSSL_AUTO_SHA256) -/* SHA256 is becoming ever more popular. This makes sure it gets added to the -list of available digests. */ -EVP_add_digest(EVP_sha256()); -#endif -} - - - /* Called from both server and client code, to do preliminary initialization of the library. We allocate and return a context structure. Arguments: - ctxp returned SSL context host connected host, if client; NULL if server - dhparam DH parameter file - certificate certificate file - privatekey private key + ob transport options block, if client; NULL if server ocsp_file file of stapling info (server); flag for require ocsp (client) addr address if client; NULL if server (for some randomness) - cbp place to put allocated callback context + caller_state place to put pointer to allocated state-struct errstr error string pointer Returns: OK/DEFER/FAIL */ static int -tls_init(SSL_CTX **ctxp, host_item *host, uschar *dhparam, uschar *certificate, - uschar *privatekey, +tls_init(host_item * host, smtp_transport_options_block * ob, #ifndef DISABLE_OCSP uschar *ocsp_file, #endif - address_item *addr, tls_ext_ctx_cb ** cbp, + address_item *addr, exim_openssl_state_st ** caller_state, tls_support * tlsp, uschar ** errstr) { SSL_CTX * ctx; -long init_options; +exim_openssl_state_st * state; int rc; -tls_ext_ctx_cb * cbinfo; -cbinfo = store_malloc(sizeof(tls_ext_ctx_cb)); -cbinfo->tlsp = tlsp; -cbinfo->certificate = certificate; -cbinfo->privatekey = privatekey; -cbinfo->is_server = host==NULL; -#ifndef DISABLE_OCSP -cbinfo->verify_stack = NULL; -if (!host) +if (host) /* client */ { - cbinfo->u_ocsp.server.file = ocsp_file; - cbinfo->u_ocsp.server.file_expanded = NULL; - cbinfo->u_ocsp.server.olist = NULL; + state = store_malloc(sizeof(exim_openssl_state_st)); + memset(state, 0, sizeof(*state)); + state->certificate = ob->tls_certificate; + state->privatekey = ob->tls_privatekey; + state->is_server = FALSE; + state->dhparam = NULL; + state->lib_state = ob->tls_preload; + } +else /* server */ + { + state = &state_server; + state->certificate = tls_certificate; + state->privatekey = tls_privatekey; + state->is_server = TRUE; + state->dhparam = tls_dhparam; + state->lib_state = state_server.lib_state; } -else - cbinfo->u_ocsp.client.verify_store = NULL; -#endif -cbinfo->dhparam = dhparam; -cbinfo->server_cipher_list = NULL; -cbinfo->host = host; -#ifndef DISABLE_EVENT -cbinfo->event_action = NULL; -#endif -tls_openssl_init(); +state->tlsp = tlsp; +state->host = host; -/* Create a context. -The OpenSSL docs in 1.0.1b have not been updated to clarify TLS variant -negotiation in the different methods; as far as I can tell, the only -*_{server,client}_method which allows negotiation is SSLv23, which exists even -when OpenSSL is built without SSLv2 support. -By disabling with openssl_options, we can let admins re-enable with the -existing knob. */ +if (!state->lib_state.pri_string) + state->server_cipher_list = NULL; -#ifdef EXIM_HAVE_OPENSSL_TLS_METHOD -if (!(ctx = SSL_CTX_new(host ? TLS_client_method() : TLS_server_method()))) -#else -if (!(ctx = SSL_CTX_new(host ? SSLv23_client_method() : SSLv23_server_method()))) +#ifndef DISABLE_EVENT +state->event_action = NULL; #endif - return tls_error(US"SSL_CTX_new", host, NULL, errstr); + +tls_openssl_init(); /* It turns out that we need to seed the random number generator this early in order to get the full complement of ciphers to work. It took me roughly a day @@ -2128,40 +2648,14 @@ of work to discover this by experiment. On systems that have /dev/urandom, SSL may automatically seed itself from there. Otherwise, we have to make something up as best we can. Double check -afterwards. */ - -if (!RAND_status()) - { - randstuff r; - gettimeofday(&r.tv, NULL); - r.p = getpid(); - - RAND_seed(US (&r), sizeof(r)); - RAND_seed(US big_buffer, big_buffer_size); - if (addr != NULL) RAND_seed(US addr, sizeof(addr)); - - if (!RAND_status()) - return tls_error(US"RAND_status", host, - US"unable to seed random number generator", errstr); - } - -/* Set up the information callback, which outputs if debugging is at a suitable -level. */ +afterwards. -DEBUG(D_tls) - { - SSL_CTX_set_info_callback(ctx, (void (*)())info_callback); -#if defined(EXIM_HAVE_OPESSL_TRACE) && !defined(OPENSSL_NO_SSL_TRACE) - /* this needs a debug build of OpenSSL */ - SSL_CTX_set_msg_callback(ctx, (void (*)())SSL_trace); -#endif -#ifdef OPENSSL_HAVE_KEYLOG_CB - SSL_CTX_set_keylog_callback(ctx, (void (*)())keylog_callback); -#endif - } +Although we likely called this before, at daemon startup, this is a chance +to mix in further variable info (time, pid) if needed. */ -/* Automatically re-try reads/writes after renegotiation. */ -(void) SSL_CTX_set_mode(ctx, SSL_MODE_AUTO_RETRY); +if (!lib_rand_init(addr)) + return tls_error(US"RAND_status", host, + US"unable to seed random number generator", errstr); /* Apply administrator-supplied work-arounds. Historically we applied just one requested option, @@ -2172,8 +2666,24 @@ grandfathered in the first one as the default value for "openssl_options". No OpenSSL version number checks: the options we accept depend upon the availability of the option value macros from OpenSSL. */ -if (!tls_openssl_options_parse(openssl_options, &init_options)) - return tls_error(US"openssl_options parsing failed", host, NULL, errstr); +if (!init_options) + if (!tls_openssl_options_parse(openssl_options, &init_options)) + return tls_error(US"openssl_options parsing failed", host, NULL, errstr); + +/* Create a context. +The OpenSSL docs in 1.0.1b have not been updated to clarify TLS variant +negotiation in the different methods; as far as I can tell, the only +*_{server,client}_method which allows negotiation is SSLv23, which exists even +when OpenSSL is built without SSLv2 support. +By disabling with openssl_options, we can let admins re-enable with the +existing knob. */ + +if (!(ctx = state->lib_state.lib_ctx)) + { + if ((rc = lib_ctx_new(&ctx, host, errstr)) != OK) + return rc; + state->lib_state.lib_ctx = ctx; + } #ifndef DISABLE_TLS_RESUME tlsp->resumption = RESUME_SUPPORTED; @@ -2212,21 +2722,41 @@ will never be used because we use a new context every time. */ /* Initialize with DH parameters if supplied */ /* Initialize ECDH temp key parameter selection */ -if ( !init_dh(ctx, dhparam, host, errstr) - || !init_ecdh(ctx, host, errstr) - ) - return DEFER; +if (state->lib_state.dh) + { DEBUG(D_tls) debug_printf("TLS: DH params were preloaded\n"); } +else + if (!init_dh(ctx, state->dhparam, host, errstr)) return DEFER; + +if (state->lib_state.ecdh) + { DEBUG(D_tls) debug_printf("TLS: ECDH curve was preloaded\n"); } +else + if (!init_ecdh(ctx, host, errstr)) return DEFER; /* Set up certificate and key (and perhaps OCSP info) */ -if ((rc = tls_expand_session_files(ctx, cbinfo, errstr)) != OK) - return rc; +if (state->lib_state.conn_certs) + { + DEBUG(D_tls) + debug_printf("TLS: %s certs were preloaded\n", host ? "client":"server"); + } +else + { +#ifndef DISABLE_OCSP + if (!host) + { + state->u_ocsp.server.file = ocsp_file; + state->u_ocsp.server.file_expanded = NULL; + state->u_ocsp.server.olist = NULL; + } +#endif + if ((rc = tls_expand_session_files(ctx, state, errstr)) != OK) return rc; + } /* If we need to handle SNI or OCSP, do so */ #ifdef EXIM_HAVE_OPENSSL_TLSEXT # ifndef DISABLE_OCSP - if (!(cbinfo->verify_stack = sk_X509_new_null())) + if (!(state->verify_stack = sk_X509_new_null())) { DEBUG(D_tls) debug_printf("failed to create stack for stapling verify\n"); return FAIL; @@ -2240,33 +2770,33 @@ if (!host) /* server */ the option exists, not what the current expansion might be, as SNI might change the certificate and OCSP file in use between now and the time the callback is invoked. */ - if (cbinfo->u_ocsp.server.file) + if (state->u_ocsp.server.file) { SSL_CTX_set_tlsext_status_cb(ctx, tls_server_stapling_cb); - SSL_CTX_set_tlsext_status_arg(ctx, cbinfo); + SSL_CTX_set_tlsext_status_arg(ctx, state); } # endif /* We always do this, so that $tls_sni is available even if not used in tls_certificate */ SSL_CTX_set_tlsext_servername_callback(ctx, tls_servername_cb); - SSL_CTX_set_tlsext_servername_arg(ctx, cbinfo); + SSL_CTX_set_tlsext_servername_arg(ctx, state); } # ifndef DISABLE_OCSP else /* client */ if(ocsp_file) /* wanting stapling */ { - if (!(cbinfo->u_ocsp.client.verify_store = X509_STORE_new())) + if (!(state->u_ocsp.client.verify_store = X509_STORE_new())) { DEBUG(D_tls) debug_printf("failed to create store for stapling verify\n"); return FAIL; } SSL_CTX_set_tlsext_status_cb(ctx, tls_client_stapling_cb); - SSL_CTX_set_tlsext_status_arg(ctx, cbinfo); + SSL_CTX_set_tlsext_status_arg(ctx, state); } # endif #endif -cbinfo->verify_cert_hostnames = NULL; +state->verify_cert_hostnames = NULL; #ifdef EXIM_HAVE_EPHEM_RSA_KEX /* Set up the RSA callback */ @@ -2279,8 +2809,7 @@ The period appears to be also used for (server-generated) session tickets */ SSL_CTX_set_timeout(ctx, ssl_session_timeout); DEBUG(D_tls) debug_printf("Initialized TLS\n"); -*cbp = cbinfo; -*ctxp = ctx; +*caller_state = state; return OK; } @@ -2430,20 +2959,17 @@ repeated after a Server Name Indication. Arguments: sctx SSL_CTX* to initialise - certs certs file or NULL + certs certs file, expanded crl CRL file or NULL host NULL in a server; the remote host in a client - optional TRUE if called from a server for a host in tls_try_verify_hosts; - otherwise passed as FALSE - cert_vfy_cb Callback function for certificate verification errstr error string pointer Returns: OK/DEFER/FAIL */ static int -setup_certs(SSL_CTX *sctx, uschar *certs, uschar *crl, host_item *host, BOOL optional, - int (*cert_vfy_cb)(int, X509_STORE_CTX *), uschar ** errstr) +setup_certs(SSL_CTX *sctx, uschar *certs, uschar *crl, host_item *host, + uschar ** errstr) { uschar *expcerts, *expcrl; @@ -2459,7 +2985,7 @@ if (expcerts && *expcerts) if (!SSL_CTX_set_default_verify_paths(sctx)) return tls_error(US"SSL_CTX_set_default_verify_paths", host, NULL, errstr); - if (Ustrcmp(expcerts, "system") != 0) + if (Ustrcmp(expcerts, "system") != 0 && Ustrncmp(expcerts, "system,", 7) != 0) { struct stat statbuf; @@ -2487,8 +3013,8 @@ This is inconsistent with the need to verify the OCSP proof of the server cert. if ( !host && statbuf.st_size > 0 - && server_static_cbinfo->u_ocsp.server.file - && !chain_from_pem_file(file, server_static_cbinfo->verify_stack) + && state_server.u_ocsp.server.file + && !chain_from_pem_file(file, state_server.verify_stack) ) { log_write(0, LOG_MAIN|LOG_PANIC, @@ -2505,7 +3031,8 @@ This is inconsistent with the need to verify the OCSP proof of the server cert. if ( (!file || statbuf.st_size > 0) && !SSL_CTX_load_verify_locations(sctx, CS file, CS dir)) - return tls_error(US"SSL_CTX_load_verify_locations", host, NULL, errstr); + return tls_error(US"SSL_CTX_load_verify_locations", + host, NULL, errstr); /* On the server load the list of CAs for which we will accept certs, for sending to the client. This is only for the one-file @@ -2520,11 +3047,15 @@ This is inconsistent with the need to verify the OCSP proof of the server cert. if (file) { STACK_OF(X509_NAME) * names = SSL_load_client_CA_file(CS file); + int i = sk_X509_NAME_num(names); if (!host) SSL_CTX_set_client_CA_list(sctx, names); - DEBUG(D_tls) debug_printf("Added %d certificate authorities.\n", - sk_X509_NAME_num(names)); + DEBUG(D_tls) debug_printf("Added %d additional certificate authorit%s\n", + i, i>1 ? "ies":"y"); } + else + DEBUG(D_tls) + debug_printf("Added dir for additional certificate authorities\n"); } } @@ -2580,12 +3111,6 @@ This is inconsistent with the need to verify the OCSP proof of the server cert. } #endif /* OPENSSL_VERSION_NUMBER > 0x00907000L */ - - /* If verification is optional, don't fail if no certificate */ - - SSL_CTX_set_verify(sctx, - SSL_VERIFY_PEER | (optional ? 0 : SSL_VERIFY_FAIL_IF_NO_PEER_CERT), - cert_vfy_cb); } return OK; @@ -2596,13 +3121,11 @@ return OK; /************************************************* * Start a TLS session in a server * *************************************************/ - /* This is called when Exim is running as a server, after having received the STARTTLS command. It must respond to that command, and then negotiate a TLS session. Arguments: - require_ciphers allowed ciphers errstr pointer to error message Returns: OK on success @@ -2612,11 +3135,13 @@ Returns: OK on success */ int -tls_server_start(const uschar * require_ciphers, uschar ** errstr) +tls_server_start(uschar ** errstr) { int rc; uschar * expciphers; -tls_ext_ctx_cb * cbinfo; +exim_openssl_state_st * dummy_statep; +SSL_CTX * ctx; +SSL * ssl; static uschar peerdn[256]; /* Check for previous activation */ @@ -2631,16 +3156,13 @@ if (tls_in.active.sock >= 0) /* Initialize the SSL library. If it fails, it will already have logged the error. */ -rc = tls_init(&server_ctx, NULL, tls_dhparam, tls_certificate, tls_privatekey, +rc = tls_init(NULL, NULL, #ifndef DISABLE_OCSP tls_ocsp_file, #endif - NULL, &server_static_cbinfo, &tls_in, errstr); + NULL, &dummy_statep, &tls_in, errstr); if (rc != OK) return rc; -cbinfo = server_static_cbinfo; - -if (!expand_check(require_ciphers, US"tls_require_ciphers", &expciphers, errstr)) - return FAIL; +ctx = state_server.lib_state.lib_ctx; /* In OpenSSL, cipher components are separated by hyphens. In GnuTLS, they were historically separated by underscores. So that I can use either form in my @@ -2651,13 +3173,16 @@ for TLS 1.3 . Since we do not call it at present we get the default list: TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256 */ -if (expciphers) +if (state_server.lib_state.pri_string) + { DEBUG(D_tls) debug_printf("TLS: cipher list was preloaded\n"); } +else { - for (uschar * s = expciphers; *s; s++ ) if (*s == '_') *s = '-'; - DEBUG(D_tls) debug_printf("required ciphers: %s\n", expciphers); - if (!SSL_CTX_set_cipher_list(server_ctx, CS expciphers)) - return tls_error(US"SSL_CTX_set_cipher_list", NULL, NULL, errstr); - cbinfo->server_cipher_list = expciphers; + if (!expand_check(tls_require_ciphers, US"tls_require_ciphers", &expciphers, errstr)) + return FAIL; + + if ( expciphers + && (rc = server_load_ciphers(ctx, &state_server, expciphers, errstr)) != OK) + return rc; } /* If this is a host for which certificate verification is mandatory or @@ -2670,37 +3195,48 @@ tls_in.dane_verified = FALSE; server_verify_callback_called = FALSE; if (verify_check_host(&tls_verify_hosts) == OK) - { - rc = setup_certs(server_ctx, tls_verify_certificates, tls_crl, NULL, - FALSE, verify_callback_server, errstr); - if (rc != OK) return rc; server_verify_optional = FALSE; - } else if (verify_check_host(&tls_try_verify_hosts) == OK) - { - rc = setup_certs(server_ctx, tls_verify_certificates, tls_crl, NULL, - TRUE, verify_callback_server, errstr); - if (rc != OK) return rc; server_verify_optional = TRUE; +else + goto skip_certs; + + { + uschar * expcerts; + if (!expand_check(tls_verify_certificates, US"tls_verify_certificates", + &expcerts, errstr)) + return DEFER; + DEBUG(D_tls) debug_printf("tls_verify_certificates: %s\n", expcerts); + + if (state_server.lib_state.cabundle) + { DEBUG(D_tls) debug_printf("TLS: CA bundle for server was preloaded\n"); } + else + if ((rc = setup_certs(ctx, expcerts, tls_crl, NULL, errstr)) != OK) + return rc; + + if (expcerts && *expcerts) + setup_cert_verify(ctx, server_verify_optional, verify_callback_server); } +skip_certs: ; #ifndef DISABLE_TLS_RESUME -SSL_CTX_set_tlsext_ticket_key_cb(server_ctx, ticket_key_callback); +SSL_CTX_set_tlsext_ticket_key_cb(ctx, ticket_key_callback); /* despite working, appears to always return failure, so ignoring */ #endif #ifdef OPENSSL_HAVE_NUM_TICKETS # ifndef DISABLE_TLS_RESUME -SSL_CTX_set_num_tickets(server_ctx, tls_in.host_resumable ? 1 : 0); +SSL_CTX_set_num_tickets(ctx, tls_in.host_resumable ? 1 : 0); # else -SSL_CTX_set_num_tickets(server_ctx, 0); /* send no TLS1.3 stateful-tickets */ +SSL_CTX_set_num_tickets(ctx, 0); /* send no TLS1.3 stateful-tickets */ # endif #endif /* Prepare for new connection */ -if (!(server_ssl = SSL_new(server_ctx))) +if (!(ssl = SSL_new(ctx))) return tls_error(US"SSL_new", NULL, NULL, errstr); +state_server.lib_state.lib_ssl = ssl; /* Warning: we used to SSL_clear(ssl) here, it was removed. * @@ -2721,7 +3257,7 @@ make them disconnect. We need to have an explicit fflush() here, to force out the response. Other smtp_printf() calls do not need it, because in non-TLS mode, the fflush() happens when smtp_getc() is called. */ -SSL_set_session_id_context(server_ssl, sid_ctx, Ustrlen(sid_ctx)); +SSL_set_session_id_context(ssl, sid_ctx, Ustrlen(sid_ctx)); if (!tls_in.on_connect) { smtp_printf("220 TLS go ahead\r\n", FALSE); @@ -2731,21 +3267,21 @@ if (!tls_in.on_connect) /* Now negotiate the TLS session. We put our own timer on it, since it seems that the OpenSSL library doesn't. */ -SSL_set_wfd(server_ssl, fileno(smtp_out)); -SSL_set_rfd(server_ssl, fileno(smtp_in)); -SSL_set_accept_state(server_ssl); +SSL_set_wfd(ssl, fileno(smtp_out)); +SSL_set_rfd(ssl, fileno(smtp_in)); +SSL_set_accept_state(ssl); DEBUG(D_tls) debug_printf("Calling SSL_accept\n"); ERR_clear_error(); sigalrm_seen = FALSE; if (smtp_receive_timeout > 0) ALARM(smtp_receive_timeout); -rc = SSL_accept(server_ssl); +rc = SSL_accept(ssl); ALARM_CLR(0); if (rc <= 0) { - int error = SSL_get_error(server_ssl, rc); + int error = SSL_get_error(ssl, rc); switch(error) { case SSL_ERROR_NONE: @@ -2755,8 +3291,8 @@ if (rc <= 0) DEBUG(D_tls) debug_printf("Got SSL_ERROR_ZERO_RETURN\n"); (void) tls_error(US"SSL_accept", NULL, sigalrm_seen ? US"timed out" : NULL, errstr); - if (SSL_get_shutdown(server_ssl) == SSL_RECEIVED_SHUTDOWN) - SSL_shutdown(server_ssl); + if (SSL_get_shutdown(ssl) == SSL_RECEIVED_SHUTDOWN) + SSL_shutdown(ssl); tls_close(NULL, TLS_NO_SHUTDOWN); return FAIL; @@ -2771,7 +3307,7 @@ if (rc <= 0) || r == SSL_R_VERSION_TOO_LOW #endif || r == SSL_R_UNKNOWN_PROTOCOL || r == SSL_R_UNSUPPORTED_PROTOCOL) - s = string_sprintf("%s (%s)", s, SSL_get_version(server_ssl)); + s = string_sprintf("%s (%s)", s, SSL_get_version(ssl)); (void) tls_error(s, NULL, sigalrm_seen ? US"timed out" : NULL, errstr); return FAIL; } @@ -2800,7 +3336,7 @@ ERR_clear_error(); /* Even success can leave errors in the stack. Seen with anon-authentication ciphersuite negotiated. */ #ifndef DISABLE_TLS_RESUME -if (SSL_session_reused(server_ssl)) +if (SSL_session_reused(ssl)) { tls_in.resumption |= RESUME_USED; DEBUG(D_tls) debug_printf("Session reused\n"); @@ -2811,31 +3347,31 @@ if (SSL_session_reused(server_ssl)) adjust the input functions to read via TLS, and initialize things. */ #ifdef SSL_get_extms_support -tls_in.ext_master_secret = SSL_get_extms_support(server_ssl) == 1; +tls_in.ext_master_secret = SSL_get_extms_support(ssl) == 1; #endif -peer_cert(server_ssl, &tls_in, peerdn, sizeof(peerdn)); +peer_cert(ssl, &tls_in, peerdn, sizeof(peerdn)); -tls_in.ver = tlsver_name(server_ssl); -tls_in.cipher = construct_cipher_name(server_ssl, tls_in.ver, &tls_in.bits); -tls_in.cipher_stdname = cipher_stdname_ssl(server_ssl); +tls_in.ver = tlsver_name(ssl); +tls_in.cipher = construct_cipher_name(ssl, tls_in.ver, &tls_in.bits); +tls_in.cipher_stdname = cipher_stdname_ssl(ssl); DEBUG(D_tls) { uschar buf[2048]; - if (SSL_get_shared_ciphers(server_ssl, CS buf, sizeof(buf))) + if (SSL_get_shared_ciphers(ssl, CS buf, sizeof(buf))) debug_printf("Shared ciphers: %s\n", buf); #ifdef EXIM_HAVE_OPENSSL_KEYLOG { BIO * bp = BIO_new_fp(debug_file, BIO_NOCLOSE); - SSL_SESSION_print_keylog(bp, SSL_get_session(server_ssl)); + SSL_SESSION_print_keylog(bp, SSL_get_session(ssl)); BIO_free(bp); } #endif #ifdef EXIM_HAVE_SESSION_TICKET { - SSL_SESSION * ss = SSL_get_session(server_ssl); + SSL_SESSION * ss = SSL_get_session(ssl); if (SSL_SESSION_has_ticket(ss)) /* 1.1.0 */ debug_printf("The session has a ticket, life %lu seconds\n", SSL_SESSION_get_ticket_lifetime_hint(ss)); @@ -2845,7 +3381,7 @@ DEBUG(D_tls) /* Record the certificate we presented */ { - X509 * crt = SSL_get_certificate(server_ssl); + X509 * crt = SSL_get_certificate(ssl); tls_in.ourcert = crt ? X509_dup(crt) : NULL; } @@ -2853,10 +3389,10 @@ DEBUG(D_tls) See description in https://paquier.xyz/postgresql-2/channel-binding-openssl/ */ { uschar c, * s; - size_t len = SSL_get_peer_finished(server_ssl, &c, 0); + size_t len = SSL_get_peer_finished(ssl, &c, 0); int old_pool = store_pool; - SSL_get_peer_finished(server_ssl, s = store_get((int)len, FALSE), len); + SSL_get_peer_finished(ssl, s = store_get((int)len, FALSE), len); store_pool = POOL_PERM; tls_in.channelbinding = b64encode_taint(CUS s, (int)len, FALSE); store_pool = old_pool; @@ -2890,7 +3426,7 @@ return OK; static int tls_client_basic_ctx_init(SSL_CTX * ctx, - host_item * host, smtp_transport_options_block * ob, tls_ext_ctx_cb * cbinfo, + host_item * host, smtp_transport_options_block * ob, exim_openssl_state_st * state, uschar ** errstr) { int rc; @@ -2914,21 +3450,33 @@ else if (verify_check_given_host(CUSS &ob->tls_try_verify_hosts, host) == OK) else return OK; -if ((rc = setup_certs(ctx, ob->tls_verify_certificates, - ob->tls_crl, host, client_verify_optional, verify_callback_client, - errstr)) != OK) - return rc; + { + uschar * expcerts; + if (!expand_check(ob->tls_verify_certificates, US"tls_verify_certificates", + &expcerts, errstr)) + return DEFER; + DEBUG(D_tls) debug_printf("tls_verify_certificates: %s\n", expcerts); + + if (state->lib_state.cabundle) + { DEBUG(D_tls) debug_printf("TLS: CA bundle was preloaded\n"); } + else + if ((rc = setup_certs(ctx, expcerts, ob->tls_crl, host, errstr)) != OK) + return rc; + + if (expcerts && *expcerts) + setup_cert_verify(ctx, client_verify_optional, verify_callback_client); + } if (verify_check_given_host(CUSS &ob->tls_verify_cert_hostnames, host) == OK) { - cbinfo->verify_cert_hostnames = + state->verify_cert_hostnames = #ifdef SUPPORT_I18N string_domain_utf8_to_alabel(host->certname, NULL); #else host->certname; #endif DEBUG(D_tls) debug_printf("Cert hostname to check: \"%s\"\n", - cbinfo->verify_cert_hostnames); + state->verify_cert_hostnames); } return OK; } @@ -3063,7 +3611,7 @@ if (tlsp->host_resumable) static int tls_save_session_cb(SSL * ssl, SSL_SESSION * ss) { -tls_ext_ctx_cb * cbinfo = SSL_get_ex_data(ssl, tls_exdata_idx); +exim_openssl_state_st * cbinfo = SSL_get_ex_data(ssl, tls_exdata_idx); tls_support * tlsp; DEBUG(D_tls) debug_printf("tls_save_session_cb\n"); @@ -3129,12 +3677,12 @@ if (tlsp->host_resumable) SSL_clear_options(ssl, SSL_OP_NO_TICKET); tls_exdata_idx = SSL_get_ex_new_index(0, 0, 0, 0, 0); - if (!SSL_set_ex_data(ssl, tls_exdata_idx, client_static_cbinfo)) + if (!SSL_set_ex_data(ssl, tls_exdata_idx, client_static_state)) { tls_error(US"set ex_data", host, NULL, errstr); return FALSE; } - debug_printf("tls_exdata_idx %d cbinfo %p\n", tls_exdata_idx, client_static_cbinfo); + debug_printf("tls_exdata_idx %d cbinfo %p\n", tls_exdata_idx, client_static_state); } tlsp->resumption = RESUME_SUPPORTED; @@ -3231,14 +3779,15 @@ tlsp->tlsa_usage = 0; } #endif -rc = tls_init(&exim_client_ctx->ctx, host, NULL, - ob->tls_certificate, ob->tls_privatekey, +rc = tls_init(host, ob, #ifndef DISABLE_OCSP (void *)(long)request_ocsp, #endif - cookie, &client_static_cbinfo, tlsp, errstr); + cookie, &client_static_state, tlsp, errstr); if (rc != OK) return FALSE; +exim_client_ctx->ctx = client_static_state->lib_state.lib_ctx; + tlsp->certificate_verified = FALSE; client_verify_callback_called = FALSE; @@ -3299,9 +3848,9 @@ else #endif - if (tls_client_basic_ctx_init(exim_client_ctx->ctx, host, ob, - client_static_cbinfo, errstr) != OK) - return FALSE; +if (tls_client_basic_ctx_init(exim_client_ctx->ctx, host, ob, + client_static_state, errstr) != OK) + return FALSE; #ifndef DISABLE_TLS_RESUME tls_client_ctx_resume_prehandshake(exim_client_ctx, tlsp, ob, host); @@ -3369,7 +3918,7 @@ if (request_ocsp) if (request_ocsp) { SSL_set_tlsext_status_type(exim_client_ctx->ssl, TLSEXT_STATUSTYPE_ocsp); - client_static_cbinfo->u_ocsp.client.verify_required = require_ocsp; + client_static_state->u_ocsp.client.verify_required = require_ocsp; tlsp->ocsp = OCSP_NOT_RESP; } #endif @@ -3381,7 +3930,7 @@ if (!tls_client_ssl_resume_prehandshake(exim_client_ctx->ssl, tlsp, host, #endif #ifndef DISABLE_EVENT -client_static_cbinfo->event_action = tb ? tb->event_action : NULL; +client_static_state->event_action = tb ? tb->event_action : NULL; #endif /* There doesn't seem to be a built-in timeout on connection. */ @@ -3461,17 +4010,18 @@ return TRUE; static BOOL tls_refill(unsigned lim) { +SSL * ssl = state_server.lib_state.lib_ssl; int error; int inbytes; -DEBUG(D_tls) debug_printf("Calling SSL_read(%p, %p, %u)\n", server_ssl, +DEBUG(D_tls) debug_printf("Calling SSL_read(%p, %p, %u)\n", ssl, ssl_xfer_buffer, ssl_xfer_buffer_size); ERR_clear_error(); if (smtp_receive_timeout > 0) ALARM(smtp_receive_timeout); -inbytes = SSL_read(server_ssl, CS ssl_xfer_buffer, +inbytes = SSL_read(ssl, CS ssl_xfer_buffer, MIN(ssl_xfer_buffer_size, lim)); -error = SSL_get_error(server_ssl, inbytes); +error = SSL_get_error(ssl, inbytes); if (smtp_receive_timeout > 0) ALARM_CLR(0); if (had_command_timeout) /* set by signal handler */ @@ -3495,8 +4045,8 @@ switch(error) case SSL_ERROR_ZERO_RETURN: DEBUG(D_tls) debug_printf("Got SSL_ERROR_ZERO_RETURN\n"); - if (SSL_get_shutdown(server_ssl) == SSL_RECEIVED_SHUTDOWN) - SSL_shutdown(server_ssl); + if (SSL_get_shutdown(ssl) == SSL_RECEIVED_SHUTDOWN) + SSL_shutdown(ssl); tls_close(NULL, TLS_NO_SHUTDOWN); return FALSE; @@ -3587,7 +4137,8 @@ if (n > 0) BOOL tls_could_read(void) { -return ssl_xfer_buffer_lwm < ssl_xfer_buffer_hwm || SSL_pending(server_ssl) > 0; +return ssl_xfer_buffer_lwm < ssl_xfer_buffer_hwm + || SSL_pending(state_server.lib_state.lib_ssl) > 0; } @@ -3610,7 +4161,8 @@ Only used by the client-side TLS. int tls_read(void * ct_ctx, uschar *buff, size_t len) { -SSL * ssl = ct_ctx ? ((exim_openssl_client_tls_ctx *)ct_ctx)->ssl : server_ssl; +SSL * ssl = ct_ctx ? ((exim_openssl_client_tls_ctx *)ct_ctx)->ssl + : state_server.lib_state.lib_ssl; int inbytes; int error; @@ -3660,7 +4212,8 @@ tls_write(void * ct_ctx, const uschar * buff, size_t len, BOOL more) size_t olen = len; int outbytes, error; SSL * ssl = ct_ctx - ? ((exim_openssl_client_tls_ctx *)ct_ctx)->ssl : server_ssl; + ? ((exim_openssl_client_tls_ctx *)ct_ctx)->ssl + : state_server.lib_state.lib_ssl; static gstring * server_corked = NULL; gstring ** corkedp = ct_ctx ? &((exim_openssl_client_tls_ctx *)ct_ctx)->corked : &server_corked; @@ -3772,8 +4325,7 @@ void tls_close(void * ct_ctx, int shutdown) { exim_openssl_client_tls_ctx * o_ctx = ct_ctx; -SSL_CTX **ctxp = o_ctx ? &o_ctx->ctx : &server_ctx; -SSL **sslp = o_ctx ? &o_ctx->ssl : &server_ssl; +SSL **sslp = o_ctx ? &o_ctx->ssl : (SSL **) &state_server.lib_state.lib_ssl; int *fdp = o_ctx ? &tls_out.active.sock : &tls_in.active.sock; if (*fdp < 0) return; /* TLS was not active */ @@ -3802,8 +4354,8 @@ if (shutdown) if (!o_ctx) /* server side */ { #ifndef DISABLE_OCSP - sk_X509_pop_free(server_static_cbinfo->verify_stack, X509_free); - server_static_cbinfo->verify_stack = NULL; + sk_X509_pop_free(state_server.verify_stack, X509_free); + state_server.verify_stack = NULL; #endif receive_getc = smtp_getc; @@ -3818,9 +4370,7 @@ if (!o_ctx) /* server side */ /* Leave bits, peercert, cipher, peerdn, certificate_verified set, for logging */ } -SSL_CTX_free(*ctxp); SSL_free(*sslp); -*ctxp = NULL; *sslp = NULL; *fdp = -1; } @@ -3862,28 +4412,20 @@ while (*s != 0) { if (*s == '_') *s = '-'; s++; } err = NULL; -#ifdef EXIM_HAVE_OPENSSL_TLS_METHOD -if (!(ctx = SSL_CTX_new(TLS_server_method()))) -#else -if (!(ctx = SSL_CTX_new(SSLv23_server_method()))) -#endif +if (lib_ctx_new(&ctx, NULL, &err) == OK) { - ERR_error_string_n(ERR_get_error(), ssl_errstring, sizeof(ssl_errstring)); - return string_sprintf("SSL_CTX_new() failed: %s", ssl_errstring); - } + DEBUG(D_tls) + debug_printf("tls_require_ciphers expands to \"%s\"\n", expciphers); -DEBUG(D_tls) - debug_printf("tls_require_ciphers expands to \"%s\"\n", expciphers); + if (!SSL_CTX_set_cipher_list(ctx, CS expciphers)) + { + ERR_error_string_n(ERR_get_error(), ssl_errstring, sizeof(ssl_errstring)); + err = string_sprintf("SSL_CTX_set_cipher_list(%s) failed: %s", + expciphers, ssl_errstring); + } -if (!SSL_CTX_set_cipher_list(ctx, CS expciphers)) - { - ERR_error_string_n(ERR_get_error(), ssl_errstring, sizeof(ssl_errstring)); - err = string_sprintf("SSL_CTX_set_cipher_list(%s) failed: %s", - expciphers, ssl_errstring); + SSL_CTX_free(ctx); } - -SSL_CTX_free(ctx); - return err; } diff --git a/src/src/tls.c b/src/src/tls.c index e5aabc6b4..ffcc8598c 100644 --- a/src/src/tls.c +++ b/src/src/tls.c @@ -36,6 +36,15 @@ functions from the OpenSSL or GNU TLS libraries. */ #ifndef MACRO_PREDEF +static void tls_per_lib_daemon_init(void); +static void tls_per_lib_daemon_tick(void); +static void tls_server_creds_init(void); +static void tls_server_creds_invalidate(void); +static void tls_client_creds_init(transport_instance *, BOOL); +static void tls_client_creds_invalidate(transport_instance *); + + + /* This module is compiled only when it is specifically requested in the build-time configuration. However, some compilers don't like compiling empty modules, so keep them happy with a dummy when skipping the rest. Make it @@ -45,7 +54,9 @@ loops. */ #ifdef DISABLE_TLS static void dummy(int x) { dummy(x-1); } -#else +#else /* most of the rest of the file */ + +const exim_tlslib_state null_tls_preload = {0}; /* Static variables that are used for buffering data by both sets of functions and the common functions below. @@ -96,6 +107,154 @@ return TRUE; } +#ifdef EXIM_HAVE_INOTIFY +/* Add the directory for a filename to the inotify handle, creating that if +needed. This is enough to see changes to files in that dir. +Return boolean success. + +The word "system" fails, which is on the safe side as we don't know what +directory it implies nor if the TLS library handles a watch for us. + +The string "system,cache" is recognised and explicitly accepted without +setting a watch. This permits the system CA bundle to be cached even though +we have no way to tell when it gets modified by an update. + +We *might* try to run "openssl version -d" and set watches on the dir +indicated in its output, plus the "certs" subdir of it (following +synlimks for both). But this is undocumented even for OpenSSL, and +who knows what GnuTLS might be doing. + +A full set of caching including the CAs takes 35ms output off of the +server tls_init() (GnuTLS, Fedora 32, 2018-class x86_64 laptop hardware). +*/ +static BOOL +tls_set_one_watch(const uschar * filename) +{ +uschar * s; + +if (Ustrcmp(filename, "system,cache") == 0) return TRUE; + +if (!(s = Ustrrchr(filename, '/'))) return FALSE; +s = string_copyn(filename, s - filename); +DEBUG(D_tls) debug_printf("watch dir '%s'\n", s); + +if (inotify_add_watch(tls_watch_fd, CCS s, + IN_ONESHOT | IN_CLOSE_WRITE | IN_DELETE | IN_DELETE_SELF + | IN_MOVED_FROM | IN_MOVED_TO | IN_MOVE_SELF) >= 0) + return TRUE; +DEBUG(D_tls) debug_printf("add_watch: %s\n", strerror(errno)); +return FALSE; +} + + +/* Create an inotify facility if needed. +Then set watches on the dir containing the given file or (optionally) +list of files. Return boolean success. */ + +static BOOL +tls_set_watch(const uschar * filename, BOOL list) +{ +rmark r; +BOOL rc = FALSE; + +if (tls_watch_fd < 0 && (tls_watch_fd = inotify_init1(O_CLOEXEC)) < 0) + { + DEBUG(D_tls) debug_printf("inotify_init: %s\n", strerror(errno)); + return FALSE; + } + +if (!filename || !*filename) return TRUE; + +r = store_mark(); + +if (list) + { + int sep = 0; + for (uschar * s; s = string_nextinlist(&filename, &sep, NULL, 0); ) + if (!(rc = tls_set_one_watch(s))) break; + } +else + rc = tls_set_one_watch(filename); + +store_reset(r); +return rc; +} + + +void +tls_client_creds_reload(BOOL watch) +{ +for(transport_instance * t = transports; t; t = t->next) + if (Ustrcmp(t->driver_name, "smtp") == 0) + { + tls_client_creds_invalidate(t); + tls_client_creds_init(t, watch); + } +} + +static void +tls_daemon_creds_reload(void) +{ +tls_server_creds_invalidate(); +tls_server_creds_init(); + +tls_client_creds_reload(TRUE); +} + + +/* Called, after a delay for multiple file ops to get done, from +the daemon when any of the watches added (above) fire. + +Dump the set of watches and arrange to reload cached creds (which +will set up new watches). */ + +static void +tls_watch_triggered(void) +{ +DEBUG(D_tls) debug_printf("watch triggered\n"); +close(tls_watch_fd); +tls_watch_fd = -1; + +tls_daemon_creds_reload(); +} + + +/* Utility predicates for use by the per-library code */ +static BOOL +opt_set_and_noexpand(const uschar * opt) +{ return opt && *opt && Ustrchr(opt, '$') == NULL; } + +static BOOL +opt_unset_or_noexpand(const uschar * opt) +{ return !opt || Ustrchr(opt, '$') == NULL; } + +#endif /* EXIM_HAVE_INOTIFY */ + + +/* Called every time round the daemon loop */ + +void +tls_daemon_tick(void) +{ +tls_per_lib_daemon_tick(); +#ifdef EXIM_HAVE_INOTIFY +if (tls_watch_trigger_time && time(NULL) >= tls_watch_trigger_time + 5) + { + tls_watch_trigger_time = 0; + tls_watch_triggered(); + } +#endif +} + +/* Called once at daemon startup */ + +void +tls_daemon_init(void) +{ +tls_per_lib_daemon_init(); +} + + /************************************************* * Timezone environment flipping * *************************************************/ diff --git a/src/src/transports/smtp.c b/src/src/transports/smtp.c index cdee92822..0a3d8f1e9 100644 --- a/src/src/transports/smtp.c +++ b/src/src/transports/smtp.c @@ -163,23 +163,12 @@ void smtp_transport_closedown(transport_instance *tblock) {} /* Default private options block for the smtp transport. */ smtp_transport_options_block smtp_transport_option_defaults = { - .hosts = NULL, - .fallback_hosts = NULL, - .hostlist = NULL, - .fallback_hostlist = NULL, + /* All non-mentioned elements 0/NULL/FALSE */ .helo_data = US"$primary_hostname", - .interface = NULL, - .port = NULL, .protocol = US"smtp", - .dscp = NULL, - .serialize_hosts = NULL, - .hosts_try_auth = NULL, - .hosts_require_auth = NULL, .hosts_try_chunking = US"*", #ifdef SUPPORT_DANE .hosts_try_dane = US"*", - .hosts_require_dane = NULL, - .dane_require_tls_ciphers = NULL, #endif .hosts_try_fastopen = US"*", #ifndef DISABLE_PRDR @@ -187,19 +176,6 @@ smtp_transport_options_block smtp_transport_option_defaults = { #endif #ifndef DISABLE_OCSP .hosts_request_ocsp = US"*", /* hosts_request_ocsp (except under DANE; tls_client_start()) */ - .hosts_require_ocsp = NULL, -#endif - .hosts_require_tls = NULL, - .hosts_avoid_tls = NULL, - .hosts_verify_avoid_tls = NULL, - .hosts_avoid_pipelining = NULL, -#ifndef DISABLE_PIPE_CONNECT - .hosts_pipe_connect = NULL, -#endif - .hosts_avoid_esmtp = NULL, -#ifndef DISABLE_TLS - .hosts_nopass_tls = NULL, - .hosts_noproxy_tls = NULL, #endif .command_timeout = 5*60, .connect_timeout = 5*60, @@ -210,35 +186,15 @@ smtp_transport_options_block smtp_transport_option_defaults = { .hosts_max_try_hardlimit = 50, .message_linelength_limit = 998, .address_retry_include_sender = TRUE, - .allow_localhost = FALSE, - .authenticated_sender_force = FALSE, - .gethostbyname = FALSE, .dns_qualify_single = TRUE, - .dns_search_parents = FALSE, .dnssec = { .request= US"*", .require=NULL }, .delay_after_cutoff = TRUE, - .hosts_override = FALSE, - .hosts_randomize = FALSE, .keepalive = TRUE, - .lmtp_ignore_quota = FALSE, - .expand_retry_include_ip_address = NULL, .retry_include_ip_address = TRUE, -#ifdef SUPPORT_SOCKS - .socks_proxy = NULL, -#endif #ifndef DISABLE_TLS - .tls_certificate = NULL, - .tls_crl = NULL, - .tls_privatekey = NULL, - .tls_require_ciphers = NULL, - .tls_sni = NULL, .tls_verify_certificates = US"system", .tls_dh_min_bits = EXIM_CLIENT_DH_DEFAULT_MIN_BITS, .tls_tempfail_tryclear = TRUE, -# ifndef DISABLE_TLS_RESUME - .tls_resumption_hosts = NULL, -# endif - .tls_verify_hosts = NULL, .tls_try_verify_hosts = US"*", .tls_verify_cert_hostnames = US"*", #endif @@ -247,24 +203,7 @@ smtp_transport_options_block smtp_transport_option_defaults = { #endif #ifndef DISABLE_DKIM .dkim = - {.dkim_domain = NULL, - .dkim_identity = NULL, - .dkim_private_key = NULL, - .dkim_selector = NULL, - .dkim_canon = NULL, - .dkim_sign_headers = NULL, - .dkim_strict = NULL, - .dkim_hash = US"sha256", - .dkim_timestamps = NULL, - .dot_stuffed = FALSE, - .force_bodyhash = FALSE, -# ifdef EXPERIMENTAL_ARC - .arc_signspec = NULL, -# endif - }, -# ifdef EXPERIMENTAL_ARC - .arc_sign = NULL, -# endif + { .dkim_hash = US"sha256", }, #endif }; diff --git a/src/src/transports/smtp.h b/src/src/transports/smtp.h index 189ad9caa..7feb8110b 100644 --- a/src/src/transports/smtp.h +++ b/src/src/transports/smtp.h @@ -13,6 +13,26 @@ #define PENDING_OK (PENDING + OK) +#if !defined(DISABLE_TLS) && defined(EXIM_HAVE_INOTIFY) +/* Flags structure for validity of TLS configuration */ + +typedef struct { + BOOL conn_certs:1; /* certificates etc. loaded */ + BOOL cabundle:1; /* CA certificates loaded */ + BOOL crl:1; /* CRL loaded */ + BOOL pri_string:1; /* cipher priority-string cache loaded */ + BOOL dh:1; /* Diffie-Helman params loaded */ + BOOL ecdh:1; /* EC Diffie-Helman params loaded */ + + BOOL ca_rdn_emulate:1; /* do not advertise usable-cert list */ + BOOL ocsp_hook:1; /* need hshake callback on session */ + + void * libdata0; /* library-dependent preloaded data */ + void * libdata1; /* library-dependent preloaded data */ +} exim_tlslib_state; +#endif + + /* Private structure for the private options and other private data. */ typedef struct { @@ -105,6 +125,9 @@ typedef struct { #ifdef EXPERIMENTAL_ARC uschar *arc_sign; #endif +#if !defined(DISABLE_TLS) && defined(EXIM_HAVE_INOTIFY) + exim_tlslib_state tls_preload; +#endif } smtp_transport_options_block; #define SOB (smtp_transport_options_block *) diff --git a/test/confs/1102 b/test/confs/1102 new file mode 100644 index 000000000..2bab6e804 --- /dev/null +++ b/test/confs/1102 @@ -0,0 +1,27 @@ +# Exim test configuration 1102 + +.include DIR/aux-var/tls_conf_prefix + +primary_hostname = myhost.test.ex + +# ----- Main settings ----- + +tls_advertise_hosts = * + +tls_certificate = DIR/tmp/certs/servercert +tls_privatekey = DIR/tmp/certs/serverkey +#tls_verify_certificates = DIR/aux-fixed/cert2 +tls_verify_certificates = system,cache + +queue_only +log_selector = +millisec + +# --- ACL --- + +acl_smtp_rcpt = acl_check_rcpt + +begin acl +acl_check_rcpt: + accept logwrite = server cert: CN=${certextract{subject,CN}{$tls_in_ourcert}} + +# End diff --git a/test/confs/1103 b/test/confs/1103 new file mode 100644 index 000000000..b937ee99c --- /dev/null +++ b/test/confs/1103 @@ -0,0 +1,43 @@ +# Exim test configuration 1103 + +.include DIR/aux-var/tls_conf_prefix + +primary_hostname = myhost.test.ex + +# ----- Main settings ----- + +tls_advertise_hosts = * + +tls_certificate = DIR/tmp/certs/servercert +tls_privatekey = DIR/tmp/certs/serverkey +tls_try_verify_hosts = * +tls_verify_certificates = DIR/aux-fixed/cert2 +#tls_verify_certificates = system,cache + +queue_only +log_selector = +millisec + +# --- ACL --- + +acl_smtp_rcpt = accept + +# ---- + +begin routers + +all: + driver = accept + transport = smtp + +begin transports + +smtp: + driver = smtp + hosts = 127.0.0.1 + allow_localhost + port = PORT_D + tls_certificate = DIR/aux-fixed/cert2 + tls_verify_certificates = DIR/aux-fixed/cert1 + tls_verify_cert_hostnames = : + +# End diff --git a/test/confs/2025 b/test/confs/2025 index 8c08abebe..5ddeb7573 100644 --- a/test/confs/2025 +++ b/test/confs/2025 @@ -16,13 +16,8 @@ queue_only queue_run_in_order tls_advertise_hosts = * - tls_require_ciphers = NORMAL:-VERS-ALL:+VERS-TLS1.2:-MAC-ALL:+SHA256 - -# Set certificate only if server - -tls_certificate = ${if eq {SERVER}{server}{DIR/aux-fixed/cert1}fail} -tls_privatekey = ${if eq {SERVER}{server}{DIR/aux-fixed/cert1}fail} +tls_certificate = DIR/aux-fixed/cert1 # ----- Routers ----- diff --git a/test/confs/2100 b/test/confs/2100 index 827d93811..63a2f255a 100644 --- a/test/confs/2100 +++ b/test/confs/2100 @@ -22,11 +22,13 @@ tls_advertise_hosts = * # Set certificate only if server -tls_certificate = ${if eq {SERVER}{server}{DIR/aux-fixed/cert1}fail} -tls_privatekey = ${if eq {SERVER}{server}{DIR/aux-fixed/cert1}fail} +#tls_certificate = ${if eq {SERVER}{server}{DIR/aux-fixed/cert1}fail} +tls_certificate = DIR/aux-fixed/cert1 +#tls_privatekey = ${if eq {SERVER}{server}{DIR/aux-fixed/cert1}fail} tls_verify_hosts = * -tls_verify_certificates = ${if eq {SERVER}{server}{DIR/aux-fixed/cert2}fail} +#tls_verify_certificates = ${if eq {SERVER}{server}{DIR/aux-fixed/cert2}fail} +tls_verify_certificates = DIR/aux-fixed/cert2 # ----- Routers ----- diff --git a/test/confs/2113 b/test/confs/2113 index fc2c72215..b992747e2 100644 --- a/test/confs/2113 +++ b/test/confs/2113 @@ -66,7 +66,6 @@ send_to_server: port = PORT_D hosts_try_fastopen = : hosts_noproxy_tls = PEX - tls_try_verify_hosts = : tls_verify_certificates = DIR/aux-fixed/cert1 tls_verify_cert_hostnames = : diff --git a/test/confs/4060 b/test/confs/4060 index f3aa84d66..b6e071202 100644 --- a/test/confs/4060 +++ b/test/confs/4060 @@ -21,7 +21,11 @@ gecos_name = CALLER_NAME dns_cname_loops = 9 chunking_advertise_hosts = OPT tls_advertise_hosts = * -tls_certificate = ${if eq {SERVER}{server}{DIR/aux-fixed/cert1}fail} +tls_certificate = DIR/aux-fixed/cert1 + +.ifdef _HAVE_TLS_CA_CACHE +tls_verify_certificates = system,cache +.endif .ifdef _HAVE_DMARC dmarc_tld_file = diff --git a/test/log/1103 b/test/log/1103 new file mode 100644 index 000000000..28d97d389 --- /dev/null +++ b/test/log/1103 @@ -0,0 +1,9 @@ +2017-07-30 18:51:05.712 10HmaX-0005vi-00 <= CALLER@myhost.test.ex U=CALLER P=local S=sss + +******** SERVER ******** +2017-07-30 18:51:05.712 exim x.yz daemon started: pid=pppp, -q7s, listening for SMTP on port PORT_D +2017-07-30 18:51:05.712 Start queue run: pid=pppp +2017-07-30 18:51:05.712 10HmaY-0005vi-00 <= CALLER@myhost.test.ex H=localhost (myhost.test.ex) [127.0.0.1] P=esmtps X=TLS1.x:ke-RSA-AES256-SHAnnn:xxx CV=yes S=sss id=E10HmaX-0005vi-00@myhost.test.ex +2017-07-30 18:51:05.712 10HmaX-0005vi-00 => test@example.com R=all T=smtp H=127.0.0.1 [127.0.0.1] X=TLS1.x:ke-RSA-AES256-SHAnnn:xxx CV=yes C="250 OK id=10HmaY-0005vi-00" +2017-07-30 18:51:05.712 10HmaX-0005vi-00 Completed +2017-07-30 18:51:05.712 End queue run: pid=pppp diff --git a/test/runtest b/test/runtest index 4972aedd8..f61b016f9 100755 --- a/test/runtest +++ b/test/runtest @@ -1067,6 +1067,24 @@ RESET_AFTER_EXTRA_LINE_READ: # this is timing-dependent next if /^OpenSSL: creating STEK$/; + # only OpenSSL speaks of these + next if /^TLS: preloading DH params for server/; + next if /^Diffie-Hellman initialized from default/; + next if /^TLS: preloading ECDH curve for server/; + next if /^ECDH OpenSSL [\d.+]+ temp key parameter settings: default selection$/; + next if /^watch dir '\/etc\/pki\/tls'$/; + + # only GnuTLS speaks of these + next if /^GnuTLS global init required$/; + next if /^TLS: basic cred init, server/; + next if /^TLS: preloading cipher list for server: NULL$/; + s/^GnuTLS using default session cipher\/priority "NORMAL"$/TLS: not preloading cipher list for server/; + next if /^GnuTLS<2>: added \d+ protocols, \d+ ciphersuites, \d+ sig algos and \d+ groups into priority list$/; + + # there happen in different orders for OpenSSL/GnuTLS/noTLS + next if /^TLS: not preloading (CA bundle|cipher list) for server$/; + next if /^TLS: not preloading server certs$/; + # drop lookups next if /^Lookups \(built-in\):/; next if /^Loading lookup modules from/; @@ -1275,6 +1293,14 @@ RESET_AFTER_EXTRA_LINE_READ: next if /^date:\w+,\{SP\}/; next if /^DKIM \[[^[]+\] (Header hash|b) computed:/; + # Timing variable over runs. Collapse repeated memssages. + if (/notify triggered queue run/) + { + my $line = $_; + while (/notify triggered queue run/) { $_ = ; } + $_ = $line . $_; + } + # Not all platforms support TCP Fast Open, and the compile omits the check if (s/\S+ in hosts_try_fastopen\? (no \(option unset\)|no \(end of list\)|yes \(matched "\*"\))\n$//) { diff --git a/test/scripts/1100-Basic-TLS/1101 b/test/scripts/1100-Basic-TLS/1101 index 0bcefa9fd..41407e80b 100644 --- a/test/scripts/1100-Basic-TLS/1101 +++ b/test/scripts/1100-Basic-TLS/1101 @@ -1,4 +1,5 @@ -# TLS server: uncork in pipelining mode, fixed in bd95ffc2ba87fbd3c752df17bc8fd9c01586d45a +# TLS server: uncork in pipelining mode +# fixed in bd95ffc2ba87fbd3c752df17bc8fd9c01586d45a exim -DSERVER=server -bd -oX PORT_D:PORT_S **** client-anytls 127.0.0.1 PORT_D diff --git a/test/scripts/1100-Basic-TLS/1102 b/test/scripts/1100-Basic-TLS/1102 new file mode 100644 index 000000000..862d26a6e --- /dev/null +++ b/test/scripts/1100-Basic-TLS/1102 @@ -0,0 +1,51 @@ +# TLS server: creds caching +# +# +mkdir -p DIR/tmp/certs +cp DIR/aux-fixed/cert1 DIR/tmp/certs/servercert +cp DIR/aux-fixed/cert1 DIR/tmp/certs/serverkey +# +#exim -d-all+tls+receive+timestamp -DSERVER=server -bd -oX PORT_D +exim -DSERVER=server -bd -oX PORT_D +**** +client-anytls 127.0.0.1 PORT_D +??? 220 +EHLO rhu.barb +????250 +STARTTLS +??? 220 +EHLO rhu.barb +????250 +MAIL FROM:<> +RCPT TO:test@example.com +??? 250 +??? 250 +QUIT +??? 221 +**** +sleep 1 +# Now overwrite the cert. key? +cp DIR/aux-fixed/exim-ca/example.com/server1.example.com/server1.example.com.pem DIR/tmp/certs/servercert +cp DIR/aux-fixed/exim-ca/example.com/server1.example.com/server1.example.com.unlocked.key DIR/tmp/certs/serverkey +# The watch mech waits 5 sec after the last trigger, so give that time to expire the send another message +sleep 7 +client-anytls 127.0.0.1 PORT_D +??? 220 +EHLO rhu.barb +????250 +STARTTLS +??? 220 +EHLO rhu.barb +????250 +MAIL FROM:<> +RCPT TO:test@example.com +??? 250 +??? 250 +QUIT +??? 221 +**** +# +killdaemon +# +sudo rm -fr DIR/tmp +no_msglog_check diff --git a/test/scripts/1100-Basic-TLS/1103 b/test/scripts/1100-Basic-TLS/1103 new file mode 100644 index 000000000..de97e32ef --- /dev/null +++ b/test/scripts/1100-Basic-TLS/1103 @@ -0,0 +1,21 @@ +# TLS client: creds caching +# +# +mkdir -p DIR/tmp/certs +cp DIR/aux-fixed/cert1 DIR/tmp/certs/servercert +cp DIR/aux-fixed/cert1 DIR/tmp/certs/serverkey +# +# load up one message in the queue +exim test@example.com +**** +# +# start the daemon, with a queue-run interval +# this will send the queued message and the receive will re-queue it +#exim -d-all+tls+receive+timestamp -DSERVER=server -bd -q7s -oX PORT_D +exim -DSERVER=server -bd -q7s -oX PORT_D +**** +sleep 1 +killdaemon +# +sudo rm -fr DIR/tmp +no_msglog_check