From: Todd Lyons Date: Thu, 26 Jul 2012 20:31:20 +0000 (-0700) Subject: Bug #198: Add remove_header ACL modifier. X-Git-Tag: exim-4_81_RC1~39 X-Git-Url: https://git.exim.org/users/heiko/exim.git/commitdiff_plain/e7568d513f9b5a853e5cdb5db8b2cc0d53f79981?hp=9471b034a78d53e4bfb951649e9f556cf4e6018c Bug #198: Add remove_header ACL modifier. Used patch from Magnus Holmgren dated 2007-02-20. Added documentation. Added tests to detect proper operation. --- diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt index 19fb321ea..1b5c94787 100644 --- a/doc/doc-docbook/spec.xfpt +++ b/doc/doc-docbook/spec.xfpt @@ -26508,8 +26508,8 @@ duplicates to be written, use the &%logwrite%& modifier instead. If &%log_message%& is not present, a &%warn%& verb just checks its conditions and obeys any &"immediate"& modifiers (such as &%control%&, &%set%&, -&%logwrite%&, and &%add_header%&) that appear before the first failing -condition. There is more about adding header lines in section +&%logwrite%&, &%add_header%&, and &%remove_header%&) that appear before the +first failing condition. There is more about adding header lines in section &<>&. If any condition on a &%warn%& statement cannot be completed (that is, there is @@ -26951,6 +26951,12 @@ all the conditions are true, wherever it appears in an ACL command, whereas effect. +.vitem &*remove_header*&&~=&~<&'text'&> +This modifier specifies one or more header names in a colon-separated list + that are to be removed from an incoming message, assuming, of course, that +the message is ultimately accepted. For details, see section &<>&. + + .vitem &*set*&&~<&'acl_name'&>&~=&~<&'value'&> .cindex "&%set%& ACL modifier" This modifier puts a value into one of the ACL variables (see section @@ -27265,7 +27271,7 @@ Remotely submitted, fixups applied: use &`control = submission`&. .section "Adding header lines in ACLs" "SECTaddheadacl" .cindex "header lines" "adding in an ACL" .cindex "header lines" "position of added lines" -.cindex "&%message%& ACL modifier" +.cindex "&%add_header%& ACL modifier" The &%add_header%& modifier can be used to add one or more extra header lines to an incoming message, as in this example: .code @@ -27307,7 +27313,7 @@ passing data between (for example) the MAIL and RCPT ACLs. If you want to do this, you can use ACL variables, as described in section &<>&. -The &%add_header%& modifier acts immediately it is encountered during the +The &%add_header%& modifier acts immediately as it is encountered during the processing of an ACL. Notice the difference between these two cases: .display &`accept add_header = ADDED: some text`& @@ -27356,6 +27362,77 @@ system filter or in a router or transport. +.section "Removing header lines in ACLs" "SECTremoveheadacl" +.cindex "header lines" "removing in an ACL" +.cindex "header lines" "position of removed lines" +.cindex "&%remove_header%& ACL modifier" +The &%remove_header%& modifier can be used to remove one or more header lines +from an incoming message, as in this example: +.code +warn message = Remove internal headers + remove_header = x-route-mail1 : x-route-mail2 +.endd +The &%remove_header%& modifier is permitted in the MAIL, RCPT, PREDATA, DATA, +MIME, and non-SMTP ACLs (in other words, those that are concerned with +receiving a message). The message must ultimately be accepted for +&%remove_header%& to have any significant effect. You can use &%remove_header%& +with any ACL verb, including &%deny%&, though this is really not useful for +any verb that doesn't result in a delivered message. + +More than one header can be removed at the same time by using a colon separated +list of header names. The header matching is case insensitive. Wildcards are +not permitted, nor is list expansion performed, so you cannot use hostlists to +create a list of headers, however both connection and message variable expansion +are performed (&%$acl_c_*%& and &%$acl_m_*%&), illustrated in this example: +.code +warn hosts = +internal_hosts + set acl_c_ihdrs = x-route-mail1 : x-route-mail2 +warn message = Remove internal headers + remove_header = $acl_c_ihdrs +.endd +Removed header lines are accumulated during the MAIL, RCPT, and predata ACLs. +They are removed from the message before processing the DATA and MIME ACLs. +There is no harm in attempting to remove the same header twice nor is removing +a non-existent header. Further header lines to be removed may be accumulated +during the DATA and MIME ACLs, after which they are removed from the message, +if present. In the case of non-SMTP messages, headers to be removed are +accumulated during the non-SMTP ACLs, and are removed from the message after +all the ACLs have run. If a message is rejected after DATA or by the non-SMTP +ACL, there really is no effect because there is no logging of what headers +would have been removed. + +.cindex "header lines" "removed; visibility of" +Header lines are not visible in string expansions until the DATA phase when it +is received. Any header lines removed in the MAIL, RCPT, and predata ACLs are +not visible in the DATA ACL and MIME ACLs. Similarly, header lines that are +removed by the DATA or MIME ACLs are still visible in those ACLs. Because of +this restriction, you cannot use header lines as a way of controlling data +passed between (for example) the MAIL and RCPT ACLs. If you want to do this, +you should instead use ACL variables, as described in section +&<>&. + +The &%remove_header%& modifier acts immediately as it is encountered during the +processing of an ACL. Notice the difference between these two cases: +.display +&`accept remove_header = X-Internal`& +&` `&<&'some condition'&> + +&`accept `&<&'some condition'&> +&` remove_header = X-Internal`& +.endd +In the first case, the header line is always removed, whether or not the +condition is true. In the second case, the header line is removed only if the +condition is true. Multiple occurrences of &%remove_header%& may occur in the +same ACL statement. All those that are encountered before a condition fails +are honoured. + +&*Warning*&: This facility currently applies only to header lines that are +present during ACL processing. It does NOT remove header lines that are added +in a system filter or in a router or transport. + + + + .section "ACL conditions" "SECTaclconditions" .cindex "&ACL;" "conditions; list of" diff --git a/doc/doc-txt/ChangeLog b/doc/doc-txt/ChangeLog index 108f6051e..2021a2ad8 100644 --- a/doc/doc-txt/ChangeLog +++ b/doc/doc-txt/ChangeLog @@ -35,6 +35,9 @@ TL/01 Bugzilla 1258 - Refactor MAIL FROM optional args processing. TL/02 Add +smtp_confirmation as a default logging option. +TL/03 Bugzilla 198 - Implement remove_header ACL modifier. + Patch by Magnus Holmgren from 2007-02-20. + JH/01 Bugzilla 1201 & 304 - New cutthrough-delivery feature, with TLS support. JH/02 Support "G" suffix to numbers in ${if comparisons. diff --git a/doc/doc-txt/NewStuff b/doc/doc-txt/NewStuff index e684344c9..94307c8b6 100644 --- a/doc/doc-txt/NewStuff +++ b/doc/doc-txt/NewStuff @@ -109,6 +109,9 @@ Version 4.81 11. Routers and transports can now have multiple headers_add and headers_remove option lines. The concatenated list is used. +12. New ACL modifier "remove_header" can remove headers before message gets + handled by routers/transports. + Version 4.80 ------------ diff --git a/src/src/acl.c b/src/src/acl.c index 5cd0c3507..3b23a915b 100644 --- a/src/src/acl.c +++ b/src/src/acl.c @@ -88,6 +88,7 @@ enum { ACLC_ACL, #ifdef WITH_CONTENT_SCAN ACLC_REGEX, #endif + ACLC_REMOVE_HEADER, ACLC_SENDER_DOMAINS, ACLC_SENDERS, ACLC_SET, @@ -150,6 +151,7 @@ static uschar *conditions[] = { #ifdef WITH_CONTENT_SCAN US"regex", #endif + US"remove_header", US"sender_domains", US"senders", US"set", #ifdef WITH_CONTENT_SCAN US"spam", @@ -280,6 +282,7 @@ static uschar cond_expand_at_top[] = { #ifdef WITH_CONTENT_SCAN TRUE, /* regex */ #endif + TRUE, /* remove_header */ FALSE, /* sender_domains */ FALSE, /* senders */ TRUE, /* set */ @@ -340,6 +343,7 @@ static uschar cond_modifiers[] = { #ifdef WITH_CONTENT_SCAN FALSE, /* regex */ #endif + TRUE, /* remove_header */ FALSE, /* sender_domains */ FALSE, /* senders */ TRUE, /* set */ @@ -465,6 +469,12 @@ static unsigned int cond_forbids[] = { (1<next) break; #endif + case ACLC_REMOVE_HEADER: + setup_remove_header(arg); + break; + case ACLC_SENDER_DOMAINS: { uschar *sdomain; diff --git a/src/src/globals.c b/src/src/globals.c index 21122f0f9..bcbe12d82 100644 --- a/src/src/globals.c +++ b/src/src/globals.c @@ -196,6 +196,7 @@ uschar *acl_not_smtp = NULL; uschar *acl_not_smtp_mime = NULL; #endif uschar *acl_not_smtp_start = NULL; +uschar *acl_removed_headers = NULL; uschar *acl_smtp_auth = NULL; uschar *acl_smtp_connect = NULL; uschar *acl_smtp_data = NULL; diff --git a/src/src/globals.h b/src/src/globals.h index 783eb7ba3..16caa41e9 100644 --- a/src/src/globals.h +++ b/src/src/globals.h @@ -142,6 +142,7 @@ extern uschar *acl_not_smtp; /* ACL run for non-SMTP messages */ extern uschar *acl_not_smtp_mime; /* For MIME parts of ditto */ #endif extern uschar *acl_not_smtp_start; /* ACL run at the beginning of a non-SMTP session */ +extern uschar *acl_removed_headers; /* Headers deleted by an ACL */ extern uschar *acl_smtp_auth; /* ACL run for AUTH */ extern uschar *acl_smtp_connect; /* ACL run on SMTP connection */ extern uschar *acl_smtp_data; /* ACL run after DATA received */ diff --git a/src/src/receive.c b/src/src/receive.c index 636913b96..7b51805dc 100644 --- a/src/src/receive.c +++ b/src/src/receive.c @@ -936,6 +936,40 @@ add_acl_headers(uschar *acl_name) { header_line *h, *next; header_line *last_received = NULL; +int sep = ':'; + +if (acl_removed_headers != NULL) + { + DEBUG(D_receive|D_acl) debug_printf(">>Headers removed by %s ACL:\n", acl_name); + + for (h = header_list; h != NULL; h = h->next) + { + int i; + uschar *list; + BOOL include_header; + + if (h->type == htype_old) continue; + + include_header = TRUE; + list = acl_removed_headers; + + int sep = ':'; /* This is specified as a colon-separated list */ + uschar *s; + uschar buffer[128]; + while ((s = string_nextinlist(&list, &sep, buffer, sizeof(buffer))) + != NULL) + { + int len = Ustrlen(s); + if (header_testname(h, s, len, FALSE)) + { + h->type = htype_old; + DEBUG(D_receive|D_acl) debug_printf(" %s", h->text); + } + } + } + acl_removed_headers = NULL; + DEBUG(D_receive|D_acl) debug_printf(">>\n"); + } if (acl_added_headers == NULL) return; DEBUG(D_receive|D_acl) debug_printf(">>Headers added by %s ACL:\n", acl_name); diff --git a/src/src/smtp_in.c b/src/src/smtp_in.c index db7f133ca..b1fea9daf 100644 --- a/src/src/smtp_in.c +++ b/src/src/smtp_in.c @@ -1041,6 +1041,7 @@ cancel_cutthrough_connection("smtp reset"); message_linecount = 0; message_size = -1; acl_added_headers = NULL; +acl_removed_headers = NULL; queue_only_policy = FALSE; rcpt_smtp_response = NULL; rcpt_smtp_response_same = TRUE; diff --git a/test/confs/0567 b/test/confs/0567 new file mode 100644 index 000000000..7348b1fe5 --- /dev/null +++ b/test/confs/0567 @@ -0,0 +1,91 @@ +# Exim test configuration 0532 + +CONNECTCOND= + +exim_path = EXIM_PATH +host_lookup_order = bydns +primary_hostname = myhost.test.ex +rfc1413_query_timeout = 0s +spool_directory = DIR/spool +log_file_path = DIR/spool/log/%slog +gecos_pattern = "" +gecos_name = CALLER_NAME + +# ----- Main settings ----- + +acl_smtp_connect = connect +acl_smtp_mail = mail +acl_smtp_rcpt = rcpt +acl_smtp_predata = predata +acl_smtp_data = data +acl_not_smtp = notsmtp + +qualify_domain = test.ex +trusted_users = CALLER + +hostlist internal_headers = x-mail-2 : x-mail-3 + + +# ----- ACL ----- + +begin acl + +connect: + accept CONNECTCOND + +mail: + accept remove_header = x-mail-1 + senders = mailok@test.ex + # Won't work because doesn't expand + remove_header = +internal_headers + accept + +rcpt: + accept local_parts = rcptok + remove_header = x-rcpt-4 : x-rcpt-2 + set acl_m_hdr = x-predata-1 + deny add_header = RCPT: denied $local_part + + +predata: + warn remove_header = x-predata-3 : $acl_m_hdr + # Won't work because doesn't use wildcards + accept remove_header = x-not-* + +data: + warn log_message = Verified previously removed header X-Rcpt-2 + condition = ${if eq{$h_x-rcpt-2:}{}} + warn remove_header = x-data-1 : x-data-4 + condition = ${if eq{$h_cond:}{accept}} + remove_header = x-data-3 + # Won't delete this header because condition fails before the modifier + warn condition = ${if eq{$h_cond:}{reject}} + remove_header = x-data-2 + warn log_message = Verified removed header X-Data-3 in this ACL still visible + condition = ${if !eq{$h_x-data-3:}{}} + accept + +notsmtp: + # Will remove a required header (Date) if told to + accept remove_header = x-notsmtp-1 : date + + +# ----- Routers ----- + +begin routers + +r1: + driver = accept + transport = t1 + + +# ----- Transports ----- + +begin transports + +t1: + driver = appendfile + file = DIR/test-mail/$local_part + user = CALLER + +# End diff --git a/test/log/0567 b/test/log/0567 new file mode 100644 index 000000000..1bad60ee9 --- /dev/null +++ b/test/log/0567 @@ -0,0 +1,10 @@ +1999-03-02 09:44:33 U=CALLER F= rejected RCPT +1999-03-02 09:44:33 10HmaX-0005vi-00 U=CALLER Warning: Verified previously removed header X-Rcpt-2 +1999-03-02 09:44:33 10HmaX-0005vi-00 U=CALLER Warning: Verified removed header X-Data-3 in this ACL still visible +1999-03-02 09:44:33 10HmaX-0005vi-00 <= mailok@test.ex U=CALLER P=local-smtp S=sss +1999-03-02 09:44:33 10HmaX-0005vi-00 => rcptok R=r1 T=t1 +1999-03-02 09:44:33 10HmaX-0005vi-00 Completed +1999-03-02 09:44:33 10HmaY-0005vi-00 <= CALLER@test.ex U=CALLER P=local S=sss +1999-03-02 09:44:33 10HmaY-0005vi-00 => rcptok R=r1 T=t1 +1999-03-02 09:44:33 10HmaY-0005vi-00 Completed +1999-03-02 09:44:33 U=CALLER temporarily rejected connection in "connect" ACL: cannot use remove_header condition in connection ACL diff --git a/test/mail/0567.rcptok b/test/mail/0567.rcptok new file mode 100644 index 000000000..deffea8fc --- /dev/null +++ b/test/mail/0567.rcptok @@ -0,0 +1,45 @@ +From mailok@test.ex Tue Mar 02 09:44:33 1999 +Received: from CALLER by myhost.test.ex with local-smtp (Exim x.yz) + (envelope-from ) + id 10HmaX-0005vi-00 + for rcptok@test.ex; Tue, 2 Mar 1999 09:44:33 +0000 +cond: accept +X-Data-2: Line two +X-Data-5: Line five +X-Not-1: Testing wildcard one +X-Not-2: Testing wildcard two +X-Rcpt-1: Line six +X-Rcpt-3: Line eight +X-Rcpt-5: Line ten +X-Mail-2: Line twelve +X-Mail-3: Line thirteen +X-Mail-4: Line fourteen is also really long, but it won't get + removed by these ACL's. +X-Mail-5: Line fifteen +X-Predata-5: Line sixteen +X-Predata-4: Line seventeen +X-Predata-2: Line nineteen +X-NotSMTP-1: Line twenty-one +X-NotSMTP-2: Line twenty-two +X-NotSMTP-3: Line twenty-three +Message-Id: +From: mailok@test.ex +Date: Tue, 2 Mar 1999 09:44:33 +0000 +RCPT: denied notok + +Test message + +From CALLER@test.ex Tue Mar 02 09:44:33 1999 +Received: from CALLER by myhost.test.ex with local (Exim x.yz) + (envelope-from ) + id 10HmaY-0005vi-00 + for rcptok@test.ex; Tue, 2 Mar 1999 09:44:33 +0000 +Message-Id: +From: CALLER_NAME + +Test non-SMTP message. Make sure it doesn't blow up when a header +it wants to remove is not present. This one also overrides the +fixup of adding a Date header because we specified to remove it! +Allow the admin to shoot himself in the foot if he really and +truly wants to. + diff --git a/test/rejectlog/0567 b/test/rejectlog/0567 new file mode 100644 index 000000000..980e2e851 --- /dev/null +++ b/test/rejectlog/0567 @@ -0,0 +1,2 @@ +1999-03-02 09:44:33 U=CALLER F= rejected RCPT +1999-03-02 09:44:33 U=CALLER temporarily rejected connection in "connect" ACL: cannot use remove_header condition in connection ACL diff --git a/test/scripts/0000-Basic/0567 b/test/scripts/0000-Basic/0567 new file mode 100644 index 000000000..5abd06fd6 --- /dev/null +++ b/test/scripts/0000-Basic/0567 @@ -0,0 +1,49 @@ +# remove_header modifier in ACLs +exim -bs -odi +mail from: +rcpt to: +rcpt to: +data +cond: accept +X-Data-1: Line one +X-Data-2: Line two +X-Data-3: Line three +X-Data-4: Line four +X-Data-5: Line five +X-Not-1: Testing wildcard one +X-Not-2: Testing wildcard two +X-Rcpt-1: Line six +X-Rcpt-2: Line seven +X-Rcpt-3: Line eight +X-Rcpt-4: Line nine is really long, so long in fact that it wraps + around to the next line. +X-Rcpt-5: Line ten +X-Mail-1: Line eleven +X-Mail-2: Line twelve +X-Mail-3: Line thirteen +X-Mail-4: Line fourteen is also really long, but it won't get + removed by these ACL's. +X-Mail-5: Line fifteen +X-Predata-5: Line sixteen +X-Predata-4: Line seventeen +X-Predata-3: Line eighteen +X-Predata-2: Line nineteen +X-Predata-1: Line twenty +X-NotSMTP-1: Line twenty-one +X-NotSMTP-2: Line twenty-two +X-NotSMTP-3: Line twenty-three + +Test message +. +quit +**** +exim -odi rcptok@test.ex +Test non-SMTP message. Make sure it doesn't blow up when a header +it wants to remove is not present. This one also overrides the +fixup of adding a Date header because we specified to remove it! +Allow the admin to shoot himself in the foot if he really and +truly wants to. +**** +exim -bs -odi -DCONNECTCOND="remove_header=CONNECT: won't do this" +**** +no_msglog_check diff --git a/test/stdout/0567 b/test/stdout/0567 new file mode 100644 index 000000000..65fd23c6f --- /dev/null +++ b/test/stdout/0567 @@ -0,0 +1,8 @@ +220 myhost.test.ex ESMTP Exim x.yz Tue, 2 Mar 1999 09:44:33 +0000 +250 OK +250 Accepted +550 Administrative prohibition +354 Enter message, ending with "." on a line by itself +250 OK id=10HmaX-0005vi-00 +221 myhost.test.ex closing connection +451 Temporary local problem - please try later