-$Cambridge: exim/doc/doc-txt/ChangeLog,v 1.140 2005/05/23 15:28:37 fanf2 Exp $
+$Cambridge: exim/doc/doc-txt/ChangeLog,v 1.141 2005/05/23 16:58:55 fanf2 Exp $
Change log file for Exim from version 4.21
-------------------------------------------
TF/03 Added the control = fakedefer ACL modifier.
+TF/04 Added the ratelimit ACL condition. See NewStuff for details.
+
Exim version 4.51
-----------------
-$Cambridge: exim/doc/doc-txt/NewStuff,v 1.44 2005/05/23 15:44:06 fanf2 Exp $
+$Cambridge: exim/doc/doc-txt/NewStuff,v 1.45 2005/05/23 16:58:55 fanf2 Exp $
New Features in Exim
--------------------
to be duplicated when the sender retries. Therefore you should not use
fakedefer if the message will be delivered normally.
+TF/04 There is a new ratelimit ACL condition which can be used to measure
+ and control the rate at which clients can send email. This is more
+ powerful than the existing smtp_ratelimit_* options, because those
+ options only control the rate of commands in a single SMTP session,
+ whereas the new ratelimit condition works across all connections
+ (concurrent and sequential) to the same host.
+
+ The syntax of the ratelimit condition is:
+
+ ratelimit = <m> / <p> / <options> / <key>
+
+ If the average client sending rate is greater than m messages per time
+ period p then the condition is true, otherwise it is false.
+
+ The parameter p is the smoothing time constant, in the form of an Exim
+ time interval e.g. 8h for eight hours. A larger time constant means it
+ takes Exim longer to forget a client's past behaviour. The parameter m is
+ the maximum number of messages that a client can send in a fast burst. By
+ increasing both m and p but keeping m/p constant, you can allow a client
+ to send more messages in a burst without changing its overall sending
+ rate limit. Conversely, if m and p are both small then messages must be
+ sent at an even rate.
+
+ The key is used to look up the data used to calcluate the client's
+ average sending rate. This data is stored in a database maintained by
+ Exim in its spool directory alongside the retry database etc. For
+ example, you can limit the sending rate of each authenticated user,
+ independent of the computer they are sending from, by setting the key
+ to $authenticated_id. The default key is $sender_host_address.
+
+ Each ratelimit condition can have up to two options. The first option
+ specifies what Exim measures the rate of, and the second specifies how
+ Exim handles excessively fast clients.
+
+ The per_mail option means that it measures the client's rate of sending
+ messages. This is the default if none of the per_* options is specified.
+
+ The per_conn option means that it measures the client's connection rate.
+
+ The per_byte option limits the sender's email bandwidth. Note that it
+ is best to use this option in the DATA ACL; if it is used in an earlier
+ ACL it relies on the SIZE parameter on the MAIL command, which may be
+ inaccurate or completely missing. You can follow the limit m in the
+ configuration with K, M, or G to specify limits in kilobytes,
+ megabytes, or gigabytes respectively.
+
+ The per_cmd option means that Exim recomputes the rate every time the
+ condition is processed, which can be used to limit the SMTP command rate.
+ The alias per_rcpt is provided for use in the RCPT ACL instead of per_cmd
+ to make it clear that the effect is to limit the rate at which recipients
+ are accepted. Note that in this case the rate limiting engine will see a
+ message with many recipients as a large high-speed burst.
+
+ If a client's average rate is greater than the maximum, the rate
+ limiting engine can react in two possible ways, depending on the
+ presence of the strict or leaky options. This is independent of the
+ other counter-measures (e.g. rejecting the message) that may be
+ specified by the rest of the ACL. The default mode is leaky, which
+ avoids a sender's over-aggressive retry rate preventing it from getting
+ any email through.
+
+ The strict option means that the client's recorded rate is always
+ updated. The effect of this is that Exim measures the client's average
+ rate of attempts to send email, which can be much higher than the
+ maximum. If the client is over the limit it will be subjected to
+ counter-measures until it slows down below the maximum rate.
+
+ The leaky option means that the client's recorded rate is not updated
+ if it is above the limit. The effect of this is that Exim measures the
+ client's average rate of successfully sent email, which cannot be
+ greater than the maximum. If the client is over the limit it will
+ suffer some counter-measures, but it will still be able to send email
+ at the configured maximum rate, whatever the rate of its attempts.
+
+ As a side-effect, the ratelimit condition will set the expansion
+ variables $sender_rate containing the client's computed rate,
+ $sender_rate_limit containing the configured value of m, and
+ $sender_rate_period containing the configured value of p.
+
+ Exim's other ACL facilities are used to define what counter-measures
+ are taken when the rate limit is exceeded. This might be anything from
+ logging a warning (e.g. while measuring existing sending rates in order
+ to define our policy), through time delays to slow down fast senders,
+ up to rejecting the message. For example,
+
+ # Log all senders' rates
+ warn
+ ratelimit = 0 / 1h / strict
+ log_message = \
+ Sender rate $sender_rate > $sender_rate_limit / $sender_rate_period
+
+ # Slow down fast senders
+ warn
+ ratelimit = 100 / 1h / per_rcpt / strict
+ delay = ${eval: 10 * ($sender_rate - $sender_rate_limit) }
+
+ # Keep authenticated users under control
+ deny
+ ratelimit = 100 / 1d / strict / $authenticated_id
+
+ # System-wide rate limit
+ defer
+ message = Sorry, too busy. Try again later.
+ ratelimit = 10 / 1s / $primary_hostname
+
+ # Restrict incoming rate from each host, with a default rate limit
+ # set using a macro and special cases looked up in a table.
+ defer
+ message = Sender rate $sender_rate exceeds \
+ $sender_rate_limit messages per $sender_rate_period
+ ratelimit = ${lookup {$sender_host_address} \
+ cdb {DB/ratelimits.cdb} \
+ {$value} {RATELIMIT} }
+
Version 4.51
------------
-# $Cambridge: exim/src/OS/Makefile-AIX,v 1.2 2005/02/16 16:40:22 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-AIX,v 1.3 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for AIX
# Written by Nick Waterman (nick@cimio.co.uk)
# Needed for vfork() and vfork() only?
-LIBS = -lbsd
+LIBS = -lbsd -lm
# End
-# $Cambridge: exim/src/OS/Makefile-BSDI,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-BSDI,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for BSDI. Its antique link editor
# cannot handle the TextPop overriding.
XLFLAGS=-L$(X11)/lib
X11_LD_LIB=$(X11)/lib
-LIBS_EXIMON=-lSM -lICE -lipc
+LIBS_EXIMON=-lSM -lICE -lipc -lm
EXIMON_TEXTPOP=
EXIWHAT_PS_ARG=-ax
-# $Cambridge: exim/src/OS/Makefile-CYGWIN,v 1.2 2004/11/10 10:36:48 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-CYGWIN,v 1.3 2005/05/23 16:58:55 fanf2 Exp $
# OS-specific file for Cygwin.
HAVE_ICONV = yes
CFLAGS= -g -Wall -O2
-LIBS= -lcrypt -lresolv
+LIBS= -lcrypt -lresolv -lm
LIBS_EXIM= -liconv
EXIWHAT_PS_ARG=-as
EXIWHAT_KILL_SIGNAL=-USR1
-# $Cambridge: exim/src/OS/Makefile-DGUX,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-DGUX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for DGUX
#
CFLAGS=-O2
RANLIB=@true
-LIBS=-lsocket -lnsl
+LIBS=-lsocket -lnsl -lm
LIBRESOLV=-lresolv
DBMLIB=-ldbm
-# $Cambridge: exim/src/OS/Makefile-FreeBSD,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-FreeBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for FreeBSD
# There's no setting of CFLAGS here, to allow the system default
HAVE_SA_LEN=YES
# crypt() is in a separate library
-LIBS=-lcrypt
+LIBS=-lcrypt -lm
# FreeBSD always ships with Berkeley DB
USE_DB=yes
-# $Cambridge: exim/src/OS/Makefile-GNU,v 1.3 2005/01/12 12:25:56 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-GNU,v 1.4 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for GNU and variants.
DBMLIB = -ldb
USE_DB = yes
-LIBS = -lnsl -lcrypt
+LIBS = -lnsl -lcrypt -lm
LIBRESOLV = -lresolv
X11=/usr/X11R6
-# $Cambridge: exim/src/OS/Makefile-GNUkFreeBSD,v 1.1 2005/01/04 10:25:58 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-GNUkFreeBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for GNU and variants.
DBMLIB = -ldb
USE_DB = yes
-LIBS = -lnsl -lcrypt
+LIBS = -lnsl -lcrypt -lm
LIBRESOLV = -lresolv
X11=/usr/X11R6
-# $Cambridge: exim/src/OS/Makefile-GNUkNetBSD,v 1.1 2005/01/04 10:25:58 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-GNUkNetBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for GNU and variants.
DBMLIB = -ldb
USE_DB = yes
-LIBS = -lnsl -lcrypt
+LIBS = -lnsl -lcrypt -lm
LIBRESOLV = -lresolv
X11=/usr/X11R6
-# $Cambridge: exim/src/OS/Makefile-IRIX,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-IRIX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for IRIX
BASENAME_COMMAND=/sbin/basename
HOSTNAME_COMMAND=/usr/bsd/hostname
CFLAGS=-OPT:Olimit=1500
-LIBS=-lmld
+LIBS=-lmld -lm
XINCLUDE=-I/usr/include/X11
vfork=fork
RANLIB=@true
-# $Cambridge: exim/src/OS/Makefile-IRIX6,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-IRIX6,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for IRIX6 on 64-bit systems
HOSTNAME_COMMAND=/usr/bsd/hostname
CFLAGS=-O2 -n32 -OPT:Olimit=4000
LFLAGS=-n32
-LIBS=-lelf
+LIBS=-lelf -lm
XINCLUDE=-I/usr/include/X11
XLFLAGS=
vfork=fork
-# $Cambridge: exim/src/OS/Makefile-IRIX632,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-IRIX632,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for IRIX 6 on 32-bit systems.
# There seems to be some variation. The commented settings show
CFLAGS=-32
LFLAGS=-32
#LIBS=-lmld
-LIBS=-lelf
+LIBS=-lelf -lm
XINCLUDE=-I/usr/include/X11
vfork=fork
RANLIB=@true
-# $Cambridge: exim/src/OS/Makefile-IRIX65,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-IRIX65,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for IRIX 6.5
LFLAGS=-Wl,-LD_MSG:off=85
LFLAGS=
# nlist has moved from libmld to libelf
-LIBS=-lelf
+LIBS=-lelf -lm
XINCLUDE=-I/usr/include/X11
vfork=fork
RANLIB=@true
-# $Cambridge: exim/src/OS/Makefile-Linux,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-Linux,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for Linux. This is for modern Linuxes,
# which use libc6.
DBMLIB = -ldb
USE_DB = yes
-LIBS = -lnsl -lcrypt
+LIBS = -lnsl -lcrypt -lm
LIBRESOLV = -lresolv
X11=/usr/X11R6
-# $Cambridge: exim/src/OS/Makefile-NetBSD,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-NetBSD,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for NetBSD (ELF object format)
HAVE_SA_LEN=YES
HAVE_IPV6=YES
-LIBS=-lcrypt
+LIBS=-lcrypt -lm
X11=/usr/X11R6
XINCLUDE=-I$(X11)/include
-# $Cambridge: exim/src/OS/Makefile-NetBSD-a.out,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-NetBSD-a.out,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for NetBSD (a.out/COFF object format)
HAVE_SA_LEN=YES
HAVE_IPV6=YES
-LIBS=-lcrypt
+LIBS=-lcrypt -lm
X11=/usr/X11R6
XINCLUDE=-I$(X11)/include
-# $Cambridge: exim/src/OS/Makefile-OSF1,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-OSF1,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for OSF1
CFLAGS=-O
-LIBS=-liconv
+LIBS=-liconv -lm
HAVE_CRYPT16=yes
HAVE_ICONV=yes
HOSTNAME_COMMAND=/usr/bin/hostname
-# $Cambridge: exim/src/OS/Makefile-OpenUNIX,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-OpenUNIX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for OpenUNIX
CFLAGS=-O -I/usr/local/include
LFLAGS=-L/usr/local/lib
-LIBS=-lsocket -lnsl -lelf -lgen -lresolv
+LIBS=-lsocket -lnsl -lelf -lgen -lresolv -lm
EXTRALIBS_EXIMON=-lICE -lSM
RANLIB=@true
-# $Cambridge: exim/src/OS/Makefile-QNX,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-QNX,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific makefile for QNX
RANLIB=@true
DBMLIB=-ldb
USE_DB=yes
-LIBS=-lsocket
+LIBS=-lsocket -lm
X11=/usr/X11R6
XINCLUDE=-I$(X11)/include
-# $Cambridge: exim/src/OS/Makefile-SCO,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-SCO,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for SCO
RANLIB=@true
DBMLIB=-lndbm
ERRNO_QUOTA=0
-LIBS=-lsocket
+LIBS=-lsocket -lm
HAVE_ICONV=yes
X11=/usr/lib/X11
-# $Cambridge: exim/src/OS/Makefile-SCO_SV,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-SCO_SV,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for SCO_SV release 5 (tested on 5.0.5 & 5.0.5)
# (see the UNIX_SV files for SCO 4.2)
CFLAGS=-melf -O3 -m486
LFLAGS=-L/lib -L/usr/lib -L/usr/local/lib
-LIBS=-ltinfo -lm -lsocket
+LIBS=-ltinfo -lsocket -lm
HAVE_ICONV=yes
-# $Cambridge: exim/src/OS/Makefile-SunOS5,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-SunOS5,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for SunOS5
HOSTNAME_COMMAND=look_for_it
RANLIB=@true
-LIBS=-lsocket -lnsl -lkstat
+LIBS=-lsocket -lnsl -lkstat -lm
LIBRESOLV=-lresolv
EXIWHAT_MULTIKILL_CMD=pkill
-# $Cambridge: exim/src/OS/Makefile-SunOS5-hal,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-SunOS5-hal,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for SunOS5 on a HAL
LIBIDENTCFLAGS="-KV7 -O -DHAVE_ANSIHEADERS"
LIBIDENTNAME=sunos5
RANLIB=@true
-LIBS=-lsocket -lnsl -lkstat
+LIBS=-lsocket -lnsl -lkstat -lm
LIBRESOLV=-lresolv
X11=/usr/X11R6
XINCLUDE=-I$(X11)/include
-# $Cambridge: exim/src/OS/Makefile-UNIX_SV,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-UNIX_SV,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for SCO SVR4.2MP (and maybe Unixware)
#
RANLIB=@true
DBMLIB=-lgdbm -L/usr/local/lib
ERRNO_QUOTA=0
-LIBS=-lsocket -lelf -lgen -lnsl -lresolv
+LIBS=-lsocket -lelf -lgen -lnsl -lresolv -lm
X11=/usr/lib/X11
XINCLUDE=-I/usr/include/X11
-# $Cambridge: exim/src/OS/Makefile-USG,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-USG,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for Unixware 2.x
#
DBMLIB=-ldb -L/usr/local/lib
USE_DB=YES
ERRNO_QUOTA=0
-LIBS=-lsocket -lelf -lgen -lnsl -lresolv
+LIBS=-lsocket -lelf -lgen -lnsl -lresolv -lm
X11=/usr/lib/X11
XINCLUDE=-I/usr/include/X11
-# $Cambridge: exim/src/OS/Makefile-Unixware7,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-Unixware7,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for Unixware7
# Based on information from James FitzGibbon <james@ehlo.com>
HAVE_ICONV=yes
-LIBS=-lsocket -lnsl -lelf -lgen -lresolv
+LIBS=-lsocket -lnsl -lelf -lgen -lresolv -lm
# Removed on the advice of Larry Rosenman
# EXTRALIBS=-lwrap
-# $Cambridge: exim/src/OS/Makefile-mips,v 1.1 2004/10/06 15:07:39 ph10 Exp $
+# $Cambridge: exim/src/OS/Makefile-mips,v 1.2 2005/05/23 16:58:55 fanf2 Exp $
# Exim: OS-specific make file for RiscOS4bsd
EXIT_FAILURE=1
EXIT_SUCCESS=0
LIBRESOLV=-lresolv
-LIBS=-liberty
+LIBS=-liberty -lm
XINCLUDE=-I/usr/X11R6/include
CFLAGS=-O
-/* $Cambridge: exim/src/src/acl.c,v 1.33 2005/05/23 15:28:38 fanf2 Exp $ */
+/* $Cambridge: exim/src/src/acl.c,v 1.34 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
#ifdef WITH_CONTENT_SCAN
ACLC_MIME_REGEX,
#endif
+ ACLC_RATELIMIT,
ACLC_RECIPIENTS,
#ifdef WITH_CONTENT_SCAN
ACLC_REGEX,
#ifdef WITH_CONTENT_SCAN
US"mime_regex",
#endif
+ US"ratelimit",
US"recipients",
#ifdef WITH_CONTENT_SCAN
US"regex",
#ifdef WITH_CONTENT_SCAN
TRUE, /* mime_regex */
#endif
+ TRUE, /* ratelimit */
FALSE, /* recipients */
#ifdef WITH_CONTENT_SCAN
TRUE, /* regex */
#ifdef WITH_CONTENT_SCAN
FALSE, /* mime_regex */
#endif
+ FALSE, /* ratelimit */
FALSE, /* recipients */
#ifdef WITH_CONTENT_SCAN
FALSE, /* regex */
~(1<<ACL_WHERE_MIME), /* mime_regex */
#endif
+ 0, /* ratelimit */
+
(unsigned int)
~(1<<ACL_WHERE_RCPT), /* recipients */
+/*************************************************
+* Handle rate limiting *
+*************************************************/
+
+/* Called by acl_check_condition() below to calculate the result
+of the ACL ratelimit condition.
+
+Note that the return value might be slightly unexpected: if the
+sender's rate is above the limit then the result is OK. This is
+similar to the dnslists condition, and is so that you can write
+ACL clauses like: defer ratelimit = 15 / 1h
+
+Arguments:
+ arg the option string for ratelimit=
+ log_msgptr for error messages
+
+Returns: OK - Sender's rate is above limit
+ FAIL - Sender's rate is below limit
+ DEFER - Problem opening ratelimit database
+ ERROR - Syntax error in options.
+*/
+
+static int
+acl_ratelimit(uschar *arg, uschar **log_msgptr)
+{
+double limit, period;
+uschar *ss, *key = arg;
+int sep = '/';
+BOOL have_key = FALSE, leaky = FALSE, strict = FALSE;
+BOOL per_byte = FALSE, per_cmd = FALSE, per_conn = FALSE, per_mail = FALSE;
+int old_pool, rc;
+tree_node **anchor, *t;
+open_db dbblock, *dbm;
+dbdata_ratelimit *dbd;
+struct timeval tv;
+
+/* Parse the first two options and record their values in expansion
+variables. These variables allow the configuration to have informative
+error messages based on rate limits obtained from a table lookup. */
+
+/* First is the maximum number of messages per period and maximum burst
+size, which must be greater than or equal to zero. Zero is useful for
+rate measurement as opposed to rate limiting. */
+
+sender_rate_limit = string_nextinlist(&arg, &sep, NULL, 0);
+if (sender_rate_limit == NULL)
+ limit = -1.0;
+else
+ {
+ limit = Ustrtod(sender_rate_limit, &ss);
+ if (tolower(*ss) == 'k') { limit *= 1024.0; ss++; }
+ else if (tolower(*ss) == 'm') { limit *= 1024.0*1024.0; ss++; }
+ else if (tolower(*ss) == 'g') { limit *= 1024.0*1024.0*1024.0; ss++; }
+ }
+if (limit < 0.0 || *ss != 0)
+ {
+ *log_msgptr = string_sprintf("syntax error in argument for "
+ "\"ratelimit\" condition: \"%s\" is not a positive number",
+ sender_rate_limit);
+ return ERROR;
+ }
+
+/* Second is the rate measurement period and exponential smoothing time
+constant. This must be strictly greater than zero, because zero leads to
+run-time division errors. */
+
+sender_rate_period = string_nextinlist(&arg, &sep, NULL, 0);
+if (sender_rate_period == NULL) period = -1.0;
+else period = readconf_readtime(sender_rate_period, 0, FALSE);
+if (period <= 0.0)
+ {
+ *log_msgptr = string_sprintf("syntax error in argument for "
+ "\"ratelimit\" condition: \"%s\" is not a time value",
+ sender_rate_period);
+ return ERROR;
+ }
+
+/* Parse the other options. Should we check if the per_* options are being
+used in ACLs where they don't make sense, e.g. per_mail in the connect ACL? */
+
+while ((ss = string_nextinlist(&arg, &sep, big_buffer, big_buffer_size))
+ != NULL)
+ {
+ if (strcmpic(ss, US"leaky") == 0) leaky = TRUE;
+ else if (strcmpic(ss, US"strict") == 0) strict = TRUE;
+ else if (strcmpic(ss, US"per_byte") == 0) per_byte = TRUE;
+ else if (strcmpic(ss, US"per_cmd") == 0) per_cmd = TRUE;
+ else if (strcmpic(ss, US"per_conn") == 0) per_conn = TRUE;
+ else if (strcmpic(ss, US"per_mail") == 0) per_mail = TRUE;
+ else if (strcmpic(ss, US"per_rcpt") == 0) per_cmd = TRUE; /* alias */
+ else have_key = TRUE;
+ }
+if (leaky + strict > 1 || per_byte + per_cmd + per_conn + per_mail > 1)
+ {
+ *log_msgptr = US"conflicting options for \"ratelimit\" condition";
+ return ERROR;
+ }
+
+/* Default option values */
+if (!strict) leaky = TRUE;
+if (!per_byte && !per_cmd && !per_conn) per_mail = TRUE;
+
+/* We use the whole of the argument list as the lookup key, because it doesn't
+make sense to use the same stored data if any of the arguments are different.
+If there is no explicit key, use the sender_host_address. If there is no
+sender_host_address (e.g. -bs or acl_not_smtp) then we simply omit it. */
+
+if (!have_key && sender_host_address != NULL)
+ key = string_sprintf("%s / %s", key, sender_host_address);
+
+HDEBUG(D_acl) debug_printf("ratelimit condition limit=%.0f period=%.0f key=%s\n",
+ limit, period, key);
+
+/* If we are dealing with rate limits per connection, per message, or per byte,
+see if we have already computed the rate by looking in the relevant tree. For
+per-connection rate limiting, store tree nodes and dbdata in the permanent pool
+so that they survive across resets. */
+
+anchor = NULL;
+old_pool = store_pool;
+
+if (per_conn)
+ {
+ anchor = &ratelimiters_conn;
+ store_pool = POOL_PERM;
+ }
+if (per_mail || per_byte)
+ anchor = &ratelimiters_mail;
+
+if (anchor != NULL && (t = tree_search(*anchor, key)) != NULL)
+ {
+ dbd = t->data.ptr;
+ /* The following few lines duplicate some of the code below. */
+ if (dbd->rate > limit) rc = OK;
+ else rc = FAIL;
+ store_pool = old_pool;
+ sender_rate = string_sprintf("%.1f", dbd->rate);
+ HDEBUG(D_acl)
+ debug_printf("ratelimit found pre-computed rate %s\n", sender_rate);
+ return rc;
+ }
+
+/* We aren't using a pre-computed rate, so get a previously recorded
+rate from the database, update it, and write it back. If there's no
+previous rate for this key, create one. */
+
+dbm = dbfn_open(US"ratelimit", O_RDWR, &dbblock, TRUE);
+if (dbm == NULL)
+ {
+ store_pool = old_pool;
+ sender_rate = NULL;
+ HDEBUG(D_acl) debug_printf("ratelimit database not available\n");
+ *log_msgptr = US"ratelimit database not available";
+ return DEFER;
+ }
+dbd = dbfn_read(dbm, key);
+
+gettimeofday(&tv, NULL);
+
+if (dbd == NULL)
+ {
+ HDEBUG(D_acl) debug_printf("ratelimit initializing new key's data\n");
+ dbd = store_get(sizeof(dbdata_ratelimit));
+ dbd->time_stamp = tv.tv_sec;
+ dbd->time_usec = tv.tv_usec;
+ dbd->rate = 0.0;
+ }
+else
+ {
+ /* The smoothed rate is computed using an exponentially weighted moving
+ average adjusted for variable sampling intervals. The standard EWMA for
+ a fixed sampling interval is: f'(t) = (1 - a) * f(t) + a * f'(t - 1)
+ where f() is the measured value and f'() is the smoothed value.
+
+ Old data decays out of the smoothed value exponentially, such that data n
+ samples old is multiplied by a^n. The exponential decay time constant p
+ is defined such that data p samples old is multiplied by 1/e, which means
+ that a = exp(-1/p). We can maintain the same time constant for a variable
+ sampling interval i by using a = exp(-i/p).
+
+ The rate we are measuring is messages per period, suitable for directly
+ comparing with the limit. The average rate between now and the previous
+ message is period / interval, which we feed into the EWMA as the sample.
+
+ It turns out that the number of messages required for the smoothed rate
+ to reach the limit when they are sent in a burst is equal to the limit.
+ This can be seen by analysing the value of the smoothed rate after N
+ messages sent at even intervals. Let k = (1 - a) * p/i
+
+ rate_1 = (1 - a) * p/i + a * rate_0
+ = k + a * rate_0
+ rate_2 = k + a * rate_1
+ = k + a * k + a^2 * rate_0
+ rate_3 = k + a * k + a^2 * k + a^3 * rate_0
+ rate_N = rate_0 * a^N + k * SUM(x=0..N-1)(a^x)
+ = rate_0 * a^N + k * (1 - a^N) / (1 - a)
+ = rate_0 * a^N + p/i * (1 - a^N)
+
+ When N is large, a^N -> 0 so rate_N -> p/i as desired.
+
+ rate_N = p/i + (rate_0 - p/i) * a^N
+ a^N = (rate_N - p/i) / (rate_0 - p/i)
+ N * -i/p = log((rate_N - p/i) / (rate_0 - p/i))
+ N = p/i * log((rate_0 - p/i) / (rate_N - p/i))
+
+ Numerical analysis of the above equation, setting the computed rate to
+ increase from rate_0 = 0 to rate_N = limit, shows that for large sending
+ rates, p/i, the number of messages N = limit. So limit serves as both the
+ maximum rate measured in messages per period, and the maximum number of
+ messages that can be sent in a fast burst. */
+
+ double this_time = (double)tv.tv_sec
+ + (double)tv.tv_usec / 1000000.0;
+ double prev_time = (double)dbd->time_stamp
+ + (double)dbd->time_usec / 1000000.0;
+ double interval = this_time - prev_time;
+
+ double i_over_p = interval / period;
+ double a = exp(-i_over_p);
+
+ /* We must avoid division by zero, and deal gracefully with the clock going
+ backwards. If we blunder ahead when time is in reverse then the computed
+ rate will become bogusly huge. Clamp i/p to a very small number instead. */
+
+ if (i_over_p <= 0.0) i_over_p = 1e-9;
+
+ dbd->time_stamp = tv.tv_sec;
+ dbd->time_usec = tv.tv_usec;
+
+ /* If we are measuring the rate in bytes per period, multiply the
+ measured rate by the message size. If we don't know the message size
+ then it's safe to just use a value of zero and let the recorded rate
+ decay as if nothing happened. */
+
+ if (per_byte)
+ dbd->rate = (message_size < 0 ? 0.0 : (double)message_size)
+ * (1 - a) / i_over_p + a * dbd->rate;
+ else
+ dbd->rate = (1 - a) / i_over_p + a * dbd->rate;
+ }
+
+if (dbd->rate > limit) rc = OK;
+ else rc = FAIL;
+
+/* Update the state if the rate is low or if we are being strict. If we
+are in leaky mode and the sender's rate is too high, we do not update
+the recorded rate in order to avoid an over-aggressive sender's retry
+rate preventing them from getting any email through. */
+
+if (rc == FAIL || !leaky)
+ dbfn_write(dbm, key, dbd, sizeof(dbdata_ratelimit));
+dbfn_close(dbm);
+
+/* Store the result in the tree for future reference, if necessary. */
+
+if (anchor != NULL)
+ {
+ t = store_get(sizeof(tree_node) + Ustrlen(key));
+ t->data.ptr = dbd;
+ Ustrcpy(t->name, key);
+ (void)tree_insertnode(anchor, t);
+ }
+
+/* We create the formatted version of the sender's rate very late in
+order to ensure that it is done using the correct storage pool. */
+
+store_pool = old_pool;
+sender_rate = string_sprintf("%.1f", dbd->rate);
+
+HDEBUG(D_acl)
+ debug_printf("ratelimit computed rate %s\n", sender_rate);
+
+return rc;
+}
+
+
+
/*************************************************
* Handle conditions/modifiers on an ACL item *
*************************************************/
break;
#endif
+ case ACLC_RATELIMIT:
+ rc = acl_ratelimit(arg, log_msgptr);
+ break;
+
case ACLC_RECIPIENTS:
rc = match_address_list(addr->address, TRUE, TRUE, &arg, NULL, -1, 0,
&recipient_data);
-/* $Cambridge: exim/src/src/dbstuff.h,v 1.2 2005/01/04 10:00:42 ph10 Exp $ */
+/* $Cambridge: exim/src/src/dbstuff.h,v 1.3 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
/* Basic DB type */
typedef struct {
- GDBM_FILE gdbm; /* Database */
- datum lkey; /* Last key, for scans */
+ GDBM_FILE gdbm; /* Database */
+ datum lkey; /* Last key, for scans */
} EXIM_DB;
/* Cursor type, not used with gdbm: just set up a dummy */
} dbdata_serialize;
+/* This structure records the information required for the ratelimit
+ACL condition. */
+
+typedef struct {
+ time_t time_stamp;
+ /*************/
+ int time_usec; /* Fractional part of time, from gettimeofday() */
+ double rate; /* Smoothed sending rate at that time */
+} dbdata_ratelimit;
+
+
/* End of dbstuff.h */
-/* $Cambridge: exim/src/src/exim.h,v 1.13 2005/05/10 22:39:20 tom Exp $ */
+/* $Cambridge: exim/src/src/exim.h,v 1.14 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
#include <ctype.h>
#include <locale.h>
+#include <math.h>
#include <signal.h>
#include <stdarg.h>
#include <stddef.h>
-/* $Cambridge: exim/src/src/exim_dbutil.c,v 1.3 2005/01/04 10:00:42 ph10 Exp $ */
+/* $Cambridge: exim/src/src/exim_dbutil.c,v 1.4 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
/* Identifiers for the different database types. */
-#define type_retry 1
-#define type_wait 2
-#define type_misc 3
-#define type_callout 4
+#define type_retry 1
+#define type_wait 2
+#define type_misc 3
+#define type_callout 4
+#define type_ratelimit 5
usage(uschar *name, uschar *options)
{
printf("Usage: exim_%s%s <spool-directory> <database-name>\n", name, options);
-printf(" <database-name> = retry | misc | wait-<transport-name> | callout\n");
+printf(" <database-name> = retry | misc | wait-<transport-name> | callout | ratelimit\n");
exit(1);
}
if (Ustrcmp(argv[2], "misc") == 0) return type_misc;
if (Ustrncmp(argv[2], "wait-", 5) == 0) return type_wait;
if (Ustrcmp(argv[2], "callout") == 0) return type_callout;
+ if (Ustrcmp(argv[2], "ratelimit") == 0) return type_ratelimit;
}
usage(name, options);
return -1; /* Never obeyed */
dbdata_retry *retry;
dbdata_wait *wait;
dbdata_callout_cache *callout;
+ dbdata_ratelimit *ratelimit;
int count_bad = 0;
int i, length;
uschar *t;
print_cache(callout->random_result));
}
+ break;
+
+ case type_ratelimit:
+ ratelimit = (dbdata_ratelimit *)value;
+
+ printf("%s.%06d rate: %10.3f key: %s\n",
+ print_time(ratelimit->time_stamp), ratelimit->time_usec,
+ ratelimit->rate, keybuffer);
+
break;
}
store_reset(value);
dbdata_retry *retry;
dbdata_wait *wait;
dbdata_callout_cache *callout;
+ dbdata_ratelimit *ratelimit;
int i, oldlength;
uschar *t;
uschar field[256], value[256];
break;
}
break;
+
+ case type_ratelimit:
+ ratelimit = (dbdata_ratelimit *)value;
+ switch(fieldno)
+ {
+ case 0:
+ if ((tt = read_time(value)) > 0) ratelimit->time_stamp = tt;
+ else printf("bad time value\n");
+ break;
+
+ case 1:
+ ratelimit->time_usec = Uatoi(value);
+
+ case 2:
+ ratelimit->rate = Ustrtod(value, NULL);
+ break;
+
+ default:
+ printf("unknown field number\n");
+ verify = 0;
+ break;
+ }
+ break;
}
dbfn_write(dbm, name, record, length);
callout->random_result);
}
break;
+
+ case type_ratelimit:
+ ratelimit = (dbdata_ratelimit *)value;
+ printf("0 time stamp: %s\n", print_time(ratelimit->time_stamp));
+ printf("1 fract. time: .%06d\n", ratelimit->time_usec);
+ printf("2 sender rate: % .3f\n", ratelimit->rate);
+ break;
}
}
-/* $Cambridge: exim/src/src/expand.c,v 1.21 2005/05/10 10:19:11 ph10 Exp $ */
+/* $Cambridge: exim/src/src/expand.c,v 1.22 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
{ "sender_host_name", vtype_host_lookup, NULL },
{ "sender_host_port", vtype_int, &sender_host_port },
{ "sender_ident", vtype_stringptr, &sender_ident },
+ { "sender_rate", vtype_stringptr, &sender_rate },
+ { "sender_rate_limit", vtype_stringptr, &sender_rate_limit },
+ { "sender_rate_period", vtype_stringptr, &sender_rate_period },
{ "sender_rcvhost", vtype_stringptr, &sender_rcvhost },
{ "sender_verify_failure",vtype_stringptr, &sender_verify_failure },
{ "smtp_active_hostname", vtype_stringptr, &smtp_active_hostname },
-/* $Cambridge: exim/src/src/globals.c,v 1.25 2005/05/23 15:28:38 fanf2 Exp $ */
+/* $Cambridge: exim/src/src/globals.c,v 1.26 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
uschar *queue_smtp_domains = NULL;
unsigned int random_seed = 0;
+tree_node *ratelimiters_conn = NULL;
+tree_node *ratelimiters_mail = NULL;
uschar *raw_active_hostname = NULL;
uschar *raw_sender = NULL;
uschar **raw_recipients = NULL;
BOOL sender_host_unknown = FALSE;
uschar *sender_ident = NULL;
BOOL sender_local = FALSE;
+uschar *sender_rate = NULL;
+uschar *sender_rate_limit = NULL;
+uschar *sender_rate_period = NULL;
uschar *sender_rcvhost = NULL;
BOOL sender_set_untrusted = FALSE;
uschar *sender_unqualified_hosts = NULL;
-/* $Cambridge: exim/src/src/globals.h,v 1.17 2005/05/23 15:28:38 fanf2 Exp $ */
+/* $Cambridge: exim/src/src/globals.h,v 1.18 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
extern uschar *queue_smtp_domains; /* Ditto, for these domains */
extern unsigned int random_seed; /* Seed for random numbers */
+extern tree_node *ratelimiters_conn; /* Results of connection ratelimit checks */
+extern tree_node *ratelimiters_mail; /* Results of per-mail ratelimit checks */
extern uschar *raw_active_hostname; /* Pre-expansion */
extern uschar *raw_sender; /* Before rewriting */
extern uschar **raw_recipients; /* Before rewriting */
extern BOOL sender_host_unknown; /* TRUE for -bs and -bS except inetd */
extern uschar *sender_ident; /* Sender identity via RFC 1413 */
extern BOOL sender_local; /* TRUE for local senders */
+extern uschar *sender_rate; /* Sender rate computed by ACL */
+extern uschar *sender_rate_limit; /* Configured rate limit */
+extern uschar *sender_rate_period; /* Configured smoothing period */
extern uschar *sender_rcvhost; /* Host data for Received: */
extern BOOL sender_set_untrusted; /* Sender set by untrusted caller */
extern uschar *sender_unqualified_hosts; /* Permitted unqualified senders */
-/* $Cambridge: exim/src/src/smtp_in.c,v 1.18 2005/05/23 15:28:38 fanf2 Exp $ */
+/* $Cambridge: exim/src/src/smtp_in.c,v 1.19 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
#endif
body_linecount = body_zerocount = 0;
+sender_rate = sender_rate_limit = sender_rate_period = NULL;
+ratelimiters_mail = NULL; /* Updated by ratelimit ACL condition */
+ /* Note that ratelimiters_conn persists across resets. */
+
for (i = 0; i < ACL_M_MAX; i++) acl_var[ACL_C_MAX + i] = NULL;
/* The message body variables use malloc store. They may be set if this is
-/* $Cambridge: exim/src/src/string.c,v 1.2 2005/01/04 10:00:42 ph10 Exp $ */
+/* $Cambridge: exim/src/src/string.c,v 1.3 2005/05/23 16:58:56 fanf2 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
break;
/* %f format is inherently insecure if the numbers that it may be
- handed are unknown (e.g. 1e300). However, in Exim, the only use of %f
- is for printing load averages, and these are actually stored as integers
- (load average * 1000) so the size of the numbers is constrained. */
+ handed are unknown (e.g. 1e300). However, in Exim, %f is used for
+ printing load averages, and these are actually stored as integers
+ (load average * 1000) so the size of the numbers is constrained.
+ It is also used for formatting sending rates, where the simplicity
+ of the format prevents overflow. */
case 'f':
case 'e':
git://git.exim.org / users / jgh / exim.git / commitdiff