X-Git-Url: https://git.exim.org/exim.git/blobdiff_plain/dd2a32ad5058ad7831621b99a1eadc264d4982c9..692b0b1ab0384ebfad82706881b9a547aa334422:/test/README diff --git a/test/README b/test/README index 7bf4ceea5..8df1678b6 100644 --- a/test/README +++ b/test/README @@ -3,8 +3,8 @@ EXPORTABLE EXIM TEST SUITE This document last updated for: -Test Suite Version: 4.67 -Date: 20 February 2007 +Test Suite Version: 4.87 +Date: 30 January 2016 BACKGROUND @@ -73,12 +73,13 @@ In order to run this test suite, the following requirements must be met: Defaults:exim-build timestamp_timeout=480,!tty_tickets -(3) The login under which you run the tests must be in the exim group so that - it has access to logs, spool files, etc. The login should not be one of the - names "userx", "usery", "userz", or a few other simple ones such as "abcd" - and "xyz" and single letters that are used in the tests. The test suite - expects the login to have a gecos name; I think it will now run if the - gecos field is empty but there may be anomalies. +(3) The login under which you run the tests must have the exim group as a + secondary so that it has access to logs, spool files, etc. However, it + should have a different primary group (eg. "users" vs. "eximgroup"). The + login should not be one of the names "userx", "usery", "userz", or a few + other simple ones such as "abcd" and "xyz" and single letters that are used + in the tests. The test suite expects the login to have a gecos name; I think + it will now run if the gecos field is empty but there may be anomalies. The login must not contain a dash or an equal sign. (Otherwise some tests about local_from_{suffix,prefix} will fail.) @@ -124,7 +125,7 @@ In order to run this test suite, the following requirements must be met: DISABLE_D_OPTION must not be used. If ALT_CONFIG_PREFIX is used, it must contain the directory of the test-suite. WHITELIST_D_MACROS should contain: - DIR:EXIM_PATH:AA:ACL:ACLRCPT:ACL_MAIL:ACL_PREDATA:ACL_RCPT:AFFIX:ALLOW:ARG1:ARG2:AUTHF:AUTHS:AUTH_ID_DOMAIN:BAD:BANNER:BB:BR:BRB:CERT:COM:COMMAND_USER:CONNECTCOND:CONTROL:CREQCIP:CREQMAC:CRL:CSS:D6:DATA:DCF:DDF:DEFAULTDWC:DELAY:DETAILS:DRATELIMIT:DYNAMIC_OPTION:ELI:ERROR_DETAILS:ERT:FAKE:FALLBACK:FILTER:FILTER_PREPEND_HOME:FORBID:FORBID_SMTP_CODE:FUSER:HAI:HAP:HARDLIMIT:HEADER_LINE_MAXSIZE:HEADER_MAXSIZE:HELO_MSG:HL:HOSTS:HOSTS_AVOID_TLS:HOSTS_MAX_TRY:HVH:IFACE:IGNORE_QUOTA:INC:INSERT:IP1:IP2:LAST:LDAPSERVERS:LENCHECK:LIMIT:LIST:LOG_SELECTOR:LS:MAXNM:MESSAGE_LOGS:MSIZE:NOTDAEMON:ONCE:ONLY:OPT:OPTION:ORDER:PAH:PEX:PORT:PTBC:QDG:QOLL:QUOTA:QUOTA_FILECOUNT:QWM:RCPT_MSG:REMEMBER:REQUIRE:RETRY:RETRY1:RETRY2:RETURN:RETURN_ERROR_DETAILS:REWRITE:ROUTE_DATA:RRATELIMIT:RT:S:SELECTOR:SELF:SERVER:SERVERS:SREQCIP:SREQMAC:SRV:STD:STRICT:SUB:SUBMISSION_OPTIONS:TIMEOUTDEFER:TIMES:TRUSTED:TRYCLEAR:UL:USE_SENDER:UTF8:VALUE:WMF:X:Y + DIR:EXIM_PATH:AA:ACL:ACLRCPT:ACL_MAIL:ACL_PREDATA:ACL_RCPT:AFFIX:ALLOW:ARG1:ARG2:AUTHF:AUTHS:AUTH_ID_DOMAIN:BAD:BANNER:BB:BR:BRB:CERT:COM:COMMAND_USER:CONNECTCOND:CONTROL:CREQCIP:CREQMAC:CRL:CSS:D6:DATA:DCF:DDF:DEFAULTDWC:DELAY:DETAILS:DRATELIMIT:DYNAMIC_OPTION:ELI:ERROR_DETAILS:ERT:FAKE:FALLBACK:FILTER:FILTER_PREPEND_HOME:FORBID:FORBID_SMTP_CODE:FUSER:HAI:HAP:HARDLIMIT:HEADER_LINE_MAXSIZE:HEADER_MAXSIZE:HELO_MSG:HL:HOSTS:HOSTS_AVOID_TLS:HOSTS_MAX_TRY:HVH:IFACE:IGNORE_QUOTA:INC:INSERT:IP1:IP2:LAST:LDAPSERVERS:LENCHECK:LIMIT:LIST:LOG_SELECTOR:MAXNM:MESSAGE_LOGS:MSIZE:NOTDAEMON:ONCE:ONLY:OPT:OPTION:ORDER:PAH:PEX:PORT:PTBC:QDG:QOLL:QUOTA:QUOTA_FILECOUNT:QWM:RCPT_MSG:REMEMBER:REQUIRE:RETRY:RETRY1:RETRY2:RETURN:RETURN_ERROR_DETAILS:REWRITE:ROUTE_DATA:RRATELIMIT:SELECTOR:SELF:SERVER:SERVERS:SREQCIP:SREQMAC:SRV:STRICT:SUB:SUBMISSION_OPTIONS:TIMEOUTDEFER:TIMES:TRUSTED:TRYCLEAR:UL:USE_SENDER:UTF8:VALUE:WMF (10) Exim must *not* be built with USE_READLINE, as the test-suite's automation assumes the simpler I/O model. @@ -213,10 +214,12 @@ THE RUNTEST SCRIPT ------------------ If you do not supply any arguments to ./runtest, it searches for an Exim -source tree at the same level as the test suite directory. It then looks for an -Exim binary in a "build" directory of that source tree. If there are several -Exim source trees, it chooses the latest version of Exim. Consider the -following example: +source tree at the same level as the test suite directory. A source tree +is a source tree, if it contains a build-* directory. + +It then looks for an Exim binary in a "build" directory of that source +tree. If there are several Exim source trees, it chooses the latest +version of Exim. Consider the following example: $ ls -F /source/exim exim-4.60/ exim-4.62/ exim-testsuite-x.xx/ @@ -260,6 +263,17 @@ There are some options for the ./runtest script itself: (If it turns out that most people prefer to use diff, I'll change the default.) + -FLAVOR + -FLAVOUR + This allows "overrides" for the test results. It's intended + use is to deal with distro specific differences in the test + output. The default flavour is "foo". If during the test + run differences between the current and the expected output + are found and no flavour file exists already, you may update + the "common" expected output or you may create a flavour + file. If a flavour file already exists, any updates will go + into that flavour file! + -KEEP Normally, after a successful run, the test output files are deleted. This option prevents this. It is useful when running a single test, in order to look at the actual output before it is @@ -827,6 +841,12 @@ terminated by four asterisks. Even if no data is required for the particular usage, the asterisks must be given. + background + +This command takes one script line and runs it in the background, +in parallel with following commands. For external daemons, eg. redis-server. + + catwrite [nxm[=start-of-line-text]]* This command operates like the "write" command, which is described below, @@ -908,7 +928,9 @@ input, details of which are given below. A number of options are implemented: -d causes the server to output debugging information -t sets a timeout (default 5) for when the server is - awaiting an incoming connection + awaiting an incoming connection. If negative, the + absolute value is used and a timeout results in a + nonfailure exit code -noipv4 causes the server not to set up an IPv4 socket