X-Git-Url: https://git.exim.org/exim.git/blobdiff_plain/1fb7660fc893265f0e37ceb965396d9e005a0b74..1cf59ee7f68960237ad5cd3d599512fdbe5b3954:/test/README diff --git a/test/README b/test/README index 4f43b0b1a..5d9bed72d 100644 --- a/test/README +++ b/test/README @@ -3,8 +3,8 @@ EXPORTABLE EXIM TEST SUITE This document last updated for: -Test Suite Version: 4.67 -Date: 20 February 2007 +Test Suite Version: 4.87 +Date: 30 January 2016 BACKGROUND @@ -73,12 +73,13 @@ In order to run this test suite, the following requirements must be met: Defaults:exim-build timestamp_timeout=480,!tty_tickets -(3) The login under which you run the tests must be in the exim group so that - it has access to logs, spool files, etc. The login should not be one of the - names "userx", "usery", "userz", or a few other simple ones such as "abcd" - and "xyz" and single letters that are used in the tests. The test suite - expects the login to have a gecos name; I think it will now run if the - gecos field is empty but there may be anomalies. +(3) The login under which you run the tests must have the exim group as a + secondary so that it has access to logs, spool files, etc. However, it + should have a different primary group (eg. "users" vs. "eximgroup"). The + login should not be one of the names "userx", "usery", "userz", or a few + other simple ones such as "abcd" and "xyz" and single letters that are used + in the tests. The test suite expects the login to have a gecos name; I think + it will now run if the gecos field is empty but there may be anomalies. The login must not contain a dash or an equal sign. (Otherwise some tests about local_from_{suffix,prefix} will fail.) @@ -114,7 +115,7 @@ In order to run this test suite, the following requirements must be met: is also an IPv6 address, additional tests are run when the Exim binary contains IPv6 support. There are checks in the scripts for a running IPv4 interface; when one is not found, some tests are skipped (with a warning - message). + message). The local net may not be in 10.0/8 as that is used by the suite. (9) Exim must be built with TRUSTED_CONFIG_LIST support, so that the test configs can be placed into it. A suitable file location is .../exim/test/trusted_configs @@ -213,10 +214,12 @@ THE RUNTEST SCRIPT ------------------ If you do not supply any arguments to ./runtest, it searches for an Exim -source tree at the same level as the test suite directory. It then looks for an -Exim binary in a "build" directory of that source tree. If there are several -Exim source trees, it chooses the latest version of Exim. Consider the -following example: +source tree at the same level as the test suite directory. A source tree +is a source tree, if it contains a build-* directory. + +It then looks for an Exim binary in a "build" directory of that source +tree. If there are several Exim source trees, it chooses the latest +version of Exim. Consider the following example: $ ls -F /source/exim exim-4.60/ exim-4.62/ exim-testsuite-x.xx/ @@ -260,6 +263,17 @@ There are some options for the ./runtest script itself: (If it turns out that most people prefer to use diff, I'll change the default.) + -FLAVOR + -FLAVOUR + This allows "overrides" for the test results. It's intended + use is to deal with distro specific differences in the test + output. The default flavour is "foo". If during the test + run differences between the current and the expected output + are found and no flavour file exists already, you may update + the "common" expected output or you may create a flavour + file. If a flavour file already exists, any updates will go + into that flavour file! + -KEEP Normally, after a successful run, the test output files are deleted. This option prevents this. It is useful when running a single test, in order to look at the actual output before it is @@ -827,6 +841,12 @@ terminated by four asterisks. Even if no data is required for the particular usage, the asterisks must be given. + background + +This command takes one script line and runs it in the background, +in parallel with following commands. For external daemons, eg. redis-server. + + catwrite [nxm[=start-of-line-text]]* This command operates like the "write" command, which is described below, @@ -908,7 +928,9 @@ input, details of which are given below. A number of options are implemented: -d causes the server to output debugging information -t sets a timeout (default 5) for when the server is - awaiting an incoming connection + awaiting an incoming connection. If negative, the + absolute value is used and a timeout results in a + nonfailure exit code -noipv4 causes the server not to set up an IPv4 socket @@ -1144,9 +1166,6 @@ fake zone files. These are: data block. The addresses that are generated are in the 10.250.0.0/16 network. -The domain names that are handled directly by Exim, without being passed to -fakens, are: - test.again.dns This always provokes a TRY_AGAIN response, for testing the handling of temporary DNS error. If the full domain name starts with digits, a delay of that many seconds occurs. @@ -1154,10 +1173,6 @@ fakens, are: test.fail.dns This always provokes a NO_RECOVERY response, for testing DNS server failures. -This special handling could now be done in the fakens program, but while the -old test suite is still being used it has to be done in Exim itself, so for the -moment it remains there. - The use of gethostbyname() and its IPv6 friends is also subverted when Exim is running in the test harness. The test code handles a few special names directly; for all the others it uses DNS lookups, which are then handled as