Clarification about the installation process

diff --git a/Installation.md b/Installation.md
index cfedf6ddd8a724b83cc13f5e6d6e289333328520..5abe32ea5edfab1b2811065435744c87dd68630d 100644 (file)
--- a/Installation.md
+++ b/Installation.md
@@ -7,9 +7,12 @@ If you want to run more than one instance of the build farm to test multiple dif
 
 ## Install Steps
 1. Make sure that all required packages for Exim to build successfully are installed. The minimal package requirements are the basic suite of compiler tools and basic libraries.  You will also need to install the ccache package.
 
 ## Install Steps
 1. Make sure that all required packages for Exim to build successfully are installed. The minimal package requirements are the basic suite of compiler tools and basic libraries.  You will also need to install the ccache package.

-2. Create the master exim user (i.e. the user that Exim changes uid to when it runs in normal operation).  I suggest the user **exim**.  If you already have the Exim package provided by your distro installed, chances are good that you already have a user named **exim** or **exim4**.  That user is fine.  This howto also assumes you use the groupname **exim**.
+
+2. Create the master exim user (i.e. the user that Exim changes uid to when it runs in normal operation).  I suggest the user **exim**.  If you already have the Exim package provided by your distro installed, chances are good that you already have a user named **exim**, **exim4**, or **Debian-exim**.  That user is fine.  This howto also assumes you use the groupname **exim**.
+

 3. Create a user that will run the Build Farm.  I suggest the user **farm**.  Due to some assumptions that are made in the test suite, the group name must also be **farm**. It is ok if you make it a secondary group or the primary group.  You also must assign a geckos name, or some tests will fail.  I suggest "Exim BuildFarm".  An example command that should work for most distributions:
 `useradd -m -c 'Exim BuildFarm' farm`
 3. Create a user that will run the Build Farm.  I suggest the user **farm**.  Due to some assumptions that are made in the test suite, the group name must also be **farm**. It is ok if you make it a secondary group or the primary group.  You also must assign a geckos name, or some tests will fail.  I suggest "Exim BuildFarm".  An example command that should work for most distributions:
 `useradd -m -c 'Exim BuildFarm' farm`

+

 4. Add **exim** as a secondary group to the **farm** user.  (You should **NOT** add **farm** as a secondary group to the exim user; this will cause test errors.)  Here is a sample user/group configuration:
 
     ````
 4. Add **exim** as a secondary group to the **farm** user.  (You should **NOT** add **farm** as a secondary group to the exim user; this will cause test errors.)  Here is a sample user/group configuration:
 
     ````


@@ -18,27 +21,46 @@ If you want to run more than one instance of the build farm to test multiple dif
     farm@buildclient:~/code$ id exim
     uid=480(exim) gid=137(exim) groups=137(exim)
     ````
     farm@buildclient:~/code$ id exim
     uid=480(exim) gid=137(exim) groups=137(exim)
     ````

+

 5. Configure sudo to allow user **farm** to elevate to root without a password.  Run `visudo` and add:
 5. Configure sudo to allow user **farm** to elevate to root without a password.  Run `visudo` and add:

-`farm    ALL=(ALL)       NOPASSWD: ALL`
+`farm    ALL=(ALL)       NOPASSWD: ALL`. You should check this with
+`sudo -U farm -l`.
+

 6. Change to the **farm** user.
 6. Change to the **farm** user.

+

 7. Checkout the client code:
 `git clone git://git.exim.org/buildfarm-client.git code`
 This will create the repo checkout in the directory *~/code/*.
 7. Checkout the client code:
 `git clone git://git.exim.org/buildfarm-client.git code`
 This will create the repo checkout in the directory *~/code/*.

+

 8. Change into the *~/code/* directory and copy the config template to the default filename:
 `cp build-farm.conf.template build-farm.conf`
 8. Change into the *~/code/* directory and copy the config template to the default filename:
 `cp build-farm.conf.template build-farm.conf`

+

 9. Edit the *build-farm.conf* file and adjust to your local configuration.  The only options that absolutely require changing are the _animal_ and _secret_ configuration settings.  Your machine's sysname alias must be assigned to *animal* and the secret key assigned to *secret*.  The rest of the default build configuration should work to build a basic Exim configuration with no added features/capabilities and it will run the test suite.
 9. Edit the *build-farm.conf* file and adjust to your local configuration.  The only options that absolutely require changing are the _animal_ and _secret_ configuration settings.  Your machine's sysname alias must be assigned to *animal* and the secret key assigned to *secret*.  The rest of the default build configuration should work to build a basic Exim configuration with no added features/capabilities and it will run the test suite.

+

 10. Whatever directory is set as the *build_root* must exist before anything will run.  If you didn't change it, this will be all that's needed:
 `mkdir $HOME/buildfarm`
 **CAUTION:** Do not ever change anything in the git repo under that is created in *$HOME/buildfarm/HEAD/exim/*. It will cause the build to fail in the Git-Checkout phase.
 10. Whatever directory is set as the *build_root* must exist before anything will run.  If you didn't change it, this will be all that's needed:
 `mkdir $HOME/buildfarm`
 **CAUTION:** Do not ever change anything in the git repo under that is created in *$HOME/buildfarm/HEAD/exim/*. It will cause the build to fail in the Git-Checkout phase.

+

 11. Directory permissions must be lax enough for the **exim** user running the test suite to be able to access the files that user farm has checked out.  One recommendation that we know works is to make the **farm** user's home directory be both group and world readable and group and world searchable:
 11. Directory permissions must be lax enough for the **exim** user running the test suite to be able to access the files that user farm has checked out.  One recommendation that we know works is to make the **farm** user's home directory be both group and world readable and group and world searchable:

-`chmod og+rx $HOME`.  Additionally, every subdirectory between */* and *$HOME/buildfarm* should have this permissive access.  Some users have reported they must set *$HOME/buildfarm* to 0775 for the test suite to run.  What do these open permissions fix?  One thing that tighter permissions will break, for example, is test 0005, because the sudo'd exim process in the test suite will be unable to read that test config file.
+`chmod og+rx $HOME`.  Additionally, every subdirectory between */* and *$HOME/buildfarm* should have this permissive access.  **Some** users have reported they must set *$HOME/buildfarm* to 0775 for the test suite to run.  What do these open permissions fix?  One thing that tighter permissions will break, for example, is test 0005, because the sudo'd exim process in the test suite will be unable to read that test config file.
+

 12. Make sure that your umask is 0022: `umask 0022`.  This also will need to be set in any script you call the run_build script.
 12. Make sure that your umask is 0022: `umask 0022`.  This also will need to be set in any script you call the run_build script.

+

 13. Test the configure process by running `./run_build --test --verbose=2 --only-steps=configure`. If there are errors, you'll need to correct them until the process succeeds (ends with OK).  You can repeat this as many times as necessary because test mode does not store the status of the git repo or the status of each stage of the build.
 13. Test the configure process by running `./run_build --test --verbose=2 --only-steps=configure`. If there are errors, you'll need to correct them until the process succeeds (ends with OK).  You can repeat this as many times as necessary because test mode does not store the status of the git repo or the status of each stage of the build.

+
+    The test might complain about missing Perl modules. Install these
+    modules.
+

 14. Test the build process by running `./run_build --test --verbose=2 --only-steps=configure,make`. If there are build errors, make adjustments, install additional packages, etc, and repeat the test.
 14. Test the build process by running `./run_build --test --verbose=2 --only-steps=configure,make`. If there are build errors, make adjustments, install additional packages, etc, and repeat the test.

-15. Test the test suite by running `./run_build --test --verbose=2 --only-steps=configure,make,test --override range_num_tests='1 2'`.  There will be one error on a standard system: the Exim test suite uses ifconfig to obtain the IP address, and this binary is frequently not in a regular user's path.  Temporarily fix this by running `export PATH="/usr/sbin:/sbin:$PATH"` to prepend the probably directories where that command is located to your regular user path. (Instructions on how to do this automatically in the cron job are below).  If there are build errors when building the test suite, or runtime errors trying to run the test suite, you may need to install additional packages (you shouldn't though).
+
+15. Test the test suite by running `./run_build --test --verbose=2 --only-steps=configure,make,test --override range_num_tests='1 2'`.  If there are build errors when building the test suite, or runtime errors trying to run the test suite, you may need to install additional packages, e.g. autoconf.
+

 16. If you enabled the documentation building process in the *build-farm.conf*, then you can try to build it with `./run_build --test --verbose=2 --only-steps=configure,make-doc`.  For documentation generation to succeed, it will require extra packages to be installed to support xml, xslt, pdf, and a few other things. Please see the comments on the [Building Documentation](./BuildingDocs.html) page.
 16. If you enabled the documentation building process in the *build-farm.conf*, then you can try to build it with `./run_build --test --verbose=2 --only-steps=configure,make-doc`.  For documentation generation to succeed, it will require extra packages to be installed to support xml, xslt, pdf, and a few other things. Please see the comments on the [Building Documentation](./BuildingDocs.html) page.

+

 17. If you can get past each of these steps, then your build farm system meets the minimum requirements.
 17. If you can get past each of these steps, then your build farm system meets the minimum requirements.

+

 18. The official process can be kicked off by running `/home/farm/code/run_cron --run-all`.  This will run the default build configuration, keep track of the git repository status, and upload the build results to the server.
 18. The official process can be kicked off by running `/home/farm/code/run_cron --run-all`.  This will run the default build configuration, keep track of the git repository status, and upload the build results to the server.

+

 19. Once that command runs with no complaints, add it to the **farm** user crontab.  You can run it at whatever frequency you choose, I suggest 2 hours.  If a previous instantiation is still running, the script will detect the lockfile and exit so as not to step on each other. **Caution:** Cron provides a highly sanitized environment to the `run_cron` script. Depending on your Cron version you may supply a `PATH=...` line in your crontab, or you need to call a wrapper script around `run_cron`:
 
     ````
 19. Once that command runs with no complaints, add it to the **farm** user crontab.  You can run it at whatever frequency you choose, I suggest 2 hours.  If a previous instantiation is still running, the script will detect the lockfile and exit so as not to step on each other. **Caution:** Cron provides a highly sanitized environment to the `run_cron` script. Depending on your Cron version you may supply a `PATH=...` line in your crontab, or you need to call a wrapper script around `run_cron`:
 
     ````


@@ -46,6 +68,7 @@ This will create the repo checkout in the directory *~/code/*.
     export PATH="/usr/local/bin:/sbin:/usr/sbin:$PATH"
     $HOME/code/run_cron.sh --run-all $@
     ````
     export PATH="/usr/local/bin:/sbin:/usr/sbin:$PATH"
     $HOME/code/run_cron.sh --run-all $@
     ````

+

 20. The default tests that are run are a limited set, from 1 to 999.  This covers basic Exim functions, but does not exercise a lot of the advanced functions.  Once a few cronjob runs complete successfully, increase the range of tests to run.  Edit the *build-farm.conf* file and change the `range_num_tests => '1 999',` to `range_num_tests => '1 5999',` and it will run more advanced tests.
 
 
 20. The default tests that are run are a limited set, from 1 to 999.  This covers basic Exim functions, but does not exercise a lot of the advanced functions.  Once a few cronjob runs complete successfully, increase the range of tests to run.  Edit the *build-farm.conf* file and change the `range_num_tests => '1 999',` to `range_num_tests => '1 5999',` and it will run more advanced tests.
 
 


@@ -53,11 +76,16 @@ This will create the repo checkout in the directory *~/code/*.
 As mentioned above, you can start at step 9 for each additional build you'll do on the same machine.  A second application must be filled out to put the appropriate data in the database because this is treated a separate BuildFarm client:
 
 1. Make a second copy of the *build-farm.conf* with a different name, such as *build-farm-alias2.conf*.
 As mentioned above, you can start at step 9 for each additional build you'll do on the same machine.  A second application must be filled out to put the appropriate data in the database because this is treated a separate BuildFarm client:
 
 1. Make a second copy of the *build-farm.conf* with a different name, such as *build-farm-alias2.conf*.

+

 2. Set the path for build_root and the CCACHE_DIR environment settings, each to a new subdirectory from the first configuration.
 2. Set the path for build_root and the CCACHE_DIR environment settings, each to a new subdirectory from the first configuration.

+

 3. Create the new build_root directory.
 3. Create the new build_root directory.

+

 4. When testing, pass this additional parameter to any build script you run: `--config /path/to/build-farm-machine2.conf`.
 4. When testing, pass this additional parameter to any build script you run: `--config /path/to/build-farm-machine2.conf`.

+

 5. Add a new cron job for this second build farm client in the same way you chose to do the first one, just make sure to specify the --config option with it.
 
 5. Add a new cron job for this second build farm client in the same way you chose to do the first one, just make sure to specify the --config option with it.
 

+

 ## Further documentation
 * Details of options in [build-farm.conf](./BuildConfigConf.html)
 * Details of [potential testing commandlines](./TestingBuilds.html)
 ## Further documentation
 * Details of options in [build-farm.conf](./BuildConfigConf.html)
 * Details of [potential testing commandlines](./TestingBuilds.html)