X-Git-Url: https://git.exim.org/exim.git/blobdiff_plain/f89d2485b9084317af07bf48bf09a81dc5780caa..61147df48889217a1c1023d8c6e2431c24967686:/doc/doc-docbook/HowItWorks.txt?ds=inline diff --git a/doc/doc-docbook/HowItWorks.txt b/doc/doc-docbook/HowItWorks.txt index 4c51ae34d..c5e328a09 100644 --- a/doc/doc-docbook/HowItWorks.txt +++ b/doc/doc-docbook/HowItWorks.txt @@ -1,5 +1,3 @@ -$Cambridge: exim/doc/doc-docbook/HowItWorks.txt,v 1.6 2007/04/11 15:26:09 ph10 Exp $ - CREATING THE EXIM DOCUMENTATION "You are lost in a maze of twisty little scripts." @@ -74,10 +72,10 @@ adopted without affecting the source maintenance. A number of issues arose while setting this all up, which are best summed up by the statement that a lot of the technology was (in 2006) still very immature. Trying to do this conversion any earlier would probably not have been anywhere -near as successful. The main problems that bother me in the XML-generated +near as successful. The main issues that bother me in the XML-generated documentation are described in the penultimate section of this document. -The major problems were originally in producing PostScript and PDF outputs. The +Initially, the major problems were in producing PostScript and PDF outputs. The available free software for doing this was and still is (we are now in 2007) cumbersome and slow, and does not support certain output features that I would like. My response to this was, over a period of two years, to write an XML @@ -85,11 +83,11 @@ processor called SDoP (Simple DocBook Processor). This program reads DocBook XML and writes PostScript, without using any of the heavyweight apparatus that is required for xmlto and fop (the previously used software). -An experimental first version of SDoP will be used for the Exim 4.67 -documentation. A full release of SDoP requires further work. SDoP's output +An experimental first version of SDoP was used for the Exim 4.67 +documentation. Subsequently SDoP was released for general use. SDoP's output includes features that are missing when xmlto/fop is used, and it also runs -about 60 times faster. The main manual can be formatted in 2 seconds instead of -2 minutes, which makes checking and fixing mistakes much easier. +about 60 times faster. The main manual can be formatted in 2.5 seconds instead +of 2.5 minutes, which makes checking and fixing mistakes much easier. The Makefile that is used to build the various forms of output will, for the moment, support both ways of producing PostScript and PDF output, though the @@ -107,13 +105,13 @@ run Gentoo Linux, and a lot of things have been installed as dependencies that I am not fully aware of. This is what I know about (version numbers are current at the time of writing): -. xfpt 0.01 +. xfpt 0.03 This converts the master source file into a DocBook XML file. -. sdop 0.00 +. sdop 0.03 - This is my new, still-very-alpha, DocBook-to-PostScript processor. + This is my new DocBook-to-PostScript processor. . ps2pdf @@ -131,25 +129,28 @@ at the time of writing): things that I have not figured out, to apply the DocBook XSLT stylesheets. . libxml 1.8.17 - libxml2 2.6.22 - libxslt 1.1.15 + libxml2 2.6.28 + libxslt 1.1.20 These are all installed on my box; I do not know which of libxml or libxml2 the various scripts are actually using. -. xsl-stylesheets-1.70.1 +. xsl-stylesheets- These are the standard DocBook XSL stylesheets. -. fop 0.20.5 + The documents use http://docbook.sourceforge.net/release/xsl/current/ which + should be mapped to an appropriate local path via the system catalogs. + +. fop 0.93 FOP is a processor for "formatted objects". It is written in Java. The fop command is a shell script that drives it. It required only if you do not want to use SDoP and ps2pdf to generate PostScript and PDF output. -. w3m 0.5.1 +. w3m 0.5.2 - This is a text-oriented web brower. It is used to produce the Ascii form of + This is a text-oriented web brower. It is used to produce the ASCII form of the Exim documentation (spec.txt) from a specially-created HTML format. It seems to do a better job than lynx. @@ -166,7 +167,7 @@ at the time of writing): . makeinfo 4.8 - This is used to make a set of "info" files from a Texinfo file. + This is used to make an "info" file from a Texinfo file. In addition, there are a number of locally written Perl scripts. These are described below. @@ -218,8 +219,8 @@ DOCBOOK PROCESSING Processing a .xml file into the five different output formats is not entirely straightforward. For a start, the same XML is not suitable for all the different output styles. When the final output is in a text format (.txt, -.texinfo) for instance, all non-Ascii characters in the input must be converted -to Ascii transliterations because the current processing tools do not do this +.texinfo) for instance, all non-ASCII characters in the input must be converted +to ASCII transliterations because the current processing tools do not do this correctly automatically. In order to cope with these issues in a flexible way, a Perl script called @@ -234,6 +235,39 @@ comments in these style files to explain what changes I have made. Some of the changes are quite significant. +XSL INCLUDES + +References to XSL paths should use the public URLs, such as: + http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl +If this fails to work for you, then there is a problem with your system +catalogs. As a work-around, you can adjust the OS-Fixups script and then: +$ make os-fixup + +As an example of how this should normally work, on a FreeBSD system the +resolution goes to /usr/local/share/xml/catalog which contains a directive: + +to pull in the file automatically maintained by the Ports system. That file +will contain: + + +and that catalog file contains: + + +and the full path is thus eventually arrived at. + +See also the tools: + xmlcatalog(1) from libxml2 + xmlcatmgr(1) for a lightweight tool written for the NetBSD Packages system. + + THE PRE-XML SCRIPT The Pre-xml script copies a .xml file, making certain changes according to the @@ -241,7 +275,7 @@ options it is given. The currently available options are as follows: -ascii - This option is used for Ascii output formats. It makes the following + This option is used for ASCII output formats. It makes the following character replacements: ’ => ' apostrophe @@ -252,14 +286,14 @@ options it is given. The currently available options are as follows: – => - en dash The apostrophe is specified numerically because that is what xfpt generates - from an Ascii single quote character. Non-Ascii characters that are not in + from an ASCII single quote character. Non-ASCII characters that are not in this list should not be used without thinking about how they might be - converted for the Ascii formats. + converted for the ASCII formats. In addition to the character replacements, this option causes quotes to be put round text items, and and to be replaced by - Ascii quote marks. You would think the stylesheet would cope with the latter, - but it seems to generate non-Ascii characters that w3m then turns into + ASCII quote marks. You would think the stylesheet would cope with the latter, + but it seems to generate non-ASCII characters that w3m then turns into question marks. -bookinfo @@ -286,17 +320,18 @@ options it is given. The currently available options are as follows: -noindex Remove the XML to generate a Concept Index and an Options index. The source - document has two types of index entry, for a concept and an options index. - However, no index is required for the .txt and .texinfo outputs. + document has three types of index entry, for variables, options, and concept + indexes. However, no index is required for the .txt and .texinfo outputs. -oneindex - Remove the XML to generate a Concept and an Options Index, and add XML to - generate a single index. The only output processors that support multiple - indexes are SDoP and the processor that produces "formatted objects" for - PostScript and PDF output for fop. The HTML processor ignores the XML - settings for multiple indexes and just makes one unified index. Specifying - two indexes gets you two copies of the same index, so this has to be changed. + Remove the XML to generate separate variables, options, and concept indexes, + and add XML to generate a single index. The only output processors that + support multiple indexes are SDoP and the processor that produces "formatted + objects" for PostScript and PDF output for fop. The HTML processor ignores + the XML settings for multiple indexes and just makes one unified index. + Specifying three indexes gets you three copies of the same index, so this has + to be changed. -optbreak @@ -479,7 +514,7 @@ so the logic is somewhat different. CREATING TEXT FILES This happens in four stages. The Pre-xml script is called with the -ascii, --optbreak, and -noindex options to convert the input to Ascii characters, +-optbreak, and -noindex options to convert the input to ASCII characters, insert line break points, and disable the production of an index. Then the xmlto command converts the XML to a single HTML document, using these stylesheets: @@ -494,7 +529,7 @@ symbol is output as "(c)" rather than the Unicode character. This is necessary because the stylesheet itself generates a copyright symbol as part of the document title; the character is not in the original input. -The w3m command is used with the -dump option to turn the HTML file into Ascii +The w3m command is used with the -dump option to turn the HTML file into ASCII text, but this contains multiple sequences of blank lines that make it look awkward. Furthermore, chapter and section titles do not stand out very well. A local Perl script called Tidytxt is used to post-process the output. First, it @@ -504,6 +539,14 @@ preceded by an extra two blank lines and a line of equals characters. An extra newline is inserted before each section heading, and they are underlined with hyphens. +The output of xmlto also contains non-ASCII Unicode characters that w3m passes +through. Fortunately, they are few, and Tidytxt cleans them up as well. Some +headings use "box drawing" characters in the range U+2500 to U+253F which are +translated into -+| as appropriate, and U+00A0 (hard space) and U+25CF (bullet) +are translated into plain spaces and asterisks. (It might be possible to do all +this in the same way as I dealt with copyright - see above - but adding a few +lines of Perl to an existing script was a lot easier.) + CREATING INFO FILES @@ -516,7 +559,7 @@ makeinfo command. These have to be changed to "cindex" and "findex" respectively in the final .texinfo file. Furthermore, the main menu lacks a pointer to the index, and indeed the index node itself is missing. These problems are fixed by running the file through a script called TidyInfo. -Finally, a call of makeinfo creates a set of .info files. +Finally, a call of makeinfo creates a .info file. There is one apparently unconfigurable feature of docbook2texi: it does not seem possible to give it a file name for its output. It chooses a name based on @@ -566,16 +609,16 @@ using SDoP instead of xmlto/fop to produce PostScript and PDF output. that is referenced, instead of to the point in the section where the index marker was set. -(4) The HTML output supports only a single index, so the concept and options - index entries have to be merged. +(4) The HTML output supports only a single index, so the variable, options, + and concept index entries have to be merged. (5) The index for the PostScript/PDF output created by xmlto/fop does not merge identical page numbers, which makes some entries look ugly. This is not a problem when SDoP is used. -(6) None of the indexes (PostScript/PDF and HTML) make use of textual - markup; the text is all roman, without any italic or boldface. For - PostScript/PDF, this is not a problem when SDoP is used. +(6) The HTML index and the PostScript/PDF indexes, when made with xmlto/fop, + make no use of textual markup; the text is all roman, without any italic + or boldface. For PostScript/PDF, this is not a problem when SDoP is used. (7) I turned off hyphenation in the PostScript/PDF output produced by xmlto/fop, because it was being done so badly. Needless to say, I made @@ -588,7 +631,8 @@ using SDoP instead of xmlto/fop to produce PostScript and PDF output. hyphenations, often for several lines in succession. (b) It uses an algorithmic form of hyphenation that doesn't always produce - acceptable word breaks. (I prefer to use a hyphenation dictionary.) + acceptable word breaks. (I prefer to use a hyphenation dictionary, + which is what SDoP does.) (8) The PostScript/PDF output produced by xmlto/fop is badly paginated: @@ -662,5 +706,6 @@ spec.xfpt xfpt source of the specification document x2man Script to make the Exim man page from the XML -Philip Hazel -Last updated: 27 March 2007 +(Originally, and for the most part: Philip Hazel) +The Exim Maintainers +Last updated: 5 July 2010