-$Cambridge: exim/doc/doc-docbook/HowItWorks.txt,v 1.6 2007/04/11 15:26:09 ph10 Exp $
-
CREATING THE EXIM DOCUMENTATION
"You are lost in a maze of twisty little scripts."
A number of issues arose while setting this all up, which are best summed up by
the statement that a lot of the technology was (in 2006) still very immature.
Trying to do this conversion any earlier would probably not have been anywhere
-near as successful. The main problems that bother me in the XML-generated
+near as successful. The main issues that bother me in the XML-generated
documentation are described in the penultimate section of this document.
-The major problems were originally in producing PostScript and PDF outputs. The
+Initially, the major problems were in producing PostScript and PDF outputs. The
available free software for doing this was and still is (we are now in 2007)
cumbersome and slow, and does not support certain output features that I would
like. My response to this was, over a period of two years, to write an XML
XML and writes PostScript, without using any of the heavyweight apparatus that
is required for xmlto and fop (the previously used software).
-An experimental first version of SDoP will be used for the Exim 4.67
-documentation. A full release of SDoP requires further work. SDoP's output
+An experimental first version of SDoP was used for the Exim 4.67
+documentation. Subsequently SDoP was released for general use. SDoP's output
includes features that are missing when xmlto/fop is used, and it also runs
-about 60 times faster. The main manual can be formatted in 2 seconds instead of
-2 minutes, which makes checking and fixing mistakes much easier.
+about 60 times faster. The main manual can be formatted in 2.5 seconds instead
+of 2.5 minutes, which makes checking and fixing mistakes much easier.
The Makefile that is used to build the various forms of output will, for the
moment, support both ways of producing PostScript and PDF output, though the
I am not fully aware of. This is what I know about (version numbers are current
at the time of writing):
-. xfpt 0.01
+. xfpt 0.03
This converts the master source file into a DocBook XML file.
-. sdop 0.00
+. sdop 0.03
- This is my new, still-very-alpha, DocBook-to-PostScript processor.
+ This is my new DocBook-to-PostScript processor.
. ps2pdf
things that I have not figured out, to apply the DocBook XSLT stylesheets.
. libxml 1.8.17
- libxml2 2.6.22
- libxslt 1.1.15
+ libxml2 2.6.28
+ libxslt 1.1.20
These are all installed on my box; I do not know which of libxml or libxml2
the various scripts are actually using.
-. xsl-stylesheets-1.70.1
+. xsl-stylesheets-<version>
These are the standard DocBook XSL stylesheets.
-. fop 0.20.5
+ The documents use http://docbook.sourceforge.net/release/xsl/current/ which
+ should be mapped to an appropriate local path via the system catalogs.
+
+. fop 0.93
FOP is a processor for "formatted objects". It is written in Java. The fop
command is a shell script that drives it. It required only if you do not
want to use SDoP and ps2pdf to generate PostScript and PDF output.
-. w3m 0.5.1
+. w3m 0.5.2
- This is a text-oriented web brower. It is used to produce the Ascii form of
+ This is a text-oriented web brower. It is used to produce the ASCII form of
the Exim documentation (spec.txt) from a specially-created HTML format. It
seems to do a better job than lynx.
. makeinfo 4.8
- This is used to make a set of "info" files from a Texinfo file.
+ This is used to make an "info" file from a Texinfo file.
In addition, there are a number of locally written Perl scripts. These are
described below.
Processing a .xml file into the five different output formats is not entirely
straightforward. For a start, the same XML is not suitable for all the
different output styles. When the final output is in a text format (.txt,
-.texinfo) for instance, all non-Ascii characters in the input must be converted
-to Ascii transliterations because the current processing tools do not do this
+.texinfo) for instance, all non-ASCII characters in the input must be converted
+to ASCII transliterations because the current processing tools do not do this
correctly automatically.
In order to cope with these issues in a flexible way, a Perl script called
changes are quite significant.
+XSL INCLUDES
+
+References to XSL paths should use the public URLs, such as:
+ http://docbook.sourceforge.net/release/xsl/current/xhtml/docbook.xsl
+If this fails to work for you, then there is a problem with your system
+catalogs. As a work-around, you can adjust the OS-Fixups script and then:
+$ make os-fixup
+
+As an example of how this should normally work, on a FreeBSD system the
+resolution goes to /usr/local/share/xml/catalog which contains a directive:
+ <nextCatalog catalog="/usr/local/share/xml/catalog.ports" />
+to pull in the file automatically maintained by the Ports system. That file
+will contain:
+ <delegateSystem
+ systemIdStartString="http://docbook.sourceforge.net/release/xsl/"
+ catalog="file:///usr/local/share/xsl/docbook/catalog" />
+ <delegateURI
+ uriStartString="http://docbook.sourceforge.net/release/xsl/"
+ catalog="file:///usr/local/share/xsl/docbook/catalog" />
+and that catalog file contains:
+ <rewriteSystem
+ systemIdStartString="http://docbook.sourceforge.net/release/xsl/current"
+ rewritePrefix="file:///usr/local/share/xsl/docbook" />
+ <rewriteURI
+ uriStartString="http://docbook.sourceforge.net/release/xsl/current"
+ rewritePrefix="file:///usr/local/share/xsl/docbook" />
+and the full path is thus eventually arrived at.
+
+See also the tools:
+ xmlcatalog(1) from libxml2
+ xmlcatmgr(1) for a lightweight tool written for the NetBSD Packages system.
+
+
THE PRE-XML SCRIPT
The Pre-xml script copies a .xml file, making certain changes according to the
-ascii
- This option is used for Ascii output formats. It makes the following
+ This option is used for ASCII output formats. It makes the following
character replacements:
’ => ' apostrophe
– => - en dash
The apostrophe is specified numerically because that is what xfpt generates
- from an Ascii single quote character. Non-Ascii characters that are not in
+ from an ASCII single quote character. Non-ASCII characters that are not in
this list should not be used without thinking about how they might be
- converted for the Ascii formats.
+ converted for the ASCII formats.
In addition to the character replacements, this option causes quotes to be
put round <literal> text items, and <quote> and </quote> to be replaced by
- Ascii quote marks. You would think the stylesheet would cope with the latter,
- but it seems to generate non-Ascii characters that w3m then turns into
+ ASCII quote marks. You would think the stylesheet would cope with the latter,
+ but it seems to generate non-ASCII characters that w3m then turns into
question marks.
-bookinfo
-noindex
Remove the XML to generate a Concept Index and an Options index. The source
- document has two types of index entry, for a concept and an options index.
- However, no index is required for the .txt and .texinfo outputs.
+ document has three types of index entry, for variables, options, and concept
+ indexes. However, no index is required for the .txt and .texinfo outputs.
-oneindex
- Remove the XML to generate a Concept and an Options Index, and add XML to
- generate a single index. The only output processors that support multiple
- indexes are SDoP and the processor that produces "formatted objects" for
- PostScript and PDF output for fop. The HTML processor ignores the XML
- settings for multiple indexes and just makes one unified index. Specifying
- two indexes gets you two copies of the same index, so this has to be changed.
+ Remove the XML to generate separate variables, options, and concept indexes,
+ and add XML to generate a single index. The only output processors that
+ support multiple indexes are SDoP and the processor that produces "formatted
+ objects" for PostScript and PDF output for fop. The HTML processor ignores
+ the XML settings for multiple indexes and just makes one unified index.
+ Specifying three indexes gets you three copies of the same index, so this has
+ to be changed.
-optbreak
CREATING TEXT FILES
This happens in four stages. The Pre-xml script is called with the -ascii,
--optbreak, and -noindex options to convert the input to Ascii characters,
+-optbreak, and -noindex options to convert the input to ASCII characters,
insert line break points, and disable the production of an index. Then the
xmlto command converts the XML to a single HTML document, using these
stylesheets:
because the stylesheet itself generates a copyright symbol as part of the
document title; the character is not in the original input.
-The w3m command is used with the -dump option to turn the HTML file into Ascii
+The w3m command is used with the -dump option to turn the HTML file into ASCII
text, but this contains multiple sequences of blank lines that make it look
awkward. Furthermore, chapter and section titles do not stand out very well. A
local Perl script called Tidytxt is used to post-process the output. First, it
newline is inserted before each section heading, and they are underlined with
hyphens.
+The output of xmlto also contains non-ASCII Unicode characters that w3m passes
+through. Fortunately, they are few, and Tidytxt cleans them up as well. Some
+headings use "box drawing" characters in the range U+2500 to U+253F which are
+translated into -+| as appropriate, and U+00A0 (hard space) and U+25CF (bullet)
+are translated into plain spaces and asterisks. (It might be possible to do all
+this in the same way as I dealt with copyright - see above - but adding a few
+lines of Perl to an existing script was a lot easier.)
+
CREATING INFO FILES
respectively in the final .texinfo file. Furthermore, the main menu lacks a
pointer to the index, and indeed the index node itself is missing. These
problems are fixed by running the file through a script called TidyInfo.
-Finally, a call of makeinfo creates a set of .info files.
+Finally, a call of makeinfo creates a .info file.
There is one apparently unconfigurable feature of docbook2texi: it does not
seem possible to give it a file name for its output. It chooses a name based on
that is referenced, instead of to the point in the section where the index
marker was set.
-(4) The HTML output supports only a single index, so the concept and options
- index entries have to be merged.
+(4) The HTML output supports only a single index, so the variable, options,
+ and concept index entries have to be merged.
(5) The index for the PostScript/PDF output created by xmlto/fop does not
merge identical page numbers, which makes some entries look ugly. This is
not a problem when SDoP is used.
-(6) None of the indexes (PostScript/PDF and HTML) make use of textual
- markup; the text is all roman, without any italic or boldface. For
- PostScript/PDF, this is not a problem when SDoP is used.
+(6) The HTML index and the PostScript/PDF indexes, when made with xmlto/fop,
+ make no use of textual markup; the text is all roman, without any italic
+ or boldface. For PostScript/PDF, this is not a problem when SDoP is used.
(7) I turned off hyphenation in the PostScript/PDF output produced by
xmlto/fop, because it was being done so badly. Needless to say, I made
hyphenations, often for several lines in succession.
(b) It uses an algorithmic form of hyphenation that doesn't always produce
- acceptable word breaks. (I prefer to use a hyphenation dictionary.)
+ acceptable word breaks. (I prefer to use a hyphenation dictionary,
+ which is what SDoP does.)
(8) The PostScript/PDF output produced by xmlto/fop is badly paginated:
x2man Script to make the Exim man page from the XML
-Philip Hazel
-Last updated: 27 March 2007
+(Originally, and for the most part: Philip Hazel)
+The Exim Maintainers
+Last updated: 5 July 2010
git://git.exim.org / exim.git / blobdiff