-$Cambridge: exim/doc/doc-docbook/HowItWorks.txt,v 1.6 2007/04/11 15:26:09 ph10 Exp $
+$Cambridge: exim/doc/doc-docbook/HowItWorks.txt,v 1.8 2007/08/31 09:13:40 ph10 Exp $
CREATING THE EXIM DOCUMENTATION
A number of issues arose while setting this all up, which are best summed up by
the statement that a lot of the technology was (in 2006) still very immature.
Trying to do this conversion any earlier would probably not have been anywhere
-near as successful. The main problems that bother me in the XML-generated
+near as successful. The main issues that bother me in the XML-generated
documentation are described in the penultimate section of this document.
-The major problems were originally in producing PostScript and PDF outputs. The
+Initially, the major problems were in producing PostScript and PDF outputs. The
available free software for doing this was and still is (we are now in 2007)
cumbersome and slow, and does not support certain output features that I would
like. My response to this was, over a period of two years, to write an XML
XML and writes PostScript, without using any of the heavyweight apparatus that
is required for xmlto and fop (the previously used software).
-An experimental first version of SDoP will be used for the Exim 4.67
-documentation. A full release of SDoP requires further work. SDoP's output
+An experimental first version of SDoP was used for the Exim 4.67
+documentation. Subsequently SDoP was released for general use. SDoP's output
includes features that are missing when xmlto/fop is used, and it also runs
-about 60 times faster. The main manual can be formatted in 2 seconds instead of
-2 minutes, which makes checking and fixing mistakes much easier.
+about 60 times faster. The main manual can be formatted in 2.5 seconds instead
+of 2.5 minutes, which makes checking and fixing mistakes much easier.
The Makefile that is used to build the various forms of output will, for the
moment, support both ways of producing PostScript and PDF output, though the
I am not fully aware of. This is what I know about (version numbers are current
at the time of writing):
-. xfpt 0.01
+. xfpt 0.03
This converts the master source file into a DocBook XML file.
-. sdop 0.00
+. sdop 0.03
- This is my new, still-very-alpha, DocBook-to-PostScript processor.
+ This is my new DocBook-to-PostScript processor.
. ps2pdf
things that I have not figured out, to apply the DocBook XSLT stylesheets.
. libxml 1.8.17
- libxml2 2.6.22
- libxslt 1.1.15
+ libxml2 2.6.28
+ libxslt 1.1.20
These are all installed on my box; I do not know which of libxml or libxml2
the various scripts are actually using.
These are the standard DocBook XSL stylesheets.
-. fop 0.20.5
+. fop 0.93
FOP is a processor for "formatted objects". It is written in Java. The fop
command is a shell script that drives it. It required only if you do not
want to use SDoP and ps2pdf to generate PostScript and PDF output.
-. w3m 0.5.1
+. w3m 0.5.2
- This is a text-oriented web brower. It is used to produce the Ascii form of
+ This is a text-oriented web brower. It is used to produce the ASCII form of
the Exim documentation (spec.txt) from a specially-created HTML format. It
seems to do a better job than lynx.
. makeinfo 4.8
- This is used to make a set of "info" files from a Texinfo file.
+ This is used to make an "info" file from a Texinfo file.
In addition, there are a number of locally written Perl scripts. These are
described below.
Processing a .xml file into the five different output formats is not entirely
straightforward. For a start, the same XML is not suitable for all the
different output styles. When the final output is in a text format (.txt,
-.texinfo) for instance, all non-Ascii characters in the input must be converted
-to Ascii transliterations because the current processing tools do not do this
+.texinfo) for instance, all non-ASCII characters in the input must be converted
+to ASCII transliterations because the current processing tools do not do this
correctly automatically.
In order to cope with these issues in a flexible way, a Perl script called
-ascii
- This option is used for Ascii output formats. It makes the following
+ This option is used for ASCII output formats. It makes the following
character replacements:
’ => ' apostrophe
– => - en dash
The apostrophe is specified numerically because that is what xfpt generates
- from an Ascii single quote character. Non-Ascii characters that are not in
+ from an ASCII single quote character. Non-ASCII characters that are not in
this list should not be used without thinking about how they might be
- converted for the Ascii formats.
+ converted for the ASCII formats.
In addition to the character replacements, this option causes quotes to be
put round <literal> text items, and <quote> and </quote> to be replaced by
- Ascii quote marks. You would think the stylesheet would cope with the latter,
- but it seems to generate non-Ascii characters that w3m then turns into
+ ASCII quote marks. You would think the stylesheet would cope with the latter,
+ but it seems to generate non-ASCII characters that w3m then turns into
question marks.
-bookinfo
-noindex
Remove the XML to generate a Concept Index and an Options index. The source
- document has two types of index entry, for a concept and an options index.
- However, no index is required for the .txt and .texinfo outputs.
+ document has three types of index entry, for variables, options, and concept
+ indexes. However, no index is required for the .txt and .texinfo outputs.
-oneindex
- Remove the XML to generate a Concept and an Options Index, and add XML to
- generate a single index. The only output processors that support multiple
- indexes are SDoP and the processor that produces "formatted objects" for
- PostScript and PDF output for fop. The HTML processor ignores the XML
- settings for multiple indexes and just makes one unified index. Specifying
- two indexes gets you two copies of the same index, so this has to be changed.
+ Remove the XML to generate separate variables, options, and concept indexes,
+ and add XML to generate a single index. The only output processors that
+ support multiple indexes are SDoP and the processor that produces "formatted
+ objects" for PostScript and PDF output for fop. The HTML processor ignores
+ the XML settings for multiple indexes and just makes one unified index.
+ Specifying three indexes gets you three copies of the same index, so this has
+ to be changed.
-optbreak
CREATING TEXT FILES
This happens in four stages. The Pre-xml script is called with the -ascii,
--optbreak, and -noindex options to convert the input to Ascii characters,
+-optbreak, and -noindex options to convert the input to ASCII characters,
insert line break points, and disable the production of an index. Then the
xmlto command converts the XML to a single HTML document, using these
stylesheets:
because the stylesheet itself generates a copyright symbol as part of the
document title; the character is not in the original input.
-The w3m command is used with the -dump option to turn the HTML file into Ascii
+The w3m command is used with the -dump option to turn the HTML file into ASCII
text, but this contains multiple sequences of blank lines that make it look
awkward. Furthermore, chapter and section titles do not stand out very well. A
local Perl script called Tidytxt is used to post-process the output. First, it
newline is inserted before each section heading, and they are underlined with
hyphens.
+August 2007: A further feature has been added to Tidytxt. The current version
+of xmlto makes HTML that contains non-ASCII Unicode characters. Fortunately,
+they are few. The heading uses "box drawing" characters in the range U+2500 to
+U+253F, and within the main text, U+00A0 (hard space) occasionally appears. The
+Tidytxt script now turns all the former into hyphens and the latter into normal
+spaces. Bullets, which are set as U+25CF, are turned into asterisks. (It might
+be possible to do all this in the same way as I dealt with copyright - see
+above - but adding three lines of Perl to an existing script was a lot easier.)
+
CREATING INFO FILES
respectively in the final .texinfo file. Furthermore, the main menu lacks a
pointer to the index, and indeed the index node itself is missing. These
problems are fixed by running the file through a script called TidyInfo.
-Finally, a call of makeinfo creates a set of .info files.
+Finally, a call of makeinfo creates a .info file.
There is one apparently unconfigurable feature of docbook2texi: it does not
seem possible to give it a file name for its output. It chooses a name based on
that is referenced, instead of to the point in the section where the index
marker was set.
-(4) The HTML output supports only a single index, so the concept and options
- index entries have to be merged.
+(4) The HTML output supports only a single index, so the variable, options,
+ and concept index entries have to be merged.
(5) The index for the PostScript/PDF output created by xmlto/fop does not
merge identical page numbers, which makes some entries look ugly. This is
not a problem when SDoP is used.
-(6) None of the indexes (PostScript/PDF and HTML) make use of textual
- markup; the text is all roman, without any italic or boldface. For
- PostScript/PDF, this is not a problem when SDoP is used.
+(6) The HTML index and the PostScript/PDF indexes, when made with xmlto/fop,
+ make no use of textual markup; the text is all roman, without any italic
+ or boldface. For PostScript/PDF, this is not a problem when SDoP is used.
(7) I turned off hyphenation in the PostScript/PDF output produced by
xmlto/fop, because it was being done so badly. Needless to say, I made
hyphenations, often for several lines in succession.
(b) It uses an algorithmic form of hyphenation that doesn't always produce
- acceptable word breaks. (I prefer to use a hyphenation dictionary.)
+ acceptable word breaks. (I prefer to use a hyphenation dictionary,
+ which is what SDoP does.)
(8) The PostScript/PDF output produced by xmlto/fop is badly paginated:
Philip Hazel
-Last updated: 27 March 2007
+Last updated: 31 August 2007
git://git.exim.org / exim.git / blobdiff