= Testing Module =

A framework of scripts and stylesheets for running [TestAlternativeProcessor any XSL formatter] on a set of [wiki:XSLTestSuites test files] (including [DocBookTestdocs DocBook tests]) and working with the test results.

The default configuration works with the xmlroff XSL Formatter.

This module serves multiple purposes for xmlroff:
 * Comparing results between different versions of xmlroff
 * Comparing results between xmlroff’s different output formats
 * Comparing xmlroff output to files produced another way, e.g., from a different formatter – FO or otherwise

The testing module uses a combination of Makefile, Perl, and commonly available utilities to:
 * Run an XSL formatter on one or more test suites conforming to the format originally defined by the W3C XSL FO subgroup for XSL 1.0 Candidate Recommendation testing.  The test definition for a single test is shown below:
{{{
<test id="expression1" fo="multiple" xml="inherited-property-value.fo"
results="inherited-property-value.pdf">Use inherited-property-value()
in expressions for font-size property, including using function with
property other than font-size and using as part of a larger expression.</test>
}}}
 * Rasterise the PDF or !PostScript output
 * Compare the rasterised output against a reference
 * Produce a report summarising all tests and an individual report for each test
 [[Image(xmlroff-test-report.png,border=1, 75%)]][[BR]]
 '''Figure 1''' – xmlroff individual test report
 * Produce a “stereo” image combining the result and reference for any page that is different from its reference version
 [[Image(xmlroff-stereo.png,border=1)]][[BR]]
 '''Figure 2''' – Detail of “stereo” comparison, where the red channel is from the current result and the blue channel is from the reference

When the reports are served by a web server, the record of the test results – which also conforms to the format defined by the W3C XSL FO subgroup – can be updated to indicate whether the result agrees with the expected result or to add a comment about the test result.  The test result for the example is shown below:
{{{
<testresult id="expression1" results="expression1.pdf" agreement="full" specproblem="no" testproblem="no"/>
}}}

== See Also ==

 * [wiki:XSLTestSuites XSLTestSuites] -- Pointers to available XSL FO test suites
 * TestAlternativeProcessor -- Testing an alternative XSL processor
 * DocBookTestdocs -- Testing with the !DocBook `testdocs` module