Aparecium, an XQuery / XSLT parser library for invisible XML

'Invisible XML' ('ixml') is a method for treating non-XML information as if it were XML; it was proposed by Steven Pemberton in 2013. The basic idea is straightforward: a context-free grammar is used to describe the structure of the information, annotations in the grammar specify how the raw parse tree of a sentence in the language described by the grammar is to be represented into XML, and an ixml parser uses the grammar to parse the non-XML document into an XML form. This allows all the tools of the XML toolbox to be applied to the data: XQuery and XSLT for general processing, XForms for creating user interfaces to the data, XML schema languages for validation, and so on.

Aparecium is an ixml parser written in XQuery and XSLT, as a library of functions callable from XQuery and XSLT. (The name is a reference to a spell in the Harry Potter novels, which makes invisible writing visible.) When used to parse external resources, Aparecium can be thought of as a replacement for the standard doc() function which can read non-XML data and deliver it as XML; it can also be used to parse strings which obey a context-free grammar, such as CSS style specifications, XSLT pattern expressions, SVG path expressions, and so on. The latter makes Aparecium useful for handling XML formats which use micro-grammars for some portions of documents.

For simplicity, Aparecium is implemented as a pipeline of processes. First the extended BNF notation allowed in ixml grammars is translated to an equivalent unextended BNF. This grammar is then used by an Earley parser to parse the input; the result is a large set of 'Earley items' describing various aspects of the parse. From the set of Earley items, Aparecium then constructs a 'parse-forest grammar' describing the set of parse trees in the input. As a final step, a parse tree is extracted from the parse-forest grammar and returned to the caller. Alternate interfaces may be used to specify that the parse-forest grammar should be returned, instead; this may be helpful in cases of ambiguity, since it allows the caller to study the ambiguity and in some cases to extract the preferred parse tree.

In some cases the caller will have the grammar in the non-XML form described in the ixml specification; in others, the grammar will be available as an XML document; sometimes the caller will have a URI for the grammar. The input may similarly be available either as a string or as a URI. Aparecium provides distinct calls for each of these situations, to simplify the use of Aparecium in constructing applications.

The talk will briefly describe the current status of Aparecium implementation and (the gods willing) show a simple demo; it will conclude with a discussion of some next steps in the work on Aparecium and in the development of broader support for invisible XML.