catalog.html revision 8a469171206edc0dbb25d115b7454c6e599ab1c5
1<?xml version="1.0" encoding="ISO-8859-1"?> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 3<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" /><link rel="SHORTCUT ICON" href="/favicon.ico" /><style type="text/css"> 4TD {font-family: Verdana,Arial,Helvetica} 5BODY {font-family: Verdana,Arial,Helvetica; margin-top: 2em; margin-left: 0em; margin-right: 0em} 6H1 {font-family: Verdana,Arial,Helvetica} 7H2 {font-family: Verdana,Arial,Helvetica} 8H3 {font-family: Verdana,Arial,Helvetica} 9A:link, A:visited, A:active { text-decoration: underline } 10</style><title>Catalog support</title></head><body bgcolor="#8b7765" text="#000000" link="#000000" vlink="#000000"><table border="0" width="100%" cellpadding="5" cellspacing="0" align="center"><tr><td width="180"><a href="http://www.gnome.org/"><img src="gnome2.png" alt="Gnome2 Logo" /></a><a href="http://www.w3.org/Status"><img src="w3c.png" alt="W3C Logo" /></a><a href="http://www.redhat.com/"><img src="redhat.gif" alt="Red Hat Logo" /></a><div align="left"><a href="http://xmlsoft.org/"><img src="Libxml2-Logo-180x168.gif" alt="Made with Libxml2 Logo" /></a></div></td><td><table border="0" width="90%" cellpadding="2" cellspacing="0" align="center" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3" bgcolor="#fffacd"><tr><td align="center"><h1>The XML C parser and toolkit for Gnome</h1><h2>Catalog support</h2></td></tr></table></td></tr></table></td></tr></table><table border="0" cellpadding="4" cellspacing="0" width="100%" align="center"><tr><td bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="2" width="100%"><tr><td valign="top" width="200" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Main Menu</b></center></td></tr><tr><td bgcolor="#fffacd"><form action="search.php" enctype="application/x-www-form-urlencoded" method="get"><input name="query" type="text" size="20" value="" /><input name="submit" type="submit" value="Search ..." /></form><ul><li><a href="index.html">Home</a></li><li><a href="intro.html">Introduction</a></li><li><a href="FAQ.html">FAQ</a></li><li><a href="docs.html">Developer Documentation</a></li><li><a href="bugs.html">Reporting bugs and getting help</a></li><li><a href="help.html">How to help</a></li><li><a href="downloads.html">Downloads</a></li><li><a href="news.html">News</a></li><li><a href="XMLinfo.html">XML</a></li><li><a href="XSLT.html">XSLT</a></li><li><a href="xmldtd.html">Validation & DTDs</a></li><li><a href="encoding.html">Encodings support</a></li><li><a href="catalog.html">Catalog support</a></li><li><a href="namespaces.html">Namespaces</a></li><li><a href="contribs.html">Contributions</a></li><li><a href="guidelines.html">XML Guidelines</a></li></ul></td></tr></table><table width="100%" border="0" cellspacing="1" cellpadding="3"><tr><td colspan="1" bgcolor="#eecfa1" align="center"><center><b>Related links</b></center></td></tr><tr><td bgcolor="#fffacd"><ul><li><a href="http://mail.gnome.org/archives/xml/">Mail archive</a></li><li><a href="http://xmlsoft.org/XSLT/">XSLT libxslt</a></li><li><a href="http://phd.cs.unibo.it/gdome2/">DOM gdome2</a></li><li><a href="http://www.aleksey.com/xmlsec/">XML-DSig xmlsec</a></li><li><a href="ftp://xmlsoft.org/">FTP</a></li><li><a href="http://www.zlatkovic.com/projects/libxml/">Windows binaries</a></li><li><a href="http://garypennington.net/libxml2/">Solaris binaries</a></li><li><a href="http://www.zveno.com/open_source/libxml2xslt.html">MacOsX binaries</a></li><li><a href="http://sourceforge.net/projects/libxml2-pas/">Pascal bindings</a></li><li><a href="http://bugzilla.gnome.org/buglist.cgi?product=libxml&product=libxml2">Bug Tracker</a></li></ul></td></tr></table></td></tr></table></td><td valign="top" bgcolor="#8b7765"><table border="0" cellspacing="0" cellpadding="1" width="100%"><tr><td><table border="0" cellspacing="0" cellpadding="1" width="100%" bgcolor="#000000"><tr><td><table border="0" cellpadding="3" cellspacing="1" width="100%"><tr><td bgcolor="#fffacd"><p>Table of Content:</p><ol><li><a href="General2">General overview</a></li> 11 <li><a href="#definition">The definition</a></li> 12 <li><a href="#Simple">Using catalogs</a></li> 13 <li><a href="#Some">Some examples</a></li> 14 <li><a href="#reference">How to tune catalog usage</a></li> 15 <li><a href="#validate">How to debug catalog processing</a></li> 16 <li><a href="#Declaring">How to create and maintain catalogs</a></li> 17 <li><a href="#implemento">The implementor corner quick review of the 18 API</a></li> 19 <li><a href="#Other">Other resources</a></li> 20</ol><h3><a name="General2" id="General2">General overview</a></h3><p>What is a catalog? Basically it's a lookup mechanism used when an entity 21(a file or a remote resource) references another entity. The catalog lookup 22is inserted between the moment the reference is recognized by the software 23(XML parser, stylesheet processing, or even images referenced for inclusion 24in a rendering) and the time where loading that resource is actually 25started.</p><p>It is basically used for 3 things:</p><ul><li>mapping from "logical" names, the public identifiers and a more 26 concrete name usable for download (and URI). For example it can associate 27 the logical name 28 <p>"-//OASIS//DTD DocBook XML V4.1.2//EN"</p> 29 <p>of the DocBook 4.1.2 XML DTD with the actual URL where it can be 30 downloaded</p> 31 <p>http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd</p> 32 </li> 33 <li>remapping from a given URL to another one, like an HTTP indirection 34 saying that 35 <p>"http://www.oasis-open.org/committes/tr.xsl";</p> 36 <p>should really be looked at</p> 37 <p>"http://www.oasis-open.org/committes/entity/stylesheets/base/tr.xsl";</p> 38 </li> 39 <li>providing a local cache mechanism allowing to load the entities 40 associated to public identifiers or remote resources, this is a really 41 important feature for any significant deployment of XML or SGML since it 42 allows to avoid the aleas and delays associated to fetching remote 43 resources.</li> 44</ul><h3><a name="definition" id="definition">The definitions</a></h3><p>Libxml, as of 2.4.3 implements 2 kind of catalogs:</p><ul><li>the older SGML catalogs, the official spec is SGML Open Technical 45 Resolution TR9401:1997, but is better understood by reading <a href="http://www.jclark.com/sp/catalog.htm">the SP Catalog page</a> from 46 James Clark. This is relatively old and not the preferred mode of 47 operation of libxml.</li> 48 <li><a href="http://www.oasis-open.org/committees/entity/spec.html">XML 49 Catalogs</a> is far more flexible, more recent, uses an XML syntax and 50 should scale quite better. This is the default option of libxml.</li> 51</ul><p></p><h3><a name="Simple" id="Simple">Using catalog</a></h3><p>In a normal environment libxml2 will by default check the presence of a 52catalog in /etc/xml/catalog, and assuming it has been correctly populated, 53the processing is completely transparent to the document user. To take a 54concrete example, suppose you are authoring a DocBook document, this one 55starts with the following DOCTYPE definition:</p><pre><?xml version='1.0'?> 56<!DOCTYPE book PUBLIC "-//Norman Walsh//DTD DocBk XML V3.1.4//EN" 57 "http://nwalsh.com/docbook/xml/3.1.4/db3xml.dtd">;</pre><p>When validating the document with libxml, the catalog will be 58automatically consulted to lookup the public identifier "-//Norman Walsh//DTD 59DocBk XML V3.1.4//EN" and the system identifier 60"http://nwalsh.com/docbook/xml/3.1.4/db3xml.dtd";, and if these entities have 61been installed on your system and the catalogs actually point to them, libxml 62will fetch them from the local disk.</p><p style="font-size: 10pt"><strong>Note</strong>: Really don't use this 63DOCTYPE example it's a really old version, but is fine as an example.</p><p>Libxml2 will check the catalog each time that it is requested to load an 64entity, this includes DTD, external parsed entities, stylesheets, etc ... If 65your system is correctly configured all the authoring phase and processing 66should use only local files, even if your document stays portable because it 67uses the canonical public and system ID, referencing the remote document.</p><h3><a name="Some" id="Some">Some examples:</a></h3><p>Here is a couple of fragments from XML Catalogs used in libxml2 early 68regression tests in <code>test/catalogs</code> :</p><pre><?xml version="1.0"?> 69<!DOCTYPE catalog PUBLIC 70 "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" 71 "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">; 72<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> 73 <public publicId="-//OASIS//DTD DocBook XML V4.1.2//EN" 74 uri="http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"/>; 75...</pre><p>This is the beginning of a catalog for DocBook 4.1.2, XML Catalogs are 76written in XML, there is a specific namespace for catalog elements 77"urn:oasis:names:tc:entity:xmlns:xml:catalog". The first entry in this 78catalog is a <code>public</code> mapping it allows to associate a Public 79Identifier with an URI.</p><pre>... 80 <rewriteSystem systemIdStartString="http://www.oasis-open.org/docbook/"; 81 rewritePrefix="file:///usr/share/xml/docbook/"/> 82...</pre><p>A <code>rewriteSystem</code> is a very powerful instruction, it says that 83any URI starting with a given prefix should be looked at another URI 84constructed by replacing the prefix with an new one. In effect this acts like 85a cache system for a full area of the Web. In practice it is extremely useful 86with a file prefix if you have installed a copy of those resources on your 87local system.</p><pre>... 88<delegatePublic publicIdStartString="-//OASIS//DTD XML Catalog //" 89 catalog="file:///usr/share/xml/docbook.xml"/> 90<delegatePublic publicIdStartString="-//OASIS//ENTITIES DocBook XML" 91 catalog="file:///usr/share/xml/docbook.xml"/> 92<delegatePublic publicIdStartString="-//OASIS//DTD DocBook XML" 93 catalog="file:///usr/share/xml/docbook.xml"/> 94<delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/"; 95 catalog="file:///usr/share/xml/docbook.xml"/> 96<delegateURI uriStartString="http://www.oasis-open.org/docbook/"; 97 catalog="file:///usr/share/xml/docbook.xml"/> 98...</pre><p>Delegation is the core features which allows to build a tree of catalogs, 99easier to maintain than a single catalog, based on Public Identifier, System 100Identifier or URI prefixes it instructs the catalog software to look up 101entries in another resource. This feature allow to build hierarchies of 102catalogs, the set of entries presented should be sufficient to redirect the 103resolution of all DocBook references to the specific catalog in 104<code>/usr/share/xml/docbook.xml</code> this one in turn could delegate all 105references for DocBook 4.2.1 to a specific catalog installed at the same time 106as the DocBook resources on the local machine.</p><h3><a name="reference" id="reference">How to tune catalog usage:</a></h3><p>The user can change the default catalog behaviour by redirecting queries 107to its own set of catalogs, this can be done by setting the 108<code>XML_CATALOG_FILES</code> environment variable to a list of catalogs, an 109empty one should deactivate loading the default <code>/etc/xml/catalog</code> 110default catalog</p><h3><a name="validate" id="validate">How to debug catalog processing:</a></h3><p>Setting up the <code>XML_DEBUG_CATALOG</code> environment variable will 111make libxml2 output debugging informations for each catalog operations, for 112example:</p><pre>orchis:~/XML -> xmllint --memory --noout test/ent2 113warning: failed to load external entity "title.xml" 114orchis:~/XML -> export XML_DEBUG_CATALOG= 115orchis:~/XML -> xmllint --memory --noout test/ent2 116Failed to parse catalog /etc/xml/catalog 117Failed to parse catalog /etc/xml/catalog 118warning: failed to load external entity "title.xml" 119Catalogs cleanup 120orchis:~/XML -> </pre><p>The test/ent2 references an entity, running the parser from memory makes 121the base URI unavailable and the the "title.xml" entity cannot be loaded. 122Setting up the debug environment variable allows to detect that an attempt is 123made to load the <code>/etc/xml/catalog</code> but since it's not present the 124resolution fails.</p><p>But the most advanced way to debug XML catalog processing is to use the 125<strong>xmlcatalog</strong> command shipped with libxml2, it allows to load 126catalogs and make resolution queries to see what is going on. This is also 127used for the regression tests:</p><pre>orchis:~/XML -> /xmlcatalog test/catalogs/docbook.xml \ 128 "-//OASIS//DTD DocBook XML V4.1.2//EN" 129http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd 130orchis:~/XML -> </pre><p>For debugging what is going on, adding one -v flags increase the verbosity 131level to indicate the processing done (adding a second flag also indicate 132what elements are recognized at parsing):</p><pre>orchis:~/XML -> /xmlcatalog -v test/catalogs/docbook.xml \ 133 "-//OASIS//DTD DocBook XML V4.1.2//EN" 134Parsing catalog test/catalogs/docbook.xml's content 135Found public match -//OASIS//DTD DocBook XML V4.1.2//EN 136http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd 137Catalogs cleanup 138orchis:~/XML -> </pre><p>A shell interface is also available to debug and process multiple queries 139(and for regression tests):</p><pre>orchis:~/XML -> /xmlcatalog -shell test/catalogs/docbook.xml \ 140 "-//OASIS//DTD DocBook XML V4.1.2//EN" 141> help 142Commands available: 143public PublicID: make a PUBLIC identifier lookup 144system SystemID: make a SYSTEM identifier lookup 145resolve PublicID SystemID: do a full resolver lookup 146add 'type' 'orig' 'replace' : add an entry 147del 'values' : remove values 148dump: print the current catalog state 149debug: increase the verbosity level 150quiet: decrease the verbosity level 151exit: quit the shell 152> public "-//OASIS//DTD DocBook XML V4.1.2//EN" 153http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd 154> quit 155orchis:~/XML -> </pre><p>This should be sufficient for most debugging purpose, this was actually 156used heavily to debug the XML Catalog implementation itself.</p><h3><a name="Declaring" id="Declaring">How to create and maintain</a> catalogs:</h3><p>Basically XML Catalogs are XML files, you can either use XML tools to 157manage them or use <strong>xmlcatalog</strong> for this. The basic step is 158to create a catalog the -create option provide this facility:</p><pre>orchis:~/XML -> /xmlcatalog --create tst.xml 159<?xml version="1.0"?> 160<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" 161 "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">; 162<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/> 163orchis:~/XML -> </pre><p>By default xmlcatalog does not overwrite the original catalog and save the 164result on the standard output, this can be overridden using the -noout 165option. The <code>-add</code> command allows to add entries in the 166catalog:</p><pre>orchis:~/XML -> /xmlcatalog --noout --create --add "public" \ 167 "-//OASIS//DTD DocBook XML V4.1.2//EN" \ 168 http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd tst.xml 169orchis:~/XML -> cat tst.xml 170<?xml version="1.0"?> 171<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" \ 172 "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">; 173<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> 174<public publicId="-//OASIS//DTD DocBook XML V4.1.2//EN" 175 uri="http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"/>; 176</catalog> 177orchis:~/XML -> </pre><p>The <code>-add</code> option will always take 3 parameters even if some of 178the XML Catalog constructs (like nextCatalog) will have only a single 179argument, just pass a third empty string, it will be ignored.</p><p>Similarly the <code>-del</code> option remove matching entries from the 180catalog:</p><pre>orchis:~/XML -> /xmlcatalog --del \ 181 "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"; tst.xml 182<?xml version="1.0"?> 183<!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" 184 "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">; 185<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"/> 186orchis:~/XML -> </pre><p>The catalog is now empty. Note that the matching of <code>-del</code> is 187exact and would have worked in a similar fashion with the Public ID 188string.</p><p>This is rudimentary but should be sufficient to manage a not too complex 189catalog tree of resources.</p><h3><a name="implemento" id="implemento">The implementor corner quick review of the 190API:</a></h3><p>First, and like for every other module of libxml, there is an 191automatically generated <a href="html/libxml-catalog.html">API page for 192catalog support</a>.</p><p>The header for the catalog interfaces should be included as:</p><pre>#include <libxml/catalog.h></pre><p>The API is voluntarily kept very simple. First it is not obvious that 193applications really need access to it since it is the default behaviour of 194libxml2 (Note: it is possible to completely override libxml2 default catalog by 195using <a href="html/libxml-parser.html">xmlSetExternalEntityLoader</a> to 196plug an application specific resolver).</p><p>Basically libxml2 support 2 catalog lists:</p><ul><li>the default one, global shared by all the application</li> 197 <li>a per-document catalog, this one is built if the document uses the 198 <code>oasis-xml-catalog</code> PIs to specify its own catalog list, it is 199 associated to the parser context and destroyed when the parsing context 200 is destroyed.</li> 201</ul><p>the document one will be used first if it exists.</p><h4>Initialization routines:</h4><p>xmlInitializeCatalog(), xmlLoadCatalog() and xmlLoadCatalogs() should be 202used at startup to initialize the catalog, if the catalog should be 203initialized with specific values xmlLoadCatalog() or xmlLoadCatalogs() 204should be called before xmlInitializeCatalog() which would otherwise do a 205default initialization first.</p><p>The xmlCatalogAddLocal() call is used by the parser to grow the document 206own catalog list if needed.</p><h4>Preferences setup:</h4><p>The XML Catalog spec requires the possibility to select default 207preferences between public and system delegation, 208xmlCatalogSetDefaultPrefer() allows this, xmlCatalogSetDefaults() and 209xmlCatalogGetDefaults() allow to control if XML Catalogs resolution should 210be forbidden, allowed for global catalog, for document catalog or both, the 211default is to allow both.</p><p>And of course xmlCatalogSetDebug() allows to generate debug messages 212(through the xmlGenericError() mechanism).</p><h4>Querying routines:</h4><p>xmlCatalogResolve(), xmlCatalogResolveSystem(), xmlCatalogResolvePublic() 213and xmlCatalogResolveURI() are relatively explicit if you read the XML 214Catalog specification they correspond to section 7 algorithms, they should 215also work if you have loaded an SGML catalog with a simplified semantic.</p><p>xmlCatalogLocalResolve() and xmlCatalogLocalResolveURI() are the same but 216operate on the document catalog list</p><h4>Cleanup and Miscellaneous:</h4><p>xmlCatalogCleanup() free-up the global catalog, xmlCatalogFreeLocal() is 217the per-document equivalent.</p><p>xmlCatalogAdd() and xmlCatalogRemove() are used to dynamically modify the 218first catalog in the global list, and xmlCatalogDump() allows to dump a 219catalog state, those routines are primarily designed for xmlcatalog, I'm not 220sure that exposing more complex interfaces (like navigation ones) would be 221really useful.</p><p>The xmlParseCatalogFile() is a function used to load XML Catalog files, 222it's similar as xmlParseFile() except it bypass all catalog lookups, it's 223provided because this functionality may be useful for client tools.</p><h4>threaded environments:</h4><p>Since the catalog tree is built progressively, some care has been taken to 224try to avoid troubles in multithreaded environments. The code is now thread 225safe assuming that the libxml2 library has been compiled with threads 226support.</p><p></p><h3><a name="Other" id="Other">Other resources</a></h3><p>The XML Catalog specification is relatively recent so there isn't much 227literature to point at:</p><ul><li>You can find a good rant from Norm Walsh about <a href="http://www.arbortext.com/Think_Tank/XML_Resources/Issue_Three/issue_three.html">the 228 need for catalogs</a>, it provides a lot of context informations even if 229 I don't agree with everything presented. Norm also wrote a more recent 230 article <a href="http://wwws.sun.com/software/xml/developers/resolver/article/">XML 231 entities and URI resolvers</a> describing them.</li> 232 <li>An <a href="http://home.ccil.org/~cowan/XML/XCatalog.html">old XML 233 catalog proposal</a> from John Cowan</li> 234 <li>The <a href="http://www.rddl.org/">Resource Directory Description 235 Language</a> (RDDL) another catalog system but more oriented toward 236 providing metadata for XML namespaces.</li> 237 <li>the page from the OASIS Technical <a href="http://www.oasis-open.org/committees/entity/">Committee on Entity 238 Resolution</a> who maintains XML Catalog, you will find pointers to the 239 specification update, some background and pointers to others tools 240 providing XML Catalog support</li> 241 <li>Here is a <a href="buildDocBookCatalog">shell script</a> to generate 242 XML Catalogs for DocBook 4.1.2 . If it can write to the /etc/xml/ 243 directory, it will set-up /etc/xml/catalog and /etc/xml/docbook based on 244 the resources found on the system. Otherwise it will just create 245 ~/xmlcatalog and ~/dbkxmlcatalog and doing: 246 <p><code>export XML_CATALOG_FILES=$HOME/xmlcatalog</code></p> 247 <p>should allow to process DocBook documentations without requiring 248 network accesses for the DTD or stylesheets</p> 249 </li> 250 <li>I have uploaded <a href="ftp://xmlsoft.org/test/dbk412catalog.tar.gz">a 251 small tarball</a> containing XML Catalogs for DocBook 4.1.2 which seems 252 to work fine for me too</li> 253 <li>The <a href="http://www.xmlsoft.org/xmlcatalog_man.html">xmlcatalog 254 manual page</a></li> 255</ul><p>If you have suggestions for corrections or additions, simply contact 256me:</p><p><a href="bugs.html">Daniel Veillard</a></p></td></tr></table></td></tr></table></td></tr></table></td></tr></table></td></tr></table></body></html> 257
