html/whatpm/readme.en.html

<!DOCTYPE html>
<html lang="en">
<head>
<title>Whatpm &mdash; Perl Modules for Web Hypertext Application 
Technologies (beta)</title>
<link rel="stylesheet" href="http://suika.fam.cx/www/style/html/xhtml">
<link rel="license" href="#license">
<link rel="author" href="#author">
<link rel=feed href="whatpm-commit" type="application/atom+xml"
    title="ChangeLog diffs">
</head>
<body>
<h1>Whatpm &mdash; Perl modules for Web hypertext application technologies
(<em>beta</em>)</h1>

<div class="section" id="introduction">
<h2>Introduction</h2>

<p><dfn>Whatpm</dfn> is a <em>work-in-progress</em> set of
<mark>P</mark>erl <mark>m</mark>odules for <mark>W</mark>eb
<mark>h</mark>ypertext <mark>a</mark>pplication
<mark>t</mark>echnologies.  It is part of the <a
href="http://suika.fam.cx/www/2006/manakai/" rel=up>manakai</a>
project.</p>

<p>Whatpm supports various Web standard technologies, including <a
href="#modules-html">HTML, XHTML</a>, <a href="#modules-xml">XML</a>,
<a hreF="#modules-css">CSS</a>, <a href="#modules-http">HTTP</a>, and
<a href="#modules-url">URL</a>.
</div>

<div class=section id=news>
<h2>What's new?</h2>

<p><a href="whatpm-commit" rel=feed type="application/atom+xml">An
Atom feed for ChangeLog diffs</a> is available.

<ul>

<li><time>2008-10-21</time>: <a href="#module-whatpm-xml-parser">An
XML parser with non-draconian error handling</a> released.
(Documentation is not available yet.)

</ul>

</div>

<div class=section id=modules>
<h2>Modules</h2>

  <p>Note that all of these modules are <em>work in progress</em>
  and have <a href="#todo">a number of unresolved problems</a>.</p>

  <p>Note also that some modules have no documentation for now.</p>

<div class=section id=modules-html-xml>
<h3>Modules for HTML and XML</h3>

<p id=modules-html>Modules related to HTML and XHTML are as follows:
<dl>
<dt id=module-whatpm-html><a href="Whatpm/HTML.html"><code>Whatpm::HTML</code></a></dt>
  <dd>An implementation of HTML5 document and fragment
  parsing algorithms.  It can be used
  to convert an arbitrary string into a 
  <abbr title="Document Object Model">DOM</abbr>.  (See also
  <a href="#demo-html-parser">demo</a>.)</dd>
<dt id=module-whatpm-html-serializer><a href="Whatpm/HTML/Serializer.html"><code>Whatpm::HTML::Serializer</code></a></dt>
  <dd>An implementation of HTML5 fragment serialization algorithm.
  (See also <a href="#demo-html-parser">demo</a>.)</dd>
<dt><a href="Whatpm/HTMLTable.html"><code>Whatpm::HTMLTable</code></a></dt>
  <dd>An implementation of the HTML5 table algorithm.  It can be
  used to extract a table structure from a DOM <code>table</code>
  element node.  (See also <a href="#demo-html-table">demo</a>.)</dd>
</dl>

<p id=modules-xml>Modules for the XML support is as follow:

<dl>

<dt id=module-whatpm-xml-parser><a
href="Whatpm/XML/Parser.html"><code>Whatpm::XML::Parser</code></a>

<dd><p>An XML parser with non-draconian error handling.  It can
construct a DOM tree from XML 1.0/1.1 documents that does <em>not</em>
rely on external entities (including the external subset entity) and
that does <em>not</em> contain general entity reference that
references an entity whose replacement text contains character
<code>&</code> or <code>&lt;</code>.  It also supports XML namespaces.

<p>It does not stop the process to construct a DOM tree even if it
detects a well-formedness or a namespace well-formedness error.  It
recovers from errors in a manner similar to HTML5's tokenization
algorithm.  It is expected that the combination of this module and a
future extension to the <code>Whatpm::ContentChecker</code> framework
will provide a mean to detect all well-formedness and validity errors,
if desired.

<p>(See also <a href="#demo-html-parser-manakai">demo</a>.)

<dt><a href="Whatpm/XMLSerializer.html"><code>Whatpm::XMLSerializer</code></a></dt>

<dd>A simple XML serializer.  It does namespace prefix fixups and
suitable for the serialization of a carefully build XML DOM tree.  It
does not assure that the output is well-formed.

</dl>

<p id=modules-cc>The module for conformance checking of a DOM tree (i.e.
a in-memory representation of an HTML or XML document) is as follows:
<dl>

<dt><a href="Whatpm/ContentChecker.html"><code>Whatpm::ContentChecker</code></a></dt>

  <dd>A DOM5 HTML (in-memory representation of a document) conformance
checker with a partial support for Atom 1.0.  (See also <a
href="#demo-html-parser">demo</a> and <a
href="#app-webhacc">application</a>.)

</dl>

<p>Currently, conformance checking of HTML/XHTML and Atom documents
is supported.

<p>For these modules, a DOM implementation that supports the manakai's
Perl binding<!-- @@ TODO: ref --> of DOM is necessary to represent a
document in memory.  The <a
href="http://suika.fam.cx/www/manakai-core/doc/web/">manakai-core</a>
package contains such an implementation,
<code>Message::DOM::Implementation</code><!-- @@ TODO: ref -->, but it
should also be possible to use any other implementation that supports
the binding.

</div>

<div class=section id=modules-css>
<h3>Modules for CSS</h3>

<p>Modules for CSS and related technologies are as follows:
<dl>
<dt><a href="Whatpm/CSS/Cascade.html"><code>Whatpm::CSS::Cascade</code></a>
  <dd>A media-independent implementation of CSS cascading and value
  computations.  (See also <a href="#demo-css-parser">demo</a>.)

<dt><a href="Whatpm/CSS/MediaQueryParser.pm"><code>Whatpm::CSS::MediaQueryParser</code></a>

  <dd>A media query parser.  Note that only CSS 2.1 media types are
supported at the moment.

<dt><a href="Whatpm/CSS/MediaQuerySerializer.pm"><code>Whatpm::CSS::MediaQuerySerializer</code></a>

  <dd>A media query serializer.  Note that only CSS 2.1 media types
are supported at the moment.

<dt><a href="Whatpm/CSS/Parser.html"><code>Whatpm::CSS::Parser</code></a>
  <dd>A CSS parser that constructs CSSOM trees from style sheets.  (See
  also <a href="#demo-css-parser">demo</a>.)
<dt><a href="Whatpm/CSS/SelectorsParser.html"><code>Whatpm::CSS::SelectorsParser</code></a></dt>
  <dd>A <a href="http://www.w3.org/TR/css3-selectors/#grouping">group of
  selectors</a> parser.  (See also <a href="#demo-css-parser">demo</a>.)</dd>
<dt><a href="Whatpm/CSS/SelectorsSerializer.html"><code>Whatpm::CSS::SelectorsSerializer</code></a></dt>
  <dd>A <a href="http://www.w3.org/TR/css3-selectors/#grouping">group of
  selectors</a> serializer.  (See also <a href="#spec-ssft">specification</a>
  and <a href="#demo-css-parser">demo</a>.)</dd>
<dt><a href="Whatpm/CSS/Tokenizer.html"><code>Whatpm::CSS::Tokenizer</code></a></dt>
  <dd>A CSS tokenizer.  (See also <a href="#demo-css-parser">demo</a>.)</dd>
</dl>

<p>For the <code>Whatpm::CSS::Parser</code> module reresenting a CSSOM
tree, modules in the <a
href="http://suika.fam.cx/www/manakai-core/doc/web/">manakai-core</a>
package are used.  Those modules also provide the serializer for the
CSSOM tree, in the form of <a href="#howtouse">the standard
<code>css_text</code> CSSOM attribute</a>.

</div>

<div class=section id=modules-http>
<h3>Modules for HTTP</h3>

<p>Modules for HTTP and related technologies are as follows:
<dl>
<dt><a href="Whatpm/ContentType.html"><code>Whatpm::ContentType</code></a></dt>
  <dd>An implementation of HTML5 Content Type sniffing algorithm.</dd>
<dt><a href="Whatpm/IMTChecker.html"><code>Whatpm::IMTChecker</code></a></dt>
  <dd>An Internet Media Type (<abbr>aka</abbr> MIME type) label
  conformance checker.</dd>
</dl>

<p>Currently, support for parsing of HTTP headers and as such is not
yet available.
</div>

<div class=section id=modules-url>
<h3>Module for URL</h3>

<p>Module for the URL support is as follows:
<dl>
<dt><a href="Whatpm/URIChecker.html"><code>Whatpm::URIChecker</code></a></dt>
  <dd>An IRI reference conformance checker.</dd>
</dl>

<p>Support for HTML5's realistic definition of URL is not available yet.
</div>

<div class=section id=modules-misc>
<h3>Modules for other technologies</h3>

<p>Following modules provide support for other Web-related technologies:
<dl>
<dt><a href="Whatpm/CacheManifest.html"><code>Whatpm::CacheManifest</code></a></dt>
  <dd>An
  <a href="http://www.whatwg.org/specs/web-apps/current-work/#manifests">HTML5
  cache manifest</a> parser.</dd>
<dt><a href="Whatpm/Charset/DecodeHandle.html"><code>Whatpm::Charset::DecodeHandle</code></a>
  <dd>A filehandle-like wrapper <a href="#doc-handles">interface</a> to
  decode byte stream encoded in some character encoding.
<dt><a href="Whatpm/Charset/UnicodeChecker.html"><code>Whatpm::Charset::UnicodeChecker</code></a>
  <dd>A Unicode character string checker.
<dt id=whatpm-charset-universalchardet><a href="Whatpm/Charset/UniversalCharDet.html"><code>Whatpm::Charset::UniversalCharDet</code></a></dt>
  <dd>A Perl interface to universalchardet character encoding detection
  library.</dd>
  <dt><a href="Whatpm/LangTag.html"><code>Whatpm::LangTag</code></a>
    <dd>A language tag parser and conformance checker, supporting both
    older RFC 3066 definition and latest RFC 4646 definition.  (See also
    <a href="#demo-langtag">demo</a>.)

<dt><a href="Whatpm/RDFXML.html"><code>Whatpm::RDFXML</code></a>
  <dd>An implementation of RDF/XML by which RDF triples can be extracted
  from RDF/XML documents.

<dt><a href="Whatpm/WebIDL.html"><code>Whatpm::WebIDL</code></a></dt>
  <dd>A WebIDL fragment parser.  It parses an IDL fragment, whether conforming
  or not, and constructs a DOM-like object model for further processing.
  Non-conforming (or broken) IDL fragment-like string will be parsed using
  CSS-like error-tolerant parsing rules, e.g. ignoring anything until next
  <code>;</code> character.
</dl>
</div>

<!-- Whatpm::ContentChecker::*, Whatpm::H2H, Whatpm::NanoDOM, and
     Whatpm::XMLParser are intentionally omitted from the list. -->

<div class=section id=howtouse>
<h3>How to use modules</h3>

<p><a href="#modules">Modules</a> listed above, which are included in
the Whatpm package, can be used by directly <code>use</code>ing or
<code>require</code>ing these modules and then invoking their native
interface.  For more information on those native interfaces, see the
document of those modules and the source code of them.

<p>In addition, some of functionality provided by those modules can be
accessed via standardized DOM interfaces implemented by modules
included in the <a
href="http://suika.fam.cx/www/manakai-core/doc/web/">manakai-core</a>
package.  See the document of the module <a
href="http://suika.fam.cx/www/manakai-core/lib/Message/DOM/DOMImplementation.html"><code>Message::DOM::DOMImplementation</code></a>
for the way to access to the DOM interfaces.

<p>The table below summarizes the relationship between Whatpm modules
and DOM methods/attributes implemented by manakai-core modules:

<table>
<thead>
<tr><th scope=col>Whatpm module
<th scope=col>DOM methods/attributes

<tbody>

<tr><td><code>Whatpm::CSS::Cascade</code>
<td><code>get_computed_style</code> (<code>ViewCSS</code>),
<code>current_style</code> (<code>ElementCSS</code>)

<tr><td><code>Whatpm::CSS::Parser</code>
<td rowspan=2><code>CSSStyleDeclaration</code>'s attributes and methods,
<code>css_text</code> (CSSOM interfaces)

<tr><td><code>Whatpm::CSS::Serializer</code>

<tr><td rowspan=2><code>Whatpm::CSS::SelectorsParser</code>
<td><code>query_selector</code>, <code>query_selector_all</code>
(<code>DocumentSelector</code>, <code>ElementSelector</code>)
<tr><td rowspan=2><code>selector_text</code> (<code>CSSStyleRule</code>)

<tr><td><code>Whatpm::CSS::SelectorsSerializer</code>

<tr><td><code>Whatpm::HTML</code>
<td rowspan=4><code>inner_html</code> (<code>HTMLDocument</code>,
<code>Element</code>)

<tr><td><code>Whatpm::HTML::Serializer</code>

<tr><td><code>Whatpm::XML::Parser</code>

<tr><td><code>Whatpm::XMLSerializer</code>

</table>

</div>

</div>

<div class=section id=documents>
<h2>Documents</h2>

<p>For the description of functionalities provided by each module, see
<abbr>pod</abbr> documentation of the module.  HTML version of
<abbr>pod</abbr> documentations are linked from the <a
href="#modules">list of modules above</a>.

<p>In addition, there are additional documents for some topics:
<dl>

<dt><a href="http://suika.fam.cx/gate/2007/html/standards">Standards
supported by WebHACC</a>

  <dd>List and description of Web standards supported by the WebHACC
conformance checker.  Although it is a documentation for the WebHACC,
it is also applicable to Whatpm in general (note that WebHACC is an
interactive user interface for the conformance checking feature
provided by Whatpm).

<dt><a href="http://suika.fam.cx/gate/2005/sw/Whatpm%20Error%20Types">List of error types</a></dt>
<!-- @@ TODO: Need to update the link - the document above is out of date -->
  <dd>Description of errors to be notified to callback functions by Whatpm
  modules.</dd>

<dt><a href="Whatpm/CSS/selectors-object">Selectors object</a></dt>
  <dd>Description of data structure for Selectors, as implemented by
  <a href="Whatpm/CSS/SelectorsParser.html"><code>Whatpm::CSS::SelectorsParser</code></a>
  (as output), and
  <a href="Whatpm/CSS/SelectorsSerializer.html"><code>Whatpm::CSS::SelectorsSerializer</code></a>
  (as input)<!--, and
  <a href="http://suika.fam.cx/www/manakai-core/lib/Message/DOM/SelectorsAPI.html"><code>Message::DOM::SelectorsAPI</code></a>-->.</dd>

<dt id=doc-user-data-names><a href="http://suika.fam.cx/gate/2005/sw/manakai/Predefined%20User%20Data%20Names">List of predefined user data names</a></dt>
  <dd>List of user data names defined by Whatpm modules.</dd>

<dt id=doc-handles><a href="Whatpm/Charset/handles">Handle objects</a>
  <dd>Description of character or byte stream input handle interfaces.
</dl>

<p>Following specifications define Whatpm-specific formats and extensions:
<dl id=spec>
<dt id=spec-ssft><a href="http://suika.fam.cx/www/markup/selectors/ssft/ssft"><abbr title="Selectors Serialization Format for Testing">SSFT</abbr>
Specification</a></dt>
  <dd>The specification for the serialization format used for
  testing Selectors-related modules.</dd>

<dt><a href="http://suika.fam.cx/gate/2005/sw/manakai/CSS%20Extensions">manakai's
CSS extensions</a>
  <dd>The specification for <code>-manakai-<var>*</var></code> properties
  and property values implemented by CSS-related modules.
<dt id=spec-manakai-selectors><a href="http://suika.fam.cx/gate/2005/sw/manakai/Selectors%20Extensions">manakai's
Selectors extensions</a>
  <dd>The specification for <code>:-manakai-<var>*</var></code>
  pseudo-classes implemented by Selectors-related modules.</dd>
</dl>
</div>

<div class="section" id="demo">
<h2>Demo</h2>

<ul id=demo-html-parser>
<li id=demo-html-parser-nanodom><a href="http://suika.fam.cx/gate/2007/html/parser-interface">HTML5 parser 
and checker demo</a>
(<a href="http://suika.fam.cx/gate/cvs/*checkout*/webroot/gate/2007/html/parser.cgi">source</a>, 
with <a href="Whatpm/NanoDOM.html">a lightweight non-conforming 
DOM implementation</a>)</li>
<li id=demo-html-parser-manakai><a href="http://suika.fam.cx/gate/2007/html/parser-manakai-interface">HTML5
parser and checker demo, with manakai's DOM implementation</a>
(<a href="http://suika.fam.cx/gate/cvs/*checkout*/webroot/gate/2007/html/parser-manakai.cgi">source</a>)</li>
<li id=demo-html-table><a href="http://suika.fam.cx/gate/2007/html/table-interface">HTML5 table
structure visualization demo</a>
(<a href="http://suika.fam.cx/gate/cvs/*checkout*/webroot/gate/2007/html/table.cgi">source</a>)</li>

<li id=demo-css-parser><a href="http://suika.fam.cx/gate/2007/css/parser-interface">CSS
tokenizer, parser, and computed style computation demo</a>
(<a href="http://suika.fam.cx/gate/cvs/*checkout*/webroot/gate/2007/css/parser.cgi">source</a>)</li>

<li id=demo-langtag><a href="http://suika.fam.cx/gate/2007/langtag/langtag-demo-interface">Language
tag parsing and conformance checking demo</a>
(<a href="http://suika.fam.cx/gate/cvs/*checkout*/webroot/gate/2007/langtag/langtag-demo.cgi">source</a>)
</ul>
</div>

<div class=section id=applications>
<h2>Applications</h2>

<ul>

<li id=app-webhacc><a
href="http://suika.fam.cx/gate/2007/html/cc/"><abbr>WebHACC</abbr>
(Web hypertext application conformance checker)</a> (See also <a
href="http://suika.fam.cx/gate/2007/html/cc-about"><cite>about
WebHACC</cite></a>).

<li><a href="http://suika.fam.cx/www/webidl2tests/readme">wttjs</a>, a
WebIDL ECMAScript binding test suite generator.

</ul>

<p>See also <a
href="http://suika.fam.cx/www/manakai-core/doc/web/#applications">a
list of applications using modules in the manakai-core package</a>;
some of them indirectly use Whatpm modules from DOM interfaces
provided by <a
href="http://suika.fam.cx/www/manakai-core/doc/web/">manakai-core</a>.

</div>

<div class="section" id="dependency">
<h2>Dependency</h2>

<dl>
<dt id=dependency-perl>Perl 5.8 or later</dt>
  <dd>It is recommended to use newer stable release of Perl 5.8 (or
  later).</dd>
  <dd id=dependency-encode>Some modules require <code>Encode</code>
  modules, which are part of standard Perl distribution.</dd>
<dt id=dependency-manakai-core>Modules from
<a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a></dt>
  <dd>
    <dl>
<dt id=dependency-error><a href="http://search.cpan.org/author/SHLOMIF/Error-0.17009/lib/Error.pm"><code>Error</code></a></dt>
  <dd>Module <code>Whatpm::HTML</code> requires <code>Error</code>,
  which is bundled in
  <a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a>.</dd>
<dt><code>Message::IMT::InternetMediaType</code></dt>
  <dd>Module <code>Whatpm::IMTChecker</code> depends on
  <code>Message::IMT::InternetMediaType</code>, which is part of 
  <a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a>.</dd>
<dt><code>Message::URI::URIReference</code></dt>
  <dd>Modules <code>Whatpm::URIChecker</code> and 
  <code>Whatpm::CacheManifest</code> depend on
  <a href="http://suika.fam.cx/www/manakai-core/lib/Message/URI/URIReference.html"><code>Message::URI::URIReference</code></a>,
  which is part of 
  <a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a>.</dd>
  <dt><code>Message::Charset::Info</code></dt>
    <dd>Module <code>Whatpm::ContentChecker</code> depends on
    <a href="http://suika.fam.cx/www/manakai-core/lib/Message/Charset/Info.html"><code>Message::Charset::Info</code></a>,
    which is part of
    <a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a>.</dd>
<dt><code>Message::DOM::DOMImplementation</code>
  <dd>Module <code>Whatpm::URIChecker</code> depends on
  <code>Message::DOM::DOMImplementation</code>,
    which is part of
    <a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a>.
<dt><code>Message::DOM::DOMImplementation</code> and related modules</dt>
  <dd><em>Testing</em> for module <code>Whatpm::ContentChecker</code>
  depends on <code>Message::DOM::DOMImplementation</code> and related modules
  in <a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a>.
  They are not required for any practical use of those modules.
    </dl>
  </dd>
<dt><a href="http://suika.fam.cx/www/manakai-charlib/readme">manakai 
charlib</a></dt>
  <dd>Module <code>Whatpm::Charset::DecodeHandle</code> depends on
  modules in <a href="http://suika.fam.cx/www/manakai-charlib/readme">manakai
  charlib</a> for decoding of <em>Japanese character encodings</em>.
  See the documentation for 
  <a href="http://suika.fam.cx/www/manakai-charlib/readme">manakai
  charlib</a> for more information.</dd>
<dt><a href="http://www.python.org/">Python</a>, Perl 
<a href="http://search.cpan.org/~neilw/Inline-Python-0.22/"><code>Inline::Python</code></a>
module, and <a href="http://chardet.feedparser.org/">Universal Encoding
Detector</a></dt>
  <dd>For the module <code>Whatpm::Charset::UniversalCharDet</code> being
  meaningful, these softwares are required on the system.  See the
  <a href="Whatpm/Charset/UniversalCharDet.html#dependency">documentation</a>
  for more information.</dd>
<dt><a href="http://search.cpan.org/~makamaka/JSON-1.14/"><code>JSON</code></a></dt>
  <dd><em>Testing</em> for modules <code>Whatpm::HTML</code> and
  <code>Whatpm::CSS::Tokenizer</code>
  depends on <a href="http://search.cpan.org/~makamaka/JSON-1.14/"><code>JSON</code> and related modules</a>.
  They are not required for any practical use of those modules.
</dl>
</div>

<div class="section" id="download">
<h2>Distribution</h2>

<p>The development version of Whatpm may be found in the 
<a href="http://suika.fam.cx/gate/cvs/markup/html/whatpm/">CVS 
repository</a>.</p>

<p><a href="http://suika.fam.cx/gate/cvs/markup/html/whatpm/whatpm.tar.gz?tarball=1">The
latest developmenet version of the Whatpm</a> is also available as a
tarball.

</div>

<div class="section" id="todo">
<h2>TO DO</h2>

<ul>
  <li>Bug fix (Test results: 
    <a href="t/content-type-result"><code>Whatpm::ContentType</code></a>,
    <a href="t/tokenizer-result">HTML tokenization</a>,
    <a href="t/tree-construction-result">HTML tree construction</a>,
    <a href="t/content-checker-result"><code>Whatpm::ContentChecker</code></a>).</li>
  <li>Merge with the <a href="http://suika.fam.cx/www/2006/manakai/">manakai-core</a>
      code tree.
  <li>Charset detection.</li>
  <li>Validation for <code>meta</code>.</li>
  <li>Validation for media queries (level 3), IRIs (against URI schemes),
    and so on.</li>
  <li>Documentations are missing for some features.</li>
  <li>In addition, each module has its own TO DO items.
    (Search for <q>## TODO</q> and <q>## ISSUE</q> in each module.)</li>
</ul>
</div>

<div class=section id=acknowledgments>
<h2>Acknowledgments</h2>

<p>Thanks to the <a href="http://code.google.com/p/html5lib/">html5lib</a> 
team for their
<a href="http://html5lib.googlecode.com/svn/trunk/testdata/">HTML5 
parser test data</a>.</p>
</div>

<div class="section" id="author">
<h2>Author</h2>

<p><a href="http://suika.fam.cx/~wakaba/who?" rel="author">Wakaba</a>.</p>
</div>

<div class="section" id="license">
<h2>License</h2>

<p>Copyright 2007$B!>(B2008 Wakaba 
<code class="mail">&lt;<a href="mailto:w@suika.fam.cx"
    rel="author">w@suika.fam.cx</a>></code>.</p>

<p>This library is free software; you can redistribute it and/or modify 
it under the same terms as Perl itself.</p>
</div>

</body>
</html>