mp4h

mp4h (Macro Processor for HTML Documents) is a text preprocessor whose macros look like HTML tags. You define a tag such as <greeting> or <mailto> once, and mp4h replaces every use of it with whatever HTML you specified, computing values, looping and branching along the way. Denis Barbier wrote it in 1999–2000 to replace the Meta-HTML interpreter inside the Website Meta Language (WML), an offline HTML generator. Its README names its two ancestors plainly: Meta-HTML “inspired mp4h syntax whereas source code is derived from” GNU m4.

That makes mp4h a curious hybrid: the front end of one macro language on the engine of another. It never had a large following of its own. It reached people mostly as the hidden second pass of WML, and through WML it helped build sites such as www.debian.org.

History and Origins

WML and Meta-HTML

WML was created by Ralf S. Engelschall as a pipeline of separate tools, each run as a numbered “pass” over a page. In the 1.x series, pass 2 was a cut-down copy of Brian J. Fox’s Meta-HTML. A WML 1.x ChangeLog entry renames its version string from “5.02” to “5.02/wml” to make clear “that this is not a complete or standard Meta-HTML”. Engelschall did not hide his feelings about it; another entry begins “because I really dislike the 3rd-party Meta-HTML sources, I’ve again stripped down the stuff.”

In the introduction to the mp4h manual, Barbier says he had maintained WML since January 1999. He explains why he wrote a replacement instead of patching Meta-HTML:

For licensing reasons, it was hard to hack Meta-HTML and so I decided to write my own macro-processor.

Built on GNU m4

He did not start from scratch. The manual says he picked GNU m4, written by René Seindal, because it was “stable, robust and very well documented”, and that mp4h “is derived from GNU m4 version 1.4n, which is a development version.” The CREDITS file lists Seindal and Fox, the authors of the two parent programs, next to Barbier, and thanks “all authors of GNU m4, who wrote the major part of mp4h sources.”

From experiment to WML 2.0

The first trace of mp4h is in the WML ChangeLog. On 29 November 1999, Barbier replaced the wml_p2_mhc program with wml_p2_mp4h, describing it as “a temporary fix to perform benchmarks and other comparisons.” The ChangeLog does not record any benchmark results.

The oldest entry in mp4h’s own ChangeLog is mp4h 0.7.0, dated 11 February 2000: “First public release.” On 13 February 2000, WML’s ChangeLog records “Replaced Meta-HTML by mp4h for pass 2”. WML 2.0.0 shipped with mp4h on 11 May 2000, and Barbier wrote a migration guide, README.mp4h, for sites whose templates had been written for Meta-HTML.

The year 1999 used on this page is the date of that first commit. Anyone outside the WML development tree would first have seen mp4h in February 2000.

Design Philosophy

The manual sets a deliberately narrow goal:

The mp4h software is not an HTML editor; its unique goal is to provide an easy way to define its own macros inside HTML documents.

Validating or tidying the output was left to other tools; the manual points readers to HTML Tidy. A few principles follow from that goal:

• Look like HTML. Macros are tags, and their arguments are attributes. The README promises that the syntax is “immediately familiar to HTML programmers.”
• Leave unknown tags alone. Ordinary HTML such as <p> or <img> passes through unchanged, so mp4h source is mostly ordinary HTML with a few custom tags mixed in.
• Generate, then stop. mp4h runs once, offline, and writes static output. It is a preprocessor, not a server-side runtime like Meta-HTML or PHP.
• Everything is text. As in m4, values are strings. Numbers are strings that arithmetic tags interpret, and arrays are strings with one element per line.

Key Features

Simple and complex tags

mp4h distinguishes simple tags (<name attrs />) from complex tags that have a body and an end tag. You create both with <define-tag>. Inside a definition, %0, %1 and so on are positional attributes, %# is the number of attributes, %attributes is all of them, and %body is the body of a complex tag.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

<define-tag greeting whitespace=delete>
Hello, <get-var name />!
</define-tag>

<define-tag mailto endtag=required whitespace=delete>
<a href="mailto:%0">%body</a>
</define-tag>

<set-var name="World" />
<p><greeting /></p>
<mailto webmaster@example.org>Write to us</mailto>

Run through mp4h 1.3.1, this produces the following (after a few blank lines left by the definitions):

1
2

<p>Hello, World!</p>
<a href="mailto:[email protected]">Write to us</a>

The whitespace=delete attribute matters more than it looks. In a macro language that works purely on text, every newline inside a definition ends up in the output unless you remove it.

Variables, arrays and control flow

mp4h has variables (<set-var>, <get-var>, <increment>), stacks for saving them (<preserve>, <restore>), and a set of array tags (<array-push>, <array-shift>, <sort> and others). Control flow is also written as tags: <if>, <ifeq>, <when>, <var-case>, <while> and <foreach>.

1
2
3
4
5

;;; Build a navigation list from an array
<set-var pages="Home\nNews\nAbout" />
<ul>
<foreach page pages>  <li><get-var page /></li>
</foreach></ul>

Lines beginning with three semicolons are comments, a feature the manual notes has “no equivalent in HTML”. Recursion works too. This factorial comes from the mp4h manual:

1
2
3
4

<define-tag factorial whitespace=delete>
<ifeq %0 1 1 <multiply %0 "<factorial <substract %0 1 /> />" /> />
</define-tag>
5! = <factorial 5 />

It prints 5! = 120. The arithmetic tag really is spelled substract. Debian’s packaging carries a lintian override for the misspelling because, as its changelog puts it, it “is a command of mp4h and cannot be changed.”

Strings and regular expressions

String tags include <substring>, <upcase>, <subst-in-string> and <printf>. <match> tests regular expressions. From July 2000 (versions 1.0.6 and 1.1.0), regular expressions use Philip Hazel’s PCRE library, so they follow Perl syntax and support the i, m, s and x modifiers.

Controlling the parser

Because mp4h has to coexist with real HTML and, later, XHTML, much of its configuration is about how strictly it parses. The -X flag takes a bit mask that decides, among other things, whether unknown tags are parsed, whether an unmatched end tag closes everything open, and whether simple tags get an XHTML-style trailing slash. By version 1.3.1 the default is 3114. Tag and variable names are case-insensitive by default; entities are case-sensitive.

Text between <@[ and ]@> is read without parsing, an escape hatch for malformed markup that Tobias Oetiker contributed in June 2000.

Safety levels and extensions

The --safety-level option disables risky primitives. Level 1 turns off <execute>, which runs shell commands, and level 2 also turns off file-system tags such as <include> and <directory-contents>. Packages of tag definitions are loaded with <use>. Builds that support it can also load compiled C modules as shared libraries, a feature the ChangeLog says was easy to add because the GNU m4 1.4n development code “already has such support.”

Evolution

The ChangeLog dates individual changes rather than releases, so only 0.7.0 (ChangeLog) and 1.3.1 (README) carry an explicit release date. The 1.3.0 date is the last change in its section, which matches Barbier’s Debian upload that same day. The 1.0.0 date comes from Debian’s 20 June 2000 upload. The 1.0.6, 1.1.0 and 1.2.0 dates are approximate, inferred from the dates of the changes in each section.

The project’s homepage moved more than once. The README, even in the 1.3.1 release, still points to engelschall.com/sw/mp4h/, the address shared with WML. A SourceForge project was registered on 28 September 2001, but according to Debian’s packaging notes its file area stops at 1.2.4. By late 2003 the distribution files were served from mp4h.tuxfamily.org, which Debian’s copyright file cites through a December 2003 Internet Archive capture. The tuxfamily address no longer serves the site, and Debian now lists the SourceForge page as the homepage “to have at least one”. A read-only copy of the sources was pushed to GitHub (mp4h/mp4h) in 2015.

Current Relevance

Upstream development stopped with 1.3.1 in January 2003, which is why mp4h is classed as historical. It has never disappeared, though:

• Debian has packaged it since February 2000. Version 1.3.1-17, uploaded in June 2018 by Axel Beckert (who also appears in mp4h’s CREDITS), is in Debian 10, 11 and 12, and later uploads carry the same upstream code into Debian 13 and the testing branch. Most of the recent packaging work concerns reproducible builds and hardening flags, not new features.
• WML is still maintained by Shlomi Fish, and its repository still contains mp4h as pass 2. The WML README is frank about the toolchain’s age. It calls WML powerful but “quirky” and says it “is not recommended for new projects”.
• Debian’s website still depends on it. The current “Using WML” guide for www.debian.org says mp4h 1.3.0 or later is needed to rebuild the translation templates, so new pages there still go through mp4h.

On this page, mp4h 1.3.1 was built from the Debian source tarball on a current macOS system, and every code example above was run through it. The 2003 C code needed permissive compiler flags (it relies on implicit function declarations that modern C compilers reject) but otherwise compiled and ran.

Why It Matters

mp4h is a small example of a larger pattern in late-1990s web tooling. Before content management systems and template engines were commonplace, many people kept static sites consistent by preprocessing HTML. Navigation bars, headers and link formats lived in macros, and a build step expanded them into static pages. Meta-HTML put HTML-shaped syntax on that idea; m4 had the engine. mp4h combined the two, turning a Unix macro processor into something web authors could read without learning a new notation.

It also shows how much of the early open-source web rested on deep stacks of reused code. A site page built with WML might pass through an HTML-syntax language derived from Meta-HTML, running on code from GNU m4, which itself descends from the Bell Labs m4 of the 1970s. That chain is still running inside the Debian website build more than two decades after mp4h’s last release.