MAGIC/L

MAGIC/L (pronounced “magical”) is a procedural, interactive programming language of the early 1980s that set out to solve a specific problem: Forth’s power came wrapped in reverse Polish notation and a culture of per-programmer dialects that made team projects hard to maintain. MAGIC/L kept what its authors considered Forth’s real advantages - an incremental compiler that let you define a routine and run it immediately, and an environment the programmer could extend without limit - and replaced the rest with the familiar shape of an algebraic language: declared, typed variables; infix expressions with assignment operators; named input arguments and local variables; record structures; and formatted PRINT and INPUT statements.

The language was conceived in 1979 by Arnold Epstein and Jeffrey Morris at the Smithsonian Astrophysical Observatory (SAO) in Cambridge, Massachusetts, where they were maintaining a suite of scientific software written in STOIC, a Forth derivative. Its commercial form, MAGIC/L, was first implemented in 1981 and sold by their company, Loki Engineering, Inc., through at least 1987. It ran on an unusually wide range of hardware for a small vendor’s product - Data General minicomputers, DEC PDP-11s and VAXes, CP/M and MS-DOS micros, UNIX systems, the Macintosh, and Motorola’s VERSAdos - and Loki claimed over 500 installations by 1985. Today it survives mainly in two documents: the January 1984 MAGIC/L User’s Manual, scanned at the Internet Archive, and a 1985 conference paper in the Journal of Forth Application and Research. No implementation is publicly available, and the language is dormant.

History and origins

STOIC at the observatory

MAGIC/L’s lineage runs through STOIC, the Stack Oriented Interactive Compiler that Jonathan Sachs - later the principal author of Lotus 1-2-3 - wrote in the mid-1970s at the MIT/Harvard Biomedical Engineering Center. STOIC was a Forth variant with a built-in assembler, floating-point package, and display editor, and it found a home at SAO, where Roger Hauck ported it to VAX/VMS and distributed it through DECUS.

By 1979 Epstein and Morris had six major software packages written in STOIC. Two of them were interactive programs for visiting scientists working with data from NASA’s Einstein Observatory, the first fully imaging X-ray telescope, launched in November 1978 with SAO’s Riccardo Giacconi as principal investigator: a database holding the satellite’s observing schedule and analysis status, and an image-processing system for the X-ray images. About ten visiting scientists a week would spend two or three days using this software, and, as the authors later wrote, “in order to provide a familiar environment to these visitors, we felt it was necessary to create a forward notation user interface for these programs.”

There was a second pressure. The two of them were becoming overburdened supporting the existing packages and wanted to hand some of that work to programmers who did not know Forth. Both problems pointed the same way: integrate ordinary forward notation into the interactive Forth-style environment itself, rather than bolting a parser onto each application.

MAGIC, 1979-1981

By the end of 1979 the SAO applications had been recoded in MAGIC, a prototype that fully supported forward notation but lacked formatted I/O, argument declarations, and compiler error checking. According to the user’s manual, this original implementation ran on a Data General Eclipse under the AOS operating system. The name came from the reaction the authors kept getting when they demonstrated it - “How did you do that?” - to which they kept answering, “It’s magic!” The acronym expansions offered in the manual (“Mnemonics Are Generally Idiotic Concoctions”; “Multi-purpose Assembler and General-purpose Interactive Compiler”) were, they admit, invented afterwards.

Over the next two years MAGIC was used for process control, hardware debugging, and image processing, and demand for a commercial version led to the founding of Loki Engineering. Its first task was to specify a language that fixed MAGIC’s shortcomings and to produce a properly documented product. That language was MAGIC/L, first implemented on a Data General Nova in 1981.

Loki Engineering and the commercial years

Loki Engineering operated from 55 Wheeler Street in Cambridge, Massachusetts. Its user’s manual, copyright 1983 and issued for Revision 2.58.1 in January 1984, credits Epstein and Morris as authors and thanks a group of “early believers” who commented on the language as it developed. The manual is deliberately implementation-independent: each release also shipped a release notice, an operating-system supplement describing access to system calls, an assembler manual for the interactive assembler on that host processor, and a toolbox manual for utilities such as command-line and definition editors. The manual’s getting-started chapter is deliberately host-neutral - typing MGL “in response to the system prompt” starts the environment and BYE returns to the host - but it acknowledges trademarks for CP/M, DEC, and Data General’s Eclipse, which together suggest the platforms of the early product. Its preface, signed “Arny Epstein” and “Jeff Morris” and dated August 1981, thanks Marian Sonnenfeld Epstein “for opening her home to a fledgling company,” which is consistent with Loki having started as a two-person operation that year. A pair of 8-inch floppy disks labelled Revisions 2.40 and 2.50 has reportedly been offered in an online collector’s auction listing, consistent with the 2.x numbering of the manual.

By the time Epstein and Claire H. Gilliatt presented “The MAGIC/L Programming Language” at the 1985 Rochester Forth Conference, the list of ports had grown considerably: DEC minicomputers under RT-11 and RSX-11M, microcomputers under CP/M-80 and MS-DOS, “various UNIX systems,” the Apple Macintosh, and completed VAX/VMS and VERSAdos implementations. The paper, published in the Journal of Forth Application and Research volume 3, number 2 (pages 9-21), states that MAGIC/L “has been in use since 1981” and “is being used at over 500 installations worldwide” in typesetting, image processing, robotics, process control, and authoring. That figure is Loki’s own and has not been independently confirmed.

The last dated trace of the company found for this article is an entry in the Computers and People Computer Directory of 15 October 1987, which lists “Loki Engineering Inc., 55 Wheeler St., Cambridge, MA 02138,” A. Epstein as president, and a single product line: “Interactive computer language ‘Magic/L’.” No later release, review, or corporate record has been located, and the language’s status is recorded as dormant.

Design philosophy

The 1985 paper is unusually explicit about its premises, and they are worth quoting because they define the language.

Forth’s strengths are interactivity and extensibility. Immediate testing of subroutines cuts development time compared with languages that need separate compile, link, and run phases; and the ability to add new words - including new classes of words via <BUILDS ... DOES> - lets the environment be tailored to the application. MAGIC/L keeps both.

Forth’s weaknesses are social, not technical. Because most words in a Forth program are user-defined and there is no formal syntax, “each programmer writes in a unique ‘dialect’ which can be difficult for others to understand. Even the author can have trouble recreating his own environment a day later.” The authors call Forth a “write-only” language in team settings: “a large programming team can easily become a Babel of programming styles.”

Memory is no longer the constraint that shaped Forth. Forth’s 1-kilobyte compiler does no syntax analysis because code is “predigested by the programmer” in postfix form. By the early 1980s, the authors argue, at least 64 KB of RAM was generally available, so “one can usually afford to have a somewhat larger compiler in order to improve code legibility.” MAGIC/L’s compiler is about 8 KB.

Consistency beats infinite tailoring. “For some people, it may be faster to write code with RPN. However, we have decided that it is more important to encourage consistency and readability than to allow an infinite number of ways to tailor the language to a particular application. This optimizes long-range productivity and improves efficiency on a team programming level.”

Portability is a language feature. MAGIC/L defines a generic set of system services for terminal and file I/O, so that application source is portable “even at the I/O call level,” and it runs under widely used operating systems rather than replacing them, so it can coexist with tools like Lotus 1-2-3 and WordStar and with other languages. The manual sums the whole thing up as “not strictly a compiler or an interpreter. We call it an ‘incremental compiler.’”

Key features

Data types and variables

Variables are declared with a type name followed by a list of names, with optional array sizes given by any constant expression evaluable at declaration time. Subscripts run from 0 to size − 1.

1
2

INTEGER X, Y, Z (10)
REAL ANGLE, SIDE, DATA (5*4)

The intrinsic types are CHAR (8-bit unsigned), INTEGER (16-bit signed), LONG (32-bit signed), REAL (32-bit single-precision floating point), and ADDRESS, which is the same size as INTEGER on 16-bit implementations and LONG on 32-bit ones - a deliberate device for keeping source compatible across the two machine classes. Strings are unsubscripted CHAR arrays. PARAMETERs are named 16-bit integer constants. Everything is case-insensitive; multiple statements on a line are separated by ;;.

Expressions and assignment

Expressions use conventional infix operators, and a variable may appear on both sides of an assignment:

1
2

X := Y * 5
Y := Y + X

The compiler decides from context whether the address or the value of Y is meant. Internally, each variable carries two code pointers ahead of its parameter field - one action routine pushes the address, the other the value - and the compiler selects the appropriate one, which is how the second line above compiles to just five words of threaded code and how the language dispenses with Forth’s ubiquitous @. One operator vocabulary (:=, +=, -=, INCREMENT, DECREMENT, SET, CLEAR, + - * /, == <> > < >= <=, NOT AND OR XOR) serves every type; the compiler emits the type-specific operation and reports an error for mixed-mode expressions or for an operator undefined on a type. Relational tests yield an INTEGER of −1 for true and 0 for false.

Record structures

Complex data - “one of the most dialectic aspects of Forth” - gets a uniform treatment. A RECORD ... ENDRECORD block defines a new type from intrinsic types and other records; variables and arrays of that type are declared exactly like intrinsic ones; and elements are accessed with the colon operator or, for a run of references, with WITH:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

RECORD DATE_REC
    INTEGER MONTH
    INTEGER DAY
    INTEGER YEAR
ENDRECORD

DATE_REC BIRTHDAY, HOLIDAYS (10)

BIRTHDAY:MONTH := 6
HOLIDAYS (2) :DAY := 4

WITH HOLIDAYS (8)
    MONTH := 12
    DAY := 25
    YEAR := 85

Control structures

All conditionals evaluate an INTEGER test expression (non-zero is true). As in Forth, loops maintain internal counters - I, J, K count up and I', J', K' count down - that can be used without being declared. The full set is:

The manual adds CASE-ENDIF, unsigned UDO/UITER loops, and EXIT. Any control structure can be typed and executed interactively at the mgl> prompt, not only inside a definition - a property the authors single out as “particularly useful … for both writing and debugging.”

Routines, arguments, and locals

A definition has up to five parts - DEFINE with an optional routine type, input-argument declarations, a LOCAL block, the body, and END - of which only DEFINE and END are required. Arguments are declared in calling-sequence order using the same syntax as record fields; locals live in a frame allocated on entry, so recursive routines get fresh locals. The paper’s example returns the sum of an integer array:

1
2
3
4
5
6
7
8

DEFINE VSUM INTEGER
    INTEGER VEC (0)
    INTEGER NVAL
    CLEAR VSUM
    ITER NVAL
        VSUM += VEC (I)
    LOOP
END

Code is compiled incrementally as it is entered; END saves the compiled routine under its name, after which it can be run from the keyboard or called from later definitions. “The ability to name input arguments and to declare local variables,” the paper notes, “is a major improvement over Forth in the area of program readability and maintainability.”

Defining new classes of words

Forth’s <BUILDS ... DOES> survives as the routine type ACTION: the “does” code becomes an ACTION routine, and BUILD or the shorthand MAKE creates instances that carry their own parameter lists:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

DEFINE SCALEFUNC INTEGER ACTION
    INTEGER VAL
    INTEGER PARAM(0)
    SCALEFUNC := VAL * PARAM(0)
END

MAKE "TIMES3" SCALEFUNC 3
MAKE "TIMES5" SCALEFUNC 5
MAKE "TIMES9" SCALEFUNC 9

mgl> PRINT TIMES3(5), TIMES9(4)
15 36

Built-in command interfaces

The feature the authors were proudest of addressed the SAO problem directly. Most languages leave the programmer to write an input parser and expression evaluator for an end-user command interface; MAGIC/L builds it in. A routine of type COMMAND is invoked without parentheses, accepts a variable number of arguments (the count is in CMDCNT, read with NXTARG), and - because arguments are compiled expressions - lets the user type arithmetic:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

DEFINE ZOOM COMMAND
    INTEGER NEWZOOM
    IF (CMDCNT == 0)
        PRINT "The Current Zoom is", CURZOOM
    ELSE
        CURZOOM := NEWZOOM
    ENDIF
END

mgl> ZOOM 5
mgl> ZOOM
The Current Zoom is 5
mgl> ZOOM CURZOOM * 4 + 2
mgl> ZOOM
The Current Zoom is 22

PARSED routines are the same idea with string arguments. Unlike Forth IMMEDIATE words, COMMANDs can be called from inside other definitions, so advanced users can build macro commands from existing ones.

Formatted I/O and the portable I/O package

PRINT takes any number of arguments of mixed types, with directives for radix (#R), integer and unsigned formatting (#I, #UI), fixed or scientific reals (#F), string justification (#S), pad character (#P), columnar tabbing (#T), and suppressing the trailing carriage return (#Z); output can go to the console, a file, or memory. INPUT converts keyboard or file text into a list of typed variables with default or user-written error handling.

1
2
3
4

CHAR ISTR(40)
INTEGER X
PRINT "Enter Name and Number: ", #Z
INPUT ISTR, X

Beneath these sit four pairs of system-independent routines - OPEN/CLOSE, RDS/WRS for binary I/O, RDTXT/WRTXT for text (a line always ends in a newline, 0A hex, whatever the host convention), and SPOS/GPOS for positioning. Implementing them on block-only systems such as RT-11 and CP/M required building a complete buffered stream I/O layer inside MAGIC/L, which the paper offers as an example of the language absorbing complexity so the programmer need not.

Library, vocabularies, and the assembler

About 100 library routines - arithmetic, trigonometric, string, mixed-mode and conversion, bit manipulation, block moves, memory access - are named “so that their functionality is clear,” and unneeded ones can be deleted to recover space. Vocabulary branches provide namespace control; EXT loads source files; a running system can be saved as an executable disk image; and each implementation includes an inline interactive assembler for its host processor.

Performance, as the authors measured it

MAGIC/L’s designers were candid about costs, and their figures are the only ones that exist. All of the following come from the 1985 paper’s comparison of MAGIC/L with “its grandparent Forth,” so they describe threaded-code implementations of the two languages on the same class of hardware rather than any independent benchmark:

• Code size: compiled MAGIC/L is “about as compact as compiled Forth code.” The overhead of LOCAL variable frames is roughly offset by no longer needing @ to fetch values. The root system, including full operator, library, and I/O support, is about 8 KB; the symbol table is kept separate from code (headerless) and can be selectively trimmed at runtime.
• Execution speed: since both languages spend their time in the inner interpreter’s “next” cycle, they are comparable. Calling a routine that has a local-variable frame costs about four extra next-cycles, which for a typical multi-line routine “tends to be under 10%.”
• Compile speed: this is where MAGIC/L paid. The compiler needs about 8 KB - “much more than Forth” - and compiles more slowly, largely because the symbol-table lookup was still a linear search; a more efficient search was planned for the next revision.

The 1984 manual’s efficiency appendix also asserts that MAGIC/L code “will run as fast as code generated by many compilers and is much faster than BASIC code” (presumably the interpreted BASICs of the era), but gives no measurements, hardware, or workload, so that claim should be read as vendor marketing rather than data.

Evolution

MAGIC/L’s development can be traced through three stages, each documented by a primary source:

The paper’s mention of a planned next revision with faster symbol lookup, and the manual’s Rev X.XX banner, imply that revisions continued past 2.58.1, but no later manual or release notice has surfaced. Whether the language was ever extended with, for example, a 68000-specific Macintosh toolbox interface or a UNIX process model is not documented.

Current relevance

MAGIC/L is dormant. No compiler, interpreter, source listing, or disk image is publicly available; the only known physical media are the 8-inch Revision 2.40 and 2.50 disks that reportedly surfaced in a collector’s listing, whose host system is not identified in the listing and which would require a period machine to run even if imaged. There is no user community, no emulator package, and no Wikipedia article. What does exist is unusually good for a language this obscure: the complete 1984 user’s manual (roughly 200 pages, OCR-searchable at the Internet Archive) and the 1985 JFAR paper (freely available from the JFAR archive maintained by MicroProcessor Engineering and mirrored at Forth, Inc.). Between them they specify the language fully enough that a re-implementation would be feasible - a project no one appears to have attempted.

Its ideas, however, did not disappear. The pattern MAGIC/L embodies - a Forth-class incremental compiler underneath a conventional infix surface syntax, with a built-in command-line evaluator for end users - reappeared repeatedly in instrument-control and laboratory software of the 1980s and 1990s, and the specific complaint it answered, that Forth is a “write-only” language for teams, is still the standard critique of Forth today. No later language documents MAGIC/L as an influence, so this page claims none; the resemblance is one of shared problem, not established descent.

Why it matters

MAGIC/L is a small but clear data point in a debate that ran through the Forth community for its whole history: whether Forth’s postfix notation and dialect-per-programmer culture were essential to what made it valuable, or accidental features of the 1-kilobyte compilers of 1970. Epstein and Morris answered from the second camp, and unusually, they answered with a shipping product rather than a proposal. They kept the interactivity, the extensibility, the DOES>-style defining words, and the tiny threaded-code footprint; they gave up postfix, implicit stack juggling, and unlimited syntactic freedom; and they added typed declarations, named arguments, locals, records, and portable I/O. The result was pitched explicitly at project management - at the moment a Forth program acquires a second programmer.

It also documents a lineage that is otherwise easy to miss: from Chuck Moore’s Forth to Jonathan Sachs’s STOIC at the Biomedical Engineering Center, to the Smithsonian Astrophysical Observatory’s Einstein Observatory software, to a two-person Cambridge company selling an interactive language on 8-inch floppies. The X-ray images that visiting astronomers analyzed at SAO in 1979 were manipulated through a language interface that would become MAGIC/L - a reminder that a good deal of scientific and industrial computing in the minicomputer era ran on languages that never appeared in a textbook, were known only to their customers, and survive today because someone kept the manual.

Further reading

• Loki Engineering, Inc. MAGIC/L User’s Manual, Revision 2.58.1, January 1984 (Arnold Epstein and Jeffrey D. Morris). Scanned copy at the Internet Archive: archive.org/details/magic-l-users-manual.
• Arnold Epstein and Claire H. Gilliatt. “The MAGIC/L Programming Language.” Proceedings of the 1985 Rochester Forth Conference, published in The Journal of Forth Application and Research, vol. 3, no. 2, pp. 9-21. PDF in the JFAR archive at vfxforth.com/flag/jfar/vol3/no2/article1.pdf; also indexed by the ACM Digital Library.
• Computers and People Computer Directory, vol. 36, no. 10B, 15 October 1987 - directory entry for Loki Engineering Inc. (scan at Bitsavers).
• On the ancestor language: the Wikipedia article on STOIC, and the STOICAL project (a STOIC adapted to Linux) for a runnable relative.