Assembler (DEC VAX)

VAX assembly language is what happens when a company decides that the instruction set itself should be the high-level language. Where the PDP-11 before it was admired for being small and regular, the VAX was admired for being complete: sixteen 32-bit registers, a dozen addressing modes that compose with almost any operand of almost any instruction, six integer sizes, four floating-point formats, character strings, packed decimal strings and variable-length bit fields as first-class machine data types, and single instructions that evaluate a polynomial, insert an element into a doubly linked queue, or format a number into a currency field with floating dollar sign and suppressed leading zeros. It is the canonical CISC architecture, and its assembly language — DEC’s VAX MACRO, also called MACRO-32 — is the language most of the VMS operating system was written in.

History and Origins

The machine

VAX design began around 1975, once DEC had accepted that the PDP-11’s 16-bit address space was the line’s fatal constraint. Gordon Bell had already called the shortage of address bits “the biggest (and most common) mistake that can be made in a computer design”, and the name of the answer said what it was: Virtual Address eXtension. Responsibility for the architecture rested with William D. Strecker, Bell’s former doctoral student at Carnegie Mellon.

The first machine, the VAX-11/780, was introduced on 25 October 1977 at DEC’s annual shareholder meeting. The -11 in the name was not nostalgia: early VAX processors implemented a PDP-11 compatibility mode, so existing user-mode PDP-11 programs that did not need the extended address space ran unchanged. Later VAX implementations offloaded compatibility mode, and some of the less-used CISC instructions, into software emulation by the operating system. DEC dropped the -11 branding as PDP-11 compatibility stopped mattering.

The assembler

VAX/VMS V1.0 shipped in August 1978, and the assembler shipped with it. The first printing of the VAX-11 MACRO Language Reference Manual is dated August 1978; it was revised in February 1979 and again in March 1980 for VAX/VMS V2. By VMS Version 4 the language manual and the instruction-set manual had been merged into a single VAX MACRO and Instruction Set Reference Manual — the April 1988 V5.0 printing (AA-LA89A-TE) records that it supersedes a Version 4.0 edition of that merged title — and the product name had lost its hyphenated 11. That manual is still in print: VMS Software, Inc. published a fresh edition in April 2019, documented against OpenVMS VAX V7.3.

The assembler’s ancestry is explicit. The VSI manual opens Chapter 8 by saying that the VAX architecture “represents a significant extension of the PDP-11 family architecture”, sharing byte addressing, similar I/O and interrupt structures and identical data formats, and that although the instruction set is not strictly compatible it “can be mastered easily by a PDP-11 programmer.” The syntax, directives, macro language and lexical substitution operators of VAX MACRO all came over from MACRO-11.

One thing did not come over, and it catches PDP-11 programmers every time: MACRO-11 defaults to octal, VAX MACRO defaults to decimal. The radix control operators ^B, ^D, ^O and ^X select binary, decimal, octal and hexadecimal explicitly, and a bare 100 on a VAX means one hundred.

Two VAX assembly languages

There are really two dialects. DEC’s MACRO-32 is the one bundled with VMS, the one VMS itself is written in, and the one this page mostly describes. Unix on the VAX went its own way: UNIX/32V (June 1979) and the Berkeley distributions that followed used the Unix as, with lowercase mnemonics, $ for immediates and % for registers, and none of MACRO’s directive vocabulary. Same instruction set, different notation — exactly the split that had already happened on the PDP-11 between MACRO-11 and Unix as.

The Language

Registers

Sixteen 32-bit general registers, R0 through R15, four of which carry a hardware role:

The PDP-11’s best idea survives: the PC is a general register, so immediate, absolute and relative addressing are not special modes but ordinary register modes applied to R15.

Data types

This is where the VAX departs from everything before it. The architecture defines, as machine data types with instructions to match:

COBOL’s decimal arithmetic, in other words, is in the hardware.

Addressing modes

The VSI manual groups the modes into general register, program counter, index and branch. The eight general register modes are register, register deferred, autoincrement, autoincrement deferred, autodecrement, displacement, displacement deferred and literal:

The PC modes add relative, relative deferred, absolute (@#address), immediate (#literal, I^#literal) and general (G^address, where the linker decides between a PC displacement and an absolute address). Index mode, written base-mode[Rx], composes with almost any other mode and scales the index by the operand’s data type, so TABLE[R3] steps by four for a longword operand and by eight for a quadword. Autoincrement and autodecrement scale the same way — they step by the size of the data the instruction is moving, not by a fixed amount.

Instruction set shape

Appendix D of the VSI reference manual lists 383 opcode mnemonics; about 63 of those belong to the optional vector extension, leaving roughly 320 scalar instructions. They are organised into groups:

• Integer arithmetic and logical — in two-operand and three-operand forms across every integer size, so ADDL2 A,B and ADDL3 A,B,C are both instructions.
• Address — MOVA and PUSHA in each operand size.
• Variable-length bit field — EXTV, INSV, FFS (find first set) and friends.
• Control — conditional branches, plus the loop primitives ACB (add, compare and branch), AOBLEQ/AOBLSS (add one and branch) and SOBGEQ/SOBGTR (subtract one and branch).
• Procedure call — CALLS, CALLG and RET, described below.
• Queue — INSQUE/REMQUE for absolute doubly linked queues and the self-relative INSQHI/INSQTI/REMQHI/REMQTI, which are interlocked and usable across processors.
• Floating point — four formats, with POLY evaluating a polynomial from a degree, an argument and a coefficient table, and EMOD splitting a product into integer and fraction.
• Character string — MOVC3, MOVC5, CMPC, LOCC, SKPC, MATCHC, SPANC, SCANC.
• Cyclic redundancy check — CRC, a single instruction, table-driven.
• Decimal string — packed decimal add, subtract, multiply, divide, compare and convert, plus EDITPC, whose operand is a little pattern-operator program for laying out a formatted number.

Instructions are variable length — one to 56 bytes — because each operand carries its own addressing-mode specifier. This is the design that the RISC movement was, a decade later, reacting against.

Statement format

A VAX MACRO statement has the familiar four fields, with a colon ending a label, a semicolon starting a comment, and DEC’s recommended tab stops at columns 1, 9, 17 and 41:

1

LABEL:  MOVL    #10,R0          ; label, operator, operand, comment

A single colon defines a label locally; a double colon (TEST::) makes it global and linkable. User-defined symbols can be up to 31 characters and may contain _, $ and . — which is why VMS system symbols look like RPB$L_IOVEC and why $-bearing names are conventionally DEC’s. A hyphen as the last non-blank character continues a statement onto the next line.

Directives begin with a dot: .TITLE, .PSECT for program sections with attributes and alignment, .ENTRY for a callable entry point, .BYTE/.WORD/.LONG/.QUAD/.OCTA for data, .BLKx for reserved storage, .ASCII/.ASCIZ/.ASCIC/.ASCID for strings, .MACRO/.ENDM/.MCALL/.MDELETE for the macro facility, and .IF/.IIF/.ENDC for conditional assembly.

The CALL standard, in hardware

The most VAX thing in the whole architecture is the procedure call. A procedure begins with a two-byte entry mask — a bit per register — and CALLS or CALLG reads it and saves exactly those registers, along with PC, FP, AP and the processor status, building a standard call frame. RET unwinds it. The register-mask unary operator ^M writes the mask readably:

1

        .ENTRY  CALC,^M<R2,R3,R7>   ; CALL will preserve R2, R3 and R7

Bits 0, 1, 12 and 13 — R0, R1, AP and FP — are reserved to the call interface, and the assembler rejects an entry mask that sets them. Routines entered with BSB or JSB instead of CALLS have no mask and must not use .ENTRY. The result is that the calling standard is not a convention compilers agree on; it is encoded in the instruction stream, which is also why linear disassembly of VAX code is awkward — there is data embedded in the code.

Code Examples

Hello, world under VMS

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

        .TITLE  HELLO

        .PSECT  DATA, NOEXE, WRT
MSG:    .ASCID  /Hello, world!/         ; string preceded by a descriptor

        .PSECT  CODE, EXE, NOWRT
        .ENTRY  START, ^M<>             ; entry mask: preserve nothing
        PUSHAL  MSG                     ; push address of the descriptor
        CALLS   #1, G^LIB$PUT_OUTPUT    ; RTL routine, one argument
        RET
        .END    START

.ASCID is the detail worth noticing: it emits a string descriptor — a two-byte length, two bytes of descriptor information, and a position-independent four-byte pointer — in front of the text. VMS routines are called with descriptors rather than bare pointers, and the assembler has a directive for building them, because the calling standard reaches that far down.

Three-operand arithmetic

1
2
3

        ADDL3   R1,R2,R3        ; R3 := R1 + R2, leaving R1 and R2 alone
        SUBL2   #10,R0          ; R0 := R0 - 10
        MULL3   #4,R5,R6        ; R6 := R5 * 4

Most double-operand instructions come in both a 2 form that overwrites its destination and a 3 form that does not, in every integer size and floating format.

A counted loop

1
2
3

        MOVL    #100,R0         ; loop 100 times (decimal — this is not the PDP-11)
10$:    ...
        SOBGTR  R0,10$          ; subtract one, branch while greater than zero

10$ is a local label, scoped between ordinary labels, inherited from MACRO-11.

One instruction, one string copy

1
2
3
4

        MOVC3   #80,SOURCE,DEST         ; copy 80 bytes
        MOVC5   #12,NAME,#^A/ /,#20,FIELD
                                        ; copy 12 bytes into a 20-byte field,
                                        ;   space-filling the remainder

MOVC5 is the architecture in miniature: source length, source, fill character, destination length, destination — a strncpy with padding, as a single opcode.

Evolution

The instruction set barely changed; what changed was where it lived. The VAX-11/780 was built from TTL and filled a cabinet roughly four feet by five. Through the 1980s the high end moved to ECL gate arrays — the VAX 8600, the 8800, and finally the VAX 9000 mainframes of October 1989, which proved too complex and expensive and were abandoned not long after introduction. The low end went the other way: the MicroVAX I, then the MicroVAX II of 1985 on the 78032 CPU, then V-11, CVAX, Rigel, Mariah and NVAX.

For an assembly programmer the important consequence was the MicroVAX architecture. It was not yet possible to put the full VAX on a chip, so the packed decimal, character-string and other less-used opcodes were moved out of microcode and emulated in software. The instruction set stayed complete from the programmer’s point of view, but part of it was now, quietly, a trap handler. The same technique later absorbed PDP-11 compatibility mode.

The architecture’s last extension was vector processing, defined by DEC STD 032 (EL-00032-00, dated January 1990) and documented as Chapter 10 of the MACRO manual: vector registers, vector control registers, masked operations, and a set of VV/VS-prefixed instructions with their own synchronization rules.

Then the industry moved. DEC had extrapolated from Moore’s Law that a 32-bit design would remain viable until roughly 1999, and did not foresee how quickly RISC would deliver more performance per dollar. DEC shipped MIPS-based DECstations in 1989 and its own Alpha AXP in 1992. Compaq announced in August 2000 that the remaining VAX models would be discontinued by year end, and OpenVMS VAX V7.3 in June 2001 was the last release for the architecture.

A footnote on benchmarks

The VAX-11/780 became, for about a decade, the unit in which computer speed was quoted, so the numbers attached to it need their context stated. It was initially marketed as a one-MIPS machine — not because it retired a million instructions per second, but because its performance was comparable to an IBM System/360 running at one MIPS, and System/360 implementations were the existing de facto standard. Measured natively — the figure usually quoted is Joel Emer’s, taken on an 11/780 under a time-sharing load — the machine executed roughly 500,000 instructions per second, which drew complaints of marketing exaggeration, and the industry settled on the circular but honest definition of a “VAX MIPS” or VUP (VAX Unit of Performance): one 11/780. A machine rated at 27 VAX MIPS ran the same program about 27 times faster than an 11/780. The BRL-CAD benchmark still uses the 11/780 as its baseline metric.

Current Relevance

VAX MACRO is the rare dead-architecture assembly language that is still a supported, shipping product — because too much of VMS was written in it to do anything else.

• The MACRO-32 compiler. When OpenVMS was ported to Alpha, the team faced the fact that most of the kernel was VAX assembly. Rather than rewrite it, they built a compiler that takes VAX MACRO as a source language and emits native object code, “just as a FORTRAN compiler might”. It was later carried to Itanium and then to x86-64, following OpenVMS V9.2 in July 2022. VSI’s own guidance is blunt about its purpose: it exists for porting, and new development should use mid- and high-level languages.
• It is a compiler, not an assembler, and it shows. VSI’s porting guide warns that the MACRO-32 compiler may move, replicate, remove and interleave code, and that faulting behaviour may differ from the VAX original. Most real modules need entry-point directives added, and non-portable constructs removed, before they compile. The compiler also exposes Alpha features the VAX assembler never had — 32 registers, 64-bit addressing, a subset of Alpha instructions — mapped onto equivalents on Itanium and x86-64.
• Emulation. SIMH simulates VAX models and boots original VMS, Ultrix and BSD distributions; Stromasys CHARON-VAX is the commercial equivalent used to keep production VAX workloads running on modern hardware.
• Modern toolchains. GCC still carries a VAX back end in gcc/config/vax, and NetBSD still ships a vax port — currently a Tier II port, present in the NetBSD 11.0 release — so VAX assembly can be written, assembled and run today without vintage hardware. OpenBSD discontinued its VAX port after the 5.9 release of March 2016.
• Documentation. VSI’s April 2019 editions of the VAX MACRO and Instruction Set Reference Manual and the MACRO Compiler Porting and User’s Guide are freely downloadable, and DEC’s originals back to the August 1978 first printing are archived on Bitsavers.

Why It Matters

The VAX is the high-water mark of the idea that a computer’s instruction set should meet programmers — and compilers — most of the way. DEC’s stated goals for it were maximal PDP-11 compatibility, high bit efficiency through rich data types and addressing modes, a “systematic, elegant instruction set with orthogonality of operators, data types, and addressing modes”, and extensibility for data types not yet invented. On its own terms it succeeded completely. VAX assembly is genuinely pleasant to write; the addressing modes compose without exceptions; the call instructions do what a calling standard needs doing; EDITPC really will format your currency field.

It is also the design that RISC was an argument against, and the argument was won so thoroughly that the VAX is now taught mainly as a contrast case. Variable-length instructions of one to 56 bytes, microcoded operations whose cost the programmer cannot see, and data embedded in the instruction stream all turned out to be expensive to pipeline. DEC’s own successor, Alpha, went the other way entirely — and had to carry a VAX MACRO compiler with it to bring the operating system along.

That compiler is the real legacy. Most assembly languages die with their hardware. VAX MACRO survived onto three subsequent architectures because an operating system’s kernel was written in it, which is a fair measure of both how much code was written in this language and how serious DEC’s customers were about never rewriting it. Nearly half a century after the VAX-11/780 was announced, VAX MACRO source is still being compiled — on x86-64.