Comega

Cω – usually written Comega or Cw where the Greek letter is inconvenient – was an experimental extension of C# built at Microsoft Research and first released as a public compiler preview in 2004. It was not a product, and Microsoft said so plainly: the project pages stated there were no plans for commercial development or Visual Studio integration. What Comega was instead is one of the most consequential research languages of its decade, because two of its ideas escaped the lab. The data half became LINQ. The concurrency half became the Joins library, and the ancestor of a generation of join-pattern implementations in other languages.

The language attacked two problems at once, along what its designers called the data dimension and the control dimension. In the data dimension, Comega aimed at what everyone in 2004 called the impedance mismatch: an object-oriented program that needs to touch a relational database and an XML document ends up with three incompatible type systems, two of them hiding inside strings. In the control dimension, it aimed at the fact that writing correct asynchronous code in C# 1.x meant hand-rolling locks, monitors, and callback plumbing.

History & Origins

Two research projects, one language

Comega is the merger of two separate efforts inside Microsoft Research.

The concurrency work came first, from Microsoft Research Cambridge, where Nick Benton, Luca Cardelli, and Cédric Fournet built Polyphonic C#. They presented it as “Modern Concurrency Abstractions for C#” at ECOOP 2002 in Málaga, with an expanded treatment in ACM Transactions on Programming Languages and Systems in September 2004. Polyphonic C# took the join calculus – a process calculus Fournet had helped develop, and the basis of the JoCaml language – and asked what its synchronization primitive would look like as an object-oriented language feature. The answer was the chord.

The data work came from Erik Meijer, Wolfram Schulte, and Gavin Bierman, and circulated under the codenames X# and then Xen before settling on Comega. The Xen papers of 2003 – “Programming with Rectangles, Triangles, and Circles” at the XML 2003 conference, and “Unifying Tables, Objects and Documents” – argued that the object, relational, and semi-structured data models could be given a single type-theoretic treatment inside the .NET Common Type System, rather than bolted together with mapping layers.

The Microsoft Research project page for Comega gives an establishment date of April 8, 2004, and the two lines of work shipped as one language: Xen’s data model with Polyphonic C#’s chords folded in. The compiler preview followed later that year, distributed through the Microsoft Download Center as Comega1_1.0.4220.0.msi.

The preview and what happened next

The preview compiler was exactly that – a research release for people who wanted to try the ideas, shipped with sample programs – reportedly including a typed rendering of the W3C XQuery use cases – to show off the query and type machinery. It targeted the .NET Framework on Windows; the Download Center listing has since been refreshed and currently names Windows 7, 8, and 10 as supported systems for the installer, though the software itself dates from the mid-2000s and Microsoft does not document it as tested on those releases.

There was never a Comega 2.0. Instead, the ideas were harvested. Microsoft announced the LINQ Project at the Professional Developers Conference in September 2005, with Meijer among its designers, and LINQ shipped in production with C# 3.0, the .NET Framework 3.5, and Visual Studio 2008 in November 2007. Meijer told the whole story – the ambition, the compromises, and what it takes to get a research idea into a shipping language – in “Confessions of a Used Programming Language Salesman” at OOPSLA 2007.

Design Philosophy

Comega’s guiding conviction was that the type system should extend to cover the data a program actually manipulates, rather than stopping at the boundary of the CLR heap. Three commitments follow from it:

• Queries belong in the language. The advantage the project claimed for its SQL operators was not brevity but timing: a query written as language syntax is checked by the compiler, whereas a query embedded in a string is checked, if at all, by the database at runtime.
• XML is a type, not a document format. Sequences, optionality, choice, and document order – the things XML schemas express – were given direct counterparts in the type system, so that XML-shaped data could be manipulated with ordinary member access.
• Concurrency should be declarative. Rather than telling threads what to do, a Comega class declares which combinations of messages it is prepared to handle, and the runtime does the queuing, matching, and dispatch.

A fourth commitment is implicit and worth stating: Comega remained C#. Every construct was an extension to a language its intended users already knew, which is precisely why the ideas were portable to the shipping product later.

Key Features

Chords and asynchronous methods

Comega splits methods into synchronous ones, which behave as in C#, and asynchronous ones, which have no return value and return control to the caller immediately – so they behave less like calls and more like messages sent.

The novelty is that a body is attached not to a single method but to a set of methods, a construction called a chord. The body runs only when every method in the chord’s header has been invoked:

1
2
3
4
5
6
7

public class Buffer {
    public async Put(string s);

    public string Get() & Put(string s) {
        return s;
    }
}

That is a complete, thread-safe one-place buffer. Put returns instantly and its argument is queued. A call to Get blocks until some Put is available to pair with, at which point the chord fires and the queued string is returned. There are no locks, no condition variables, and no Monitor.Wait in sight – the synchronization is the declaration.

The semantics are precise about the awkward cases: if no chord is enabled, the invocation is queued; if several are enabled, one is selected non-deterministically; and a chord whose header contains only asynchronous methods runs its body on a new thread, since nobody is waiting for a result.

Stream types

A trailing * denotes a stream – an ordered, homogeneous, possibly lazy sequence. Streams flatten rather than nest, which is what makes them behave like XML content models instead of like nested collections:

1
2
3
4
5

public string* LoTR() {
    yield return "The Fellowship of the Ring";
    yield return "The Two Towers";
    yield return "The Return of the King";
}

C# 2.0’s iterators, which shipped with Visual Studio 2005, use a closely related yield return generator style, though the two were developed in the same period and C# iterators are not documented as derived from Comega.

Generalized member access – “the power is in the dot”

The slogan from the ECOOP 2005 paper captures the central move. A member access applied to a stream is applied to every element, and the results are collected back into a stream. So LoTR().Length is not an error; it is a stream of three integers.

On top of that, Comega borrowed XPath’s navigation axes as operators: .* selects all fields of a value, ... descends transitively through a structure, and a type-qualified form filters that descent by type. This is how an object graph gets queried the way an XML document does, without a separate query API and without losing static types.

Anonymous structs and content classes

Because XML has document order, unnamed content, and repeated element names, Comega’s tuple-like anonymous structs allow fields that have no name and fields that share one:

1
2
3
4
5

struct{ int; string; string; DateTime date; string; } x =
    new {47, "Hello World", "Dare Obasanjo", date=DateTime.Now, "This is my first story"};

Console.WriteLine(x[1]);          // positional access
DateTime pubDate = x.date;        // access by name

Accessing a name shared by several fields yields a stream of them. Content classes then compose these into schema-shaped declarations, with * for repetition and ? for optionality – the same vocabulary a DTD or XML Schema uses:

1
2
3
4
5
6
7
8

public class Books {
    struct {
        string title;
        string author;
        string publisher;
        string? onloan;
    }* Book;
}

XML literals

XML could be written directly as an expression, with braces escaping back into Comega code:

1
2
3
4

NewsItem news = <NewsItem title="Hello World" author="Dare Obasanjo">
                  <date>{DateTime.Now}</date>
                  <body>I am the first post of the New Year.</body>
                </NewsItem>;

This feature did not survive into C#, but it did ship: Visual Basic 9.0 included XML literals and axis properties in Visual Studio 2008.

SQL-style query operators

Query syntax was part of the grammar, with select, from, where, order by, group by, and the usual aggregates:

1
2

results = select Title, Artist from CDs where Style == CDStyle.HipHop;
rows    = select ContactName, Phone from DB.Customers order by ContactName desc;

The projection’s type is inferred, the field names are checked against the source, and a typo is a compile error rather than a runtime exception from the database driver. Anyone who has written LINQ will recognize the shape immediately – the most visible difference is clause order, since C# 3.0 moved from to the front – a change its designers have explained in part as letting editors offer completion on the range variable before the projection is written.

Evolution

Comega itself barely evolved; it was released, studied, written about, and closed. Its evolution happened by transplant, and the transplants went in three directions.

The concurrency transplant is the most instructive. Claudio Russo’s Joins library, released in 2006 and presented at PADL 2007, showed that once C# 2.0 had generics, join patterns no longer needed a custom compiler – they could be a class library. And because the library builds patterns at runtime, it can express synchronizations that Comega’s compile-time chords could not, since the number and shape of the patterns need not be fixed in the source text. The research language proved the idea was worth having; the library found a cheaper way to have it.

Current Relevance

Comega is dormant. The compiler preview is still downloadable from Microsoft, and the project pages and papers remain online, but there has been no release since the mid-2000s and no successor bearing the name. Nobody should start a project in it.

Its relevance is entirely as ancestry, and that ancestry is unusually easy to trace because the people involved wrote it down. If you use LINQ, you are using the surviving form of the Comega data model. If you use async/await, you are downstream of a broader Microsoft Research effort on asynchrony in .NET to which the Polyphonic C# and Comega work belongs – though await derives from the Task-based model rather than from chords directly. If you have used a join-pattern library in Scala, .NET, or elsewhere, you are using an idea that Polyphonic C# and Comega were among the first to give a mainstream object-oriented syntax.

For anyone studying language design, Comega is also a case study in how a research language succeeds. It never gained a production user base, never shipped as a product, and never reached version 2 – and by the standard that matters, it worked completely.

Why It Matters

Comega settled an argument that had been running for years about where data access belongs. The prevailing answer in 2004 was “in a library, with the query in a string.” Comega’s answer was that a query is a program, so the compiler should read it. Once LINQ shipped, that answer became conventional wisdom across the industry, and query syntax integrated into general-purpose languages spread well beyond .NET.

It matters a second time for showing that a process calculus can become an ergonomic language feature. The join calculus is a piece of theory; a chord is something a working programmer can read in ten seconds and use correctly. That translation – from calculus to syntax to library – is the whole job of a research language, and Comega did it twice in one language.

Sources

• Comega project, Microsoft Research
• Comega: Introduction, Microsoft Research
• Comega compiler preview, Microsoft Download Center
• “The Essence of Data Access in Comega”, Microsoft Research
• “The Joins Concurrency Library”, Microsoft Research
• Polyphonic C#, Nick Benton
• “Introducing Comega”, XML.com
• Cω, Wikipedia