Only about three weeks after Google launched App Engine, a scalable application hosting platform as preview release to a limited number of developers, a first third party service has popped up, covering features which are not (or just not yet ?) part or Google App Engine. The service we are most interested in here is their Comet daemon. It has several limitations, no pricing has been announced yet, but it’s a start! Currently the following browsers and transport techniques are supported (but not yet fully tested):

• Firefox 2 (XHR streaming)
• Safari/Webkit 3.1 (XHR streaming)
• Opera 9.2 (server sent events)
• Internet Explorer 7 (Activex(”htmlfile”))
• Internet Explorer 6 (Activex(”htmlfile”))
• Internet Explorer 5.5 (Activex(”htmlfile”))
• Konqueror (iframe)

The client-side Javascript is based on orbited.js from the Orbited project

Since the introductory article about Erlang and Comet, a new open source project by Torbjörn Törnkvist is giving some additional choice: domerl. As the name suggests, its focus is on serverside triggered DOM manipulation. The Comet implementation is currently rather simple, just XHR-long-polling, no Bayeux and no bells and whistles, but less is sometimes more! The source is in a git repository, contains some examples and can be cloned as follow:

git clone git://www.tornkvist.org/domerl

The project dependencies are yaws (Erlang web server) and jQuery.

While looking for a JavaScript application that was easy to “Cometify” I discovered sketch, a Dojo example app recently committed by Liu Cougar. It fully implements the idea of pipelining the drawing process based on sequential commands for adding, modifying or deleting objects. It even implements an UndoStack, and is therefore perfectly suited for adding Comet-based collaborative drawing capabilities. But I didn’t want to start from scratch when adding that, so I took a chat application (which probably represents the “hello world” app of Comet) and wrapped it around the drawing app. At the basic level, the two application types are similar. With chat the sent objects are words, while with collaborative drawing the objects are drawing commands.

Very little additional code was necessary to combine the chat app and the drawing app into a collaborative example app. I added a single line of code to the drawing app to let it publish its drawing commands using Comet. Another line was added to the chat app for sending incoming drawing commands to the drawing application (any additional code only served to make things prettier, more readable, or easier to extend).

The application has been tested on ErlyComet, but there are no server-side dependencies, so in theory it should run equally well on any Bayeux-compliant Comet server. The source code is available on the ErlyComet website. And there is also a short screencast.

What is missing to make such an example application suitable for real life tasks? A lot, of course: the whole server-side logic, for one. And deciding (automatically or based on human interaction) which incoming changes should be applied to the master document (like conflict resolution in version control systems, just in quasi-real time). But as this example app demonstrates, once you have a Comet infrastructure in place there are applications which can be “Cometified” with little effort and huge results.

In this article I will explore the possibilities and limits of Erlang for Comet applications. The most popular frameworks and tools related to Comet will be introduced and a practical example of an existing Erlang Comet implementation will be presented.

Erlang is a virtual-machine based, functional programming language developed by Ericsson. While other functional languages like Haskell or OCaml are popular mainly in the academic world, Erlang has been used commercially for nearly 20 years now in massive distributed fault-tolerant telecoms systems. In 1998 Erlang became open source software. For a number of reasons, Erlang is particularly well suited for Comet:

Concurrency, distribution and scalability

Erlang is designed from the ground up for massive parallelism. Instead of native threads, it uses lightweight processes (also called microthreads), which use little memory overhead and can communicate with each other through message passing, even if the processes are located on different hosts. In a typical Erlang Comet application, each connection is represented by such a lightweight process.

Performance

The response to a Comet request often needs to be sent to multiple clients, and upgrading a non-Comet web application to Comet requires significantly more computing power. Erlang has a highly optimized byte code compiler and is also able to generate platform specific native code. On multicore servers, nearly linear speedups are achieved.

Fault tolerance

If things go wrong, and sooner or later they will go wrong, Erlang provides a built-in monitoring and heartbeat mechanism which supervises and restarts the application if necessary.

Full application stack in Erlang

The web server, Comet server, database, and maintenance scripts can all be written in one single language.

Possible problems and limits

• Because Erlang is a functional language, it has a steep learning curve, especially for developers with experience only in object oriented programming.
• While Erlang is good at solving the trick parts of distributed Comet applications, the situation looks different for standard tasks and tools related to web development, such as MVC frameworks, template engines and integration with any kind of web services. While the other languages popular for web development have lots of mature frameworks which solve all these specific web problems, Erlang hasn’t been used yet widely in this field, only few web tools and frameworks are available and often the wheel needs to be partially reinvented.
• If you have legacy code you depend on, the beauty of running the whole application stack in one single programming language is gone. While interfacing Erlang with other languages such as Java or C is possible and often necessary, it introduces a lot of dependencies and if not done very carefully, it reduces fault tolerance and scalability.

Erlang based web servers and frameworks

Currently the following Erlang based web servers and frameworks are available and have been used for Comet style applications:

Yaws

Yaws is the most popular Erlang web server. In active development since 2002. Many contributors, lots of add-ons and large code base, and therefore somehow comparable to Apache, although according to a benchmarks by Joe Armstrong (he is one of the principal authors of Erlang), yaws easily manages 80,000 simultaneous connections while Apache on the same hardware dies after 4,000 connections. There is also ErlyWeb, a MVC web framework tailored to yaws and six times faster than RubyOnRails, according to a benchmark done by Yariv Sadan, the author of the framework. If you are considering adapting yaws for Comet, Chris Double’s article about yaws IFRAME based HTTP push might be a helpful resource. Yaws currently has a limitation for long-polling or callback-polling: long-lived connections have a hard coded timeout after 30 seconds of inactivity. It is easy to apply a patch to fix the issue, but it is a maintenance nightmare to keep a separate source tree and apply the patch whenever the official yaws gets upgraded.

MochiWeb

Since November 2007, MochiWeb is a googlecode.com open source project. Very clean and beautiful designed code, easy to extend and perfectly suited for implementing a Comet server. MochiWeb is not a web server itself, it’s just a HTTP a framework, so you have to build a HTTP server from scratch, but don’t worry, MochiWeb lets you implement a simple web server with just a few lines of code.

ErlyComet

Probably the first Comet implementation based on MochiWeb is ErlyComet. (Disclaimer: the author of this article is also principal author of ErlyComet). Long-polling, callback-polling and a demo application with a server side counter, asynchronous RPC and a simple chat have been implemented, and the screencast shows how easy it is to set up the server and run the demos. If you start one ErlyComet server then you have one Erlang virtual machine running, independent of how many connections the server is handling. If you start on the same host another ErlyComet server listening on a different port you can test cross domain Comet and simulate a cluster, which is handy during development and functional testing. The state of each connection is shared between cluster nodes with a distributed in-memory database.

Conclusion

Erlang makes it easy to solve Comet scalability and distribution problems. Frameworks and open source Comet implementations exist, so one can jump start straight into Erlang Comet development. Just be aware that it might take some time to get comfortable with the functional nature of the Erlang language!