In this article I will explore the possibilities and limits of Erlang for Comet applications. The most popular frameworks and tools related to Comet will be introduced and a practical example of an existing Erlang Comet implementation will be presented.

Erlang is a virtual-machine based, functional programming language developed by Ericsson. While other functional languages like Haskell or OCaml are popular mainly in the academic world, Erlang has been used commercially for nearly 20 years now in massive distributed fault-tolerant telecoms systems. In 1998 Erlang became open source software. For a number of reasons, Erlang is particularly well suited for Comet:

Concurrency, distribution and scalability

Erlang is designed from the ground up for massive parallelism. Instead of native threads, it uses lightweight processes (also called microthreads), which use little memory overhead and can communicate with each other through message passing, even if the processes are located on different hosts. In a typical Erlang Comet application, each connection is represented by such a lightweight process.

Performance

The response to a Comet request often needs to be sent to multiple clients, and upgrading a non-Comet web application to Comet requires significantly more computing power. Erlang has a highly optimized byte code compiler and is also able to generate platform specific native code. On multicore servers, nearly linear speedups are achieved.

Fault tolerance

If things go wrong, and sooner or later they will go wrong, Erlang provides a built-in monitoring and heartbeat mechanism which supervises and restarts the application if necessary.

Full application stack in Erlang

The web server, Comet server, database, and maintenance scripts can all be written in one single language.

Possible problems and limits

• Because Erlang is a functional language, it has a steep learning curve, especially for developers with experience only in object oriented programming.
• While Erlang is good at solving the trick parts of distributed Comet applications, the situation looks different for standard tasks and tools related to web development, such as MVC frameworks, template engines and integration with any kind of web services. While the other languages popular for web development have lots of mature frameworks which solve all these specific web problems, Erlang hasn’t been used yet widely in this field, only few web tools and frameworks are available and often the wheel needs to be partially reinvented.
• If you have legacy code you depend on, the beauty of running the whole application stack in one single programming language is gone. While interfacing Erlang with other languages such as Java or C is possible and often necessary, it introduces a lot of dependencies and if not done very carefully, it reduces fault tolerance and scalability.

Erlang based web servers and frameworks

Currently the following Erlang based web servers and frameworks are available and have been used for Comet style applications:

Yaws

Yaws is the most popular Erlang web server. In active development since 2002. Many contributors, lots of add-ons and large code base, and therefore somehow comparable to Apache, although according to a benchmarks by Joe Armstrong (he is one of the principal authors of Erlang), yaws easily manages 80,000 simultaneous connections while Apache on the same hardware dies after 4,000 connections. There is also ErlyWeb, a MVC web framework tailored to yaws and six times faster than RubyOnRails, according to a benchmark done by Yariv Sadan, the author of the framework. If you are considering adapting yaws for Comet, Chris Double’s article about yaws IFRAME based HTTP push might be a helpful resource. Yaws currently has a limitation for long-polling or callback-polling: long-lived connections have a hard coded timeout after 30 seconds of inactivity. It is easy to apply a patch to fix the issue, but it is a maintenance nightmare to keep a separate source tree and apply the patch whenever the official yaws gets upgraded.

MochiWeb

Since November 2007, MochiWeb is a googlecode.com open source project. Very clean and beautiful designed code, easy to extend and perfectly suited for implementing a Comet server. MochiWeb is not a web server itself, it’s just a HTTP a framework, so you have to build a HTTP server from scratch, but don’t worry, MochiWeb lets you implement a simple web server with just a few lines of code.

ErlyComet

Probably the first Comet implementation based on MochiWeb is ErlyComet. (Disclaimer: the author of this article is also principal author of ErlyComet). Long-polling, callback-polling and a demo application with a server side counter, asynchronous RPC and a simple chat have been implemented, and the screencast shows how easy it is to set up the server and run the demos. If you start one ErlyComet server then you have one Erlang virtual machine running, independent of how many connections the server is handling. If you start on the same host another ErlyComet server listening on a different port you can test cross domain Comet and simulate a cluster, which is handy during development and functional testing. The state of each connection is shared between cluster nodes with a distributed in-memory database.

Conclusion

Erlang makes it easy to solve Comet scalability and distribution problems. Frameworks and open source Comet implementations exist, so one can jump start straight into Erlang Comet development. Just be aware that it might take some time to get comfortable with the functional nature of the Erlang language!