Using maven to build your project is fantastic for managing your dependencies and avoiding having dependencies (and their dependencies) checked into your own svn. The only fly in the ointment, is projects that don’t publish maven artifacts, and the Ajax Dojo Toolkit and Cometd have been one of these. Until now that is !

The new repositories are at

• http://download.dojotoolkit.org/maven2-snapshot/
• http://download.dojotoolkit.org/maven2/

I have blogged at length about the maven incantations needed to use these.

Currently only 1.2-SNAPSHOTs of Dojo and 1.0-SNAPSHOTs of parts of Cometd are published. Releases (including dojo 1.1.1) will be forthcoming once we have some feedback.

It is an oft repeated misnomer that Comet is about long-lived HTTP connections, but this is simply not the case. Comet uses long-held HTTP requests rather than long-lived connections!

Almost every HTTP client out there strives to use persistent connections (either HTTP/1.0 keep-alive or HTTP/1.1 by default) over which many HTTP requests are sent. With today’s web, you more often than not have to take action to stop a browser from having long lived connections. So web-1.0 apps, Ajax apps and Comet apps will all use long-lived HTTP connections!

With Comet, the server holds onto a HTTP request while waiting for events. With streaming Comet, a partial HTTP response is sent as each event occurs and the request remains held. With long-polling Comet, a complete HTTP response is sent as each event occurs and the browser sends a new HTTP request to get subsequent events.

Wikipedia’s Cometd page currently contains the following incorrect definitions:

Comet

In web development, Comet is a neologism coined in 2006 to describe application architecture in which a long-lived HTTP connection allows a web server to push data to a client asynchronously and with no need for the client to explicitly request it.

Streaming

In an application using streaming Comet, the browser opens a single persistent connection to the server for all Comet events, which is handled incrementally on the browser side; each time the server sends a new event, the browser interprets it, but neither side closes the connection.

Long polling

As the name suggests, long polling requires a new request for each event (or set of events). The browser makes an Ajax-style request to the server, which is kept open until the server has new data to send to the browser. After sending such an event, the server closes the connection, and the browser immediately opens a new one.

And before we all go renew the battle of Cometd at wikipedia, we should realize that the reason that wikipedia has these incorrect definitions, is that almost every article, blog, or paper I can find on Comet makes the same mistake. They all say connection when they should say request! So it is simply not a matter of correcting wikipedia, as that would violate their original research rule. We need to build a body of diverse articles that use correct language when describing Comet.

I think I’ll start a virtual swearing jar - into which a virtual dollar will be put every time a Comet Daily contributor uses the phrase “long lived connection” instead of “long held request”. This virtual fund can eventually be realized as the bar tab at some future Comet Daily get together.

Eventually I’d like to see the definitions on wikipedia and else where be something along the lines of:

Comet

In web development, Comet is a neologism coined in 2006 to describe application architecture in which a long-held HTTP requests allows a web server to push data to a client asynchronously.

Streaming

In an application using streaming Comet, the browser sends a single HTTP request to the server for all Comet events. The server delivers events to the client by sending them in partial HTTP responses, which are handled incrementally on the browser side. Each time the server sends a new event, the browser interprets it, and the server keeps the HTTP response incomplete for subsequent events.

Long polling

As the name suggests, long polling requires the client to poll the server for an event (or set of events). The browser makes a long poll in an Ajax-style request to the server. Unlike traditional polling, with long polling the server holds the request until there is new data to send to the browser, which is sent in a complete HTTP response. In order to receive subsequent events, the browser sends a new long poll in a new HTTP request.

Language is very important for understanding. We all must strive to put an end to the long-lived connection misnomer.

In recent weeks, there has been an ongoing battle at wikipedia over the definition and details of the Comet (programming) page, with the end result being that a 6000 word page on the history, techniques and implementations of Comet, has been cut down to three sentences that barely define what Comet is.

Much of this dispute has been due to culture clash and strange wikipedia policies that result in the contributions from those with direct experience in Comet having their contributions devalued due to fears of advertising and conflicts of interest, with the result that the text that is favoured is written by people without Comet experience?

However, at the root of the issue is that there is no really good definition of what Comet is! I think some of the contributors to the wikipedia page were hoping that it would become that good definition, but there is a wikipedia policy that prohibits any original thought, so any clarification of the term cannot be organized there.

“Ask Alex Russell for the definition, as he created the term!” I hear you say. Unfortunately, as far as wikipedia is concerned, we can’t look at the creation of the term by Alex, as he is apparently a self publishing non-notable tech blogger (in wikipedia terms), and thus the original blog is not a sufficient citation. In some part I agree with wikipedia on this, namely that we should look to current usage rather than historical intent when seeking a definition of what is and is not a Comet web application.

Thus I’d like to pose a couple of questions in this musing about the definition of Comet:

Is Comet Ajax, or just Ajax-like?

This question goes somewhat to what is the definition of Ajax. But if the use of plugins such as flash or applets is sufficient to render an in-browser application non-Ajax, is it enough to render it also non-Comet? In Alex’s original blog, he only describes Comet as Ajax-like. Does the display technology of the application have to be Ajax? If Cometd was used to deliver events to a Java applet, would that application be Comet? Is it simply sufficient for the communication to be delivered to an Ajax capable environment (i.e. JavaScript callback) for it to be Comet?

Is Comet HTTP only?

The original definition, and many Comet articles use the phrase: “long-lived HTTP connections” to describe Cometd transports. Can a Comet application use a raw TCP/IP socket from flash and still be called Comet? Can a Comet transport used the FTP protocol from the browser and still be called Comet?

Does a browser have to be involved

If Bayeux is used to communicate over HTTP to a non browser client (e.g. a Java swing application), is that Comet?

Does any of this matter?

Ajax itself is probably not the most well defined term. Is a good definition of what Comet is needed (other than to permit a more complete page to be created on wikipedia)?

My opinion

In my not so humble opinion, I believe that a good definition is Comet could be: “An Ajax web application that uses open standards based protocols to transport asynchronous events from the server to the client with minimal latency”. I like this definition because declaring Comet as a subset of Ajax means that any browser developments that become an acceptable part of Ajax, become an acceptable part of Comet. I also believe open standards are important for Ajax, and thus should be important for Comet. Using flash plugins would render a transport to be Comet-like, but not Comet.

To answer my questions above:

• Comet is a subset of Ajax
• Comet is not HTTP only, but it does need to use an open standard. FTP and BOSH would qualify, but something over a raw TCP/IP socket would not, unless socket access became part of the browser.
• Comet is to the browser. A swing client using Bayeux would be using a Comet transport, but would not itself be a Comet app
• Not really, but I’d like to hear some feedback and alternative thoughts anyway.

Cometd Bayeux is a publish/subscribe messaging system, and this has drawn some criticism that PubSub is not the most flexible paradigm for an Ajax/Comet transport. This article attempts to address that by showing the details of the Bayeux PubSub mechanism and the extensions that makes it easy and efficient to transport many styles of messaging.

PubSub

The classic Bayeux/Cometd example is chat, which is a perfect match for publish subscribe. Clients may join a chat room with a subscribe:

dojox.cometd.subscribe("/chat/room",myObject,myHandler);

and anybody can publish chat with:

dojox.cometd.publish("/chat/room",{from: "gregw", chat: "Hello cruel world"});

Request/Response

This communication paradigm is probably the most frequent paradigm used and the most familiar to developers. However it is not always the best choice for Ajax Comet applications, nor does it have a simple and efficient mapping to pure PubSub.

I say that request/response is not always the best choice, because developers can use it to create very chatty UIs that send many separate message to the server: getThis, getThat, doSomething, setAValue, etc. Batching can help reduce some of the chatter, but adds complexity. Often it is better to design your application communications as exchange of state message rather than RPC style request/response. However, even state message often needs to be done in a request/response style and pure PubSub still does not well support this.

The issue is that a client sending a request to the server does not want that request and/or published to all other clients that are subscribe to the channel. The pure PubSub response to this approach is to create per client channels with random names only know to those clients: the client subscribes for a response on the secret channel name and then publishes the request. While this works well enough, it is very wasteful of server resources and data structures and queues need to be created on the server side for each service for each channel, resulting in huge numbers of channels.

Thus Bayeux supports channels that are not published to non-server clients. The /meta/subscribe and /meta/unsubscribe channels are examples of such non publishing channels and messages sent by a client to these channels are never delivered to other non-server clients, even if they attempted to subscribe to those channels. All /meta/* channels are like this, but these are reserved for the use of the protocol. Bayeux also defines the /service/* channel space to be a non-publishing space. Messages (requests) published on /service/channels are only delivered to server side handlers and/or clients. Non-server clients may subscribe to these channels, but will only receive messages (responses) delivered explicitly to the client them by the server.

For example, if a chat room wanted the ability to show some chat history as soon as a member joined, then this could be implemented by the chat client subscribing to /service/chat and publishing a request message with some query (chat room name, time range of message wanted, max chats to send etc). e.g.:

dojox.cometd.subscribe("/service/chat",myObject,myHandler);
dojox.cometd.publish("/service/chat",{room: "/chat/room", max: 100});

A server side client would be subscribed to the /service/chat client to handle all such requests, and would handle them by running the query and preparing a response method that they would deliver to the requesting client (see dojox.cometd.Client.deliver(…) ).

YUCK! you say? Well I agree, there is still some issues and ugliness:

• The subscribe is persistent and needs either to be unsubscribed or able to handle multiple responses
• The responses for multiple request must be associated with the correct response and potentially a different callback
• It doesn’t look like request/response! which is not a trivial matter.

But these are issues of the client API and not issues of the protocol. Bayeux service channels are capable of efficiently transporting the request and response and Bayeux has the ID mechanism required to correlate responses to requests. Client libraries should be able to present this mechanism as request/response and manage the short term callback and request/response ID matching required. This may be done either as a standardized API, or in an API style best suited to a particular framework.

This API facade over the services mechanism has already been done to some extend on the server side with the BayeuxService class (although that is not yet part of the “standard” dojox.cometd java server API). A similar packaging of service channels on the clients could well present this mechanism as pure request/response.

Request/Multiple-Response

The same /service mechanism can be used to support request/multi-response, where a single request message may result in multiple response messages sent to the client. The way the protocol transports the messages does not change, with the main difference is that the callback should not be unsubscribed after the first response. There may be more that can be done in the client API to support this mode.

Private Messages

Often with PubSub, developers feel the need to create a channel per user in order to deliver private messages to a client. For example, if a trading system wants to notify a user of completed trades, the temptation is to create a channel like /trades/a_user_id and each user will subscribe to their own channel. This approach works, but is not the most resource sensible way of solving this issue and requires security code to prevent unauthorized clients subscribing to other users channels.

Instead, the services channel space can be used together with the deliver API on the server. All clients subscribe to /service/trades and the server delivers and individual users trades to them with code like:

bayeux.getClient(clientId).deliver(serviceClient,"/service/trade",trades,null);

Analogy with IP

There have been calls for Bayeux to be simplified and/or replaced with a layered protocol suit. However, I would maintain that Bayeux is already layered and represents a minimal protocol suite analogous to the IP (as in TCP/IP or UDP/IP):

Conclusion

The basic communication paradigm of Bayeux is flexible and efficient. pub/sub can be viewed as the default routing algorithm for bayeux, but by use of the Client.deliver API and service channel, arbitrary message forwarding strategies can be developed. The current client APIs are very much written to be PubSub, so there is almost certainly more work that can be done in those APIs to support the commonly used communication paradigms.

More importantly, the cometd needs to do a much better job of documenting and advocating the features that it already has, so that the conversation about how it can be improved can be conducted from a knowledgeable basis. I believe the specification document needs to be refactored to breakout specific transports and to make it clear that PubSub is just the default routing mechanism. I have also discussed some extended routing features with Alex Russell that I think can be added to a cleaned up specification.

To follow-up on Dylan’s post about Comet games in more detail, the folks at chess.com have launched live.chess.com running on the Jetty implementation of Cometd with the Dojo client and an ext.js UI. This site does exactly what it says on the tin! This has only been available for a few days, yet already you will typically find several hundred players online watching or participating in 50 to 100 games. Most of the games are run under tight time limits, so the action comes fast and furious (Ok, it’s not world of warcraft, but fast and furious for chess!). The user interface allows you to watch/participate in multiple games at once and to customize the appearance of the the boards and pieces.

In part 3 of this series, Greg responds to Michael’s criticisms of Bayeux from part 2.

Part 1: Greg Wilkins explains the need for Bayeux
Part 2: Michael Carter criticizes the current state of Bayeux
Part 3: Greg Wilkins responds to Michael Carter
Andrew Betts’ thoughts (from a related article)
Part 4: Michael Carter responds to Greg Wilkins
Part 5: Kris Zyp’s thoughts
Part 6: Alex Russell responds to Michael Carter
Part 7: Michael Carter responds to Alex Russell

Welcome to Comet Celebrity knock down!

I see the criticism contained in Michael’s article as mostly indicating that the Cometd project has failed to communicate the features and intention of Bayeux. This is mostly due to our poor project documentation (insert pathetic excuse about project infrastructure, time etc.)

So I think the best rebuttal I can give is to correct a few misconceptions that Michael has.

Broadcast - Michael suggests that broadcast use-cases do not need the “cost” of Bayeux and that “The URL itself is enough of a subscription request for this information”. But broadcast does need a very important feature of Bayeux: transport negotiation! Without a standard for that, your broadcast application will be tied to a single transport.

A single request is needed to handshake most transports, and then another request will subscribe to whatever broadcast channels you are interested in (e.g. earthquakes and storms, but not floods and fires). If you want to specify that in a URL, then use a transport that allows URL encoding of subscribe messages.

Peer to Peer - Michael correctly identifies that there is unacceptable cost in having user specific channels for user specific messages. That is why Bayeux includes /meta/* and /service/* channels. These channels are not broadcast channels, and when a client publishes a message on /service/takeItem, it is only delivered to the server and not to other clients subscribed to /service/takeItem. The server that handles the /service/takeItem channel can deliver message(s) back to a single client via the same channel so that the client can be informed of the success or failure of their take item (and what the item is etc.) in Michael’s example.

The use of /service/* channels greatly removes the need for most per client channels and dramatically reduces server resource requirements. It is a failing of the Cometd project that we have not well documented this feature.

I also disagree with Michael’s premise that the “cost” of any unused features in Bayeux will stop developers from implementing it. We don’t expect that many developers will implement Bayeux, unless they have some new idea, language, transport etc. The vast majority of developers will simply use one of the growing number of existing implementations. And how many of us really use all the features of the frameworks that we have at hand? The important question here is what the cost of unused features is. For Bayeux, the cost is a single request to negotiate the transport before the business of subscription commences. Even a subscribe request can return valuable payload. The cost of Bayeux is small.

I’m not going to comment much on Michael’s proposed layered solution other than to say that I’m not opposed to layering. The Jabber/BOSH protocol appears to have done a very good job of specifying the use of 2 HTTP connections to provide full duplex communications. I’d love to see BOSH as a transport option for Bayeux. Even though there is some overlap, BOSH is definitely pitched more at a specific transport while Bayeux is pitched more at a specific semantic.

Finally, take Michael’s prediction that Bayeux may not be long lived and that better protocols will emerge. That may be so, but if Bayeux is successful in weaning developers off XHR (and wrappers), of getting them thinking of messaging (instead of request/response), and of separating transport concerns from the application, then Bayeux will have done its job. I dare say that to transition to some Bayeux-killer will be almost trivial once XHR is dead and transport is no longer specified in magic headers and parameters buried deep within an application’s business logic.

Bayeux makes no claim to being the end of the road on Comet evolution. Just to being a big step in the approximate right direction, and to having low entry barriers so that it does not need to affect your choice of language, server, framework etc.

We’re starting a new feature comparing opposing perspectives. This time the subject is the Bayeux protocol and the general question of a standard for Comet.

Part 1: Greg Wilkins explains the need for Bayeux
Part 2: Michael Carter criticizes the current state of Bayeux
Part 3: Greg Wilkins responds to Michael Carter
Andrew Betts’ thoughts (from a related article)
Part 4: Michael Carter responds to Greg Wilkins
Part 5: Kris Zyp’s thoughts
Part 6: Alex Russell responds to Michael Carter
Part 7: Michael Carter responds to Alex Russell

What the Problem Is

On one level, Comet is very easy: pick up your nearest XHR-for-dummies book, send a request to the server and wait for something to happen before sending a response.

Yet when you get into the details of Comet, it has some very difficult resource management issues that need to be dealt with. Most of these issues stem from the 2 TCP/IP connection limit imposed by RFC2616 for HTTP/1.1 for a HTTP client communicating with a given HTTP server.

The limit is applied to the entire client (i.e. browser) and not just to a given frame, tab or window. A single Comet connection consumes both of the available TCP/IP connections: one for the long poll or streaming response and another for any outgoing calls. Thus in an entire browser, only 1 Comet instance can be connected to a given host.

But there are many use-cases where it is easy for multiple Comet connections to be needed:

• Two frameworks on the same page independently implementing Comet
• A page with multiple frames, each with its own Comet instance
• Multiple tabs or windows pointing at the same web application
• Multiple Comet applications that have been mashed up together or combined by a portlet server.

Even without considering Comet, the semantics of XHR as used by Ajax also has some related resource limitations. If you have 10 widgets on a page that all need to update their state from the server, these requests have to be squeezed over the shared 2 connections. If the widgets use XHR wrappers to do this, then the user will have to wait for the latency of multiple round trips to the server as each of the 10 requests and responses queue up behind the two available connections.

Some people advocate the use of host name aliases to circumvent the 2 connection limit. But this is a false hope. 4 or 8 connections will soon be insufficient as we mashup more widgets onto a single page. But more importantly, the 2 connection limit is there for reason: to conserve expensive server resources. To double the connections from a single client is to halve the number of simultaneous users a server can handle.

But the good news is that dealing with limited resources is not an unfamiliar problem for computer science. After all, most computers have only 1 or 2 CPUs, and we manage to share those between many disparate processes. In order to share limited resources we use common infrastructure, libraries, and/or operation systems.

But the problem in the Ajax Comet world is that the common infrastructure is the browser and its JavaScript API, which offers no support at all for sharing connections. In fact, its security model impedes sharing of resources between frames/tabs/windows, and its standard API gives no control over important aspects such as pipelining requests over connections. Worse still, there is no reasonable prospect of browsers with such support being commonly deployed within the next 3 years.

So in the Ajax/Comet world, our infrastructure, libraries and “operating system” come in the form of JavaScript frameworks that we download. But these frameworks are not common, and widgets, frames, pages, tabs and windows are free to use what ever framework(s) that they wish.

The only way we can share the scarce connections is to have a common infrastructure, but we will never agree on one and only one Ajax framework, and the frameworks themselves can’t even agree on a getElementById wrapper, let alone limiting and controlling access to XHR, script elements and anything else that may consume connections.

How Bayeux Addresses the Problem

So what does all this have to do with Bayeux? Isn’t it just another framework trying to be the one and only one? Well NO IT’S NOT.

Bayeux is not a framework, it is a specification.

And the Bayeux specification has two great strengths that will help us all get past the limited resources problem:

1. Bayeux specifies a publish subscribe messaging semantic, which is a vastly better abstraction layer than XHR or request/response for achieving flexibility, scalability and resource sharing.
2. Bayeux specifies that transport is negotiated between a client and server and that the details of that transport are a separate concern to the use of the Bayeux messaging layer.

Bayeux recognizes that we all have to agree on something if we are going to share the limited resources of the browser. But it also recognizes that we are not all going to agree on long polling or streaming, or on stopping innovation on new transports. So transport is negotiable in Bayeux.

Bayeux recognizes that we are not all going to agree on the same language on the server or the client. Bayeux has already been implemented in JavaScript, Java, Perl, Python, Flash, and C++, and more are coming.

Bayeux recognizes that we will not all agree on an API or even how callback and subscriptions etc. will be abstracted within a client and/or server. Thus multiple implementations and framework integrations exist and more are on their way.

Bayeux recognizes that not everybody is a JSON fan, and while the content of the Bayeux messages is specified in JSON, there is no requirement that they are sent as JSON, and implementations of XML (e.g. BOSH), binary, or telepathic transports would be welcome.

Bayeux recognizes that it has not solved all the problems. For example, the solutions for multi-tab are still very rudimentary and there is no support for inter-framework message bridges. However, Bayeux provides an environment in which innovation can strive for better solutions to these problems without requiring applications to be re-written.

So yes, I know the default transports may not be optimal for all use-cases. I know the Dojo client can be considered a bit fat, and I know the fast Java servlet only scales well on Jetty. I know we need to support inter-framework message bridges.

These are concerns that I share—but they are concerns that Bayeux separates from the application developer by a semantic divide. These are concerns that framework and server developers can innovate, differentiate and compete about.

Finally, the initial Bayeux implementations are provided by the Cometd project at the Dojo Foundation. While these implementations are not hugely mature, they have been developed by a team with real world experience of developing deployable and scalable web solutions. The Cometd implementation of Bayeux contains that experience as well as innovation and differentiation that allows it to be deployed reliably on today’s browsers, scale to enterprise levels and compete well with other Comet options out there on its own merits.

Techniques for replicating session data include distributing: to all nodes in a cluster; to a subset of the nodes in the cluster; to a database; to a cluster server; etc. etc. To this range of solutions, Cometd/Bayeux now adds the possibility of using the client machine to store a replicated session.

I have blogged about a prototype BayeuxSessionManager for Jetty.

While it sounds strange, if you think about it, the client machine is the perfect place to store a replicated copy of the server session data. It is a machine that is always in communication with the correct web server, and if the client machine crashes, the session is by definition over so you don’t need to timeout the replicated data. Client machines scale with the number of users, so if you have more users then you have more machines available to store replicated session.

After some recent optimizations, the Dojo Cometd implementation of the Bayeux protocol running on the Jetty web server can now handle up to 20,000 simultaneous users per server while maintaining sub-second latency.

Previous benchmarking efforts had been done on low end hardware with early versions of Cometd, yet had showed promising results, with 10,000 simultaneous users being possible but with latency blowing out to 7 seconds or more.

This latest testing has taken place on more realistic hardware with the more mature implementation of Cometd that is provided by Dojo and Jetty.

The test machines

The test machines used were mid-sized Amazon EC2 virtual servers: 7.5 GB of memory, 2×2 EC2 Compute Units, 64-bit platform running Ubuntu 7.10 and Sun JVM 1.5.0_13. A single virtual machine was used as the Cometd server and between 1 and 3 virtual machines were used to generate the load of 20,000 clients.

The test server software

The Cometd server used was the Cometd demo bundled with Jetty 6.1.7, with some modifications:

webapps/comet.war

The war was expanded into a webapps/root directory and the war file removed

contexts/*.xml,webapps/test*

All other contexts and webapps deleted

etc/jetty.xml

Initial benchmarking past 10,000 showed that the limiting factor was lock starvation on the BoundedThreadPool. A new version of the thread pool (QueuedThreadPool) was written that used 4 locks instead of 1 and was configured with a maximum of only 50 threads.
The number of acceptors on the SelectChannelConnector were increased to 4
The lowResourcesConnections was increased to 30,000

webapps/root/WEB-INF/web.xml

The timeout was increased from 120,000 to 240,000 ms

webapps/root/WEB-INF/filters.json

All filters were removed for the test

The application tested was the simple chat room (the “hello world” of Comet) using the BayeuxLoadGenerator class shipped with Jetty.

The test client(s)

The load generator uses the Jetty asynchronous HTTP client so that 20,000 Bayeux users can be simulated with only a few threads. While it is somewhat remarkable that the Cometd/Jetty server can scale to 20,000 users, it is truly remarkable that the test client can also handle such loads.

But no matter how good the test client is, 20,000 simulated users on 1 JVM on 1 machine just can’t generate/handle the same load as 20,000 users each with their own computer and network infrastructure. To understand how this affected the results, this test measured the latency of all 20,000 users simulated from 1 machine and compared that with the latency experience on a machine simulating only 1,000 users with the remaining 19,000 users being simulated by 2 other machines.

The test load

The test load generated was the most difficult for Cometd: a constant stream of messages without any introduced jitter that would allow the server to catch up on backlogs of messages. The load was a 50 byte payload sent in a burst to 10 randomly selected chat rooms at an interval fixed for each test. The interval was selected so that a steady state was obtained with the server CPU at approximately 10% and 50% idle.

Tests were performed for 1,000, 5,000, 10,000 and 20,000 simultaneous users for 100 and 1,000 chat rooms. This gave a range of 1 to 200 users per room.

Results

The raw benchmark results are available and are plotted in a bubble diagram above. You can download the results in a Gnumeric spreadsheet (with the diagram) or CSV format (without the diagram). The bubble size relates to the message throughput (a maximum 3,800 messages per second were delivered) and the bubble position shows the round trip latency.

The key result is that sub-second latency is achievable even for 20,000 users. There is an expected latency vs. throughput tradeoff: for example, for 5,000 users, 100ms latency is achievable up to 2,000 messages per second, but increases to over 250ms for rates over 3,000 messages per second.

Above 10,000 users, the test client became brittle, and while it was able to maintain good average latency, the maximum latency could blow out to beyond some of the timeouts. This appears to indicate that lock starvation is still occurring in the test client. The tests were repeated with the load generated from 3 clients instead of 1 and the latency measurements were taken from a client limited to 1,000 simulated users. This approach gave significantly better latency and throughput for 20,000 users and of course real traffic will not suffer from this type of limitation.

Summary

The Cometd implementation from Dojo and Jetty has demonstrated scalability for web-2.0 applications to levels that meet or even exceed node utilization levels expected for web-1.0 applications.

One of the oft-cited advantages of forever frame over long polling is that it does not suffer from the 3x max latency issue. This is when an event occurs the instant after a long poll response is sent to a client, so the event must wait for that response and the subsequent long poll request before sending a response containing that event. Thus while the average latency of long polling is very good, the theoretical max latency is 3x the average latency, which is the time taken to transit the network one-way.

Forever frame is said not to suffer from this issue, as it can send a response back at any time, even the instant after a previous event has been sent. Strictly speaking, that is not always the case, as forever frame implementations also need to terminate responses and issue new requests, at the very least to prevent memory leaks on the client. But for the purposes of this musing, let’s assume that it is true.

Does this theoretical lowering of the maximum latency actually enable any applications to be developed that would be impossible with the 3x max latency? For example, could forever frame be used to implement a first person shooter game that would be unplayable with long polling injecting a 3x latency on occasions (normally just as you charge into the room full of enemy guns lagging…).

Unfortunately, I think not. The problem is that comet will never be suitable for any application that cannot accept a bit of jitter in the application latency. Comet can achieve great average latency, often <100ms over the internet, but it is always going to suffer from the possibility of an occasional long delay.

The reason is that TCP/IP is by definition the transport that will be used for comet (that adheres to open standards) and TCP/IP is simply not a protocol that can guarantee constant low latency. Like long polling, TCP/IP gives very good average latency, but all it takes is 1 dropped packet and you will incur a TCP/IP timeout and resend, which by definition will be at least 3 x the network traversal time (sender must wait at least 2x network time before deciding that the ack will never come, then it must resend). Sure TCP/IP has lots of tricks and optimizations that are designed to help with latency for missed packets (e.g. fast resend, piggyback ack), but they rely on other traffic being sent in order to quickly detect the dropped packet. If a lone event is sent in a single packet, then at least 3x latency will result. One could even argue that the client’s need to send a new poll request with long polling will provide a convenient data packet on which an ack can piggyback, and could improve latency in some situations.

So any application that cannot tolerate 3x max latency is an application that should not be considered for comet. Comet is ideal for applications that thrive on good average latency, but that can tolerate the odd delay. For such applications, long polling is a good match and the theoretical latency gains of forever frame are probably just that—theoretical.