Polling, Long Polling and HTTP Streaming
The most popular technique used with Ajax currently is called polling. With polling, an Ajax application will poll the server for data every time the content of the page requires an update. Most of the responses the server returns will not contain any data (blank response).
The second technique, considered the emerging technique, is called long polling (a.k.a. Comet), where you open a persistent connection and wait for the server to execute the push operation. In contrast with the polling technique, the server never sends a blank response, and instead waits until data is available to initiate a push operation.
Finally, the third technique is called HTTP streaming. HTTP streaming is similar to the long polling technique except the connection is rarely closed, even after the server executes push operations. With HTTP streaming, an Ajax application will only send a single request and receive chunked (partial) responses as they come, re-using the same connection for a really long time.
Simple enough. Now with long polling and HTTP streaming, many problems might happen when the server executes its push operations. The one that happens frequently is what I call Delayed Push Operations, or DPO.
What is Delayed Push Operations (DPO)?
Before explaining delayed push operations, let me describe what a push operation is. A push operation is the phase when a server-side component decides to send message(s) back to the client using a long polling or HTTP streaming connection. The server component decides, most of the time, to execute the push operation based on some application-specific logic. As an example, the server-side component of a stock quote application will execute push operations every time new quotes are available, and a server-side component of a chat application will execute push operations as soon as a new chat message becomes available. The server component maintains a pool of long polling/HTTP streaming connections, and as soon as an event occurs, it starts push operations by updating the connection pool with the new data.
Delayed push operations happens when the client is not getting updated as fast as it should be or expects to be. As an example, a stock quote application that starts getting updates every 5 seconds instead of every 1 second will suffer the delayed push operations problem. Of course, the application still works, but the promise of real time data that Comet offers is severely impacted, and more importantly, the perception that the application provides real time data is severely reduced. Worse, most of the time the problems will be discovered very late in the development cycle, and having to re-architect the server-side component to fix the delayed push operations problem might impact the application architecture, its delivery schedule and again, perception of Comet as a real time technology.
When Does DPO happen?
Delayed push operations are measurable when the following symptoms start appearing:
- Server congestion: The OS/server/application is running out of resources. This problem arises most of the time when there are too many simultaneous connections open on a single server.
- Client congestion: The clients aren’t reading messages fast enough and the server is blocked while waiting for clients to read the messages. If the server uses a single threaded strategy for executing its push operations, one client might delay other clients as the server is waiting for that one client to read its messages.
- Data congestion: The server is not able to write messages fast enough, so messages accumulate and push operations are delayed.
The problem of delayed operations is better understood for applications that use the polling technique instead of long polling or HTTP streaming. Server and client congestion solutions are widely defined and used, and multiple solutions exist, like using load balancers, clusters of servers, etc. With the polling technique, you rarely suffer data congestion directly, but rather as a consequence of server/client congestion. Hence as soon as you fix your server/client congestion, your data congestion usually goes away.
Unfortunately, that’s not the case with the long polling and HTTP streaming techniques. With those techniques, a small number of client connections might easily produce data congestion that produces delayed push operations. How? In a Comet application, all long polling connections to the server might eventually share information amongst each other. To put it differently, all long polling connections might be virtually connected to each other. When one client sends data, the server will respond and then start its push operations, updating all the long polling connections virtually connected to each other. In comparison, with the polling technique, the server just needs to send a response back, with no extra operation.
So far so good. Now if all clients decide to respond to the first server push, the server will ends up having to initiate push operations for every client’s response. The server will work on executing its first push operations and the amount of operations it will have to execute will grow as soon as a single message is pushed. There might not be any data congestion for a relatively small amount of clients, but every new client added will increase the probability of creating data congestion. For a small load the problem might never show up, but increasing the number of virtually connected connections will eventually produce data congestion.
Want to see how this can happen? Just take a look at the JMaki Grizzly Comet demo or the Jetty Cometd demo. Those applications are quite cool to demo. Now just open more and more browsers and see how bad the situation can become. Just start moving things and see how other browser are updated….now find 50 of your friends and ask them to move things simultaneously…boom!!
There is no simple solution to avoid data congestion and reduce the probability of delayed push operations. There are several ways to reduce the probability of creating data congestion:
- Aggregate messages: Instead of executing a server push operation every time new data is available, aggregate messages and execute a single push operation containing multiple messages.
- Filter messages: Some messages might not need to be delivered under load and can be dropped.
- Delay messages: Some messages might not need to be delivered in real time, so delaying them can reduce the probability of producing delayed push operations.
When writing Comet-based applications, it is important to reduce the probability of delayed push operations. Since Comet is a new technology and optimal solutions for reducing delayed push operations still need to be explored, aggregating, filtering, or delaying push operations might significantly improve the usability of your application under load.