Comet and HTTP Pipelining

by Kris ZypFebruary 22nd, 2008

Comet is about creating duplex communication where both client and server can send messages at any time. HTTP remains an essential foundation for Comet communication since it works across proxied routes. However, current browser technology forces us to utilize two TCP connections to facilitate duplex conversation, despite that fact that TCP is a duplex protocol. In order to move towards future Comet capabilities that are optimally efficient we need to be able to duplex on single connection using HTTP.

Previously I had suggested a possible technique for duplex communication in HTTP based on request content streaming in XHR requests. A request could use “chunked” transfer encoding, and progressively send parts of the request body. While the client streaming of the request body is taking place, the response could then be returned to the user agent and also stream data for the response body. However, this approach relied on the capability of user agents being able to handle the response after sending part of the request, but prior to finish sending the entirety of the request. The HTTP specification is vague on the specifics of timing, simply describing a request being sent and response being returned for normal operation. After further testing it turns out that most browsers can not currently handle receiving the response before the request is finished: only Opera can handle this. All other browsers simply ignore the response if they have not yet finished the request. However, this was a proposal for native support in browsers, so it could be argued that since the browser must be modified anyway, why not modify HTTP handling characteristics. Unfortunately this same behavior extends to proxy servers as well. Changing the behavior of browsers doesn’t fix the intermediaries.

So the question remains, is it possible to have a full duplex conversation in HTTP over a single TCP connection where both client and server can asynchronously send messages? Actually, the answer is yes. HTTP pipelining makes this possible. Once a request has been fully sent to a server, the client can add nothing to that request, but with HTTP pipelining, additional requests can be delivered on that same connection without having to wait for the response to finish. Most browsers do not have pipelining enabled (only Opera, and soon Firefox 3 by default), and no browsers provide any means for the client JavaScript code to control the routing of requests for pipelining. However, if this functionality was exposed, and XHR interactive mode was supported, full duplex conversations would be possible. A client can make an initial HTTP request on a connection which would receive a streamed response that stays open indefinitely. The server can asynchronously send messages over this streamed response at any time to the client, and the client can send messages to the server any time by sending pipelined requests over this connection. Furthermore, since the pipelined requests are queued behind an indefinite request/response, no response will ever be returned. Essentially we can sidestep the normal request/response ordering: clients can send messages at any time without requiring a subsequent response, and the server can send messages at any time without requiring a preceding request (except for the setup request), all on a single connection leveraging the HTTP protocol. And since this is following the HTTP protocol, this will work over proxies (unless they completely fail to handle inbound pipelined requests, but almost all proxies can handle this), albeit the proxy may split the connections up to the server, but this does not affect the functionality.

Advantages

So what advantage would there be to using controlled pipelining for duplex conversations? We can already have a duplex conversation today by keep a connection open, and clients can send messages to servers by simply issuing HTTP requests over a second TCP connection. What advantage is gained by pipelining these requests over a single connection? There are few reasons:

  • There is not always a second open TCP connection immediately available. When persistent keep-alive connections are used, a second connection may be available if there has been recent activity. However, if there has not been recent activity on the second connection or if it was never built, then a new TCP connection must be created. Creating a TCP connection requires a TCP handshake which involves an additional round trip and therefore increases the latency for sending the message. This communication overhead should not be necessary, since there is already an open TCP connection that could be utilized for sending data.
  • Building or keeping a second connection open puts additional strain on the server. Connection handling is one of the most resource intensive operations for the server.
  • Client sent messages do not require a response, and therefore less bandwidth is utilized. Normally all requests require a server response. However, if the response body stream is kept open indefinitely (until the connection is closed), requests that are pipelined on the same connection will never need to have a response. Responses must take place on the same connection, but the connection never finishes the current response to allow other responses, so no response is needed. In typical Comet architectures, many client messages like subscription requests do not require a response at the application level, and could operate more efficiently with one. At the transport level, as long as the TCP connection is open, the TCP protocol handles the acknowledgment of the message delivery and resending the message if necessary for reliable delivery of messages. It is important to note that only idempotent messages are supposed to be pipelined (subscription requests can be idempotent, publishing or triggering new messages are often not).
  • Subscription tracking can be handled without inter-connection state sharing. If subscription requests are all delivered over the same connection as the streamed response, all subscription information can be isolated to a single connection. Therefore, once a connection is closed, a server could immediately free all the resources associated with that connection such as the list of subscriptions.

The Comet and Pipelining Hazard

At the present, some browsers have pipelining ability, most do not have pipelining enabled, and none provide any mechanism for controlling the connection used for the pipelining. However, pipelining is most likely going to increase in usage in the browser. Firefox 3 will probably have pipelining enabled by default. Without connection control, this actually creates a very significant future hazard for Comet. Pipelining in conjunction with Comet without control of which connection is used for pipelining can lead to indefinitely unfulfilled requests. This might be desirable for subscription requests, but if a browser pipelines resource requests that should receive a response behind a Comet request, the responses may never come back to the browser, since they are required by HTTP to come after the first response (the Comet long-lived response in this case) is finished. Without XHR pipelining connection control, when a Comet connection is open on a pipelining enabled browser, Ajax and other requests may “mysteriously” hang.

Current XMLHttpRequest Standardization

I have made a proposal to the Web API Working Group to add pipelining control to the XMLHttpRequest specification. In my original proposal, pipelining could be disabled or enabled by XHR users. However, pipelining has traditionally been either disabled or not implemented in browsers due to concerns about some proxy servers and origin servers not supporting it, and the working group was hesitant to allow users to enable it. However, even if enabling pipelining can’t be controlled, I believe it is extremely critical that there is some means to advise the browser on which connection to use to pipeline a request when pipelining is enabled. Pipelining has the potential to be a tremendous aid or a terrible hazard to Comet. Controlling the destination connection for pipelined requests not only provides a means for duplex HTTP communication, but also may be used to prevent the hanging requests problem described above, and could even have potential performance benefits for optimally ordering other Ajax requests. If you would like to see this ability added to the XMLHttpRequest standard or have suggestions on how this should be done, email the working group’s public mailing list. My proposal to standardize a way to advise the browser on which connection to pipeline requests is currently being actively discussed by the working group on their mailing list (the archived discussion is in the online archives starting in February 2008).

8 Responses to “Comet and HTTP Pipelining”

  1. Martin Tyler Says:

    Hi Kris,

    I thought about this a few years ago, but didnt get far once I hit the block of the browsers not supporting it.

    Have you tried this technique through any proxy servers? I wondered if your idea of only using the initial requests response and not responding to subsequent requests could cause proxies (or the browser) to timeout and return errors. Although, if it works, your way would have bandwidth benefits i was thinking each new request would cause the current response to end and a new response to start.

  2. Kris Zyp Says:

    Martin,
    I did some further tests with a proxy, and there is indeed problems with making this work across a proxy server. The apache proxy (which I believe is similar to most current proxies) servers can handle inbound pipelined requests, but they do not forward subsequent pipelined requests until they have received a full response from the first request. This means that pipelined requests would not be delivered to the origin server as long as the Comet response is still open.
    Of course, this isn’t a showstopper, as long as you have a fallback mechanism for users behind a proxy, other users with a direct path (which generally represents the majority) would still enjoy the benefits of the improved performance of duplex on a single connection. And future proxies may be more capable of pipelining (although you have to be able to detect that capability).

  3. Steven Roussey Says:

    You shouldn’t have other requests “mysteriously” hang if the comet request is on a separate subdomain, right? Or am I missing something?

  4. Kris Zyp Says:

    Steven, you are correct, a connection can’t be shared with a different subdomain, so if there is only a comet request being sent to a subdomain and no other requests, then there would be no pipelining of requests on that connection.

  5. Comet Daily » Blog Archive » Introducing HTTP Channels Says:

    [...] HTTP. However, full duplex HTTP is not possible within the constraints of the browser because of a lack of sufficient control of connections and pipelining. Therefore, the Channels transport also includes the necessary mechanism to emulate a single [...]

  6. Anders Both Says:

    Can you disable Pipelining on the Server, lets say on IIS ?
    (or via asp.net configuration).

    I know it wont help for the two connection limit, but I would not want a browser to start pipelining to the server, if the server host advanced comet services.

    ?

  7. Dave Says:

    I believe that this is because the HTTP spec requires pipelining to keep the requests/responses in order.
    I’m not sure why the HTTP spec requires this and it sure seems counter intuitive to me in that I would love to pipeline a bunch of stateless requests. I don’t care about the order of responses or even if all requests even get responses.

    http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html

    Does anyone know why HTTP Spec requires pipeline ordering? Why not leave that open to the use case for the application?

  8. Adam Says:

    Great post, Thanks,
    You can test pipeline using this free HTTP pipeline tool and test the response return from the web when sending different HTTP Get or post with pipeline:
    http://soft-net.net/SendHTTPTool.aspx


Copyright 2015 Comet Daily, LLC. All Rights Reserved