Alex Russell, one of the writers of the Bayeux protocol, says that Michael’s complaints are fundamentally flawed.

Part 1: Greg Wilkins explains the need for Bayeux
Part 2: Michael Carter criticizes the current state of Bayeux
Part 3: Greg Wilkins responds to Michael Carter
Andrew Betts’ thoughts (from a related article)
Part 4: Michael Carter responds to Greg Wilkins
Part 5: Kris Zyp’s thoughts
Part 6: Alex Russell responds to Michael Carter
Part 7: Michael Carter responds to Alex Russell

Vaguely Correct vs. Precisely Wrong

My attention was drawn today to re-reading the Battle of Bayeux series again, and as I re-read Michael’s response to Greg which said that Bayeux has an “unrealistic agenda”, I couldn’t help but feel that Michael (and many others I’ve talked to) have simply missed the entire point of Bayeux, which is to specify enough structure to grow the number of interoperable Comet clients and servers while leaving enough to the implementations to allow integration of piecemeal evolution until such a point as the protocol itself is unnecessary.

Reading many of Michael’s complaints about Bayeux, I’m struck by his ardent desire for the protocol he’s implementing to be the right one. Or rather, for the description to be so air-tight as to enable full automated testability, interoperation, and even the integration of concerns such as URL mapping and eliding handshakes as first-order priorities for any “worthy” spec. Perhaps if the authors-to-date who have contributed to Bayeux could count on the world not changing, we might agree that Bayeux should define client APIs or that it should get super-specific about how to handle multi-frame. But why? And why right now? And who is that good for?

Let’s assume for a moment that Bayeux specified a multi-frame strategy. Today, given the limits of browsers, any such specification will be writing into stone what is essentially a series of browser-specific hacks. Indeed, while the entire Bayeux spec is a reaction to browser deficiencies, it seeks to avoid prematurely writing those bugs into stone tablets. Today, given the limits of browsers, any such specification will be blessing what is essentially a series of browser-specific hacks. Indeed, while the entire Bayeux spec is a reaction to browser deficiencies, it seeks to avoid prematurely standardizing hacks. Had we fully specified a multi-frame solution when we started work on Bayeux clients and servers, the demands of terseness and clarity would have demanded that we write into stone an “IPC over cookies” protocol which uses timers in cooperating frames to send/receive packets of encoded data which is re-constructed and re-evaluated on the other side. Today, the “duh” thing to do would simply be to use the HTML 5 DOM Storage provision and notify library authors to hack around it for old browsers for the time being. In another 18 months, the entire point will likely be moot as browser vendors get wise to Comet and provide us with ways around the 2-connection limit at an HTTP level. When that happens, multi-frame concerns are just legacy features. The point here isn’t that one solution is better. It’s that 18 months ago, we didn’t know the future, but were sure enough of our own fallibility to avoid wild goose chases for what is, essentially, a client concern. The “right” solution has changed in the last 18 months. More to the point, how multiple Bayeux clients in a single process communicate to a server has zero bearing on the semantics of the messages which Bayeux exchanges. Addressability amongst peering tabs is, from one view, purely a client issue which clients should be free to experiment and iterate on. Indeed, much of Michael’s argument seems to be predicated on the supposition that he can do things better. Of course he can! That’s the whole point!

The furthest down the path of picking sides that the protocol currently goes is to specify that servers MUST implement at least the long-polling transport, and leaves the API which is exposed on either end to implementers. Clients and servers may then implement other transports and negotiate which one to use via the specified mechanism (although this can also be elided without violating the spec). This is entirely by design, and we expect that there will be revisions and addendum’s which outline recommended APIs once enough clients and servers exist to know what the right thing to do is. The long-polling transport is the small, critical, core of network-level semantics that when combined with client-level semantics can serve to guarantee interoperability. Not only do we expect that the long-polling transport will be the worst performing transport for a given combination of client and server, we also anticipate that it will be the least likely to be used in large-scale production sites. But you don’t have to care about that if you’re just adding Comet to a small blog….all you want to care about is that your server and your client can talk to each other. It is this entry-level sector of the market which Bayeux is aimed at enabling, while still providing enough play in the joints to enable those with different requirements to handle them within the loose framework of the spec. This isn’t not doing the hard work, it’s being smart enough to know that the hard work we do today will not survive long should we over-adapt to a particular environment only to find that is changing underneath us.

As with many kinds of engineering, the best performance requires the greatest adaptation to the current environmental constraints. Bayeux seeks to enable that evolution to local constraints without over-burdening the market with fundamentally incompatible clients and servers over the long haul. Where Bayeux “waves its hands”, it does so for good reasons; ones which we feel outweigh the short-term costs (which Michael is quick to enumerate).

Bayeux, as a spec, is built with the understanding that the environment is changing, and that competition amongst implementations is good only insofar as it doesn’t cause huge amounts of lock-in. By specifying a thin layer of semantics about messages and transports and by building extension and plug-in points directly into the protocol, Bayeux has lead to successful evolution already. From cross-domain long-polling using JSON-P to Flex clients which speak optimized wire formats to a profusion of interoperable servers in many languages, Bayeux is succeeding by leaving enough play in the joints to meet reality, today’s and tomorrow’s, half way and then letting implementations figure out the rest.

There’s a long way to go in making Bayeux better, and we know we’re not done yet, but let’s be clear that by omitting things from the core of the spec, we are not calling them unimportant, just unsettled and not absolutely required for interoperability.