Kris joins the discussion in part 5 of this series on Bayeux.
Part 1: Greg Wilkins explains the need for Bayeux
Part 2: Michael Carter criticizes the current state of Bayeux
Part 3: Greg Wilkins responds to Michael Carter
Andrew Betts’ thoughts (from a related article)
Part 4: Michael Carter responds to Greg Wilkins
Part 5: Kris Zyp’s thoughts
Part 6: Alex Russell responds to Michael Carter
Part 7: Michael Carter responds to Alex Russell
Bayeux has a pluggable design that opens the door for future improvements in Comet communication. Bayeux defines a default level of facilities such that any conforming client and server can communicate at the publish/subscribe level of meaning using very basic transports. Bayeux provides transport negotiation and discovery, and the mechanisms for rudimentary transports to participate in duplex conversations using connection management and encoding definitions. However, while other transports may be provided by Bayeux compliant agents, they may not necessarily need or want all the facilities provided by Bayeux. Bayeux provides the default mechanisms, but more advanced transports may not need them.
This means that a server can provide a new high-efficiency transport (that is not defined in the Bayeux spec), and still implement Bayeux by advertising this transport with available Bayeux handshakes and providing the required long-polling and callback-polling mechanisms as a backup to the more advanced transport. In addition, the transport may or may not need to utilize the other facilities of Bayeux. Perhaps, if the transport was an XML transport, it may already have the inherent capability for encoding meta data/properties, and would not need to use the Bayeux JSON encoding specification. A more compact compressed transport may also have such capabilities and not require JSON encoding. However, a simple transport that only can only transfer plain text payloads can leverage the JSON encoding for meaningful and interoperable transfer of properties. It should be noted that this is certainly the intent of Bayeux, even though the current draft (seemingly mistakenly) insists that that every transport use JSON encoding.
A new transport might already have reliable duplex connection capabilities. In this case the transport would not need to send “meta/connect” messages nor the clientId-based session management. Also, a client may have a priori information about which transport to use, in which case a Bayeux handshake may not be necessary to begin a connection to the transport. Any capability that the transport inherently provides, it can do so itself. Capabilities necessary for a minimum level of coherent server delivered messaging that the transport does not provide can realized by implementing the protocol definitions from Bayeux. Bayeux provides a default mechanism for every part of the layers of communication, but each of these layers can optionally be implemented by the transport (which can span all the layers, or simply the lowest layer). However, it should be noted that Bayeux does not provide long-term guaranteed message delivery or message ordering, although this can be provided by transports.
As a transport takes on more of the responsibility of meaningful duplex conversations, Bayeux becomes more of an API than a format or protocol. With simple text-based transports, Bayeux is a protocol defining the order of operation and format of the messages. With more advanced duplex meta data capable transports, Bayeux is intended to become an API, defining higher level abstract concepts like starting and terminating a connection, sending and receiving messages, and defining certain properties that should be transferred in messages for API compliance, but leaving the formatting and operation up to the transport.
Unfortunately, this pluggable API-based transport capability is not spelled out very clearly in the Bayeux specification and there is no API defined in the specification. However, the client reference implementation (dojox.cometd) provides a very reasonable transport API that could certainly seem to be a good candidate for being a Bayeux API. A transport must simply implement the following methods:
This is called to initialize a transport.
This is called with an array of objects that represent messages to be published. Each object has properties corresponding to the properties defined in the Bayeux specification for messages (channel, data, clientId, etc.).
This is a method that can be replaced or listened to by services that use this transport. Whenever a transport receives a message from the server, it should call deliver(message) with an object with properties corresponding to those defined in the Bayeux specification for a server sent message (channel, data, clientId, etc.).
This is called to disconnect the transport.
- check(types, version, xdomain)
This can be called by Bayeux implementations to determine if this transport is appropriate for use for the current server. The types parameter is an array of server supported transports, and xdomain indicates if the Comet connection must cross domains.
This is a reasonable API, and it allows a lot of flexibility in how transports actually behave. Transports have ultimate control over what is transmitted on the wire. There is still a need for standardization of something like this, so that new transports can plug into generic Bayeux implementations.
Essentially a client/server can utilize their own sophisticated communication technique, yet the server can still be Bayeux compliant by providing Bayeux service advertising and implementing the required transports of Bayeux as a backup. The client can still be Bayeux compliant if it can connect to other Bayeux servers and at least handle the basic transports (although it would seem most vendors probably will not find as much value in providing a Bayeux compliant client, since it is rather heavy, and there is already a good one in Dojo). A Comet application could even be built using Flash as the primary mechanism for its own client/server transport, but the server could additionally provide Bayeux for clients from other sites to use. Much of the resistance to Bayeux seems to be predicated on the concept that Bayeux always requires a certain format/protocol to be used by all transports. I believe this is simply a mistake in the current draft, and needs to be corrected. The client and server should be allowed to perform Comet communication with their own specific transport without violating Bayeux.
There is still work to be done to develop better standard transports. The Bayeux default transport, Bayeux long-polling (not to be confused with the general technique of long-polling which could be used as a mechanism for other transports as well), is a very simple naïve transport. This is a good because it is easily implemented, and when used as a backup for agents that are not privy to more advanced transport, does not require significant implementation effort. However, Bayeux long-polling is inadequate for many Comet applications. It is not very efficient, does not take advantage of more advanced future and current capabilities like XHR streaming, does not have guaranteed message delivery or ordering, and does not effectively use HTTP features. In addition, publish/subscribe is not the best paradigm for every Comet application, and a transport can certainly expose more relevant constructs. Symmetrical triggers are not always appropriate, and additional meta data information such as REST semantics can be extremely important in some applications and provide message ordering stipulations.
There would be great value in a standardized, efficient, and reliable (message delivery guarantee) full-duplex transport that addresses these concerns and leverages standards-based mechanisms such that could be widely used (since proprietary transports lose their value as soon as a different agent is involved). As Michael said, “it makes strategic sense for the Comet standard to closely resemble and interoperate with current standards.” I completely agree, and hope a future Comet standard can be developed that does build upon the foundation we have in HTTP, rather than fighting it. I think my proposal for Comet communications using HTTP semantics is a good candidate, but it does need more refinements.