Kris Zyp recently categorized Comet applications into event-based and data-centric Comet. I will try to expand on this by mapping such definitions to two general types of publish/subscribe architectures. Then I will describe two models of publishing.
Comet Pub/Sub: Symmetric vs. Asymmetric
The typical representation of a Comet-based pub/sub architecture is shown in Figure 1. Two actors are involved: the Comet Client and the Comet Server. The Client can act both as a publisher and a subscriber. The Comet Server is the hub that manages the subscriptions and the message routing. This architecture can be mapped to Kris’ Event-based Comet and I will refer to it as symmetric Comet pub/sub.
But there are well known situations where the publishers and the subscribers are completely separated, from both a logical and a physical perspective. Figure 2 shows this topology, where Clients act only as subscribers while the role of the publisher is played by a new actor: the Data Feed. This architecture can be mapped to Kris’ Data-centric Comet and I will refer to it as asymmetric Comet pub/sub.
Asymmetry between publishers and subscribers is a natural property for many Comet applications. For example, in the Finance industry the most successful application of push-technology is Market Data Distribution (or Dissemination). Each market (or exchange) produces a high volume of real-time data that characterizes the state of a financial security or instrument. Each state is made up of a set of fields (e.g. last price, timestamp, quantity, volume, bid, ask, etc) that can number in the hundreds. There exist some subsets of the state that can change at a very fast pace (the “market depth” can produce hundreds or thousands of updates per second for a single security). These state changes are driven by the orders (buy and sell) submitted by the traders. Collecting, organizing and selling this real-time data flow is the business of several specialized companies (the Financial Information Providers) such as Reuters and Bloomberg. They provide specific platforms together with APIs to make the real-time data flow available to their customers. This infrastructure is usually referred to as the Data Feed.
Different Comet implementations are more oriented towards one or the other type of pub/sub. For example, Cometd is based on symmetric pub/sub, while Lightstreamer is based on asymmetric pub/sub. That does not mean that an asymmetric pub/sub solution does not enable client-side publishing, rather that the exposed interface is ready and optimized for server-side publishing (client-side publishing is usually obtained by having the client behave like a feed through a server-side Adapter).
Asymmetric solutions should offer a way to integrate the Comet Server with any Data Feed. This is often accomplished by an Adapter-based architecture, borrowed from Enterprise Application Integration (EAI) systems. An Adapter (also referred to as a Connector) is a software component that performs two-way communication and implements the interfaces exposed by the two ends (the Server and the Feed). In other words, the Adapter adapts the Feed’s protocol to the Server’s interface.
The Comet Server should expose an interface that is flexible and efficient at the same time (this is a trade-off), so that third parties can develop Adapters for any Data Feeds. For example, Lightstreamer exposes three types of Adapter interfaces, based on a Java API, a .NET API, and a socket-level protocol.
This kind of architecture has some business implications too. In fact, off-the-shelf Adapters are beginning to appear on the market (e.g. Migratory Data Systems is a European company that ships a ready-made Reuters RMDS Adapter for Lightstreamer) and some system integrators are specializing in the development of Comet Adapters. The Comet space is becoming “structured”. That is, multiple parts are potentially involved in an enterprise Comet project: the Comet Server vendor, the Adapter vendor, the Information Provider, the AJAX toolkit vendor, and the AJAX system integrator.
Publish vs. “Publish On-Demand”
Another property that naturally applies to the two families of Comet architectures (Event-based/Symmetric and Data-centric/Asymmetric) regards how the publishing process takes place over time.
In symmetric Comet, publishers usually deliver data to the Server totally at their own discretion. When they have something to publish, they send it. The publishing life cycle for each topic usually corresponds to the life cycle of the publisher itself.
In asymmetric Comet, the Comet Server can often be seen as a Web-oriented add-on to a messaging system that exists on the server side (basically, the infrastructure within and behind the Data Feed). The role of the Comet Server is to dispatch the messages received from the publisher to all the users subscribed to a certain topic. If no user is subscribed to the topic to which a newly received message pertains, that message should be discarded. For performance reasons (that is, to avoid processing potentially high volumes of unnecessary data), the Comet Server should not receive messages for which there is no subscriber. When a client subscribes to a topic that is not currently subscribed to by any other client, the Comet Server activates the same subscription on the Data Feed and initializes a “channel” to start receiving updates on that topic. If a second client subscribes to the same topic, no action takes place towards the Data Feed. When all the clients unsubscribe from that topic, the Comet Server will deactivate that subscription on the Data Feed and stop delivering data on that topic. This model can be defined as publish on-demand (be aware that this is not fully compliant with a purist’s definition of pub/sub) and is reflected by the interface exposed by some asymmetric Comet Server implementations.
The architectural notions of symmetric and asymmetric Comet, and of publish-on-demand, are useful to understand which solution better fits the application requirements. Although the implementations of the two Comet models can be arranged to behave as the alternative model (or as a mix of the two models), it is important to understand the very architecture of a Comet solution in order to verify that it matches the actual needs.