They recently added Web Push capabilities, so that any changes to the data are reflected in real-time through a web front-end. The Comet server used by GroovyDB is Lightstreamer.

Their site hosts a live demo of news pushed in real-time from the GroovyDB.

The Lightstreamer Adapter they developed is also available for download.

Dobrica writes:

I already blogged about RFID readers but now I really need to replace antique 3M software with something better. So, basically, I started with specification:

User should put RFID tag on reader and get output from Koha

Nice and simple. However, readers are connected to Windows machine. And we need some way for reader to push data about new tag to browser. Hmmm… push? Sound like Comet, doesn’t it?

Let’s see what we need:

1. Comet server to display page in web browser and push updates
2. RFID reader program installable on Windows to communicate with serial port and push messages to Comet server
3. User with a browser

I particularly like this because RFID driver software is often appalling, and it’s nice to see some of the principles of building the rich internet applications that we love to use extended to breathe fresh life into the way we interact with embedded systems.

Twitterfall is a real time twitter feed hosted on EC2 and using Meteor for Comet. It simply streams tweets onto your page based on filters you can define. Unfortunately for the Telegraph, the only filter they set up was the hashtag #budget.

See the results for yourself:

And so it went on, and on, until the widget made a sudden and mysterious disappearance about three hours after it was published. The moral of this story? Try to resist the urge to stream unfiltered user generated content directly onto your prestige pages.

Some of the more mature Comet servers also provide APIs in other languages to access the same functionality that the browser JavaScript client can access. These APIs have mainly been for standalone applications using Java and .Net or for testing, but also for other browser technologies such as Flex, and more recently Silverlight.

At Caplin, a number of our customers use our Java client API, for applications and for browser applets. Recently some customers have also used our .Net client API.

Our focus has always been on the browser though and in recent years we have invested a lot of time on our products surrounding the core Comet server, including Caplin Trader, a rich Trading browser front end. Being Ajax, Caplin Trader can host Flex, Silverlight, and Java components. We demonstrate the use of these technologies in Caplin Trader, but as yet do not provide easy to use extension points or APIs. In Caplin Trader all of our standard display components, the business logic, and the layout management are written in JavaScript. Few browser applications are as advanced as this. Maybe some of the web desktop style demos come close, but unlike Caplin Trader, they appear to be technical showcases rather than real world applications.

Flex

Despite the buzz around Ajax in many communities, it is still a barrier for some large corporations and we come up against people wanting to write Trading front-ends in Flex. We have both won and lost these battles, and time may tell who was right, but most likely there will not be a clear cut conclusion.

I have thought long and hard about the choice. It is not a straight Ajax versus Flex debate, because we are selling a framework that happens to use Ajax and has lots of pre-built functionality, on both the client and the server-side. Our goal is to make our front end is a lot more customisable and extensible without having to write any JavaScript at all—at the moment some customisation is like this and some requires you to break out the text editor and write some code. This is often the scary part for a customer; the tech team may be unfamiliar with JavaScript development, but have people more attuned to Flex development, or at least with Flex they feel they are using something more akin to the kind of development they are used to. Flex provides a more straightforward development package. It is provided by a single source, Adobe. Some people may be put off by having to pick and choose from a whole host of Ajax libraries and tools.

Flex can give you a fairly instant feel-good factor. It has an Eclipse based GUI builder that makes you think you have achieved something very quickly. Adding any real world functionality to a dumb GUI obviously takes some time though, but often the technology decision has been made by then.

Adobe also provides Comet like abilities through their LiveCycle Data Services product. This can be integrated with the back-end using JMS; again the Developer familiarity bell rings here, but that does not necessarily make it the right tool for the job. It is generic, which can be advantageous, but you will probably be reinventing the wheel.

So why do people choose Flex? Being an Ajax focused Comet site I’d expect a slightly biased view here, but I am interested in what it is people like about Flex. Are there things the Ajax community needs to do to combat some of things mentioned here?

Silverlight

Then we get to Silverlight. Unfortunately Microsoft has not made it as easy for software vendors to create Silverlight APIs from a standalone .Net API as you would hope. I believe Lightstreamer has had a .Net client API for some time, but have only just released a beta of a Silverlight API. I am assuming this was not because the Silverlight API is totally different, but due to the hurdles of the technology.

Silverlight and Flex are both moving targets to a certain extent. Some aspects of Flex require the latest Flash plug-ins that do not have the vast install base of the previous versions. Silverlight 3 will be released sometime this year, and appears to add a lot of new functionality.

How do people in this space view Silverlight? At Caplin, we have not yet had to face off against someone wanting to develop their own Silverlight trading application, but I can see this happening in the not too distant future. I imagine that Silverlight has many of the advantages of Flex, and possibly many more, but will it catch on and will it pose a threat to Ajax?

Conclusion

Flex and Silverlight both provide advantages over Ajax development, or at least perceived advantages. The wealth of Ajax technology out there is both an advantage and a disadvantage, for some the simplicity of going with the single vendor solution is the easy choice. In a large organisation the old adage ‘Nobody ever got fired for buying IBM’ might apply here, I think some development teams are choosing the safe route, which in a lot of cases might not be the best option.

When the choice to build something from scratch is Flex/Silverlight versus Ajax, I can understand why some would go with Flex or Silverlight. However, when the choice is building it yourself with Flex/Silverlight versus using a more complete existing Ajax solution the balance is tipped firmly towards Ajax. This does not just apply to Caplin Trader—there are other Ajax based solutions in all kinds of sectors that offer far more than a programming library.

I have not said much about native Comet support in Flex or Silverlight, as I do not think it is the key factor here. Adobe provides LiveCycle Data Services, Microsoft do not seem to provide anything like this yet—and some other vendors support Flex and/or Silverlight with their Comet servers. Alternatively JavaScript can be used to bridge into these technologies which allows you to mix and match.

The initial concept of cometD was to provide “web quality” communications between the server and the browser, i.e. it was not intended to provide reliable, transactional and/or atomic delivery of messages. The premise was that if your network failed, then messages can be lost. While this is sufficient for many applications, it is not sufficient for mission critical applications. Luckily, the Bayeux protocol implemented by cometD provides for an extension mechanism, which cometd-jetty, cometd-dojox and cometd-jquery have now used to provide an acknowledged message mechanism for reliable message delivery.

Server-side

To enable cometd-jetty support for acknowledged messages, the extension must be added to the bayeux instance during initialization:

  bayeux.addExtension(new AcknowledgedMessagesExtension());

The AcknowledgedMessageExtension is a per server extension that monitors handshakes from new clients, looking for clients that also support the acknowledged message extension and then adds the AcknowledgedMessagesClientExtension to each client on handshake.

Once added to a client, the AcknowledgedMessagesClientExtension prevents messages being delivered on any request other than a /meta/connect to prevent the possibility of out of order delivery. The extension also maintains a list of unacked messages and intercepts the /meta/connect traffic to insert and check ack IDs.

DojoX Client

The client-side for Dojo is provided by dojox/cometd/ack.js which was released in Dojo 1.3.0b2 (but can also be applied by patch to Dojo 1.2.x). To enable the client-side, the dojo.require mechanism is used:

  dojo.require("dojox.cometd.ack");

This is sufficient to enable the extension, however it may then be programmatically disabled/enabled before initialization by setting the ackEnabled boolean field:

  dojox.cometd.ackEnabled = (dojo.query("#ackInit").attr("checked") == "true");
  dojox.cometd.init(cometdUrl);

jQuery Client

The client-side for jQuery is enabled by including the jquery.cometd-ack.js file (bundled with jetty 6.1.15):

  <script type="text/javascript" src="../../jquery/jquery.cometd.js"&gt;&lt;/script>

Details

To enable message acknowledgement, both the client and server must indicate that they support message acknowledgement. This is negotiated during the handshake. On handshake, the client sends "ext":{"ack": "true"} to indicate that it supports message acknowledgement. If the server also supports message acknowledgment, it likewise replies with "ext":{"ack": "true"}.

The extension does not insert ack IDs to every message, as this would impose a significant burden on the server for messages sent to multiple clients (which would need to be reserialized to JSON for each client). Instead the ack ID is inserted in the ext field of the /meta/connect messages that are associated with message delivery. Each /meta/connect request contains the ack ID of the last received ack response: "ext":{"ack": 42}. Similarly, each ack response contains an ext ack ID that uniquely identifies the batch of responses sent.

If a /meta/connect message is received with an ack ID lower that any unacknowledged messages held by the extension, then these messages are requeued prior to any more recently queued messages and the /meta/connect response sent with a new ack ID.

Demo

There is an example of acknowledged messages in the DojoX chat demo that comes bundled with cometd-jetty.

To run the demo, download the Jetty implementation of cometD, then:

  cd contrib/cometd/demo
  mvn jetty:run

Point your browser to http://localhost:8080/examples/chat/ and make sure to check “Enable reliable messaging?”.

Use two different browser instances to initialize a chat session, then briefly disconnect one browser from the network (the work offline option will do this). While one browser is disconnected, type a chat message in the other browser and this will be received when the disconnected browser is reconnected to the network.

Note that if the disconnected browser is disconnected for an excess of maxInterval (the default is 10s), then the client will be timed out and the unacknowledged queue is discarded.

SIP messages follow the MIME/HTTP style syntax, and SIP has also been extended to provide a powerful integration with HTTP for monitoring and subscribing to resource changes (along with an extension for efficient subscription to multiple resources at once). This technique bears a strong correspondence with the REST Channels mechanism, providing a means with integrating with a REST architecture using the semantics from HTTP (REST Channels is specifically designed for Comet though), allowing a client to subscribe to resources and be notified of changes.

Since SIP is transport agnostic, it would also be possible to layer on top of Comet transport layer like js.io or WebSockets.

The thread executing the event listener is a web-request thread and in this context the document (ItsNatDocument) is locked by this thread, so there is no need for synchronization. This is an opportunity to change the server DOM, to send custom JavaScript, etc. Prior to this release, the background thread usually updated the client (synchronizing the document first). Now, the event listener approach is preferred, but the old approach is still valid.

The following example shows how a background thread notifies the client every 2 seconds. There are two notifications (two alerts), the first one pushed by the background thread and second alert is pushed by the event listener. Of course the method CometNotifier.notifyClient() needs to be called by the background thread, otherwise the first alert is queued but not sent and the Comet event is not fired and dispatched.

    final ItsNatDocument itsNatDoc = ...;
 
    final CometNotifier notifier =
        itsNatDoc.getClientDocumentOwner().createCometNotifier();
    EventListener listener = new EventListener()
    {
        public void handleEvent(Event evt)
        {
           itsNatDoc.addCodeToSend("alert('Tick From Event');");
        }
    };
    notifier.addEventListener(listener);
 
    Thread backgroundThr = new Thread()
    {
        public void run()
        {
            System.out.println("Background server task started");
            long t1 = System.currentTimeMillis();
            long t2 = t1;
            do
            {
                try
                {
                    Thread.sleep(2000);
                }
                catch(InterruptedException ex) { }
 
                if (!notifier.isStopped())
                {
                    synchronized(itsNatDoc)
                    {
                       itsNatDoc.addCodeToSend("alert('Tick From Thread');");
                    }
                    notifier.notifyClient();
                }
                t2 = System.currentTimeMillis();
            }
            while( (t2 - t1) < 10*60*1000 ); // Max 10 minutes
 
            notifier.stop();
 
            System.out.println("Background server task finished");
        }
    };
 
    backgroundThr.start();

Another interesting feature introduced in ItsNat v0.6 is a draft of the full remote control capability. In full remote control, two or more users can “share” the same web page, that is, if a user does something and the page changes, the page of other users are updated accordingly. Polling and long-polling can be easily used to update the page the users attached to the original page (if the original page need to be updated automatically an Ajax timer or CometNotifier can also be used). In previous versions, only read-only monitoring was possible (”remote views”).

The full remote control capability is disabled by default, and must be enabled by using an internal flag, if you need this feature now contact with me to learn how to enable it.