With today’s release announcement for Orbited 0.6.0, Orbited is now officially Beta software.

From the start, the 0.6.0 branch has been exclusively about increasing the stability and reliability of existing features over adding new features. In particular, we’ve stress tested against all of the memory leaks and performance issues that are often associated with Comet development. In addition, we’ve adopted Dojo’s DOH test framework for automated browser-side testing of the Orbited.TCPSocket object. The TCPsocket interface today is very reliable, robust both to browser defects and intermittent networks. After two years of development, Orbited is now considered stable enough for more widespread production deployment.

In Orbited, we sometimes send multiple events in a single payload, particularly with the long-polling and polling transports. After we parse the data of each event, we call some event handler that a user has attached to the Orbited connection. Imagine code that looks something like this:

function rawOnReceive(rawData) {
    frames = parseFrames(rawData);
    for (var i = 0; i < frames.length; ++i) {
        self.onread(frames[i]);
    }
}

Unfortunately, sometimes user code can throw an exception. If this happens, we don’t want to stop sending the remainder of the frames, so we revised our original code to look more like this:

function rawOnReceive(rawData) {
    frames = parseFrames(rawData);
    for (var i = 0; i < frames.length; ++i) {
        try {
            self.onread(frames[i]);
        }
        catch(e) { }
    }
}

But now we have a problem — that is, if the user code fails, we hide the error, and then dispatch the next packet. This is a terrible solution because it will be next to impossible for users to track down bugs.

Consider window.setTimeout, a mechanism generally used to schedule calls after a set amount of time has expired. It has a side effect though; it moves code out of the callstack. So we use setTimeout with 0ms as the specified timeout in order to make the appropriate onread calls without worrying whether they throw exceptions or not:

function rawOnReceive(rawData) {
    frames = parseFrames(rawData);
    for (var i = 0; i < frames.length; ++i) {
        window.setTimeout(function() { self.onread(frames[i]) }, 0);
    }
}

This means that all of the onread callbacks will be scheduled to be executed, but none of them will actually execute until our code, the rawOnReceive function, finishes. Unfortunately, there is a second problem — the value of i used for each onread callback will be the final value of i. We need to hold on to each value separately for each callback. Consider this code:

function rawOnReceive(rawData) {
    frames = parseFrames(rawData);
    for (var i = 0; i < frames.length; ++i) {
        (function() {
            var j = i;
            window.setTimeout(function() { self.onread(frames[j]) }, 0);
        })();
    }
}

Note the extra function and the “var j = i” — Our solution is to create a closure that saves each value of i as locally scoped variable j. This code will allow us to make all the appropriate onread calls, whether or not one calls user code that causes an exception.

When I first tried this out, I thought it would fail due to the ordering of the messages — after all, we go through a lot of trouble in Orbited to guarantee message order. It would be truly tragic if we destroyed that ordering right at the last possible moment. But its actually not a problem whatsoever. Regardless of how the various browsers implement setTimeout, they have managed to do it in a way that preserves ordering if multiple consecutive calls have the same timeout value. Of course, I would feel more comfortable if this was documented somewhere!

In light of the recent discussion about sockets and protocols and Comet, I’ve been thinking about browser protocols:

XMPP:

• Put IM on your website
• Manage XMPP components from a browser

IMAP:

• Write Gmail in 20 minutes

IRC:

• Collaborate on Open (and otherwise) Software from a browser

STOMP:

• Message Queue integration — Browsers can be first-class clients in MQ systems

We have already started the above choices with the Orbited project. But there are hundreds of additional protocol choices. What protocols do you want in the browser?

A few weeks ago MySpace invited me to give a presentation about using Orbited to create scalable Comet applications. While I can’t speak to any plans on the part of MySpace, I did get permission to post my speaking notes and slides. One point to keep in mind is that this presentation was made shortly before the naming of WebSocket was decided in the HTML5 spec, so the slides refer to the then-current name “TCPConnection”.

Slides

Speaking Notes

Slide: Orbited Logo + Michael Carter

-My name is Michael Carter
- Founder Lead developer of Orbited
- 2 years in development
- countless architectures tested
- now we’ve found the best
- javascript work

Slide: Comet

- Here to tell you about Comet
- Everything I’ve learned in two years of Comet work
- You probably already know what Comet is. But i’ll say it again just in case.

Slide: What is Comet

- Its about asynchronous updates in the browser (Pushing data)
- Not necessarily about a particular technology
- could be flash
- polling
- javascript hacks (streaming)
- Its about an interaction model — user is notified asap with new data
- Its about the applications that you can build with that interaction model
- Chat, clearly
- Financials
- Gaming
- Auctions
- Sports reporting
- Collaborative Editing
- Network monitor reporting / system administration
- Election Monitoring (Andrew Betts / London Paper)
- Air traffic control (no joke)
- To name a few
- In practice though, everyone wants Comet without the need for flash or bad performance of polling
- Just plain old js and html, low latency
- Which brings us to Transports

Slide: Transports
- These are the actual mechanisms for delivering data
- There are many

Slide: Iframe

- Originally we had script tag streaming
- but it made sense to put it in an iframe
- So we refer to it now as iframe streaming

Slide: Iframe Diagram

- Application page has a callback “receive”
- Finishes loading, creates and iframe
- Iframe causes an HTTP request to the server
- the server sends headers
- (transfer-encoding chunked for http 1.1)
- And then stops
- The key to this transport is incremental rendering
- Browser will try to display any open/close tag pair that it receives
- This includes displaying invisible elements, like script tags
- So the server can cause the browser to execute arbitrary javascript
-

Slide: Iframe Code
- Easy to implement
- Works in all browsers with the same code
- Suffers from..

Slide: Iframe Usability
- Loading Bars
- Hour glasses
- clicking noises?!

Slide: Iframe Unacceptable
- Modern applications can’t have clicking noises and hour glasses.
- The users would get too confused
- So Iframe streaming isn’t the most popular transport
- Instead, the most popular transport is…

Slide: XHR Long polling

- Long polling is what the cool kids do.
- (Meebo and Facebook, for instance)

Slide: XHR diagram

- Browser says, ‘Give me an event’
- Server says, ‘…’
- Server says, ‘event.’
- Browser says, ‘Give me an event’
- server says ‘…’
- Its cross-browser

Slide: Long polling code
- still easy to implement
- and no usability issues!
- seems like a big win
- but how does long polling performance compare to iframe streaming?
- Consider an island off the coast
- its surrounded by water

Slide: Island
- We want to send conveys to the island, so we build a bridge

Slide: Bridge
- Conveys can now go to the island as they please
- But maybe a big ugly bridge is not what we wanted… its too unsightly
- So instead we build a boat

Slide: A boat
- The convey arrives at the boat, and the boat takes it to the island
- no problem
- But what about the second convey that arrived just after the boat leaves?
- It waits for the boat to go to the island
- It waits for the boat to come back
- It gets on, and waits for the boat to go to the island
- 3x travel duration
- Well Iframe streaming is the bridge, and the boat is long polling
- Meaning, long polling suffers from a 3x latency performance hit
- But thats not all
Slide: Bandwidth comparison
- You also have to send headers all the time
- Your buddy says ‘hows it hanging’
- But the browser says ‘GET /some/url HTTP/1.1\r\nUser-Agent: …etc’
- And the server says ‘HTTP/1.1 200 ok\r\nContent-type….’
- your 15 byte message can easily incur 300 bytes of headers, or more!
- thats a 20x bandwidth increase in that case

Slide: Long polling unacceptable
- 3x latency is too much
- 20x bandwidth is too much

Slide: Strike 2, you’re out
- Strike 2, you’re out
- or so we thought

Slide: Htmlfile Object
- htmlfile object to the rescue

Slide: ActiveX!
- but safe for scripting
- hides usability issues

Slide: Garbage crossout
- leaks live connections on navigations
- But we can fix that by explicitly calling the garbage collector

Slide: Htmlfile works great
- Streaming
- But No side effects
Slide: Explorer only
- partial solution
Slide: Xhr Streaming
- transport for ff and safari
Slide: ready state 3
- don’t do one event per request/response cycle
- do a hundred. a thousand…
- ready state 3 saves the day
- callbacks for download progress from server
- responseText is just plain text without any inherent meaning

Slide: Manually parse
- So invent our own format
- parse it manually
- And we’re set for firefox and safari
- Without loading bars or anything
- and the same bandwidth and latency constraints of iframe streaming
- So we’ve got all the browsers
- Except opera

Slide: SSE

- Opera goes above and beyond the call of duty
- I don’t ask that browser’s make my life easy
- They can keep their nasty forest of terrible hacks
- All I ever asked from browsers is that they make it possible to do comet
- But Opera actually makes comet a first class citizen of the browser
- Create an event source, name a url, add a listener. The end.

Slide: Html5

- Then they went on to propose that the this be included in HTML5
- after discussions with other vendors, gained support, a new spec came up

Slide: deprecated

- so the new one isn’t implemented
- but thats okay
- we can use the deprecated one in opera 8+

Slide: Transport Conclusions 10.18.20
- Ten minutes to explain
- Years to discover

Slide: hacks
- And yes, its hacks, hacks hacks

Slide: aim carefully
- Developing these transports is like trying to throw darts at a board

Slide: moving target
- but the board moves after you’ve thrown the dart
- That is to say, just imagine the most frustrating game you can
- That game is twice as fun as building transports

Slide: sleek bridge
- We can avoid the boat transport (long polling)
- And provide great user experience
- get a nice looking bridge

Slide: sleek bridge image with orbited logo superimosed
- And orbited does just that

Slide: Scalability
- Once you have a way to do comet, you need to scale it
- where transports are frustrating, scaling is hard
- Step 1,

Slide: Vertical Scalability
- How do you put lots of users on one machine?
- Seems like such a hard problem, but really there’s a simple trick
- And once you know the trick, its possible
- Don’t use threads.
- People have tried this with servers like apache
- A reasonable thread count, of 20, will service 20 users
- A better user count is a 1000 though
- But a 1000 threads brings a system to its knees with context switches
- No comet, just thrashing

Slide: Comet + threads != scalable
- So use the little trick of omitting threads, and you’re golden
- Build event-based servers
- You don’t want your comet apps to be like nature preserves
- You don’t want your app to be a lonely desert with the occasional wanderer
- it should be a riot, with users tearing down the walls, sucking your resources dry, demanding more…

Slide: desert -> riot
- So use event based programming for your comet server
- And I guarantee, your app will be a riot, not a desert.

Slide: zzzz
This is the easy problem
The harder problem is the other “half” of scaling,

Slide: Horizontal Scalability
- If one server supports 1000 users, does 2 support 2000? and n, n-1000?
- This is a truly tough problem
- Really though, it depends on the application
- No one answer.
- But a few core rules are in order
- 1. Don’t share state between comet servers
- 2. Only do Comet in the Comet server. no Pub/sub, no jabber/game server, no load balancing, just comet
- 3. Treat comet as the last leg of the journey

Slide: Two Comet servers (sharded by user) doing pub sub (ajaxworld slide)
- For some reason, comet servers are all trying to provide a pub/sub api
- Developers like it, great.
- But it was already a hard problem before Browser’s existed
- And the people solving the problem, RabbitMQ/ActiveMQ, aren’t part of the comet community. (there have been minor inroads)
- Integrating streaming comet sith real publish/subscribe message queues would be great
- But, thats just a specific case of a larger issue
- The real issue with scaling comet applications is that
- Comet is difficult
- Comet imposes architectural constraints on web apploications that we aren’t used to (can’t do round robin balancing per http request)
- Its a moving part that has to be closely tied to the other pieces of the system
- But integrating comet with existing technologies would allow us to solve the horizontal scaling problem however these existing technologies already solve it.
- How could you scale a thick client deployment?
- We want that to be the only question to answer when scaling a comet application.
- So How do you integrate comet servers with X?

Slide: Offboard Comet integrated with Jabber
- This is an example of how a typical comet server is integrated with jabber.
- the browser maintains a connection with the comet server and the “bridge”
- The bridge is an http server (hopefully event based as well!)
- and also an xmpp client
- and also a “Comet client” (dispatch protocol)
- Message flow goes browser -> bridge -> jabber -> bridge -> comet -> browser

Slide: A lolcat tangled up
- This is a bit of a mess
- complex systems are hard to scale

Slide: previous slide (Offbard comet integrated with jabber)
- you can’t scale this with your typical web load balancer
- the mapping between comet servers, bridges, and xmpp servers are almost certainly not giong to be 1-1
- so you have the complex task of figuring out load balancing between each part
- and then you have to implement each client twice
- once for the bridge, once for the browser
- the bridge speaks xmpp to the jabber server, but json-chat (made up protocol) to the browser
- Clearly not the solution…

Slide: Victory march picture, maybe the planting of the american flag in some battle
- Standards save the day.
- HTML5 has specified a TCPConnection standard for the browser

Slide: TCPConnection (word)
- I’ve been working closely with whatwg/w3c to specify the protocol and api that solves all of our problems
- tcpconnection is just like a socket, but for javsacript in browsers
- It will
-traverse forward proxies
- speak ssl
- include cookies (auth integration)
- use a single socket for both directions
- revolutionize web applications
- Think about it — tcp connections revolutionized network programming in the 70s.
Slide: Iron Man screenshot during test sequence
- tcp/ip. Thats how dad did it, thats how america does it, and its worked out pretty well so far
- And we don’t even have to wait for browser vendors

Slide: The future is now (sci-fi-ish picture + words?)
- Orbited provides a TCPConnection to all major browsers, today.

Slide: Browser < - > Comet Proxy < - > Jabber diagram
- Orbited speaks comet transports on the front
- and plain old tcp on the back
- So you put it in front of ANY tcp server
- and the browsers can interact as if they’d connected directly to the backend server over tcp.
- and the server doesn’t need to be altered whatsoever.
- No need for a bridge or dispatch protocol
- Much less complex means easier to scale
- Load Balancing?
- We can simplify this picture still

Slide: Browser < - > ( comet Proxy + Jabber Diagram single machine)
- Treat the comet proxy as part of the server
- One comet proxy per backend server
- Now you can all sigh in relief

Slide: Sigh of relief
- Because Comet isn’t going to get in the way
- Scaling Comet applications horizontally isn’t a twofold problem anymore
- If you know how to scale a network application, then you can scale a comet application.

Slide: Jabber server cloud with half Comet proxies, half normal
- And whats more, there’s no difference between the browser and the desktop

Slide: Auth + persistence (a pair of keys and some kind of storage)
- There are a couple of details: authentication and persistence

Slide: Jabber server with comet proxy on left, web app on the right, sql server behind both
- Just have the network server and the web app coordinate on the back end
- put the sql server behind both
- User creates an account over the web app
- but can log in to jabber
- jabber saves logs to the database
- and the user can see those logs via the web app
- And your problems are over.

Slide:Conclusion (Bullseye (with orbited logo somewhere))
- Transports are frustrating, but Orbited implements them (indeed, a number of projects take our implementation directly)
- Vertical Scalability and threads don’t mix. Orbited use event-based network IO.
- Horizontal scalability is tough because you have two problems.
- If you can solve integration, then you can get a handle on horizontal scalability by normal means
- TCPConnection is the perfect integration point. It works with all past, present, and future tcp/ip servers. Out of the box.
- Orbited provides that functionality today.

A recent set of HTML5 discussions are changing the course of Comet. First, a recap of the last two years of Comet: With long-polling we set the bar to cross-browser push. With XHR streaming and ActiveXObject(’htmlfile’) we raised it to cross-browser streaming. With SSE we’ve been trying to raise the bar to native, cross-browser streaming. And there we’ve sat, hoping that browser vendors actually implement the latest SSE spec.

I say we’ve been selling ourselves short. All this time pushing for a native server->client streaming transport, but we still lack client->server streaming, and anything resembling a standard transport for bi-directional communication. The Holy Grail of Comet development has always been native browser support of a full-duplex, single-connection communication’s channel, otherwise known as a TCP socket. But we’ve been mired down in hacks so long that we’ve lost the vision.

No Longer. The HTML5 specification now offers WebSocket, a full-duplex communications channel that operates over a single socket. I have been listening closely, and in some cases contributing, to the process of ensuring that WebSocket will:

• Seamlessly traverse firewalls and routers
• Allow duly authorized cross-domain communication
• Integrate well with cookie-based authentication
• Integrate with existing HTTP load balancers
• Be compatible with binary data

The API of WebSocket is very straightforward. You create a WebSocket and point it at a url:

var conn = new WebSocket("ws://www.example.com/livedemo")

Then you attach three callbacks:

conn.onopen = function(evt) { alert("Conn opened"); }
conn.onread = function(evt) { alert("Read: " + evt.data); }
conn.onclose = function(evt) { alert("Conn closed"); }

And finally, you can send upstream data with a simple function call:

conn.send("Hello World")

The browser will perform an HTTP handshake with the target web server to determine support, and then a direct stream will be exposed via the onread and send functions. The uri scheme “ws://” is used for WebSocket connections, and the “wss://” URI scheme is for secure WebSocket connections.

After the handshake, bi-directional framed communication ensues. Each frame can be either binary or text, thus allowing for swapping the encoding mid-stream. You can find more information about the protocol itself at the network section of the whatwg HTML5 draft page

While the HTML5 specification is not in a finalized stage, the first public draft was published by the W3C in January 2008, and browser vendors have already began targeting features in the specification. The idea of putting a duplex channel into the spec is not a new one; the TCPConnection API and protocol was initially drafted more than two years ago. Unfortunately there were many significant problems with TCPConnection that held back browser adoption. Ian Hickson, the editor of the HTML5 specification, tackled these problems head on and under his guidance the standard has evolved to a usable state with WebSocket. About this new feature, Mr. Hickson comments:

Comet

WebSocket in a browser is terrific because it drastically cuts down the complexity of the Comet server by an order of magnitude. What’s more though, it provides a straightforward, understandable API to JavaScript developers. The most important part of the specification is that developers can wrap their heads around the API in about five seconds. That’s because it looks so much like a socket.

If the future prospect of a native WebSocket isn’t enough good news, I am proud to announce that the Orbited project has implemented WebSocket for all major browsers, today. We do this by communicating over various Comet transports with the browsers, then performing the WebSocket handshake with the remote server, and proxying data in between. This means that today you can write a WebSocket server and application, start Orbited up, and be on your way. Tomorrow, you won’t need to change any of your server or client code whatsoever. Your application will fall forward to the native implementation of WebSocket for improved performance.

TCPSocket

The single most voiced criticism to this specification has been that a WebSocket isn’t quite the same as a raw TCP socket, because a WebSocket server needs to understand a specific handshake in order for browsers to connect directly, and as such a WebSocket can’t connect to existing servers. If we did allow raw TCP sockets in the browser, a malicious site could cause any visitors to open up a TCP connection to an SMTP server, for instance, turning a casual web visitor into a spambot. There are many variations on this scenario, but the general problem is that a raw socket connections in a browser will allow any sites that a user visits to access network services as if they were the user, in the same network security context as the user. We need to therefore make this an opt-in process or we’ll catch existing servers off-guard. Furthermore, very few protocols have any kind of cross-domain authorization or security mechanisms built in. If we were to allow raw TCP, then we would be opening all manner of cross-site security holes. We could fix these by limiting TCP connections to the origin domain and port, (meaning a direct sockets back to the webserver only) but that would limit any usefulness the TCP socket could provide.

I fully understand the criticism though; Earlier this week I discussed exactly why having a raw socket in the browser is so desirable. You could, for instance, quickly prototype a Gmail clone using a raw socket, an IMAP client, and an XMPP client in the browser.

We have a clear problem then: Direct access to existing network servers could greatly simplify application architecture, but due to security restrictions it’s a non-starter; we absolutely must retrofit each network server with the new WebSocket protocol first. I hope that happens, but we can’t count on it, at least not right away. What we really need is a way to allow the server to opt-in without putting it in the protocol, a way to seamlessly layer access control in front of the back-end server. It turns out that this problem has already been solved for traditional networks. That is, if we have two end-points communicating over TCP, and we need transparent access control in between, then we can use a well known device: A firewall. The beauty of a firewall is that server behind it requires no re-programming, or even re-configuration, yet gains all of the access control/security benefits. What we really need in the browser case, is a custom firewall that can listen for WebSocket connections from the browser, enforce access control, and relay TCP to a back-end server.

That is why Orbited provides this feature under the API name TCPSocket. Orbited is the firewall that sits between the back-end server and the browser. It understands WebSocket protocol for browser communication, and uses whitelist security to accept or reject requests to proxy TCP data to and from a back-end server. That’s right, you can fire up a stock XMPP server, and Orbited, and write the XMPP client entirely in JavaScript. This works cross-browser today. We also offer a binary mode that uses an intermediate encoding to allow the browser to read raw bytes (in the form of JavaScript integer arrays) from a remote server. Here is a diagram of the architecture:

The Future

Now its up to the browser developers to implement Websocket. I expect some will be very quick on the uptake, while it may take years for others. I expect to see a common pattern emerge where application servers listen to the WebSocket protocol directly from new browsers, but fallback to Orbited’s emulation layer for legacy browsers. The key here is that we don’t have to wait on the browser vendors to get started. We can all develop these WebSocket applications now, and when browsers have native support, we’ll all get a performance boost.

We will probably never get a native (raw) TCP socket in the browser, for the security reasons I outlined. It’s okay though — we can use the firewall pattern I outlined. For more information about installing and configuring Orbited, check out the documentation and the getting started section.

I am pleased to announce, with the Orbited 0.5.x series, we provide a TCP socket for the browser.

When I used tell people about Comet, I’d have to ask if they’ve heard about Reverse Ajax, or HTTP Push. Or a number of other arbitrary phrases for the same technology. “You know, where you have data on the server, and you need to get it to the browser without the browser needing to ask for it.” Now I don’t have to deal with that, because very, very few people have heard of Reverse Ajax and not also heard of Comet. But still, when I walk into an arbitrary user group meeting, I ask who has heard of Comet, only about 25% of the audience raises their hands.

Now I have a new tactic. I ask, “How many of you have heard of Comet? Keep your hands up…” and a quarter have their hands up. “If you’ve heard of a socket — yes, a TCP socket — then also put your hand up.” And at this point, the entire audience has their hands raised. With the Orbited project we’ve found our explanation of purpose: We provide a socket API to the browser using the best methods available, which in most cases are Comet-based. This week I can make it official because for all intents and purposes, the Orbited project provides a TCP socket in the browser.

The name of the API is TCPSocket and you can use it like this:

var conn = new TCPSocket(hostname, port)
 
conn.onopen = function() { alert('connection opened!') }
conn.onread = function(data) { alert('RECEIVE: ' + data) }
conn.onclose = function(data) { alert('connection closed!') }
 
conn.send('Hello World');

The above code code is all you need to know. It will open a TCP connection to hostname:port, and will send the data “Hello World”. Any data the server sends back will be alerted with the onread handler.

The exact mechanism behind this innovation is pretty straightforward: Orbited is a router and firewall for incoming TCPSocket connections from the browser. It uses Comet techniques to establish bi-directional communication with the browser, then forwards all data over a raw TCP socket to the end point. Configuration allows access control enforcement so that the TCPSocket can only be connected to pre-approved end-points, so that the Orbited server isn’t an open relay.

While this presents a paradigm shift in the way developers are tackling real-time, web-based applications today, it’s actually a return to the original method of writing network applications. Instead of writing applications based on web frameworks and throwing a Comet server in the mix, you can simply use the client-server architecture where the browser is the client, and the server is an arbitrary TCP server. Let’s take Gmail as a brief case study.

The Gmail application features a nice gui that clearly took some time to build. The real trouble with building Gmail is figuring out a way to bridge email notification to the browser. You could create a server that uses IMAP to discover the new messages, then use a Comet transport to push those messages to the browser. You also need to write a similar adapter for XMPP. That means you’ll need a server that acts as an XMPP client for each user and transcodes the XMPP protocol to something JSON-based for easy access in the browser. You’ll need to then integrate these and find some way of doing load balancing. While this isn’t rocket science, its damn hard.

On the other hand, assume that your base tools include a pure-JavaScript IMAP and XMPP client in the browser. On page load the browser can ask the IMAP server directly for the 20 most recent emails, including their send times, read status, and subject. Next, the browser can issue an IDLE command which will cause the IMAP server to send immediate notification of any new mail. The browser also can connect to the XMPP server and send/receive presence information to/from the user’s buddy list. Message dialogs and typing notifications are all handled by callbacks attached the XMPP client, and sending messages to buddies is a trivial one-liner. There’s not a whole lot else to do besides make a great user interface, which can take as much time as you can give.

I’m not trying to cast a shadow on the engineering behind Gmail — its a brilliantly engineered product. We’ve been stuck with web application servers (read, jumped up HTTP servers) for so long though, that we take complicated architecture for granted. I’m just pointing out that you don’t need to be a brilliant engineer to create Gmail if you have the right tools. The right tools aren’t web application servers, they’re sockets and clients, and with those tools we could put together a tutorial titled “How to write your own Real-time Email/Chat application in 20 minutes.”

The Future

Once you have a TCP socket in the browser, no web application architecture is impossible, or even that hard. Only now it is apparent how far JavaScript lags behind all other programming languages for networking support. We need to start developing every major protocol in JavaScript. For now, Orbited ships with STOMP (ActiveMQ and RabbitMQ), IRC, and XMPP implementations, but we plan on soon shipping a Daemon that serves implementations of all major communication protocols.

These developments will transform the way we think about the web. Many thick-client systems no longer need to be re-engineered for the web — its just a matter of writing a user interface. It is conceivable to write a fully featured browser application with no web application server whatsoever. All the applications that use complicated architectures to bridge various protocols to the browser over HTTP can now be simplified many times over. JavaScript developers no longer have to live in a ghetto; we can use a socket just like in all other programming languages.

I will be giving a talk this Saturday titled “Impact of the Reverse AJAX (Comet), TCP Connection and HTML5 to the architecture of the Web” at the iPhone BarCamp this Saturday in Palo Alto, CA.

There are under ten slots still available for this event (registration cap is 100) so if you’re interested, you can still register. This talk will cover a range of Comet-related topics, including what Comet is and how it works, how it will impact the architecture of your application, and the formalization of Comet with regards to the HTML5 specification. Additional information can be found at the BarCamp Wiki. I hope to see you there, but if not, I’ll post my slides and speaking notes for later review.

I’ve recently started a new job at Kaazing, a company focused on re-engineering the Web for real-time. Kaazing’s vision for the future of Comet closely parallels my own vision, allowing this architectural position to be mutually beneficial both for Kaazing’s Enterprise customers and the Orbited project. To those ends, Kaazing has generously allotted a substantial portion of my hours towards committing improvements directly to the Orbited codebase. Many of those improvements will be soon made available with the upcoming 0.5 release of Orbited. Furthermore, Kaazing is encouraging my direct involvement with the HTML5 specification as a forum for driving Comet standards. These exciting developments are great news for me, Orbited, Kaazing, and the entire Comet community.

Michael Carter says that his concerns about the Bayeux protocol have gone unanswered, and that Greg Wilkins’ and Alex Russell’s arguments for Bayeux have focused on peripheral issues.

Part 1: Greg Wilkins explains the need for Bayeux
Part 2: Michael Carter criticizes the current state of Bayeux
Part 3: Greg Wilkins responds to Michael Carter
Andrew Betts’ thoughts (from a related article)
Part 4: Michael Carter responds to Greg Wilkins
Part 5: Kris Zyp’s thoughts
Part 6: Alex Russell responds to Michael Carter
Part 7: Michael Carter responds to Alex Russell

I’ve already made clear, technical arguments about Bayeux’s flaws, but those issues still haven’t been addressed. In his original article, Greg brought up multi-frame connection sharing as one of Bayeux’s core features, and in an effort to keep the dialogue centered on facts I pointed out that Bayeux actually doesn’t solve the problem whatsoever. Alex, on the other hand, has a different opinion on the matter. But while Greg is claiming that Bayeux’s strength is that it supports connection sharing, and Alex is proclaiming that actually it’s the lack of that support that is the real strength, let me say that I honestly couldn’t care less about multi-frame anything; a proposal to solve this was in the works long before I even entered the realm of Comet development. There are so many more important issues.

Bayeux is a bad foundation for Comet communication. It implies the wrong API, and by not specifying an actual API, an application can’t swap one Bayeux client out for another. It lacks any message reliability or ordering guarantees. There is no standard way of integrating new transports—that’s completely unspecified. The worst part is that these flaws are, according to Alex, not errors of omission; rather, they are by design. The Bayeux authors don’t care about client interoperability or transport interoperability, which are the only two parts of Comet that are even possible to standardize. Instead they’ve chosen a communication protocol (note my previous musing that lacks many necessary features, demands the use of publish/subscribe messaging, and will almost certainly be obsolete within the year, when long-polling, the “critical core” of Bayeux, goes out of style.

On the other hand, the API for bi-directional communication has been known for more than forty years. It’s a socket, and it supports open, close, send, and receive. Long polling happens to be one way to implement the receive method, and as Alex suggests, it’s far from the best. We already know we want nothing to do with the current methods of communication a few years down the line, so let’s stop trying to standardize a communication protocol and instead choose a good API. Why on earth would we have to hem and haw on the ideal standard Comet API when it’s so clearly a TCP Socket, and in fact already defined in HTML5? I say, declare the TCPConnection as the base for Bayeux’s communication and be done with it. When all the browsers support bi-directional communication, preferably via the actual TCPConnection specification, then we can throw out all the hacks and get on with our life. And in the meantime, the rest of us aren’t saddled with the publish/subscribe layer that’s a useless burden to many applications.

It makes the most sense to say that a standards-compliant Comet server is something that serves some JavaScript that exposes the TCPConnection API. How the JavaScript actually implements that is something that we can “compete about”, as the Bayeux authors like to put it, but higher level functionality like publish/subscribe or application-specific protocols can be implemented on top of the TCP socket. Developers just want a socket. It’s unrealistic to foist a messaging protocol API on everyone and say “live with it.” Not all developers want it. Those who want a messaging API (ala Bayeux) can use go use one, and it won’t hurt the rest of us.

Of course, Bayeux also tries to allow interoperability between an arbitrary client and server. I detailed previously exactly why I think that this is absolutely the wrong target for public standardization, because the important interface is between applications developers and a Comet implementation, not between Comet servers and clients. That said, we would do well to have our own APIs and even protocols that allow us to share client side code. In particular, we need a Comet transport API that defines a simple interface for connecting to a remote server, identifying the connection to that server, and receiving data. It also makes sense to allow an implementation of this API to specify an upstream communication mechanism. A Comet client would really just be an implementation of a TCP socket built on top of the Comet transports. If Orbited defined a fast new Comet transport, anyone’s Comet TCP implementation could use the code without modification.

The story I’m outlining for interoperability is that your browser-side application includes a comet.js from the Comet server, which exposes the TCPConnection API. If you want a new Comet server, just start it up and link to that new server’s comet.js file. In the end, the interface is always the same. If developers want publish/subscribe, then they can use Bayeux, which would be built on the TCPConnection interface. This is the ideal world for application developers, and even for the Bayeux authors who want to push a particular mode of interaction for Comet. They can worry about publish/subscribe but have a reasonable communication layer. This would work out well for server developers too, as we would probably end up sharing much of the comet.js code that implements the transports and protocol necessary to expose the proper interface, but we wouldn’t muddy the waters for the general public—they’d just have a single interface to worry about. In the end, we’ll get some kind of native TCP connection in a browser, and suddenly Comet is irrelevant, and our applications continue to work, but even more efficiently.

Simple integration with current technologies is a very real problem standing in the way of widespread Comet adoption. Developers are trained to use a language or framework like PHP, Django, or Rails, where all state information is pushed into the database. There is generally no state beyond what’s needed for the duration of the request. But real-time applications need to keep track of more information and access it more often. Even some very trivial functionality is difficult to integrate with these frameworks.

Consider a Comet application where the application broadcasts a single piece of information every thirty seconds. An administrator can change that piece of information by logging into the application via a web-based GUI. Given a framework like Django, the GUI would be obviously simple to make. But what about the timer? This involves creating a separate thread for the timer itself, and could potentially be complicated depending on kind of server Django happened to be running on top of. This simple app turns out to be very hard to implement in practice due to the nature of synchronous web frameworks.

The timer may seem trivial, but what if your app needs to keep track of connection state? The application could send a PING command out to all of the clients every 30 seconds. If the application doesn’t get a reply (via an XHR requests, let’s say) before the next ping is broadcast, then that browser would be considered disconnected. There is a very real need for this type of feature in many Comet applications, but it is particularly hard to implement with synchronous frameworks.

This could be solved with two steps:

1. 1) Having the Comet server handle connection state, including pings and ping responses.
2. 2) Having the Comet server make HTTP callbacks to the application when timeouts occur.

Now the application need only implement a timeout callback as a particular URL, like any other function of the web application; it’s just an HTTP request with some form parameters. The application can make a note of that disconnect in the database, or inform the other clients as needed. It could even store undelivered messages (from the body of the callback request) in the database for future delivery. The best part is that no timers or any other complicated mechanisms are needed.

In Orbited 0.4 we are including some pre-defined callbacks to make development with synchronous frameworks much easier. These callbacks include message delivery success/failure, signon, and signoff/timeout. We have future plans to include publish/subscribe-specific callbacks so that the web application need only implement a single URL as the publish callback if it needs to react to published information. We also are including callbacks for authentication, which will make it trivial to tie publish/subscribe security to the application’s pre-existing authentication framework.

I believe that the future of widespread Comet adoption is to provide a scalable server that can be easily integrated with existing methods and technologies. No one wants to completely re-learn web development, but they may be willing to learn a couple simple steps towards achieving Comet integration. Providing a simple mechanism to integrate real-time functionality with PHP, Rails, or Django will be a huge win for the Comet community.