Webtide
SitePen Support

Sockets in the Browser

by Michael CarterJuly 1st, 2008

I am pleased to announce, with the Orbited 0.5.x series, we provide a TCP socket for the browser.

When I used tell people about Comet, I’d have to ask if they’ve heard about Reverse Ajax, or HTTP Push. Or a number of other arbitrary phrases for the same technology. “You know, where you have data on the server, and you need to get it to the browser without the browser needing to ask for it.” Now I don’t have to deal with that, because very, very few people have heard of Reverse Ajax and not also heard of Comet. But still, when I walk into an arbitrary user group meeting, I ask who has heard of Comet, only about 25% of the audience raises their hands.

Now I have a new tactic. I ask, “How many of you have heard of Comet? Keep your hands up…” and a quarter have their hands up. “If you’ve heard of a socket — yes, a TCP socket — then also put your hand up.” And at this point, the entire audience has their hands raised. With the Orbited project we’ve found our explanation of purpose: We provide a socket API to the browser using the best methods available, which in most cases are Comet-based. This week I can make it official because for all intents and purposes, the Orbited project provides a TCP socket in the browser.

The name of the API is TCPSocket and you can use it like this:

var conn = new TCPSocket(hostname, port)
 
conn.onopen = function() { alert('connection opened!') }
conn.onread = function(data) { alert('RECEIVE: ' + data) }
conn.onclose = function(data) { alert('connection closed!') }
 
conn.send('Hello World');

The above code code is all you need to know. It will open a TCP connection to hostname:port, and will send the data “Hello World”. Any data the server sends back will be alerted with the onread handler.

The exact mechanism behind this innovation is pretty straightforward: Orbited is a router and firewall for incoming TCPSocket connections from the browser. It uses Comet techniques to establish bi-directional communication with the browser, then forwards all data over a raw TCP socket to the end point. Configuration allows access control enforcement so that the TCPSocket can only be connected to pre-approved end-points, so that the Orbited server isn’t an open relay.

While this presents a paradigm shift in the way developers are tackling real-time, web-based applications today, it’s actually a return to the original method of writing network applications. Instead of writing applications based on web frameworks and throwing a Comet server in the mix, you can simply use the client-server architecture where the browser is the client, and the server is an arbitrary TCP server. Let’s take Gmail as a brief case study.

The Gmail application features a nice gui that clearly took some time to build. The real trouble with building Gmail is figuring out a way to bridge email notification to the browser. You could create a server that uses IMAP to discover the new messages, then use a Comet transport to push those messages to the browser. You also need to write a similar adapter for XMPP. That means you’ll need a server that acts as an XMPP client for each user and transcodes the XMPP protocol to something JSON-based for easy access in the browser. You’ll need to then integrate these and find some way of doing load balancing. While this isn’t rocket science, its damn hard.

On the other hand, assume that your base tools include a pure-JavaScript IMAP and XMPP client in the browser. On page load the browser can ask the IMAP server directly for the 20 most recent emails, including their send times, read status, and subject. Next, the browser can issue an IDLE command which will cause the IMAP server to send immediate notification of any new mail. The browser also can connect to the XMPP server and send/receive presence information to/from the user’s buddy list. Message dialogs and typing notifications are all handled by callbacks attached the XMPP client, and sending messages to buddies is a trivial one-liner. There’s not a whole lot else to do besides make a great user interface, which can take as much time as you can give.

I’m not trying to cast a shadow on the engineering behind Gmail — its a brilliantly engineered product. We’ve been stuck with web application servers (read, jumped up HTTP servers) for so long though, that we take complicated architecture for granted. I’m just pointing out that you don’t need to be a brilliant engineer to create Gmail if you have the right tools. The right tools aren’t web application servers, they’re sockets and clients, and with those tools we could put together a tutorial titled “How to write your own Real-time Email/Chat application in 20 minutes.”

The Future

Once you have a TCP socket in the browser, no web application architecture is impossible, or even that hard. Only now it is apparent how far JavaScript lags behind all other programming languages for networking support. We need to start developing every major protocol in JavaScript. For now, Orbited ships with STOMP (ActiveMQ and RabbitMQ), IRC, and XMPP implementations, but we plan on soon shipping a Daemon that serves implementations of all major communication protocols.

These developments will transform the way we think about the web. Many thick-client systems no longer need to be re-engineered for the web — its just a matter of writing a user interface. It is conceivable to write a fully featured browser application with no web application server whatsoever. All the applications that use complicated architectures to bridge various protocols to the browser over HTTP can now be simplified many times over. JavaScript developers no longer have to live in a ghetto; we can use a socket just like in all other programming languages.

[Slashdot] [Digg] [Reddit] [del.icio.us] [Facebook] [Technorati] [Google] [StumbleUpon]
Comet Support by SitePen

33 Responses to “Sockets in the Browser”

  1. Joe Walker Says:

    It’s a minor point. With DWR, we use ‘Reverse Ajax’ when we’re talking about comet or polling, or any other mechanism of getting async data to the client.

    Reverse Ajax != Comet
    Reverse Ajax = Comet + Polling + Piggyback

  2. Jörn Zaefferer Says:

    That sounds great and well, but like everything about Comet, the lack of documentation makes it hard to get started.

    Apart from the rather slim wiki page, there is no information about the implementation: What does it mean to have a JavaScript TCP Socket? Is it just a wrapper API for other techniques or something brand new? Is it strained by the usual two-connection limit? Does it scale?

  3. Steffen Says:

    Michael,

    I think this is a great idea.

    However, you always need to go through port 80 for consumer apps, as many firewalls block any ports other than 80, right? Also, how will proxy servers handle your requests/data when you are not using HTTP?

    Steffen

    (apologies, I replied to the wrong posting earlier)

  4. Marcus Cavanaugh Says:

    @Steffen: This TCP Socket technique involves using a comet server (Orbited) as a *router* to essentially proxy a socket connection to a browser, with the last mile being HTTP.

    I’ve always thought that the comet servers, up through now, have been going about finding a solution inefficiently. All we’re trying to do is emulate a streaming connection, so why not make it *act* like one? If HTML5’s sockets go through, a socket interface for the browser will be standardized anyway — so if we adopt that paradigm now, when support for sockets actually _comes_ in the browser, our applications will be ready.

    Applications should not care about the low-level protocol used to usher data over the wire. Sockets hide all of the complexity associated with different comet hacks, allowing application developers to concentrate on what matters most. A higher-level message protocol can always be built on a stable foundation.

  5. Brian H Says:

    “While this presents a paradigm shift in the way developers are tackling real-time, web-based applications today, it’s actually a return to the original method of writing network applications.”

    Please, get over yourself.

    Your gmail critique is ridiculous. It’s already proven to be a viable implementation using “just” AJAX. Moreover, they don’t kill resources on their servers by allocating a open socket to each user.

  6. Ajaxian » TCPSocket: Sockets in the browser Says:

    [...] Carter of Orbited has written about how he now likes to call Comet sockets in the browser, and has an implementation available that looks like this: PLAIN TEXT [...]

  7. Charlie Says:

    I’ve been fooling around with using the browser as a TCP client for a while now, using Flash as a gateway (via AFLAX) and a super simple JAVA socket server.

    The nifty bit is that the JAVA is implemented via Rhino, enabling me to do all of the code on both the server and the client in javascript. I’m running Prototype on the server side, it’s fun.

    I’ve built a basic IRC-like chat server out of it so far. I’m no code guru so I’m sure it will need a lot of work to be scalable and robust in the wild, but it’s stable enough so far in small controlled tests.

    Feel free to email tenebrious_mockingbird (a-t) yahoo for the source if you’re interested.

  8. rektide Says:

    where is a live demo (so we can snoop your protocol) and where are the protocol specs? news is fun and all, but theres a lot of ways you could implement this and really thats where all the fun parts will be.

  9. rektide Says:

    how much of the information on the blog about the skipped orbited .4 release is true?:
    http://orbited.org/blog/2008/02/upcoming-orbited-040-information/

    did stomp get implemented?

  10. Phill Kenoyer Says:

    Yeah, I’ll just keep my Flash socket client. It’s much easier and works better. AFLAX, Flex, Juggernaut.

  11. Geoffrey Lee Says:

    Looks really interesting. Does the last mile HTTP connection offer the same TCP delivery and packet ordering guarantees?

  12. Jeff Bisbee Says:

    You have been able o get sockets in javascript via both java and flash for some time now and now with ActionScript 3 you actually have binary sockets and compression. At work we’ve been doing this since 2001.

  13. Geoffrey Lee Says:

    Jeff, the problem with client-side sockets is that they’re blocked when people use proxies. This is particularly problematic in corporate environments. HTTP tunneling is a solution to this problem, and that’s one of the reasons for the Orbited TCP Socket.

  14. Frank Salim Says:

    Brian:
    If you are bridging to XMPP, you need to have one open socket per client as well. The criticism of Gmail is not that it doesn’t work, but that there is a much simpler way to build a similar application if you have access to a socket.

    rektide:
    The 0.4 development branch is pretty different from 0.5. Orbited comes with a STOMP client in JavaScript that uses TCPSocket and runs in the browser, so you can dispatch messages to the browser using STOMP and a STOMP broker. It is in /static/protocols/stomp, and there is a demo in /static/demos/stomp served by the Orbited daemon. (It expects a STOMP broker running on localhost:61613.)

    Geoffrey:
    Yes, message ordering and reliability are preserved within an Orbited TCPSocket stream.

  15. Steffen Says:

    This sounds really great!

    How stable is the current version of Orbitd? Does it work with FF and IE?

    Does Orbitd scale well (consumer apps)?

    Thanks!

  16. JS Says:

    This is stupid and pointless. The whole point of Javascript and browsers is to not have to install external applications. Orbited requires users to install Python with the Twisted framework and this Orbited package on top. Not only is this forcing a reliance on a 3rd party app, but this won’t work well for machines behind firewalls that restrict all traffic to ports 80 and 443.

  17. Michael Carter Says:

    JS: I think you misunderstand both Orbited and Comet. Orbited runs on the *server* side, and listens on port 80 and 443 (or any port you like.) Therefore, users install *nothing* and can connect to remote serves on any port by proxying through Orbited.

    For instance, if you navigate to the Orbited homepage ( http://www.orbited.org ) and click on the prominent “Live Demo” link, you will notice that it works without you, the user, installing anything whatsoever.

  18. Steffen Says:

    Michael,

    The live demo is not working. Nothing happens after you enter your nick name - there is no button ie. “Sign In”, nor does the RETURN key work. You might want to have a look at this, as everyone wants to try Orbited using this demo. It might be a browser issue, but I am using FF 2, so it *should* work.

    Thanks,

    Steffen

  19. narcvs - spun thoughts » Blog Archive » Ajaxlights: TCP, bindings, jQ file tree, on-demand js Says:

    [...] “TCP sockets” in the browser: [...]

  20. Comet Daily » Blog Archive » Independence Day: HTML5 WebSocket Liberates Comet From Hacks Says:

    [...] fully understand the criticism though; Earlier this week I discussed exactly why having a raw socket in the browser is so desirable. You could, for instance, quickly [...]

  21. Orbited Blog » Blog Archive » Socket Proliferation Says:

    [...] discussing browser sockets,, I’ve been doing some research on other implementations. For some time now, David Davis has [...]

  22. Hari Says:

    Dude,
    I finally see real time web happening..:)
    This is great!! Thanks a ton!

  23. Comet Daily » Blog Archive » TCP Sockets and Athena Says:

    [...] Carter’s recent Sockets in the Browser has received significant interest. JP Calderone wrote an example of a TCP sockets implementation [...]

  24. Orbited Blog » Blog Archive » Comet for the Non-Web Programmer Says:

    [...] solving formerly complex problems in minutes instead of months (see Michael’s discussion of gmail), with minimal code and zero “web programming.” Because let’s face it: “web [...]

  25. Orbited Blog » Blog Archive » Talk at OSCON 2008 Says:

    [...] Sockets in the Browser [...]

  26. Shaun Says:

    forgive me for being ignorant..

    but how is this a PUSH protocol??
    All i see is the browser is sending a packet to the server, and waiting for a reply.

    There are no methods for the server sending information to the browser..

    Am i missing something here??

  27. Orbited Blog » Blog Archive » Integrating Orbited with Web App Frameworks Says:

    [...] clients in many languages. These benefits are similar to the benefits of the biggest change in 0.5: TCPSocket, which lets you use existing protocols for communication to the [...]

  28. Comet Daily » Blog Archive » Expanding Cross-Site Comet Options Says:

    [...] it has the advantage of being secure (the target site can not exploit the requesting site). Note:Orbited directly exposes the WebSocket API in their Comet implementation, making it easy for Orbited users to switch to using the native [...]

  29. Comet Daily » Blog Archive » Why Flash must adopt Comet Says:

    [...] The browser implicitly routes Comet requests through the proxy because they are HTTP. As I’ve noted previously Comet on its own isn’t the best developer-facing API to deal with; a socket would be much [...]

  30. Ayden Says:

    Can someone explain something in english for me please.
    I want to use Adobe AIR, and avoid java script, and prefer PhP and HTML.
    So what is the difference between FLASH socket, BlazeDS, COMET and Orbited?
    All I can tell is that Long Polling consumes a lot of CPU resources i.e. 100-200 users max per CPU which isnt good when you want tens of thousands of users..
    Whats best to use with AIR? which uses HTML/PhP

    Please help!

  31. Anonymous Says:

    While not saying anything about author’s use cases, I suggest,
    that this techique can be easily modified to provide a proxy for other protocols like UDP or SCTP
    and make these accessible as UDPSocket and SCTPSocket within browser.
    Again, no plugins and OS support for these protocols required on the client side, and that is cool.
    Thanks.

  32. Peter Says:

    Great concept but the back-end seems pretty heavy - wouldn’t it be possible to achieve this with PHP instead of Python? PHP supports sockets and there are various ways of getting around script timing limitations.

    thx,

    – p

  33. MultiUser Chat using XMPP and Orbited (Using Ruby-on-Rails) « Sid__ Says:

    [...] we begin try running this snippet obtained from Micheal Carter’s Sockets in the Browserarticle on CometDaily.com. Add this snippet to the _tcpsocket.html.erb partial that is included in [...]

Leave a Reply



Copyright 2014 Comet Daily, LLC. All Rights Reserved