Introducing ItsNat, a New Java-based Comet Tool

by Jose Maria ArranzAugust 25th, 2008

ItsNat is a new Java web framework with Comet capabilities. In this article we’re going to see how ItsNat fits in the ecosystem of Comet technologies, and how ItsNat works and provides support for Comet.

When Dylan Schiemann invited me to write for Comet Daily I thought: “What can I contribute to Comet Daily?” I’m not involved in client-side technologies (e.g. Dojo) because I’m a supporter of server centric development so I don’t have experience with Comet oriented servers typically controlled by JavaScript code in the browser (Lightstreamer, Caplin, Orbited, Liberator, Cometd, etc.).

I’m the author of ItsNat, a Java web framework with Comet capabilities. In the beginning of ItsNat development I knew of Comet, but it wasn’t in my list of ideas for ItsNat until I realized that Comet was a perfect fit. Today, the most interesting ItsNat features like remote page monitoring, server-sent events and functional testing are based on Comet.

In response to my initial question, “What can I contribute to Comet Daily?”, I realized that I can be the “Java Web Framework guy” of Comet Daily. This auto-assigned “position” is interesting for Comet Daily because adds a new piece to the Comet puzzle:

  • Lightstreamer, Caplin, Orbited, Liberator, Cometd, erlycomet, Persevere: are Comet servers. In a web environment “the application” is usually coded in JavaScript.
  • Dojo: a powerful JavaScript library ready to talk with Comet servers using the Bayeux protocol (for instance with Cometd), XMPP, and other Comet protocols.
  • Jetty & Grizzly: both provide servlet implementations with scalable asynchronous requests that are very useful for Comet. Among other things, they provide low level infrastructure for Comet in the Java server side.
  • DWR: a browser-servlet bridge using Ajax. DWR can perhaps be thought of as a “low level” Java web framework for Ajax and Comet (in addition to its remoting capabilities).

Where does ItsNat fit in this scenario?

ItsNat is a new technology. A Google search for Java web framework Comet surprisingly shows an article about ItsNat on the first page of results.

With the exception of DWR and IceFaces, Java web frameworks generally do not provide integrated support for Comet. I think this is going to change because Comet in my humble opinion is going to be one of the most influential technologies in the future, changing how we build and use the web. Of course, Flex, Silverlight even JavaFX will have a role and will use Comet, but if the Open Web continues to flourish, Comet, using pure web technologies, will have a very important role in “Web 3.0″. If “Web 2.0″ = web + Ajax (from a technical perspective), will be “Web 3.0″ = web with Ajax + Comet = real time web?

ItsNat and DWR comparison

ItsNat is a server centric Java web framework with out of the box” Comet capabilities. It might be perceived as being in the same space as DWR, but there are several key differences with ItsNat:

  • Provides a complete infrastructure for building web applications. No additional JavaScript library is needed (but of course can be used).
  • Promotes server centric development, no user defined JavaScript is coded, and the view logic is coded in the server generating the necessary JavaScript to update the view.
  • Like DWR, custom JavaScript code can be sent to the browser. Furthermore, ItsNat automatically sends any changes to the view performed “in the server” to update the client accordingly in a way transparent to the developer.

There is actually not much overlap between DWR and ItsNat, because DWR is focused on browser centric development and ItsNat on server centric development.

How ItsNat works

ItsNat approach to web development is called TBITS, “The Browser Is The Server”, because ItsNat simulates a Java W3C browser in the server:

  • Keeps a DOM tree in the server of the page seen by the user. The browser DOM is a clone of the server DOM. Most of the work of the framework is to synchronize both trees.
  • If the DOM tree changes in the server using Java W3C DOM (Level 2) APIs, this change is automatically detected by the framework and custom JavaScript code is generated and sent to the client to update the DOM client tree, usually as the result of a client to server request.
  • Any server DOM tree element can receive client events that are transported to the server using Ajax. These browser events are received as Java W3C DOM Events and dispatched to the Java W3C DOM EventListeners registered in the server. A client event is transported to the server only if a server listener matching the target node is registered.

Load Process

Event Process

The main benefit of this approach is the templating technique: ItsNat templates are pure X/HTML or SVG files because any view logic is coded using plain Java calling the W3C DOM APIs. The way to code view logic in the server with ItsNat is basically the same as JavaScript is used to manipulate the DOM in the client.

ItsNat provides some typical components like buttons, labels, lists, tables or trees. Built on top of the basic infrastructure described previously, the use of these components is optional and with little hassle, any developer can build their own custom components using Java W3C DOM and DOM Events APIs.

ItsNat is focused on Ajax, and doesn’t provide much for classic page based navigation. I’m a firm believer in the “One Single Web Page” (OSWP) paradigm, especially for web applications. To help with the OSWP approach, ItsNat provides page fragments, a pure X/HTML or SVG page where only the markup contained in the body element is used. The page fragment is loaded as DOM and used as the pattern to replace some or all of the main page with no page navigation. In this OSWP scenario, Comet may have a very important role.

A very brief overview of Comet in ItsNat

Except for the initial page load, server to client communication is performed using JavaScript (Ajax events). When the server code changes the server DOM, the JavaScript code generated to update the client is queued until an Ajax request is processed. The developer can add custom JavaScript code to the queue at any time using a special method (addCodeToSend). A received Ajax request clears the queue and transports the JavaScript code to the client updating the client view and executing the custom user code. Nothing prevents a non-web thread from changing the server DOM state and queuing custom JavaScript code to be sent to the client when the next Ajax request arrives.

ItsNat provides traditional polling and long polling, fully controlled in the server. The polling technique uses client timers and the long-polling version uses a special method (notifyClient) to finalize a pending request to transport any pending JavaScript code and to recreate and hold a new request.

In the next article, I’ll go into more detail on how to use Comet in ItsNat.

[Slashdot] [Digg] [Reddit] [del.icio.us] [Facebook] [Technorati] [Google] [StumbleUpon]
Webtide

17 Responses to “Introducing ItsNat, a New Java-based Comet Tool”

  1. Carfield Yim Says:

    Look like this is same as wicket, what is the difference?

  2. Jose Maria Arranz Says:

    First of all, this is not the site to talk generically about Java web frameworks.

    Anyway… ItsNat and Wicket share some ideas, for instance the separation of layout (HTML) and behaviour coded in Java.

    The main difference is in the Java part: in ItsNat the layout is fully controlled using Java W3C DOM APIs, every piece of markup “is alive”, you can modify any text node, any attribute, add and remove any element when you want and where you want.

    Wicket in some way follows the approach of “dead markup” in templates (like almost any web framework), basically the template is plain text for the framework, only marked zones with special tags or special attributes are alive parts. You only can “modify” the markup following the rules of provided Java components.

    One example, the tree component. In Wicket is defined using a special tree tag and Wicket generates all the markup, in this case there is no separation of user defined markup and behaviour (markup is generated by the framework). Following the “dead markup” approach there is no alternative, the alternative would be adding tons of special attributes and tags to the user defined markup designed to be coupled to the tree component.

    In ItsNat the layout is designed by the developer: ul, li, tables in tables, table where every row is a node (tree-tables)… A tree with a single node as HTML is enough to “explain” what the concrete layout of the component is. This layout is “alive” for the framework and coupled to the tree component in Java, the tree component is almost HTML agnostic, it detects the structure used by the designer and uses this pattern as the pattern to create and add new nodes to the tree (in fact this “structure” understanding can be optionally user defined in Java).

    In ItsNat components are an add-on, they are optional and any user can build their own components using Java W3C DOM and Events APIs.

    This approach is relevant for COMET: the server can change the client layout as required with almost no limitation imposed by components or framework design.

    Another main difference: Wicket is focused on page based navigation, ItsNat is focused on AJAX and One Single Web Page.

    This isn’t an anti-Wicket comment, Wicket is nice too.

  3. Jose Maria Arranz Says:

    And Wicket has some features missing in ItsNat.

  4. Alex Says:

    These applications are suposed to be scalable ?

  5. Jose Maria Arranz Says:

    A Comet application or a normal Ajax app.?
    about memory?
    or the throughput of many concurrent users?
    please specify.

  6. zkuser Says:

    You can’t beat zk for the ease of programming.

    This framework may be better for other reasons, but for a programmer it’s more of the same nightmare that he has to face for developing web applications: numerous separate files for the simplest of tasks.

  7. Jose Maria Arranz Says:

    One single phrase “Separation of Concerns”

    http://en.wikipedia.org/wiki/Separation_of_concerns

    Please don’t use this reasoning in a job interview with an experienced developer. This has nothing to do with Zk.

    If you want the burden of mixing view layout (markup) and code ItsNat is not for you (constructing markup using W3C Java APIs and ItsNatDocument.toDOM() are recommended only for small bits of markup).

    The best tool for this approach is by far classic JSP of course including pure and strong Java (forget “weak” expression languages).

  8. Eduardo Says:

    The architecture looks a lot like ICEFaces, at least in the part of having a Server side representation of the DOM tree. There are some potential drawbacks with this approach, which we certainly encountered using ICEFaces:

    - Keeping a server-side representation of the DOM doesn’t scale well. Consider the case of a spreadsheet type interface on the browser, with hundreds of rows. This will make for a pretty big DOM tree object, and as more and more clients connect, the server has to keep constructing and keeping track of all this complex DOM tree proxies. By our experience with ICEFaces, I can say you will run into scaling problems. Perhaps this may be alleviated by a prepackaged, optimized grid component…

    - If you have a big table, at least in ICEFaces, to keep the DOM trees in sync between browser and server, the framework has to iterate over the multiple items in the tree. For tables on medium to large sizes this results in a very slow response, defeating the Ajax promise.

    - I think wanting to do everything in Java, and avoiding Javascript (the server centric approach) is a poor justification for an architecture. Javascript is the runtime embedded in the browser, why not use it in an optimal way? It’s not assambler, its a very easy and powerful language. So why all this workarounds to avoid using it?

    My 2cts, granted I am speaking only from your descriptions here, I have not used ItsNat.

  9. anzaan Says:

    Frameworks are terrible things in general….

    They force you to write your code in certain ways … in undesired ways at times…
    I think the biggest turn-off for java developers are the plethora of frameworks that claim to be different from others and that claim to solve problems the others couldn’t.

    What we really need are more tools and libraries that we can use or drop into our applications, and write some code to solve our problems, the way we want to.

    Tools and libraries don’t restrict the way we write applications, although there sure are exceptions, where as frameworks are nightmares, generally these days just about anybody is coming up with frameworks….. Framework must be the new BLACK of the programming world.
    This framework for Comet applications, that framework for ajax apps and so forth.
    How about this library to help you write comet apps and that library to help you write ajax and so on?? SO that I can use them as I please?

    There are great merit to using libraries to build apps as opposed to using frameworks.
    Firstly, there’s no gurantee that the framework u used in your last job will be used in your next job, which means u might have to learn another mammothian framework to be productive again, such a shame.

    If u tool or library, u can always just include that library and start coding to get your job done….

    I would like to think this obsession with frameworks to develop just about any crap is a fad and will go away at some point in future……

  10. anzaan Says:

    And by the way, itsNat’s sourceforge homepage is one of the most terrible piece of work I have come across.
    Its one of the best example of what would happen if you let some server-side junkies build web applications using all but server-side code( I presume……).
    It reminds us why we need designers to layout designs and it also reminds us not to hit every nail with server-side hammer just because we have got a server-side-hammer.

    It also reminds us why spitting out javascript from server using some framework is a bad idea, because to modify the auto-generated code to suit the requirements for an app, u’ll require a javascript programmer to dechiper the code.

    The world still seem plagued with people who think javascript is evil and they need to sanitise developers from the mighty evil of javascript by conjuring frameworks that spit our javascript code for them so they don’t have to hand write any javascript.

    The problem with server-side approach to generating javascript is that its not portable. U’ll end-up writing applications after applications after application, and with every application u end up reinvent a new wheel.

    I think that’s a terrible way of writing a softwares…
    I’d much rather write less applications and more libraries that I can reuse over wide variety of applicaitons. I’d much rather dig into javascript learn it and write simple libraries as I go so I can use it in future apps than instantly gratify myself with auto-spitted javascript code - which will be useless in my next project………

  11. Jose Maria Arranz Says:

    Hi Eduardo.

    Sorry for the delay, I haven’t seen your message before.

    First of all: neither server-centric nor client-centric is the panacea, both approaches have advantages and shortcomings.

    I can’t talk about IceFaces because is not my job, the main difference between IceFaces and ItsNat is DOM management is public in ItsNat and in IceFaces is an “implementation detail” (of course a very important detail) hidden to the end user.

    Eduardo: - Keeping a server-side representation of the DOM doesn’t scale well.

    There are two factors in scalability: memory and speed.

    Memory these days is a bargain (roughly 15 USD per Gb) and current 64 bit architectures have broken the 4Gb barrier.

    In ItsNat “static parts” of a web page are saved as text in memory not DOM (you can mark static/dynamic subtrees with a special attribute), anyway this not apply because your example (the grid) is dynamic.

    Eduardo: - If you have a big table, at least in ICEFaces, to keep the DOM trees in sync between browser and server…

    This is not a serious problem in ItsNat, ItsNat uses node caching techniques to look for nodes, in server and in client (both caches are replicated and automatically kept in sync). For instance, if you change an attribute of a concrete DOM element in server, this server action is propagated to the client of course as a setAttribute call (in a W3C browser, may be different in MSIE). How the client node is searched in the client? ItsNat uses internal ids (generated by the framework), the JavaScript command sent to the client is something like “itsNat.getNodeFromPath(“an_id”).setAttribute(…), the value “an_id” is used to search the target node in a internal registry, this avoid traversing the tree from the top (I agree this task may be very slooow). Any decent browser will use a hash map or a b-tree map coded in C++ to implement the client registry (a property map), in the server this registry is backed by a HashMap (could be a TreeMap). For instance, a b-tree has a performance in the worst case of N “checks” per 2^N, an example: in a registry of 2^32 nodes (aprox 4.000.000.000 nodes) in the worst case the node is found with 32 comparisons (“hey DOM node this id is your id?”). The method getElementById is not used by the framework because is usually implemented (in browsers and in Xerces) using the “brute force” algorithm (searching from top to down).

    Disabling node caching in ItsNat (yes you can) introduces severe performance degradation.

    IceFaces uses some kind of client-server node caching? I’m not sure (inspecting the source code of a web page getElementById seems to be intensively used).

    Another performance technique is innerHTML, ItsNat uses innerHTML automatically instead normal DOM when possible because innerHTML is several orders magnitude faster.

    Force brute/traversing the tree is avoided when possible, in client and in server. For instance, you have very big element list or table and you need the DOM element of the item list or table row 3000, if you use straightforward DOM in ItsNat you need to traverse 2999 elements including text nodes with spaces (ItsNatTreeWalker can help). You can two more options (supposed an element list):

    Using a DOM utility: ElementList.getElementAt(index)

    Or

    Using a DOM list component: ItsNatListUI.getElementAt(index)

    (there are similar interfaces for tables and trees).

    Both methods look for your required element from an internal java.util.ArrayList, an ArrayList is basically a (resizable) Java array, and accessing any item in a Java array is size independent.

    Eduardo: - I think wanting to do everything in Java, and avoiding Javascript (the server centric approach) is a poor justification for an architecture. Javascript is the runtime embedded in the browser, why not use it in an optimal way?

    In ItsNat you’re coding your application in JavaScript… err in Java, but using the same code you would use in JavaScript but in the server. ItsNat simulates a Java W3C browser in the server including events. As you know IceFaces and ItsNat are “AJAX intensive”, so then they are JavaScript intensive too, in IceFaces “JavaScript” is fully hidden, this is a legitimate option, in ItsNat “JavaScript” is exposed to the developer as Java W3C DOM APIs, ItsNat provides cross-browser and some optimizations “for free” and the goodness of Java, strong typing and OOP programming.

    Finally, nothing prevents you can use custom JavaScript code or a JavaScript framework in an ItsNat application (with care). In fact, the Java method ScriptUtil.getNodeReference(node) returns a JavaScript reference to the specified node in the client, ItsNatDocument.addCodeToSend can be used to send custom JS code to the client, and “user defined events” can be used to call (and send data) to the server from user defined JavaScript in client, this “user event” is received in the server as a DOM event.

    Un saludo Eduardo.

  12. Jose Maria Arranz Says:

    anzaan your comments are SO MAXIMALIST, SO EXTREME, that I don’t know what to say.

    You seem angry with the software industry (or Java world) as a whole, may be you are afraid about innovation on software or you have some problems to deal the diversity. May be your place in this world is as a “passive developer” but not a “technologist”, this web is focused on developers with a “technologist” spirit because Comet introduce a “new” paradigm to create a “new” kind of applications, and the word “new” is not for you.

    Your reflections about libraries vs frameworks are very simplistic. Servlet web frameworks are built on top of the servlet technology, the servlet technology is SO BASIC that is UNAVOIDABLE to build LAYERS over SO we have FRAMEWORKS. In fact JSP is a LAYER (=FRAMEWORK) on top of the servlet standard.

    aazan: “And by the way, itsNat’s sourceforge homepage is one of the most terrible piece of work I have come across. Its one of the best example of what would happen if you let some server-side junkies build web applications using all but server-side code( I presume……). ”

    ItsNat web site is PHP, MOSTLY PLAIN HTML hosted on Sourceforge. I’m a SOFTWARE ENGINEER I don’t spend very much time developing “useless beauty”, this is the work for a web designer, FOR ME THE BEAUTY IS IN THE CODE. I think this web page may be more interesting to you: http://home.disney.go.com/

    aazan: “The problem with server-side approach to generating javascript is that its not portable. U’ll end-up writing applications after applications after application, and with every application u end up reinvent a new wheel.”

    Definitively you are completely lost.

  13. anzaan Says:

    @Jose Maria Arranz

    u said:”May be your place in this world is as a “passive developer” but not a “technologist”

    I think Passive developers are Lambs, they follow the herd, I neither follow the herd nor do I get fooled by a 1 page article by a “technologist” who claims innovation by giving us a brand new framework with actually nothing new.

    you said “….Comet introduce a “new” paradigm to create a “new” kind of applications, and the word “new” is not for you”.

    Are you trying to tell me that comet technology sprang out just about yesterday and I totally missed the 6.30 news??
    I’ll tell you exactly what comet means to me.
    Comet is just a fancy name given to an old hack so that every tom-dick-harry with a bit of programming skills can write up some code and surround it with the buzzword COMET and sell it to unsuspecting programmers and clients.
    It seems to work too.

    So to me your ” new” word is just a marketing pitch /punchline.

    And itsNat’s GNU license explains your marketing zeal to some extend.

    u said: ItsNat web site is PHP, MOSTLY PLAIN HTML hosted on Sourceforge. I’m a SOFTWARE ENGINEER I don’t spend very much time developing ‘useless beauty’ “.

    Looks like self-importance has been chewing up your rationale, because rationale tells me this :
    ” IF YOU DEVELOP A WEB FRAMEWORK AND CREATE A HOMEPAGE, AT THE LEAST DEVELOP THE SITE USING YOUR OWN FRAMEWORK ! ”
    Hope that makes at least some sense.

    “useless beauty” is one thing but a tasteless jumbled eyesore homepage of a framework from a Technologist Innovator is something else.

    Using heavy handed technique to simulate browser at the server is more a case of software mis-engineering than software engineering.
    Good software engineering is not just about writing code to make things happen, its also about making the best out of available resource while eliminating duplications and not doing things just because u can.
    Your framework advocates VIEW duplication with a copy in both browser and server and it promotes doing things just because you can. And if it served a unique purpose then I would understand.

    Try as I may, I can’t think of a rationale behind such bloated misengineering approach.
    With client-side solution, all evens can be handled instantanously resulting in rich user experience where as with server-centric approach there’s always a roundtrip for everything. To me its a terrible waste of user’s time and system resource.
    Looks like usability issues are non issues for Technologists these days.

    Oh, by the way, do u also control mouse-over event for menus from the server???
    Does it take a round trip to handle mouse-over??

    this was what I found at itsNats homepage:

    ‘ ItsNat is server centric using a unique approach called TBITS, “The Browser Is The Server”.

    Anyone who creates acronyms such as that to describe their piece of work have worrisome case of megalomania or are over-zealous in their marketing push.

    And by the way your comparision of itsNat and DWR in your article was laughable.
    There’s a paragraph of it with a big heading and three bulet points about your framework but not a single comparision between the two. What exactly did you mean by comparision?
    Who are you kidding?

    I like DWR in that at least its unobtrusive and it doesn’t shackle me as a developer and creep into my application code either.

    Need some quality articles here at cometdaily.com…….lately it has become a marketplace for salesmen.

  14. anzaan Says:

    @Jose Maria Arranz

    when I said ..”And itsNat’s GNU license explains your marketing zeal to some extend.”
    I meant GPL license.

  15. Jose Maria Arranz Says:

    I’m absolutely agree with you, I’ve seen the light and my salvation.

    Thanks Very Much!
    Good Bye

  16. Softball Says:

    This advice is really going to help, thanks.

  17. mark Says:

    anzaan

    I think itsNat is an awesome breakthrough in eliminating framework stupidity. You seem to have a grudge against Jose Maria Arranz. Is there something personal going on here?

Leave a Reply



Copyright 2014 Comet Daily, LLC. All Rights Reserved