Comet Support by SitePen

Is it Raining Comets and Threads… Again?

by Jose Maria ArranzNovember 21st, 2008

A scientific theory says the oceans were created by millions of comets and asteroids colliding with the Earth many millions of years ago.

A long polling application is very similar, like a rain of:

  • Requests
  • Connections
  • … and Threads??

These requests have arrived to stay in the server like water in the ocean, but do these Comets contain threads?

Threads and ItsNat

When I was adding long-polling to ItsNat, I followed the standard approach of one request per thread; this relationship is mandated by the current servlet standard. When a long-polling request arrives to the server, ItsNat queries for pending data (in JavaScript form) to send to the client (for instance, a DOM node has been added/changed/removed or new custom JavaScript is queued by an asynchronous process). If there is data to send, the request returns to the client with new code and a new long-polling request is sent again. If there is no pending data the thread is stalled waiting for a new notification of the mysterious user defined asynchronous process working behind the scenes.

I thought that because this approach followed the standard, worked on any servlet container, and most of the time threads will be asleep, there would be no problem with having so many sleeping threads!

My happiness was very short; soon I realized the ItsNat approach did not scale for many concurrent users because too many threads were needed.

There is significant evidence, people, and products focused on the goodness of asynchronous servlet containers or using techniques avoiding threads in general: IceFaces offers a custom asynchronous servlet container, while GlassFish, Tomcat and Jetty are pushing solutions of containers with the promise of very few threads servicing many clients. However, in my opinion this is the reason that NIO was invented: to avoid threads blocking when reading and writing streams. The typical pattern of classic IO (if a thread is blocked and a new connection has arrived a new thread must be created or this connection will be stalled too), and of course thread synchronization is time consuming and affects performance. Everybody seemed to know NIO was superior, in performance and scalability, to IO because of threads.

So at this point, while the ItsNat approach might satisfy the needs of many Comet applications (not every web site/application is Google or eBay), it failed with too many concurrent requests. Detecting and using the new servlet containers become a priority in my “TODO” list (in spite of current solutions not being standardized). For a while, I left this scalability problem alone and simply cited in in the ItsNat documentation about our long-polling Comet implementation.

The “Revelation”

As a frequent reader of javalobby.org, one day I found an article on avoiding NIO to get better throughput. Dan Dyer cited the blog and a paper of Paul Tyma, a Software Engineer at Google. In his research paper, Paul destroys, with humor and with some tests, the typical well established myths about the superiority of performance and scalability of NIO against IO. I must recognize that I was shocked with Paul’s investigations and soon realized the correlation with long-polling.

The origins of the NIO approach and why threads were “bad”

To understand Paul’s insights, first we must understand some background with thread programming. First of all, in any system, threads are basically pieces of memory: stacks and state registries. There is no correspondence between threads and hardware. Furthermore, usually any processor (or core) is almost a single-thread processor. in other words, a very very small number of running threads completely fulfills the “real” parallelism of any average computer. The parallelism of threading is an illusion and thread management is basically a software problem with some necessary support of hardware.

Managing tons of threads is not a hardware problem, but one with software. For example, consider a single core processor saturated with two threads doing some non-blocking tasks. If in a single-real-thread-processor one task needs X time to complete, Y threads doing the same need X*Y as a minimum and in ideal circumstances (in this extreme case threading cost is 0), but the parallelism sensation is important and justifies threading.

The problem of many threads is that the performance cost is significant:

  1. Thread switching has a cost, and this may increase when more threads are added.
  2. Uncontended synchronization (calling a synchronized method not blocked) has a cost.
  3. Contended synchronization has a cost.
  4. The thread stack occupies memory.

NIO was invented as an alternative to thread-based communications (one socket per thread blocking when necessary). The basic idea of NIO is to avoid threads, where one single thread iterates over all open connections asking for and writing new data with no blocking. In other words, if there is no data pending read or no data cannot be written in a connection, nothing is done, and the next connection is tried. This way the illusion of parallelism is fulfilled (every effective read/write is a small buffer) and thread costs are gone. This idea is heavily based on Matt Welsh’s Ph.D. thesis and the SEDA project.

Welsh’s thesis was right; the SEDA project started around 2001, and scalability of threading in Linux 2.4 kernel was bad, really bad: only a few hundreds threads were enough to stall an average Linux system. NIO was introduced in Java 1.4 in 2002, heavily based on Welsh’s ideas.

As time passes, NIO in the real world uses some threads (very few) to avoid unavoidable locks and to make better use of multi-core systems (NIO is not thread-free), and now the JVM is better managing threads reducing costs, and most importantantly: Linux Threads were re-engineered in the 2.6 version of the kernel as Linux NTPL:

In tests, NPTL succeeded in starting 100,000 threads on a IA-32 in two seconds. In comparison, this test under a kernel without NPTL would have taken around 15 minutes

Paul Tyma’s investigations at Google about NIO and IO

In spite of living in the multi-core age (some threading is absolutely necessary in any server, IO or NIO), Linux NTPL, and Java JVM threading improvements, the conclusion of superior performance with NIO was not questioned. That is, until Paul Tyma took a look and found he was not alone in questioning this assumption:

For a tangential purpose I was benchmarking NIO and IO (simple “blast data” benchmark) I could only get NIO to transfer data up to about 75% of IO’s speed, asynchronous (blocking NIO was just as fast), I blamed myself because we all know asynchronous is faster than synchronous right? Started doing some emailing and googling looking for benchmarks, experiential reports. Yes, everyone knew NIO was faster. No, no one had actually personally tested it. Started formalizing my benchmark then found: http://www.theserverside.com/discussions/thread.tss?thread_id=26700 and http://www.realityinteractive.com/rgrzywinski/archives/000096.

TheServerSide.com page revealed the same initial conclusions of Paul, in this case by Rahul Bhargava, CTO of Rascal Systems:

As you can see the last line indicates vanilla blocking server (thread per connection) produced the best throughput even with 1700 threads active in the JVM.

No one commented on this post. To be fair, the Discussions section of TheServerSide.com was never very popular compared to the News section, but still, not a single person commented to agree or disagree with this comment.

When Paul realized that NIO presuppositions could be false or may be not so definitive, new tests were done on Linux (kernel 2.6), Windows XP and a very big multicore system of Azul (a beast of 768 cores), trying to validate or to debunk some well established presuppositions against classic blocking and thread based IO:

  1. Asynchronous I/O is faster
  2. Thread context switching is slow and synchronization among threads will kill your performance
  3. Threads take up too much memory
  4. One thread per connection does not scale

The results of Paul’s tests using up to 1000 threads state the contrary:

  1. IO outperforms asynchronous IO in a single client/server scenario (including on Windows XP)
  2. Thread context switching, idle thread and uncontented synchronization cost is near zero. Sometimes explicit synchronization can be avoided using utilities for non-blocking concurrency; in fact IO methods used under the hood concurrency utilities when possible. Furthermore, NIO is not free of context switching cost when querying and managing client data associated to the keys returned by the selector.
  3. The Java stack can be reduced to 48KB (minimum), so 41666 stacks (threads) fulfills 2GB of memory (obviously more space is need for the OS, applications, etc). This stack “automatically” saves the state of the communication. In the NIO solution, the client state is associated to the selector keys and must also be saved to keep track of communication flow and phases.
  4. Performance is greatly independent of the number of threads (up to 1000). In the case of Azul, more threads clearly implies more performance.

And finally Paul states two advantages of classic IO:

  1. Coding is much simpler: the code contains the communication flow and states.
  2. Make better use of multi-cores

Finally Paul tells “The story of Rob Van Behren (as remembered by me from a lunch with Rob)”:

Set out to write a high-performance asynchronous server system found that when switching between clients, the code for saving and restoring values/state was difficult. Took a step back and wrote a finely-tuned, organized system for saving and restoring state between clients, when he was done, he sat back and realized he had written the foundation for a threading package.

Conclusions: Not So Fast…

First of all, Paul’s study and conclusions are based on raw socket connections, not about HTTP and servlets. A concrete study applied to HTTP, standard servlets and modern asynchronous approaches, would be necessary to validate Paul’s conclusions on higher levels.

For me, the most important conclusion is NIO (asynchronous in general) and IO approaches are comparable, and there is no longer one definitive single correct answer. There may be some scenarios where NIO or IO is better and other scenarios where it is irrelevant. And of course there is space for technology improvement.

This case remind me the old battle about performance between Java and C. Java is no longer the snail technology it was, and many agree that C and Java performance is today very similar, and speed is no longer an important criterion when selecting the technology in most typical scenarios.

In the case of ItsNat, supporting new asynchronous servlet containers is desirable but no longer an absolute priority for scaling, and no longer problematic. Because ItsNat is based on Java 1.4 and new containers are compiled with Java 1.5, reflection will be the path we take, and a new asynchronous servlet standard is around the corner!

[Slashdot] [Digg] [Reddit] [del.icio.us] [Facebook] [Technorati] [Google] [StumbleUpon]
Orbited

40 Responses to “Is it Raining Comets and Threads… Again?”

  1. Markus Kohler Says:

    Very good summary of the NIO vs IO discussion!

    One comment from my side:
    Even if you have a single processor you may need more than one thread for the best performance.

    Imagine you have an operation that requires IO and some costly computation. If there’s no direct dependency between IO and the computation you can just run the IO operation in one thread and the computation in another thread.
    This will lead to better performance than just serially running them.

    At the end I think, ideally we want to use NIO with more than a few threads, tuned to what is best for the application and the given hardware (maybe memory is the limitation factor maybe something else).

    Regards,
    Markus

  2. Robert Says:

    Excellent article. I found the paper you linked to quite enlightening.
    It’s also exposed me to your ItsNat project.
    I found the idea intriguing, the DOM manipulations are done directly on the server and just synchronized down to the client. That events on the client are sent back up to the server to be handled.
    It reminded me of the Echo framework which is several years old. It too has the notion of the server is where everything lives. However it’s not the same since there is a bunch more abstraction in Echo, you don’t directly modify the DOM.

    Even though in your ItsNat demo I see response times in the the 40ms to 60ms range, I worry that this will turn into an issue. That every single modification or event has to go over the network. Where on the client triggering a hundred events is no big deal CPU wise (even if it’s a bad idea design wise). Having a hundred events triggered with ItsNat would introduce significant network traffic.

    I like the idea, but like most technologies it won’t be tenable for all web application types.

  3. Neil Berkman Says:

    There’s quite a bit of discussion of Rahul Bhargava and Paul Tyma’s findings here:

    http://www.theserverside.com/news/thread.tss?thread_id=48449

  4. Jose Maria Arranz Says:

    Markus: Even if you have a single processor you may need more than one thread for the best performance.

    Yes I know “some” real parallelism is possible in modern processors (=cores), in fact Pentium 4 with hyperthreading enabled is reported as a two core processor (of course this never was true, may be 1,5 cores or lower).

    My point is: threading is mostly a software problem, very much the same problem of context/client switching in a NIO application. In some way is the question “what is more appropriated?” to use a kernel based capability ready to be “easily” used (threads)? or a custom/specific solution doing basically the same (NIO) ?

    The “real native thread of the processor” dispatching to the “native threads” (without “real” and “processor”) is basically doing the same job (context switching simulating parallelism) as the NIO thread dispatching to the clients. Similar problem, similar approach, similar performance expected.

  5. Jose Maria Arranz Says:

    Robert: I worry that this will turn into an issue. That every single modification or event has to go over the network.

    In ItsNat only events with associated (registered) server-side listeners are sent to the client, a single event can change most of the web page (or completely) if necessary.

    Robert: Having a hundred events triggered with ItsNat would introduce significant network traffic. I like the idea, but like most technologies it won’t be tenable for all web application types.

    ItsNat is in the space of server-centric web frameworks with AJAX capabilities, like Echo, Wicket + AJAX or JSF flavours with AJAX (and many more). Usually in these frameworks visual layout is cooked in the server. This approach has pros and cons.

    I’ve seeing your amazing WorldOfSolitaire.com client web application and I agree with you ItsNat IS NOT for this type of web applications (nor Echo, nor Wicket, nor JSF … nor server-centric frameworks in general). Dojo, Yahoo UI (used by you) etc are more appropriated.

    Greater than 90% of the typical web sites and web applications are database driven (or data intensive in general saved on the server), this percentage is mine, but I’m sure is actually bigger. In this type of applications almost every user event queries/changes data in the server. This is the space of ItsNat.

    Another emergent space for ItsNat is web sites designed for mobile, today all of modern mobile browsers include AJAX, these browsers are heavily limited in memory and performance.

    Memory limited supposes that no very much JavaScript code can be loaded, the SOFEA approach fails if the web application is complex.

    Performance/speed is a problem too, mobile processors are slow and JavaScript is sloooow in these devices. More power to the server more power to the device in general. Two interesting examples are Opera Mini and SkyFire, in these browsers JavaScript and layout rendering are executed in a server so rendering and JavaScript execution may be speeder than full blown mobile browsers like iPhone or Opera Mobile (a good network connection is the key).

    http://blog.laptopmag.com/mobile-browser-showdown-iphone-3g-vs-opera-mobile-and-skyfire

    Anyway ItsNat can be used alongside client-centric frameworks, joining forces. Several reasons:

    1) Developer has absolute control of the layout and onXXX methods are not used by ItsNat.
    2) Static DOM subtrees in the server may be dynamic subtrees in the client (peace for all). In fact static DOM subtrees usually are cached as plain text.
    3) Custom JavaScript code can be sent to the client as a response of any event (and in load time).
    4) Custom user events can be sent to the server fired by JavaScript code.
    5) DOM changes in the server can be optionally not propagated to the client (to manually synchronize server with client changes).
    6) DOM changes in the client can be sent to the server to update the server DOM (used with care because of security).

    These reasons open an unexplored new world of cooperation between server and client frameworks.

  6. Dong Liu Says:

    For static large file read tasks, Java IO is faster than NIO in my experiments. Throughput is different from scalability though.

    Whether to choose thread-per-connection or thread-per-request is never a question for servlet containers. The latter is preferred.

    The fact that NIO can support more concurrent users/requests than IO has been proved by many.

    In order to see the difference, design your now benchmark and test your server in two configurations, you will know your mileage.

  7. Jose Maria Arranz Says:

    Dong Liu: “The fact that NIO can support more concurrent users/requests than IO has been proved by many. In order to see the difference, design your now benchmark and test your server in two configurations, you will know your mileage.”

    “I want to believe” said Mulder in X-Files, but… where are the numbers?

    First of all: any test trying to calculate what is the *maximum* number of concurrent users can a system manage is a bit stupid because in this maximum the system is actually down, useless, you can create thousand of threads too.

    The most interesting scalability test is: what is the max number of concurrent users a system can manage with no significative increment of processing time per user, that is to say, user management/switching cost doesn’t increase when a new user is added. In Paul Tyma’s tests, 1000 threads-concurrent users can be managed with almost no cost, this number is not bad, of course the system can manage many more but in some number of threads the management/switching cost will start to increase.

    I have a test:

    “Can a Grizzly run faster than a Coyote?” comparing GlassFish against Tomcat (this test is outdated). GlassFish using NIO and Tomcat 5.5 using IO and 300 concurrent requests.

    http://weblogs.java.net/blog/jfarcand/archive/2006/03/can_a_grizzly_r.html

    And yes, GlassFish seems to be the winner but this victory is pyrrhic.

    You must understand, this test is not a pure NIO vs IO test, more layers are involved and this test is comparing apples with oranges, two different servlet containers, and “GlassFish Catalina is based on Tomcat 5.0.x (plus severals performance improvement)”, that is to say, in those days GlassFish was an “improved Tomcat” (Tomcat would be defeated using IO too).

    The only test with a significative difference is a “Real world benchmark, the purpose of my second benchmark is to stress the server with a real world application that contains complex Servlet, JSP and Database transaction”… too many things in the bag to compare NIO and IO approaches.

    In summary, again a test confirming my “thesis”: NIO in some way does in essence the same as IO + kernel does, we only can expect minor differences.

  8. Richard Maher Says:

    So there are no scalibility issues with one-thread-per-connection? Excellent! Why not go one-process-per-connection and be done with it? I guess the future is WebSockets and INETd, and I like it. If you don’t believe me, “where are the numbers?” :-)

    Alternatively, you could have a configurable (min, max, idle-timeout) pool of worker threads (or processes) and not have to have 1000 attaches to the database(s) or files, and 1000 stacks and copies of memory that don’t do much else except get paged in/out every now and then. But anyone opting for this strategy must surely be clinging to security-blankets and the TP Monitor functionality of yester-year?

    Sorry for the scepticism/sarcasm and for being too lazy to look up the “tests” in question to discover exactly what work the threads involved were required to perform, but if they were doing more than retrieving (probably) cached/buffered information then I am impressed. If they were attached to a database and performing update and locking activity and there was still no benefit from having the worker-pool model then I think this should be on the cover of every IT magazine and in every University tomorrow!

    Regards Richard Maher

  9. Comet Daily » Blog Archive » Liberator Performance and Architecture Says:

    [...] And here on Comet Daily: Is it Raining Comets and Threads? [...]

  10. Dong Liu Says:

    “The most interesting scalability test is: what is the max number of concurrent users a system can manage with no significative increment of processing time per user, that is to say, user management/switching cost doesn’t increase when a new user is added. ”

    In my test, I always do in that way. To scale does not mean loss of responsiveness or reliability. For the application cases I tested, NIO is better than IO in term of the capacity for concurrent users in a single node system.

    From a high level point of view, “Languages Don’t Scale. Frameworks Don’t Scale. Architectures Scale.”

  11. Jose Maria Arranz Says:

    Dong: “NIO is better than IO in term of the capacity for concurrent users in a single node system.”

    Capacity = number of ? I think throughput (data per second) is a better benchmark and how this throughput is affected by the number of users (connections).

    How much is better? I can’t find a test/benchmark confirming your facts, it would be interesting if you post your results.

  12. Martin Tyler Says:

    There seems to be quite a lack of detail for these ‘tests’. Ie what each thread, or ‘client’ was actually doing.

    I would add latency to throughput as a criteria for a benchmark.

    Also if the application in question requires the client connection to do anything based on anything other than reading from the client (ie sending chat message from someone else), do you send the message from another thread? if so the arguments about simple coding and following the control flow through a single thread is mute. Basically a lot of server applications are inherently asynchronous and event based, which can make a thread-per-client model just as complicated as a fully async model. The implication that an async model has do loads of work ’saving state’ seems only relevant if you are comparing it to the most simple of applications using a thread-per-client model.

  13. Jose Maria Arranz Says:

    Martin I agree with you, in “normal” (non-long polling) web applications, NIO and IO approaches are just internal issues with almost no impact on developer code, NIO and IO differences are very important when some kind of communication protocol is used (in this case IO is by far simpler). In web, most of the input and output data is raw (for instance a HTML page returned is not “interpreted” in some way by the application server).

    The problem is long-polling in Java servlets. In the standard servlet specification client requests are considered “short”, processed as soon as possible, in this scenario a small pool of threads can serve to many concurrent connections, before processing the request, a thread is associated to a connection, this is mandated by the servlet specification.

    In long polling is different, a long polling request can take very much time (when there is no data to send to the client), as you need to “hold” the request object you must understand if this request can be hold without the associated thread, in a normal request if the associated thread is not hold (locked) this request object will become invalid. To avoid thread locking Java based servlet containers are offering alternative non-standard asynchronous approaches for long-polling, in this case the details are exposed to the web developer doing Comet.

    For a web developer using ItsNat, NIO vs IO is not an issue because in some way long-polling is event based in ItsNat too. NIO and IO smackdown is important for me as the ItsNat creator dealing with IO internals :)

  14. Jose Maria Arranz Says:

    I forgot to say:

    In custom non-standard asynchronous requests the approach for the developer is “not hold the thread”.

    Paul Tyma’s investigations suggest that threading is not an issue for scalability and performance, so ANY web developer can do long polling “easily” *in ANY servlet container* (NIO or IO based) without special artifacts, simply locking the thread of the long polling request with an object monitor; of course this monitor in some way must be registered before in some place to be “notified” by some “background thread” to unlock this request thread when new data must be notified to the client. Almost any web developer with some background of threading can do this without special Comet and servlet container infraestructure (in some way this code is a mini-long polling framework in server).

  15. Dong Liu Says:

    @Jose,

    My comments on IO/NIO are *not* about ItsNat framework. And I have not done any experiments with ItsNat. Different application scenario will see different benchmarking result. My work is about the scalability of web services and service orchestrations. Those services and service orchestrations are developed by some well-known java-based frameworks and deployed on servlet containers like tomcat and jetty. I would not comment on tomcat vs jetty. For jetty, NIO is always better than IO for my orchestration cases in term of number of concurrent requests with acceptable response time. NIO is also required for async processing. http://blogs.webtide.com/ has some posts about the benefit of NIO for jetty.

    Request per second is more interesting than Byte per second in my case. And the number of concurrent request the server can support is important. My definition of scalability is “a service system’s ability to sustain acceptable mean service time for increasing number of concurrent jobs in proportion to the computing or operational resources added into the system without affecting its reliability and other features”. For a single node system, larger capability for concurrent requests implies better scalability.

    In my cases, thread number is definitely a factor for system’s throughput and responsiveness for java applications. See http://books.google.ca/books?id=DpHvFpK2omsC page 33 figure 2-1. IO and NIO may have a little difference for “light load”, but for “heavy load” there will be a big difference.

    The C10K page (http://www.kegel.com/c10k.html) has many stuff about IO/NIO and threading.

  16. Martin Tyler Says:

    Jose,

    It sounds like if you are limiting yourself to work within the serlvet spec then you could be right, i havent tested it. I am thinking about more generic standalone server applications - which i think the main article you referenced was also talking about.

    I am not claiming async IO is better for all apps, and I admit when I wrote Liberator the threading wasnt up to it, as he mentions. I would like to see more about his tests though.. 1000 threads isnt a huge amount, and he doesnt give much idea about what those threads do - as i said, in many complex apps you would need more than just 1 thread per connection as they have to react to lots of different things, so cant just sit there blocking on a socket.

  17. Jose Maria Arranz Says:

    This is not just an auto-limiting issue because APIs like F. Arcand’s Atmosphere makes integration with diverse application servers easier.

    The problem goes beyond an API, only the newest application servers can use this kind of asynchronous technology. This excludes millions of application server deployments, many of them were the result of very expensive purchasing (I’m talking about Java).

    The fact of threading scalability opens Comet (long polling) to millions of already running web applications and related application servers, that is to say “yes your old application server can scale a long polling application too”.

    Martin: 1000 threads isnt a huge amount, and he doesnt give much idea about what those threads do.

    I agree, I think the test just read raw data form a server on socket level.

    If 1000 threads don’t impact the throughput (this is the claim) a significative higher number could be used with no problem, may be beyond 1000 threads the throughput could start to decrease. Anyway this is just speculation.

  18. Jonathan Myer Says:

    [...] And here on Comet Daily: Is it Raining Comets and Threads? [...]

  19. Jonathan Myer Says:

    Hi Jose,

    Your idea is not too bad, apart from the fact that its old and its been done before and done much much better.
    I have seen a few demos of GWT (smartGWT) , IceFaces and Echo and ItsNats demos don’t even come any close to those demos.

    The IO vs NIO debate is a pointless one unless you believe that the Sun dudes who came up with the NIO idea after some many years are completely retarded. .
    And I don’t see why you hold on to the servlet specs since its going to change anyway , like everything. Evolution is as much true in biology as in software engineering.

    you said : “If 1000 threads don’t impact the throughput (this is the claim) a significative higher number could be used with no problem,may be beyond 1000 threads the throughput could start to decrease”

    I think your statement refutes itself. IF a significant higher number could be used with no problem ( surely significant as in lot more than 1000 threads), why would thoroughput start to decrease beyond 100 threads??
    That statement has a mystical sound to it.
    And when did a thousand connection or even few thousand concurrent connection become significant?

  20. Jonathan Myer Says:

    …continued( as cometdaily refused to post my post as a whole)….

    And doing too much in the server is a very tempting invititation to trouble. What if my user base becomes larger than say your 1-2000. What if my user-base is say 40000??
    Do I ditch ItsNat completely and write another applicaiton with another framework from scratch??
    I would sure have to because I won;t have any generic codebase to work from.
    What you seem to have missed besides other things like scalibility is the fact that front-end engineering is as much a fact as back-end engineering. Since your framework moulds the two, if I have to ditch ItsNat for ,say, scalibility reason, I would have to ditch almost everything and start from scratch.
    What if I need a a system that can handle say 50,000 connections concurrently and perform all the heavy duty tasks like database/file access and manipulation. Do I run off to Erlang??
    Or do I dive into Scala??
    Of course I can use Rhino with javaScript generators to come up with a decent light-weight pseudo-thread ( or shall I all it process) library to tackle the complexity.

    So, it is a given fact that ItsNats is only so good as the applicaiton is expected to have around 1000 concurrent connecitons at a time??
    Even then the user-experience is going to suffer through round-trips to server for even trivial event responses.

    My conclusion is that ItsNat is a very heavy-handed approach to application development. Please correct me if I’m wrong. And please publish your test/benchmark results such that they can be reproduced, so that what you claim can be justified. There’s a difference between science and relegion, namely in the fact that science is based upon empirical facts that can be tested and proved while religion is mere clain and requires absolute faith.

  21. Jonathan Myer Says:

    ahh… in my first post I meant ….why would thoroughput start to decrease beyond 1000 threads??
    instead of “why would thoroughput start to decrease beyond 100 threads??”

  22. Jose Maria Arranz Says:

    First of all I’m reluctant to talk about Java web development in general on cometdaily.com. Other web sites like javalobby.org, theserverside.com etc are more appropriated. Fortunately comments don’t “pollute” very much the focus of cometdaily.com

    J. Myer: Your idea is not too bad, apart from the fact that its old and its been done before and done much much better.

    Where? Before starting ItsNat I was “investigating” the approach used in ItsNat, the result was “I can’t believe no one has tried this approach , is so clean, so easy, so straightforward, so NATURAL”. If some competent framework (not a pet/sunday project) had used this technique ItsNat wouldn’t be born. Of course I’m absolutely biased.

    J. Myer: I have seen a few demos of GWT (smartGWT) , IceFaces and Echo and ItsNats demos don’t even come any close to those demos.

    Let me answer with some cites you can find in ItsNat documents:

    http://www.innowhere.com:8080/itsnat/feashow_servlet?itsnat_doc_name=feashow.main&feature=feashow.featShowcase.doc

    “Feature Showcase’s examples are very simple and oriented to show how to use the APIs, they are not shocking examples to impress you, ItsNat is not a picture is much like a painter’s palette. The HTML used is very very simple because any piece of markup should be understood easily, HTML (and XML) templates and final output, to show you have complete control of the layout. We invite you to select output markup regions in every example using the marvelous Mozilla/FireFox’s “View Selection Source” contextual menu option to see how the markup is finally rendered.”

    http://www.innowhere.com:8080/itsnat/feashow_servlet?itsnat_doc_name=feashow.main

    “Previous features are defined in the ItsNat Core level, ItsNat provides a Component system (constructed over the Core level) too… The most interesting feature is: any element can be a component, for instance any element can be a normal button, a check box, a radio button, any element list may be a combo or a list component, for instance a group of SVG circles may be a multiple selection list component. Furthermore components can be used to generate XML!! ItsNat components are not intrusive, they are not “black boxes”, you can customize and control every element used, every attribute. Components are highly extensible and customizable, for instance there is no default decoration when selection ItsNat tells you when and what has been selected, ItsNat may be seen as a meta-framework. Doing user defined components, for instance compound components, is amazingly easy.”

    “No XML configuration (use Spring or the configuration technique you like more), not declarative navigation, no JSP, no custom tags, not declarative programming, no view-Java bindings, no logic mixed with view, no new (expression or templating) languages, no “pull/active views” (ItsNat pushes data/markup to the view), not page based development (ItsNat is event based), not highly closed not customizable and intrusive components, not obscure HTML and JavaScript generation, not fully programmatic view definition (ItsNat uses pure XML/HTML/SVG templates), not JavaScript centric, no big and obscure JavaScript libraries hard to debug and maintain open to modifications by malicious users in runtime, not AJAX as a forced add-on, not oriented to tools (use your favorite X/HTML designer and pure Java IDE), not developer/designer substitution by the framework, no loss of developer control. “

  23. Jose Maria Arranz Says:

    Part II:

    If you’re looking for *already made* “cutting edge” components out of the box may be ItsNat is not for you.

    In spite of this do you want “cool” demos? Try this Ext-JS like Table:

    http://www.innowhere.com:8080/inexperiments/

    It was developed reusing the HTML layout generated by this example:

    http://www.greeneyed.org/test/indexJSON.test

    ItsNat version is not complete and some DHTML effect is missing, for instance resizing (but I’m not going to spend more time in this kind of “cool” stuff). This “experiment” is using CSS code of ExtJS but no ExtJS JavaScript code is used.

    I’m going to release the source code soon. You are not going to find something weird: pure HTML code as templates with the layout generated by ExtJS (some cleaning was needed) and some (very little) Java code to add the view logic.

    I, you, he, she, anyone can clone with ItsNat the cool layout of your favorite framework with no very much problem :)

    J. Myer: The IO vs NIO debate is a pointless one unless you believe that the Sun dudes who came up with the NIO idea after some many years are completely retarded.

    A very, very, very bad argument and too simple. I’m not going to answer this, furthermore, I’ve already answered this in my article:

    “Welsh’s thesis was right; the SEDA project started around 2001, and scalability of threading in Linux 2.4 kernel was bad, really bad: only a few hundreds threads were enough to stall an average Linux system. NIO was introduced in Java 1.4 in 2002, heavily based on Welsh’s ideas.”

    One analogy: Swing was very bad, SWT was born to replace the “wrong” approach of Swing, today SWT is the “wrong” approach and Swing approach shines. Tomorrow? I don’t know.

    J. Myer: And I don’t see why you hold on to the servlet specs since its going to change anyway , like everything. Evolution is as much true in biology as in software engineering.

    It is going to change… but not changed, and when changed real adoption for masses will need several years. Some statistics say around 25% of developers are still stuck to Java 1.4. You should know WebSphere 6.1, the first release with Java 1.5 support, was released on May 2006!!

    http://www.infoq.com/news/WebSphere-6.1-Released

    The only thing I was trying to say is: YES, your old Java application server CAN scale long polling based applications. Of course if you can upgrade, please upgrade, today we have amazing application servers and servlet containers and most of them are free, but you should know upgrading from Java 1.4 to Java 1.5 is not ever possible (for free).

  24. Jose Maria Arranz Says:

    Part III:

    J. Myer: I think your statement refutes itself.

    I mean: imaging throughput is X Mb/s with 1000 threads, right? Throughput with 1001 threads is 0.9999*X Mb/s, fine. Now with 2000 threads is 0.80*X Mb/s, 20% lower… can I say throughput is not affected with 2000 threads? NO. Can I lose 20% of throughput to serve 1000 new users? YES. For any other specific question about Paul Tyma’s study please try to ask to Paul.

    J. Myer: And doing too much in the server is a very tempting invititation to trouble. What if my user base becomes larger than say your 1-2000. What if my user-base is say 40000??

    Ask this to Struts, Wicket, JSF, Spring MVC or Tapestry 4 users. Most of web applications out there are based on full reload per click, most of the time with no significative use of JavaScript. Many of these applications were developed when the bandwith was 5Kb/s (dial-up)… and worked. I’m not talking about a UML tool based on web, yes the server centric approach doesn’t work in this kind of applications, but >90% of web applications are data intensive (not graphic intensive), the server side approach simply works.

    If your user base becomes large… buy a new server, is cheap, your 2000 concurrent users worth the effort.

    If your user-base is 40000? Do you mean 40000 CONCURRENT users??!! Forget technical issues and call to Google they will be really interested in your web site and by the way, they know how to deal with 40000 concurrent users, for instance try to ask to Paul Tyma :)

    J. Myer: So, it is a given fact that ItsNats is only so good as the applicaiton is expected to have around 1000 concurrent connecitons at a time??

    I think you are not talking about Comet and long polling, if I’m correct then you have missed the point of the article. ItsNat on non-long polling applications behaves basically the same as any other typical server centric AJAX web framework; ItsNat can run in ANY servlet container including asynchronous based, in normal Java based web applications asynchronous communication is provided transparently and the framework is in no way “conscious” of this. ItsNat scalability is heavily influenced and limited by the servlet container used as any other framework.

  25. Jose Maria Arranz Says:

    J. Myer: Even then the user-experience is going to suffer through round-trips to server for even trivial event responses.

    I think many of your complaints are related with server centric approach of web development and nothing to do with long-polling, NIO or IO. Am I wrong? As I said before ItsNat has no problem to be mixed with client centric JavaScript frameworks, ItsNat CAN, this cannot be said with other intrusive frameworks with almost no control of the layout.

    Finally, have you tried to search a word in Google? Yes course. Have you seen the amazing Google suggest feature? Don’t say to anyone but the browser is requesting the server every time you press a key.

    Rendering the aesthetic as HTML in the server of minor changes of the page transported with AJAX is a minor issue in server power, because, HTML is mainly data, there is no “drawing” you’re not rendering images.

    Remember… the “love is in the air” and the “data is in the server”.

  26. Jose Maria Arranz Says:

    I’ve found this interesting article:

    http://www.theserverside.com/tt/articles/article.tss?l=ScalingYourJavaEEApplications

    The chapter “Non-Blocking IO vs. Blocking IO” shows how a NIO based application server (GlassFish) outperforms Tomcat using IO

    One big problem with this test: “In this testing, we set “maxThreads” to “1000″” trying to
    serve 7000 users… This is not a fair tuning for Tomcat IO, what if tried 7000 threads? One interesting thing is 1000 threads seem to work fine until 4000 users!

    Finally this test is testing “More complex scenarios instead of simple JSPs and Servlets, involving EJB, Database, file IO, JMS and transactions”, again too may things to conclude NIO is superior.

    Final notes:

    * I’m not trying to devaluate GlassFish, GlassFish is an amazing beast. I just can find tests using GlassFish.

    * NIO is a valid path for servers, of course, because in spite of is doing basically the same as the kernel with threads, it gives more freedom and more possibilities to the developer (synchronous and asynchronous long polling) and more resource control to the server.

  27. Jonathan Myer Says:

    Hi Jose,

    I had been meaning to reply back to the clarifications you made in your above posts but due to the volume of your post I find it incredibly difficult to post a reply without ending up writing up a book.
    I’ll just pick on a few points.

    “If some competent framework (not a pet/sunday project) had used this technique ItsNat wouldn’t be born. Of course I’m absolutely biased”.
    I’m a little disapponted that you decided to post biased opinion on cometdaily rather than doing it in ItsNat’s site or other appropriate place. As a contributer I kind of feel that you have a responsibility to answer queries without being biased towards your own work.
    Otherwise your article and comments have little more than marketing value. Forgive me if I sound blunt and forgive me hereafter if I sound biased as a result of your bias.

    “We invite you to select output markup regions in every example using the marvelous Mozilla/FireFox’s “View Selection Source” contextual menu option to see how the markup is finally rendered.” ”
    Was there something I missed? because I saw plain HTML markups as HTML markups are supposed to be and rather unsemantic im some cases.
    I write applications that has minimalist markups and I always find ways to do things without polluting elements with unsemantic properties. I can’t speak for other here though.

  28. Jonathan Myer Says:

    “A very, very, very bad argument and too simple. ….”
    Your are right about that :-)

    “One analogy: Swing was very bad, SWT was born to replace the “wrong” approach of Swing, today SWT is the “wrong” approach and Swing approach shines. Tomorrow? I don’t know.”

    You seemed to have missed out the whole politics behing SWT. Or the politics that is at play behind the scene of the Eclipse foundation. And personally I never liked the idea behind SWT and never used it. I like to steer clear of political agenda and rhetorics and bias too. :-)

    “Some statistics say around 25% of developers are still stuck to Java 1.4″
    That doesn’t say anything about the percantage of web applications still dependent on Java 1.4.
    And it says far less about the chances of those applications developed in Java 1.4 that will be needing comet support and it says even less about why those applicaitons cannot be upgraded to Java 1.5 or greater. That statistic seems irrelevent to the discussion at hand here, if I may say.
    Isn’t JDK 7 around the corner?

    “Most of web applications out there are based on full reload per click, most of the time with no significative use of JavaScript”
    That’s only because they chose to do so and probably because of the delusion of grandeur suffered by many java developers who think javascript is for kiddies and they don’t need it and can write grand web applicaiton without using javascript. Its a cultural problem with java developers and has little to do with Struts, Spring MVC or other frameworks.

  29. Jonathan Myer Says:

    “If your user-base is 40000? Do you mean 40000 CONCURRENT users??!! Forget technical issues and call to Google they will be really interested in your web site and by the way, they know how to deal with 40000 concurrent users, for instance try to ask to Paul Tyma”

    Bill Gates said once that 640 KB of RAM should be enough for anything anybody would ever want to do on a computer.
    Your cynicysm of 40000 concurrent users shows the same attitude as did Mr Gates back then. Hasn’t the world changed a lot since?
    And I didn’t realise that as soon as the magnitude of the problem at hand is large everyone ran off to Google for answers. If everyone has to find answers from google for such problems or if everyone has to get their solution from google for such problems, lord almight help the software industry, because development would seem stagnant everywhere else except at mighty Gogle Headquarters. Of course Google hires the smartest people, but I believe pool of smart people is much larger than Google alone can accommodate.
    And oh, I don;t really care if google is interested in our application and I can tell you google won’t be swallowing our company any soon.

  30. Jonathan Myer Says:

    “Finally, have you tried to search a word in Google? Yes course. Have you seen the amazing Google suggest feature? Don’t say to anyone but the browser is requesting the server every time you press a key.”
    Thanks for the enlightenment, but I have been developing rich-client applicaiton and I use YUI and JQuery on the client side, and I can tell you my server side code does what its supposed to do and doesn’t try to spit out client-side code and markups .
    And what I don’t understand is why you seem to think that web application development is such a complex matter to start with. ITs totally beyond me. I thought the complexity has everything to do with business logic and business requirement not web development per se.
    For your information, I use YUI’s datagrid for pretty much all my crud-centric application and I swear to god I never send more than JSON data from the server and I have never had to change the small library I build on top of YUI for any of the projects I did. And I don’t have to write a single line of code to get the data for any of the CRUD operation. And what’s more our users have the option of inline-editing and panel-based editing, whatever suits their need.
    And the server is just happy to send JSON data and the client is more than happy to recieve JSON data and work with it. And the server works great with the help of just 2 utility classes.
    And oh, I use the same YUI library for my php applicaitons too with absolute no modifiaction. :-)
    And the PHP library consists of just one class :-)
    Excuse me here but I really really like reusable code.

  31. Jonathan Myer Says:

    ” Remember… the “love is in the air” and the “data is in the server”. ”
    Remember there’s javascript enabled browsers out there and PCs these days have helluva lot of processing power. I like to shift the processing burder a little to the browser so that my server can remain happy.

    And oh I’m so over threads these days. I like light-weight concurrent processes better these days.

    As a suggestion, if you want to write up educational or informational articles I suggest you read “Asynchronous HTTP and Comet architectures” by Gregory Roth http://www.javaworld.com/javaworld/jw-03-2008/jw-03-asynchhttp.html?page=1 .
    He’s the author of XSocket NIO Library (which I use) but yet his article has no bias whatsoever. he’s even gone on to provide comet examples using Jetty and Tomcat besides examples with XSocket.
    sadly I have lost faith in cometdaily a bit.

    ( cometdaily::And is there a way to post comments without breaking them into small chunks?)

  32. DylanSchiemann Says:

    Jonathan,

    I’ll look into the comment length issue. Also, with Comet Daily, we’ve tried to assemble contributors representing a variety to technical approaches to solving the scaling problem, Obviously my personal approach (Dojo + whatever on the server) does not really align with the approach Jose Maria takes with ItsNat, but I think his different take on things is at least interesting.

  33. Jose Maria Arranz Says:

    I’m sorry about the comment of “40000 concurrent users” is my bad, you must recognize your tone was a bit aggressive too and your post was first. The “40000″ comment, yes is cynic, but you must read it as a joke too because if your web site has 40000 concurrent requests you are fortunate, your site is a big success in popularity, of course I’m not talking about bills :)

    About “bias” and “lost faith” in CometDaily…

    No one is “bias” free, no one, including you, I have my preferences and interests.

    Alessandro Alinone promotes Lightstreamer
    Jean-Francois Arcand promotes GlassFish
    Andrew Betts promotes Meteor
    Michael Carter and Jacob Rus promotes Orbited
    Alex Russell and Dylan Schiemann promotes Dojo
    Martin Tyler promotes Caplin
    Joe Walker promotes DWR (and Dojo)
    Greg Wilkins promotes Jetty
    Kris Zyp promotes Persevere

    And this is fine and desirable, we aren’t children.

    Anyway I’m trying to be fair, for instance if you read my articles and comments you will read phrases like “every approach has pros and cons”, “NIO gives more flexibility” (in spite of ItsNat doesn’t support asynchronous long polling at the time of writing), “this type of application is not for ItsNat” and so on.

    When I say “I’m biased” I’m saying “don’t have absolute faith in me, my opinion may be “the truth” but is polluted with my interest”, in fact this is a sign of fair play. We are talking about computing sciences not about irrational faith in persons, I don’t want to be a Madoff asking faith to his clients :)

    Finally as you see Comet Daily is bigger than me.

    Merry Christmas

  34. Jonathan Myer Says:

    @DylanSchiemann
    Thanks for the reply.

    I acknowledge the need for diversity in opinion and the value of having different contributers, who have different and interesting take on tackling the scaling problem, and other issues that relate to Comet.
    I’m aware that pretty much all the contributers are aligned to one or another comet related products out there.

    But as a regular reader of this site I really don’t like to waste my time reading articles full of slippery slope arguments and full of arguments more suited to used-car salesmen.
    The problem is I only realize I have wasted my time after reading articles and thus I only wish that there is some better way of not wasting my time or time of other readers like me.
    Its a funny paradox.

    I think that there are a lot of very interesting comet -related products and frameworks out there that I would like to see included in cometdaily articles. It would be great if to see the creators of those frameworks and products contribute to cometdaily.
    MochiWeb(Erlang), Lift(Scala), Apache Cocoon (Java) for their interesting use of javascript for continuation, - to name a few.

  35. Jonathan Myer Says:

    @Jose Maria Arranz

    “you must recognize your tone was a bit aggressive too ”
    Like I mentioned above, I dislike wasting my time reading articles and I like the idea of gaining at least some knowledge or insight into interesting products and concepts by reading articles rather than going through poorly formed arguments and mere rhetoric and bias, especially in sites like cometdaily. And the realisation of wasted time does get me annoyed and that annoyance may reflect in the comments I post.

    “No one is “bias” free, no one, including you, I have my preferences and interests. ”
    Ah, but you seem to forget that your are a contributor and I’m not. As a reader I can express my bias and displeasure in more colourful ways, whereas as a contributor to cometdaily you really should watch what you are saying. You see like you said cometdaily is bigger than you and if you contribute towards lowering the standard of cometdaily with your bias and half-baked articles and comments, cometdaily suffers and so do readers.

    And as for faith, why do u think IPhone enjoys such popularity??
    Is it just because IPhones are flashy - I have seen lots of other flashy phones? Or could it be that people have come to trust Apple to bring out products that meets certain level of user expectations?
    Or in other words isn’t it because people have faith in apple products?
    Did I mention standard somewhere?

  36. Jose Maria Arranz Says:

    I’m absolutely shocked, an article of 1778 words trying to expose the studies made with real tests of other guy (by the way a Google software engineer) is qualified as “poorly formed arguments and mere rhetoric and bias”

    You have made my day.

  37. Jose Maria Arranz Says:

    I forgot to say…

    Merry Christmas

  38. Jonathan Myer Says:

    merry christmas and happy new year too :-)

    and may the gods at google protect your from all the evils of the world

  39. Jonathan Myer Says:

    Ahh, I really wanted to end our arguments here and agree to disagree - but also I feel this need to clarify my stand.
    Those 1778 words would really have been meaningful if you hadn’t tried to make the test outcome made by the google engineer to push your own vested interests. your article really has less to do with the tests done by the revered google god, oops I meant engineer, than your desire to push your own agenda.
    Your reply to other reader’s comments also betray your salesman position (for ItsNat).

    Here’s how your arguments sound like to me:
    Christians are all good people, they fear god and do good things. Since Jose is a Christian he must be be good person and must do good things too.
    Replace Christians with Paul Tyma , and Jose with ItsNat and tweak the grammer, reaarrange a few words and u’ll see where I’m heading.
    Its a terrible terrible way of putting your case forward.

    C’mon give us a break, not everyone who reads cometdaily are idiots who can be fooled by your sloppy arguments.

    I really really insist you read “Asynchronous HTTP and Comet architectures” by Gregory Roth http://www.javaworld.com/javaworld/jw-03-2008/jw-03-asynchhttp.html?page=1 .

    I hope you can learn something from there and in turn write articles that gives readers the same kind of learning opportunity as provided by that article.

    Excuse me here but I have a certain distaste for over-zealous salesmen.

    Happy new year

  40. Bestes OS f Says:

    [...] not blocked) has a cost. Contended synchronization has a cost. The thread stack occupies memory. Comet Daily

Leave a Reply



Copyright 2014 Comet Daily, LLC. All Rights Reserved