Today, many Comet servers come with vague information regarding their performance. However, performance is one of the most important features of a Comet server, a primary reason to exist besides a traditional web server. The purpose of this article is to present five important metrics and explain why they are essential when looking at a Comet server from a performance perspective. Some concrete values for each metric are also provided, which can be used as a good basis for comparison.
The concrete values of the metrics are obtained from a recent benchmark of Migratory Push Server. A complete kit to replay the benchmark can be freely obtained from Migratory.
High Vertical Scalability
The scalability of a Comet server is determined by the number of users it can handle concurrently from a single instance.
A high-performance Comet server should achieve high scalability when running on a less expensive machine and it should achieve even higher scalability when running on more expensive machines. When this is true, one can say that the high-performance Comet server scales vertically.
Another method to achieve scalability is to use multiple instances of a Comet server. This is called horizontal scalability. While support for horizontal scalability is a must for any enterprise Comet server – because it usually represents the basis for other important features of a Comet server such as load-balancing, fault-tolerance, and guaranteed message delivery – horizontal scalability alone could increase significantly the total cost of ownership or be simply insufficient in some situations.
At Migratory, it is not unusual to receive inquires asking for HTTP streaming to millions of concurrent users. A Comet server handling thousands of concurrent users is clearly not an option, even if it scales horizontally. Obviously, in such situations, multiple instances of a Comet server must be used on multiple machines, but each instance must be vertically scaled as far as possible before scaling horizontally by adding more machines.
Example — Migratory Push Server is able to handle 1 million concurrent users from a single instance running on an entry-level machine (see the section “Extreme Number of Clients” of the benchmarks).
Low Message Latency
The latency of a Comet server is defined as the time for a message to propagate from the server side to the user. It can be accurately computed as the delta between the time a message is created on the server side and the time the message is received by the user.
Concerning the latency of a Comet server, the most important statistics to look at are its average and standard deviation.
Example — Migratory Push Server is able to push real-time data to 10,000 users where each user receives 100 messages/second, each message of 50 bytes, with average latency of 143 milliseconds and standard deviation of 79 milliseconds, while running on an entry-level machine utilizing a 1Gb Ethernet network (see the section “Very High Update Rates” of the benchmarks).
Low Bandwidth Consumption
To achieve low bandwidth consumption, the protocol used by the Comet server to deliver data to its users must be as efficient as possible. It should add only a small overhead to the actual data payload.
Example — Migratory Push Server adds to each message the following overhead to the actual data payload: 42 bytes when pushing data to the Internet Explorer, and 20 bytes for all the other browsers. The overhead is constant and also includes information used for the guaranteed delivery of data.
The bandwidth used to push real-time data with Migratory Push Server to 16,000 users where each user receives 5 messages/second, each message of 1 kilobyte, is 85 megabyte/second, while the actual data payload is about 79 megabyte/second (see the section “Large Data Messages” of the benchmarks).
The throughput is the average rate of data delivered by a Comet server.
Example — Migratory Push Server running on an entry-level machine saturates 1 Gb Ethernet being able to push real-time data with a throughput of 106 megabyte/second to 10,000 users where each user receives 5 messages/second, each message of 2 kilobytes (see the section “2-Kilobytes Messages” of the benchmarks).
High Connection Rate
The connection rate of a Comet server is defined as the rate of new, established connections it can accept.
This metric is particularly important for an application where many users connect frequently to it. Also, in a fault-tolerant deployment using multiple instances of a Comet server, after the failure of one instance, the remaining instances should be able to rapidly absorb the users of the failed instance.
Example — Migratory Push Server is able to accept new users with an average connection rate of 5000 users/second.
The five dimensions of performance above are essential for any high-performance Comet server. They can be measured independently of the specific architecture of a particular Comet server. In this way, they could be used for benchmark comparisons of the Comet servers.