“Hello World” for Sockets with Lightstreamer

by Alessandro AlinoneJuly 29th, 2008

This is the third article in a series aimed at illustrating how to develop Lightstreamer v.3.x Adapters based on various technologies:

  1. “Hello World” with Lightstreamer: An introduction to Lightstreamer’s data model, Web Client API, Java Data Adapters, and Server deployment, through the development of a very basic “Hello World” application.
  2. “Hello World” for .NET with Lightstreamer: The .NET version of the Data Adapter used in the “Hello World” application, showing both a C# and a Visual Basic port.

In this third installment, I will show a Data Adapter that communicates with the Lightstreamer Server through plain TCP sockets, instead of leveraging higher level abstractions as I did with the Java API and the .NET API. The rationale for this is to enable the development of Data Adapters based on technologies other than Java and .NET. This way, it is possible to inject real-time updates into Lightstreamer Server from programs written in C, PHP, Ruby, or any other language that allows client TCP socket programming.

The Architecture

On the client side, we will keep the same exact HTML front-end used in the two previous installments. For an explanation of the HTML/JavaScript code, please see the “Creating the front-end” section of the first article.

On the server side, we will leverage the Lightstreamer Adapter Remoting Infrastructure (ARI):

Adapter Remoting Infrastructure

This is the same architecture used in the .NET example, but in this case the Remote Data Adapter is not a .NET application, but any process that opens two sockets with the Proxy Data Adapter and implements the ARI Protocol over TCP. As in the previous examples, we will not code a custom Metadata Adapter, but will use a default one. So the Remote Metadata Adapter will not be present.

Let’s recap. On the server side, this new example will be comprised of:

  • Proxy Data Adapter: A ready-made Data Adapter, based on Java, which is provided as part of the Adapter Remoting Infrastructure (available in the free and commercial distributions of Lightstreamer Server v. 3.x).
  • Remote Data Adapter: The subject of this article…
  • LiteralBasedProvider: A ready-made Metadata Adapter, based on Java, which is provided as part of all the free and commercial distributions of Lightstreamer Server v. 3.x. The LiteralBasedProvider replaces the Proxy Metadata Adapter in the diagram above.

So, what about the Remote Data Adapter? We could implement it in any language that supports socket programming, but in this article we will do something more interactive, which does not require any programming. We will use a very normal telnet client to connect to the Proxy Data Adapter and will manually play the ARI Network Protocol. Should be fun…

The Proxy Data Adapter listens on two TCP ports and the Remote Data Adapter has to create two sockets. One socket is used for interactions based on a request/response paradigm (this is a synchronous channel). The other socket is used to deliver asynchronous events from the Remote Adapter to the Proxy Adapter (this is an asynchronous channel). Therefore, our Remote Data Adapter will be comprised of two telnet windows.

Setting Up the System

If you didn’t already do it in the previous installments, download and install Lightstreamer Moderato.

Now, let’s deploy the Adapter pair (Proxy Data Adapter + LiteralBasedProvider) in a new folder. Go to the “adapters” folder of your Lightstreamer Server installation. You could already have a “StockList” folder (containing a pre-installed demo Adapter), plus a “HelloWorld” and a “ProxyHelloWorld” folders (if you followed the previous two articles). In any case, you don’t need these three folders, as we are going to create a new one for this example.  Let’s call this new folder (created inside “adapters”) ”ProxyHelloWorldSockets“. Then create a “strong>lib” folder inside “ProxyHelloWorldSockets” and copy the “ls-proxy-adapters.jar” file from “Lightstreamer/DOCS-SDKs/sdk_adapter_remoting_infrastructure/lib” to “Lightstreamer/adapters/ProxyHelloWorldSockets/lib”.

Create a new file in “Lightstreamer/adapters/ProxyHelloWorldSockets”, call it “adapters.xml“, and use the following contents:

<?xml version="1.0"?>
<adapters_conf id="PROXY_HELLOWORLD_SOCKETS">
 
  <metadata_provider>
    <adapter_class>com.lightstreamer.adapters.metadata.LiteralBasedProvider</adapter_class>
  </metadata_provider>
 
  <data_provider>
    <adapter_class>com.lightstreamer.adapters.remote.data.NetworkedDataProvider</adapter_class>
      <param name="request_reply_port">7001</param>
      <param name="notify_port">7002</param>
      <param name="timeout">36000000</param>
  </data_provider>
 
</adapters_conf>

I have chosen TCP port 7001 for the request/response channel and TCP port 7002 for the asynchronous channel. Feel free to use different ports if such numbers are already used on your system.

Notice the “timeout” parameter. It sets the maximum time the Proxy Adapter will wait for a response from the Remote Adapter after issuing a request. The default value is 10 seconds, but in this case the Remote Adapter is played by humans, so I have configured a very high value (10 hours), to do relaxed experiments without the pressure of any timeouts.

Finally, let’s deploy the simple Web resources. Create a “HelloWorldSockets” folder under the “Lightstreamer/pages” folder. Put in this new folder the “index.htm” file available from this archive (the page is very similar to those of two previous articles; we just changed the Adapter name to “PROXY_HELLOWORLD_SOCKETS” and added the “pushtable.setSnapshotRequired(true)” line to always get the current state of the fields). Now we have to deploy the JavaSscript libraries used by our page. Let’s create an “LS” folder under “HelloWorldSockets” and copy to it all the files located in “Lightstreamer/DOCS-SDKs/sdk_client_web/lib”. For further details on the page deployment, please see the first article.

Let’s Play

Start Lightstreamer Server from a command or shell window (I will call this the “log window“). You should see something like this:

[...]
28.Jul.08 17:56:01,437 < INFO> Lightstreamer Server starting in Moderato edition
28.Jul.08 17:56:01,468 < WARN> JMX management features not available with the current license
28.Jul.08 17:56:01,500 < INFO> Started HTML Adaptor for JMX on port 6666
28.Jul.08 17:56:01,531 < INFO> Started JMXMP Connector for JMX on port 9999
28.Jul.08 17:56:01,578 < INFO> Loading Metadata Provider PROXY_HELLOWORLD_SOCKETS
28.Jul.08 17:56:01,578 < INFO> Loading Data Provider PROXY_HELLOWORLD_SOCKETS
28.Jul.08 17:56:01,593 < INFO> Connecting...
.

The Server is waiting for the “PROXY_HELLOWORLD_SOCKETS” Remote Data Adapter to connect to the two TCP ports we specified above (7001 and 7002). The Server initialization will complete only when these connections succeed.

Now open other two command or shell windows.
In the first one (which I will call the “request/response window“) type:

     telnet localhost 7001

In the second one (which I will call the “async window“) type:

     telnet localhost 7002

The Server initialization will complete and in the log window you should see something like this:

[...]
28.Jul.08 17:56:01,437 < INFO> Lightstreamer Server starting in Moderato edition
28.Jul.08 17:56:01,468 < WARN> JMX management features not available with the current license
28.Jul.08 17:56:01,500 < INFO> Started HTML Adaptor for JMX on port 6666
28.Jul.08 17:56:01,531 < INFO> Started JMXMP Connector for JMX on port 9999
28.Jul.08 17:56:01,578 < INFO> Loading Metadata Provider PROXY_HELLOWORLD_SOCKETS
28.Jul.08 17:56:01,578 < INFO> Loading Data Provider PROXY_HELLOWORLD_SOCKETS
28.Jul.08 17:56:01,593 < INFO> Connecting...
28.Jul.08 18:31:05,078 < INFO> Connected
28.Jul.08 18:31:05,093 < INFO> Loading Metadata Provider STOCKLISTDEMO
28.Jul.08 18:31:05,093 < INFO> Loading Data Provider STOCKLISTDEMO
28.Jul.08 18:31:05,234 < INFO> Pump pool size set by default at 1
28.Jul.08 18:31:05,265 < INFO> Notify receiver '#1' starting...
28.Jul.08 18:31:05,265 < INFO> Events pool size set by default at 1
28.Jul.08 18:31:05,265 < INFO> Lightstreamer Server 3.4.8 build 1398 starting...
28.Jul.08 18:31:05,281 < INFO> Request sender '#1' starting...
28.Jul.08 18:31:05,281 < INFO> Reply receiver '#1' starting...
28.Jul.08 18:31:05,312 < INFO> Server "Lightstreamer HTTP Server" listening to *:8080 ...
.

Now we are ready to interact with the Server.

Let’s connect a client to activate a subscription. Open a browser window and go to: http://localhost:8080/HelloWorldSockets

In the log window you will see some information regarding the HTTP interaction between the browser and the Lightstreamer Server.

In the browser window you will see:

loading...
loading...

The “greetings” item has been subscribed too by the Client, with a schema comprised of the “message” and “timestamp” fields. The Server has then subscribed to the same item through our Remote Adapter (due to the fact that Lightstreamer Server v.3.x is based on a “Publish On-Demand” paradigm). This subscription will manifest itself as a request in the requets/response window, similar to the following:

10000011b6a823e31|SUB|S|greetings

The first string is the unique ID of that request and will change every time. Let’s respond saying that we accept such subscription. We can do this by typing the following string in the requests/response window and hitting Enter:

     10000011b6a823e31|SUB|V

Note: Replace “10000011b6a823e31″ with the actual ID you received, otherwise the subscription will not succeed and you will see a warning in the log window.

Our Remote Data Adapter has now accepted to serve events on the “greetings” item. It’s time to inject some events by hand, through the async window. With most telnet applications you will not see anything will typing in the async window, so it is better to use copy and paste. Paste the following string, then hit Enter:

     0|UD3|S|greetings|S|10000011b6a823e31|B|0|
     S|timestamp|S|Now is the time|S|message|S|Hello socket world!

Note: Make sure to paste everything on a single line (the text above was split on two lines to fit in the page). And again, replace “10000011b6a823e31″ with the actual ID you received.

Now look at the browser window and enjoy the results of this effort:

Hello socket world!
Now is the time

We can push more events on the “greetings” item, leveraging the same two fields (”message” and “timestamp”) ans sending arbitrary data. For example, paste this in the async window (always on a single line and replacing the ID):

     0|UD3|S|greetings|S|10000011b6a823e31|B|0|
     S|message|S|What do you call a fish with no eyes?|S|timestamp|S|A fsh

The Network Protocol

In the examples above we scratched the surface of the ARI Network Protocol. By delving into deeper details, you will see that it is quite straightforward. The full specification is available in the “ARI Protocol.pdf” document, which is available in the “Lightstreamer/DOCS-SDKs\sdk_adapter_remoting_infrastructure\doc” folder of any distribution of Lightstreamer Server v.3.x.

The Remote Data Adapter can only receive two synchronous requests: subscribe and unsubscribe. It can send three asynchronous events: update, end of snapshot, and failure.
The Remote Metadata Adapter (which was not covered in this article) can receive more synchronous requests, as its interface is a bit more complex than the Data Adapter, but it does not send any asynchronous events at all (in fact it uses one TCP socket only).

Final Notes

In reality, of course, you will never implement a “human-driven” Adapter as we did in this article, but this approach was useful, from a didactive perspective, to illustrate the basic principles of the Lightstreamer Adapter Remoting Infrastructure (ARI). If you need to develop an Adapter based on technologies other than Java and .NET, the ARI makes the trick.

Should you develop any Adapter in PHP, Ruby, Python , Perl, or any other language, feel free to let us know, by posting a comment here on the Lightstreamer Forums.

10 Responses to ““Hello World” for Sockets with Lightstreamer”

  1. john casley Says:

    #!/usr/bin/perl

    use IO::Socket;

    $control = IO::Socket::INET->new(”127.0.0.1:7001″) or die “$@\n”;

    $feed = IO::Socket::INET->new(”127.0.0.1:7002″) or die “$@\n”;

    $answer = ;
    (@tuples) = split(/\|/,$answer);
    print $control “$tuples[0]|SUB|V\n”;

    while () {
    ($sec,$min,$hour,$mday,$month,$year,$wday,$yday,$isdst) = localtime();
    $timestamp = sprintf(”%d/%02d/%02d %02d:%02d:%02d”,$year+1900,$month+1,$mday,$hour,$min,$sec);
    print $feed “0|UD3|S|greetings|S|$tuples[0]|B|0|S|timestamp|S|$timestamp|S|message|S|Hello socket world!\n”;
    print “$timestamp\n”;
    sleep(1);
    }

    close($control);
    close($feed);

  2. Jarius Says:

    Python virtuall identical to Perl:

    #!/usr/bin/env python

    import socket, sys, time

    try:
    control = socket.socket(socket.AF_INET, socket.SOCK_STREAM).connect((”localhost”,7001))
    feed = socket.socket(socket.AF_INET, socket.SOCK_STREAM).connect((”localhost”,7002))
    except socket.error, e:
    sys.exit(”Error! %s\n” %(e))
    answer = control.recv(2048).split(”|”)[0]

    control.send(”%s|SUB|V\n” % (answer))

    while True:
    ts = time.strftime(”%x %X”, time.localtime())
    feed.send(”0|UD3|S|greetings|S|%s|B|0|S|timestamp|S|%s|S|message|S|Hello socket world!\n” % (answer, ts))
    time.sleep(1)
    control.close()
    feed.close()

  3. Jarius Says:

    Sorry, correction the control and feed entries:
    import socket, sys, time

    try:
    control = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    control.connect((”localhost”,7001))
    feed = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    feed.connect((”localhost”,7002))
    except socket.error, e:
    sys.exit(”Error! %s\n” %(e))
    answer = control.recv(2048).split(”|”)[0]

    control.send(”%s|SUB|V\n” % (answer))

    while True:
    ts = time.strftime(”%x %X”, time.localtime())
    feed.send(”0|UD3|S|greetings|S|%s|B|0|S|timestamp|S|%s|S|message|S|Hello socket world!\n” % (answer, ts))
    time.sleep(1)
    control.close()
    feed.close()

  4. Alessandro Alinone Says:

    By the way, if you want the Lightstreamer Server not to shut down in case the Remote Adapter disconnects (in the example above, if you close the telnet connection), just change the data_provider adapter class.

    In adapters.xml, change of from:
    com.lightstreamer.adapters.remote.data.NetworkedDataProvider
    to:
    com.lightstreamer.adapters.remote.data.RobustNetworkedDataProvider

  5. Ryan Says:

    If I wanted to write my own chat system (or similar) in PHP, I’m not really understanding when my PHP script connects to the LS server and when it disconnects. I was able to use this example http://www.lightstreamer.com/vb/showthread.php?t=515 to create the Hello World example, but that works by me running the php script from the commandline and have it continuously running. I want something more real world. A user enters some text and clicks “Send.” That data is then POSTed to my Apache server where PHP picks it up. I can then use PHP to connect (like the example I listed), but I don’t want to run in an infinite loop. I just want to write the message and leave. But removing the loop makes this fail after one POST. Any ideas?

    Thanks in advance.

  6. Alessandro Alinone Says:

    @Ryan: Your Remote Adapter (be it based on PHP or any other technology) will need to keep persistent TCP connections to Lightstreamer Server (as shown in the first picture above) and to manage these sockets both synchronously and asynchronously. In other words, your Remote Adapter needs to have its own thread of execution. I’m not sure if you can accomplish that without and infinite loop in PHP.

  7. Ryan Says:

    Thanks for the reply, but now I’m more confused. It sounds like you are sayign I need to write my own web server in PHP to be constantly listening for messages. Tehn once it receives them, it pass them to the LS server? Is that correct? Sounds like overkill.

    Let’s start from scratch. I want to create a chat app where one use types a message and all users receive that message. How do I do that with Lightstreamer?

  8. Alessandro Alinone Says:

    Actually, you don’t need to write a Web Server in PHP, but just a Data Feeder, which connects to Lightstreamer Server via TCP sockets. So you need to find a way for your PHP code to keep a persistent TCP connection to Lightstreamer Server.

  9. Ryan Says:

    Is there no way to directly input data into the Lightstreamer with Javascript?

    Also, why the necessity for a persistent connection to Lightstreamer just to input data? What about the RobustNetworkedDataProvider feature?

    Thanks in advance.

  10. Alessandro Alinone Says:

    The background idea is that on the back-end, the Lightstreamer Server and its Adapters are tightly coupled, thus communicating over dedicated persistent connections. The RobustNetworkedDataProvider is intended to provide some resiliency over short network disconnections between the Server and the Adapters, but is not meant to provide offline operations, as we are talking about a real-time system.

    If you just need to send messages to the Lightstreamer Server from the browser, in order to dispatch them to some users, you probably don’t need any PHP code at all, as this is easily handled by very simple Java code, as shown in the online examples. Please see the following demos:
    - http://www.lightstreamer.com/demos.htm#ChatDemo
    - http://www.lightstreamer.com/demos.htm#RoundTripDemo
    - http://www.lightstreamer.com/demos.htm#MessengerDemo


Copyright 2015 Comet Daily, LLC. All Rights Reserved