On my recent excursion into non-blocking servers I came across Comet, server push technologies and then Web Sockets. I was late arriving at the Comet party, but I think I have arrived at the Web Sockets party just in time. The final standard is still being evovled and at the time of writing, the only browser supporting it by default is Chrome. So to get started, I had a look at what was around. Initially, I wasn't going to write a blog article about it, I just wanted to learn about this new thing.
My only requirement was that the server implementation was Java based. The reason is simple - my entire technology stack is based on Java EE and I want to be able to integrate all my existing stuff into any new server, without the need for an integration bus. I like being able to drop a JAR file from one project into another one and to be up and running immediately.
So I had a quick look at jWebSockets, Grizzly (Glassfish), Jetty and Caucho's Resin. All similar, yet all somewhat different. Most different, was jWebSockets, because they have gone to the extent of building their own server, instead of basing their solution on an existing server. I had a good long rant about that kind of thing when I blogged about node.js, so I won't start again. But jWebSockets has a real issue to face in the coming years. JSR-340 talks about support for web technologies based around HTML5 and there is talk that this JSR will be part of Servlet 3.1 and be included in Java EE 7 at the end of next year. If that happens, jWebScokets will have the problem that their server implementation will become deprecated. And because their server doesn't offer anything else from Java EE, it is likely to lose a large part of it's market share. Don't get me wrong, they are still innovating on the client side, with a bridge to allow any browser to use websockets, and creating things like Java SE web socket clients. But server side isn't looking so hot - just take a look at the huge config file which you need to supply to get your plugins to work.
Anyway, I wasn't really enthusiastic about any of the solutions I saw. Some are servlet based (which is good), and all rely on you creating a new instance of a listener, handler, plugin or whatever they choose to call it. But what struck me was, by doing that, you lose the benefits of running inside the container. What I mean by this is that if you think about a servlet or an EJB (and I'm talking Java EE 6 and beyond here), the components you write have the benefit that the container can take care of cross cutting concerns like transactions, security and concurrency as well as provide you with injected resources, so that all you have left to do is write business code. So the reason I was not enthusiastic about any of the solutions listed above, was that what I felt to be glaringly obvious, has been missed. Sure, an inner class within a servlet could make use of resources from the servlet, but nothing I found on the net suggested that should be the normal way to program a web socket solution. And I'm not convinced that a reference in an inner class to a resource from the servlet couldn't go stale between messages received. I would rather the container always injected fresh resources just before my event handling method gets called, just as happens in the lifecycle of a servlet or EJB instance. Just think about a database connection timeout occurring between messages? If the container ensured the connection was always fresh, then the app developer wouldn't need to worry about technicalities.
HttpServlet has methods for handling GETs, POSTs, et al. Why shouldn't a component handling a Web Socket not be a
WebSocketServlet and have methods for handling the events for handshaking, opening, messaging, closing and error handling? I would benefit, because the servlet could have resources injected into it before the event handler methods get called, and the container could start any transactions that are required, or check security, etc.
So without realising that this would actually be quite hard to implement, I set upon taking Tomcat 7 apart, and building web sockets into it, in a way which fulfilled my requirements.
What resulted was (what I think is) a pretty sexy servlet solution. Please take a few minutes to study the code and especially the comments below, because it is the essence of this article.
To me, this is what a serverside web socket component should look like. It is really similar to a sevlet or an EJB or a web service - things we are now familiar with. The learning curve is mini.
But getting Tomcat to use this thing was a little harder. First off, my approach was to simply write a new Connector and configure it in the
server.xml. I gave it a port number and Tomcat willingly instantiated my new class:
Of course, I cheated here a bit and stuck my class in a Coyote package. To get this to work, I unzipped the Tomcat 7.0.14 source and told Eclipse where to find it. I set up the build folder and added that to the classpath in the
catalina.bat batch, before any of the other JARs on the classpath, and I did this just near the end of the batch where the call to the Java process is made. I then ran my modified Tomcat by simply starting the catalina batch, using remote debugging to make life easier.
I initially based my non-blocking connector on the non-blocking server which I have written and extended for my past few blog articles. My idea was intercept any web socket handshake requests that arrived and to quickly return a self built handshake response. I would then call the requested servlet and run it using Servlet 3.0 async support, so that the container would keep the connection open. The messages which the client then sent after the handshake would be handled by that servlet.
The idea didn't quite work though, because the servlet used the async support and my connector got callbacks from Tomcat in order for me to do stuff with threads, handle events and state changes. I was a little lost because I didn't understand the implementation details of Tomcat well enough, and after a while I gave up, because it was becoming clear that I would have to build quite a lot of stuff that already existed in the HTTP connectors which shipped with Tomcat. I came to realise that instead of reinventing the wheel, I would do better by creating my own versions (or even sub-classes) of the Coyote
Http11NioProtocol which Tomcat uses to implement the HTTP connectors. While these are somewhat complicated, the advantages of specialising them are firstly that I am not reinventing the wheel, and secondly that I would get HTTPS (or WSS as it is with Web Sockets) for free (theoretically, I haven't yet tested it, but it is simply encapsulated so it should work with almost no problems).
While it was straight forward to copy the two relevant classes, the next headache was to get the browser to accept the handshake response. I wasn't sending the handshake response as a self built String like I had with my own non-blocking connector. Instead I was making as much use of servlet technology as I could, and building the response by setting the relevant headers on it and pushing the 16 byte response key onto the output stream of the response. Unlike a normal HTTP response, the browser expects any handshake responses to stick to a pretty strict specification. Things like a simple date header in the response (which Tomcat kindly sends by default) cause the browser to close the web socket. So with a bit of fiddling in the
WebSocketsNioProcessor#prepareResponse() method, I soon had the response header looking perfect, and the browser started to play along.
But the next problem wasn't as simple to fix. The message handling didn't work, at all. Connections kept getting closed quickly and I couldn't figure out what was going on.
After a few headaches, I thought back to how Comet is implemented in Tomcat. I slowly realised, that what I needed was to simply piggy back the Comet infrastructure in Tomcat. In Tomcat, you create a Comet handler by simply implementing the
CometProcessor interface in your servlet. The container checks during the initial request whether the servlet which will service the request implements the interface, and if it does, the request is marked as being a comet request. That means the container keeps the connection open and forwards future events (e.g. bytes / messages) to the relevant handler method which your servlet implements. I made my base servlet (
WebSocketsServlet) implement the
CometProcessor interface, and added the
event(CometEvent event) method using the final keyword, so that application developers wouldn't ever think of overriding it, meaning they were protected from having the option of knowing about Comet. This base class servlet then encapsulated the web sockets events by hiding the fact that I had cheated by using Comet. The event method from the Comet processor simply passes the event on to the relevant doXXX method in my base servlet, which application programmers would subclass. This is pretty much what the
HttpServlet base class does in it's service method.
WebSocketsServlet is the Web Sockets equivalent of the
HttpServlet, and as such, I put it into the
javax.servlet.websockets package - i.e. to be supplied as part of the JSR which gets around to specifying the way in which Java EE containers should handle web sockets.
WebSocketsServlet also deals with
WebSocketSession objects, rather than HTTP equivalents. Each of these derives from the relevant
ServletXXX rather than
HttpServletXXX object, because when handling a web socket frame, it makes no sense what so ever to be able to do things like setting the headers on the response. Headers are only relevant to the handshake, and not anything which an application developer needs to concern themselves with. Again, these interfaces also sit in the
javax.servlet.websockets package, because they are the analagous classes belonging to the specification, rather than just Tomcat. Sadly, the
ServletRequest class has stuff in it which is too specific to HTTP, like
getParameter(String), as well as other stuff which I would prefer to hide for web sockets, like
getInputStream(). In order to make the
WebSocketsServletRequest fit really well into the
javax package, I think the
ServletRequest might need to be broken into a super and a sub-class. Whether that would be possible, considering the billions of lines of Java code already out there, I don't know...
Things like wrapping the response String (bytes) in a web socket frame (i.e. leading 0x00 and trailing 0xFF byte) are also handled transparently by the
WebSocketsServletResponse implementation. In fact, I went a step further. After listening to two interesting videos (with Jerome Dochez, the Architect of Glassfish - The Future of Java EE and Jerome discusses early plans for Java EE 7) I was inspired by some of the things he talked about. He spoke about not wanting to mess around with parsing strings and getting the container to handle XML and JSON. So... I added that to Tomcat too. You'll notice in the code near the top, the handler method which is called for handling message events is called
doJSONMessage(WebSocketsServletRequest, WebSocketsServletResponse) rather than say simply
doMessage. The reason is, the web sockets client (running in the browser) lets you send a header containing the protocol which it will use. This protocol appears to be a free text which the application can choose. In my implementation I went with XML, JSON, TEXT and BIN (binary), although XML and BIN aren't fully implemented. So when the client creates the web socket like this:
new WebSocket("ws://localhost:8085/nio-websockets/TestServlet", "JSON");
that JSON parameter is used by the container to decide which exact handler method in the servlet it will call. It gets better too, because instead of then having to manually handle the JSON string, the container does the magic, and from the
WebSocketsServletRequest which is passed into the message handler method, I can retrieve the object, using the
#getDataObject() method. And thanks to generics, it even passes that object back to me without me having to use a cast - the application servlet which implements the
WebSocketsServlet base servlet specifies what type of data object it expects.
How does that magic work, I hear you asking? Well, its not that complicated. The base servlet receives a byte array from the Comet event. It looks at the original request header and realises it should treat that request as a JSON object. It then uses XStream which knows XML as well as JSON to unmarshal the incoming request. One last bit of magic that is required is that it needs to know which class say a "container" object in the JSON string maps too - i.e. which class should it use to stick the JSON data into? Well, XStream lets you define such aliases. I had a think about where else this happens, and JAX-Binding does this. So instead of the
@XmlType annotation for JAXB, I create the
javax.json.bind.annotation.JsonType annotation, which is applied to data objects, like this:
Here, the "name" attribute of the annotation says that this class is mapped to JSON objects whose name is "container". So JSON like this:
is simply unmarshalled into an instance of the
CommandContainer class, so that the Java application can use the object immediately, rather than screwing around with strings and unmarshalling itself. The Tomcat container parses annotations during startup and I simply stuck some extra code in there to note down these mappings so that when the base servlet sets the raw data into the request, the request implementation (which is a Coyote object, rather than a javax object) can use something like XStream and these mappings to do the unmarshalling. I had to watch out for the class loader here to - luckily XStream lets you set the classloader, because if you don't set it, you are stuck with one that doesn't know the webapp. I simply took the classloader from the servlet context (which I got out of the
HttpServletRequest which the container actually passes to the Comet processor in the Comet Event), and it has a classloader which knows the webapp classes. Regardless of whether such JSON annotated data classes are in the web-inf/classes folder, or in jars, the container can deal with them.
onerror callback method. Let's cross our fingers hey?
Resources, like EJBs, Entity Managers (JPA) and Web Services are also easily used - because in Java EE 6 I can inject them all using the container. While Tomcat doesn't support EJBs/WebServices per se, I still stuck some into the above example, to illustrate how such a servlet might look in a fully fledged Java EE app server. To make it work, I hacked Tomcat to look for those annotations and if nothing was found in the JNDI tree, it simply injected a new instance of the class - ok, its cheating, but great for a proof of concept ;-)
Note that while the servlet shown above might not be a typical application needing web sockets (because its a basic request/response app), it demonstrates the way I would like to see web sockets added to Java EE. It wouldn't be hard at all to write a servlet like this which handled a chat app, or for example an app where a user draws on their screen, and all other members see the drawing update on their screens in pretty much realtime. I might go on to play with something like that, but this article already discusses some of the issues facing Web Sockets.
A few weeks ago, I blogged about how node.js would need to provide the things like I have described in this article to become mature. Java EE has exactly the same challenges if it is to remain mature in the future, as new technologies are integrated into it.
Well, that's pretty much it. What do you think?
Patches for Tomcat 7.0.14 are available here.
The webapp containing the servlet shown above can be downloaded here.
Other useful links:
- A blog article about Jetty websockets
- Source code for the Grizzly (Glassfish) web sockets solution
- A discussion on whether Tomcat will have websockets soon
- jWebSockets - a standalone websocket server
- XStream - marshals/unmarshals XML & JSON
- A video of the conference presentation about the future of Java EE
- A video interview with Glassfish's architect
- Tomcat docs for Comet
- Caucho's Resin Websockets
- HTML5 Web sockets spec
- The link to start the demo I created, once you install it locally - http://localhost:8080/nio-websockets/index.jsp
Copyright © 2011 Ant Kutschera
In my previous blog posting, I wrote about what I would expect node.js to do in order to become mature. I introduced the idea of having a framework which lets you define a protocol and some handlers in order to let the developer concentrate on writing useful business software, rather than technical code, in a very similar manner to which Java EE does. And through that posting, I came to learn about a thing called Comet. I had stated that using a non-blocking server wouldn't really be any more useful than a blocking one in a typical web application (HTTP), and so I created an example based on my own protocol and a VOIP server, for streaming binary data to many concurrently connected clients.
I have now read up on Comet and come to realise there is indeed a good case for having a non-blocking server in the web. That case is pushing data back to the client, like for continuously publishing latest stock prices. While this example could be solved using polling, true Comet uses long-polling or even better, full on push. A great introduction I read was here. The idea is that the client makes a call to the server and instead of the server returning data immediately, it keeps the connection open and returns data at some time in the future, potentially many times. This is not a new idea - the term Comet seems to have been invented in about 2006 and the article I refer to above is from 2009. I think I've arrived at this party very late :-)
My new found case of server push for non-blocking HTTP, and a strong curiosity drove me to knuckle down and start coding. Before long, I had a rough implementation of the HTTP protocol for my little framework, and I was able to write an app for my server using handlers that subclassed the
HttpHandler, which was for all intents and purposes, a Servlet.
To get Comet push to work properly, you get the client to "log in" and register itself with the server. In my demo, I didn't check authorisations against a database, like you might do for a real app, but I had the concept of a channel, to which any browser client could subscribe. During this login, the client says which channel it wants to subscribe to, and the server adds the clients non-blocking connection to its model. The server responds using chunked transfer encoding, because that way, the connection stays open, and you don't need to state up front how much data you will send back. At some time in the future, when someone publishes something, the server can use the connection which is still open contained in its model, to push that published data back to the subscribed client, by sending another chunk of data.
The server implementation wasn't too hard, but the client posed a few problems, until I realised that the data was arriving in the ajax client with a ready state of 3, rather than the more usual 4. The ajax client's
onreadystatechange callback function was also given every byte of data in it's
responseText, rather than just the new stuff, so I had some fiddling around until I could get the browser to just append the new stuff to the
innerHTML attribute of a
div on my page. Anyway, after just a few hours, I had an app that worked quite well. But it wasn't entirely satisfactory, partly because as I stated in the previous posting, the server is still a little buggy, especially when the client drops a connection, because for example the browser page is closed. I had also ended up implementing the HTTP protocol for my framework, which seemed to be reinventing the wheel - servlet technology does all this stuff already, and much better than I can hope to do it. One of the reasons that I don't like node.js is that everything is being reinvented.
So, like the article which I referenced above indicated, version 3.0 of Servlets should be able to handle Comet. I downloaded Tomcat 7.0, which has a Servlet 3.0 container, and I ported my app code to proper Servlets. It took a while to work out exactly how to use the new asynchronous parts of servlets because there aren't all that many accurate tutorials out there. The servlet specs (JSR 315) helped a lot. Once I cracked how to use the async stuff properly, I had a really really satisfying solution to my push requirements.
The first step, was to reconfigure Tomcat so that it uses non-blocking (NIO) for its connector protocol. The point of this is that I want to keep a connection open to the client in order to push data to it. I can't rely on the one-thread-per-request paradigm, because context switching and thread memory requirements are likely to be a performance killer. In Tomcat's
server.xml file, I configured the connector's protocol:
rather than the normal:
All you need to do to get Tomcat to turn into an NIO server is change the protocol attribute to the slightly longer class name.
The second step, was to create two servlets. The first
LoginServlet handles the client "logging in" and subscribing to a channel. That servlet looks like this:
The inline comments describe most of the choices I made. I added a listener to the context so that we get an event in the case of a disconnected client, so that we can tidy up our model - the full code is in the ZIP at the end of this article. Importantly, this servlet does no async processing itself. It simply prepares the request and response objects for access at some time in the future. We stick them (via the async context) into a model which is in application scope. That model can then be used by the servlet which receives a request to publish something to a given channel:
Again, there are plenty of comments in the code. In this servlet, we actually do some (potentially) long running task. In many online examples of Servlet 3.0 Async Support, they show handing off work to an Executor. The async context provides the ideal way to do this via the container though, using its
start(Runnable) method. The container implementation then descides how to handle the task, rather than the developer having to worry about spawning threads, which on app servers like WebSphere is illegal and leads to errors. In true Java EE fashion, the developer can concentrate on business code, rather than technicalities.
Something else which might be important in the above code, is that the publishing is done on a different thread. Imagine publishing data to ten thousand clients. In order to finish within a second, each push it going to have to complete in less than a tenth of a millisecond. That means doing something useful like a database lookup is not going to be feasible. On a non-blocking server, we can't afford to take a second to do something, and many would argue a second is eternity in such an environment. The ability to hand off such work to a different thread is invaluable, and sadly, something node.js cannot currently do, although you can hand off the task to a different process, albeit potentially messier than that shown here.
The important parts here are that we need to listen for events on both ready state 3 and 4, rather than the usual 4. While the connection is kept open, the client only receives data in ready state 3, and everytime it receives a chunk, the
ajax.responseText attribute contains all chunks since login, rather than just the newest chunk. This could be bad, if the connection receives tons of data - eventually the browser will run out of memory! You could measure the number of bytes sent to any client on the server, and when its above a given threshold, force the client to disconnect by ending the stream (call the
complete() method on the async context of the relevant client just after publishing a message to it). The client above automatically logs itself back into the server when the server disconnects.
Instead of the reconnect solution for dropped / timed-out connections (which is very similar to long polling) we could add a heartbeat which the server sends to each subscribed client. The heartbeat period would need to be slightly less than the timeout. The exact details could get messy - do you consider sending a heartbeat every second, but only do it to clients who need it? Or do you send it to every client, say every 25 seconds, if the timeout is say 30 seconds? You could use performance tuning to determine if that is better than the reconnecting I have shown above. Then again, a heartbeat is good for culling closed connections, because it tests the connection every so often, and gets an exception if the push fails. And then again, the container informs the listener that we added to the login async context if a disconnect occurs too, so perhaps we don't need a heartbeat - you decide :-)
Now, we need a way to publish data - that's easy - I just type the following URL into the browser, and it sends a GET request to the publish servlet:
Keep refreshing the browser window with that protocol, and the other window almost instantly gets updates, showing the latest message at the bottom. I tested it using Firefox as the subscriber, and Chrome as the publisher.
I haven't checked out scalability, because I have assumed that Tomcat's NIO connector has been well tested and performs well. I'll let someone else play with the scalability of Servlet 3.0 for a push solution. This article is about showing how easy it is to implement Comet push, using Java EE Servlets. Note, it used to be easy too, because servers like Jetty and Tomcat and others provided bespoke Comet interfaces, but now, with the advent of Servlet 3.0, there is a standardised way to do it.
The complete code for this demo is downloadable here.
Copyright © 2011 Ant Kutschera