The Kitchen in the Zoo - tomcat tag

Tomcat, WebSockets, HTML5, jWebSockets, JSR-340, JSON and more

Tue, 21 Jun 2011 21:12:00 GMT

On my recent excursion into non-blocking servers I came across Comet, server push technologies and then Web Sockets. I was late arriving at the Comet party, but I think I have arrived at the Web Sockets party just in time. The final standard is still being evovled and at the time of writing, the only browser supporting it by default is Chrome. So to get started, I had a look at what was around. Initially, I wasn't going to write a blog article about it, I just wanted to learn about this new thing.

My only requirement was that the server implementation was Java based. The reason is simple - my entire technology stack is based on Java EE and I want to be able to integrate all my existing stuff into any new server, without the need for an integration bus. I like being able to drop a JAR file from one project into another one and to be up and running immediately.

So I had a quick look at jWebSockets, Grizzly (Glassfish), Jetty and Caucho's Resin. All similar, yet all somewhat different. Most different, was jWebSockets, because they have gone to the extent of building their own server, instead of basing their solution on an existing server. I had a good long rant about that kind of thing when I blogged about node.js, so I won't start again. But jWebSockets has a real issue to face in the coming years. JSR-340 talks about support for web technologies based around HTML5 and there is talk that this JSR will be part of Servlet 3.1 and be included in Java EE 7 at the end of next year. If that happens, jWebScokets will have the problem that their server implementation will become deprecated. And because their server doesn't offer anything else from Java EE, it is likely to lose a large part of it's market share. Don't get me wrong, they are still innovating on the client side, with a bridge to allow any browser to use websockets, and creating things like Java SE web socket clients. But server side isn't looking so hot - just take a look at the huge config file which you need to supply to get your plugins to work.

Anyway, I wasn't really enthusiastic about any of the solutions I saw. Some are servlet based (which is good), and all rely on you creating a new instance of a listener, handler, plugin or whatever they choose to call it. But what struck me was, by doing that, you lose the benefits of running inside the container. What I mean by this is that if you think about a servlet or an EJB (and I'm talking Java EE 6 and beyond here), the components you write have the benefit that the container can take care of cross cutting concerns like transactions, security and concurrency as well as provide you with injected resources, so that all you have left to do is write business code. So the reason I was not enthusiastic about any of the solutions listed above, was that what I felt to be glaringly obvious, has been missed. Sure, an inner class within a servlet could make use of resources from the servlet, but nothing I found on the net suggested that should be the normal way to program a web socket solution. And I'm not convinced that a reference in an inner class to a resource from the servlet couldn't go stale between messages received. I would rather the container always injected fresh resources just before my event handling method gets called, just as happens in the lifecycle of a servlet or EJB instance. Just think about a database connection timeout occurring between messages? If the container ensured the connection was always fresh, then the app developer wouldn't need to worry about technicalities.

An HttpServlet has methods for handling GETs, POSTs, et al. Why shouldn't a component handling a Web Socket not be a WebSocketServlet and have methods for handling the events for handshaking, opening, messaging, closing and error handling? I would benefit, because the servlet could have resources injected into it before the event handler methods get called, and the container could start any transactions that are required, or check security, etc.

So without realising that this would actually be quite hard to implement, I set upon taking Tomcat 7 apart, and building web sockets into it, in a way which fulfilled my requirements.

What resulted was (what I think is) a pretty sexy servlet solution. Please take a few minutes to study the code and especially the comments below, because it is the essence of this article.

/**
 * a demo of what a web socket servlet could look like
 */
@WebServlet(urlPatterns={"/TestServlet"})
@ServletSecurity(@HttpConstraint(rolesAllowed = "registered"))
public class TestServlet extends WebSocketsServlet<CommandContainer> {

  @WebServiceRef
  private PricingSystem pricingSystem;
  
  @EJB
  private SalesLog salesLog;
  
  @Override
  protected void doJSONMessage(
      WebSocketsServletRequest<CommandContainer> request,
      WebSocketsServletResponse response) throws IOException {
    
    //see how generics is helping me here, 
    //when i grab the request payload?
    //CommandContainer is a class defined in the app!
    CommandContainer c = request.getDataObject();
    
    if("CLOSE".equals(c.getCommand())){
    
      //closing connection to client
      response.close();

      //handle updates to say my model or whatever      
      actualClose(EventSubType.SESSION_END, request.getSession());
      
      return;

    }else if("PRICE".equals(c.getCommand())){

      //hey wow - a call to an injected web service!
      BigDecimal price = pricingSystem.getPrice(c.getData());
      c.setResult(price.toString());

    }else if("BUY".equals(c.getCommand())){
      
      //data is e.g. GOOG,100@200.25 - equally I could have 
      //put it in the model
      int idx = c.getData().indexOf(',');
      String shareCode = c.getData().substring(0, idx);
      idx = c.getData().indexOf('@');
      String numShares = c.getData().substring(shareCode.length()+1, idx);
      String price = c.getData().substring(idx+1, c.getData().length()-1);

      //awesome - a call to an injected EJB!
      salesLog.logPurchase(shareCode, 
                           new BigDecimal(price), 
                           Integer.parseInt(numShares));
      
      //and again cool - I can access the logged in user
      //in a familiar way (security)
      c.setResult("sold " + numShares + " of " + shareCode + 
                  " for " + request.getUserPrincipal().getName());

    }else{
      
      c.setResult("unknown command " + c.getCommand());
      
    }

    log("handled command " + c.getCommand() + " on session " + 
                 request.getSession().getId() + ": " + c.getResult());
  
    //the response takes care of the outputstream, framing, 
    //marshalling, etc - my life is really easy now!
    response.sendJSONResponse(c);
  }

  @Override
  protected void doError(EventSubType eventSubType, 
                         WebSocketsSession session, 
                         Exception e) {

    System.out.println("error: " + eventSubType + 
                       " on session " + session.getId() + 
                       ".  ex was: " + e);
  }

  @Override
  public void doClose(EventSubType eventSubType, 
                      WebSocketsSession session) {

    actualClose(eventSubType, session);
  }

  /** 
   * this is a method where we might do stuff like
   * tidy up our model, free resources, etc.
   */  
  private void actualClose(EventSubType eventSubType,
                           WebSocketsSession session) {
  
    System.out.println("closed session " + 
                       session.getId() + ": " + eventSubType);
  }

}

To me, this is what a serverside web socket component should look like. It is really similar to a sevlet or an EJB or a web service - things we are now familiar with. The learning curve is mini.

But getting Tomcat to use this thing was a little harder. First off, my approach was to simply write a new Connector and configure it in the server.xml. I gave it a port number and Tomcat willingly instantiated my new class:

  <!-- NIO WebSockets connector -->    
    <Connector port="8082" protocol="org.apache.coyote.http11.WebSocketsNioProtocol" 
               connectionTimeout="20000" 
             />

Of course, I cheated here a bit and stuck my class in a Coyote package. To get this to work, I unzipped the Tomcat 7.0.14 source and told Eclipse where to find it. I set up the build folder and added that to the classpath in the catalina.bat batch, before any of the other JARs on the classpath, and I did this just near the end of the batch where the call to the Java process is made. I then ran my modified Tomcat by simply starting the catalina batch, using remote debugging to make life easier.

I initially based my non-blocking connector on the non-blocking server which I have written and extended for my past few blog articles. My idea was intercept any web socket handshake requests that arrived and to quickly return a self built handshake response. I would then call the requested servlet and run it using Servlet 3.0 async support, so that the container would keep the connection open. The messages which the client then sent after the handshake would be handled by that servlet.

The idea didn't quite work though, because the servlet used the async support and my connector got callbacks from Tomcat in order for me to do stuff with threads, handle events and state changes. I was a little lost because I didn't understand the implementation details of Tomcat well enough, and after a while I gave up, because it was becoming clear that I would have to build quite a lot of stuff that already existed in the HTTP connectors which shipped with Tomcat. I came to realise that instead of reinventing the wheel, I would do better by creating my own versions (or even sub-classes) of the Coyote Http11NioProcessor and Http11NioProtocol which Tomcat uses to implement the HTTP connectors. While these are somewhat complicated, the advantages of specialising them are firstly that I am not reinventing the wheel, and secondly that I would get HTTPS (or WSS as it is with Web Sockets) for free (theoretically, I haven't yet tested it, but it is simply encapsulated so it should work with almost no problems).

While it was straight forward to copy the two relevant classes, the next headache was to get the browser to accept the handshake response. I wasn't sending the handshake response as a self built String like I had with my own non-blocking connector. Instead I was making as much use of servlet technology as I could, and building the response by setting the relevant headers on it and pushing the 16 byte response key onto the output stream of the response. Unlike a normal HTTP response, the browser expects any handshake responses to stick to a pretty strict specification. Things like a simple date header in the response (which Tomcat kindly sends by default) cause the browser to close the web socket. So with a bit of fiddling in the WebSocketsNioProcessor#prepareResponse() method, I soon had the response header looking perfect, and the browser started to play along.

But the next problem wasn't as simple to fix. The message handling didn't work, at all. Connections kept getting closed quickly and I couldn't figure out what was going on.

After a few headaches, I thought back to how Comet is implemented in Tomcat. I slowly realised, that what I needed was to simply piggy back the Comet infrastructure in Tomcat. In Tomcat, you create a Comet handler by simply implementing the CometProcessor interface in your servlet. The container checks during the initial request whether the servlet which will service the request implements the interface, and if it does, the request is marked as being a comet request. That means the container keeps the connection open and forwards future events (e.g. bytes / messages) to the relevant handler method which your servlet implements. I made my base servlet (WebSocketsServlet) implement the CometProcessor interface, and added the event(CometEvent event) method using the final keyword, so that application developers wouldn't ever think of overriding it, meaning they were protected from having the option of knowing about Comet. This base class servlet then encapsulated the web sockets events by hiding the fact that I had cheated by using Comet. The event method from the Comet processor simply passes the event on to the relevant doXXX method in my base servlet, which application programmers would subclass. This is pretty much what the HttpServlet base class does in it's service method.

So, my WebSocketsServlet is the Web Sockets equivalent of the HttpServlet, and as such, I put it into the javax.servlet.websockets package - i.e. to be supplied as part of the JSR which gets around to specifying the way in which Java EE containers should handle web sockets.

The WebSocketsServlet also deals with WebSocketServletRequest, WebSocketServletResponse and WebSocketSession objects, rather than HTTP equivalents. Each of these derives from the relevant ServletXXX rather than HttpServletXXX object, because when handling a web socket frame, it makes no sense what so ever to be able to do things like setting the headers on the response. Headers are only relevant to the handshake, and not anything which an application developer needs to concern themselves with. Again, these interfaces also sit in the javax.servlet.websockets package, because they are the analagous classes belonging to the specification, rather than just Tomcat. Sadly, the ServletRequest class has stuff in it which is too specific to HTTP, like getParameter(String), as well as other stuff which I would prefer to hide for web sockets, like getInputStream(). In order to make the WebSocketsServletRequest fit really well into the javax package, I think the ServletRequest might need to be broken into a super and a sub-class. Whether that would be possible, considering the billions of lines of Java code already out there, I don't know...

Things like wrapping the response String (bytes) in a web socket frame (i.e. leading 0x00 and trailing 0xFF byte) are also handled transparently by the WebSocketsServletResponse implementation. In fact, I went a step further. After listening to two interesting videos (with Jerome Dochez, the Architect of Glassfish - The Future of Java EE and Jerome discusses early plans for Java EE 7) I was inspired by some of the things he talked about. He spoke about not wanting to mess around with parsing strings and getting the container to handle XML and JSON. So... I added that to Tomcat too. You'll notice in the code near the top, the handler method which is called for handling message events is called doJSONMessage(WebSocketsServletRequest, WebSocketsServletResponse) rather than say simply doMessage. The reason is, the web sockets client (running in the browser) lets you send a header containing the protocol which it will use. This protocol appears to be a free text which the application can choose. In my implementation I went with XML, JSON, TEXT and BIN (binary), although XML and BIN aren't fully implemented. So when the client creates the web socket like this:

new WebSocket("ws://localhost:8085/nio-websockets/TestServlet", "JSON");

that JSON parameter is used by the container to decide which exact handler method in the servlet it will call. It gets better too, because instead of then having to manually handle the JSON string, the container does the magic, and from the WebSocketsServletRequest which is passed into the message handler method, I can retrieve the object, using the #getDataObject() method. And thanks to generics, it even passes that object back to me without me having to use a cast - the application servlet which implements the WebSocketsServlet base servlet specifies what type of data object it expects.

How does that magic work, I hear you asking? Well, its not that complicated. The base servlet receives a byte array from the Comet event. It looks at the original request header and realises it should treat that request as a JSON object. It then uses XStream which knows XML as well as JSON to unmarshal the incoming request. One last bit of magic that is required is that it needs to know which class say a "container" object in the JSON string maps too - i.e. which class should it use to stick the JSON data into? Well, XStream lets you define such aliases. I had a think about where else this happens, and JAX-Binding does this. So instead of the @XmlType annotation for JAXB, I create the javax.json.bind.annotation.JsonType annotation, which is applied to data objects, like this:

@JsonType(name="container")
public class CommandContainer {

  private String command;
  private String result;
  private String data;
.
.
.

Here, the "name" attribute of the annotation says that this class is mapped to JSON objects whose name is "container". So JSON like this:

var data = {"container":
               {"command": "BUY", 
                "data": "GOOG,100@200.25", 
                "result": "sold"}
           };

is simply unmarshalled into an instance of the CommandContainer class, so that the Java application can use the object immediately, rather than screwing around with strings and unmarshalling itself. The Tomcat container parses annotations during startup and I simply stuck some extra code in there to note down these mappings so that when the base servlet sets the raw data into the request, the request implementation (which is a Coyote object, rather than a javax object) can use something like XStream and these mappings to do the unmarshalling. I had to watch out for the class loader here to - luckily XStream lets you set the classloader, because if you don't set it, you are stuck with one that doesn't know the webapp. I simply took the classloader from the servlet context (which I got out of the HttpServletRequest which the container actually passes to the Comet processor in the Comet Event), and it has a classloader which knows the webapp classes. Regardless of whether such JSON annotated data classes are in the web-inf/classes folder, or in jars, the container can deal with them.

So what else is left? Well, security for starters. Chrome is very descent and because the Web Sockets path resides (in this case) under the same host as the one which serves the HTML containing the Javascript which opens the socket, it sends the JSESSIONID to the server during the Web Sockets handshake. That is really great, because so long as you had to log in to get the HTML, all web socket requests occur in the same session - so application developers have full access to the security framework supplied by Java EE! Hence I can mark my Servlet as requiring the user to be in a specific role in order to use it, as well as check their roles programmatically in my handler methods, using the request object. If the browser had not been so kind, my plan was to send the session ID to the server in the handshake as a request parameter in the query string of the requested URL, by putting the session ID into the URL using JSP.

On small problem with this security solution is that if the user isn't logged in, there is no way for the browser to handle the return code in the handshake properly. At least not in Chrome. In Chrome, the connection is simply closed, and the Javascript error console shows a slightly cryptic message indicating that a 403 code was returned. To avoid each browser implementing their own way to handle such problems, the IETF web sockets specification needs to clearly state that such problems should be sent as an event to the client's onerror callback method. Let's cross our fingers hey?

Resources, like EJBs, Entity Managers (JPA) and Web Services are also easily used - because in Java EE 6 I can inject them all using the container. While Tomcat doesn't support EJBs/WebServices per se, I still stuck some into the above example, to illustrate how such a servlet might look in a fully fledged Java EE app server. To make it work, I hacked Tomcat to look for those annotations and if nothing was found in the JNDI tree, it simply injected a new instance of the class - ok, its cheating, but great for a proof of concept ;-)

Note that while the servlet shown above might not be a typical application needing web sockets (because its a basic request/response app), it demonstrates the way I would like to see web sockets added to Java EE. It wouldn't be hard at all to write a servlet like this which handled a chat app, or for example an app where a user draws on their screen, and all other members see the drawing update on their screens in pretty much realtime. I might go on to play with something like that, but this article already discusses some of the issues facing Web Sockets.

A few weeks ago, I blogged about how node.js would need to provide the things like I have described in this article to become mature. Java EE has exactly the same challenges if it is to remain mature in the future, as new technologies are integrated into it.

Well, that's pretty much it. What do you think?

Patches for Tomcat 7.0.14 are available here.
The webapp containing the servlet shown above can be downloaded here.

Other useful links:

Social Bookmarks :

Transfer-Encoding: chunked

Wed, 24 Feb 2010 06:27:00 GMT

The J2ME HTTPConnection which comes with MIDP lets you make HTTP requests to your server. It doesn't do much at a high level, for example the API doesn't have methods like addCookie() - you need to manually add them with a request header. But the implementation is clever enough to turn any request body which is greater than around 2Kb into a chunked request.

With HTTP 1.0, the request had to contain a header called Content-Length which told the server how many bytes to read off the input stream when reading the body. HTTP 1.1 introduced the Transfer-Encoding header, which lets the client omit the Content-Length header, and instead create chunks of request body, which optimises the upload meaning that a) the server can start processing before it has everything, and b) more importantly for J2ME where memory might be a valuable resource, it lets the client send a bit of the request, free up that allocated memory and then send some more of the request.

For a POST request, with no chunking, the headers and body might look like this:

	POST /log.jsp HTTP/1.1
	User-Agent: Mozilla/4.0 (maxant J2ME Client) 
	Accept: text/html,application/xhtml+xml,application/xml
	Content-Type: application/x-www-form-urlencoded
	Content-Length: 51
	Host: wwwchaseamatecom:8089

	problem=Failed%20to%20get%20installation%20response

Chunked, that becomes:

	POST /ota/addInstallation.jsp HTTP/1.1
	User-Agent: Mozilla/4.0 (maxant J2ME Client) 
	Accept: text/html,application/xhtml+xml,application/xml
	Content-Type: application/x-www-form-urlencoded
	Host: wwwchaseamatecom:8089
	Transfer-Encoding: chunked

	problem=Failed%20to%20get%20installation%20response

You'll notice that the body of the second example, "problem=..." doesn't contain chunk headers (search Wikipedia for chunking to see an example). The reason is that I copied that text out of the TCP/IP Monitor in Eclipse 3.5 and it seems kind enough to hide that information and simply show you the unchunked body. Not great if you aren't expecting that, somewhat useful none the less.

Anyway, about time I got to the point of this blog article. Any version of Tomcat before 5.5.28 or 6.0.21 (ie. the latest versions at the time of writing!) had a nasty bug in them whereby POST requests did NOT get the body parsed and put into request.getParameter(String) or request.getQueryString(). Very nasty indeed for J2ME developers. I trawled the internet and found people were having problems from around 2002 when HTTP 1.1 became available in J2ME. Only with the following bug fix has this finally been corrected on Tomcat, some 8 years later.

https://issues.apache.org/bugzilla/show_bug.cgi?id=37794

Work arounds included using Apache to put together the chunks before sending the request on to Tomcat, or manually splitting the request on the device, into small chunks and upload each one and then joining them together at the end - what a load of work to have to do!

I'm working with an old 5.5.9 Tomcat installation and have just upgraded it to 5.5.28 to fix this problem.

However before I found this bug report I did consider creating a filter and configuring it in my web.xml. The idea was that the filter would read the request body, URL decode it, unlock the requests parameter map (Catalina has a lockable map which is locked after parsing the request), and put the request bodies request parameters into the map. Something like this:

/*  
 * Copyright (c) 2010 Ant Kutschera, maxant
 * 
 * This file is part of Ant Kutschera's blog, http://blog.maxant.co.uk
 * 
 * This is free software: you can redistribute it and/or modify
 * it under the terms of the Lesser GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 * 
 * It is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * Lesser GNU General Public License for more details.
 * You should have received a copy of the Lesser GNU General Public License
 * along with Foobar.  If not, see .
 */
import java.io.IOException;
import java.io.InputStream;
import java.net.URLDecoder;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.Map.Entry;

import javax.servlet.http.HttpServletRequest;

//nasty, but we are afterall fixing a bug in catalina!
import org.apache.catalina.util.ParameterMap;

public final class RequestHandler implements Filter {

	private static final String TRANSFER_ENCODING = "Transfer-Encoding";
	private static final String CHUNKED = "chunked";

	public void destroy() {
	}

	public void init(FilterConfig filterConfig) throws ServletException {
	}
	
	public void doFilter(ServletRequest request, ServletResponse response,
			FilterChain chain) throws IOException, ServletException {

		String header = request.getHeader(TRANSFER_ENCODING);
		
		InputStream is = null;
		try{
			if(header != null && header.equals(CHUNKED)){
				//time to work our magic

				((ParameterMap)request.getParameterMap()).setLocked(false);
				is = request.getInputStream();

				//read the body as a map
				Properties props = new Properties();
				props.load(is);
				
				//get the encoding to use for url-decoding
				String enc = request.getCharacterEncoding();
				if(enc == null){
					enc = "UTF-8";
				}

				for(Entry e : props.entrySet()){
					
					String key = (String) e.getKey();

					//need to decode it!
					key = URLDecoder.decode(key, enc);

					if(!request.getParameterMap().containsKey(key)){
						String val = (String) e.getValue();

						//need to decode it!
						val = URLDecoder.decode(val, enc);

						request.getParameterMap().put(key, val);
					}
				}
			}				
				
			((ParameterMap)request.getParameterMap()).setLocked(true);
				
		}finally{
			if(is != null) is.close();
		}
	}
}

I wasted hours because of this rubbish bug. The problem is that its full of red herrings, making you think for example that because the monitor shows the request body unchunked, that the device isn't chunking the body properly. The web is also full of very old posts related to servers not being HTTP 1.1 compatible. And there are lots of work arounds which people went to the effort of because they had no other choice until most recently. So hopefully if you have had chunking problems with J2ME, you found this article without too much hassle!

Copyright (c) 2010 Ant Kutschera

Social Bookmarks :