The Kitchen in the Zoo - design tag

100% code coverage, Hibernate Validator and Design by Contract

Mon, 28 Nov 2011 07:29:37 GMT

Code coverage in unit testing was one of the first things I learned in my software engineering career. The company I worked at taught me that you should have 100% coverage as a goal, but achieving it does not mean there are no bugs in the system. At that time, I worked at a company whose big thing was that they delivered very reliable billing software to telecomms operators. We used to invest as much time writing unit tests as we did writing the code. If you included unit tests, integration tests, system tests and acceptance tests, more time was spent testing than designing and implementing the code which ran in production. It was impressive and I have never worked with or for a company with that kind of model since, although I am sure there are many companies which operate like that.

The other day I was thinking back to those days and reading up about unit testing to refresh myself in preparation for a unit testing course I'm attending soon (don't want the instructor to know more than me!) and I started to wonder about what kind of code could be fully covered during unit testing, but which could still contain a bug. While I learned that 100% coverage should be the goal, in over 10 years I have never worked on a project which achieved that. So I have never surprised to find a bug in code which I thought was "fully" tested.

It's fun write whacky code - you don't normally try and write bugs :-) I started with the specification:

    /**      * @return String "a" or "b" depending upon the values      * passed to the init method. If variable "a" is true,      * then string "a" is returned. If variable "b" is true,      * then "b" is returned. If neither is true, then "" is      * returned. Variable "b" is important than "a", so if      * both are true then return "b".      */
Now, the tricky part was to write code which contained a bug yet was easy to cover with incomplete tests. I came up with the following. The specification above is for the "getResult()" method.

public class SomeClass {     private boolean a;     private boolean b;     public SomeClass init(boolean a, boolean b) {         this.a = a;         this.b = b;         return this;     }     public String getResult() {         String s = "";         if(a) {             s = "a";         }         if(b){             s += "b"; // <-- oops!         }         return s;     } }
The "bug" is in the method "getResult" and the problem is that instead of just an assignment, the "plus equals" operator has been used. An "else" would probably make the code a little safer too. But I think code like this is quite typical of the buggy code that finds its way into productive systems. Even the best programmers have lapses of concentration where they write typos like this (especially in open plan offices!).

The unit test which a programmer trying to achieve 100% coverage, would look something like this:

    @Test     public void test() {         assertEquals("a", new SomeClass().init(true, false).getResult());         assertEquals("b", new SomeClass().init(false, true).getResult());     }
Using Emma for Eclipse I measured complete coverage. But wait! There is still a bug. The code does not do what it says in the Javadoc spec. If I initialise the object with "true, true", then the result will be "ab", because of the "plus equals" operator. So even though I have 100% coverage, I still have a bug.

I asked myself what that means to me as a programmer. What do I need to look out for, when writing tests. Imagine the code above was tucked away among hundereds of other lines of code, then the chances of seeing it are really quite small. The test wouldn't be just two lines long and the problem wouldn't be jumping out of the page.

One way to look at the problem is to say that there is a bug because the code isn't fulfilling its contract. OK, I use "contract" in the loose sense of the word, but the Javadoc is basically a contract. It tells anyone calling that method what to expect, yet the codes is not doing what the user expects.

So perhaps one solution is to not only entirely exercise the code, but entirely exercise the contract? Is there a way to translate the Javadoc contract into something more concrete which the testing tools can help me check? Yes, namely my using some kind of Design (or Program) by Contract (DBC or PBC) framework. JSR-303 isn't strictly DBC, but close. It lets you use annotations to state your expectations about parameters passed to methods, as well as your expectation about the result being returned. You can create your own complex constraints quite easily. I added the following annotations to my code to help in my quest for bug free code:

    @Valid     @NotNull     @Size(min=0, max=1)     public String getResult() {         ...
Now, method validation (validating method calls, rather than validating the bean itself) is something which comes as an extra in Hibernate Validator, and which really isn't part of JSR-303 - it's only described in Appendix C as optional. To test this, I used Google Guice to add an AOP interceptor to any methods marked with the @Valid annotation, and in that interceptor, I call the Hibernate Method Validator. I end up with something like this:

. . .     Injector injector = Guice.createInjector(new AbstractModule(){         protected void configure() {             bindInterceptor(Matchers.any(),                     Matchers.annotatedWith(Valid.class),                     new MethodValidatorInterceptor());         }     });     SomeClass someClass = injector.getInstance(SomeClass.class);     someClass.init(true, false);     assertEquals("a", someClass.getResult());     someClass = injector.getInstance(SomeClass.class);     someClass.init(false, true);     assertEquals("b", someClass.getResult()); . . . public class MethodValidatorInterceptor<T> implements MethodInterceptor {     public Object invoke(MethodInvocation invocation) throws Throwable {         //validate all inputs         Set<MethodConstraintViolation<T>> mcvs =             methodValidator.validateAllParameters((T)invocation.getThis(),                invocation.getMethod(), invocation.getArguments());         if(mcvs.size() != 0){             throw new IllegalArgumentException(String.valueOf(mcvs));         }         //call through to the actual method being called         Object ret = invocation.proceed();         //validate result         mcvs = methodValidator.validateReturnValue((T)invocation.getThis(),             invocation.getMethod(), ret);         if(mcvs.size() != 0){             throw new IllegalArgumentException(String.valueOf(mcvs));         }         return ret;     } } . . .
The above is something which a (Java EE) container should do for me - I was just messing around with simple classes in a simple Java Project. Now, it isn't quite complete, because I still have 100% coverage, and I still have that bug, because the new annotations haven't really done anything useful. Well that isn't entirely true - the reader of the code knows that the contract is a little stricter than it was when it was simple Javadoc. The reader may assume that the system will check these constraints when the method is called. But while there is still a bug, I've laid the path for adding some full pre- or post-conditions. The next step was to add a new annotation, an interface and make use of them in the interceptor.

@Target( { ElementType.METHOD }) @Retention(RetentionPolicy.RUNTIME) @Documented public @interface PreCondition {     Class<? extends ConditionChecker> implementation(); } /** Any pre- or post-condition is written in   * an implementation of this interface   */ public interface ConditionChecker {     void checkCondition()         throws ConstraintViolationException; }
The annotation can be added to the business code, in this case to add a pre-condition. I created a similar annotation for post-conditions. When I add the pre-condition to a method, I also state which class contains the code for checking that pre-condition:

    @Valid     @NotNull     @Size(min=0, max=1)     @PreCondition(implementation=MyPreAndPostCondition.class)     public String getResult() {         ...
The interceptor can check for the presence of such a pre-condition annotation before calling through to the method being called. If the annotation is found, the interceptor attempts to create an instance of the implementation class, and calls it's "checkCondition" method.

So the final part of this puzzle is to write a pre-condition which will help me to fail to get 100% coverage when I test with the test shown near the top of this post. Here it is, implemented as a static final inner class inside the SomeClass class, so that it has access to the fields "a" and "b":

public class MyPreAndPostCondition implements ConditionChecker {     @Override     public void checkCondition()             throws ConstraintViolationException {         //im going to list all combinations of         //a and b which I support         if(!a && b) return;         if(!b && a) return;         if(!b && !a) return;         if(a && b) return;     } }
When I now test my code, I no longer get 100% coverage, because the last two lines in the pre-condition are not covered. Hooray! The tools can now tell me what I still need to test...

Now, while this technical solution works, I think it is really ugly. Like I said, if the code which is causing the problem were to be found within one or two hundred other lines of code, and were reliant on local variables rather than fields in my class, I would have no chance of using a pre-condition like this to help me locate the problem.

So to sum up, DBC isn't going to help me solve the problem that 100% code coverage can still contain errors. I think DBC frameworks (and there are a lot out there, some which do exactly what I have done here using the @PreCondition annotation) help to make contracts more concrete. If you use method validation from Hibernate Validator, you don't have to write as much Javadoc, because the reader knows that the container will give up before calling the method if anything fails to validate. To me as a programmer, that is much more satisfying than praying that some Javadoc reflects what I have really implemented.

I have known programmers who don't write tests because a DBC framework is in place and that makes them feel safe. But just because you declare a contract, does not mean the code actually works. Whether the code fails hard with a validation exception or at some time later because your code is buggy, makes no difference - both are inacceptable! From that perspective, DBC contracts are simply hints to the tester what could be useful tests, and they ensure that the code fails hard, early.

While I was refreshing my testing skills, I also learned the difference between mocks and stubs. For years I had always thought they were the same thing, but it turns out that stubs return pre-fed answers, whereby mocks check the sequence of calls to them too. On a different thread at DZone, someone made a point that unit testing was pointless because it never helped them find bugs, and it caused lots of work when refactoring, because all that does is break their tests. I'd say that this is simply a question of black box versus white box testing. Black box unit testing should never break if you refactor your code - the tests are simply clients to the code which you are refactoring, and tools like Eclipse will modify the calling code if you modify the interfaces being called, including tests. You can get pretty good testing results by just using black box tests - the large majority of the tests I write are black box tests and when I write such tests, I tend to have bug free code.

I've talked about writing contracts to help the reader determine what they should expect when they use your code. Unit tests themselves work similarly, because they show the reader examples of how to call your code and what to expect when your code changes system state. They hint to the reader how the writer intended the code to be used. While I don't advocate TDD (perhaps only because I have never been on a project which used it or at a company which valued quality enough to warrant TDD), I do encourage writing tests and using validation and pre-/post-conditions because they help document the code with the added bonus of finding the occassional bug. At the same time, I am an architect and we need to keep things like budgets in mind. You have an influence on your budget when you estimate, assuming you are allowed to do that. Your customer might push you to reduce your estimates, and that is an indication about their commitment to quality, because they won't budge on scope or delivery dates! So write as many tests as you can, within your budget and start with the code which is most complex and which gets called most often. and remember, 100% coverage is not really your best friend because bugs may still lurk.

© 2011, Ant Kutschera

Social Bookmarks :

A really simple but powerful rule engine

Sat, 12 Nov 2011 20:26:00 GMT

I have the requirement to use a rule engine. I want something light weight and fairly simple, yet powerful. While there are products out there which are super good, I don't want something with the big learning overhead. And, I fancied writing my own!

Here are my few basic requirements:

Use some kind of expression language to write the rules,
It should be possible to store rules in a database,
Rules need a priority, so that only the best can be fired,
It should also be possible to fire all matching rules,
Rules should evaluate against one input which can be an object like a tree, containing all the information which rules need to evaluate against
Predefined actions which are programmed in the system should be executed when certains rules fire.

So to help clarify those requirements, imagine the following examples:

1) In some forum system, the administrator needs to be able to configure when emails are sent.

Here, I would write some rules like "when the config flag called sendUserEmail is set to true, send an email to the user" and "when the config flag called sendAdministratorEmail is true and the user has posted less than 5 postings, send an email to the administrator".

2) A tarif system needs to be configurable, so that the best tarif can be offered to customers.

For that, I could write rules like these: "when the person is younger than 26 the Youth tarif is applicable", "when the person is older than 59, the Senior tarif is applicable", and "when the person is neither a youth, nor a senior, then they should be given the Default tarif, unless they have had an account for more than 24 months, in which case they should be offered the Loyalty tarif".

3) A train ticket can be considered to be a product. Depending upon the travel request, different products are suitable.

A rule here, could be something like: "if the travel distance is more than 100km and first class is desired, then product A is to be sold."

Finally, a more complex example, involving some iteration of the input, rather than just property evaluation:

4) Timetable software needs to deterimine when students can leave school.

A rule for that might read: "If a class contains any student under age 10, the entire class gets to leave early. Otherwise, they leave at the normal time."

So, with those requirements in mind, I went to look for an expression language. I started with the unified expression language specified in JSP 2.1. Using the jasper jar used in Tomcat and Apache Commons EL jar, I got something up and running very quickly. Then I discovered the MVEL library from Codehaus.org, which incidentally is used in Drools (the leading Java rule engine?) and it worked even better. It offers far more functionality as far as I can tell.

So, I designed my rule engine to work like this:

1) An engine is configured with some rules.

2) A rule has these attributes:
    - namespace: an engine may contain many rules, but only some may be relevant to a particular call and this namespace can be used for filtering
    - name: a unique name within a namespace
    - expression: an MVEL expression for the rule
    - outcome: a string which the engine might use if this rules expression evaluates to true
    - priority: an integer. The bigger the value, the higher the priority.
    - description: a useful description to aid the management of rules.

3) The engine is given an input object and evaluates all rules (optionally within a namespace) and either:
    a) returns all rules which evaluate to true,
    b) returns the outcome (string) from the rule with the highest priority, out of all rules evaluating to true,
    c) execute an action (defined within the application) which is associated with the outcome of the rule with the highest priority, out of all rules evaluating to true.

4) "Action"s are instances of classes which the application programmer can supply. An action is given a name. When the engine is asked to execute an action based on the rules, the name of the action matching the "winning" rule's outcome is executed.

5) A rule can be built up of "sub-rules". A subrule is only ever used as a building block on which to base more complex rules. When evaluating rules, the engine will never select a subrule to be the best (highest priority) "winning" rule, i.e. one evaluating to true. Subrules make it easier to build complex rules, as I shall show shortly.

So, time for some code!

First, let's look at the code for the tarif system:

Rule r1 = new Rule("YouthTarif", "input.person.age < 26", "YT2011", 3, "ch.maxant.someapp.tarifs", null);
Rule r2 = new Rule("SeniorTarif", "input.person.age > 59", "ST2011", 3, "ch.maxant.someapp.tarifs", null);
Rule r3 = new Rule("DefaultTarif", "!#YouthTarif && !#SeniorTarif", "DT2011", 3, "ch.maxant.someapp.tarifs", null);
Rule r4 = new Rule("LoyaltyTarif", "#DefaultTarif && input.account.ageInMonths > 24", "LT2011", 4, "ch.maxant.someapp.tarifs", null);
List<Rule> rules = Arrays.asList(r1, r2, r3, r4);

Engine engine = new Engine(rules, true);

TarifRequest request = new TarifRequest();
request.setPerson(new Person("p"));
request.setAccount(new Account());

request.getPerson().setAge(24);
request.getAccount().setAgeInMonths(5);
String tarif = engine.getBestOutcome(request);

So, in the above code, I have added 4 rules to the engine, and told the engine to throw an exception if any rule cannot be pre-compiled. Then, I created a TarifRequest, which is the input object. That object is passed into the engine, when I ask the engine to give me the best outcome. In this case, the best outcome is the string "YT2011", the name of the most suitable tarif for the customer I added to the tarif request.

How does it all work? When the engine is given the rules, it does some validation on them, and pre-compiles the rules (to improve overall performance). Notice how the first two rules refer to an object called "input"? That is the object passed into the "getBestOutcome" method on the engine. The engine passes the input object to the MVEL class together with each rules expression. Anytime an expression evaluates to "true", the rule is put to the side as a candidate to be the winner. At the end, the candidates are sorted in order of priority, and the outcome field of the rule with the highest priority is returned by the engine.

Notice how the third and fourth rules contain the '#' character. That is not standard MVEL expression language. The engine examines all rules when they are passed to it, and it replaces any token starting with a hash symbol, with the expression found in the rule named the same as the token. It wraps the expression in brackets. The logger outputs the full rule after reference rules have been resolved and replaced, just in case you want to check the rule.

In the above business case, we were only interested in the best tarif for the customer. Equally, we might have been interested in a list of possible tarifs, so that we could offer the customer a choice. In that case, we could have called the "getMatchingRules" method on the engine, which would have returned all rules, sorted by priority. The tarif names are (in this case) the "outcome" field of the rules.

In the above example, I wanted to receive any of the four outcomes, from the four rules. Sometimes however, you might want to build complex rules based on building blocks, but you might never want those building blocks to be a winning outcome. The train trip example from above can be used to show what I mean here:

Rule rule1 = new SubRule("longdistance", "input.distance > 100", "ch.maxant.produkte", null);
Rule rule2 = new SubRule("firstclass", "input.map[\"travelClass\"] == 1", "ch.maxant.produkte", null);
Rule rule3 = new Rule("productA", "#longdistance && #firstclass", "productA", 3, "ch.maxant.produkte", null);
List<Rule> rules = Arrays.asList(rule1, rule2, rule3);

Engine e = new Engine(rules, true);

TravelRequest request = new TravelRequest(150);
request.put("travelClass", 1);
List rs = e.getMatchingRules(request);

In the above code, I build rule3 from two subrules. But I never want the outcomes of those building blocks to be output from the engine. So I create them as SubRules. SubRules don't have an outcome field or priority. They are simply used to build up more complex rules. After the engine has used the sub-rules to replace all tokens beginning in a hash during initialisation, it discards the SubRules - they are not evaluated.

The TravelRequest above takes a distance in the constructor, and contains a map of additional parameters. MVEL let's you easily access map values using the syntax shown in rule 2.

Next, consider the business case of wanting to configure an forum system. The code below introduces actions. Actions are created by the application programmer and supplied to the engine. The engine takes the outcomes (as described in the first example), and searches for actions with the same names as those outcomes, and calls the "execute" method on those actions (they all implement the IAction interface). This functionality is useful when a system must be capable of predefined things, but the choice of what to do needs to be highly configurable and independent of deployment.

Rule r1 = new Rule("SendEmailToUser", "input.config.sendUserEmail == true", "SendEmailToUser", 1, "ch.maxant.someapp.config", null);
Rule r2 = new Rule("SendEmailToModerator", "input.config.sendAdministratorEmail == true and input.user.numberOfPostings < 5", "SendEmailToModerator", 2, "ch.maxant.someapp.config", null);
List<Rule> rules = Arrays.asList(r1, r2);
		
final List<String> log = new ArrayList<String>();
		
Action<ForumSetup, Void> a1 = new Action<ForumSetup, Void>("SendEmailToUser") {
  @Override
  public Void execute(ForumSetup input) {
    log.add("Sending email to user!");
    return null;
  }
};
Action<ForumSetup, Void> a2 = new Action<ForumSetup, Void>("SendEmailToModerator") {
  @Override
  public Void execute(ForumSetup input) {
    log.add("Sending email to moderator!");
    return null;
  }
};

Engine engine = new Engine(rules, true);

ForumSetup setup = new ForumSetup();
setup.getConfig().setSendUserEmail(true);
setup.getConfig().setSendAdministratorEmail(true);
setup.getUser().setNumberOfPostings(2);
			
engine.executeAllActions(setup, Arrays.asList(a1, a2));

In the code above, the actions are passed to the engine when we call the "executeAllActions" method. In this case, both actions are executed, because the setup object causes both rules to evaluate to true. Note that the actions are executed in the order of highest priority rule first. Each action is only ever executed once - it's name is noted after execution and it will not be executed again, until the engines "execute*Action*" method is called again. Also, if you only want the action associated with the best outcome to be executed, call the "executeBestAction" method instead of "executeAllActions".

Finally, let's consider the classroom example.

String expression = 
    "for(student : input.students){" +
    "	if(student.age < 10) return true;" +
    "}" +
    "return false;";

Rule r1 = new Rule("containsStudentUnder10", expression , "leaveEarly", 1, "ch.maxant.rules", "If a class contains a student under 10 years of age, then the class may go home early");
		
Rule r2 = new Rule("default", "true" , "leaveOnTime", 0, "ch.maxant.rules", "this is the default");
		
Classroom classroom = new Classroom();
classroom.getStudents().add(new Person(12));
classroom.getStudents().add(new Person(10));
classroom.getStudents().add(new Person(8));

Engine e = new Engine(Arrays.asList(r1, r2), true);
		
String outcome = e.getBestOutcome(classroom);

The outcome above is "leaveEarly", because the classroom contains one student whose age is less than 10. MVEL let's you write some pretty comprehensive expressions, and is really a programming language in it's own right. The engine simply requires a rule to return true, if the rule is to be considered a candidate for firing.

There are more examples in the JUnit tests contained in the source code.

So, the requirements are fulfiled, except for "It should be possible to store rules in a database". While this library doesn't support reading and writing rules to / from a database, rules are String based. So it wouldn't be hard to create some JDBC or JPA code which reads rules out of a database and populates Rule objects and passes them to the Engine. I haven't added this to the library, because normally these things as well as the management of rules is something quite project specific. And because my library will never be as cool or popular as Drools, I'm not sure it would be worth my while to add such functionality.

I've put the rule engine into an OSGi library with the LGPL licence and it can be downloaded from my tools site. This library depends on MVEL, which can be downloaded here (I used version 2.0.19). If you like it, let me know!

© 2011, Ant Kutschera

Social Bookmarks :

Tomcat, WebSockets, HTML5, jWebSockets, JSR-340, JSON and more

Tue, 21 Jun 2011 21:12:00 GMT

On my recent excursion into non-blocking servers I came across Comet, server push technologies and then Web Sockets. I was late arriving at the Comet party, but I think I have arrived at the Web Sockets party just in time. The final standard is still being evovled and at the time of writing, the only browser supporting it by default is Chrome. So to get started, I had a look at what was around. Initially, I wasn't going to write a blog article about it, I just wanted to learn about this new thing.

My only requirement was that the server implementation was Java based. The reason is simple - my entire technology stack is based on Java EE and I want to be able to integrate all my existing stuff into any new server, without the need for an integration bus. I like being able to drop a JAR file from one project into another one and to be up and running immediately.

So I had a quick look at jWebSockets, Grizzly (Glassfish), Jetty and Caucho's Resin. All similar, yet all somewhat different. Most different, was jWebSockets, because they have gone to the extent of building their own server, instead of basing their solution on an existing server. I had a good long rant about that kind of thing when I blogged about node.js, so I won't start again. But jWebSockets has a real issue to face in the coming years. JSR-340 talks about support for web technologies based around HTML5 and there is talk that this JSR will be part of Servlet 3.1 and be included in Java EE 7 at the end of next year. If that happens, jWebScokets will have the problem that their server implementation will become deprecated. And because their server doesn't offer anything else from Java EE, it is likely to lose a large part of it's market share. Don't get me wrong, they are still innovating on the client side, with a bridge to allow any browser to use websockets, and creating things like Java SE web socket clients. But server side isn't looking so hot - just take a look at the huge config file which you need to supply to get your plugins to work.

Anyway, I wasn't really enthusiastic about any of the solutions I saw. Some are servlet based (which is good), and all rely on you creating a new instance of a listener, handler, plugin or whatever they choose to call it. But what struck me was, by doing that, you lose the benefits of running inside the container. What I mean by this is that if you think about a servlet or an EJB (and I'm talking Java EE 6 and beyond here), the components you write have the benefit that the container can take care of cross cutting concerns like transactions, security and concurrency as well as provide you with injected resources, so that all you have left to do is write business code. So the reason I was not enthusiastic about any of the solutions listed above, was that what I felt to be glaringly obvious, has been missed. Sure, an inner class within a servlet could make use of resources from the servlet, but nothing I found on the net suggested that should be the normal way to program a web socket solution. And I'm not convinced that a reference in an inner class to a resource from the servlet couldn't go stale between messages received. I would rather the container always injected fresh resources just before my event handling method gets called, just as happens in the lifecycle of a servlet or EJB instance. Just think about a database connection timeout occurring between messages? If the container ensured the connection was always fresh, then the app developer wouldn't need to worry about technicalities.

An HttpServlet has methods for handling GETs, POSTs, et al. Why shouldn't a component handling a Web Socket not be a WebSocketServlet and have methods for handling the events for handshaking, opening, messaging, closing and error handling? I would benefit, because the servlet could have resources injected into it before the event handler methods get called, and the container could start any transactions that are required, or check security, etc.

So without realising that this would actually be quite hard to implement, I set upon taking Tomcat 7 apart, and building web sockets into it, in a way which fulfilled my requirements.

What resulted was (what I think is) a pretty sexy servlet solution. Please take a few minutes to study the code and especially the comments below, because it is the essence of this article.

/**
 * a demo of what a web socket servlet could look like
 */
@WebServlet(urlPatterns={"/TestServlet"})
@ServletSecurity(@HttpConstraint(rolesAllowed = "registered"))
public class TestServlet extends WebSocketsServlet<CommandContainer> {

  @WebServiceRef
  private PricingSystem pricingSystem;
  
  @EJB
  private SalesLog salesLog;
  
  @Override
  protected void doJSONMessage(
      WebSocketsServletRequest<CommandContainer> request,
      WebSocketsServletResponse response) throws IOException {
    
    //see how generics is helping me here, 
    //when i grab the request payload?
    //CommandContainer is a class defined in the app!
    CommandContainer c = request.getDataObject();
    
    if("CLOSE".equals(c.getCommand())){
    
      //closing connection to client
      response.close();

      //handle updates to say my model or whatever      
      actualClose(EventSubType.SESSION_END, request.getSession());
      
      return;

    }else if("PRICE".equals(c.getCommand())){

      //hey wow - a call to an injected web service!
      BigDecimal price = pricingSystem.getPrice(c.getData());
      c.setResult(price.toString());

    }else if("BUY".equals(c.getCommand())){
      
      //data is e.g. GOOG,100@200.25 - equally I could have 
      //put it in the model
      int idx = c.getData().indexOf(',');
      String shareCode = c.getData().substring(0, idx);
      idx = c.getData().indexOf('@');
      String numShares = c.getData().substring(shareCode.length()+1, idx);
      String price = c.getData().substring(idx+1, c.getData().length()-1);

      //awesome - a call to an injected EJB!
      salesLog.logPurchase(shareCode, 
                           new BigDecimal(price), 
                           Integer.parseInt(numShares));
      
      //and again cool - I can access the logged in user
      //in a familiar way (security)
      c.setResult("sold " + numShares + " of " + shareCode + 
                  " for " + request.getUserPrincipal().getName());

    }else{
      
      c.setResult("unknown command " + c.getCommand());
      
    }

    log("handled command " + c.getCommand() + " on session " + 
                 request.getSession().getId() + ": " + c.getResult());
  
    //the response takes care of the outputstream, framing, 
    //marshalling, etc - my life is really easy now!
    response.sendJSONResponse(c);
  }

  @Override
  protected void doError(EventSubType eventSubType, 
                         WebSocketsSession session, 
                         Exception e) {

    System.out.println("error: " + eventSubType + 
                       " on session " + session.getId() + 
                       ".  ex was: " + e);
  }

  @Override
  public void doClose(EventSubType eventSubType, 
                      WebSocketsSession session) {

    actualClose(eventSubType, session);
  }

  /** 
   * this is a method where we might do stuff like
   * tidy up our model, free resources, etc.
   */  
  private void actualClose(EventSubType eventSubType,
                           WebSocketsSession session) {
  
    System.out.println("closed session " + 
                       session.getId() + ": " + eventSubType);
  }

}

To me, this is what a serverside web socket component should look like. It is really similar to a sevlet or an EJB or a web service - things we are now familiar with. The learning curve is mini.

But getting Tomcat to use this thing was a little harder. First off, my approach was to simply write a new Connector and configure it in the server.xml. I gave it a port number and Tomcat willingly instantiated my new class:

  <!-- NIO WebSockets connector -->    
    <Connector port="8082" protocol="org.apache.coyote.http11.WebSocketsNioProtocol" 
               connectionTimeout="20000" 
             />

Of course, I cheated here a bit and stuck my class in a Coyote package. To get this to work, I unzipped the Tomcat 7.0.14 source and told Eclipse where to find it. I set up the build folder and added that to the classpath in the catalina.bat batch, before any of the other JARs on the classpath, and I did this just near the end of the batch where the call to the Java process is made. I then ran my modified Tomcat by simply starting the catalina batch, using remote debugging to make life easier.

I initially based my non-blocking connector on the non-blocking server which I have written and extended for my past few blog articles. My idea was intercept any web socket handshake requests that arrived and to quickly return a self built handshake response. I would then call the requested servlet and run it using Servlet 3.0 async support, so that the container would keep the connection open. The messages which the client then sent after the handshake would be handled by that servlet.

The idea didn't quite work though, because the servlet used the async support and my connector got callbacks from Tomcat in order for me to do stuff with threads, handle events and state changes. I was a little lost because I didn't understand the implementation details of Tomcat well enough, and after a while I gave up, because it was becoming clear that I would have to build quite a lot of stuff that already existed in the HTTP connectors which shipped with Tomcat. I came to realise that instead of reinventing the wheel, I would do better by creating my own versions (or even sub-classes) of the Coyote Http11NioProcessor and Http11NioProtocol which Tomcat uses to implement the HTTP connectors. While these are somewhat complicated, the advantages of specialising them are firstly that I am not reinventing the wheel, and secondly that I would get HTTPS (or WSS as it is with Web Sockets) for free (theoretically, I haven't yet tested it, but it is simply encapsulated so it should work with almost no problems).

While it was straight forward to copy the two relevant classes, the next headache was to get the browser to accept the handshake response. I wasn't sending the handshake response as a self built String like I had with my own non-blocking connector. Instead I was making as much use of servlet technology as I could, and building the response by setting the relevant headers on it and pushing the 16 byte response key onto the output stream of the response. Unlike a normal HTTP response, the browser expects any handshake responses to stick to a pretty strict specification. Things like a simple date header in the response (which Tomcat kindly sends by default) cause the browser to close the web socket. So with a bit of fiddling in the WebSocketsNioProcessor#prepareResponse() method, I soon had the response header looking perfect, and the browser started to play along.

But the next problem wasn't as simple to fix. The message handling didn't work, at all. Connections kept getting closed quickly and I couldn't figure out what was going on.

After a few headaches, I thought back to how Comet is implemented in Tomcat. I slowly realised, that what I needed was to simply piggy back the Comet infrastructure in Tomcat. In Tomcat, you create a Comet handler by simply implementing the CometProcessor interface in your servlet. The container checks during the initial request whether the servlet which will service the request implements the interface, and if it does, the request is marked as being a comet request. That means the container keeps the connection open and forwards future events (e.g. bytes / messages) to the relevant handler method which your servlet implements. I made my base servlet (WebSocketsServlet) implement the CometProcessor interface, and added the event(CometEvent event) method using the final keyword, so that application developers wouldn't ever think of overriding it, meaning they were protected from having the option of knowing about Comet. This base class servlet then encapsulated the web sockets events by hiding the fact that I had cheated by using Comet. The event method from the Comet processor simply passes the event on to the relevant doXXX method in my base servlet, which application programmers would subclass. This is pretty much what the HttpServlet base class does in it's service method.

So, my WebSocketsServlet is the Web Sockets equivalent of the HttpServlet, and as such, I put it into the javax.servlet.websockets package - i.e. to be supplied as part of the JSR which gets around to specifying the way in which Java EE containers should handle web sockets.

The WebSocketsServlet also deals with WebSocketServletRequest, WebSocketServletResponse and WebSocketSession objects, rather than HTTP equivalents. Each of these derives from the relevant ServletXXX rather than HttpServletXXX object, because when handling a web socket frame, it makes no sense what so ever to be able to do things like setting the headers on the response. Headers are only relevant to the handshake, and not anything which an application developer needs to concern themselves with. Again, these interfaces also sit in the javax.servlet.websockets package, because they are the analagous classes belonging to the specification, rather than just Tomcat. Sadly, the ServletRequest class has stuff in it which is too specific to HTTP, like getParameter(String), as well as other stuff which I would prefer to hide for web sockets, like getInputStream(). In order to make the WebSocketsServletRequest fit really well into the javax package, I think the ServletRequest might need to be broken into a super and a sub-class. Whether that would be possible, considering the billions of lines of Java code already out there, I don't know...

Things like wrapping the response String (bytes) in a web socket frame (i.e. leading 0x00 and trailing 0xFF byte) are also handled transparently by the WebSocketsServletResponse implementation. In fact, I went a step further. After listening to two interesting videos (with Jerome Dochez, the Architect of Glassfish - The Future of Java EE and Jerome discusses early plans for Java EE 7) I was inspired by some of the things he talked about. He spoke about not wanting to mess around with parsing strings and getting the container to handle XML and JSON. So... I added that to Tomcat too. You'll notice in the code near the top, the handler method which is called for handling message events is called doJSONMessage(WebSocketsServletRequest, WebSocketsServletResponse) rather than say simply doMessage. The reason is, the web sockets client (running in the browser) lets you send a header containing the protocol which it will use. This protocol appears to be a free text which the application can choose. In my implementation I went with XML, JSON, TEXT and BIN (binary), although XML and BIN aren't fully implemented. So when the client creates the web socket like this:

new WebSocket("ws://localhost:8085/nio-websockets/TestServlet", "JSON");

that JSON parameter is used by the container to decide which exact handler method in the servlet it will call. It gets better too, because instead of then having to manually handle the JSON string, the container does the magic, and from the WebSocketsServletRequest which is passed into the message handler method, I can retrieve the object, using the #getDataObject() method. And thanks to generics, it even passes that object back to me without me having to use a cast - the application servlet which implements the WebSocketsServlet base servlet specifies what type of data object it expects.

How does that magic work, I hear you asking? Well, its not that complicated. The base servlet receives a byte array from the Comet event. It looks at the original request header and realises it should treat that request as a JSON object. It then uses XStream which knows XML as well as JSON to unmarshal the incoming request. One last bit of magic that is required is that it needs to know which class say a "container" object in the JSON string maps too - i.e. which class should it use to stick the JSON data into? Well, XStream lets you define such aliases. I had a think about where else this happens, and JAX-Binding does this. So instead of the @XmlType annotation for JAXB, I create the javax.json.bind.annotation.JsonType annotation, which is applied to data objects, like this:

@JsonType(name="container")
public class CommandContainer {

  private String command;
  private String result;
  private String data;
.
.
.

Here, the "name" attribute of the annotation says that this class is mapped to JSON objects whose name is "container". So JSON like this:

var data = {"container":
               {"command": "BUY", 
                "data": "GOOG,100@200.25", 
                "result": "sold"}
           };

is simply unmarshalled into an instance of the CommandContainer class, so that the Java application can use the object immediately, rather than screwing around with strings and unmarshalling itself. The Tomcat container parses annotations during startup and I simply stuck some extra code in there to note down these mappings so that when the base servlet sets the raw data into the request, the request implementation (which is a Coyote object, rather than a javax object) can use something like XStream and these mappings to do the unmarshalling. I had to watch out for the class loader here to - luckily XStream lets you set the classloader, because if you don't set it, you are stuck with one that doesn't know the webapp. I simply took the classloader from the servlet context (which I got out of the HttpServletRequest which the container actually passes to the Comet processor in the Comet Event), and it has a classloader which knows the webapp classes. Regardless of whether such JSON annotated data classes are in the web-inf/classes folder, or in jars, the container can deal with them.

So what else is left? Well, security for starters. Chrome is very descent and because the Web Sockets path resides (in this case) under the same host as the one which serves the HTML containing the Javascript which opens the socket, it sends the JSESSIONID to the server during the Web Sockets handshake. That is really great, because so long as you had to log in to get the HTML, all web socket requests occur in the same session - so application developers have full access to the security framework supplied by Java EE! Hence I can mark my Servlet as requiring the user to be in a specific role in order to use it, as well as check their roles programmatically in my handler methods, using the request object. If the browser had not been so kind, my plan was to send the session ID to the server in the handshake as a request parameter in the query string of the requested URL, by putting the session ID into the URL using JSP.

On small problem with this security solution is that if the user isn't logged in, there is no way for the browser to handle the return code in the handshake properly. At least not in Chrome. In Chrome, the connection is simply closed, and the Javascript error console shows a slightly cryptic message indicating that a 403 code was returned. To avoid each browser implementing their own way to handle such problems, the IETF web sockets specification needs to clearly state that such problems should be sent as an event to the client's onerror callback method. Let's cross our fingers hey?

Resources, like EJBs, Entity Managers (JPA) and Web Services are also easily used - because in Java EE 6 I can inject them all using the container. While Tomcat doesn't support EJBs/WebServices per se, I still stuck some into the above example, to illustrate how such a servlet might look in a fully fledged Java EE app server. To make it work, I hacked Tomcat to look for those annotations and if nothing was found in the JNDI tree, it simply injected a new instance of the class - ok, its cheating, but great for a proof of concept ;-)

Note that while the servlet shown above might not be a typical application needing web sockets (because its a basic request/response app), it demonstrates the way I would like to see web sockets added to Java EE. It wouldn't be hard at all to write a servlet like this which handled a chat app, or for example an app where a user draws on their screen, and all other members see the drawing update on their screens in pretty much realtime. I might go on to play with something like that, but this article already discusses some of the issues facing Web Sockets.

A few weeks ago, I blogged about how node.js would need to provide the things like I have described in this article to become mature. Java EE has exactly the same challenges if it is to remain mature in the future, as new technologies are integrated into it.

Well, that's pretty much it. What do you think?

Patches for Tomcat 7.0.14 are available here.
The webapp containing the servlet shown above can be downloaded here.

Other useful links:

Social Bookmarks :

Non-blocking (NIO) Server Push and Servlet 3

Sun, 05 Jun 2011 18:40:00 GMT

In my previous blog posting, I wrote about what I would expect node.js to do in order to become mature. I introduced the idea of having a framework which lets you define a protocol and some handlers in order to let the developer concentrate on writing useful business software, rather than technical code, in a very similar manner to which Java EE does. And through that posting, I came to learn about a thing called Comet. I had stated that using a non-blocking server wouldn't really be any more useful than a blocking one in a typical web application (HTTP), and so I created an example based on my own protocol and a VOIP server, for streaming binary data to many concurrently connected clients.

I have now read up on Comet and come to realise there is indeed a good case for having a non-blocking server in the web. That case is pushing data back to the client, like for continuously publishing latest stock prices. While this example could be solved using polling, true Comet uses long-polling or even better, full on push. A great introduction I read was here. The idea is that the client makes a call to the server and instead of the server returning data immediately, it keeps the connection open and returns data at some time in the future, potentially many times. This is not a new idea - the term Comet seems to have been invented in about 2006 and the article I refer to above is from 2009. I think I've arrived at this party very late :-)

My new found case of server push for non-blocking HTTP, and a strong curiosity drove me to knuckle down and start coding. Before long, I had a rough implementation of the HTTP protocol for my little framework, and I was able to write an app for my server using handlers that subclassed the HttpHandler, which was for all intents and purposes, a Servlet.

To get Comet push to work properly, you get the client to "log in" and register itself with the server. In my demo, I didn't check authorisations against a database, like you might do for a real app, but I had the concept of a channel, to which any browser client could subscribe. During this login, the client says which channel it wants to subscribe to, and the server adds the clients non-blocking connection to its model. The server responds using chunked transfer encoding, because that way, the connection stays open, and you don't need to state up front how much data you will send back. At some time in the future, when someone publishes something, the server can use the connection which is still open contained in its model, to push that published data back to the subscribed client, by sending another chunk of data.

The server implementation wasn't too hard, but the client posed a few problems, until I realised that the data was arriving in the ajax client with a ready state of 3, rather than the more usual 4. The ajax client's onreadystatechange callback function was also given every byte of data in it's responseText, rather than just the new stuff, so I had some fiddling around until I could get the browser to just append the new stuff to the innerHTML attribute of a div on my page. Anyway, after just a few hours, I had an app that worked quite well. But it wasn't entirely satisfactory, partly because as I stated in the previous posting, the server is still a little buggy, especially when the client drops a connection, because for example the browser page is closed. I had also ended up implementing the HTTP protocol for my framework, which seemed to be reinventing the wheel - servlet technology does all this stuff already, and much better than I can hope to do it. One of the reasons that I don't like node.js is that everything is being reinvented.

So, like the article which I referenced above indicated, version 3.0 of Servlets should be able to handle Comet. I downloaded Tomcat 7.0, which has a Servlet 3.0 container, and I ported my app code to proper Servlets. It took a while to work out exactly how to use the new asynchronous parts of servlets because there aren't all that many accurate tutorials out there. The servlet specs (JSR 315) helped a lot. Once I cracked how to use the async stuff properly, I had a really really satisfying solution to my push requirements.

The first step, was to reconfigure Tomcat so that it uses non-blocking (NIO) for its connector protocol. The point of this is that I want to keep a connection open to the client in order to push data to it. I can't rely on the one-thread-per-request paradigm, because context switching and thread memory requirements are likely to be a performance killer. In Tomcat's server.xml file, I configured the connector's protocol:

	<!-- NIO HTTP/1.1 connector -->    
    <Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol" 
               connectionTimeout="20000" 
               redirectPort="8443" />

rather than the normal:

    <Connector port="8080" protocol="HTTP/1.1" 
               connectionTimeout="20000" 
               redirectPort="8443" />

All you need to do to get Tomcat to turn into an NIO server is change the protocol attribute to the slightly longer class name.

The second step, was to create two servlets. The first LoginServlet handles the client "logging in" and subscribing to a channel. That servlet looks like this:

/*  
 * Copyright (c) 2011 Ant Kutschera
 * 
 * This file is part of Ant Kutschera's blog, 
 * http://blog.maxant.co.uk
 * 
 * This is free software: you can redistribute
 * it and/or modify it under the terms of the
 * Lesser GNU General Public License as published by
 * the Free Software Foundation, either version 3 of
 * the License, or (at your option) any later version.
 * 
 * It is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied
 * warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE.  See the Lesser GNU General Public License for
 * more details. 
 * 
 * You should have received a copy of the
 * Lesser GNU General Public License along with this software.
 * If not, see http://www.gnu.org/licenses/.
 */
package ch.maxant.blog.nio.servlet3;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;

import javax.servlet.AsyncContext;
import javax.servlet.ServletContext;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import ch.maxant.blog.nio.servlet3.model.Subscriber;

@WebServlet(name = "loginServlet", urlPatterns = { "/login" }, asyncSupported = true)
public class LoginServlet extends HttpServlet {

	public static final String CLIENTS = "ch.maxant.blog.nio.servlet3.clients";

	private static final long serialVersionUID = 1L;

	public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException {

		// dont set the content length in the response, and we will end up with chunked 
		// encoding so that a) we can keep the connection open to the client, and b) send
		// updates to the client as chunks.
		
		// *********************
		// we use asyncSupported=true on the annotation for two reasons. first of all, 
		// it means the connection to the client isn't closed by the container.  second 
		// it means that we can pass the asyncContext to another thread (eg the publisher) 
		// which can then send data back to that open connection.
		// so that we dont require a thread per client, we also use NIO, configured in the 
		// connector of our app server (eg tomcat)
		// *********************

		// what channel does the user want to subscribe to?  
		// for production we would need to check authorisations here!
		String channel = request.getParameter("channel");

		// ok, get an async context which we can pass to another thread
		final AsyncContext aCtx = request.startAsync(request, response);

		// a little longer than default, to give us time to test.
		// TODO if we use a heartbeat, then time that to pulse at a similar rate
		aCtx.setTimeout(20000L); 

		// create a data object for this new subscription
		Subscriber subscriber = new Subscriber(aCtx, channel);

		// get the application scope so that we can add our data to the model
		ServletContext appScope = request.getServletContext();

		// fetch the model from the app scope
		@SuppressWarnings("unchecked")
		Map

The inline comments describe most of the choices I made. I added a listener to the context so that we get an event in the case of a disconnected client, so that we can tidy up our model - the full code is in the ZIP at the end of this article. Importantly, this servlet does no async processing itself. It simply prepares the request and response objects for access at some time in the future. We stick them (via the async context) into a model which is in application scope. That model can then be used by the servlet which receives a request to publish something to a given channel:

/*  
 * Copyright (c) 2011 Ant Kutschera
 * 
 * This file is part of Ant Kutschera's blog, 
 * http://blog.maxant.co.uk
 * 
 * This is free software: you can redistribute
 * it and/or modify it under the terms of the
 * Lesser GNU General Public License as published by
 * the Free Software Foundation, either version 3 of
 * the License, or (at your option) any later version.
 * 
 * It is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied
 * warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE.  See the Lesser GNU General Public License for
 * more details. 
 * 
 * You should have received a copy of the
 * Lesser GNU General Public License along with this software.
 * If not, see http://www.gnu.org/licenses/.
 */
package ch.maxant.blog.nio.servlet3;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Map;

import javax.servlet.AsyncContext;
import javax.servlet.ServletContext;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import ch.maxant.blog.nio.servlet3.model.Subscriber;

@WebServlet(name = "publishServlet", urlPatterns = { "/publish" }, asyncSupported = true)
public class PublishServlet extends HttpServlet {

	private static final long serialVersionUID = 1L;

	public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException {

		// *************************
		// this servlet simply spawns a thread to send its message to all subscribers.
		// this servlet keeps the connection to its client open long enough to tell it 
		// that it has published to all subscribers.
		// *************************
		
		// add a pipe character, so that the client knows from where the newest model has started.
		// if messages are published really quick, its possible that the client gets two at
		// once, and we dont want it to be confused!  these messages also arrive at the 
		// ajax client in readyState 3, where the responseText contains everything since login,
		// rather than just the latest chunk.  so, the client needs a way to work out the 
		// latest part of the message, containing the newest version of the model it should 
		// work with.  might be better to return XML or JSON here!
		final String msg = "|" + request.getParameter("message") + " " + new Date();

		// to which channel should it publish?  in prod, we would check authorisations here too!
		final String channel = request.getParameter("channel");

		// get the application scoped model, and copy the list of subscribers, so that the 
		// long running task of publishing doesnt interfere with new logins
		ServletContext appScope = request.getServletContext();
		@SuppressWarnings("unchecked")
		final Map

Again, there are plenty of comments in the code. In this servlet, we actually do some (potentially) long running task. In many online examples of Servlet 3.0 Async Support, they show handing off work to an Executor. The async context provides the ideal way to do this via the container though, using its start(Runnable) method. The container implementation then descides how to handle the task, rather than the developer having to worry about spawning threads, which on app servers like WebSphere is illegal and leads to errors. In true Java EE fashion, the developer can concentrate on business code, rather than technicalities.

Something else which might be important in the above code, is that the publishing is done on a different thread. Imagine publishing data to ten thousand clients. In order to finish within a second, each push it going to have to complete in less than a tenth of a millisecond. That means doing something useful like a database lookup is not going to be feasible. On a non-blocking server, we can't afford to take a second to do something, and many would argue a second is eternity in such an environment. The ability to hand off such work to a different thread is invaluable, and sadly, something node.js cannot currently do, although you can hand off the task to a different process, albeit potentially messier than that shown here.

Now, we just need to create a client to subscribe to the server. This client is an ajax request object which is created when the HTML is loaded which runs some JavaScript in a library I have written. The HTML looks like this:

<html>
<head>
	<script language="Javascript" type="text/javascript" src="push_client.js"></script>
</head>
<body>
<p>Boo!</p>
<div id="myDiv"></div>
</body>
<script language="Javascript" type="text/javascript">

function callback(model){
	//simply append the model to a div, for demo purposes
	var myDiv = document.getElementById("myDiv");
	myDiv.innerHTML = myDiv.innerHTML + "<br>" + model;
}

new PushClient("myChannel", callback).login();

</script>
</html>

As you can see, it simply needs to define a callback function which will handle each message published from the server. The published message could be text, XML or JSON - the publisher chooses. The JavaScript in that library is a little more complicated, but basically creates an XHR requester which sends a request to the login servlet. Any data it receives from the server, it parses, and returns the newest part of the data back to the callback.

/*  
 * Copyright (c) 2011 Ant Kutschera
 * 
 * This file is part of Ant Kutschera's blog, 
 * http://blog.maxant.co.uk
 * 
 * This is free software: you can redistribute
 * it and/or modify it under the terms of the
 * Lesser GNU General Public License as published by
 * the Free Software Foundation, either version 3 of
 * the License, or (at your option) any later version.
 * 
 * It is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied
 * warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE.  See the Lesser GNU General Public License for
 * more details. 
 * 
 * You should have received a copy of the
 * Lesser GNU General Public License along with this software.
 * If not, see http://www.gnu.org/licenses/.
 */

function PushClient(ch, m){

	this.channel = ch;
	this.ajax = getAjaxClient();
	this.onMessage = m;

	// stick a reference to "this" into the ajax client, so that the handleMessage 
	// function can access the push client - its "this" is an XMLHttpRequest object
	// rather than the push client, coz thats how javascript works!
	this.ajax.pushClient = this;
	
	function getAjaxClient(){
		/*
		 * Gets the ajax client
		 * http://en.wikipedia.org/wiki/XMLHttpRequest
		 * http://www.w3.org/TR/XMLHttpRequest/#responsetext
		 */
	    var client = null;
	    try{
			// Firefox, Opera 8.0+, Safari
			client = new XMLHttpRequest();
		}catch (e){
			// Internet Explorer
			try{
				client = new ActiveXObject("Msxml2.XMLHTTP");
			}catch (e){
				client = new ActiveXObject("Microsoft.XMLHTTP");
			}
		}
		return client;
	};
	
	/** 
	 * pass in a callback and a channel.  
	 * the callback should take a string, 
	 * which is the latest version of the model 
	 */
	PushClient.prototype.login = function(){

		try{
			var params = escape("channel") + "=" + escape(this.channel);
			var url = "login?" + params;
			this.ajax.onreadystatechange = handleMessage;
			this.ajax.open("GET",url,true); //true means async, which is the safest way to do it
			
			// hint to the browser and server, that we are doing something long running
			// initial tests only seemed to work with this - dont know, perhaps now it 
			// works without it?
			this.ajax.setRequestHeader("Connection", "Keep-Alive");
			this.ajax.setRequestHeader("Keep-Alive", "timeout=999, max=99");
			this.ajax.setRequestHeader("Transfer-Encoding", "chunked");
			
			//send the GET request to the server
			this.ajax.send(null);
		}catch(e){
			alert(e);
		}
	};

	function handleMessage() {
		//states are:
		//	0 (Uninitialized)	The object has been created, but not initialized (the open method has not been called).
		//	1 (Open)	The object has been created, but the send method has not been called.
		//	2 (Sent)	The send method has been called. responseText is not available. responseBody is not available.
		//	3 (Receiving)	Some data has been received. responseText is not available. responseBody is not available.
		//	4 (Loaded)
		try{
			if(this.readyState == 0){
				//this.pushClient.onMessage("0/-/-");
			}else if (this.readyState == 1){
				//this.pushClient.onMessage("1/-/-");
			}else if (this.readyState == 2){
				//this.pushClient.onMessage("2/-/-");
			}else if (this.readyState == 3){
				//for chunked encoding, we get the newest version of the entire response here, 
				//rather than in readyState 4, which is more usual.
				if (this.status == 200){
					this.pushClient.onMessage("3/200/" + this.responseText.substring(this.responseText.lastIndexOf("|")));
				}else{
					this.pushClient.onMessage("3/" + this.status + "/-");
				}
			}else if (this.readyState == 4){
				if (this.status == 200){
					
					//the connection is now closed.
					
					this.pushClient.onMessage("4/200/" + this.responseText.substring(this.responseText.lastIndexOf("|")));

					//start again - we were just disconnected!
					this.pushClient.login();

				}else{
					this.pushClient.onMessage("4/" + this.status + "/-");
				}
			}
		}catch(e){
			alert(e);
		}
	};
}

The important parts here are that we need to listen for events on both ready state 3 and 4, rather than the usual 4. While the connection is kept open, the client only receives data in ready state 3, and everytime it receives a chunk, the ajax.responseText attribute contains all chunks since login, rather than just the newest chunk. This could be bad, if the connection receives tons of data - eventually the browser will run out of memory! You could measure the number of bytes sent to any client on the server, and when its above a given threshold, force the client to disconnect by ending the stream (call the complete() method on the async context of the relevant client just after publishing a message to it). The client above automatically logs itself back into the server when the server disconnects.

Instead of the reconnect solution for dropped / timed-out connections (which is very similar to long polling) we could add a heartbeat which the server sends to each subscribed client. The heartbeat period would need to be slightly less than the timeout. The exact details could get messy - do you consider sending a heartbeat every second, but only do it to clients who need it? Or do you send it to every client, say every 25 seconds, if the timeout is say 30 seconds? You could use performance tuning to determine if that is better than the reconnecting I have shown above. Then again, a heartbeat is good for culling closed connections, because it tests the connection every so often, and gets an exception if the push fails. And then again, the container informs the listener that we added to the login async context if a disconnect occurs too, so perhaps we don't need a heartbeat - you decide :-)

Now, we need a way to publish data - that's easy - I just type the following URL into the browser, and it sends a GET request to the publish servlet:

http://localhost:8080/nio-servlet3/publish?channel=myChannel&message=javaIsAwesome

Keep refreshing the browser window with that protocol, and the other window almost instantly gets updates, showing the latest message at the bottom. I tested it using Firefox as the subscriber, and Chrome as the publisher.

I haven't checked out scalability, because I have assumed that Tomcat's NIO connector has been well tested and performs well. I'll let someone else play with the scalability of Servlet 3.0 for a push solution. This article is about showing how easy it is to implement Comet push, using Java EE Servlets. Note, it used to be easy too, because servers like Jetty and Tomcat and others provided bespoke Comet interfaces, but now, with the advent of Servlet 3.0, there is a standardised way to do it.

There are plenty of profesional and open source solutions which do what I have done in this article. See this article, which lists many, whackiest of all, APE - another project which puts JavaScript on the server!? Well, better than PHP I guess :-)

The complete code for this demo is downloadable here.

Copyright © 2011 Ant Kutschera

Social Bookmarks :

Java non-blocking servers, and what I expect node.js to do if it is to become mature

Sun, 22 May 2011 19:36:09 GMT

node.js is getting a lot of attention at the moment. It's goal is to provide an easy way to build scalable network programs, e.g. build web servers. It's different, in two ways. First of all, it brings Javacript to the server. But more importantly, it's event based, rather than thread based, so relies on the OS to send it events when say a connection to the server is made, so that it can handle that request.

The argument goes, that a typical web server "handles each request using a thread, which is relatively inefficient and very difficult to use." As an example they state that "Node will show much better memory efficiency under high loads than systems which allocate 2MB thread stacks for each connection."

They go on to state that "users of Node are free from worries of dead-locking the process — there are no locks. Almost no function in Node directly performs I/O, so the process never blocks. Because nothing blocks, less-than-expert programmers are able to develop fast systems".

Well, reading those statements makes me think that I have problems in my world which need addressing! But then I look at my world, and realise that I don't have problems. How can that be? Well first of all, because we don't have a thread per request, rather we use a thread pool, reducing the memory overhead, and queuing incoming requests if we can't run them in a thread instantly, until a thread becomes free in the pool.

The second reason I don't have threading problems is because I work with Java EE. I write business software which is written as components which get deployed to an app server. That app server is what does all the hard work with threads. All I have to do is follow some simple and common sense rules to avoid threading issues, like ensuring that shared resources like a Map(dictionary) are instantiated using the thread safe API which Java offers. In more extreme cases, I use the synchronized keyword to protect methods or objects being shared.

All this means that we Java programmers have become very efficient at writing software to solve business problems, rather than writing software to solve technical problems like concurrency, scalability, security, transactions, resources, object and resource pooling, etc., etc.

So when do I need to solve those problems which lots of threads introduce, namely memory problems? Any server which needs to maintain an open connection to the client, in order to say stream video or Voice over IP (VOIP), or handle instant messaging, is a server where I'm going to have problems with memory if I have a thread per connection.

Now, the Java EE specifications tend to focus on multithreaded servers. And to my knowledge, there is no Java EE specification which tells vendors how to make a server which lets people deploy software components to it, which are handled in a non-blocking manner. Sure, you could write a Servlet compliant server like Tomcat with a non-blocking engine under the hood, but Java EE doesn't talk about streaming. And probably 99% of websites out there don't do the kind of streaming which is going to cause memory issues.

So there certainly are times when something like node.js will be useful. Or at least the idea of a non-blocking server. But looking into the details, node.js isn't something that I would seriously use if I needed such a server. The main problems are that:

you deploy Javascript to the server
it doesn't seem to specify a standardised way of writing components so that I can concentrate on writing business software rather than solving technical issues
while it is rumoured to be fast, studies such as this suggest it's half as slow as Java
compared to Java, the node.js API and Javascript libraries available today are immature - think about things like sending email, ORM, etc.
node.js has political problems because it relies on Google. If Google don't want to support Javascript on the server (their V8 engine is designed for Chrome, client side), and node.js needs a patch or development on V8 to handle a serverside issue, they might never get it. OK, like I have a chance to get a Java bug patched by Oracle ;-)

It started for me, with this code snippet, taken from the node.js site:

    var net = require('net');

    var server = net.createServer(function (socket) {
      socket.write("Echo server\r\n");
      socket.pipe(socket);
    });

    server.listen(1337, "127.0.0.1");

Well, if the point of node.js is to make it really easy to create a server, and pass it a function for handling requests, I can do that in Java too:

TCPProtocol protocol = new TCPProtocol(){

    public void handleRequest(ServerRequest request, 
                              ServerResponse response) {

        //echo what we just received
        try {
            response.write(request.getData());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
};

Configuration config = new Configuration(1337, protocol);
ServerFactory.createServer(config).listen();

OK, it's a tiny bit more complicated. But the reason is, I decided to write a non-blocking server which can be configured to handle any kind of request. The configuration takes a port number for the server to run on, and a protocol object. The protocol object is a subclass of an abstract protocol, and its job is to look at the data coming in from the wire and decide how to handle it. A TCPProtocol as shown above is simplest protocol, in that it does nothing. You need to tell it what to do, by overriding the handleRequest() method (analagous to supplying a function in Javascript), for example as shown above, by returning to the client what it sent to the server.

But I wanted to do something more useful, so I extended what I wrote to handle voice over IP, or at least a client streaming an MP3 to a different client, to simulate a phone call between the two of them.

The first step was to design the protocol. At the byte level, the first byte in a packet sent to the server contains the command, or action, the second byte contains the length of the payload, if any, and the subsequent bytes contain the payload.

The protocol then lets a caller login (returning a session ID), start a call (returning an OK/NOK), send data for the call, end a call, and quit (logout).

After a few hours I had my laptop playing my favourite MP3s, streamed to my laptop from a different PC. But it looked clunky. I had a think, and took inspiration from the Servlet and EJB specifications in Java EE. Namely, I wanted to have deployable components whose sole job was to handle a business requirement.

In .NET and Java you can define a web service or web page (Servlet) by annotating the class. In the annotation, you provide a URL path, which the server uses to determine when to call your servlet or web service. This is similar to a "command" or "action" as I had in my VOIP protocol.

So I wrote a little container, which sits inside the non-blocking server. The server takes the incoming request and once it has handled the low level non-blocking stuff, it passes the request over to the configured protocol. The VOIP protocol then extracts the command (first byte) from the incoming request. It then uses a "context" object, which it gets injected into it from the container, to send the incoming request to a Handler. The handler is a class which has an annotation to state which command it can handle. The handler is then analagous to a web service or servlet. It is a piece of busienss code which handles an incoming request, based on the command (path) which the client is requesting.

Put together, it looks as follows. The first bit is the protocol's handler method which determines the command and sends it to a handler:

    public void handleRequest(ServerRequest request,
	                          ServerResponse response) {

        String criterion = String.valueOf(request.getData()[0]);
        try{
            //call through to the context for help - it
            //will call the handler for us.
            getContext().handleRequest(criterion, request, response);
        } catch (UnknownHandlerException e) {
            //send NOK to client because of unknown command
            response.write(new Packet(UNKNOWN_COMMAND).marshal());
        }
    }

A Packet object in the above code is simply an encapsulation of the data sent over the wire, but knows that the first byte is the command, the second is the length and subsequent bytes are the payload. In an HTTP request, this Packet object would be something which knew about the HTTP header's attributes and the HTTP payload, rather than the ServerRequest and ServerResponse objects which only know about byte arrays and sockets/channels.

The second part of the solution are then the handlers. These are contained in the configuration, either created programatically as shown in the TCP example above, or created using XML which the server reads upon startup. The container then locates, instantiates and calls these handlers on behalf of the protocol, when the protocol object calls getContext().handleRequest(criterion, request, response) in the above snippet. A typical handler looks like this:

@Handler(selector=VoipProtocol.LOGIN)
public class VoipLoginHandler extends VoipHandler {

    public void service(VoipRequest request,
                        VoipResponse response)
                        throws IOException {
		
        String name = request.getPacket().getPayloadAsString();
        Participant p = new Participant(
                        name, response.getSocketChannel());
        getProtocol().getModel().addParticipant(p);
        request.getKey().attach(p);
		
        //ACK
        String sessId = p.getSessId().getBytes(VoipProtocol.CHARSET);
        Packet packet = new Packet(sessId, VoipProtocol.LOGIN)
        response.write(packet);
	}

}

So, the service method is the only one in a handler, and is in charge of doing business stuff. There is no techy stuff here. The code works out who is logging in, creates an instance of them (the Participant object) in the model. The model is contained in the protocol object, which exists just once in the server. Compared to a servlet, the call to get the model is analagous to putting data into the application scope. The code then attaches the participant object to the client connection using the attach(Object) method (see Java NIO Package) so that we can always get straight to the relevant part of the data model from the connection object, when subsequent requests arrive. Finally, the code responds to the client with the session ID as an aknowledgement. Note that here, I haven't bothered to authenticate the password - the payload only contains the username. But if I were writing a complete app, I would have more data in my payload, perhaps even something like XML or JSON, and I would authenticate the username and password.

The @Handler annotation at the top of the handler class is analagous to the @WebServlet annotation applied to Java EE Servlets. It has a "selector" attribute which the container uses to compare to the criterion which the protocol extracts from the request. This is analagous to the URL patterns attribute in @WebServlet which tells a Java EE web container to which path the servlet should be mapped. There is a little bit of hidden magic too - the service method in the handler already knows VoipRequest and VoipResponse, rather than ServerRequest and ServerResponse. The superclass does this magic, by implementing the standard service method and calling the specialised abstract service method, implemented in subclasses.

So, being creative, I added a further attribute to the @Handler annotation. It is called runAsync and defaults to false. But, if it is set to true, then the container sends the handler to a thread pool for execution sometime in the future. I don't actually use this in the VOIP example, but I did this, to show that it is perfectly feasible, that an app server can do such things. The developer doesn't need to worry about threads or anything - they simple configure the annotation and the container handles the hard parts. This is typical of Java EE! And it becomes extremely useful in cases where a request needs a little more time to execute. In a non-blocking single threaded process, while connections are concurrently connected to the server, they are serviced sequentially, meaning that they MUST return fast, if you don't want those in the queue to wait too long. This is something which node.js CANNOT do, because it has no way of starting threads. Their solution is to send an async request to a different process. But to do that, the developer is spending time working on technical issues, rather than getting on and trusing the container to do it for them, so that they can spend more time writing cost effective business code. One of the main reasons to use Java EE is that the developer can spend more time writing business software and less time handling techie problems. There is no reason why a handler couldn't also have other annotations:

@Handler(selector=VoipProtocol.QUIT)
@RolesAllowed({"someRole", "someOtherRole"})
@TransactionManagement(TransactionManagementType.CONTAINER)
public class VoipQuitHandler extends VoipHandler {

    @PersistenceContext(unitName="persistenceUnitName")
    private EntityManager em;
    .
    .
    .
}

The @RolesAllowed@TransactionManagement annotation means that transactions will be handled by the container rather than by the programmer. When the handler is executed, the @PersistenceContext annotation causes the container to inject a JPA entity manager so that the business code has access to a database. This is exactly the same way that a Servlet or EJB gets resources from the container. Those resources are created based on the app server configuration, in a standardised way and are managed by the container too (pooling, reconnecting, etc.), again, relieving the programmer from that burden.

So what do we have now? Instead of a low level API like node.js provides, we have a high level container for running software components within a non-blocking server. What we don't have, is Javascript running on the server, because it seemed like the ideal language for handling callbacks in a non-blocking environment, because it has an event queue and function pointers.

Could this little exercise be the basis of a JSR? Well, is it really useful? Only really in cases where you have clients which need to keep connections open to the server for a long time, and you have thousands of clients. There won't be many people needing to do this, and there are more important JSRs awaiting acceptance and implementation. But who knows what the future will bring.

To summarise, node.js makes me uneasy, because I don't like the idea of Javascript and its immature stack of libraries being deployed on a production server. Whenever I develop with Javascript I spend a lot more time with the debugger than I would like to, because the language is not entirely checkable using static analysis because it is duck typed. But the idea of building a non-blocking server using Java - now that's interesting (to me at least). But like I said, I'm just not sure how often will it be useful.

It seems to me, that the revolution that node.js is starting isn't really about Javascript on the server. It is more about using a non-blocking server. But the problem is, probably 99% of our needs are already satisfied with standard multithreaded servers. And we can already write scalable websites, without all those problems which node.js claims we have. So let's not rebuild the world based on non-blocking I/O, just because node.js has arrived. Let's build/rebuild just those very special cases, where non-blocking I/O will really help us.

Does node.js deserve the hype it is getting? I don't think that it deserves hype because you can run Javascript on the server - that's a bad thing. Douglas Crockford (senior JavaScript architect at Yahoo!) even hints this, when he says:

"The big surprise for me in this is we're about to take maybe the most important step we've ever taken in terms of the technology of the web, and JavaScript is leading the way."

He seems to be saying that Javascript is leading the way in the most important step we are ever taking, and not that Javascript IS the most important step we are taking. That is, non-blocking servers are the most important step, and having an event loop on the server is the way forward. The way I understand it, he is saying that non-blocking is the revolution. The problem I have in joining in on this revolution is that non-blocking servers are not ncessarily the best thing ever. They only really help, when you have thousands of clients needing to keep their connections to the server open. HTTP (99% of the web) doesn't need non-blocking I/O to become the technology leader of the web - it already is. So instead of joining in this revolution and all the hype that node.js is stirring up, I will ignore it and continue building software the way I have been.

The code for the examples given above are in two Eclipse projects, which you can download here. The first project is the framework itself, including the non-blocking server, container and relevant classes and interfaces. The second project is an example of how to use the framework to build (business) apps. It contains the TCPServerRunner class for running a TCP echo server. It also contains the VoipServerRunner which starts the VOIP Server. To run the streaming example, first run the ListeningClient, followed by the SendingClient. Change line 74 of the SendingClient to use your favourite MP3, and you should hear it stream the first 20 seconds. The VOIP server itself isn't 100% reliable - especially once clients have disconnected. But I guess node.js wasn't that stable after only a small number of hours of development. Good luck!

PS. Performance: streaming at 192 kilobits a second, the server told me it was running around half a percent load (i.e. the event loop was idle 99.5% of the time). GSM, rather than high quality MP3 uses around 30 Kbps, so for a real VOIP server, you could probably get away with 1200 simultaneous calls. That doesn't seem that many, but I have no idea really. I guess I have 2 CPU cores which I could use so that I could use a load balancer to spread the load among two processes, which is the node.js way of using all the cores. But the load balancer wouldn't be doing much less work than my server does, so might not be able to handle any extra load itself. Or I could stick my handlers in a thread pool, using the runAsync attribute of my @Handler annotation. So a low cost commodity server could handle nearly 2500 simultaneous calls. Still not many? Again, I really don't know. But I should say that I didn't try optimising the server. My packet sizes are based on sending one packet of sound data every 12 milliseconds. Perhaps I could get away with sending them less frequently which might improve throughput and still have good call quality with low latency. Who knows - anyone wanting to optimise it, let me know the results! One thing for sure is that a server using one thread per connection with 2500 simultaneous connections, will struggle. While my previous blog article showed it is possible to have many thousand threads open, the context switching is likely to become the bottle neck. A non-blocking server is certainly the way forward for this use case.

Download the code here.

Copyright © 2011 Ant Kutschera

Social Bookmarks :