Tag: scala

Choosing the right language to write a simple transformation tool

Recently, a colleague asked for help in writing a little tool to transform a set of XML files into a non-normalised single table, so that their content could be easily analysed and compared, using Excel. The requirements were roughly: Read XML from several files, with the structure shown below, Write a file containing one row per combination of file and servlet name, and one column per param-name (see example below), It should be possible to import the output into Excel. Example input:In the example input above, there can be any number of servlet tags, each containing at least a name, and optionally any number of name-value pairs, representing input parameters to the servlet. Note that each servlet could contain totally different parameters! The output should then have the following structure. We chose comma separated values (CSV) so that it could easily be imported into Excel.Note how the output contains empty cells, because not every servlet has to have the same parameters. The algorithm we agreed on was as follows: Read files in working directory (filtering out non-XML files), For each file:     For each servlet:         For each parameter name-value pair:             Note parameter name             Note combination of file, servlet, parameter name and value Sort unique parameter names Output a header line for the file column, servlet column, and one column for each unique parameter name For each file:     For each servlet:         For each sorted unique parameter name:             Output a "cell" containing the corresponding parameter value,             or an empty "cell" if the servlet has…

Read more

A reactive and performant Spray + Akka solution to “Playing with concurrency and performance in Java and Node.js”

In my previous post I examined a fictitious trading engine and compared a Java based blocking solution to a Node.js based non-blocking solution. At the end of the post I wrote that: I suspect that following the recent success of Node.js, more and more asynchronous Java libraries will start to appear. Well such libraries already exist, for example: Akka, Spray, and this Mysql async driver. I set myself the challenge of creating a non-blocking Java based solution using exactly those libraries, so that I could compare its performance to that of the Node.js solution created for the last article. The first thing you might have noticed is that these are all Scala based libraries, but I wrote this solution in Java even though it is a little less syntactically elegant. In the last article I introduced a solution based upon Akka whereby the trading engine was wrapped in an actor. Here, I have dropped Tomcat as the HTTP server and replaced it with Spray, which neatly integrates the HTTP server straight into Akka. In theory this should make no difference to performance, because Spray is NIO just as Tomcat 8 is, out of the box. But what attracted me to this solution was that overall, the number of threads is greatly reduced, as Spray, Akka and the async Mysql library all use the same execution context. Running on my Windows development machine, Tomcat has over 30 threads compared to just a few over 10 for the solution built here, or…

Read more

Non-Blocking Asynchronous Java 8 and Scala’s Try/Success/Failure

Inspired by a recent newsletter from Heinz Kabutz as well as Scala's Futures which I investigated in my recent book, I set about using Java 8 to write an example of how to submit work to an execution service and respond to its results asynchronously, using callbacks so that there is no need to block any threads waiting for the results from the execution service. Theory says that calling blocking methods like get on a java.util.concurrent.Future is bad, because the system will need more than the optimum number of threads if it is to continuously do work, and that results in wasting time with context switches. In the Scala world, frameworks like Akka use programming models that mean that the frameworks will never block - the only time a thread blocks is when a user programs something which blocks, and they are discouraged from doing that. By never blocking, the framework can get away with using about one thread per core which is many less than say a standard JBoss Java EE Application Server, that has as many as 400 threads just after startup. Due largely to the work of the Akka framework, Scala 2.10 added Futures and Promises, but these don't (yet?) exist in Java. The following code shows the goal I had in mind. It has three parts to it. Firstly, new tasks are added to the execution service using the static future method found in the class ch.maxant.async.Future. It returns a Future, but not one from the…

Read more

Play 2.0 framework and XA transactions

XA transactions are useful and out of the box, Play 2.0 today does not have support for them.  Here I show how to add that support: First off, some examples when XA is useful: - JPA uses two physical connections if you use entities from two different persistence.xml - those two connections might need to be committed in one transaction, so XA is your only option - Committing a change in a database and at the same time committing a message to JMS.  E.g. you want to guarantee that an email is sent after you successfully commit an order to the database, asynchronously.  There are other ways, but JMS provides a transactional way to do this with little overhead in having to think about failure. - Writing to a physically different database because of any of several political reasons (legacy system, different department responsible for different database server / different budgets). - See http://docs.codehaus.org/display/BTM/FAQ#FAQ-WhywouldIneedatransactionmanager So the way I see it, XA is something Play needs to "support". Adding support is very easy.  I have created a play plugin which is based on Bitronix.  Resources are configured in the Bitronix JNDI tree (why on earth does Play use a config file rather than JNDI?! anyway...)  You start the transaction like this, "withXaTransaction":     def someControllerMethod = Action {                     withXaTransaction { ctx =>                         TicketRepository. addValidation(user.get, bookingRef, ctx)                         ValidationRepository.addValidation(bookingRef, user.get, ctx)                     }    …

Read more

A really simple but powerful rule engine

UPDATE: Version 2.2.0, JavaScript (Nashorn) rules in the JVM - see here. UPDATE: Version 2.1.0, Java 8, Maven, GitHub - see here. UPDATE: Version 2.0 of the library now exists which supports Scala. It is a breaking change in that "Action" instances as shown below are now "AbstractAction", and the "Action" class is only supported in Scala, where functions can be passed to the action constructor instead of having to override the AbstractAction. Scala collections are also supported in the engine. I have the requirement to use a rule engine. I want something light weight and fairly simple, yet powerful. While there are products out there which are super good, I don't want something with the big learning overhead. And, I fancied writing my own! Here are my few basic requirements: Use some kind of expression language to write the rules, It should be possible to store rules in a database, Rules need a priority, so that only the best can be fired, It should also be possible to fire all matching rules, Rules should evaluate against one input which can be an object like a tree, containing all the information which rules need to evaluate against Predefined actions which are programmed in the system should be executed when certains rules fire. So to help clarify those requirements, imagine the following examples: 1) In some forum system, the administrator needs to be able to configure when emails are sent. Here, I would write some rules like "when the config flag…

Read more