Let's start right at the beginning. Bear with me, it might get long...

The following snippet of Java code could be used to create a server which receives TCP/IP requests:

class Server implements Runnable {
    public void run() {
        try {
            ServerSocket ss = new ServerSocket(PORT);
            while (!Thread.interrupted())
                Socket s = ss.accept();
                s.getInputStream(); //read from this
                s.getOutputStream(); //write to this 
        } catch (IOException ex) { /* ... */ }
    }
}

This code runs as far as the line with ss.accept(), which blocks until an incoming request is received. The accept method then returns and you have access to the input and output streams in order to communicate with the client.

There is one issue with this code. Think about multiple requests coming in at the same time. You are dedicated to completing the first request before making the next call to the accept method. Why? Because the accept method blocks. If you decided you would read a chunk off the input stream of the first connection, and then be kind to the next connection and accept it and handle its first chunk before continuing with the original (first) connection, you would have a problem, because the accept method blocks. If there were no second request, you wouldn't be able to finish off the first request, because the JVM blocks on that accept method. So, you must handle an incoming request in its entirety, before accepting a second incoming request.

This isn't so bad, because you can create the ServerSocket with an additional parameter, called the backlog, which tells it how many requests to queue up before refusing further connections. While you are busy handling the first request, subsequent requests are simply queued up.

This strategy would work, although it's not really efficient. If you have a multicore CPU, you will only be doing work on one core. It would be better to have more threads, so that the load can be balanced across the cores (watch out, this is JVM and OS dependent!).

A more typical multi-threaded server gets built like this:

class Server implements Runnable {
    public void run() {
        try {
            ServerSocket ss = new ServerSocket(PORT);
            while (!Thread.interrupted())
                new Thread(new Handler(ss.accept())).start();
                // one thread per socket connection every thread 
                // created this way will essentially block for I/O
        } catch (IOException ex) { /* ... */ }
    }
}

The above code hands off each incoming request to a new thread, allowing the main thread to handle new incoming requests, while spawned threads handle individual requests. This code also balances the load across CPU cores, where the JVM and OS allow it. Ideally, we probably wouldn't create a thread per new request, but rather hand off the request to a thread pool executor (see the java.util.concurrent package). On the other hand, there are times when a thread per request is required. If the conversation between server and client is longer lasting (rather than a simple HTTP request that is typically serviced in anything from milliseconds to seconds), then the socket can stay open. An example of when this is required are things like chat servers, or VOIP, or anything else where a continual conversation is required. But in such situations, the above code, even though it is multi-threaded, has it's limits. Those limits are actually because of the threads! Consider the following code:

public class MaxThreadTest {

    static int numLive = 0;
	
    public static void main(String[] args) {
        while(true){
            new Thread(new Runnable(){
                public void run() {
                    numLive++;
					
                    System.out.println("running " + Thread.currentThread().getName() + " " + numLive);
                    try {
                        Thread.sleep(10000L);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }

                    numLive--;
                }
            }).start();
        }
    }
}

This code creates a bunch of threads, until the process crashes. With 64 MB heap size, it crashed (out of memory) after around 4000 threads, while testing on my Windows XP Thinkpad laptop. I upped the heap size to 256 MB and Eclipse crashed while in debug mode... I started the process from the command line and managed to open 5092 threads, but it was unstable and unresponsive. Interestingly, I upped the heap size to 1 GB, and then I could only open 2658 threads... This shows, I don't really understand the OS or JVM at this level! Anyway, if we were writing a system to handle a million simultaneous conversations, we would probably need over two hundred servers. But theoretically, we could reduce our costs to less than 10% of that, because we are allowed to open just over 65,000 threads per server (well, say 63,000 by the time we account for all the ports used by the OS and other processes). We could theoretically get away with just having 16 servers per million simultaneous connections.

The way to do this is, is to use non-blocking I/O. Since Java 1.4 (around 2002?), the java.nio package has been around to help us. With it, you can create a system which handles many simultaneous incoming requests using just one thread. The way it works is roughly by registering with the OS to get events when something happens, for example when a new request is accepted, or when one of the clients sends data over the wire.

With this API, we can create a server, which is, sadly, a little more complicated than those above, but which handles lots and lots of sockets all from one thread:

public class NonBlockingServer2 {

	public static void main(String[] args) throws IOException {
		System.out.println("Starting NIO server...");
		Charset charset = Charset.forName("UTF-8");
		CharsetDecoder decoder = charset.newDecoder();
		CharsetEncoder encoder = charset.newEncoder();
		
		ByteBuffer buffer = ByteBuffer.allocate(512);

		Selector selector = Selector.open();
		ServerSocketChannel server = ServerSocketChannel.open();
		server.socket().bind(new InetSocketAddress(30032));
		server.configureBlocking(false);
		SelectionKey serverkey = server.register(selector, SelectionKey.OP_ACCEPT);

		boolean quit = false;
		while(!quit) {
			selector.select(); //blocks until something arrives, of type OP_ACCEPT
			Set keys = selector.selectedKeys();
			for (SelectionKey key : keys) {
				if (key == serverkey) {
					if (key.isAcceptable()) {
						SocketChannel client = server.accept();
						if(client != null){ //can be null if theres no pending connection
							client.configureBlocking(false);
							SelectionKey clientkey = client.register(selector,
									SelectionKey.OP_READ); //register for the read event
							numConns++;
						}
					}
				} else {
					SocketChannel client = (SocketChannel) key.channel();
					if (!key.isReadable()){
						continue;
					}
					int bytesread = client.read(buffer);
					if (bytesread == -1) {
						//whens this happen?
						key.cancel();
						client.close();
						continue;
					}
					buffer.flip();
					String request = decoder.decode(buffer).toString();
					buffer.clear();
					
					if (request.trim().equals("quit")) {
						client.write(encoder.encode(CharBuffer.wrap("Bye.")));
						key.cancel();
						client.close();
					}else if (request.trim().equals("hello")) {
						String id = UUID.randomUUID().toString();
						key.attach(id);
						String response = id + "\r";
						client.write(encoder.encode(CharBuffer.wrap(response)));
					}else if (request.trim().equals("time")) {
						numTimeRequests++;
						String response = "hi " + key.attachment() + " the time here is " + new Date() + "\r";
						client.write(encoder.encode(CharBuffer.wrap(response)));
					}
				}
			}
		}
		System.out.println("done");
	}
}

The above code is based on that found here. By reducing the number of threads being used, and not blocking, but rather relying on the OS to tell us when something is up, we can handle many more requests. I tested some code very similar to this to see how many connections I could handle. Windows XP proved its high reliability, when reproducibly and consistently, more than 12,000 connections lead to blue screens of death! Time to move to Linux (Fedora Core). I had no problems creating 64,000 clients all simultaneously connected to my server. Let me re-prase... I didn't have problems having the clients simply connect and keep the connection open, but getting the server to also handle just 100 requests a second caused problems. Now 100 requests a second on a web server, on hardware which was a 5 year old cheap Dell laptop, sounds quite impressive to me. But on a server with 64,000 concurrent connections, that means each client making a request every ten minutes! Not very good for a VOIP application... The connection speeds also slowed down from around 3 milliseconds with 500 concurrent connections, down to 100 milliseconds with 60,000 concurrent connections.

So, perhaps I better get to the point of this posting? A few days ago, I read about Voxer, and Node.js on The Register. I had difficulty with this article. Why would anyone want to build a framework for Javascript on the server? I have developed plenty of rich clients, and have the experience to understand how to do rich client development. I have also developed plenty of rich internet apps (RIA), which use Javascript, and I can only say, it's not the best. I'm not some script kiddie or script hacker who doesn't know how to design Javascript code, and I understand the problems of Javascript development well. And I have developed lots and lots of server side code, mostly in Java and appreciate where Java out punches Javascript.

It seems to me, that the developers of Node.js, and those following it and using it, don't understand server development. While writing in Javascript might initially be quicker, the lack of tools and libraries in comparison to Java make it a non-competition in my opinion.

If I were a venture capitalist, and knew my money was being spent on application development based on newly developed frameworks, instead of extremely mature technologies, when the mature technologies suffice (as shown with the non-blocking server code above), I would flip out and can the project.

Maybe though, this is why I have never worked at a start up!

To wrap up, let's consider a few other points. Before anyone says that the performance of my example server was poor because it's just Java which is slow, let me comment. First of all, Java will always be faster than Javascript. Secondly, using top to monitor the server, I noticed that 50% of the CPU time was spent by the OS working out what events to throw, rather than Java handling those requests.

In the above server, everything runs on one thread. To improve performance, once a request comes in, it could be handed off to a thread pool to respond. This would help balance load across multiple cores, which is definitely required to make the server production ready.

While I'm at it, here is a quote from Node JS's home page:

"But what about multiple-processor concurrency? Aren't threads necessary to scale programs to multi-core computers? Processes are necessary to scale to multi-core computers, not memory-sharing threads. The fundamentals of scalable systems are fast networking and non-blocking design—the rest is message passing. In future versions, Node will be able to fork new processes (using the Web Workers API ) which fits well into the current design."

Actually, I'm not so sure... Java on Linux can spread threads across cores, so individual processes are not actually required. And the above statement just proves that Node JS is not mature for building really professional systems - I mean come on, no threading support?!

So, in the interests of completion, here is the client app I used to connect to the server:

public class Client {

	private static final int NUM_CLIENTS = 3000;

	static Timer serverCallingTimer = new Timer("servercaller", false);
	
	static Random random = new Random();

	/**
	 * this client is asynchronous, because it does not wait for a full response before 
	 * opening the next socket.
	 */
	public static void main(String[] args) throws UnknownHostException, IOException, InterruptedException {
		
		final InetSocketAddress endpoint = new InetSocketAddress("192.168.1.103", 30032);
		System.out.println(new SimpleDateFormat("HH:mm:ss.SSS").format(new Date()) + " Starting async client");

		long start = System.nanoTime();
		for(int i = 0; i < NUM_CLIENTS; i++){
			startConversation(endpoint);
		}

		System.out.println(new SimpleDateFormat("HH:mm:ss.SSS")
				.format(new Date())
				+ "Done, averaging "
				+ ((System.nanoTime() - start) / 1000000.0 / NUM_CLIENTS)
				+ "ms per call");
	}

	protected static void startConversation(InetSocketAddress endpoint) throws IOException {
		final Socket s = new Socket();
		s.connect(endpoint, 0/*no timeout*/);
		s.getOutputStream().write(("hello\r").getBytes("UTF-8")); //protocol dictates \r is end of command
		s.getOutputStream().flush();

		//read response
		String str = readResponse(s);
		System.out.println("New Client: Session ID " + str);

		//send a request at regular intervals, keeping the same socket! eg VOIP
		//we cannot use this thread, its the main one which created the socket
		//simply create another task to be carried out by the scheduler at a later time

		//the interval below is 4 minutes, otherwise the server gets REALLY slow handling 
		//so many requests.  This is equivalent to ~260 reqs/sec
		
		serverCallingTimer.scheduleAtFixedRate(
				new ConversationContainer(s, str), 
				random.nextInt(240000/*in the next 4 mins*/), 
				240000L/*every 4 mins*/);
	}
	
	private static String readResponse(Socket s) throws IOException {
		InputStream is = s.getInputStream();
		int curr = -1;
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		while((curr = is.read()) != -1){
			if(curr == 13) break; //protocol dictates a new line is the end of a response
			baos.write(curr);
		}
		return baos.toString("UTF-8");
	}

	private static class ConversationContainer extends TimerTask {
		Socket s;
		String id;
		public ConversationContainer(Socket s, String id){
			this.s = s;
			this.id = id;
		}
		
		@Override
		public void run() {
			try {
				s.getOutputStream().write("time\r".getBytes("UTF-8")); //protocol dictates \r is end of command
				s.getOutputStream().flush();

				String response = readResponse(s);
				
				if(random.nextInt(1000) % 1000 == 0){
					//we dont want to log everything, because it will kill our server!
					System.out.println(id + " - server time is '" + response + "'");
				}
				
			} catch (Exception e) {
				e.printStackTrace();
			}
		}
	}
}

Copyright © 2010 Ant Kutschera

Social Bookmarks :                                              

Responses[1]

Posted by Ant Kutschera on March 5, 2011 10:36:00 PM CET #

My current client has a service which connects to an old IBM z/OS application (legacy system). The data centre charges for each message sent to this legacy system, rather than using a processor or hardware based pricing model. The output from this legacy system is always the same, since the calculations are idempotent. The application calculates prices for travelling along a given route of the train network. Prices are changed only twice a year through an administration tool. So in order to save money (a hundered thousand dollars a year), the service which connects to this legacy system had an in-memory least-recently-used (LRU) cache built into it, which removes the least recently used entries when it gets full in order to make space for new entries. The cache is quite large, thus avoiding making costly calls to the legacy system. To avoid losing the cache content upon server restarts, a background task was later built to periodically persist the latest data inside this cache. Upon starting a server, the cache content is re-read. Entries within the cache have a TTL (time to live) so that stale entries are discarded and re-fetched from the legacy system.

This cache was great in the beginning because it saved the customer a lot of money, but in the mean time several similar caches have been added, as well as more general caches for avoiding repeated database reads, causing our nodes to need over 1.5 GB of RAM. Analysis has showed that the caches are consuming over 500 MB alone. The platform is WebSphere and IBM only recommends running with 512 MB per node... So as you can see, we are on the road to having some memory issues. In order to avoid these issues, and potentially also save money by requireing less hardware for our applications, I have taken time to think about possible architectures which could solve the problems.

One of the problems is that the cache is an internal part of the service, i.e. it is a glorified Map referenced as a singleton. That means, each node has its own cache instance. Worse actually, because of the class loaders in application servers, each application has its own instance of the service and hence its own instance of the cache, and because the service is deployed per application, each cache exists multiple times per node in the cluster. The result is an architecture as follows, where there are for example 8 instances of the same cache:



While these in memory caches are quick, that is not the main reason for having created them. The main reasons were to avoid hammering either the database or the legacy systems with repeated queries which result in the same outcome. While the caches theoretically speed up the the applications, the associated increase in garbage collection might actually result in lower overall speed because the garbage collector needs to run more frequently because there is less memory available for use by the application itself, because the caches consume so much.

This got me thinking about creating a centralized cache, of which there is one single instance for use by all applications. The architecture would then look like this:



The requirements for the cache are:

• Persistent, so that server restarts do not cause all content to be lost,
• Single, so that the entire landscape contains just one cache rather than the same data being cached lots of times as it is today,
• Relatively fast,
• No impact on existing infrastructure, i.e. if we stick our cache into our database, that influences the reading/writing of other data by making the database perform less well.

So, what are the options for creating such a cache?

• In Memory, but remote, backed by persistence, similar to what we have today, only with the service deployed as a remote EJB, rather than being called locally,
• A bigtable style, NoSQL cache, because they are based on key/value pairs and can handle lots of data,
• LDAP, because LDAP is persistent as well as optimised for read access.

A fourth option might be to use a database cache, i.e. a table devoted to key/value pairs cached in the database, which gets read every time you need data, but this would affect overall database performance, so isn't really an option. I have however included such a solution in the solutions that I tried below, simply to compare performance.

While the concept of a bigtable style database is aimed at large amounts of data (billions of rows and millions of columns), what is of interest in this case is that it is based on key/value pairs, which is exactly what we need for our cache. The caches used today are LRU caches, because to store all possible data would require too much memory (there are millions of routes on the train network), which is also the reason why we wouldn't want to store all possible routes in the database. However, because a bigtable style database supports large data sets while maintaining performance, we can potentially just let it contain all possible routes. The strategy for filling such a cache would be to check the cache for the required entry, if it doesn't exist, to then query the legacy system, and its answer be cached for future use, until it becomes stale.

So first things first, I started up Glassfish for Eclipse and MySQL to do some tests. I created a set of tables to contain 100,000 train stations, generated randomly. Then I added 100,000 random routes, each containing between 3 and twenty stations. It is these routes which the legacy system calculates prices for. The first solution I implemented was a pure database cache, where a query to the service which encapsulates the legacy system first checked if the database contained a cache entry for the key, made up of the "from" station and the "to" station codes. If there was no hit on the cache, the legacy system was queried and its result was cached. I filled the cache with all 100,000 routes and began making measurements. My first results were rather poor, with requiring several hundred milliseconds to read from the cache, even though the cache key was the primary key, that is indexed! OK, the test system was my aging IBM T60 Intel Core Duo (2 core CPU) laptop with only 2 GB RAM, running Windows XP, Glassfish, MySQL, and a handful of other (memory intensive) applications and services.

I started reading up about Cassandra, the Apache application based on bigtable, and in the performance tuning section it mentioned about using two disks for seperate files which it writes. That made me think about what was going on on my laptop and I realised that one reason why my first solution was so slow was that I was running applications using over 2 GB of RAM, so I was well into the region where the OS was using virtual memory and swapping from RAM to disk continuously. Sure enough, during tests, my hard disk light was going crazy. I have been wanting to upgrade this laptop for a while, but it belongs to the client, so I cannot really upgrade it. But I did a little more reading and decided to try out a USB 3.0 external hard disk (500 GB "My Passport" from Western Digital), together with an Expressbus USB 3.0 adapter. This gave me a very fast external hard drive. Random read/write tests showed that this external hard drive was actually up to three times faster than the internal one!

So, I got back to testing, and installed my apps (Glassfish and MySQL, etc) on the external hard drive, leaving the internal harddrive for the OS and disk swapping. In the ideal world, one would do this the other way around and install an SSD for the OS and its swap file, leaving a slower more conventional spinning hard disk for applications and data. Anyway, instead of a single drive having to try to do swapping and reading of say the database simultaneously, this configuration let the internal drive handle swapping, while the external drive handled reading for the database, basically allowing parallel tasks.

Sure enough, I was able to get random cache read times for this database cache, down to 15 milliseconds. It still wasn't that fast, but interestingly, this database was doing nothing else at all, simply servicing cache reads (i.e. reads out of a single table with a cache key as the primary key, and a varchar for the value (a serialised Java object). Since this solution influences the existing infrastructure, i.e. the database, it would not fulfil the requirements I created earlier in this post.

I next downloaded Cassandra from Apache, and read a few introductions like the ones here and here.

What I wanted to do was not complex at all, so I created a structure with a single ColumnFamily (like a table). I filled it with all 100,000 routes, and started performance testing, and got average read times similar to MySQL. I read up a little more about performance tuning and I discovered that I could control the number of entries cached in memory inside Cassandra. I set this to be 99% and sure enough, my results improved to the point that after a few hundred thousand reads (which would happen fast in production), I was getting read times of around 2 milliseconds. Cassandra was running as a single process on the same laptop on which GlassFish was running, which contained a service which wrapped calls to Cassandra. The data structures were simple with each cache entry stored within a single key in Cassandra, basically having a single column for the ColumnFamily, containing the serialised route object.

Further reading introduced the idea that Cassandra is optimised for reading many columns within a single "row" (key). I tried restructuring the data so that the key for each "row" was the "from" station code, and for each destination from that station, a column was added within the row. So I had many less keys in Cassandra, but many more columns per row. And because these bigtable solutions do not require each key to have identical columns, allowing sparsely filled tables, it was perfect for this restructuring.

I refilled the cache using this new structure, and sure enough, once Cassandra had loaded a few hundred thousand keys and they were all cached in its memory (set at 256 MB for the process), average random read times sank to just 1 millisecond. Compared to MySQL things were looking good.

Time for the next proposed solution, which was to make calls to a remote EJB running in its own node (or rather cluster). I implemented this by creating a simple stateless session EJB which contained a static reference to a Hashtable (so that it was thread safe). I didn't bother implementing the persistent part in the first instance, because I was interested in how long it would take to simply call the remote EJB and do the lookup in the Map. I deployed the EJB to Apache OpenEJB running as a simple process on the same laptop and after filling the cache completely, the average time for random reads was around 2 milliseconds, i.e. similar to Cassandra, albeit a tiny bit slower.

The final option was to try out LDAP. I only considered this because LDAP is optimised for reading, which is what this service is doing. I used ApacheDS that also as a process on my laptop. I filled LDAP with my 100,000 routes by basically filling my organisation unit with new sub-organisational units per key, with the Base64 encoded serialised object attached to the sub-OU as an attribute, such that the cache entry had a URL like ldap://localhost:10389/ou=8500101_8500103,ou=verbindungen,ou=system. Sadly I am no expert on LDAP and was not too sure how to optimise it, beyond increasing the cache size attribute of the system partition in server.xml inside ApacheDS. The results were far from promising with average random read times around 50 milliseconds.

In all cases, I optimised my code so that connections to the remote processes were created post EJB construction, and just once, until the EJB instance was destroyed (using the @PostConstruct EJB 3 annotation). In all cases, the average read times were measured per cache entry, based on 1000 random serial cache reads, after reading over 100,000 entries to ensure that the external process had had time to optimised its own in memory key cache.

The results are summarised as follows:

The nice part about using Cassandra would be that we don't have to write the code for this cache, nor maintain it. Additionally, there are other areas within our landscape that could make good use of a bigtable style solution because the amounts of data are getting very slow to handle, so it would be useful to start getting to know the new world.

The not so nice part about Cassandra, is that it is not standard software at this client. We would need to create a request for architecture and invest time convincing some enterprise architects that we need Cassandra and that our platform people and data centre should invest in order to support it. This could end up involving software evalutations and eventual licence costs if they chose a commercial solution. All this effort could easily cost more than we could possibly save by implementing a single cache in order to reduce the amount of hardware required today. Politics are great, eh :-| I shall be putting the proposals forward in the new year, let's see what they bring.

One quick final note: the times measured here are not realistic of the target environment - a laptop doesn't compare to a cluster! But they can be used as relative indicators to suggest which solutions might be worth further investigation. Only final deployment to production can give the real answer, as shown by this example, although just because Digg had problems, doesn't mean Cassandra is bad. It does show that you need to think long and hard about your architecture though.

Copyright © 2010 Ant Kutschera

Social Bookmarks :                                              

Add a comment

Posted by Ant Kutschera on December 30, 2010 10:43:00 PM CET #

• Trip: Consists of a Train and a list of TripSegments.
• Train: consists of a list of Locomotives, and a list of Wagons. The assembly of a train comes from the Timetable.
• Locomotive: Has an efficiency, a weight and rolling friction factor.
• Wagon: has a weight and rolling friction factor.
• TripSegment: has a Segment, and the average speed along which the Train will travel on that segment, during its trip.
• Segment: a stretch of track, which has a length, positive hill climbs and negative hill decents. Has a starting station and an end station, which are not relevant to the energy calculations - it is the distance which counts.
• Timetable: a service which provides all Trips for a given time period.

@Stateless
@Context
public class ForecastingContext implements ForecastingContextLocal {

	/**
	 * an asynchrounous method for determining the energy requirements of the given trip.
	 */
	@Asynchronous
    public Future forecastEnergyRequired(Trip trip) {

		BehaviourInjector bi = new BehaviourInjector(this);
		
		//the context simply casts objects into roles, and starts the interaction
		EnergyConciousTrip t = bi.assignRole(trip, EnergyConciousTrip.class);
		double energy = t.forecastEnergyRequirements();
		
		return new AsyncResult(energy);
    }
}

/**
 * this role is played by a trip so that it is able to 
 * give a prediction about how much energy will be 
 * needed to undertake the trip.
 */
@Role(contextClass=ForecastingContext.class, 
		implementationClass=EnergyConciousTrip.EnergyConciousTripImpl.class)
public interface EnergyConciousTrip {

	double forecastEnergyRequirements(); //role
	List getTripSegments(); //data
	Train getTrain(); //data
	
	static class EnergyConciousTripImpl{ //role implementation
		
		@CurrentContext
		private IContext ctx;
		
		@Self
		private EnergyConciousTrip self;

		/**
		 * energy consists of a longitudinal component (ie along the track due to 
		 * aerodynamic forces and rolling resistance) as well as altidudal component
		 * (ie due to climbing or descending).
		 * 
		 * E = mgh //ie potential energy for climbing / descending
		 * E = Fd  //ie energy required to overcome resistive forces
		 */
		public double forecastEnergyRequirements(){
			
			//give the train some scientific behaviour and determine
			//its total weight and average efficiency
			ScientificTrain train = ctx.assignRole(self.getTrain(), ScientificTrain.class);
			double totalWeight = train.getTotalWeightKg();
			double avgEff = train.getAverageEfficiency();
			
			//add up all energy per segment
			double energy = 0.0;
			for(TripSegment seg : self.getTripSegments()){
				
				double segmentEnergy = train.getResistanceForce(seg.getSpeed()) * seg.getSegment().getLengthMeters();
				
				segmentEnergy += totalWeight * seg.getSegment().getAltitudeChange();
				
				//each locomotive can pull, but has an efficiency which needs to be factored in!
				//it needs more energy than just pulling wagons, because its inefficient
				segmentEnergy /= avgEff;
				
				energy += segmentEnergy;
			}
			
			return energy;
		}
	}
}


/**
 * played by a train, when it needs to provide scientific details about itself.
 */
@Role(contextClass=ForecastingContext.class,
		implementationClass=ScientificTrain.ScientificTrainImpl.class)
public interface ScientificTrain {

	double getResistanceForce(double speed); //role
	int getTotalWeightKg(); //role
	double getAverageEfficiency(); //role
	List getWagons(); //data 
	List getLocomotives(); //data
	double getDragCoefficient(); //data
	
	static class ScientificTrainImpl { //role impl

		@Self
		private ScientificTrain self;

		/** 
		 * resistance force is the rolling friction as a 
		 * result of weight, as well as aerodyamic drag.
		 */
		public double getResistanceForce(double speed){
			double resistanceForce = 0.0;
			for(Wagon w : self.getWagons()){
				resistanceForce += w.getWeightKg() * w.getRollingFriction();
			}
			for(Locomotive l : self.getLocomotives()){
				resistanceForce += l.getWeightKg() * l.getRollingFriction();
			}
			
			resistanceForce += speed * speed * self.getDragCoefficient();
			
			return resistanceForce;
		}

		/** total weight of wagons and locs. */
		public int getTotalWeightKg(){
			int weight = 0;
			for(Wagon w : self.getWagons()){
				weight += w.getWeightKg();
			}
			for(Locomotive l : self.getLocomotives()){
				weight += l.getWeightKg();
			}
			return weight;
		}

		/** average of all locs in the train */
		public double getAverageEfficiency(){
			double avg = 0.0;
			for(Locomotive l : self.getLocomotives()){
				avg += l.getEfficiency();
			}
			return avg / self.getLocomotives().size();
		}
	}	
}

@Stateless
public class EnergyForecastingService implements EnergyForecastingServiceLocal {

	@EJB
	private TimetableLocal timetable;

	@EJB
	private ForecastingContextLocal forecastingContext;
	
	@Override
	public EnergyRequirementResult forecast() throws InterruptedException, ExecutionException {
		
		long start = System.nanoTime();
		List> results = new ArrayList>();
		EnergyRequirementResult result = new EnergyRequirementResult();
		result.setTrips(timetable.getTripsForPeriod(null, null));
		for(Trip trip : result.getTrips()){
			//start it in a parallel thread
			//it will be simulated using a DCI context...
			Future e = forecastingContext.forecastEnergyRequired(trip);
			results.add(e);
		}
		
		//ok, all started, lets wait until all are done...
		result.setEnergyRequirement(waitForResults(results));
		result.setCalculationTimeNs(System.nanoTime() - start);
		
		return result;
	}

	private double waitForResults(List> results) throws InterruptedException, ExecutionException {
		double energyRequired = 0.0;
		while(!results.isEmpty()){
			List> assimilated = new ArrayList>();
			for(Future result : results){
				if(result.isDone()){
					assimilated.add(result);
					energyRequired += result.get().doubleValue();
				}
			}
			results.removeAll(assimilated);
			
			//lets give CPU time to other threads while we wait...
			Thread.sleep(5L);
		}
		
		return energyRequired;
	}
}

Social Bookmarks :                                              

Add a comment

Posted by Ant Kutschera on November 20, 2010 10:29:00 PM CET #

The Data, Context, and Interaction (DCI) architecture paradigm introduces the idea of thinking in terms of roles and contexts. See some of my white papers for a more detailed introduction into DCI, but for this blog article, consider the following example: a human could be modelled in object oriented programming by creating a super huge massive class which encapsulates all a humans attributes, their behaviours, etc. You would probably end up with something much too complex to be really maintainable. Think about when a human becomes a clown for a kids party; most of that behaviour has little to do with being a programmer, which is a different role which the human could play.

So, DCI looks at the problem differently than OOP and solves it by letting the data class be good at being a data class, and putting the behaviours specific to certain roles into "roles", which in my DCI Tools for Java library are classes.

Certain roles interact with each other within a given context, such as a clown entertaining kids at a birthday party. The roles which belong to an interaction are part of the context, and in DCI the context is a class which puts data objects into specific roles, and makes them interact. The context and its roles form the encapsulation of the behaviour.

I have updated my library, so that there are two new Annotations, namely the @Context and @Role annotations. The @Context annotation is simply a marker to show that a class is a context. The @Role annotation is more important. It is placed onto the role interface class and tells the context where to find the role implementation, and additionally requires that the programmer specifically state to which context the role belongs.

The code ends up looking looking as follows. First a unit test which creates and calls the context:

    @Test
    public void testExecute() {
    	
    	//create an adult (the clown) and some children, 
    	//and let the party start...
        
    	Human adult = new Human();
        List children = new ArrayList();
        Human child = new Human();
        child.setName("Johnny");
        children.add(child);
        child = new Human();
        child.setName("Jane");
        children.add(child);

        ClownContext cc = new ClownContext(adult, children.toArray());
        cc.startParty();

        //check the party went well...
        assertEquals(0, adult.getHappiness());
        for (Human c : children) {
            assertEquals(1, c.getHappiness());
        }
    }
}

There is nothing special about the code above. It simply creates some data objects and passes them to the context, and then starts the party (the interaction, or use case). Next, the context class is shown:

/** the context in which a clown entertains some kids. */
@Context
public class ClownContext extends BehaviourInjector {

    /** the object who will play the clown role */
    private Object adult;
    
    /** the objects who will play the kid role */
    private List

The context has a few noteworthy points. First of all, it extends the BehaviourInjector. By doing this, it has access to the assignRole(Object, Class) and getIterable(Iterator, Class) methods. Note the @Context annotation at the top - it's important for not only helping the programmer spot that it's a context, but also for the Eclipse plugin which I created and which I introduce below.

You might have spotted that the context above is not tightly coupled with the Human class. The reason is simply to make the context and its roles usable in the future, with data classes which I currently know nothing about. It is feasible that a robot might one day do parties by dressing up as a clown!

Assigning the role is where the behaviour is "injected" into the data object. The iterable which is created is a special role, which can automatically cast the human children into the role of a kid attending a party.

Next, take a look at the role interface. This class defines the methods which data object must have to play the role, as well as the behaviour which will be added to the object.

@Role(contextClass = ClownContext.class, 
	  implementationClass = ClownImpl.class)
public interface Clown {

	/** put some makeup on (initialise) */
	public void applyMakeup(); // role

	/** makes kids laugh */
	public void makeKidsLaugh(IIterable kids); // role
	
	public void setHairColour(Color c); // data
	
}

The role interface has the @Role annotation which tells the behaviour injector firstly to which context the role belongs, and secondly where the actual implementation (behaviour) can be found. In the clown role specified above, two behaviours are added (applying make up and making kids laugh). The application of make up additionally requires that the clown be able to set his hair colour, so that method which the human class must have is also specified here.

Finally, the role implementation class:

public class ClownImpl {

	@Self
	private Clown self;
	
	/** @see Clown#applyMakeup() */
	public void applyMakeup(){
		self.setHairColour(Color.GREEN);
	}

	/** @see Clown#makeKidsLaugh(IIterable) */
	public void makeKidsLaugh(IIterable kids) {
		for (Kid kid : kids) {
			kid.laugh();
			System.out.println();
		}
	}
}

The role implementation does not need to implement the role interface! It is simply a class which the behaviour injector relies upon to find behaviour which it cannot find in the data object. The @Self annotation on the clown field called self is a reference to the object playing the clown role, and is similar to "this", just that "this" would be a reference to the instance of the role implementation, rather than the object playing the role. It allows you to access the data methods of the data object.

Now, as well as helping the programmer to mentally associate contexts with roles, these two annotations allow static analysis of the code to be performed. Using the Eclipse JDT framework, it is possible to obtain individual tokens of source code, even when the class files don't compile, in order to build up a model of contexts and roles within a Java project. I created a little DCI Outline view for Eclipse, similar to the class outline, except rather than showing classes, it shows the contexts and their associated roles. Here's a screen shot:



The view shows all contexts which it finds in the selected Java project. For each context, it shows the package, followed by a list of its methods, and then all the roles that belong to it.

For each role interface, the package is listed, followed by a list of methods which the role interface contains. At the bottom of a role branch, is the role implementation class.

The idea of this plugin is to give the programmer a view of what contexts are available and what roles they contain. It puts contexts, roles and their methods into the mind of the programmer letting them think in terms of DCI, exactly the same way which a standard class browser lets the programmer think in terms of classes.

Double clicking on a role interface, role implementation or context will open it in the editor.

The plugin has limited validation capabilities and can determine for example, if a context specified in a role annotation cannot be found within the project, and highlights this so that the programmer can see there is a problem. In such cases, the view looks like this:



The code from this example can be downloaded here.

A full instruction manual, together with the the DCI Tools for Java library, and the Eclipse Update site can be found at here.

Attention: No clowns were harmed during the writing of this blog.

© 2010 Ant Kutschera

Social Bookmarks :                                              

Responses[4]

Posted by Ant Kutschera on November 16, 2010 9:58:00 PM CET #

Have you ever had to create a mock object in which most methods do nothing and are not called, but in others something useful needs to be done?

EasyMock has some newish functionality to let you stub individual methods. But before I had heard about that, I had built a little framework (one base class) for creating mock objects which stubs those methods you want to stub, as well as logging every call made to the classes being mocked.

It works like this: you choose a class which you need to mock, for example a service class called FooService, and you create a new class called FooServiceMock. You make it extend from AbstractMock<T>, where T is the class you are mocking.

As an example:

public class FooServiceMock extends AbstractMock<FooService> {

    public FooServiceMock() {
        super(FooService.class);
    }

It needs to have a constructor to call the super constructor passing the class being mocked too. Perhaps that could be optimised, I don't have too much time right now.

Next, you implement only those methods you expect to be called. For example:

public class FooServiceMock extends AbstractMock<FooService> {

    public FooServiceMock() {
        super(FooService.class);
    }

    /** 
     * this is a method which exists in FooService, 
     * but I want it to do something else.
     */
    public String sayHello(String name){
	    return "Hello " + name + 
              ", Foo here!  This is a stub method!";
    }	

To use the mock, you'll notice that it doesn't extend the class which it mocks, which might be problematic... Well, there are good reasons. To do the mocking, the abstract base class is actually going to create a dynamic proxy which wraps itself behind the interface of the class being mocked. To the caller, it looks like the FooService, but it's not actually anything related to it. Anytime a call to the FooService is made, the first thing which the proxy does is log that call, using XStream to create an XML representation of the parameters being passed into the method. Then, the proxy goes and looks in the instance of the mock class to see if it can find the method being called (well at least a method which takes the same parameters and has the same name and return type). If it finds such a method, it calls it. In our example, the sayHello(String) method would get called. It returns the result if there is one, to the caller.

In the case where it cannot find the method, it throws an exception, because it assumes that if it was not implemented, you didn't expect it to be called. You could of course change this to suit your needs, maybe even calling the actual FooService.

So, how to you use the FooServiceMock to create a FooService instance which you can use to mock your service? In the test, where you setup the class under test, you do this:

    FooServiceMock fooService = new FooServiceMock();
	
    //perhaps tell it about objects you would
    //like it to return...
	
    instanceOfClassUnderTest.setFooService(
                               fooService.getMock());	

The setFooService(FooService) method on the instance of the class you are testing is in my case present, but you might not have it and may need to use reflection to do it. It's a question of how testable you write your classes, and is a design choice.

The getMock() method on the AbstractMock class is the method which creates the dynamic proxy which wraps the instance of the mock.

You can now test the class. There is however still something useful you can do after testing, i.e. assert that the right calls were made in the correct order with the right parameters. You do this in the test class to:

    assertEquals(1, fooService.getCalls().size());
    assertEquals("[sayHello: <String>Ant</String>]", 
	                  fooService.getCalls().toString());

The above tests that the sayHello(String) method was called just once, and passed the name "Ant".

There are times when you might want to clear the call log, between parts of the test. For that, call the clearCalls() method on the mock object:

        fooService.clearCalls();

Finally, here is the code for the AbstractMock, which you might want to tweak, depending upon your needs:

/*  
 * Copyright (c) 2010 Ant Kutschera, maxant
 * 
 * This file is part of Ant Kutschera's blog, 
 * http://blog.maxant.co.uk
 * 
 * This is free software: you can redistribute
 * it and/or modify it under the terms of the
 * Lesser GNU General Public License as published by
 * the Free Software Foundation, either version 3 of
 * the License, or (at your option) any later version.
 * 
 * It is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied
 * warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
 * PURPOSE.  See the Lesser GNU General Public License for
 * more details. You should have received a copy of the
 * Lesser GNU General Public License along with the blog.
 * If not, see .
 */
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.ArrayList;
import java.util.List;
import com.thoughtworks.xstream.XStream;

/**
 * base class for mocks who need to log their calls.
 *
 * @author   Ant Kutschera
 */
public abstract class AbstractMock {

  /** 
   * xstream is used for creating a serialised xml
   * representation of objects
   */
  private XStream xstream = new XStream();

  /** every call to the class being mocked is logged */
  private List calls = new ArrayList();

  /** 
   * the interface which needs to be exposed, ie the
   * class of the object being mocked
   */
  private Class interfaceClass;

  /**
   * Creates a new AbstractMock object.
   *
   * @param  interfaceClass  the class of the object
   *                  being mocked
   */
  public AbstractMock(Class interfaceClass){
    this.interfaceClass = interfaceClass;
  }

  /**
   * @return  a list of strings, one for each call
   *    containing the method name followed by each
   *  parameter as an XML string as created by XStream.
   */
  public List getCalls() {
    return calls;
  }

  /**
   * resets the list of calls.
   */
  public void clearCalls() {
    calls.clear();
  }

  /**
   * @return  a proxy which wraps the instance, so
   *          that it can automatically handle
   *          unimplemented methods.
   */
  @SuppressWarnings({ "unchecked", "rawtypes" })
  public T getMock() {
    ClassLoader classLoader = getClass().getClassLoader();
    Class[] interfaces = new Class[] { interfaceClass };
    InvocationHandler ih = new InvocationHandler() {
	
      @Override
      public Object invoke(Object proxy, Method method, 
                          Object[] args) throws Throwable {

        Method m = getMethod(method);

        if (m == null) {
          //TODO you could change this if you like... 
          //maybe make it configurable?
          throw new RuntimeException("unexpected call to "+
                    "a method not in the mock: " + method);
        } else {
          String s = m.getName();
          if (args != null) {
            for (Object o : args) {
              s += ":" + xstream.toXML(o);
            }
          }

          //trim white space between xml tags as well as
          //new lines anywhere!
          s = s.replaceAll("\r", "").replace("\n", "");
          for (int i = 0; i < 10; i++) {
            char[] spaces = new char[i * 2];
            for (int j = 0; j < spaces.length; j++) {
              spaces[j] = ' ';
            }
            s = s.replaceAll(">" + new String(spaces) + 
                                                "<", "><");
          }

          //log the call...
          calls.add(s);

          //now call the mock implementation!
          return m.invoke(AbstractMock.this, args);
        }
      }
    };

    return (T) Proxy.newProxyInstance(classLoader, 
                                           interfaces, ih);
  }

  /**
   * the method with the same name/params, if it exists in
   * this class, otherwise null.
   */
  private Method getMethod(Method method) throws 
        IllegalAccessException, InvocationTargetException {

    for (Method domainObjectMethod : 
                                 getClass().getMethods()) {

      if (match(method, domainObjectMethod)) {
        return domainObjectMethod;
      }
    }

    return null;
  }

  private static boolean match(Method method1, 
                                          Method method2) {
      return namesAreEqual(method1, method2) 
	                    && typesAreEqual(method1, method2);
  }

  private static boolean namesAreEqual(Method method1, 
                                          Method method2) {
    return method2.getName().equals(method1.getName());
  }

  private static boolean typesAreEqual(Method method1, 
                                          Method method2) {
    Class[] types1 = method1.getParameterTypes();
    Class[] types2 = method2.getParameterTypes();

    if (types1.length != types2.length) {
      return false;
    }

    for (int i = 0; i < types1.length; ++i) {
      if (!types1[i].equals(types2[i])) {
        return false;
      }
    }
    return true;
  }
}

Have fun!
©2010 Ant Kutschera

Social Bookmarks :                                              

Responses[3]

Posted by Ant Kutschera on November 3, 2010 8:45:00 PM CET #