We all know about the term 'Code Monkey' which refers to junior developers... Well, I'm calling my ants 'Code Ants'!
Anyway, slight problem with them - I put hand cream on the edge of their tupperware home, so that if they tried to venture out, they would think twice and wouldn't drown. Sadly, the cream has dried out, and more and more are going over the edge, losing grip and drowning!
Lessons Learned for Project Management:
Don't allow your code ants boundaries to become unclear! Give them the slightest opportunity to get off the project, they will take it, even at the cost of their own life. Keep reinforcing their boundaries so they know their place.
At the weekend, I was in the park playing with my son and he was intrigued by the beetles and ants. So we popped home, picked up some tupperware and a spoon and went back to get some ants to have our very own ant farm! They are in a tupperware container within a bath, so that if they do climb over the edge they won't be able to escape and scare my wife out of the house. See below for what it looks like.
Anyway, its a bit like a fish tank in that its very relaxing to watch them building and feeding, etc. As I was watching them pulling a pupa from where I had tossed it out of the original tupperware container into their new home, it got me thinking. They can't see the bigger picture, yet they make a good job in getting there in the end...
For example, they were pulling at this poor thing to try and get it over a blade of grass, but couldn't. So they went left, the long way round. They should have gone right! But there was no one there to guide them along the way. In the end however, they got to where they were going. Now, having turned left, I can't say that they didn't change their plans and continue going that way, because at this stage, there was no established ant hill, so they were just looking to see where everyone else was going I guess. I don't know enough about entomological sociology to understand these simple creatures! But this morning, sure enough, there was an ant hill, and all the pupae were gone, I assume underground. So even though they had no architect, no project manager, they got there in the end, through what seems to be chaos.
I can only imagine that they did this because ants follow a small number of rules. From what I do know, they put pheromones on the ground using their tails and depending upon which pheromone, it has a different meaning. Good examples are the 'alert' pheromone - attack a nest, and you will soon see there are hundreds of ants running around. That is because the ants you attack lay this special pheromone which attracts all the others. Its not brilliant, because they don't seem to know when to retreat. Anyway, yes, back to the plot, they only have simple rules. Things like:
- if you find food, take it back to the nest. if its too big, lay a trail of 'food' phermone on the way back so the others can find it!
- if you come across a pupae, check the temperature is good, then move it if required. Check its got enough food, if not, give it some from your second stomach
- if attacked, lay 'alert' phermone to get help!
- if your'e tired, go rest, if you've put in your hours for the day
- you are a small part working for the good of the greater whole
Also, they have distinct roles. Some look after the pupae, some forage for food, some guard the nest, some attend to the queen (they will have quite a boring time in their new home, as we didn't catch any queens). So I am wondering, if you took a set of software developers and gave them some simple rules on how to get the job done, whether they could do it without the project management and the architect? Let's hypothesise:
- Use standard design patterns
- Follow coding standards
- When the whole thing screws up, scream and shout for help from everyone else
- Go home at the end of the day and sleep - no overtime, nor any coding at home!
- you are a small part working for the good of the greater whole - your'e in a team, so use that team to help you to help the entire project become successful
OK - some of these rules need an architect at the start, for example to define the patterns. But once they are defined, you could implant them into the developers DNA so that the architect isn't needed any more.
In theory, if you have a good set of experienced developers, strong regression testing built as part of the project to allow lots of refactoring and some requirements which are CLEAR and written SIMPLY, then you should be successful as the ants.
I've discussed this before and in my opinion, success is a combination of staying within budget (in terms of time and/or cost), meeting quality requirements and meeting functionality requirements. Ants have successful projects in terms of meeting their functionality requirements. The quality is sometimes lacking, for example when a tunnel collapses, but there is generally no loss of life, and the repairs are quickly done, albeit at additional cost. And the time/cost doesn't appear to be an issue, so long as there is plenty of food and water. And lets face it, what else would ants do...
Perhaps these other attributes of success (cost, time and quality) make it impossible to compare ants to software developers? Then again, perhaps what ants do is so simple that it cannot be compared to software systems in the enterprise anyway?!
Still, if you have some spare money and want to potentially throw it away (like most large companies who have software it would seem), then get in touch and we can put together a project and run it like an ant farm...
This is one of my favourite software related cartoons. Unfortunately I don't know where it came from, so I cannot give credit to the originator. Sorry!
(scroll right down to the bottom of the page, since the blog software is trying to be clever...)
Right... The contract at my current client is finally coming to an end after a few years and now I'm having to look for something else - I'm looking local, as I want to hang around. Hence, I've created this blog, and posted all my previous postings from a company internal blog, or at least the more interesting ones...
Job wise, I'm currently unsigned, and without transfer fee! So if you're managing a team and need a striker, I'm your man... Be quick though, because I have a few options on the cards. Alternatively, it will all go wrong, which means it will actually go right, because I will be able to collect on my unemployment insurance and get 80% salary! And the beach down by the lake is so lovely this time of year...
PS - now my blog is public, people who I don't know might start asking who I am and what I do, and what maxant is all about... Well, if you really really want to know, mail me on ant_at_maxant_dot_co_dot_uk!
Apparently a big off shore IT firm is desperately hiring anyone it can, to fulfil its contractual obligations and as a last resort to find resources considers hiring some cannibals. "Look, we will hire you, but we will NOT tolerate you eating any other staff members. Is that understood?" asked the HR manager to the cannibal leader. He agreed and shortly afterwards, him and his tribe get to work.
Six weeks later the HR manager learns that a techie has gone missing and runs to the cannibal chief. "What have you done?! I explicitly told you, no eating the staff! You are ALL fired!" The chief turns to his tribe and asks "Who the hell ate the programmer? We agreed to only eat the managers. We've been doing that for 6 weeks and no one noticed..."
Model Driven Architecture apparently...
Just been reading about it at http://www.omg.org/mda. The site isn't too good at giving away what this is, until you start to read a little more deeply, but it seems to me that its a development process / architecture which allows you to model your business needs in UML and automagically generate the entire application. As this is basically what Chordiant does, I thought it might be interesting for Chordiant people to read up about. Perhaps some of you have already heard about it? Apparently it's something that software vendors provide the tools for, so I guess that's what Chordiant does in that respect.
I'm wondering if the clever people at Chordiant thought of it before it became main stream, but haven't updated their software to be compliant, exactly what, in my opinion, happened with their front end when you compare it to Struts?
Anyway, if Chordiant haven't already thought of it, we could suggest that they make their framework more MDA compliant, hence making it more sellable because it complies with some OMG standards! They might like us for suggesting something like that. Alternatively, they might react like SeeBeyond did when I presented our proposed testing tool set to them - negatively with comments like "you can't do that - it shows our product has weaknesses!" I don't bother trying to help software vendors improve their products now, I just build the tools myself and use them, occasionally putting a BA badge on it (e.g. AntTailer and the Test Harness that I built in the last two years)...
Anyone into Sudoku?
Personally, I don't get on well with the normal Sudoku, but at http://sudoku.maxant.co.uk there are lots of different puzzle sizes (e.g. 2x2 and 2x3 which are dead quick to play) - and it records your times so you can challenge your mates too.
So - can anyone beat 1 minute 36 seconds for this one? http://sudoku.maxant.co.uk/index.jsp?puzzleUID=114
This week has been a bit of a fast track when it comes to training. As well as spending two days on a SAP XI (Exchange Infrastructure / EAI) training workshop, I have had some time to check out JBoss SEAM.
The SAP XI stuff relates to the Client needing to upgrade their existing EAI platform (eGate) to a newer version. As it will eventually involve recoding of all interfaces, due to no backward compatability, they are taking the opportunity to look at other platforms. And since 80% of their systems are SAP based, XI seems to the the natural and logical solution.
So the workshop I attented was aimed at gaining a high level understanding of the product. It's sold as something related to J2EE, but in my opinion isn't really related to J2EE. Sure it lives inside a J2EE app server, but as far as the developer is concerned, all they do is create a toolbox of objects (data structures, mapping rules, business processes, etc). And this is all done graphically! Then they hook it all together and configure it. OK - I haven't done anything hands on, nor seen anything complex done, but I am sure that does involve writing a bit of Java at some stage. Out of interest, mappings are done using XSLT, which is run on the data structures which are defined as XML docs.
I also spent time looking at JBoss SEAM, which is a project that combines your database, Hibernate (ORM), Ant and the JBoss IDE (in Eclipse) to mean that all you need to do to create a complete back end is the following:
1) Create a config file with database connection details
2) Click a button on a GUI to generate a project
3) Build and deploy that project to JBoss
The result is that a web app gets deployed which connects to EJBs to provide all the functionality to read/update/insert rows in the DB.
While the GUI isn't that useful (although its a basic starting point for a JSF GUI), what is useful is that the entire data layer is built. All you need to do is write some services (in an SOA) / business logic to implement the business processes to hook the datalayer together as you want.
OK - Chordiant has been doing this for years... but SEAM is free and based purely on EJB standards (3.0).
So, this week I have spent a lot of time learning technologies which remove the requirement to actually code. For a few years I have been reading that this is the way programming is going (building using wizards and then just worrying about config), but its finally coming to fruition...
I'm still sceptical about how techy you need to be to use these tools/frameworks. I'm of the opinion that as soon as an error occurs, you need to be able to get out the debugger and dig deep. I'm lucky that when I learned to code, it was using a text editor, so I have the skills to do this. Sadly lots of newbie developers (especially off shore ones!) don't have those skills. They learn to build apps using tools like this and then can't debug them or optimise them, etc.
So, if you are mentoring someone who is building systems like this, make sure you get them to spend time learning the technologies (Hibernate, JSF, XSLT, EJB, etc) using a text editor, command line compiler, command line deployer, etc. When they have to start debugging a complex problem, they will have the skills needed.
Well, its not a bank holiday in Switzerland, so while you are all having fun in the sun or spending your day in the pub as is traditional for the May bank holidays, I'm sitting here working on an interesting performance problem.
We are trying to transfer 200,000 rows of data from SAP in Lausanne to MS SQL Server in the Ukraine (1000 miles away?). Its averaging 0.5 seconds per row and so will complete in roughly 27 hours. That causes the customer a few problems because its scheduled for 3am and has to be complete by 6am... The techy implementation is to use JDBC to do an insert/update statement depending upon if the row already exists, so we are actually doing several JDBC calls per row of data.
Originally we thought that the bandwidth to the Ukraine together with a crackily/dodgy/unreliable phone line was the problem, but a quick calculation for the 1MB ADSL line that services the Ukrainian network, showed that it should be transferring in the range of several gigabytes in 27 hours, not a measly 25 megabytes...
So along came our DB expert who suggested that calling a stored proc would be better because it would involve sending less bytes of data (the call to a stored proc does not name the columns being updated) as well as be optimised by the engine. Furthermore, it could do the check to see if the row already exists, reducing the amount of calls we had to do. The theory was that using a stored proc would be about 5 times quicker. So away I went and did a test. The results were that a single insert statement was taking around 146 milliseconds (average of 100 calls) and the stored proc was taking an average of 140 milliseconds. Hmmm.... so only 6 milliseconds in it!
So I got thinking a little harder and wondered if the problem was the amount of time taken to send the data over the wire to the Ukraine because of its distance, AKA latency. I have a server in the States which I use for my wife's jewellery web site (www.dragonflyjane.com - yes another shameless plug - sorry) and it can take up to half a second longer to display a page than a site in the UK. Its because as soon as it gets to that wire that goes across the Atlantic floor, it slows right down. A quick 'trace route' using something like 'tracert www.dragonflyjane.com' and comparing it to 'tracert www.google.co.uk' (from the dos prompt) shows the difference. It gives a print out of the average time to get from each server to the next, along the route taken to get to the target server. Sure enough, the result of the trace route to the Ukraine showed an average time of around 150 milliseconds, which is almost exaclty the time we saw it taking to execute each SQL statement.
So, we decided to bunch our statements together and send them at one time. That means only a single delay of 150 milliseconds, as opposed to a delay once per row. Sure enough it worked! We haven't completed the code changes yet, but are thinking it is going to take in the region of 15 minutes to do the updates for all 200,000 rows of data.
Now, the sad part is that we only discovered these issues in production and the customer is complaining that we did a bad job. Lets look at this a little more... Firstly, the customer did sign off the implentation (through some basic UAT tests). Their UAT tests did not include any kind of serious load testing. They knew that they would see 200 or maybe 300 thousand rows of data in production, but only tested with 10% of that. Second, we warned them that the interface was slow and asked if it was acceptable. We chased them about it too, but never had a response. By the time we went to UAT we decided it was their problem as we had done all we could. Thirdly, in the specifications that were provided and signed off by the customer, the estimated amount of data that we would process in a day was not more than 50 kilobytes. The final problem here was that because this interface is running in batch and is dealing with a lot of data, it would have been better to implement it as an ETL project, not EAI. But that decision is strangely out of our hands and in the hands of the customer, for some strange political reason. And sadly the customer had never heard of ETL. There is another issue in that the SAP connector for the ETL package is something like 500,000 US dollars. And even though there has been a lot of rework using expensive consultants, we cost a lot less than 500,000 USD!
1) Get the requirements right!
2) If you notice slow transactions during unit testing, chase it up and start doing some performance testing and thinking of how to fix it. Do root cause analysis to really work out what the problem is, so that it takes less time to analyse in production.
3) If the customer is not experienced enough to do realistic testing in UAT, give them a hand by suggesting some scenarios.
4) Really get the requirements right! We were asked to build the equivalent of a car to drive from London to Edinburgh and back, a few times a day and the customer wasnt happy that he couldnt send his entire army within 24 hours!