The Kitchen in the Zoo - philosophy tag

Houses of Cards

Sat, 10 Oct 2009 12:12:00 GMT

I have often heard of software systems being compared to a house of cards, meaning that they were poorly built and are ready to topple at any time. The system I am currently helping to maintain has from time to time also been labeled in that way, yet it always manages to go live with quite a few new functions, and apart from the odd all-nighter to fix a few last bugs, it works to the customers satisfaction.

So while I was doing the washing up today, I noticed something. I am well known amongst family and ex-house mates for building what look like unstable piles of washed up dishes, as they dry. Yet I have never ever had a pile collapse on me. Never once have I lost it all. And remember, there are slippery suds involved in holding up these piles!

As I thought a little more about it, it didn't take long to come to the simple conclusion that a pile of washing up is stronger than a house of cards. Cards are uniformly shaped and have no edges or surfaces which help to lock their neighbours in place. Plates, dishes, cups, pans and cutlery on the other hand can be placed tactically so that they lock together forming a strong structure. Notice in the picture below that the heavy pan is placed at the top!

It still looks ugly though. But as long as its just left to dry it will be safe. Just like our software that is left out in production where it runs and runs and delivers all the time.

Where my piles of washing up do go wrong, are when my inexperienced wife comes along and tries to take it down to put the dishes away. It has been known to collapse at such times (although luckily never spectacularly badly). Similarly, software needs careful hands and experience of how it was built, when taking it down and restructuring it.

Perhaps also relevant to software, is why I build such dodgy looking stacks of dishes. Well I find it the most efficient way. I simply start piling up in the order that the dishes come out of the bowl (please note that my software is designed and planned properly, unlike my dish washing!). What ever comes out of the bowl at that time, finds a good place to be anchored in. Pans get done last, since they are dirtiest and hardest to clean. Just like in software the hardest parts get fixed last because they take the longest to get right. They are put to the side with some water in them to soak away at the hardest parts. And when you are working to a deadline, you don't always have the time to rearrange your pile of dishes and fit those hard parts in, where they would look safe. You just build them in where they fit best. It looks unstable, but you built it and you know the history of why you did it that way. As long as you are involved in the next phase of the project or maintenance, everything will be fine (if you are not, it doesn't matter - see here). It's not ideal, but the software was delivered on time, on budget and it works. And just like with the washing up you don't have to spend your Saturday afternoon in the kitchen.

Copyright (c) 2009 Ant Kutschera

Social Bookmarks :

Creativity - Yet another driver towards agile processes

Mon, 28 Sep 2009 19:51:00 GMT

Software developers are often creative people, having entered into software development precisely because of its need for creativity during implementation. While solving a logical problem might sound unrelated to creativity, the ability to solve that problem in many different ways and choosing the right way, is what requires the creativity. This is what turns a mundane data entry job into an exciting and fun job.

Not only are there many different ways to implement algorithms or user interfaces, so that they provide the same inputs and outputs as required, there are many different ways depending upon the architectural or design view points taken. For example, should the implementation use as much reusable code as possible, or should the code be as simple, readable and hence maintainable as possible? Or is breaking architectural policy allowed, in order to ensure maintainability, productivity or other aspects, etc...

So if programmers, designers and architects are so creative, is it a problem? Well it can be, when requirements specifications are rigid. Developers need a certain amount of leeway in order to feel that they are being creative whilst doing their job. If a specification goes into exact detail about how to implement something (which a requirements specification shouldn't actually need to do, unless the requirement is defining the inputs or outputs of the system), then a developer can feel there is no need for creativity, and the fun part of their job disappears quickly. Once that happens, developer motiviation becomes reduced and project success starts to suffer. Indeed a measure of software success is the level of developer happiness at the end, because you need these people to be motivated for the next phase or project.

The alternative to rigid requirements is a more agile approach. If the customer* can work with the development team in an agile manner, defining the outcome of the software together and allowing the development team to provide input on say usability aspects, or technological options, then everyone is a winner. The developers feel creative and the customer gets something which he can definitely use and definitely wants, because the agile process is an iterative one with short cycles between feedback and response (delivery of an updated version of the system). Often when writing rigid specifications, they suffer from "specification rot" (exactly the same thing as code rot, which is when code quality reduces over successive iterations of changes because new changes fail to take old ones into consideration). Basically said, the specifications become incomplete, containing conflicting requirements and illogical portions. Close and agile work between customer and development team allows rigid specifications to become living documents which can be adapted and perfected.

An advantage to the customer is also that they may become aware of other possible solutions to their problem. For example there may be a cooler way to make the user interface interact with the users. "Cool" can be a very important metric when it comes to web site design. Don't believe me? Take a look at iPhone sales related to the "cool" factor. Developers may also offer better or more efficient ways to solving the customers problems.

Of course, there are two sides to any coin, so agile processes must have disadvantages. Here are a few:

    - Scope Creep - the customer realises quickly with each delivery that they need something other than what they have specified. But I have experienced plenty of projects where the customer doesn't realise that, until six months or a year down the line and by then all of their budget has been spent and it's too late to fix the problem. At least in an agile process, the customer specifies through priorities, what is most important to them within their fixed budget, and that priority list is allowed to change as the customer discovers the system.

    - Rigid release cycles - large corporations whose software is all interlinked may use say two fixed releases a year in order to synchronise releases and versions of software in order to reduce the complexity of running multiple versions of the same software at the same time. While such strategies do indeed make sense, there is no reason why individual projects may not be run lean and agile. The customer can certainly be given access to their software frequently as it is being built. The excuse that the software is not ready because interfaces to other systems in the big release cycle are not ready does not fly, because otherwise the software could not be built until those systems are completed (using stubs, mocks or similar are the solution here).

    - Reluctance to change - many customers who have aquired software in the past only know how to do it using rigid and old fashion methods. It is quite human to dis-like change and go with what you know. But I encourage you to try going agile on your next project, even if it is only to prove me wrong, that agile projects are more successful than rigid ones.

    - Inexperienced customers - to be a useful and good customer on an agile project requires that you are open to change and suggestion. Many customers create a picture in their mind of what they think they want and define that in a rigid way. They are afraid that if their rigid requirements are not implemented that the software will be wrong in the end. Well here's the news: it will be anyway! There are very few individuals out there who a) know what they really need without going through an iterative cycle of trying it and then improving it, and b) who even if they do indeed know what they really need, can communicate it so that it gets implemented that way. So instead of going through the disappointment of only playing with it when its finished, try playing with it as it comes to life, just like a new home owner inspects their house as it is being built, or an engineer tests their mechanics as they are built. In aeronautical engineering, the "customer" is generally considered to be the company designing the new plane, rather than airlines who purchase the planes. No aeronautical engineering outfit has ever built the entire plane before testing that the waste disposal systems worked as required! Indeed such systems are put through acceptance tests before they are even integrated into the airplane.

Out of these, the two important ones are inexperienced customers and scope creep / change. Prioritising EVERY TASK on the project is the key to keeping the project on track. Experienced customers know that arguing over the layout of the UI down to the last pixel while important, may be costly. They can decide if it is more important to the project (note: not them as individuals) where the priorities lie. Experienced agile developers know this too and together the customer and developers can ensure a successful project.

So, as the world moves on, there are still many non-agile and rigid projects happening throughout the world. If you are in charge of software requisition and have some budget to start a new project, why not try making the project an agile one, if only because the creativity involved makes it a more fun project!

*
Let's quickly define a "customer". In the sense of software development and aquisition, this group of people are the stake holders who are paying for the work to be done. They are not the users although should be part of that group in order to be able to specify what is really needed. In a world where a software product is to be built which will be sold to users, for example like what Microsoft's core business does, the customer here is not the end customer, but rather the group of people who decide what should be sold in the first place. The customer group typically includes business types who are in charge of the money and strategy, as well as people representing the user community.

Copyright (c) 2009 Ant Kutschera

Social Bookmarks :

NEWS FLASH: Agile does not mean there is no plan

Sun, 20 Sep 2009 19:53:00 GMT

Why aren't all projects run using agile methods? Working in an enterprise where we have maybe 50 applications which all go live at the same time twice a year (as well as another 450 which are also in some stage of the software lifecycle), I was told that it would not be possible to plan such a release without tight control over the planning, and frankly agile methods do not cater for such planning...

Erm... I see it a little different. Agile to me means that there is a plan and you know the maximum amount of money that you can to spend. To me it also means there is a fixed deadline. As quality is also fixed (come on, it has to be bug free!), it is scope which changes. Or admittedly to some degree also quality, as you decide that some bugs are tolerable and you can live with them.

What is different about agile projects, is that the project plan and scope (functionality) are alive. They are dynamic. They change from day to day as everyone on the project learns and discovers. Just like your day to day plans change as events unfold.

It should be illegal for a project manager to state that a project plan is fixed. It shouldn't be allowed that the project manager, upon discovering that a plan needs to change, states that it will not be changed because it has already been communicated to the customer. This view is certainly un-agile. But it is also simply wrong! It is like saying "I have stated my intentions to walk to the moon by Christmas, and I shall do so, regardless of whether it is possible". In my world, it is very acceptable to admit your plan is wrong, and to re-state your intentions, especially if the deadline doesn't actually change ("I shall now fly there instead!").

Transparency to the customer, that you have discovered something that you did not put in your original plan, shows the customer that he can have faith in you, for being able to be honest about it and to re-plan, being able to handle such events.

The customer must also be able to tolerate such unplanned events, and doing so shows that the customer is mature enough to understand that this happens in software development. It also happens through out life, in all kinds or projects. When was the last time that a building was built on time? Planning is done wrong in all walks of life, and it is perfectly natural to re-plan. We should not be scared of admitting that our plans are wrong and we should certainly not be working in environments where one feels the need to hide such facts.

Copyright (c) 2009 Ant Kutschera

Social Bookmarks :

The Key to Maintenance is Compartmentalisation

Sun, 21 Jun 2009 19:16:00 GMT

Over the last couple of large projects I have noticed a trend in the way software developers/engineers/architects like to come out with statements like "that code is shit". I'm not just talking about me saying it, or how they react to the code that I write (come on, I write great code!). But general sweeping statements like this crop up all the time on these projects, related to everyones code. And managers then perpetuate these statements.

I think the reason is easy to understand. When a developer takes over responsibility for some code they want to make it clear that any problems with that code are unrelated to them. They don't want to take responsibility for the code which someone else has written. Because writing code is inherently creative and the same functionality can be written many different ways, you are almost guaranteed that a developer taking over some code "would have written it differently".

What I have also noticed is that the developers coming out with statements like this do not know or do not want to understand the conditions under which the code was originally written. As an architect, I realise there are many many factors which influence the creation of code, budget being a major one. What some developers also do not fully appreciate is that the perfect implementation not only doesn't exist (because every developer would do it differently) but that it often isn't required. The perfect solution considers many cases which don't need to be programmed, as they "may" be required in the future, but currently are not. From a financial point of view it is possible to determine whether or not to implement functionality in software (see my previous posting). So from that basis, a developer should work to a simple 80/20 rule and implement something that will suffice, as opposed to something that will exist for the next quarter of a century, unchanged because of its greatness.

Often developers which make statements that the code they are inheriting is poor, want to refactor it. They give reasons that the enlightened manager/team-leader/developer understands that refactoring is part and parcel of modern coding. Or perhaps they give reasons like without refactoring or rewriting, the code is unmaintainable. But is that really true? Does experience show that once a project has gone into production that there are many bugs or many change requests? During maintenance what proportion of code will ever actually be changed because of bugs or change requests? The project I am working on now has some code it in which we have not touched in 18 months (since going into production). The code works, although today we do not fully understand that code, but importantly we have its source code! This code is without a doubt poor. It has relatively little documentation. We have a list of all changes that are likely to come in the next years (perhaps 5 years ahead), none of which a related to this code. The intended lifespan of the code is at least another 20 years. After 18 months in production there have been zero reported bugs related to this code. So should we refactor in order to be able to optimise the code and make it a nicer design so that we understand it today? I am of the opinion that we should not. Doing so would not add any value.

Indeed does a developer need to fully understand the code in order to be capable of maintaining it? I am of the opinion (and please feel free to disagree) that a good developer doing maintenance should be capable of maintaining code that they currently do not fully understand and which has been implemented using different styles/patterns/techniques than they would. In fact I would go as far as saying it is imperative for a maintenance developer to be able to do this. The reason is simple: to keep maintenance budget to a minimum. At the time of going to production, you almost certainly do not know the areas of code which will need maintaining due to bugs. You may be aware of future change requests or further project phases where changes will be made. The result is that a maintenance developer should not start refactoring code just to make it "more correct", unless that developer intends to support the software for the rest of its life. If that is not the case (and in my experience most developers move on to different projects within at least three years), then changing the code is useless as the next maintenance developer to come along will simply claim that the code is unmaintainable and poorly implemented and want to start the refactoring all over again.

The skill of being able to inherit code and appreciating that at that time you do not need to fully understand the code is in my opinion extremely important for reducing maintenance costs. The key is "compartmentalisation". A typical example in maintenance is that you will often come across code which is duplicated. The gut reaction is to refactor it because a "law" of OO is to have code reuse. How often though does it happen that during the refactoring you realise that getting the common code out is painful and not optimal? Often. You might even end up having to build in a work around (guaranteed to cause the next maintenance developer to claim you wrote crap code). And certainly, such refactoring may introduce new bugs, requiring it to be fully regression tested.

The solution then, is to "compartmentalise" the code. The maintenance developer should become at ease with the situation and make a mental note (or even in a erm... document) that this situation exists. Without knowing whether this code has a problem with bugs and without being able to anticipate any changes which may come, you cannot guarantee that the changes you make during refactoring will be optimal. However, by understanding such code and having several related "compartments" you can plan correct refactoring, should a bug or change come your way which affects that code.

What I am about to write now is controversial, but please read it with an open mind. If you make the assumption that your maintenance developers can compartmentalise and live with code duplication, then during the project phase where you implement your code, you can use compartmentalisation to your advantage: parallel development. If you have two developers implementing similar things and you get them to work together to create a single design, it will cost you more money. Firstly, because of the communication required (the positive communication as well as the negative whereby they disagree and spend time comprimising). Secondly because while one does the implementation of the common code, the other will be potentially idle. Thirdly, a common design may not be an optimum because of comprimise in order to make it fit both/all situations.

Hey, if worst comes to worst, at least if a bug turns up in on half of some duplicated code, the users will only see it once and not in every instance of some commonly used core code. There is a rumour in our office that aeronautical systems are developed parallel in duplicate by two teams who do not communicate. This ensures that critical systems can continue to run if a bug crops up in one of them... Of course you would actually need three such systems in order to be able to determine which one is actually displaying the bug.

Anyway, if you have got this far without claiming I am a nut case then congratulations for having an open mind. However please realise I am not advocating code duplication for the sake of it. As a designer or architect you should be in a position to determine where code duplication might occur and to plan it sufficiently correctly so that your team can work efficiently and use common components as required. But if your plan cannot efficiently include such a common component, or the complexity of it means that at the planning stage you are not able to determine the usefulness of common code, then don't get caught up in the "code reuse is imperative" mantra that OO brought us. When OO was invented, the idea of code reuse was first at the method level - OO empowered programmers to reuse functions very easily. Making a law that code duplication is illegal simpy constrains management/architects/designers from using their brain to consider the options (and be creative, which after all puts the fun back into software development).

Copyright (c) 2009 Ant Kutschera

Social Bookmarks :

Complexity makes for exponential maintenance costs?!

Sat, 20 Sep 2008 20:31:00 GMT

The title of this article isn't a fact, just a thought. However the thought comes from the experience of putting a very complex application into production and watching what happens. So long as there is stuff to do on the application, e.g. further releases adding more functionality, the maintenance costs are minimal, because you can plan the people who do the maintenance to do spend say 20% of their week on maintenance tasks while spending the rest of their time on building the new functionality.

But what happens when there is no more new functionality to build? What happens if suddenly there is no budget for new functionality? Well, you normally scale down the team, reducing head count to the minimum required to keep the knowledge about the application. You will probably put them part-time on other projects.

But now it gets hard. If the application is very complex or uses many differing technologies, it might not be possible to efficiently (in terms of reducing knowledge loss) reduce the size of the team below a specific size. If you application uses say a BPEL Engine, an Oracle Database, a Java Application Server, has a web front end and connects to several external business partners requiring knowledge of complex business rules, you probably already have the requirement to keep more than five people for maintenance. Mostly, this doesn't cause a problem, because those people can happily staff other projects. But if the application is so complicated that they are certain to forget parts of the system if they start working elsewhere, then you face a problem - knowledge loss is guaranteed if these people are not kept on the project for 100% of their time. Saying that, it's not even that simple, because the people might start forgetting parts of the system if they are not actively using / updateing / maintaining them. Just because you are on a project doesn't mean you are not forgetting how it works.

What can you do if you have such a complex project? First of all, from the start of development, ensure it is well documented, has good unit and regression tests. These will help reduce maintenance costs because it helps remind developers of the pitfalls in the system and checks that any changes they make don't introduce further bugs. But we know that we should do this on all projects anyway right? What else can you do? Well, the example I am thinking of used to have 50 people working on it. That was cut down to around 10, of which 5 were developers. If none of the developers are happy to work on anything else for more than 50% of the time because they would lose knowledge of the system, if you include server costs, an application manager and a few other comparatively small costs, you soon come to a budget of a million dollars a year just to keep your application in production. That cost is not related to number of bugs or number of technology updates (e.g. upgrading the application server in line with company policy to keep up with supported versions). That cost is just to keep the application running and have a safety net, in case a bug crops up (although it does of course include the cost to fix such bugs). Oh, and that price assumes there are other projects for your staff to go to - which might not be the case, if for example there is no budget, or if other projects are only interested in resources who are at least 75% available... If these problems apply, you suddenly need to double your maintenance budget because you have to keep these people on board 100% of their time!

Other options... Well if you compare this phase of a project to an earlier one where you have your staff doing 20% maintenance and 80% development. In this early project phase, maintenance time is optimum because the people are still working with the system and have all its nuances at the forefront of their minds. In the later phase (call it the maintenance phase where no new functionality is delivered), maintenance at 20% might actually cost the equivalent of 30% in the early phase, since the people are idling (working elsewhere) for 50% of their time and forgetting how the system works. So of their 50% availability, they only have 20% available (one day a week) in which to potentially build new functionality, should there be such requirements. Compare that to the earlier phase, a developer has 80% of their time to devote to new functionality (4 days a week) - they are four times as productive. But they are there for 100% of the time, so its costing you double compared to the later phase. In total, that makes them TWICE as efficient in terms of what is built per dollar!

Another problem with having staff working for only 50% on maintenance tasks is that they will get bored and want to move on. Bad morale is never a good thing.

Another option is to just ramp down regardless of knowledge loss and save money up front. That saved money can be put in a pot and be used to pay for any ramp up time needed if and when bugs or requirements for new functionality come your way (although it is unlikely that it will remain in that pot, since other more active projects are likely to use it). As a techy I would shy away from this approach, because experience has shown that ramping up a project at a later time almost guarantees that the newly built stuff will be badly integrated making the entire thing less maintainable, ensuring that maintenance costs in the future will be increased. Still it's your project, your choice. Since software decisions are often based on political decisions these days, you probably don't care if the project fails during a later maintenance stage, because it was successful while you were there (after all, you saved loads of money on maintenance back then!), and the person who took over after you left, must have screwed up because it now costs more to maintain or upgrade than when you were there. That sounds like good cause for another promotion for you! Just start praying now, that there are no bugs or change requests between the start of the maintenance phase where you slash the head count and the time you are no longer responsible for the project...

Side Note: The last paragraph reminds me of a project from some years back, where a project manager totally disregarded software quality and maintainability in order to get the first phase of the project (in fact entire platform) into production as fast as possible. He turned the whole project into a high profile one and span it all in his favour so that he delivered a great new platform on time and under budget guaranteeing his promotion into a role well above the previous one. By the time anyone realised that the platform was very expensive to maintain he was well out of sight and the person who took over spent a large proportion of their time defending the platform and trying to improve it. But from that place far above where seagulls fly, it was the new guy who looked bad, not the one who delivered a high profile project successfully... (note the reference to why directors are like seagulls, because they go around with their heads in the clouds crapping on the people below them).

Anyhow... it appears from the discussion above that there are three options when it comes to deciding how to put a very complex project into a maintenance phase.

Do it traditionally, which is inefficient in terms of developers time
Cut all staff to a bear minimum, and pray you are long gone before the crap hits the fan
Don't do it at all! Keep developing, and justify it by showing that development is twice as efficient in doing so...

OK, well that last option does depend upon there actually being something to do. But most enterprise applications that I have worked on always went into the maintenance phase before they were completely finished. And of course more seriously, any software should always show a return on its investment - you can't just keep developing to the point that you are streaming live TV through your application because there was nothing else left to do! But I guess the important point is this: delay your maintenance phase until the ABSOLUTE last possible moment. If you intend on putting a project into a maintenance phase and then developing some other functionality in a year or twos time, it will be far more successful if you replan, keep your team running at 100% and bring the budgets for the new functionality forwards.

Social Bookmarks :