What’s so special about Agility?

Gosh, I was looking at all those individuals who have signed up at http://agilemanifesto.org/sign/display.cgi. Hey, if you think there is a different way, add a comment, I can’t be the only lone voice – can I?

So, the Agile Manifesto is based on a dozen ‘principles’. Let’s take a look and see. I’m concerned that the Agilites somehow think they are unique in their beliefs and that there is no other way in which quality software can be produced.

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This is the fast food philosophy to software development – clearly the Agilites can produce you something quickly however it is unlikely to be something that will be of quality, will inspire you or satisfy you beyond you immediate needs. Many customers are looking for something that will be substantial beyond their immediate short-term gratification. It is arrogant to assume that all customers are motivated and share this priority. Clearly time to market is important, but slapping out versions of half-baked software is not always the answer.

I’m reminded of Fred Brook’s preface to chapter 2 of The Mythical Man Month, actually the chapter named after the book. He quotes from the menu of restaurant Anoine, New Orleans – ‘Good cooking takes time. If you are made to wait, it is to serve you better, and to please you’.

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Have these guys never seen effective change management processes? Even ‘waterfall’ techniques can accommodate change late in development when an effective change management process is in operation. Further, the customer is given an informed choice based on an impact assessment (these are really cheap and easy to do) irrespective of the development stage. The Agilites would press ahead without any foresight – I know some customer’s who wouldn’t feel comfortable with that. Can you imagine the scenario – humm let’s try that idea out and see if it works? I’m not sure what impact it will have longer term but let’s give this a try? This is a rather speculative and potentially risky strategy at times don’t you think?

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Continuous integration is possible with an iterative architecture driven approach. I would have it no other way, daily builds once construction has started. It is just a question of when does construction start? Perhaps a little later than the Agilites would like.

4. Business people and developers must work together daily throughout the project.

Do the Agilites live in a different Universe? If the customer is paying, they’d sure like it for the business to talk to the developers. I’ve never seen this not happen – why is this exclusive to Agile development?

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Yep – done this, but not with Agile techniques. I’ve known developers wanting to defer their holidays because they could see the value of a plan based, architecture, design lead approach.

6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

True. But I hope they write it down – the mind is a fickle thing and miscommunication is easy. We can all speak the same words but mean totally different things.

7. Working software is the primary measure of progress.

Why? This is based on the premise that customer’s are not educated, or are to too stupid to understand that progress can be measured in other ways. If I’m having an extension added to my house I hardly expect the builders to turn up and start to dig the foundation and lay bricks until we’ve agreed the detailed blueprints. With an Agile approach they’d lay the foundations in the wrong place (costly to change) or build walls that would need to be knocked down. Why would a customer pay for an army of Agilites to start cutting code which will be wrong and require modification. Surely paying someone to think before that code is a far more prudent plan.

8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

What on earth does this mean? Can someone explain this principle, it seems to imply that sustainable progress is only unique to Agile approaches.

9. Continuous attention to technical excellence and good design enhances agility.

How is this possible – on the Agile projects that I have observed everyone is racing along so quickly that they don’t have time to think. Technical excellence is rarely achieved and poorly thought-out designs occur. Somehow there is an expectation that some combination of rudimentary code refactorings will salvage a poor design. This is rarely the case.

10. Simplicity--the art of maximizing the amount of work not done--is essential.

Yes I subscribe to the Ludwig Mies van der Rohe ‘less is more’ principle.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

What effective teams aren’t self-organising?

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Continue review is an excellent principle, but again this isn’t exclusive to Agile approaches.

SOA, Designing Web Service APIs

Service Orientation is all the rage right now and has been for a couple of years. I get a little upset by the trend of the IT industry to keep reinventing the wheel and in the process to forget everything it previously learnt. Here I would like to present some areas for consideration for the design of Web Service or distributed computing APIs.

I often get involved in discussions concerning the design of Web Service APIs. Invariably people get hung up on the implementation technology, WSDL, SOAP etc and loose sight of the underlying issues. Most people’s initial stab at a Web Service API is to think “oh, it’s like an local API call but with slower response times”. Well almost, but there is more to it than this.

Below I have identified five areas in which the design of local and remote APIs differ. I think these are useful when thinking about Web Services, what do you think? In addition I believe they can be reused in the future when Service Orientation becomes passé and is rebranded into something like Super Lightening Integration Metaphor (SLIM) or whatever …

API Consideration Areas

1. Calling Semantic

Remote API

Call by value. Parameters and return values adopt a pass by value semantics (e.g. objects are serializable)

Local API

Both call by value and reference can be supported. The later may be desirable for performance purposes and is usually desirable for local APIs.

2. Function Granularity

Remote API

Due to the communication overhead (principally latency) the API has to be designed so that the number of interactions/invocations is minimised. Typically this encourages a coarse grained API. This is also consistent with the calling semantics above.

Tens to Hundreds of invocations per second are possible. Beware if you are porting an application that used a local API previously.

Local API

Since calling is local, there is a minimal communication and invocation overhead. Millions of invocations per second are possible.

3. Security Issues

Remote API

Two aspects:

• Authentication – is the caller who they say they are (how do we prevent client identity being spoofed)
• Encryption – is some of the data being exchanged in each function invocation sensitive? The information may be transported via an untrusted channel. Plain text passwords are an obvious example, but there may be commercially sensitive information that also should not be passed in the clear.

Local API

With local invocation it isn’t usually necessary to worry about these issues.

4. Recovery Semantics

Remote API

The API must be designed to support recovery following communication failure or the loss of messages. This applies to both function invocation and response.

Think of a simple banking application which has a ‘Debit Bank Account Transaction’ as an example here we wish to debit £10 from a customer account we need to ensure that the transaction or function call is executed once. Technically this is called idempotency.

The message used for the invocation of the function may get lost, or the invocation message may be received successful but the return response message gets lost on the way back.

There are two strategies for dealing with this;

• the use of a token for each unique invocation. The recipient can check to see if the request has been processed previously. See Synchronizer Token pattern (http://www.refactoring.com/catalog/introduceSynchronizerToken.html)
• the provision of a ‘query’ function. If the caller is unsure that function has been executed a request can be made to determine the state.

Local API

A local API doesn’t need to worry about these issues as communication is typically within a machine/server.

5. API State

Remote API

To support scalability and recovery typically the API is designed to be stateless. The provider of the API is not in contact directly with the caller. The caller may crash without the provider realising. For stateful APIs there are recovery strategies, but typically these revolve around the use of timeouts which increase the complexity of the API implementation and use of more resources (as these need to be kept until the timeout has expired).

Local API

With local invocation, and typically as a result of adopting call by reference semantics, a stateful API can be implemented and is often desirable for performance reasons.

Beware of the Agilites

The development of quality engineered software to time and budget is hard. In recent years the hub-hub of agile development methods has increased to deafening levels. There are agile development processes, agile modelling and even agile UP. Many people embrace the agile approach on blind faith. The writing in this area is seductively persuasive, but few readers take a moment to reflect on what is actually being said.

The underlying theme of the Agilites (the advocates of agile methods and approaches) is to pooh-pooh ‘waterfall’ development processes and to claim that there is only one approach, the agile one, to the application of iterative development techniques. We all accept that a pure waterfall development is not ideal for the development of large-scale software, however they reject so much by dismissing it completely, this is like throwing out the baby with the bath water. Many of the Agilites base their views on not understanding large-scale development, being poor practitioners of it or having experienced it being done poorly. Iterative development can be conducted in a non-Scrum, non-XP, non-FDD manner without being a pure ‘waterfall’.

Agile projects do fail just as often as ‘waterfall’ ones, I’ve experienced all kinds and seen both fail. It has nothing to do with the approach and all to do with the experience of the practitioners.

You can see the misunderstands of the Agilites by examining the Agile manifesto.

We are uncovering better ways of developing software by doing it and helping others do it.

Through this work we have come to value:

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Let me take each of these points in turn.

• Point 1. As the old adage goes ‘a fool with a tool is still a fool’. The application of a development process and the appropriate use of tools requires skill and experience. A development process is only as good as the people (no fools please) and an endeavour of any complexity requires interactions. No development process espouses the absence of individuals interacting. Many neo-Agilites reading this value interpret this as a rejection of process and tools. They roll their eyes at the merest mention of the dreaded ‘P’ word and heaven forbid if you suggest that capturing a UML diagram in a tool can provide anything other than an inconvenience. I’ve applied both processes and tools very successfully thank-you.

• Point 2. This value is based on the premise that there is no benefit to be gained in producing artefacts other than software. The word ‘comprehensive’ is used in an almost disparaging way. All documentation should be produced for a purpose. We’ve all seen people/projects who go through the motions producing documentation that serves no purpose. This is not a problem with documentation per-say. Who can dispute the creation of a succinct domain model that provides insight into a business problem? These things cannot be uncovered by hacking software or if they can the problem domain must be trivially simple.

• Point 3. Can’t really dispute this point. However, I would say there are few of us who are in the luxurious position where commercial accountability does not come to bear. We all have to produce software on time, to quality and on budget.

• Point 4. This completely mis-understands the purpose of a plan and the art of project management. This is more a problem with poor project management (managers) not plans. A plan isn’t, as implied here, something that is created at the offset and blindly followed. It is a valuable tool that enables progress to be tracked, dependencies to be managed, changes to be easily assessed and is continually refined and revised (dare I say agilely!?). A project that follows a plan is not to the exclusion of accommodating change, far from it. It is only poor project plan practitioners who create a rigid plan and blindly stick to it. A plan should be revised daily (and not to just reflect progress).

In my next entry I will discuss how an architectural driven approach can be used to drive a plan based project which is adaptable to change and incorporates iterative techniques. It embraces the pragmatic application of precision and formalism whilst rejecting the ‘dash to code’ and ‘random walk’ approach to development software proffered by the Agilites.