Sunday, July 09, 2006

What’s so special about Agility?

Gosh, I was looking at all those individuals who have signed up at Hey, if you think there is a different way, add a comment, I can’t be the only lone voice – can I?

So, the Agile Manifesto is based on a dozen ‘principles’. Let’s take a look and see. I’m concerned that the Agilites somehow think they are unique in their beliefs and that there is no other way in which quality software can be produced.

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

This is the fast food philosophy to software development – clearly the Agilites can produce you something quickly however it is unlikely to be something that will be of quality, will inspire you or satisfy you beyond you immediate needs. Many customers are looking for something that will be substantial beyond their immediate short-term gratification. It is arrogant to assume that all customers are motivated and share this priority. Clearly time to market is important, but slapping out versions of half-baked software is not always the answer.

I’m reminded of Fred Brook’s preface to chapter 2 of The Mythical Man Month, actually the chapter named after the book. He quotes from the menu of restaurant Anoine, New Orleans – ‘Good cooking takes time. If you are made to wait, it is to serve you better, and to please you’.

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Have these guys never seen effective change management processes? Even ‘waterfall’ techniques can accommodate change late in development when an effective change management process is in operation. Further, the customer is given an informed choice based on an impact assessment (these are really cheap and easy to do) irrespective of the development stage. The Agilites would press ahead without any foresight – I know some customer’s who wouldn’t feel comfortable with that. Can you imagine the scenario – humm let’s try that idea out and see if it works? I’m not sure what impact it will have longer term but let’s give this a try? This is a rather speculative and potentially risky strategy at times don’t you think?

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Continuous integration is possible with an iterative architecture driven approach. I would have it no other way, daily builds once construction has started. It is just a question of when does construction start? Perhaps a little later than the Agilites would like.

4. Business people and developers must work together daily throughout the project.

Do the Agilites live in a different Universe? If the customer is paying, they’d sure like it for the business to talk to the developers. I’ve never seen this not happen – why is this exclusive to Agile development?

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Yep – done this, but not with Agile techniques. I’ve known developers wanting to defer their holidays because they could see the value of a plan based, architecture, design lead approach.

6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

True. But I hope they write it down – the mind is a fickle thing and miscommunication is easy. We can all speak the same words but mean totally different things.

7. Working software is the primary measure of progress.

Why? This is based on the premise that customer’s are not educated, or are to too stupid to understand that progress can be measured in other ways. If I’m having an extension added to my house I hardly expect the builders to turn up and start to dig the foundation and lay bricks until we’ve agreed the detailed blueprints. With an Agile approach they’d lay the foundations in the wrong place (costly to change) or build walls that would need to be knocked down. Why would a customer pay for an army of Agilites to start cutting code which will be wrong and require modification. Surely paying someone to think before that code is a far more prudent plan.

8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

What on earth does this mean? Can someone explain this principle, it seems to imply that sustainable progress is only unique to Agile approaches.

9. Continuous attention to technical excellence and good design enhances agility.

How is this possible – on the Agile projects that I have observed everyone is racing along so quickly that they don’t have time to think. Technical excellence is rarely achieved and poorly thought-out designs occur. Somehow there is an expectation that some combination of rudimentary code refactorings will salvage a poor design. This is rarely the case.

10. Simplicity--the art of maximizing the amount of work not done--is essential.

Yes I subscribe to the Ludwig Mies van der Rohe ‘less is more’ principle.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

What effective teams aren’t self-organising?

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Continue review is an excellent principle, but again this isn’t exclusive to Agile approaches.

Wednesday, July 05, 2006

SOA, Designing Web Service APIs

Service Orientation is all the rage right now and has been for a couple of years. I get a little upset by the trend of the IT industry to keep reinventing the wheel and in the process to forget everything it previously learnt. Here I would like to present some areas for consideration for the design of Web Service or distributed computing APIs.

I often get involved in discussions concerning the design of Web Service APIs. Invariably people get hung up on the implementation technology, WSDL, SOAP etc and loose sight of the underlying issues. Most people’s initial stab at a Web Service API is to think “oh, it’s like an local API call but with slower response times”. Well almost, but there is more to it than this.

Below I have identified five areas in which the design of local and remote APIs differ. I think these are useful when thinking about Web Services, what do you think? In addition I believe they can be reused in the future when Service Orientation becomes passé and is rebranded into something like Super Lightening Integration Metaphor (SLIM) or whatever …

API Consideration Areas

1. Calling Semantic

Remote API

Call by value. Parameters and return values adopt a pass by value semantics (e.g. objects are serializable)

Local API

Both call by value and reference can be supported. The later may be desirable for performance purposes and is usually desirable for local APIs.

2. Function Granularity

Remote API

Due to the communication overhead (principally latency) the API has to be designed so that the number of interactions/invocations is minimised. Typically this encourages a coarse grained API. This is also consistent with the calling semantics above.

Tens to Hundreds of invocations per second are possible. Beware if you are porting an application that used a local API previously.

Local API

Since calling is local, there is a minimal communication and invocation overhead. Millions of invocations per second are possible.

3. Security Issues

Remote API

Two aspects:
  • Authentication – is the caller who they say they are (how do we prevent client identity being spoofed)
  • Encryption – is some of the data being exchanged in each function invocation sensitive? The information may be transported via an untrusted channel. Plain text passwords are an obvious example, but there may be commercially sensitive information that also should not be passed in the clear.

Local API

With local invocation it isn’t usually necessary to worry about these issues.

4. Recovery Semantics

Remote API

The API must be designed to support recovery following communication failure or the loss of messages. This applies to both function invocation and response.

Think of a simple banking application which has a ‘Debit Bank Account Transaction’ as an example here we wish to debit £10 from a customer account we need to ensure that the transaction or function call is executed once. Technically this is called idempotency.

The message used for the invocation of the function may get lost, or the invocation message may be received successful but the return response message gets lost on the way back.

There are two strategies for dealing with this;

  • the use of a token for each unique invocation. The recipient can check to see if the request has been processed previously. See Synchronizer Token pattern (
  • the provision of a ‘query’ function. If the caller is unsure that function has been executed a request can be made to determine the state.

Local API

A local API doesn’t need to worry about these issues as communication is typically within a machine/server.

5. API State

Remote API

To support scalability and recovery typically the API is designed to be stateless. The provider of the API is not in contact directly with the caller. The caller may crash without the provider realising. For stateful APIs there are recovery strategies, but typically these revolve around the use of timeouts which increase the complexity of the API implementation and use of more resources (as these need to be kept until the timeout has expired).

Local API

With local invocation, and typically as a result of adopting call by reference semantics, a stateful API can be implemented and is often desirable for performance reasons.