1. Don’t bet against human ingenuity.   If your prediction involves the concept that “people will never be able to overcome this challenge”, you’re wrong.
2. Don’t bet against the laws of physics.   Duh.  Except that sometimes, humans manage to find loopholes.
3. Don’t misunderstand what people want.   People don’t want cars.  They want a way to get to where they want to go.   They don’t want “music from popular artists” - they want to look “with it.” - Don’t build trend predictions based on the wrong thing.
4. Don’t bet against the law of supply and demand.   It’s like arguing against evolution - it just makes you look dumb.
5. Don’t bet against long term trends - they are long term trends for a reason.  Just make sure (re #3) that you understand the actual trend.
   
6. Don’t bet against people acting in their own self interest.   People look out for themselves and their families first.   That’s not going to change.
7. Don’t confuse a technology with an application - the word is full of technologies that are impossible/impractical to use.
8. People always overestimate what can be accomplished in one year, and underestimate what can be accomplished in 5.  So be safe, and assume it will take 3 years.
9. If your prediction involves more than two critical coincidences, it’s almost certainly not going to happen.
10. Even a blind pig finds an acorn once in a while. If you make an outlandish prediction that turns out to be true, it doesn’t mean you’re smart.  It means you’re lucky.

Subtitled: The ever-changing API and the Path to Madness

Agile Software Development is (at least partially) about the fact that change happens in software projects, and, as practitioners, it is clear that resisting change seems to be more expensive and error-inducing than simply accepting it.

But the fact that this is generally true certainly doesn’t make it always true. Consider the noble API.

Imagine, if you will, that someone, let’s call them Calidon Systems, produces an API that gives you access to their cool database of information about, say, movies. Your organization writes a bunch of applications using the API, and produces some excellent work.

And then Calidon Systems abruptly change the API, in substantial and semantically meaningful ways (like, for example, replacing a web-service/SOAP based API with a REST API).

Suddenly, you’re obliged to do a whole bunch of unanticipated work trying to repair your application to use the new API.

And then, a few months later, Calidon Systems announce that they have performance issues with the REST approach, and are switching to a Message-Queue style communications model.

I don’t know about you, but if I was a customer of this API, I would be pissed as hell. If I had do, I’d grimace and make the changes (again), but a voice at the back of my head would be whispering “Never use an API from Calidon Systems again”

If your API is going to be used by more than a handful of people, you owe it to them, and to your reputation as a developer, to change it rarely, incrementally, and, as much as possible, in a backwards-compatible way.

If you’re building APIs for your internal development co-workers, this probably isn’t an issue, since you and they are both working towards the same overall goal. In that kind of environment, changing your API to deal with new features, etc, is probably a good thing. But that’s because the number of users and the people affected by the change is small.

If your API is public - if it is used by many programs, third-party developers, etc, it almost instantly becomes legacy code. Don’t refactor it every week. Don’t introduce changes to the interface calls, new objects, etc on a regular basis. Keep it extremely well documented. Do your very best to get it right the first time, and do your very best to maintain backwards compatibility for years.

Public is the key word in this whole post - once an API is public, you should consider it to be nigh-unchangeable. You do not have the luxury of fiddling with it, at least not with your reputation intact.

The best strategy I can suggest is to try a two-pass approach:

• Beta: Do your level best to build an API that satisifies a number of known customers. Leaving the “Beta” tag attached, release it to a relatively small group of people, with the understanding that it will change based on their feedback.
• Release: Once you get feedback, release a final version, and keep it as stable as you can for at least a year, preferably two or more.

This video on the subject was very helpful in reinforcing that my opinions here are not completely off base

I really, really enjoy this blog, and this video by the progenitor of the blog is a very nice introduction to the concepts.

All of us have biases that we don’t recognize.  It makes us better analysts to realize we have them, and to work on methods to compensate for them.

Subtitled:  You don’t build a nuclear weapon like you build a cabinet.

The first of my heresies - Don’t use agile techniques to build mission critical apps.

One commenter replied “Why not do agile with a zero defect mindset.”

It’s a fair response, if every defect could be found through “typical” testing.

But I’ve done enough mission critical application development, and read enough about others doing it, that I know that the problem is that there are incredibly subtle bugs, that are nearly impossible to discover, through any amount of brute-force testing.   Bugs with timing, with unusual states, with unusual inputs, with combinations of all three.

If you’re going to be building something where people’s lives depend on its proper functioning, you owe it to them (and to your internal customers) to go down every path, inspect every variable, bullet-proof every pathway.  And the most effective way to do this, is, unfortunately, by designing the logic up front - focusing a lot of attention and care with a number of brains working in parallel on exactly how to build certain subsystems and various elements of the system.    Every change, every tweak, everything needs to be inspected and approved and debated before it is ever written.

Having said this - this is an incredibly expensive way to build software.   And 99.9% (if not more) of software, including a lot of stuff that is important and valuable, doesn’t need to be written this way.

And to anticipate the natural objection - yes, I know that bugs still happen, even with this approach.   It’s just the best way that I believe one can build software that is so important that it must get done right the first time, where cost and time are not the primary concern.

Having said that, I believe that one can isolate those software components, and build the rest of the system (the “mundane” 99.9+%) using agile methods.   I believe that you can also build the mission-critical components using a TDD approach for your own benefit, that you can build it in stages, and provide agile-style visibility of your progress as you go.

You just can’t invent it as you go along.

I look forward to your comments.

My opinions:

When to be agile

• When you’re dealing with constantly changing requirements
• When there’s a lot of uncertainty about how to build the system
• When there’s a lot of variability - priority changes, multiple projects, etc
• When you want the team to “gel” quickly
• When you want the team to take ownership of their projects and tasks
• Pretty much all the time except the following:

When not to be (too) agile:

• When you’re building “infrastructure”
  • Protocols/APIs
  • Frameworks/Platforms
•  When your client expects and is paying for a thorough analysis and extensive documentation
• When you’re building something “mission critical”

I’ll address these in more detail in follow-up posts.

• A Method for Madness, by David Gerrold. The previous book came out in 1994
• A Dance with Dragons, by George R.R. Martin. The previous book came out in Oct, 2005.

What else have people been waiting a long time for?

*Ack, I can’t believed I typed “fore” instead of “for”.*

From the New York Times:

Until I talked to Nick Bostrom, a philosopher at Oxford University, it never occurred to me that our universe might be somebody else’s hobby. I hadn’t imagined that the omniscient, omnipotent creator of the heavens and earth could be an advanced version of a guy who spends his weekends building model railroads or overseeing video-game worlds like the Sims.

Moo haahahahaha

Now that everyone at work knows, I can tell my loyal readers (all 2 of you) that I have taken a new job, as the CTO of the Sunlight Foundation. It’s a non-profit, based in DC, that focuses on using technology and the Internet to spotlight waste, fraud, corruption and “questionable” behavior in Congress.

I start in September. Half of my family is nervously excited about the move, the other half is nervously anxious.

For some reason, the wp_comments table has disappeared. I may have lost all of my comments

Hopefully, I’ll be able to get new comments to start appearing, however.

*Update* - The MySQL web front-end “Repair” button worked wonders, once I figured out which database the blog was connected to.

Courtesy of 800-CEO Read  (who I normally find quite solid)

I don’t mind, so much, when people say “Don’t pay off your house, invest that money in the stock market”.  I don’t agree, but I don’t mind.

But when you combine that with this bizarre BS “double asset” accounting, I get seriously frustrated.

Let’s review:

• The house is not your asset.   It is the bank’s asset until you buy it.   How do I know?  Try selling it.
• Even if you do consider it your asset, if you don’t pay it off, you have a serious liability to go along with it.  You can’t just wave your hands and pretend that the liability doesn’t exist.
• When you have a large mortgage, that puts iron clamps on a significant chunk of your income every month - money that you have to pay to the bank.  This has the following effects:
  • It is harder for you to move, because you have to sell your house first
  • It creates an income floor - you simply cannot afford to not earn less than X per year
  • A run of bad luck on your investments, and you have no fallback plan, because you have no equity to draw from.

Essentially, this is the doctrine of “profit over flexibility”.   But do you really want to tie yourself down like that, create incredibly stressful obligations that leave you with significantly fewer opportunities to experiment, to explore, to take time away from work?

Welcome to the August 3rd Carnival of the Agilists. Today’s theme is: I’m running late.

Ideas

• Do it *bleeping* now - Jeff Atwood
• Why is my project late?  - Jack Vinson
• Analysis and discussion of the Ten Undeniable Truths of Project Management - Phillip McGuin
  
• George Dinwiddie (who wisely submitted via email) on “Running Interference” for your project.

Discussions

• Evan Hoff wants to know your thoughts on Agile Development, Six Sigma and CMM.  Evan also wisely used the email address to submit his article.
• Marc Andressen thinks Group Brainstorming is a Bad Idea.
• Never one to shy away from a good scrap, Glen Alleman says (amongst other things) that Waterfall is already dead.

Resources

• Applying Agile to Ruby - a video lecture/presentation by Fred George.

Books

• Managing Humans: Biting and Humorous Tales of a Software Engineering Manager - looks like a fun read.
• This was released in February, but I hadn’t seen it before:  Rails for Java Developers
• Another Martin Fowler series book: Continuous Integration

Submissions

Want to be included in the carnival? You’re more likely to get noticed if you send a link to your posts to agilists.carnival@gmail.com.

Add your Voice

• Think we’re a bunch of yahoos, and you could do this whole Carnival thing a thousand times better? Let us know, and you’ll get the chance to produce your own edition of the Carnival.