Glen Alleman says yes, I say absolutely, positively not.

First, his key points:

• Everything interacts with everything else - a project is a “system.” All the elements of the project - cost, schedule, scope, risk, resources, etc. - all interact with each other in know and possibly unknown ways. Most of these interactions are non-linear as well.
• Everything goes somewhere - projects are closed systems - all the elements of the project have to go somewhere. There are no left over parts in terms of cost, schedule, performance, risk, resources, etc.
• There is no such thing as a free lunch - the net effort to deliver the project is the same no matter what the methodology used. The seemingly extra effort of higher ceremony project methods must be paid by someone when using agile methods.

Everything interacts with everything else.

I totally agree with this.  Complexity is a fundamental aspect of software development.  There is no silver bullet.

Everything goes somewhere.

Totally disagree, and I’ll explain why in a moment.

There is no such thing as a free lunch.

Totally agree, but no one has ever said that agile represents a free lunch.  It represents a discounted lunch.

My Take

Glen, IMO, has a couple of very serious “big project” filters that are misleading him:

• Not every project is delivered
• I have personally been a part of many projects that were never actually finished.  In “high ceremony” projects, a massive amount of documentation was produced, then a small amount of code, then the project was cancelled.  In agile projects, a small amount of doc was produced, a small amount of code, and then the project was cancelled.  Clearly, agile represents a significant savings over high ceremony in this case.
• Not every project, once delivered is maintained
• I have also gone through the effort of producing systems that, for a variety of reasons were never used.  All of that time spent on producing extensive system documentation was wasted.
• Not every project, once delivered, requires significant maintenance
• If you deliver a project, and it is well-used, but for a variety of reasons requires very little ongoing maintenance, the value of a comprehensive technical documentation suite is very suspect.

But even more fundamentally, I disagree with Glen’s assertion in the general case.  Even if a project is delivered, and used, and requires significant maintenance, doesn’t mean that the costs are the same.  For example:

• It can be (and IMO usually is) much easier to write comprehensive documentation for a working system than a theoretically working one.
• It is also easier to train developers/maintainers on a working system than a theoretical one
• It is also easier to gauge the impact of infrastructure changes on working software than theoretical software
• Many (most?) risks are more easily mitigated when they are found earlier than when they are found later.  High Ceremony projects often fail to detect risks as early as agile projects.

Summary

Glen is absolutely right to point out that agile methodologies “move the problem around” instead of “getting rid of the problem.”  There are always drawbacks to every approach.  But I think the key here is to realize that costs change over time.  Some things are expensive at the beginning, and cheaper later on. Others are cheap at the beginning, and more expensive over time.   If you can do the cheaper-at-the-beginning things earlier, and the cheaper-later-on things afterwards, you, and your project win.

A timely and relevant cartoon for your enjoyment, related to my previous post.  But wait, there’s more.
Hat tip: Obie Fernandez

To follow the path:
Look to the master;
Follow the master;
Walk with the master;
See through the master;
Become the master.

I’m not a wine drinker.  Red wine makes me itchy, white wine always tastes sharp and unpleasant.

However, over the last month I have deliberately walked through the wine aisles at my supermarket and at a fancy wine store to see if they have some Stormhoek, so I can raise a glass to Hugh.  Hugh trusts his readers to do right by him, and that’s just plain decent of him.

I updated this Base64 Encoder to work with ActionScript 3 for Flex 2.0 Beta 3.

New version:  Base64.zip

Also, there’s this version, which is binary data oriented:

Note that I’ve only done rudimentary testing with these.   It may not work perfectly.

hyperparallelipiped

 Examples:

• Yes, but does this data take the form of a hyperparallelipiped?
• Can we model this as a hyperparallelipiped?
• What kinds of hyperparallelipipeds do you see in this data?
• This is best described as the product of two hyperparallelipidpeds.

A co-worker of mine appears to have a motto.  I say appears, because I doubt he would actually say this, or claim that it was actually his motto.  But here it is:

Never build an application when you can build an application framework.

I must say that I admire his persistence.  I’ve been working with him for two years, and he’s never actually gotten a framework worth a damn.  Maybe its because he gives up just when it goes past “conceptual prototype.”  Maybe it’s because he applies his personal biases and assumption to the framework, to the point that any deviation from the framework bias renders it unusable.  Maybe its because his frameworks relentlessly grow more and more complicated (and, of course, undocumented) as he adds features, to the point that they become maintenance nightmares par excellance - lacking in any of the -ilities, other than perhaps fatility.

It is, perhaps, a good thing that those of us who must live in the shadow of the builder-builder have built a support group, with wonderful code words and phrases that send tears of laughter into our eyes, or daggers of rage into our hearts at a moment’s notice.   Sound effects, catch phrases (”Yeah, but you have to support it!”), props (our latest is a box of crayons as the “graph builder-builder”) and wonderful realms of conceptual fancy where frameworks hold sway over the earth, enslaving the poor lowly applications in the iron chains of single-purposedness.

At one point in my career, I might have been jealous of the builder-builder, living without consequence, without accountability, without deadlines, without domain expertise, without any real-world constraints on what he architects and prototypes.  But now, I like building things that work, that actually get delivered, that get used by customers.   So we keep our props, and our catch phrases (”It’s completely configurable, All you need is to put some code in the XML.”) close at hand, to serve both as a source of amusement, and as a solemn reminder of what not to do.

I’ve been an informal betatester for InfoQ for the last few days.  It’s a news aggregator and original content magazine for numerous topics in software development.  Worth a look (or two!)

is up!

From the NYTimes:

Failure, Mr. Petroski shows, works. Or rather, engineers only learn from things that fail: bridges that collapse, software that crashes, spacecraft that explode. Everything that is designed fails, and everything that fails leads to better design. Next time at least that mistake won’t be made: Aleve won’t be packed in child-proof bottles so difficult to open that they stymie the arthritic patients seeking the pills inside; narrow suspension bridges won’t be built without “stay cables” like the ill-fated Tacoma Narrows Bridge, which was twisted to its destruction by strong winds in 1940.

Because of the volatile nature of software development, failure is a spectre that lingers over every project, all the time. Failures in communications, in comprehension, in reliability, in efficiency, in usability, in execution. Failure, as Mr. Petroski says, is both inevitable and the key to learning and growing.

Iterative development is an example of the improving state-of-the-art, given that failures are going to happen (and going to happen often), Iterations allow the failures to be recognized and studied soon after they occur, instead of 1,2,3 years down the road. Iterations allow failures to be repaired/addressed earlier too, which leads to a better overall design. The next time someone says ‘We need to think this through before we do it’, remember that engineers thought the Tacoma Narrows bridge through too. And yet it failed. And the failure was studied, and learned from, and since then bridges have been better (Although we appear to be overdue for another major bridge failure).

Unless you are your own customer, you will never have a precise understanding of what they want, and they will never have a precise understanding of what you are going to give them until they get it. That’s what I would call a failure-rich environment. Iterative development allows those failures to be identified early, discovered in a controlled way, and dealt with immediately.

Compare and contrast with this story, where a fairly innocuous change to the implementation of the architect almost led to major catastrophe.   If the architect had been more dismissive of the student’s question, or if he had decided to remain quiet to avoid disgrace, things might have ended tragically.

I can’t help but notice that it was a technical change that created the problem, and a human interaction that brought it to light.

From Authentic Happiness, my 5 signature strengths are:

• Creativity, ingenuity, and originality
• Humor and playfulness
• Capacity to love and be loved
• Fairness, equity, and justice
• Zest, enthusiasm, and energy

(Hat tip: Will Wilkinson)

Having recently purchased and listened to Dr. Robert Cialdini’s 45 minute lecture on persuasion and influence, I’ve been working on thoughts on how to apply those lessons to Agile Development, in terms of:

• How to convince the necessary managers that switching to agile techniques is a good idea
• How to convince the developers that switching to agile techniques is a good idea.
• How to convince the customers that they should support the process
• How to get the team involved in delivery to do their best.

First: thanks to Dave Taylor for the suggestion to download the lecture. It is probably the most valuable $2.95 I have ever spent. It isn’t a perfect lecture, but it is extremely good.
Second: Quick overview - Dr. Cialdini’s lecture talks about the keys of persuasion, centering around the concepts of reciprocity, commitment, exclusivity, gain and loss, and authority. He lays out techniques for influencing and persuading people, using these concepts. What I am going to do is try to cement those concepts to the heavy lifting of
getting agile projects underway.

Third: I am sure that these will seem manipulative techniques to many people, especially introverts (like me) who don’t like having to persuade. They are. But at the end of the day, Dr. Cialdini’s techniques seem very solid, well supported by experimental evidence. I would rather persuade someone to do agile development than fume and grow bitter and angry in a non-agile environment.
How to convince management to switch to agile techniques

1. Do you actually need to ask? Many agile experts suggest that since the management team doesn’t understand programming, they won’t really care if you change the methodology
2. Ask to convert the whole organization, or as much of it as you think could succeed. The key here is to think big - go for the big win.
1. If they say no, back down gracefully, and ask “Well, what about a pilot project?” According to Dr. Cialdini, people are more likely to agree with you if they think you’re trying to find common ground (giving them a concession)
3. Don’t talk about how much there is to gain from agile development - describe the change in terms of what they will lose if they don’t switch:
1. Less responsive to change
2. Less effective feedback
3. More likely to go over time and over budget
4. If you believe it to be true, find exclusive information that justifies your position. For example, perhaps you have a report from a consultant, talking about how agile development is transforming your industry, a report that you have paid for and is not widely known.
5. Or, alternatively, if there is already significant movement towards agile development in your industry, play up this consensus - try to get management to feel that they are lagging behind.

Again, buy and listen to the podcast if you want more insight into why I chose these particular 4 topics. And if you have suggestions on how I could add to them or improve them, please feel free to share.

How to convince the developers to switch to agile techniques

1. Many of them will already be familiar with agile techniques. Some may be doubters, some may be evangelists. Try to see if you can get a sense of consensus - that most of the team is with you. It will help move the doubters your way.
2. Admit to a failing of agile development in your industry - perhaps the documentation requirements aren’t quite good enough, or the iterations might be too short. Then, talk about the benefits - faster response to change, better involvement from customers, faster starts. Most importantly, bring up the weakness before anyone else. Be honest about it, don’t try to sugarcoat it, and don’t try to hand-wave it away. Instead, say - this is a weakness, yes, but all these other strengths outweigh it.
3. If the developers continue to resist, consider saying “this doesn’t have to be a permanent arrangement - why don’t we try it for a few iterations and see how it goes?” You want to concede a little bit here, in some meaningful way, but (the tricky part) without conceding so much that you end up losing the power of the combined agile techniques.

How to convince customers to support your agile process

1. Point out that the customer will have exclusive access to your development group, and exclusive information about their status and their progress.
2. Remind them of what they stand to lose if you go away for a year and end up building the wrong thing.
3. Point out before they do that heavy customer involvement is a challenge and a sacrifice. Then point out all the good things that agile development will bring to them (insight, better knowledge about the progress, regular demos, etc), and say that the good things outweight the negatives.
4. Ask for the maximum amount of customer involvement you could possibly want - for example, two of your people on site, all the time, dedicated to our project. If they balk, back down to something less stringent, but still effective - “well, what about 1 person then” or “How about two people, each half time”.

How to get the team to deliver their best

1. Ask for their verbal (or even better) written commitment that they will do their best to get the project done on time. I personally would follow that up with my own commitment that I will hold true to the fixed scope of the iteration, and do my best to remove all impediments to progress, up to and including taking care of their laundry, bringing in food, etc.
2. Be like them - if at all possible, find common ground. Make them feel that you are part of their “tribe.” They are much more likely to work hard for someone they respect and trust than someone they barely know and can’t relate to.
3. Remind them of what they stand to lose if you are forced to go back to a waterfall process - death march deliveries, brittle designs without unit tests, weeks or months spent making no progress on the development while the clock ticks.

What not to do

• Do not, under any circumstances, concede the core agile techniques away. If you do, you are inviting failure on many, many levels. Concede the duration and scope of the agile experiment, not the agile process. But don’t concede what you believe to be the keys to agility. I can’t tell you what those keys are for you, I can only tell you what they are for me (Iterative Dev, Unit Testing, TDD, Continuous Integration, paper prototyping, customer involvement or excellent customer proxies and user stories/informal use cases). If you trade those away, you’re effectively gutting the process. It would be better to just walk away than to pilot a project that is doomed from moment #1.

Thanks again for the time you took to read this, and let me know a) What you think, b) If you try it out and it works.

This most excellent list is provided by Fred Monjo, from the Scrum Development Yahoo Group 

• Work as an unbreakable united team
• As a contributor, customer (PO) is the key for the project’s success
• Welcome changes as a competitive advantage
• Communicating well is better than programming well
• Documentation must be useful and significant
• High quality is the key to high speed
• Keep your architecture flexible
• Think simple, efficient, and incremental
• A good team in a good environment

#3 is probably the hardest one for non-agilists to get - they see change as decay, entropy, disruption and pain, not as opportunity.

More support for my assertion that feedback is one of the most valuable of the agile benefits, and one that isn’t as easily appreciated as you might think:
From Mary Poppendieck, on the Agile Project Management Yahoo Group:

Being a control engineer (and using control in the sense it is used in control engineering) I know that the shorter the feedback cycle, the more predictable the process, up to a very find-grained point where extremely rapid feedback will create thrashing.  Companies that have processes tuned to rapid feedback are some of the most predictable in the world, because they are poised to RESPOND to events as they unfold, rather than predict the future and then act on that prediction as if it were fact. One of the fundamental principles of lean thinking is precisely that – you get far more predictable results when you are able to wait until the last responsible moment before you make a decision, because then your response will be the most appropriate. 

For example:  Dell was able to respond exactly correctly to the ten day dock strike on the West Coast a couple years back because by culture it is able to keep its supply chain flowing despite any interruption.  Most other companies were thrown into chaos by the strike.  Zara – a clothing manufacturer in Spain – was able to design and distribute to its stores the outfit Madonna wore to her first concert in Spain recently, so that by her last concert that outfit was worn by teens in the audience.  Zara is no small chain – it is a 5 billion Euro company that competes very effectively with Gap and H&M and achieves higher margins because it can respond much faster to fashion trends – in two weeks if need be – rather than try to set them.  Toyota can put a new car on the market in 15 months when it has to, and routinely does it in 18 months.  In all of these cases, predictable results come BECAUSE the organization is able to respond correctly to the latest information while competitors’ slow response time causes them to react prematurely and thus base decisions on guesses instead of facts.  

I contend that the really competitive companies that compete on the basis of speed are the ones who have created a sustainable competitive advantage, and those who learn how to do this in software development will be similarly blessed with extremely repeatable, predictable performance.  I don’t think your McDonalds analogy is correct – repeatable, predictable, fast performance is most certainly NOT inevitably tied to mediocre results.  There are a multitude of companies that have found a way to have speed, quality, and low cost all at the same time – look at Google for a software example.  

This is in response to Robert Bogue, who writes:

It’s actually the reverse.  To increase predicatability you have to add more feedback metrics which will consume capacity and reduce velocity.  I wrote an article a few weeks/months back titled “How Softawre Development is Like Fast Food Restaurants” (http://www.developer.com/design/article.php/3585031), this may illuminate my statement that predictability requires “control” (i.e. metrics) which does tend to make things slower.  I believe this view is shared by several others.  Let me know if you would like me to dig through references to find some of the others in the community who share this perspective.

As you can probably imagine, I come down strongly on the Mary side of this debate.  Feedback is critical, and companies that effectively use feedback to manage their day-to-day operations seem to execute better than companies that try to optimize feedback out and replace it with upfront predictions.

From an article in the Wall Street Journal about evolution comes this quote from biologist Joan Roughgarden of Stanford University:

The idea that females choose the genetically best males is wrong. Instead of choosing mates who will increase the genetic quality of their offspring, females make choices that will increase their number of offspring

This is an example of a framing problem endemic in debates about Waterfall and Agile development. People assume that the option they find most compelling is, by definition, the best one. But the best option is the one that delivers the goods, not the one that “should deliver the goods.”

Specifically referencing the quote above, notice the contradition - “genetically best” in her mind does not, apparently, mean the same thing as “increase their number of offspring. Huh? Consider the following:

The idea that project managers choose the theoretically best software development methodology X is wrong. Instead of choosing methodologies that increase the quality of the project, project managers make choices that will increase the number of projects delivered on time and on budget.

Isn’t “deliverying on time and on budget” a realistically meaningful definition of quality? Not the only one, to be sure, but certainly a compelling one, no? And in the case of the birds, isn’t one definition of genetic quality contributing to the number of offspring? There’s a cognitive bias here that says that “Attribute X” makes someone’s genes better, but that is simple human arrogance. God and Nature define what genetic success is for a bird, not Man (or in this case, Woman).
Now, the next time someone says that BDUF improves the “quality” of the project, make sure that you and they are on the same page about what “quality” means.

Note - in all fairness to Ms/Dr. Roughgarden, she may have been completely misquoted. The problem, unfortunately, remains.

Is Stellarium, the program that lets you scan and zoom and spin and take control of the night sky, from anywhere on earth, at any point in the past or future.

It is free and open source, and very cool.

Also, if I haven’t mentioned it before, DesktopEarth is also very cool, although not open source.  But it is free.

Welcome to the May 4th edition of the Carnival of Agilists - the bimonthly roster of the best posts from the Agile Blogosphere.

Individuals and interactions over processes and tools

• Ron Jeffries tried baseball, and it didn’t work.
• Skip Angel writes a one page memo, describing his vision for his department. There’s a lot of insightful team-oriented thought in here, well worth a read.

Working software over comprehensive documentation

• Alex Pukinskis shares his thoughts on ending an Iteration.. Acceptance, Demos, Reviews, Metrics, Lunch and more.
• Johanna Rothman insists that Architects must write code and gives a bunch of reasons why.
• Victor Lombardi provides us with early thoughts from his book, discussing how to use metrics to in big, visible ways.

Customer collaboration over contract negotiation

• In order to collaborate with your customer, you have to listen to them. Mishkin Berteig writes about Eight Barriers to Effective Listening and adds some thoughts of his own
• Ed Gibbs suggests that you don’t let the methodology get in the way of common sense.

Responding to change over following a plan

• Achilles (the BDUF software architect) and Tortoise (Ron in disguise) debate the merits and demerits of the Extreme Programming approach to software design

Case Studies and Analysis

• Kevin Rutherford provides some notes from an insightful analysis on the differences between Scrum and Lean.

Fun!

• High Moon Studios provides a video on how they use Scrum to build their games.
• How to mock a Singleton
• This product sounds really cool.. I wish I knew what it did.

Previous Editions
All previous editions of the Carnival are referenced at the Agile Alliance website.

Join in the Fun!
Have something that you think is worth sharing? Don’t be shy! We love new ideas and insights. Send us a link to your post at agilists.carnival@gmail.com.

The Carnival is published on the first and third Thursday of each month. If you would like to participate, please send us a link to your post at agilists.carnival@gmail.com. Or, if you prefer, use this handy dandy carnival submission form.

Contributors

My thanks to all of the following for contributing!

• Kevin Rutherford
• Ron Jeffries
• Skip Angel
• Johanna Rothman
• Mishkin Berteig
• Ed Gibbs
• Alex Pukinskis
• Victor Lombardi

When you’re trying to explain a technical product/concept/architecture/whatever to someone less technical, be careful that you don’t end up looking like this man.

Technically sophisticated:  You can describe complex things.

Technically smart:  You can explain complex things, and make them seem simple.

From the ScrumDev yahoo group, I found this quote:

“An Agile project begins when testers convert high-level requirements into testable specifications.”

At first, it resonated with me. Then I started to question it.

• “What if the requirements aren’t fully fleshed out?”

But then, I realized the answer to that was “If the high level requirements aren’t fully fleshed out, you can’t realistically start the project. That’s like saying your backlog isn’t fixed for the 30 days - if it isn’t fixed, you’re already hosed.

• “What does ‘testable specifications’ mean?”

Well, our company uses ‘day in the life’ stories that go through most of the core, commonly used functionality. I don’t know if that meets Tobias’s standards, but it certainly seems to get the job done here.

So, after questioning it, I find that it still makes a lot of sense. Huzzah!

Many people are afraid of outsourcing, even though it is becoming pretty clear that the actual decrease in cost of outsourcing is much less significant than most expected.

But I believe that there are still many, many more ideas on how to use software to make life and business easier than there are people to implement those ideas.   And as long as that is true, there will always be significant demand for software developers, demand that will soak up all the “excess supply” at the price it can afford.

And, most importantly, the more software that is written, the more ideas and the more new opportunities for software there are.   Software isn’t a zero sum game.  There isn’t a fixed pool of software projects that we are drawing from.  Quite the opposite - the more software produced, the more new ways people come up with to use it, extend it and obviate it.   All of which, of course, is done in software.