Trial-and-error with a feedback cycle


(This essay appears in the e-book What I've Learned From Failure: A quarter-century of experience shipping software, distilled into fixnum bittersweet essays)

Don’t EVER make the mistake that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle.

That's giving your intelligence much too much credit.

trial and error

A common mistake when trying to be “Agile” (or trying to appear to be Agile without actually being Agile) is to shorten milestones while still maintaining the idea that we have a plan for every milestone done up at the beginning of the project and we consider changing the plan to be a management failure.

The whole point of trial and error is to make our plan, but to accept the inevitability that our plan will not survive contact with reality. I was working on one project, and the first two-week cycle went badly sideways. The project manager complained that she hates missing milestones early in a project, it foreshadows all sorts of problems. She wanted to impose overtime to hit the date.

I urged restraint. If we were wrong about our velocity, or wrong about some of our technical predictions, or wrong about some of the requirements, it would be far better to adjust our expectations. The two-week cycles were a bonus because they were providing the feedback we needed to adjust our plans.

If you ignore the evidence, if you try to twist the appearance of reality to suit your plan rather than twist your plan to suit reality, the feedback is wasted.

feedback

Feedback is so critical, that even if you do everything else right, and even if you think you are gathering and responding to feedback, you can still fail. A recent flurry of blog posts discussed The Waterfall Trap. This is when you build software incrementally instead of iteratively.

Increments are components of the finished software that do not provide value in and of themselves, like a database and its associated ORM doo-hicky. When you build an increment, what feedback do you have? Feedback about how well you are executing against your plan.

This is like deciding to march through the jungle and measuring how many steps you take per day. Are you going in the right direction? Who knows, but you know you how fast you are going, and that is worth something, isn’t it?

I can tell you exactly what it is worth. At some meeting later on, when the project has been judged an expensive failure, you can prove it was a well-managed failure under your watch. Congratulations.

For feedback to really work, you must have feedback about how well the finished software delivers value to its users. The only way to do that is to deliver it to those users and get their feedback. Thus, the other approach: to build the software iteratively. This means to build things that are valuable in and of themselves, so users can judge how well they work.

productization

My Golden Hammer for building software iteratively is to divide it into separate products. We sit down and look at the pile of functionality on the plan and ask, “Which of these pieces could be useful things in and of themselves?”

On one project, there was a complex bit of loan eligibility reasoning that needed to be built into a database application. We decided that although the finished application would have the reasoning built into certain input screens, the reasoning could be a separate product. We imagined this thing sitting in its own screen alongside what their existing process.

There would be some double entry involved, but we knew that if we built this as a stand-alone product, it could be used in production and it could provide value. So we did. And sure enough, we got a lot of feedback from the users. We didn't end up deploying it in production, they wanted to wait for the rest of the pieces to be built, but they could play with it, they could see how it worked, they could give us real feedback.

Did I mention they decided against deploying it in production until the rest of the project was ready? That was the most critical piece of feedback yet. It told us it wasn’t as important as we thought, and we were able to reprioritize around other things.

That kind of feedback is only possible when you get feedback about the value you are providing. And that is exactly the feedback Linus is talking about: the ruthless feedback of users who don’t care about your plans or your years of experience or your blog or whether you work for a billion-dollar company or whether you use Java or JRuby, or whether your servers are MSFT or Linux.

The feedback you get from people who just want to get stuff done.   ¶ 11:45 AM