raganwald
Sunday, December 16, 2007
  Trial-and-error with a feedback cycle



(This essay appears in the e-book What I've Learned From Failure: A quarter-century of experience shipping software, distilled into fixnum bittersweet essays)


Don’t EVER make the mistake that you can design something better than what you get from ruthless massively parallel trial-and-error with a feedback cycle.

That's giving your intelligence much too much credit.
Linus Torvalds


trial and error

A common mistake when trying to be “Agile” (or trying to appear to be Agile without actually being Agile) is to shorten milestones while still maintaining the idea that we have a plan for every milestone done up at the beginning of the project and we consider changing the plan to be a management failure.

The whole point of trial and error is to make our plan, but to accept the inevitability that our plan will not survive contact with reality. I was working on one project, and the first two-week cycle went badly sideways. The project manager complained that she hates missing milestones early in a project, it foreshadows all sorts of problems. She wanted to impose overtime to hit the date.

I urged restraint. If we were wrong about our velocity, or wrong about some of our technical predictions, or wrong about some of the requirements, it would be far better to adjust our expectations. The two-week cycles were a bonus because they were providing the feedback we needed to adjust our plans.

If you ignore the evidence, if you try to twist the appearance of reality to suit your plan rather than twist your plan to suit reality, the feedback is wasted.

feedback

Feedback is so critical, that even if you do everything else right, and even if you think you are gathering and responding to feedback, you can still fail. A recent flurry of blog posts discussed The Waterfall Trap. This is when you build software incrementally instead of iteratively.

Increments are components of the finished software that do not provide value in and of themselves, like a database and its associated ORM doo-hicky. When you build an increment, what feedback do you have? Feedback about how well you are executing against your plan.

This is like deciding to march through the jungle and measuring how many steps you take per day. Are you going in the right direction? Who knows, but you know you how fast you are going, and that is worth something, isn’t it?

I can tell you exactly what it is worth. At some meeting later on, when the project has been judged an expensive failure, you can prove it was a well-managed failure under your watch. Congratulations.

For feedback to really work, you must have feedback about how well the finished software delivers value to its users. The only way to do that is to deliver it to those users and get their feedback. Thus, the other approach: to build the software iteratively. This means to build things that are valuable in and of themselves, so users can judge how well they work.

productization

My Golden Hammer for building software iteratively is to divide it into separate products. We sit down and look at the pile of functionality on the plan and ask, “Which of these pieces could be useful things in and of themselves?”

On one project, there was a complex bit of loan eligibility reasoning that needed to be built into a database application. We decided that although the finished application would have the reasoning built into certain input screens, the reasoning could be a separate product. We imagined this thing sitting in its own screen alongside what their existing process.

There would be some double entry involved, but we knew that if we built this as a stand-alone product, it could be used in production and it could provide value. So we did. And sure enough, we got a lot of feedback from the users. We didn't end up deploying it in production, they wanted to wait for the rest of the pieces to be built, but they could play with it, they could see how it worked, they could give us real feedback.

Did I mention they decided against deploying it in production until the rest of the project was ready? That was the most critical piece of feedback yet. It told us it wasn’t as important as we thought, and we were able to reprioritize around other things.

That kind of feedback is only possible when you get feedback about the value you are providing. And that is exactly the feedback Linus is talking about: the ruthless feedback of users who don’t care about your plans or your years of experience or your blog or whether you work for a billion-dollar company or whether you use Java or JRuby, or whether your servers are MSFT or Linux.

The feedback you get from people who just want to get stuff done.
 

Comments on “Trial-and-error with a feedback cycle:
Eek, Linus' quote is just way too scary! A random walk towards a solution, builds with it a set of extended and in many cases artificial complexity. If there is one thing our intelligence CAN do, is find a cleaner abstraction than could ever be discovered via a trial-and-error feedback cycle. That's why it is special.

100,000 typing monkeys, even with some funky cool perfect feedback cycle are still not going to give you Shakespeare. If against all odds, it did happen, it will have taken nearly forever.

A "ruthless massively parallel trial-and-error approach with a feedback cycle" only works to fine-tune something, it can't take you there if it ain't already close.

Paul.
http://theprogrammersparadox.blogspot.com/
 
Paul, thanks for your comment.

BTW, readers, do not be shy with links to your blogs. Blogger's otherwise excreble comment facility does permit you to use:

<a href="http://..."> http://theprogrammersparadox.blogspot.com/ </a> for those links. Make it easy for other readers to visit you!
 
I don't think the quote is about trial and error in 'coding' specifically, but more trial and error about the specific 'work' that should get done. Hence the conversation about iterations or iterative development. Or did I just miss something?

Ponderous Programmer
 
Linus's quote applies in a very limited, non-creative area, after initial design was already done (e.g. Linux).

Otherwise Microsoft would be much better than Apple at doing everything and we would prefer building weapps in Java instead of Ruby.

But the article content is right: getting feedback from real users working with real app is the only way to know if your product is solving anything. Forget specifications, focus groups, and other research.

The only way to extract knowledge about problems from people is to provide them with concrete implementation. Everything else is too abstract for most people.
 
> a very limited, non-creative area, after initial design was already done (e.g. Linux)

Mmmh. Show us your operating system.
 




<< Home
Reg Braithwaite


Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

Books
What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

Share
rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

Beauty
IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

Work
The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Management
Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

Notation
The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Opinion
Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

Whimsey
The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

History
06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /