raganwald
(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Monday, January 15, 2007
  What I've Learned from Sales, Part II: Wanna Bet?


In Part I of “What I’ve Learned from Sales,” Don’t Feed the Trolls, we looked at why resistance to a new idea is expressed as a never-ending series of objections. We looked at one powerful way to avoid objections, by identifying a real, urgent problem that needs to be solved. The next installment, Part III, How to use a blunt instrument to sharpen your saw, describes the mind-set that there are opportunities for improvement to be found everywhere.

In this part, we’re going to disregard my advice to avoid objections and talk about one way to respond to many of the objections raised against new ideas.


Can we overcome objections?

Right off the top, I want to say that I don’t believe you can “overcome” an objection by frontal assault. And furthermore, you shouldn’t try. You cannot persuade someone to consider an idea by debating them into submission.

My belief is that you must discover and address their true problem. The first reason to do so is that if they do not have a problem your idea can solve, there is no reason for them to “change for change’s sake.” The second reason is that if you are not solving a real, genuine problem for them you will get caught defending an idea against an endless series of objections.

That being said, there are some circumstances where it is important to respond directly to objections. Even if we have carefully qualified someone’s problem and explained how a new idea will solve their problem, a prudent person will analyse the idea carefully, looking for fatal defects that could prevent it from solving their problem.

Another common circumstance is when there are several parties involved in presenting, discussing, and analysing an idea. Although one of the parties may be bringing up irrelevant objections to resist the idea, you may need to persuade another of the parties that these irrelevant objections do not have merit.

For example, you may be suggesting that agile meta-programming will solve a company’s problems to the CEO. You may have carefully qualified the CEO’s priorities. But the company’s IT department will raise objections because your proposals do not address their personal and departmental problems. So you will need to respond to some of their objections as part of a campaign to neutralize their influence.

What is overcoming an objection?

Overcoming an objection is, to borrow a phrase from law, “A sword, not a shield.” When you overcome an objection, you point out that the reason for not considering your new idea is fallacious, or does not apply in this case.

However, overcoming the objection does not actually provide a reason to change: it merely removes a reason not to change. This is why Part I goes on and on about discovering an immediate problem your idea can fix. If you overcome an objection but have not presented a compelling reason to change, nothing will happen.

In other words, if someone says “I like your idea, however...” you should attempt to overcome the objection. If someone says “Your idea stinks because...” you really need to identify a problem to solve.

One model for responding to common objections


A gotcha objection is a proposition that your idea is false based on a premise that is true infrequently or only true for some small number of cases.


Let’s assume you are going to respond to an objection, and you have good reason for doing so. Here’s one common form of objection, and a method for responding to the objection. In essence, I am going to share a pattern with you.

First, let’s look at one of the most common kinds of objections. Here are some examples. They all have something in common:


What these objections all have in common is what I call the “gotcha!” fallacy. The underlying assumption is that if your idea does not work 100% of the time, on 100% of the cases, it is no damn good. And thus I call an objection based on the gotcha fallacy a Gotcha Objection. A gotcha objection is a proposition that your idea is false based on a premise that is true infrequently or only true for some small number of cases.

P is for Pharmaceutical

Think about therapies in medicine. None of them are deterministic! Every pill, every technique, every therapy is described in probabilistic terms: When compared to the control group who drank a glass of red wine daily but did little exercise, 36.7% of those who combined daily exercise with a glass of red wine had an average improvement of 22.1% in their combined evaluation scores for cardiovascular health.

Try this the next time you’re at the doctor’s office: point out to your physician that you have heard that some people who exercise drop dead right after their daily run. Use this as an excuse not to exercise.

Now, software development is not the same thing as medicine, and I am not suggesting you respond to criticism by saying that since some drugs do not work for some patients that your ideas have merit regardless of evidence or suggestions to the contrary. But I will walk you through the reasoning that leads to the same conclusion.

How to respond to gotcha objections

Our pattern for responding to these gotcha objections is to establish that software development is probabilistic in practice. Once we establish this premise, we then turn the debate from whether there are cases where a particular practice does not appear to be optimal to whether the overall results of applying that practice is better than not applying that practice.

That’s it, and if you’re in a hurry you can stop right here: everything else is an elaboration of this idea.

Exempli gratia: the technique in action

Objection: “Sometimes you are gonna need it, so you’re wasting time if you don’t build it in from the beginning. Therefore, we should build what we know we’ll need.”

Response: Well, sometimes you need it, sometimes you don’t. And when you don’t need it, you save the code you would have written as well as all the other design that becomes coupled to the code you end up throwing out. Of course, sometimes you end up throwing out some stub code, but think about the possibility that you will wind up getting more features done, earlier in the cycle where we can get feedback and reduce risk. Why don’t we look at whether, in aggregate, more projects will succeed using YAGNI than will succeed using “Build everything we might need”?

Objection: “Some bugs simply can’t be found through automated unit tests, sorry. This is why we have to use a programming language with static typing.”

Response: Sure enough, some can’t be detected with automated unit tests. But our choice of a dynamic language provides us with many other benefits, most especially in the areas of metaprogramming and code reduction. By writing less code, we may even have fewer bugs overall. Shouldn’t we try to compare similar projects written in a static language against those written in a dynamic language, and see whether the projects in the dynamic language had fewer bugs and whether the projects written in the dynamic language were more likely to be successful?

Now that you have seen the results of applying the technique, we will patiently examine the reasoning in detail.

Theory D and Theory P in Software Development


Theory P states that the time and effort required to measure all of the variables influencing a software development project precisely enough to predict the outcome with certainty and in advance exceeds the time and effort required develop the software.


There are two schools of thought about the practice of managing software development (the theory of managing software development is of little use to us because “the gap between theory and practice is larger in theory than it is in practice”).


Critical Chain explains project management from the ground up in probabilistic terms. It's a significant improvement over the classical approach for managing risk and uncertainty in software development.
One school is that everything is fully deterministic in practice (“Theory D”). If development appears, from the outside, to be probabilistic, it is only because we haven’t discovered the “hidden variables” that fully determine the outcome of software development projects. And, since we are talking about development in practice, it is practical to measure the variables that determine the outcome such that we can predict that outcome in advance.

The other school of thought is that development is fully probabilistic in practice (“Theory P”), that there are no hidden variables that could be used to predict with certainty the outcome of a software development project. Theory P states that the time and effort required to measure all of the variables influencing a software development project precisely enough to predict the outcome with certainty and in advance exceeds the time and effort required develop the software.

Theory P does not mean that software development cannot be managed in such a way that the desired outcome is nearly certain: the flight of an airplane is fully probabilistic as it encounters atmospheric conditions, yet we have a huge industry built around the idea that airplanes arrive at their destinations and land on the runway as planned.

Which theory fits the evidence collected in sixty years of software development? To date, Theory P is the clear winner on the evidence, and it’s not even close. Like any reasonable theory, it explains what we have observed to date and makes predictions that are tested empirically every day.

(Sidebar: do not confuse Computer Science—the study of the properties of computing machines—with Software Development, the employment of humans to build computing machines. The relationship between Computer Science and Software Development parallels the relationship between Engineering, the hard science of the behaviour of constructions, and Project Management, the employment of humans to construct engineered artefacts.)

The first response to the gotcha objection


The race may not always be to the swift nor the victory to the strong, but that’s how you bet
—Damon Runyan


Before we can go anywhere with gotcha objections, there is something you absolutely, positively must do when you respond. And it must be the very first thing you do. You must establish the fact that the premise of the objection is not universal and not predictable, it is probabilistic.

The objection is of the form that “since your idea works out badly some of the time, your whole idea is bad.” You must respond by establishing that some of the time the idea doesn’t appear to work out, and some of the time it does appear to work out. It isn’t universally bad or universally good.

If you are in a face to face discussion, you can solicit agreement from the objecting party. For example:


In a less interactive environment like a running language flame war on Usenet, you can start by simply stating that the premise is not universal.

Having established that the premise is not universal you must then establish that the cases where the premise applies are not easily distinguished from the cases where the premise does not apply. Establish that nobody knows whether the premise will apply or not until after it has happened.

We simply can’t tell in advance whether the bugs that would be caught by a static type system will end up being significant to the outcome of a project. We can’t tell in advance which constructs will end up being a waste of time. And we can’t tell in advance which people will fail when they try to pair program. (The last point is absolutely true if the people involved are not doing the arguing about whether pair programming will work. If the programmers involved do not believe it will work, they may have a point.)

This is the other aspect of establishing that the premise is probabilistic: not only does it only apply some of the time, but we don’t have a good way of knowing in advance when it applies and when it doesn’t.

Okay, we’ve gone through all of this dry pseudo-academic talk of theories and probabilistic development. Time for a vacation to Las Vegas.

Casino Royale, follow link for rights information.
How do casinos make money?


The casino’s strategy is so secure, there is just one danger to its profits: if the casino plays very few games for very high stakes relative to their capitalization, they could lose their capital.


Casinos make money by wagering money on the outcome of games with gamblers. (If the gambling industry offends you, I apologize. We could choose to look at how insurance companies make money, it is entirely the same thing.) The games are arranged in such a way they the casino holds a small mathematical edge on each play. Over the long run, with many gamblers playing the games many, many times, the casino inexorably makes money. The casino may lose games here and there, and some gamblers may enjoy temporary winning streaks, but overall, the casino wins more than it loses.

The very briefest exploration of statistics reveals the following facts about the casino’s strategy for making money:

  1. The casino must have an edge on each play;

  2. The more games played, the more likely that the casino will profit overall;

  3. Runs of good luck for some gamblers are offset by runs of bad luck for other gamblers.


Henk Tijms’s Understanding probability explains probability in simple terms requiring very little mathematics. Examples drawn from everyday life include analyzing investment returns, lotteries, and gambling. The book continues to build on the basics, worthing through Bayes' theorem up to multivariate and conditional distributions. A must-read for those working with data or seeking to understand risk analysis.
In fact, the casino’s strategy is so secure, there is just one danger to its profits (besides the obvious fear of losing their license to print money): if the casino plays very few games for very high stakes relative to their capitalization, they could lose their capital. One celebrated “whale,” Akio Kashiwagi, won more than $19 million in one casino and on another occasion won $6 million in a siangle session playing baccarat for $200,000 a hand.

Therefore, the casino’s prime safeguard is to avoid risking large amounts of capital on games played very few times.

To summarize, the casino’s strategy is to:

  1. Arrange a small advantage on each game played;

  2. Play a very large amount of games;

  3. Ignore good and bad runs, they will offset each other;

  4. Do not risk large amounts of capital on games played very few times.

Now that we know the casino’s strategy, let’s consider how they evaluate games. Imagine them sitting around a conference table, and someone suggests, “Let’s create a new game, Alaska Freeze.” What do they use to evaluate whether to add this new game to their casino?

Space in a casino is at a premium. If Alaska Freeze goes in, something else comes out. So there has to be an Incremental Value calculation: do they make more money with Alaska Freeze in and something else out? Or less? This calculation has two simple components: does Alaska Freeze have a larger or smaller mathematical edge than whatever it replaces, and will Alaska Freeze be played more or less often than whatever it replaces?

Ok, let’s return to handling gotcha objections. I’m sure you knew all of this, I was just presenting it in a palatable morsel so you can feed it to someone objecting to your idea.

From casinos back to objections


The process of developing software is just like the business of running a casino.


You are handling an objection and you have established that its premise is neither universal nor predictable, it is probabilistic. Well, if it has an uncertain outcome, it is just like a casino game. And the process of developing software is just like the business of running a casino.

So the question is not whether a new practice ever has a case where it appears to “lose,” for the same reason that evaluating a new game for a casino does not involve worrying about whether gamblers will ever win.

The way to evaluate the idea is to examine it and see whether it fits the casino model:

  1. Does it offer a probable advantage each time it is employed?

  2. Can it be employed many times?

  3. Do streaks of “losses” and “wins” offset each other?

  4. Can we avoid risking the entire outcome of the project infrequent events?

And if it does, to identify what idea or practice it replaces to determine whether it is a net win or a net loss overall:


Now, I am not going to say that XP, YAGNI, or dynamically typed programming languages necessarily fit the casino model and are necessarily better than the practices they replace (Classical Project Management, BDUF, and static typing). But what I will say is that there is a huge difference between saying “some of the time, for some of the people, that idea loses” and saying “overall, when applied to an entire project, the project does worse than whatever idea it replaced.”

So to handle a gotcha objection, we establish that it is probabilistic in nature, then we analyse it as we would analyse any other probabilistic practice: we look at the overall effect of the practice on the entire project, comparing it to whatever practice it would replace. And I have some easy-to-remember phrases for doing that.

The second response to the gotcha objection

The next step is rather obvious. You have to state the payoff those times that your idea or practice “wins.” And to be fair, you also have to agree to the cost of your idea or practice when it “loses.”


Needless to say, if you are in a face to face meeting you should solicit agreement to this second response as well. If you can’t establish that there are any benefits to your suggested practice, you have a great deal more work to do to handle this objection.

The third, and final response to the gotcha objection

The final step is the clincher. Having established that the objection’s premise is probabilistic, and that for those times the idea or practice “wins” there is a positive payoff, it’s time to compare the overall benefit of the idea or practice to whatever it replaces. You want to shift the debate from debating the premise to debating the overall benefit.

And in fact, there are two different forms of the clincher. You can use either, or preferably both:

  1. Ask whether you can balance the benefits of using the practice when it wins against the cost of using the practice when it loses, and evaluate the overall benefit in comparison to the benefits and costs of the alternative and see whether you would get more of a benefit over one entire project, or;

  2. Ask whether you can compare the success rate of teams using the practice to the success rate of teams using the alternative in aggregate, or;

  3. Both.

And here are the example responses again, demonstrating the three forms of clincher:

Objection: “Sometimes you are gonna need it, so you’re wasting time if you don’t build it in from the beginning. Therefore, we should build what we know we’ll need.”

Response: Well, sometimes you need it, sometimes you don’t. And when you don’t need it, you save the code you would have written as well as all the other design that becomes coupled to the code you end up throwing out. Of course, sometimes you end up throwing out some stub code, but think about the possibility that you will wind up getting more features done, earlier in the cycle where we can get feedback and reduce risk. Why don’t we look at whether, in aggregate, more projects will succeed using YAGNI than will succeed using “Build everything we might need”?

Objection: “Some bugs simply can’t be found through automated unit tests, sorry. This is why we have to use a programming language with static typing.”

Response: Sure enough, some can’t be detected with automated unit tests. But our choice of a dynamic language provides us with many other benefits, most especially in the areas of metaprogramming and code reduction. By writing less code, we may even have fewer bugs overall. Shouldn’t we try to compare similar projects written in a static language against those written in a dynamic language, and see whether the projects in the dynamic language had fewer bugs and whether the projects written in the dynamic language were more likely to be successful?

Good luck handling objections. What is your experience: do you have another technique you can recommend?

A Personal Note

As Mike pointed out in the first comment, this post explains how to handle this one type of objection, the gotcha objection, by moving the debate away from the exception case and towards the overall case. But it does not follow up by presenting hard data to justify the example ideas presented.

First, I want to say that even with hard data you will not foster change with numbers: you need to show how your idea addresses an urgent priority. That should have happened before you got to this point. If you have addressed the problem correctly, it really is sufficient to point out the fallacy in the objection and allow your original argument to stand.

Second, there is a dearth of hard data about anything to do with software development. Repeat after me: “the plural of anecdote is not data.” If you have a source of hard data about any practice, be it programming languages, practices, or even interviewing techniques, I would very much like to read and learn from it.

Does this mean that we should never change, that since there’s no proof that new ideas are an improvement over old ones?

If you are happy with your current situation, maybe not. If you are unhappy with your current situation, if you want things to be better, you may want to change something. It’s your call.

Labels: ,

 

Comments on “What I've Learned from Sales, Part II: Wanna Bet?:
Nicely put.

But isn't this just Step One in the discussion? You've persuaded your interlocutor to compare the (probabilistic) results of Agile and other techniques in real-world projects. How do you then convince her that those Agile techniques will give better results on her project than the alternatives?

I can't make it to the talk on Tuesday, so consider my entry into the Q&A!
 
The important thing to remember is that fundamentalism does not lead to better software, but can result in smug developers.
 
Regarding studies on software development productivity, I noticed this yesterday on reddit. Steve McConnell summarizes the results of a study (unfortunately not available for free) comparing various factors' contribution to software development productivity. I'm sure this study isn't exhaustive and, based on its publication date, doesn't mention recent innovations.


McConnell's Editorial


Article (with abstract)

 
I echo Mike's confusion.

What's the point of saying something is probabilistic
if there is no data on which way the odds point?

Ok YAGNI is probailistic. Fine . But so is Up Front Design (say). Now what? Which one to choose?

Unless YAGNI has a success probability of x + delta where x is the probaility of success of UFD, this argument can cut both ways.

And as you correctly point out one has only anecdotes in either direction. So "probability" comes very close to "opinion".

I am sorry if I missed something obvious but I just don't get it.
 
Just to clarify, what I don't get is the benefit of saying technique X is probabilistic.

All you are saying is that it sometimes succeeds and sometimes fails. But that is true of *everything* in software.


Unless Prob(X succeeds) > Prob(X is a failure) and Prob(X succeeds) > Prob(Y -where Y is a competing way of doing this - succeeds) - and there seems to be no way of establishing this without hard data or rigorous experimentation - there *seems* to be no advantage in stating this as a supporting argument for trying X.
 
Mike and Ravi:

The answer is that nobody has proven that one software development idea or another is better using anything remotely resembling the scientific method.

(Actually, the only conclusive evidence seems to be that "hiring good people" and "not telling them to pretend to be bad people" are the things that work best).

Sorry, there is no scientific proof. Functional Programming? Object Oriented Programming? Lisp? C? Java? XP? Scrum? Good Agile? Does anybody have evidence that any of these things is better than any other?

The point of Part I was to establish a good reason for your new idea. Let's say your new idea is "OOP." (From my observation, it's a very new idea to many Java shops).

The good reason is to address an urgent problem. Let's say that the urgent problem is good people defecting because your shop is "unsexy" with its procedural style.

If you establish retention as a good reason to use OOP, and THEN somebody (the grizzled FORTRAN veteran) says ...but, here's this one situation where OOP is more unwieldy than old fashioned procedural programming, you use Part II to move the conversation to overall benefit rather than an edge case.

If someone says, "but there are no rigorous studies one way or another," you can say "That's right."

But remember the reason we talked about before Mr. Fortran objected? Retention? That's still why we should learn OOP.

This article is about neutralizing an objection. It's not some sort of jiu-jitsu where you not only neutralize the objection, you flip it over into a reason to accept your idea.

And it especially isn't an article about test driven development, dynamic programming, or any other specific practice.

I'm sure that if I quoted the little evidence I do have about some practices, we would have an even longer thread disputing the evidence when the article is about a specific pattern for handling a specifc kind of objection.
 
Ok I think I get it now.


You are using "probabilistic" as a smack down for folks who point to occasional/edge case failures of the proposed change as a reason for not making that change.

' "OOP." (From my observation, it's a very new idea to many Java shops).'

heh!

"
 
Actually, there's a good argument for various agile/quick-turnaround methodologies hiding under the probabilitic arguments: By having many short iterations that deliver code to users, you're reducing the bet size. This avoids gambler's ruin, and lets you walk away from the table at any time.

Of course, if you don't even understand how to play the game (which happens all too often in software, especially out on the cutting edge), that might be another reason to bet small.
 
By having many short iterations that deliver code to users, you're reducing the bet size. This avoids gambler's ruin, and lets you walk away from the table at any time.

This was the central message of a relatd talk I gave at XP/Agile Toronto recently: that lightweight, iterative development maximizes the profitability of the Casino.

The Casino doesn't mind losing games because the impact of each game is deliberately very small.

I contrasted this approach with the "entrepreneur" who is betting her entire company. Every business decision places the entire company at risk, and therefore she attempts to win every game she plays, at all costs.
 
Human's make decisions based on emotions not logic.

We use "facts" to prove the emotional decision we've already made about something...

(A vegetarian and a meat-eather could show you their set of "facts" "proving" their case.)

I suggest a better approach is to target a person's emotions directly rather than through logic..

Comparing your "facts" with their opposing "facts", sometimes creates an endless debate.. Another approach is NLP...

http://en.wikipedia.org/wiki/Persuasion_uses_of_NLP

Instead of dis-proving or trumping their "facts". Understand their emotional attachment to the concepts and bring an alternative perspective that they can also agree to.

Cheers,
Rajesh Duggal
 




<< Home
Reg Braithwaite


Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

Books
What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

Share
rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

Beauty
IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

Work
The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Management
Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

Notation
The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Opinion
Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

Whimsey
The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

History
06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /