Closures and Higher-Order Functions There has been a great deal of interest in closures lately, driven in great part by the fact that there is talk of adding some form of anonymous functions to the Java. Most of the time, people talk about “adding closures” to Java, and that prompts a flurry of questions of the form “what is a closure and why should I care?”

The discussion around closures tends to go on and on about the “closing over” of free variables and only lightly touch on the biggest change to Java: functions as first-class objects with a lightweight syntax for creating them. Making it easy to do something basic like define a new function is more than just a little syntactic sugar: it makes it easy to do new things with functions that were impractical when you needed a lot of boilerplate to make anything work.

Without understanding functional programming, you can’t invent MapReduce, the algorithm that makes Google so massively scalable.

I’m going to try to explain first class functions using Ruby (it is possible to write code that does exactly the same thing using the current Java feature set, however the result is so wordy that it obscures the basic idea being presented: call it accidental complexity, or perhaps yellow code.)

Ruby is a good language for demonstrating features that ought to be in Java. Like Java, Ruby uses squiggly brace syntax. Like Java, everything in Ruby is an object—whoops, Java has primitives. Okay, like Java, functions are represented as objects.

In Java you write:

interface IFromIAndI {
   Integer call(Integer a, Integer b);
}

IFromIAndI add_two_integers = new IFromIAndI() {
   public Integer call(final Integer a, final Integer b) {
      return a + b;
   }
};

(The Java convention is to name things in lowerCamelCase, but we’ll ignore that. If you need to print this essay on a dot-matrix printer you may want to make some changes first.)

In Ruby you write the function as:

add_two_integers = lambda { |a,b| a + b }

Later on, when you want to call your function in Java, you write:

add_two_integers.call(35, 42);

And if you like semicolons, you write the exact same thing in Ruby:

add_two_integers.call(35, 42);

You can do the same thing with multiplication:

multiply_two_integers = lambda { |a,b| a * b }

First Class Functions

In the examples above, functions look a little like methods. The Java version is obviously implemented as a method. But what we did in both cases was assign the resulting function to a variable. In Java, assigning a method to a variable is not particularly easy (it is possible using reflection).

Anything that can be assigned to a variable is a value. If it can also be passed as a parameter or returned from a method (or function), we say it is a first class value. Functions as first class values, or first class functions, are very interesting. For example, what can we do passing a function as a parameter to another function?

Hmmm. Well, I am breaking a cardinal rule of selling something. We’re talking about shiny new toys without identifying a problem to be solved. Let’s talk about my favourite problem: writing the same thing more than once, violating the DRY principle.

Here are two pieces of similar Ruby code:

adder_wth_acc = lambda { |acc, list|
   if list.empty?
      acc
   else
      adder_wth_acc.call(acc + list.first, list[1..-1]) # [1..-1] returns a copy of the list without the first element
   end
}
adder = lambda { |list|
   adder_wth_acc.call(0, list)
}
adder.call([1, 2, 3, 4, 5])

And:

multiplier_with_acc = lambda { |acc, list|
   if list.empty?
      acc
   else
      multiplier_with_acc.call(acc * list.first, list[1..-1]) # [1..-1] returns a copy of the list without the first element
   end
   }
   multiplier = lambda { |list|
   multiplier_with_acc.call(1, list)
}
multiplier.call([1, 2, 3, 4, 5])

What do they both do? Pretty much the same thing: they accumulate the result of some binary operation over a list of values. adder accumulates addition, and multiplier accumulates multiplication. You could call this a “Design Pattern.” If you did that, you would use the exact chunk of code everywhere. I would call that retrograde. Didn’t our predecessors invent the subroutine so we could eliminate writing the exact same piece of code over and over again?

Why can’t we do the same thing? Well, we can. A subroutine does the same thing over and over again, but it takes different parameters as it goes. What is different between adder and multiplier? Ah yes, the adding and multiplying. Functions. What we want is a function that takes a function as a parameter.

Well, we said that with first-class functions, functions are values and can be passed as parameters. Let’s try it:

folder = lambda { |default_value, binary_function, list|
   fold_with_acc = lambda { |acc, list|
      if list.empty?
          acc
      else
          fold_with_acc.call(binary_function.call(acc, list.first), list[1..-1])
      end
   }
   fold_with_acc.call(default_value, list)
}

Now we can use our function that takes functions as a parameter:

folder.call(0, add_two_integers, [1, 2, 3, 4, 5])
folder.call(1, multiply_two_integers, [1, 2, 3, 4, 5])

This is much better. When functions can take functions as parameters, we can build abstractions like folder and save ourselves a lot of code. Note that this would be a lot harder to read if we had to surround all of our functions with Object boilerplate in Java. That’s one of the key reasons why ‘syntactic sugar’—making it brief—is a big win.

And you know what? Functions are values, not just variables that happen to hold functions. These work just as well:

folder.call(0, lambda { |a, b| a + b }, [1, 2, 3, 4, 5])
folder.call(1, lambda { |a, b| a * b }, [1, 2, 3, 4, 5])

There’s just one problem (actually two, but I’m saving one for later): everywhere you use our new folder function, you need to remember that add_two_integers needs a default value of zero, but multiply_two_integers needs a default value of one. That’s bad. Sooner or later you will get this wrong.

What we need is a way to call folder without having to always remember the correct initial value. Should we extend our understanding of a function to include a default initial value for folding? If we’re thinking in Java, maybe our IFromIAndI interface needs a getDefaultFoldValue? I think not. Why should a function know anything about how it’s used? And besides, as we build other abstractions out of functions we’ll need more stuff.

If we aren’t careful, we’ll end up implementing the Visitor pattern on functions, and all of our brevity will go out the window. No, what we want is this: in one place we define that folding addition starts with a default value of zero and in another place we say we want to fold, say, [1, 2, 3, 4, 5] with addition. Then when we want to fold something else with addition, like [2, 4, 6, 8, 10], we shouldn’t have to say anything about zero again.

Adding Curry

What we need is a function that folds addition. Didn’t we say that functions are values that can be returned from functions? How about a function that makes a folding function? We should pass it our initial value and our binary function, and it should return a function that performs the fold without needing an initial value as a parameter:

fold_coder = lambda { |default_value, binary_function|
   fold_with_acc = lambda { |acc, list|
      if list.empty?
          acc
      else
          fold_with_acc.call(binary_function.call(acc, list.first), list[1..-1])
      end
   }
   lambda { |list|
      fold_with_acc.call(default_value, list)
   }
}

Now we can do the following:

adder = fold_coder.call(0, lambda { |a, b| a + b })
adder.call([1, 2, 3, 4, 5])
adder.call([2, 4, 6, 8, 10])

No more remembering that addition starts with a default of zero.

Actually, there’s a far simpler way to avoid having to remember the default value when you want to fold over addition. But let’s just play along so that we don’t have to come up with an entirely new set of examples to demonstrate the value of functions as first-class values.

Functional programmers (as opposed to the rest of us dysfunctional programmers) will recognize this as currying our folder function. Currying is when a function takes more than one parameter and you combine one of the parameters and the function to produce a function that takes fewer parameters.

Here’s a currying function in Ruby:

curry = lambda { |fn,*a|
   lambda { |*b|
     fn.call(*(a + b))
   }
}

(This is an improvement on an earlier version, thanks to Justin's comment.)

So you can use our new function to create an increment function out of our adder and a treble function out of our multiplier:

plus_one = curry.call(add_two_integers, 1)
times_three = curry.call(multiply_two_integers, 3)

If you are ever asked, “what good is currying?,” I hope I’ve given you an example you can use to explain why currying matters, and why people do it all the time (possibly without explicitly naming it). Although it doesn’t look like much when looking at trivial examples like functions that multiply by three, it’s much more useful when creating folders and mappers where you want some of the parameters to remain constant.

Composition

Our examples combined functions and non-functions to create new functions. Here’s an example from a recent post, Don’t Overthink FizzBuzz, where I give a method for composing two functions. The idea is that if you have multiple functions that each take one argument, you can combine them using compose. I also have a method that generates functions, carbnation:

# fizzbuzz.rb

def compose *lambdas
  if lambdas.empty?
    lambda { nil }
  elsif lambdas.size == 1
    lambdas.first
  else
    lambda do |n|
      lambdas.first.call(compose(*lambdas[1..-1]).call(n))
    end
  end
end

def carbonation(modulus, printable_form)
  i = 0
  lambda do |n|
    (i = (i + 1) % modulus) == 0 && printable_form || n
  end
end

print(((1..100).map   &compose(     carbonation(15, 'FizzBuzz'), carbonation(5, 'Buzz'), carbonation(3, 'Fizz'))).join(' '))

The simple explanation of how it works is that carbonation generates functions that replace every so many elements of a list with a printable string. Compose composes any two or more methods together. So if you want to print out 100 numbers, but replace every third number with “Fizz,” every fifth with “Buzz,” and all those that are third and fifth with “FizzBuzz,” you generate a function for each replacement, compose them together with compose, and then map the numbers from one to one hundred to the resulting überfunction.

When you look at this today, it seems weird and unreadable by Java standards. I wonder if adding first-class functions with simple syntax to Java will lead the Java community to a place where code like this will not appear out of place?

Just one more thing

So we started by saying that people are getting hung up on what makes a closure a closure, and there has been less emphasis on the benefits of using functions as first-class values. Did you notice that our folder function actually includes a non-trivial closure?

If you look at the fold_with_acc function, it makes use of binary_function, a variable from its enclosing lexical scope. This is not possible with the current version of Java: if you translate this to Java, when you make fold_with_acc and anonymous inner class, you will have to copy binary_function into a final member to use it. It simply won’t compile if you try an idiom-for-idiom translation, even adding explicit types.

And then if you look at the anonymous function it returns, lambda { |list| fold_with_acc.call(default_value, list) }, that anonymous function uses default_value,another variable from the enclosing lexical scope. Once again you will have to fool around with final variables to make this work, or perhaps declare full-fledged object with constructors.

(If you try writing this simple example out in Java, you quickly find yourself inventing a lot of classes or interfaces. And they have some complicated types, like a function taking an integer and a function taking two integers, returning a function taking a list of integers and returning an integer.

After twenty minutes of that, you understand why the ML and Haskell communities use type inference: If the types are that complicated, it’s incredibly helpful to have the compiler check them for you. Yet if the types are that verbose, it’s incredibly painful to write them out by hand. Even if your IDE were to write them for you, they take up half the code, obscuring the meaning.

You also get why the Ruby on Rails community doesn’t care about type checking: types for CRUD applications are way less complicated than types for first-class functional programs.)

That’s Interesting

Part of the interest in closures is in simplifying the syntax around functions, and part of the interest is in the way that access to enclosing scope would simplify a lot of code. There’s a whole debate around the value of simplification in a world where all serious languages are Turing Equivalent.

I hope you’re convinced, by now, that programming languages with first-class functions let you find more opportunities for abstraction, which means your code is smaller, tighter, more reusable, and more scalable.

For me, simpler is just nicer until something reaches a certain tipping point: when it becomes so simple that the accidental complexity of using it goes away, I will start using it without thinking about it. Tail optimization is like that: as long as recursion is slower than iteration and sometimes breaks, I have to think about it too much. But when I’m not burdened with “except…” and “when performance is not a factor…” it becomes natural.

And then something interesting happens. It changes the way I look at problems, and one day I see a whole new way to do something that I never saw before. Functions as first class values are definitely one of those things that change everything.

Further Reading

If this has whet your appetite for more, Structure and Interpretation of Computer Programs is the book on higher-order functions and how they can be used as building blocks to create more elaborate abstractions such as object-oriented programming.

The Seasoned Schemer devotes an entire book to the uses of functions. Although the examples are in Scheme, the language is dead simple to learn and the techniques in the book can be applied to Ruby and Java (or at least to a future version of Java where you do not need functors).

The second edition of Programming Ruby is an indispensable guide. Even if you will not be using Ruby immediately, pick it up and discover why so many people are lauding the language's simple, clean design and powerful Lisp-like underpinnings.

As the author says, “Higher-Order Perl: Transforming Programs with Programs is about functional programming techniques in Perl. It’s about how to write functions that can modify and manufacture other functions.

“Why would you want to do that? Because that way your code is more flexible and more reusable. Instead of writing ten similar functions, you write a general pattern or framework that can generate the functions you want; then you generate just the functions you need according to the pattern. The program doesn’t need to know in advance which functions are necessary; it can generate them as needed. Instead of writing the complete program yourself, you get the computer to write it for you.”

It’s worth reading even if you have no intention of using Perl: the ideas span languages, just as SICP is worth reading even if you don’t use Scheme at work. And be sure to read Higher-Order JavaScript and Higher-Order Ruby. They translate HOJ’s ideas to other languages.

Notable Follow-ups:

Some useful closures, in Ruby: “The Dylan programming language included four very useful functions for working with closures: complement, conjoin, disjoin and compose. The names are a bit obscure, but they can each be written in a few lines of Ruby.”

From Functional to Object-Oriented Programming: “OO allows a traceable connection between the conceptual design level and the implementation level. Concepts have names, so you can talk about them, between programmers and architects.”

HOF or OOP? Yes!: “First-class functions are a natural fit with OO, as evidenced by their presence in OO languages that aren’t glorified PDP-11 assemblers with some OO stuff bolted on the side.”

But Y would I want to do a thing like this?: “To truly learn a new tool, you must not just learn the new tool, you must apply it to new kinds of problems. It’s no good learning to replace simple iteration with first-class functions: all you’ve learned is syntax. To really learn first-class functions, you must seek out problems that aren’t easily solved with iteration.”

Labels: java, lispy, popular, ruby

  ¶ 11:55 PM

  Please design a deck of cards in Java Dear Future Colleague:

I’ve been asked to perform a quick technical evaluation of your résumé before we ask you in for an interview. You obviously have the right kind of experience and I’m looking forward to meeting you soon.

We are asking all candidates to perform a little exercise before coming in for the first meeting. I realize you have a great deal of relevant experience, so this will not take very much of your time at all.

And I’m sure you appreciate the fact that we value ability and merit above certifications and other pieces of paper.

Please design a deck of cards in Java

Imagine that we are writing a casino program. We are not going to write the whole program right now, just the part of the program that has to handle the decks of cards to be used by the rest of the program.

If you are not familiar with the kind of cards used in European card games (like Poker, Bridge, Blackjack, Crazy Eights, Rummy, and so on), please write back and I will give you a different problem. You should be comfortable with concepts like the ranks of the cards, the suits, shuffling, and dealing.

Your task is to design the interfaces and/or classes we would need for the deck(s) of cards. At a bare minimum, your classes should handle everything we need to know about decks of cards, the cards themselves, and operations on decks of cards like shuffling, dealing, removing cards from a deck (some games have one or two jokers, some don’t, some only use the cards from 9 and up, some from 7 and up, some use them all), and combining decks (Canasta uses two, four, six, or more decks of cards with jokers!)

You do not need to worry about the mechanics of games like hands of cards, ranking Poker hands, knowing about Trumps or Bowers (as in Euchre and Bridge), discard piles (as in most games), or melds (as in Gin, Rummy, and Canasta). Imagine that other people will write all of that stuff using your classes for decks and cards.

Please be specific about the interfaces, abstract classes, and concrete classes you will need. There is no need to actually write the bodies of methods, but please be specific about what the methods are called, what parameters they take, and what they return (if anything).

I am especially interested in the relationships between the classes. Although you don’t have to write out each method, if you expect that a method would call other methods in your design, please say so.

For example:

class Deck {

// calls Card.arrangeRelativeTo(Card othercard)
public void shuffle () { ... }

}

You can see that in this hypothetical design cards have some ability to arrange themselves relatively to each other and I am documenting that Decks can be shuffled and they use the Cards’ arrangeRelativeTo method to do so. Obviously, you don’t need to follow that design. It’s just an example.

This extends to constructors. Does creating a Deck create Cards? Or does it use Cards that already exist? Or are Cards not even objects but are represented by primitive types?

About UML

You can just write out the skeleton of the classes and interfaces. If you prefer to use UML, be my guest. As long as your design answers all of the questions I have, use whatever notation you prefer. Just don’t fall into the trap of letting the notation drive the design: just because a certain type of UML diagram doesn’t indicate the CREATES-A relationship, that doesn’t mean you shouldn’t add it: what’s important is the design, not the document.

Requirements?

Feel free to email or call with any questions you might have about requirements. This is not meant as a test of your ability to read my mind about what I want or to ask insightful questions. So, it’s ok to ask me questions. If you have a choice to make, it’s also ok to just decide.

For example, you might wonder how to represent which cards are left in a deck after dealings some out. Should you use a standard OO pattern like a List or Set? That might be fine if our Casino program is a single-user game. But if we are running a web-based virtual casino with many thousands of users, performance might be important enough that we consider using something like a bitfield for each deck.

If you want, you can ask what’s more important. Or you could just decide for yourself. Using a bitfield for performance is a fine decision. So is having a standard OO design. Each works well for a particular set of requirements.

So don’t be too concerned about asking a million questions. I just want to get a basic idea of your comfort level designing OO software.

Thanks,




Reg Braithwaite

p.s. This isn’t a trick question. There’s no right answer, and as I said above I don’t even expect full implementations, so you don’t even have to write code that compiles.

There is absolutely no need for a bunch of stuff I didn’t ask about, like requirements in three-ring binders or JUnit test cases. I know that you are probably tempted to show off your depth of experience and SDLC knowledge, however I really only want to know how you would approach the design in this question.

Please save the other stuff for the interview. Just because I don’t need to know about it right now doesn’t mean I don’t agree that it’s important!

Labels: java, jobs

  ¶ 10:00 AM

  The false dichotomy of choosing between your values and expediency Wil Shipley:

Sometimes the most important thing in a fight is which side you are on, and not whether that side can win or not. Sometimes we just have to believe that, eventually, good will win out, and we have ask ourselves, “How will my children judge me when I tell them the story of this time?”

When my kids ask me about the first black man who really had a shot for the presidency, will I be forced to dissemble: ‘Well, see, kids, at the time, I didn’t think he could win, so it seemed expedient for me…’

This blog is not about politics, and it is especially not about politics in another country. If you are one of my American readers, I expect you to decide for yourself what is the right thing to do with your vote, time, and money. That being said, I quoted Wil because I admire something that he is saying there, something that is on topic for this blog.

Let’s start by dispelling one ugly myth about voting “strategically” (as we say in Canada) or for someone “elect-able” (as you say in America): It isn’t about the outcome of the election.

Let’s start by dispelling one ugly myth about voting “strategically” (as we say in Canada) or for someone “elect-able” (as you say in America): It isn’t about the outcome of the election. That’s a little white lie people tell themselves so they can do something they are embarrassed about.

They are really thinking “Here’s this thing that I ought to want to do, vote for the best man who happens to be Black, but—truth be told—I don’t want to do it. But I absolutely don’t want to admit I don’t want to do it, so… I’ll make up this story about expediency and how important it is for a Donkey to beat an Elephant, so I’ll nominate this other guy to run.”

Did you read my “Don’t Feed the Trolls” post? One of the key lessons was that people often are embarrassed about their real motivations for making decisions, so they dress them up in objections that they think sound reasonable to other people. Like winning elections. I’m going to come back to this in a moment. You are a stack-based reasoner, right? So PUSH the idea that what people say about why they are doing things rarely matches why people are actually doing things.

Let’s talk about software development

Think about your software development values. A value is something you are willing to pay for. I can’t tell you what your values ought to be, but I can tell you one thing that absolutely isn’t a value. Making money is not a value.¹ It’s an expediency.

Money is almost always a “reasonable objection to tell other people.” For example, taking a job with a consulting firm because “it pays more money.” When what you really want is the prestige of going into workplaces and telling people what to do. Without bothering to actually, you know, have any real industry experience first.

Synonyms for “making money” include “getting a job,” “sales,” “raising capital,” amongst many others.

Now we have already talked about people that have a value but can’t admit it so they tell a little white lie about “expediency.” It’s not just politics. Do you think our choices in software development are any different? Of course not. We are the same whether we are choosing mates, voting, or picking a programming language. So the way we choose is the same.

But there’s an extra dimension we are not considering. If you read The Wisdom of Crowds, you remember this idea of an information cascade, where information about other people’s choices actually makes the crowd more stupid, not more wise.

Synonyms for expedient include “backwards compatible,” “maintainable,” “easy to understand,” “easy to sell to management,” and many others.

The problem is this: when we see a bunch of people doing something, it seems like a reasonable, sane option. So do you want your fourth Java Certification? It seems reasonable, a lot of people are getting their certification, it’s an expedient way of getting a job.

Or let’s pick something that isn’t such a blatant straw man. Do you want to put your business logic into stored procedures instead of in your model objects? Or perhaps do you want to build yet another rich MVC web application instead of something REST-ful or perhaps one that is continuation-based (like a Seaside application)?

Really? Or is it just expedient to do it that way (synonyms for expedient include “backwards compatible,” “maintainable,” “easy to understand,” “easy to sell to management,” and many others).

The problem with the little white lies

So you’re a reasonable person, and you see a lot of other reasonable people doing something because it’s expedient. You value something, but you are wondering whether you ought to compromise your value in favour of expediency. Life is tough some times, and you have to grow up. What’s wrong with that?

Here’s where we go to the stack: POP. Remember, lots of those other people aren’t actually doing the expedient thing when they say they’re being expedient! They’ve got a whole ‘nother motivation, but they lied about that motivation to you. It isn’t a real reason, they aren’t being honest with you (or themselves) about why they’re doing it, and you go along with it thinking it’s a real, valid option. But it’s smoke and mirrors for other people’s frailties (bless them, they are as human as we are).

For example, the argument that BDUF is the expedient way to manage a project? Look, I’ll level with you: if what you want is to demonstrate to your manager that you have everything under control, that you are in charge, then go for it. But expedient? Sorry, that’s bunk. It’s just a little white lie a lot of people have told you because they fear telling the truth, like:

• They have no idea how to manage software development activities, so forcing developers to specify everything in advance relieves their anxiety that they will have to actually manage things as they go along. Instead, they can just follow “The Plan.”


• They have no idea how to manage software development to a deadline, so forcing developers to estimate every activity in advance—no matter how far the activity is away—relieves their anxiety that they will have to ever explain whether a project is on target for release or not. Instead, they can simply parrot how much of “The Plan” seems to be complete and whether actual times for activities are ahead or behind estimates.


• They have no idea how to manage stake holder expectations and priorities, so forcing stake holders to specify all of their requirements in advance relives their anxiety that they will have to engage stake holders in a continuous and open dialogue over the life of the project.


• The road to senior management in their organization is paved with process documentation, not successful projects.

Yes, we have to grow up and consider expediency. But we also have to grow up and learn that other people are human and don’t always tell us the whole truth about why they do what they do. Which means it is difficult to judge whether something is right for us based on what other people say is right for them.

Be careful when presented with a choice between what we value and what is “expedient.” This choice is almost always a false dichotomy: the “expedient” option is usually a little white lie someone else has told us because they think it will sound reasonable.

We need to be very, very, very careful when presented with a choice between what we value and what is “expedient.” This choice is almost always a false dichotomy: the “expedient” option is usually a little white lie someone else has told us because they think it will sound reasonable, even if it isn’t what they were thinking.

Therefore my exhortation to you is this: If you really, truly want to program in Lisp, or manage your next project with BDUF, or write unit tests before you write code, because these actions reflect your values, then go ahead. I honour you for your choice, even if it’s different than my choice. Rock on!

But… if you really, truly want something else but are doing the expedient thing because that’s where the money is, or the jobs are, or there’s safety from criticism, or any of a hundred or a thousand or ten thousand little white lies, stop it right now. You need to understand that the chattering voices you hear, the ones that sound so reasonable (“Zune sucks, but MSFT always wins in the end, so buying a Zune is expedient”) don’t even reflect what those other people really think or want.

They’re just little white lies other people are telling you because they don’t have the courage to tell you the truth about themselves.

---

1. If you’re reading this blog and telling me that you—to pick just one example—choose to program .NET because “that’s where the money is,” you are misguided. Sorry, but the money is actually on Wall Street or Bay Street where even the poorest analyst makes more money with convexities and gammas than you or I will ever make with objects and tables.

Labels: passion

  ¶ 2:40 PM

  The shortest post yet written on the realtionship between Ruby Metaprogramming and slavish adherence to the Gospel of the GoF Ruby Metaprogramming elevates the Decorator Pattern from a useful tool to a core idiom for creating idioms.

Labels: ruby

  ¶ 3:19 PM

  Don't Overthink FizzBuzz I noticed some traffic coming to my post on juggling from Using FizzBuzz to Find Developers who Grok Coding. Like me, the author is having trouble with the fact that 199 out of 200 applicants for every programming job can’t write code at all. I repeat: they can’t write any code whatsoever.

UPDATE: If you think that I just claimed that 199 out of 200 working programmers cannot code, stop immediately and either read my follow-up explaining the math, or read Joel’s article on why companies aren’t as picky as they think they are. Thank you.

So the author has decided to set the bar ridiculously low. As the post says:

I’ve come to discover that people who struggle to code don’t just struggle on big problems, or even smallish problems (i.e. write a implementation of a linked list). They struggle with tiny problems.

He gives an example of a “tiny” problem that is quite suitable for a phone interview:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

One of the nice things about using a tiny problem is that anyone who has written actual code in the last six months will solve it in less than ten minutes. This is good because (a) it leaves more time for talking about important things, and (b) if the candidate can’t solve the problem you will find out right away and save yourself a lot of grief.

As you can imagine from a post like this, quite a few people have stepped forward with creative solutions. Here’s mine:

# fizzbuzz.rb
# see http://weblog.raganwald.com/2007/01/closures-and-higher-order-functions.html

def compose *lambdas
  if lambdas.empty?
    lambda { nil }
  elsif lambdas.size == 1
    lambdas.first
  else
    lambda do |n|
      lambdas.first.call(compose(*lambdas[1..-1]).call(n))
    end
  end
end

def carbonation(modulus, printable_form)
  i = 0
  lambda do |n|
    (i = (i + 1) % modulus) == 0 && printable_form || n
  end
end

print(((1..100).map
   &compose(
     carbonation(15, 'FizzBuzz'), 
     carbonation(5, 'Buzz'), 
     carbonation(3, 'Fizz'))).join(' '))

Right away I think you want to be careful if you ever trot a solution like this out in an actual interview. Here are some of the things that could go wrong:

• The interviewer merely wanted to go through the motions of demonstrating you could program. Now she’s going to have to get into a discussion with you about how your solution works, which is irritating and wastes time that could have been invested more profitably on a design question;


• The interviewer might presume that you will always find the most indirect route to the destination and decide you belong in an ivory tower;


• The interviewer might not understand your solution at first glance but will make up for this by “bikeshedding” and grilling you about some minor nit like the failure modes of using && and || instead of the if statement or the ternary operator;


• The interviewer might have just finished The Seasoned Schemer and grill you over why your solution doesn’t use the Y Combinator to implement compose (snap!).

Of course, some interviewers are just looking for an excuse to talk about technical issues and will not prejudge you in any way. Speaking for myself, I would laugh if you wrote something like this out. But then again, I once worked for someone who had won the second International Obfuscated C Contest.

Update: Don’t Overthink the Interview Either

If you are conducting an interview and you receive a cryptic answer, or an impossibly concise one-liner, or perhaps something like the above that is redolent of Higher Order Functions, do you embrace the candidate as a genius? Or perhaps avoid them as someone who cannot write clear, maintainable code?

I suggest the best answer is neither. Stay on track: you asked the question for the purpose of eliminating the 199 out of 200 that have absolutely no hope of ever producing working software. You have just identified that this person is the one in two hundred applicants who can actually code. Mission accomplished. You aren’t trying to get to HIRE or NO HIRE over the phone.

Invite them in for an interview: that’s the place where you can drill down and obtain their views on professional programming. Remember, this is a toy problem. Unless you load the question down with requirements that their answer resemble production code, why should you have any expectation that the candidate only knows one way to write software?

This is like trying to hire programmers who know both languages, FORTRAN and COBOL.

If you do ask them for production code, don’t be surprised if the best candidates decide not to work for you: they will rightly point out that FizzBuzz is not a realistic example of a production problem.

Overall I think the best approach as an interviewer is the simplest: pose the simple question. Indicate it is a screener designed to confirm that they are the person described on the resume.

If anything, tell them that there is no need for production artifacts like documentation and unit tests, you just want working code in as little time as they are able. The no-hopers won’t get it no matter how simple the question, and the good people won’t waste a lot of time with JavaDoc and xUnit.

And then make your decision based on one objective fact: does the code run and produce the desired output. Everything else is superfluous to the question of whether the candidate should be asked into your office for a face-to-face meeting.

Update: a response to some of the criticism.

Labels: jobs, lispy, ruby

  ¶ 9:49 PM

  There's a Cross-Platform Tempest in Apple's Teapot. Again. So the iPhone won’t run Java. Specifically, no applets, no desktop applications, and none of the JavaME games and applications that are already running on 80% or more of the world’s existing smart phones.

Well, we’ve seen this before. Do you remember that the Macintosh did not run DOS applications? For good reason: a DOS application is nothing like a Macintosh application. And guess what? A JavaME application is nothing like an iPhone application.


iPhone applications have been carefully designed. They use the new iPhone interface. They integrate with each other. The entire purpose of the iPhone is to make phone applications work, make them useful and accessible for the vast majority of people who are not gadget enthusiasts.

Of course, someone out there loves their XYZ application and will say, “the reason to run JavaME is so that I can buy an iPhone and run XYZ.” Sorry, that’s not a reason. That’s a reason for that person to stick with their existing handset. Lesson one of product design is to decide who is not your customer as well as who is your customer. People who want to run their DOS applications on a shiny new iPhone are not part of the iPhone’s target market.

People who buy a phone, but don’t use 95% of its features, much less install new applications, are the iPhone’s market. People who buy a phone, learn how to use most of its features, and then install even more features by downloading JavaME programs? They are not a priority.

---

Although that makes compelling sense to me, there’s a much more important reason it doesn’t matter whether the iPhone runs JavaME. It’s the same reason It isn’t important whether Apple opens up development to third parties or whether there’s a FORTRAN-to-Cocoa bridge for iPhone. The reason is that native API development is so last century. Well, not really “last” century, but try raising VC funds (or YC funds) for your great desktop application idea.

Let me put it another way: the iPhone does run Java applications like ING Direct’s online banking (a personal favourite). It also runs Ruby applications like 37 Signals’ Basecamp. And Python applications like Tabblo. It’s 100% open: anyone can write an application for iPhone. In fact, this weblog is available to read on every iPhone sold. Right out of the box, no fancy installation!

Although the idea of skipping from WiFi sites to Cingular network and back and forth seems a little sketchy, I have a feling that the WiFi connectivity will turn out to be the killer application for iPhone. There’s lots of free public WiFi around, enough to make web servers a reasonable platform for iPhone applications.

So instead of worrying about whether iPhone runs the JavaME version of Tetris, why don’t we see how much of flickr works on an iPhone? Seriously, for every person who wants their JavaME XYZ application, how many people might use their del.icio.us bookmarks, their Google docs and spreadsheets, or even their office time tracking application on an iPhone?

Exactly.  ¶ 1:47 PM

  Why you need a proper and fundamental education to be successful in life There’s a twenty-five year high school reunion party. The class nerd shows up in his M3, he worked at a start-up, got some options, did ok for himself, everyone congratulates him. The class jock went into pharmaceutical sales, is a V.P. with one of the big drug companies, doing very well, everyone congratulates her.

Then the class stoner pulls up to the school in a limo, money practically falling out of his pockets, trophy girlfriend on his arm. What gives? They all figure he’s a coke dealer.

No, he explains, he was bumming around India when he met a guy with a rubber factory. The kind you wear. Everyone’s “ewww, gross.” The stoner continues: “This Indian dude makes them in printed patterns, I ask if he can make me one with my name on it, he says why not? I like it, I order a box more. So when I get home, everyone loves them, want some for themselves, great gag gifts.”

“So I write the Indian dude and start selling them with, you know, people’s names, and nicknames and stuff, they sell like hot cakes, always sell a ton of them around Valentine’s day.”

“Well, I can’t keep up so I get distributors, and people buy ‘em from me and package them up and sell them on, put ‘em in novelty stores, or in bars, I dunno, my distributors sell ‘em everywhere, now all I do is handle the paperwork and cash the cheques.”

“I pay around three cents each and in bulk I sell them for ten cents each. Cheap, yunno.”

“Isn’t it amazing,” the stoner asks, “how much money you can make on just seven per cent mark-up?”  ¶ 8:33 AM

  Business programming standards have become higher in 2007. Learn to love it. From time to time people suggest that fundamental computer science familiarity is irrelevant to the work of a business programmer. I am talking about a knowledge of recursion, operations on data structures, code generation, and other topics that are often derided as being “unnecessary” in a business programming context.

Hmmm.

I think this is wrong in 2007. It may not have been wrong in 2002, perhaps such knowledge was a bonus but not a basic requirement. But today, I think you need to have it. And I don’t mean, you need to have it on your resumé. I mean, you will use it on your job from time to time.

Now, I know that some readers are shaking their heads, no. And some are nodding their heads, yes. It’s easy to think this is all about culture, and some kind of weird hacker fraternity, or whatever. It’s especially easy to dismiss stuff you never use: if you never needed it before, why would you need it now?

That’s the Blub talking. Your toolbox is good enough because you've never needed anything else to do a job... up to now.

No matter what you think of Lisp, Google-style interviews (do you remember when they were “Microsoft-style interviews”?), or optimizing code, please put that aside and try to read this post as objectively as possible. I’ll lay out my thinking for you.

When you say “these things are not relevant for the job,” how do you know? Ok, you have twenty years of experience. And you’ve never used recursion or you’ve only used it once or twice. So you won’t need it now, you’ll find a way to use iteration. And who cares what a suffix array is, you’ll look it up in wikipedia if you need to implement one.

That’s what people have done for quite a while: wire existing frameworks together. What programmers need to know is how to Google stuff, and what a programmer need on her desktop is an IDE that auto-completes stuff so she’ll know what the methods are called. And of course, static typing to make sure she gets the method parameters right. Good to go.

There are jobs out there like that. Last year’s jobs.

Well, there are jobs out there like that. Last year’s jobs. But how do you know the one I’m filling in 2007 is one of those? Because I told you that for this position we are working with one of the world’s largest financial institutions on a public-facing J2EE application that has been in service for more than five years?

Now, I agree that you have figured out 98% of what we do. Most of the time, we mess with XSLT, message queues, JDBC through a DAL, and other buzzword compliant tasks. The keys to success for those items is less about programming brilliance than around discipline, process, requirements management, and the other stuff the “no hard CS” folks want to talk about in the interview.

And of course, we care about skills in those areas. We have to, we can’t hire someone who can distribute data sets across a grid but is unable to negotiate requirements effectively with a business analyst. But let’s talk about the other 2%.

The top two percent

From time to time we get some challenges. Here are some recent examples:

As part of a refactoring effort last year, we wrote some Java that used Reflection and Dynamic Proxies to replace an entire layer of the application that used to include extensive hand-coding of stuff that was repetitive and error-prone. This saves us 80-90% of the code in that layer when we add new stuff to the application. A testing utility the year before used Reflection to automatically write a certain JUnit suite.

You know how bit-twiddling in Java is irrelevant because you’re waiting for the database any ways? Well, we can’t afford to wait for this particular function, it’s one of those AJAX-y things that happens in real time. We can’t wait to go back to the database.

We’re working on something right now that is highly performant. We have a seven-figure user base, and peak loads are intense. You know how bit-twiddling in Java is irrelevant because you’re waiting for the database any ways? Well, we can’t afford to wait for this particular function, it’s one of those AJAX-y things that happens in real time. We can’t wait to go back to the database. So we have to load something into memory on the server, build a compact data structure, and traverse it quickly. And oh yes, we can’t have a lot of layers of crap, we need to get a response back to the browser with every key press.

For another client, we had to build a task dispatching system. It was like building a piece of a very lightweight fault-tolerant operating system. That operated across data centres in three different cities, moving jobs around from centre to centre. If you were in an interview and someone posed one of those hypothetical “how would you design a …” problems, would you think they stole the problem from some Amazon programmer’s weblog? Would you think “you don’t need that for business programming?” Well, we built that. For an ordinary, brick and mortar business that makes physical stuff.

That 2% figure? That was in 2006. In 2007, it will be way more. Standards are rising. We’re doing more and more work that steps outside of the usual CRUD development.

Here’s the big reason why. Have you read people grousing about interviews where they’re asked about how to implement a search? And about what a waste it is because 99% of the time the database does it, and the rest of the time they stick it in a hash table? Well, in 2007 search matters. The database is a big part of that, but it’s not as easy as SELECT foo.* FROM bar WHERE ... any more.

The Google Effect

Google has become the “start page of the internet.” As a result, everyone now thinks that the way to find stuff is to do a full text search. Everyone thinks that relevant results should be first. And I mean everyone, not just your “early adopter” users, you now have Joe Average calling your customer support hotline and complaining that the search page on your application—the one with a different field for each column in each table—is too hard and why can’t he just type something and get an answer?

This stuff isn’t rocket science. And you don’t need Common Lisp or Haskell to pull it off. You can do it in Java or C#.

Just how do you plan to implement full text search? Buy it from Google or Oracle? And do you think you can do the “Google Suggest” thing with the drop down? In real time?

Users now love having a single search box. They don’t want to have one box for searching on product SKU and a drop-down for searching on supplier name. One box. And if you want to make searching for supplier name easy, give them an auto-complete. Users don’t like to be given an application that basically has the implementation protruding into the interface.

They don’t care that you store first name and last name in separate columns, they want to search for “Reg Braithwaite” and find him, even if “Reginald” is what’s stored in the first name column and there are 3,215 Braithwaites in the table. You figure it out, possibly by word stemming, possibly by statistical analysis. Or maybe you’re just doing some untrained bayesian classification to cluster the “Reginald Braithwaite” record with some other things the user is looking at right now so you put that record at the top of the list you returned.

Hmmm, we’re not in Kansas any more. It isn’t all about has_one, has_many, or has_and_belongs_to_many, and you don’t have to be a high-profile start up to care. Jane Average uses stuff like this when she reads her mail and books her vacation. But does her office HR support application work half as well? Why not?

This stuff isn’t rocket science. And you don’t need Common Lisp or Haskell to pull it off. You can do it in Java or C#: we do it and there are thousands of places just like ours where people just like us are doing it every day.

But in 2007, you do need to let go of the idea that all we’re doing with “business programming” is building web applications that are replacing the client-server applications of the eighties and nineties that themselves replaced the green screen terminal applications of the seventies and eighties.

The “leading edge” interface and ideas employed by Google, Amazon, eBay, and Yahoo! are suffusing our culture to become the standard user interface of web applications. And programming the standard user interface is a basic job requirement. Learn to love it.

---

Do you love applications like Google Mail? Would you like to write stuff like this, even if it’s less than 100% of the time? But are you looking for a stable company working on stuff you can explain to your neighbors? Michael Lucas is hiring intermediate and senior developers for positions in Toronto, Canada. To be considered for a position, please send Michael an email with your answer to the following question:

Name three features from public web ‘sites’ like Google, Amazon, and YouTube (you can pick any site or sites you like) that will make the jump to business applications in 2007.

Labels: agile, popular

  ¶ 12:59 PM

  What I've Learned from Sales, Part II: Wanna Bet? In Part I of “What I’ve Learned from Sales,” Don’t Feed the Trolls, we looked at why resistance to a new idea is expressed as a never-ending series of objections. We looked at one powerful way to avoid objections, by identifying a real, urgent problem that needs to be solved. The next installment, Part III, How to use a blunt instrument to sharpen your saw, describes the mind-set that there are opportunities for improvement to be found everywhere.

In this part, we’re going to disregard my advice to avoid objections and talk about one way to respond to many of the objections raised against new ideas.

---

Can we overcome objections?

Right off the top, I want to say that I don’t believe you can “overcome” an objection by frontal assault. And furthermore, you shouldn’t try. You cannot persuade someone to consider an idea by debating them into submission.

My belief is that you must discover and address their true problem. The first reason to do so is that if they do not have a problem your idea can solve, there is no reason for them to “change for change’s sake.” The second reason is that if you are not solving a real, genuine problem for them you will get caught defending an idea against an endless series of objections.

That being said, there are some circumstances where it is important to respond directly to objections. Even if we have carefully qualified someone’s problem and explained how a new idea will solve their problem, a prudent person will analyse the idea carefully, looking for fatal defects that could prevent it from solving their problem.

Another common circumstance is when there are several parties involved in presenting, discussing, and analysing an idea. Although one of the parties may be bringing up irrelevant objections to resist the idea, you may need to persuade another of the parties that these irrelevant objections do not have merit.

For example, you may be suggesting that agile meta-programming will solve a company’s problems to the CEO. You may have carefully qualified the CEO’s priorities. But the company’s IT department will raise objections because your proposals do not address their personal and departmental problems. So you will need to respond to some of their objections as part of a campaign to neutralize their influence.

What is overcoming an objection?

Overcoming an objection is, to borrow a phrase from law, “A sword, not a shield.” When you overcome an objection, you point out that the reason for not considering your new idea is fallacious, or does not apply in this case.

However, overcoming the objection does not actually provide a reason to change: it merely removes a reason not to change. This is why Part I goes on and on about discovering an immediate problem your idea can fix. If you overcome an objection but have not presented a compelling reason to change, nothing will happen.

In other words, if someone says “I like your idea, however...” you should attempt to overcome the objection. If someone says “Your idea stinks because...” you really need to gidentify a problem to solve.

One model for responding to common objections

A gotcha objection is a proposition that your idea is false based on a premise that is true infrequently or only true for some small number of cases.

Let’s assume you are going to respond to an objection, and you have good reason for doing so. Here’s one common form of objection, and a method for responding to the objection. In essence, I am going to share a pattern with you.

First, let’s look at one of the most common kinds of objections. Here are some examples. They all have something in common:

• “Sometimes you are gonna need it, so you’re wasting time if you don’t build it in from the beginning. Therefore, we should build what we know we’ll need.”


• “Some bugs simply can’t be found through automated unit tests, sorry. This is why we have to use a programming language with static typing.”

What these objections all have in common is what I call the “gotcha!” fallacy. The underlying assumption is that if your idea does not work 100% of the time, on 100% of the cases, it is no damn good. And thus I call an objection based on the gotcha fallacy a Gotcha Objection. A gotcha objection is a proposition that your idea is false based on a premise that is true infrequently or only true for some small number of cases.

P is for Pharmaceutical

Think about therapies in medicine. None of them are deterministic! Every pill, every technique, every therapy is described in probabilistic terms: When compared to the control group who drank a glass of red wine daily but did little exercise, 36.7% of those who combined daily exercise with a glass of red wine had an average improvement of 22.1% in their combined evaluation scores for cardiovascular health.

Try this the next time you’re at the doctor’s office: point out to your physician that you have heard that some people who exercise drop dead right after their daily run. Use this as an excuse not to exercise.

Now, software development is not the same thing as medicine, and I am not suggesting you respond to criticism by saying that since some drugs do not work for some patients that your ideas have merit regardless of evidence or suggestions to the contrary. But I will walk you through the reasoning that leads to the same conclusion.

How to respond to gotcha objections

Our pattern for responding to these gotcha objections is to establish that software development is probabilistic in practice. Once we establish this premise, we then turn the debate from whether there are cases where a particular practice does not appear to be optimal to whether the overall results of applying that practice is better than not applying that practice.

That’s it, and if you’re in a hurry you can stop right here: everything else is an elaboration of this idea.

Exempli gratia: the technique in action

Objection: “Sometimes you are gonna need it, so you’re wasting time if you don’t build it in from the beginning. Therefore, we should build what we know we’ll need.”

Response: Well, sometimes you need it, sometimes you don’t. And when you don’t need it, you save the code you would have written as well as all the other design that becomes coupled to the code you end up throwing out. Of course, sometimes you end up throwing out some stub code, but think about the possibility that you will wind up getting more features done, earlier in the cycle where we can get feedback and reduce risk. Why don’t we look at whether, in aggregate, more projects will succeed using YAGNI than will succeed using “Build everything we might need”?

Objection: “Some bugs simply can’t be found through automated unit tests, sorry. This is why we have to use a programming language with static typing.”

Response: Sure enough, some can’t be detected with automated unit tests. But our choice of a dynamic language provides us with many other benefits, most especially in the areas of metaprogramming and code reduction. By writing less code, we may even have fewer bugs overall. Shouldn’t we try to compare similar projects written in a static language against those written in a dynamic language, and see whether the projects in the dynamic language had fewer bugs and whether the projects written in the dynamic language were more likely to be successful?

Now that you have seen the results of applying the technique, we will patiently examine the reasoning in detail.

Theory D and Theory P in Software Development

Theory P states that the time and effort required to measure all of the variables influencing a software development project precisely enough to predict the outcome with certainty and in advance exceeds the time and effort required develop the software.

There are two schools of thought about the practice of managing software development (the theory of managing software development is of little use to us because “the gap between theory and practice is larger in theory than it is in practice”).

One school is that everything is fully deterministic in practice (“Theory D”). If development appears, from the outside, to be probabilistic, it is only because we haven’t discovered the “hidden variables” that fully determine the outcome of software development projects. And, since we are talking about development in practice, it is practical to measure the variables that determine the outcome such that we can predict that outcome in advance.

The other school of thought is that development is fully probabilistic in practice (“Theory P”), that there are no hidden variables that could be used to predict with certainty the outcome of a software development project. Theory P states that the time and effort required to measure all of the variables influencing a software development project precisely enough to predict the outcome with certainty and in advance exceeds the time and effort required develop the software.

Theory P does not mean that software development cannot be managed in such a way that the desired outcome is nearly certain: the flight of an airplane is fully probabilistic as it encounters atmospheric conditions, yet we have a huge industry built around the idea that airplanes arrive at their destinations and land on the runway as planned.

Which theory fits the evidence collected in sixty years of software development? To date, Theory P is the clear winner on the evidence, and it’s not even close. Like any reasonable theory, it explains what we have observed to date and makes predictions that are tested empirically every day.

(Sidebar: do not confuse Computer Science—the study of the properties of computing machines—with Software Development, the employment of humans to build computing machines. The relationship between Computer Science and Software Development parallels the relationship between Engineering, the hard science of the behaviour of constructions, and Project Management, the employment of humans to construct engineered artefacts.)

The first response to the gotcha objection

The race may not always be to the swift nor the victory to the strong, but that’s how you bet
—Damon Runyan

Before we can go anywhere with gotcha objections, there is something you absolutely, positively must do when you respond. And it must be the very first thing you do. You must establish the fact that the premise of the objection is not universal and not predictable, it is probabilistic.

The objection is of the form that “since your idea works out badly some of the time, your whole idea is bad.” You must respond by establishing that some of the time the idea doesn’t appear to work out, and some of the time it does appear to work out. It isn’t universally bad or universally good.

If you are in a face to face discussion, you can solicit agreement from the objecting party. For example:

• “Well, sometimes you need it, sometimes you don’t, sometimes you’re wasting time, sometimes you aren’t. Is that right?”


• “So there are some bugs that can’t be found with automated unit tests and some that can—am I understanding you correctly?”

In a less interactive environment like a running language flame war on Usenet, you can start by simply stating that the premise is not universal.

Having established that the premise is not universal you must then establish that the cases where the premise applies are not easily distinguished from the cases where the premise does not apply. Establish that nobody knows whether the premise will apply or not until after it has happened.

We simply can’t tell in advance whether the bugs that would be caught by a static type system will end up being significant to the outcome of a project. We can’t tell in advance which constructs will end up being a waste of time. And we can’t tell in advance which people will fail when they try to pair program. (The last point is absolutely true if the people involved are not doing the arguing about whether pair programming will work. If the programmers involved do not believe it will work, they may have a point.)

This is the other aspect of establishing that the premise is probabilistic: not only does it only apply some of the time, but we don’t have a good way of knowing in advance when it applies and when it doesn’t.

Okay, we’ve gone through all of this dry pseudo-academic talk of theories and probabilistic development. Time for a vacation to Las Vegas.


How do casinos make money?

The casino’s strategy is so secure, there is just one danger to its profits: if the casino plays very few games for very high stakes relative to their capitalization, they could lose their capital.

Casinos make money by wagering money on the outcome of games with gamblers. (If the gambling industry offends you, I apologize. We could choose to look at how insurance companies make money, it is entirely the same thing.) The games are arranged in such a way they the casino holds a small mathematical edge on each play. Over the long run, with many gamblers playing the games many, many times, the casino inexorably makes money. The casino may lose games here and there, and some gamblers may enjoy temporary winning streaks, but overall, the casino wins more than it loses.

The very briefest exploration of statistics reveals the following facts about the casino’s strategy for making money:

1. The casino must have an edge on each play;


2. The more games played, the more likely that the casino will profit overall;


3. Runs of good luck for some gamblers are offset by runs of bad luck for other gamblers.

In fact, the casino’s strategy is so secure, there is just one danger to its profits (besides the obvious fear of losing their license to print money): if the casino plays very few games for very high stakes relative to their capitalization, they could lose their capital. One celebrated “whale,” Akio Kashiwagi, won more than $19 million in one casino and on another occasion won $6 million in a siangle session playing baccarat for $200,000 a hand.

Therefore, the casino’s prime safeguard is to avoid risking large amounts of capital on games played very few times.

To summarize, the casino’s strategy is to:

1. Arrange a small advantage on each game played;


2. Play a very large amount of games;


3. Ignore good and bad runs, they will offset each other;


4. Do not risk large amounts of capital on games played very few times.

Now that we know the casino’s strategy, let’s consider how they evaluate games. Imagine them sitting around a conference table, and someone suggests, “Let’s create a new game, Alaska Freeze.” What do they use to evaluate whether to add this new game to their casino?

Space in a casino is at a premium. If Alaska Freeze goes in, something else comes out. So there has to be an Incremental Value calculation: do they make more money with Alaska Freeze in and something else out? Or less? This calculation has two simple components: does Alaska Freeze have a larger or smaller mathematical edge than whatever it replaces, and will Alaska Freeze be played more or less often than whatever it replaces?

Ok, let’s return to handling gotcha objections. I’m sure you knew all of this, I was just presenting it in a palatable morsel so you can feed it to someone objecting to your idea.

From casinos back to objections

The process of developing software is just like the business of running a casino.

You are handling an objection and you have established that its premise is neither universal nor predictable, it is probabilistic. Well, if it has an uncertain outcome, it is just like a casino game. And the process of developing software is just like the business of running a casino.

So the question is not whether a new practice ever has a case where it appears to “lose,” for the same reason that evaluating a new game for a casino does not involve worrying about whether gamblers will ever win.

The way to evaluate the idea is to examine it and see whether it fits the casino model:

1. Does it offer a probable advantage each time it is employed?


2. Can it be employed many times?


3. Do streaks of “losses” and “wins” offset each other?


4. Can we avoid risking the entire outcome of the project infrequent events?

And if it does, to identify what idea or practice it replaces to determine whether it is a net win or a net loss overall:

• Does it offer a larger or smaller mathematical edge over the practice it replaces? In other words, does it lose less often?


• Can it be applied more or less often than the idea it replaces? In other words, how much benefit can we obtain from it?

Now, I am not going to say that XP, YAGNI, or dynamically typed programming languages necessarily fit the casino model and are necessarily better than the practices they replace (Classical Project Management, BDUF, and static typing). But what I will say is that there is a huge difference between saying “some of the time, for some of the people, that idea loses” and saying “overall, when applied to an entire project, the project does worse than whatever idea it replaced.”

So to handle a gotcha objection, we establish that it is probabilistic in nature, then we analyse it as we would analyse any other probabilistic practice: we look at the overall effect of the practice on the entire project, comparing it to whatever practice it would replace. And I have some easy-to-remember phrases for doing that.

The second response to the gotcha objection

The next step is rather obvious. You have to state the payoff those times that your idea or practice “wins.” And to be fair, you also have to agree to the cost of your idea or practice when it “loses.”

• “When teams are disciplined about not writing code that won’t be needed in the current iteration, we get more of the features developed earlier, where we can analyse and learn from them. This lowers our technical and requirements risk. Of course, sometimes this means that they have to throw stub or temporary scaffolding code out later, so there is some wasted code.”


• “When projects are written using dynamic languages, they can use techniques like meta-programming to write less code and to concentrate the green code in one place. Of course, we do not have the compiler telling us about certain types of errors.”

Needless to say, if you are in a face to face meeting you should solicit agreement to this second response as well. If you can’t establish that there are any benefits to your suggested practice, you have a great deal more work to do to handle this objection.

The third, and final response to the gotcha objection

The final step is the clincher. Having established that the objection’s premise is probabilistic, and that for those times the idea or practice “wins” there is a positive payoff, it’s time to compare the overall benefit of the idea or practice to whatever it replaces. You want to shift the debate from debating the premise to debating the overall benefit.

And in fact, there are two different forms of the clincher. You can use either, or preferably both:

1. Ask whether you can balance the benefits of using the practice when it wins against the cost of using the practice when it loses, and evaluate the overall benefit in comparison to the benefits and costs of the alternative and see whether you would get more of a benefit over one entire project, or;


2. Ask whether you can compare the success rate of teams using the practice to the success rate of teams using the alternative in aggregate, or;


3. Both.

And here are the example responses again, demonstrating the three forms of clincher:

Objection: “Sometimes you are gonna need it, so you’re wasting time if you don’t build it in from the beginning. Therefore, we should build what we know we’ll need.”

Response: Well, sometimes you need it, sometimes you don’t. And when you don’t need it, you save the code you would have written as well as all the other design that becomes coupled to the code you end up throwing out. Of course, sometimes you end up throwing out some stub code, but think about the possibility that you will wind up getting more features done, earlier in the cycle where we can get feedback and reduce risk. Why don’t we look at whether, in aggregate, more projects will succeed using YAGNI than will succeed using “Build everything we might need”?

Objection: “Some bugs simply can’t be found through automated unit tests, sorry. This is why we have to use a programming language with static typing.”

Response: Sure enough, some can’t be detected with automated unit tests. But our choice of a dynamic language provides us with many other benefits, most especially in the areas of metaprogramming and code reduction. By writing less code, we may even have fewer bugs overall. Shouldn’t we try to compare similar projects written in a static language against those written in a dynamic language, and see whether the projects in the dynamic language had fewer bugs and whether the projects written in the dynamic language were more likely to be successful?

Good luck handling objections. What is your experience: do you have another technique you can recommend?

A Personal Note

As Mike pointed out in the first comment, this post explains how to handle this one type of objection, the gotcha objection, by moving the debate away from the exception case and towards the overall case. But it does not follow up by presenting hard data to justify the example ideas presented.

First, I want to say that even with hard data you will not foster change with numbers: you need to show how your idea addresses an urgent priority. That should have happened before you got to this point. If you have addressed the problem correctly, it really is sufficient to point out the fallacy in the objection and allow your original argument to stand.

Second, there is a dearth of hard data about anything to do with software development. Repeat after me: “the plural of anecdote is not data.” If you have a source of hard data about any practice, be it programming languages, practices, or even interviewing techniques, I would very much like to read and learn from it.

Does this mean that we should never change, that since there’s no proof that new ideas are an improvement over old ones?

If you are happy with your current situation, maybe not. If you are unhappy with your current situation, if you want things to be better, you may want to change something. It’s your call.

Labels: agile, popular

  ¶ 5:16 PM

  What I've Learned From Sales, Part I: Don't Feed the Trolls This is the first part of “What I’ve Learned From Sales.” In this part, “Don’t Feed the Trolls,” I present my explanation for why people act like “trolls,” raising objection after objection to new ideas, and I suggest how to side-step this behaviour and deal directly with their concerns.

(Part II, Wanna Bet?, describes how to handle one very common form of objection. Part III, How to use a blunt instrument to sharpen your saw, describes the mind-set that there are opportunities for improvement to be found everywhere.)

---

“Oh no,” you must be thinking, “another Guy Kawasaki wanna-be trying to tell me how to sell things.” Well, yes to the wanna-be accusation, but no to the proposition that this is a general-purpose article about sales. I am not going to pretend to tell you how to promote your business, turn your product idea into a money-maker, or even how to sell yourself to an employer.

What I am going to share with you is some experience I have had with sales that strongly parallels my experience discussing new ideas with people. (I know, reasoning by analogy is often faulty. But it’s what we humans do, we’re pattern-matching machines.) If you find that people seem unreasonably resistant to good ideas like more powerful programming languages, putting people before process, or valuing working code above documents, you may find this helpful.

Our model here is that the mental process of considering a new idea is the same as the mental process behind buying something. If you are discussing a new idea with someone—even if you aren’t actively trying to “sell” it to them—they are still going through the buying process. And if they have trouble accepting the idea, they will resist, or in sales jargon, they will “raise objections.”

Do you think this is specific to sales? No, when we see new ideas like the Ruby programming language, we encounter objection after objection. Some are ironic: Some Java enthusiast objects to Ruby on performance grounds: perhaps they are too young to remember when the C/C++ folks objected to Java on performance grounds? Or perhaps it will be the IDE support objection, or the Not Invented By Microsoft (a/k/a Not a Corporate Standard) objection, or any of a million others that are not completely unreasonable, but are also usually irrelevant.

What is an objection?

On the face of it, an objection is an expression of a discrepancy between your idea and what someone wants. So if they say “Lisp has too many parentheses,” you might think that they are saying “I would use Lisp if it didn’t have so many parentheses.”

The great secret we can learn from sales is that this is not true. As we will see below, people say what they think other people want them to say. So someone might be thinking “Lisp is too hard for me, all this talk of let and lambda and closures and tail recursion is confusing.” But they are embarrassed to admit this, so they seize on something that sounds more reasonable, like “the syntax is weird.”

We need to understand this, because the absolute worst thing you can do with an objection is answer it directly. If someone is really thinking “Lisp is too hard,” what good does it do to try to persuade them that parentheses are their friends? They don’t really care. Worse, if you trot out the benefits of homiconicity and its applications to macros and introspection and meta-circularity, you’re actually making Lisp sound harder, not easier.

At one time I was a Macintosh salesperson. I used to sell Mac SEs and Mac IIs in “The Dark Times” after Steve Jobs was expelled from Apple by the vile and treacherous Prince John…

Let me give you an example from my own experience in sales. At one time I was a Macintosh salesperson. I used to sell Mac SEs and Mac IIs in “The Dark Times” after Steve Jobs was expelled from Apple by the vile and treacherous Prince John… but I digress. I was a Macintosh salesperson at a time much like this time: nineteen out of twenty computer sales were PCs.

You would think that selling Macintoshes would be a lonely existence. But no, the phone would ring regularly and customers would visit the office on a daily basis, always with the same question, Why should I buy a Macintosh instead of a PC? And at first, I would answer this question. Macintoshes were superior to PCs in every way. (Actually, this was technically very true at that time. Why, you could run as many as six monitors on a Macintosh II! But I am digressing from my digression.)

I learned a funny thing about answering people’s questions. I would answer their questions, and they would argue with me. I would say that you could run multiple monitors on a Macintosh II, making yourself more productive. And they would say “but I don’t need to be that productive, so that doesn’t count.” Or I would say that the mouse and windowing interface is easier to use, you can learn to use more programs. And they would say “but all I need is a word processor and a spreadsheet, so I don’t need to learn new programs.”

(Why would they do that? It was so that if someone asked them, “did you shop around and make an intelligent decision?,” they could reply, “why yes, I shopped around, I checked out Macs and PCs, I did a lot of research, and surprise surprise, I ended up with the exact same thing that my neighbour Bob has, except mine is running at 12Mhz and poor Bob is stuck with 10Mhz.” It may sound to you like they are doing an awful lot of work just to be able to say that one thing with a straight face, but I can tell you that there is this multi-billion dollar automobile industry that works on this principle: people want to be a little better than their neighbour, but not so much better that they are different than their neighbour.)

I can give you many more examples, but the interesting thing is not whether people wanted this stuff or not, but no matter how convincingly I answered their question, they would just ask another one. Their questions had nothing to do with how they were making up their mind.

What people think often bears no relationship to how they behave

I learned very quickly that what people think often bears no relationship to how they behave. People usually say the things that they think other people expect them to say, but they go ahead and do whatever it is they always wanted to do. In the case of buying computers, my observation is that most people want to buy whatever it is that most people are buying. They want to belong, to fit in. So they are going to buy a PC. Or an iPod.

So what was really on their mind was fitting in, even though they argued about the Macintosh’s technical merits.

The lesson I learned is that before we can introduce a new idea to someone, we first need to understand what is really on their mind.

Salespeople call this “uncovering the hidden objection”. They have all these elaborate techniques for figuring out what’s really on a prospect’s mind when they encounter resistance to the sales pitch. I’m not going to suggest we do that. Instead, I’m going to suggest we avoid the objections in the first place by “qualifying.”

The most important principle of effective selling is that qualifying the customer is more important than overcoming objections.

What is “qualifying” and why is it the most important step in the sales process?

Many people say the most important step is “closing,” the art of getting the prospect to hand over the money, the end of the sale. If you judge by the behaviour of people selling time shares, fitness club memberships, and automobiles, this is the only thing salespeople work on: haranguing prospects and ‘overcoming objections’ by arguing.

Experienced and successful salespeople follow a different path. Experienced salespeople believe that the most important step is qualifying, the art of discovering whether a prospect has an actual need for the product, the beginning of the sale. Salespeople who are strong qualifiers spend almost no effort on closing because they are always working with prospects who want to buy.

On the other hand, if you do not discover their real problem, you extol virtues that have no attraction for them and neglect to address any perceived issues in their mind. The principle at work is that if you know what someone really needs, you address their needs from the very beginning. When you arrive at the conclusion, you have already addressed any questions they may have.

This is true for sales. It is also true for new ideas. If someone fears that an idea like learning Lisp (or meta-programming, or designing a program in the technical interview) would be too difficult for them, you will only be successful if you first explain how easily they will learn the new idea, and only then explain how wonderful the idea is.

If someone doesn’t have a headache, you cannot establish the value of an aspirin for them… Don’t focus on how you think your new idea can help them be better. Instead, focus on whether they have an urgent problem that your new idea can fix.

Although there are various models for understanding people’s motivations—such as Maslow’s Hierarchy of Needs, or the Greed-Belonging-Exclusivity-Fear Quartet—for development tools and methodologies my experience is that the simplest model fits best: people are motivated to solve their problems: if you can identify a problem they think they have, you can show them how to solve it.

People without problems are not good prospects for lightweight development methodologies, new development tools, programming languages, or any other “change for the better.” Just the other day I was lunching with Dmitri from Opalis. We were talking about a development tool I am trying to build, and Dmitri was suggesting that it was a “vitamin” and not an “aspirin”.

I was taken aback. Isn’t improving software development important for everyone? Then I remembered my sales training and asked him about how things were going at Opalis. Dmitri admitted that his team was performing well and that he had built a lot of trust with his organization. So he didn’t have a problem. Quite simply, if someone doesn’t have a headache, you cannot establish the value of an aspirin for them.

Now, even Dmitri’s team has room for improvement, so it is not correct to say that there is no value in improved methodologies, tools, languages, or anything else for him. However, such things may not be a priority right now. This is exactly the same case as trying to sell a Macintosh to Bob’s neighbour: I believed that absolutely everyone could have benefited from owning a Macintosh. However, Bob’s neighbour didn’t think he had a usability problem, he thought he had an urgent “keeping up with Bob” problem.

And there’s the key: Don’t focus on how you think your new idea can help them be better. Instead, focus on whether they have an urgent problem that your new idea can fix.

Discovering their priorities shouldn’t difficult. Why don’t we simply ask them? Well, there’s a trick to asking someone about their priorities. Remember, they will tell you what they think other people want to hear, not what they are thinking. Here’s an example concerning agile development:

Agilist: “What’re your priorities for the development team in the next 60-90 days?”

Manager Says: “I have a total commitment to process improvement and faster response to business initiatives,” …but thinks… CMM Level Four—or, God willing, Level Five—will get me a higher profile and a shot at the CIO position. I need some consultants in here to start imposing some bondage and discipline over our development processes.

The trick is to get specific and objective. Never take objections as evidence of their real needs, and never accept vague feel-good values at face value. Top salespeople don’t. Try calling a busy estate agent and saying you’d like to buy a house. I guarantee that the agent will ask you, “when do you need to buy a new house?” And so it is with new ideas:

You cannot position lightweight development, tools, languages, or any other type of change without being able to fit them into the specific and objective problems someone is trying to solve. You need to relentlessly pursue the immediate, urgent priority:

Startup Founder: “I’ve heard that Agile stuff is crap—it only works for star programmers who would be good no matter how you manage them.”

Agilist: “Well, there’re a lot of opinions out there. Tell me, what would you say is the most pressing issue facing your development team right now?”

Startup Founder: “Well, we have hacked together some great stuff, but we need to scale, and to scale without imploding we’ll need some discipline, some real management of the development team. That’s why we need a real methodology.”

Agilist: “I can understand the importance of scaling up. So, have you set some specific objectives for scaling up over the next month or two?”

Startup Founder: “For the next couple of months? Oh, it’s all about recruiting, definitely recruiting. I need another two or three top people to work on a new project that could be worth millions. We’ve identified some good candidates, but it’s very difficult to get them to accept an offer from a start up.”

Agilist: “You know, we really ought to consider whether using Agile might help you recruit—have you considered the possibility that some star candidates might be looking for an environment that is more Agile than the one they are leaving?”

Startup Founder: “Hmmm…”

As you can tell, once you have a specific problem with specific dates attached to when the problem needs to be resolved, you can discuss a specific solution. You’ve side-stepped the useless “objections.” One more time: do not accept vague objectives, get specific objectives with near-term dates attached to them.

If someone really doesn’t have any applicable near-term objective, you will not be able to introduce a new idea to them. So don’t be surprised if they express very little interest. But when you have an immediate, specific objective in hand, you can position the idea as a solution to their problem.

And that works for almost any idea. Say you had a new programming language designed for set-top boxes. But it turns out nobody has a “programming set top boxes” problem. So they raise objections about the speed of your virtual machine, or the fact that programmers cannot manage memory in your language, so they cannot squeeze programs into very small spaces.

Should you keep pounding away at that? Or go looking for an immediate problem people have, like building web applications?

If you side-step their objections—like memory management—and get to the root of their immediate needs, you might be able to introduce a new popular programming language. Good luck!

Part II, Wanna Bet?, describes how to handle one very common form of objection. If you liked this post, you might also like a related post, The false dichotomy of choosing between your values and expediency.

Labels: agile, popular

  ¶ 12:47 AM

  An interesting case of bias One of the subjects that seems to raise people’s hackles is how to conduct a technical interview. I recall being subjected to personal attacks because I happen to like asking senior candidates a design problem as part of the process.

Oh, I’m a terrific software architect if you want a three tier, MVC business rules-driven application where I have days or weeks to suss out requirements and fully investigate the options. But my performance designing a distributed email server, or a game, or anything else in just a few hours does not reflect my performance under actual job conditions.

I’m not going to argue the point here. But I have observed a very, very common theme to the arguments against solving puzzles, writing code, and designing ‘toy’ systems in an interview (let’s call all of these activities “juggling”): in almost every comment I’ve read objecting to asking a candidate to juggle, the poster admits that they dislike juggling, or are poor at it under the time and pressure constraints of an interview.

The argument against testing is cloaked in terms of “Oh, I’m a terrific software architect if you want a three tier, MVC business rules-driven application where I have days or weeks to suss out requirements and fully investigate the options. But my performance designing a distributed email server, or a game, or anything else in just a few hours does not reflect my performance under actual job conditions.”

The premise is since the candidate thinks they are good, and also thinks they would do poorly on the test, the test must be invalid.

I’m not saying they’re wrong. I just find it interesting how few people make the claim that they are incredibly skilled at ‘juggling’ in technical interviews but do not think the results are particularly significant.

update: Do you dislike juggling but you’d like to work for the company anyways? Take control of your interview.  ¶ 8:11 PM

  Children should be making things, communicating, exploring, sharing, not running office automation tools

One of the saddest but most common conditions in elementary school computer labs (when they exist in the developing world), is the children are being trained to use Word, Excel and PowerPoint.

I consider that criminal, because children should be making things, communicating, exploring, sharing, not running office automation tools.

Labels: passion

  ¶ 9:59 PM

  Upcoming Presentation and Discussion: "Wanna bet?" How to discuss lightweight development practices with skeptics. The Toronto XP/Agile Users Group has just accepted my talk proposal, so on January 16th I’ll be presenting “Wanna bet?” How to discuss lightweight development practices with skeptics at 7:00 pm.

The location is:

Ryerson University
George Vari Engineering and Computing Centre
245 Church St.
Room ENG 288 / 289
Toronto, Canada

I’ll be talking about various criticisms that have popped up lately—most infamously Steve Yegge’s rant about Bad Agile—and giving tips for how to constructively debate the value of lightweight or agile development practices with people who may have read this kind of thing.

If you’re in Toronto on the 16th, please join me and the XP/Agile Users group for what should be an interesting evening. The discussions around the presentations are always stellar.  ¶ 12:12 PM

  A Fellow Traveller's Road to Enlightenment

I began to see that many of the sacred GoF design patterns were not, in actuality, grand universal truths of software engineering, but simply collateral damage caused by incidental limitations in the abstractive power and object model of certain manifestly-typed programming languages.

A wonderful read.

Labels: passion

  ¶ 11:32 AM

  Where were you on Saturday, November 9, 2002? Had you dropped by MIT on November 9th, 2002, you might have seen a rather large collection of hackers, academics, tool vendors, and programming language enthusiasts gathering for the LL2 conference. It was rather like taking the current readership of Lambda the Ultimate and programming.reddit.com—no, I am wrong.

It was rather like taking the top couple of hundred of the people who write papers and blogs cited on Lambda the Ultimate and programming.reddit.com and gathering them in an auditorium together to talk about programming languages.

As the rather Spartan LL2 web page explains, the theme of the conference was:

How to get usable and useful programming tools into the hands of programmers, minimizing dogma, and maximizing flexibility.

Like any other conference, there were speakers. The speakers talked about the languages they had invented, and about how and why people ought to use them.

Had you been there, what would you have thought about the first speaker, Joe Armstrong, and his language, Erlang? Just another weird language for embedded telephone switches? Or an important language that is powering diverse applications such as a distributed, replicating database?

Joe spent a lot of time talking about unreliability. About how computers and programming languages are always designed as if everything always worked and that not working was the exception case, rather than the rule. And about how you can’t build systems out of dozens, hundreds, or thousands of pieces without assuming that everything is unreliable, all of the time.

And how there ought to be a programming language that builds that right into the system. Like Erlang. Would your eyes have glazed over? Or would you have thought about MapReduce and grid computing and would you have taken notes furiously?

Would you have thought that two different talks (one about persistence and another about asynchronous exceptions) were too much for an obscure language called Python? Would you have marked Python down as “One to Watch”?

And what would some random guy know about inventing “the next big programming language”? Oh, he was managing Microsoft’s Programming Languages Systems group. Well, there you go, one guy that Corporate America might care about. I wonder how many of his predictions about the next big programming language have come true? Good thing his talk is on line, we can check…

After lunch, some random guys talk about getting an un-typed language running on top of the Java Virtual Machine. I mean, really, who the hell would think that would ever become important?

Next up, the organizers had flown Yukihiro Matsumoto in to talk about Ruby. Yes, in 2002 the organizers wanted a bunch of Lisp-heads—the Q & A for absolutely every talk included at least one question about when macros would be added to the speaker’s language—to hear from someone who had designed a language known at the time as the most popular scripting language in Japan. Had you heard of Ruby in 2002? Did you care?

And what was with him constantly talking about making programming fun and comfortable? As if a programming language was a user interface for programmers. Everyone knows that programmers love idiosyncratic tools like Emacs, right? Who designs a language for usability? Who indeed.

There were other talks. They were all good, just as good. There was a spirited exchange of ideas over lunch in the lobby, again upstairs in the MIT researchers’ offices, and continuing late into the night at a nearby pub. It was a phenomenal gathering.

Were you there?

That isn’t important right now. If you were there, good for you, and I hope in the four years since you have taken some of those ideas, and a lot of that enthusiasm, and rolled it into what makes this industry great. The fact that anyone—even some guy from the other side of the world—can make something that other people like and use. Whether it’s free or for pay, whether it’s a language, a tool, an application, or a piece of hardware.

But even if you were there, that was then and this is now. 2007. Here’s the important question: Where is LL2 this year?

I asked “where is LL2 this year.” Actually, it may not be a Lightweight Languages conference. It might have nothing to do with programming languages. But somewhere in the world, somewhen in 2007, there is going to be a conference. Or a gathering. Maybe it’s this year’s Startup School. Maybe it’s a DemoCamp in your city.

Wherever, and whenever it is, you are going to see and hear things that are going to matter to you over the next five years.

They may not look like much in 2007, but they’re the seeds of the next Big Thing, the thing that will be big four or five years hence, in 2012.

Finding the conference

In 2002, LL2 didn’t look like much. You probably couldn’t find a job using any of the languages discussed at the conference. Unless you wanted to work on telephony. The lone representative of a mainstream company—Microsoft—wasn’t talking about their products, he was talking about things that would disrupt their products.

So if you’re looking for the right conference in 2007, there are two clues: the agenda isn’t going to include anything you can use to make money, and it isn’t going to feature anybody trying to make money off of you.

That’s right: no consultants, no white papers, no thinly veiled sales pitches. That knocks most Ruby conferences out. Maybe not all, but most. The right conference isn’t going to be about building CRUD applications on top of MySQL with AJAX goodness. That may not be out of date, but it’s so 2007, and we’re looking for 2012.

In 2002, the audience was highly interactive. Most of them were working on their own projects. They were practitioners, not tourists. They weren’t there to get a job, or goof off work. They were there to learn stuff for themselves and share what they knew with others.

In 2007, you should be looking for the same thing. For a conference that will be attended by people who are, as Joel would say, Smart and Get Things Done. Lacking a way to check the door before you show up, how do you find out?

Naturally, you find out by talking to other would-be attendees before going. LL2 had LL1’s active mailing list. In 2005, Startup School had a wiki. I don’t know what the right conference in 2007 will have, but I will bet on this: if there’s no community building in advance around a wiki, a list, or some other peer-to-peer communication, it isn’t the right conference.

The kind of conference where there is a one-way exchange between you and the organization—they tell you where to show up and you send them money—is not the right kind of conference. Look for the conferences that are built on top of active communities.

And when you find the community, pay attention to the action-to-talk ratio. I’m terrible at this, I always have to push myself to work and stop goofing off on my blog. But you know what’s worse? People who don’t have any work to begin with, it’s all just entertainment to them. They are tourists, and they are a liability to a conference.

Here’s why.

When tourists reach critical mass, the organizers start to cater to them. That means safe topics and safe speakers. Consultants. People with a good “patter” but not much to say that would disturb anyone.

Also, the tourists tend to ask the wrong questions, because they aren’t the market for anything new and interesting. Say a tourist raises her hand. She’s going to ask about something she imagines might concern a user, but you know what? She isn’t a user and she really has no idea how a user thinks, because she’s not the type to learn anything new until it’s already popular, and she doesn’t have any idea how such people think.

I think the root of your mistake is saying that macros don't scale to larger groups. The real truth is that macros don't scale to stupider groups.

So she asks something irrelevant. Like about language performance. Or bindings to Oracle. Or whether there is an Eclipse plug-in. Or how it will scale to larger programming teams.

But when a practitioner asks a question, it’s a smart question, it’s important. The speaker learns something important about their tool and the rest of the audience learns too. The next thing, someone asks a smarter question, and the energy builds.

If there are too many tourists, you can’t learn anything significant in the Q & A, and that’s a shame, because Q & A is where the action is.

So, my advice for finding the right conference is, avoid anything to do with either you or the presenters making money, find a conference built on top of an active community, and go the the conference with the highest density of practitioners.

And my question really ought to have been, will you be at this year’s important conference? and better still, will you be presenting?

Labels: passion, ruby

  ¶ 4:57 PM