(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Friday, April 20, 2007
  99% of programmers are law-abiding citizens...

When dispensing advice, people often pluck the proportion “99%” out of the air as a way of saying “overwhelmingly likely.” As in:

Now 99% is obviously a junk number, just like 199/200. But let’s hand wave and say that it’s close enough to the truth. What does that mean?

For example, if you’re deciding how to build persistence into a web application, what does “99%” tell us about using an RDBMS? I’d say, not very much. For starters, if you build seventy web application in your entire career, there’s a fifty-fifty chance one of them should have used something other than an RDBMS.

That tells me that you can’t discard other options and just say that “every web application should use an RDBMS.” 99% is a lot of them, but not so many that we can afford to ignore the 1%.

Society of Mind is Marvin Minsky’s legendary explanation of how the human brain solves problems. It’s a fascinating book, not least because of its unusual structure: lasvishly illustrated with diagrams, each page is a self-contained chapter and links between ideas form a dense network rather than a linear chain.

If we were buying one hundred stocks, we would do very well if ninety-nine of them make money and one loses… unless the one loser loses us more money than the other ninety-nine make for us. When it comes to technology choices, we don’t just make a single wild bet (“every application will have an RDBMS”). We have the option of investigating the application and our options before making a choice.

So when I read a SWAG like “iteration works just fine for 99% of the things you do with collections,” I make a mental note to look for the 1% of the cases where iteration doesn’t work.

They may be few and far between, but they’ll pop up enough to make knowing how to use map, reduce, and recursion worthwhile.

That being said, “99% of businesses should run on lock-in, proprietary operating systems and platforms” is a very interesting case. There’s really very little cost in learning how to roll your own persistence for the cases where an RDBMS is not the right choice (even if you need transactional semantics). And it is surprisingly easy to use map, reduce, and recursion… it turns out to be easier than iteration, not harder.

But learning entire platforms is an expensive proposition. It may make sense to focus on one platform where you can accumulate high expertise. After all, a programming language takes ten years to master, can an entire OS and its ecosystem be any easier? And even if it is possible to learn a new platform on the fly, employers may prefer to hire for experience rather than pay you to learn.

That doesn’t tell me that you should go with the 99% solution. The 1% of companies out there that are best served with other technologies still need to hire employees, engage suppliers, and purchase software that runs on their systems. Game theory tells us that the optimum strategy for a worker, contractor, or supplier is to roll a one hundred sided die and pick the 1% platform if they roll a “one.”

What I find really interesting is that the marketplace is highly inefficient. Given one platform having 99% share, far more than 99% of the people will choose to “specialize” in the 99% solution, creating a commodity marketplace. The very few that serve the 1% of the market enjoy disproportionally higher salaries and profits.

For this reason, when I read that 99% of the web applications should use an RDBMS, I am very interested in the 1% that shouldn’t… and figuring out how I can manoeuvre myself into a position to build one. And the news that one company has a near-monopolistic lock on the business operating system business? I consider this an opportunity for those of us comfortable with the alternatives.

So the next time you hear the magic words, “99% of…”, I encourage you to do as I do: start asking questions about the other one percent.

Comments on “99% of programmers are law-abiding citizens...:
The chance to never ever get into one of these no-RDBMs projects in 50 projects is 0.99^50 = 0.605 = 61%. Not fifty-fifty.

The chance to get in exactly one (and not more) of them is 0.01 * 0.99^49 = 0.006 = 0.6%.
actually the chance to get exactly one is 50 * 0.01^1 * 0.99^49, i.e. about 30% if the previous calculations are correct.
Thanks, I think I've corrected the math.
this post is just fantastic. cheers!

<< Home
Reg Braithwaite

Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /