Monday, October 30, 2006
A Ruby, Io, and Ocaml Programming Pattern
Every 5 minutes you spend writing code in a new language is more useful than 5 hours reading blog posts about how great the language is.
Programming Theorems
Seriously.
If I hand you a plastic disc, are we going to spend all day debating whether Ultimate is more fun than Soccer? Do you want to investigate the physics of rotating aerofoils and how gyroscopic precession affects the flight path when the disc is released from your hand at 190 degrees?
Or should we just go out and throw the Hammer already?
Reading is important. But reading (and blogging!) is no substitute for doing. And on that note, I shall return to writing unit tests...
If I hand you a plastic disc, are we going to spend all day debating whether Ultimate is more fun than Soccer? Do you want to investigate the physics of rotating aerofoils and how gyroscopic precession affects the flight path when the disc is released from your hand at 190 degrees?
Or should we just go out and throw the Hammer already?
Reading is important. But reading (and blogging!) is no substitute for doing. And on that note, I shall return to writing unit tests...
Friday, October 20, 2006
Is your IDE tilting at the dynamic language windmill?
Quite a few programmers love having powerful IDEs. These IDEs perform certain simple refactorings somewhat automatically, they can take a compiled language and make it seem a little more like an interpreted language, they can find certain types of errors as you type…
In short, they do a lot of drudge work on the programmer’s behalf. What’s not to like about this? Nothing. These are useful tools. I have been working with Eclipse lately when editing Java, although I really don’t care one way or another. I like what it does but I’m fundamentally indifferent to its existence.
Of course, there are more than a few people who espouse the idea that the most powerful features of these IDEs are only possible with statically typed languages. And that this productivity is so important that it—ummm—eclipses all other considerations such as which language allows you to express a program with the least redundancy.
How interesting.
Anyways, lots of people use Emacs for Lisp (don’t you love the symmetry? you need to use Lisp for Emacs). They invest a lot of time making themselves more productive.
So here’s what I see from my chair: There’s this group of star-bellied sneetches who say “the important thing about being a productive programmer is having a tool that makes you productive.” And there’s this group of sneetches who say the same dammned thing. The only difference between them is that the star-bellied sneetches say that the productivity of the tool is all about the special things the tool can do when the programs are written in statically typed languages. The plain sneetches love the tool that they can customize so heavily it feels like a personal extension of their mind.
Hmmm.
The right thing to do is to get two people who both agree that productivity from IDEs is important to debate how to do it. That way there’s one, clear point of debate: does an IDE + static typing beat a programmable editor by an order of magnitude?
Debating with me is going to be entertaining but inconclusive, because we’ll be mixing several arguments up: are IDEs important, and if they are, what’s the best way to make an IDE powerful, and if that is the best way, is it so good that it outweighs other considerations for choosing a programming language? If you stop by my house, that’s a conversation that is going to put a serious dent in my collection of single malts.
Kinda something not entirely unlike what happens when you sit down to improve the performance of a program. First you profile. Then you choose what to optimize. I’ve profiled myself, and what I need is more help deciding what to do than I do doing it.
I am also holding on to a somewhat axiomatic (and irrepressibly optimistic) belief that some languages have orders of magnitude higher signal-to-noise ratio than others. This is important for understanding why I am reluctant to pick a language because my IDE likes it.
For starters, one needs to understand the goal of all this refactoring: it’s to have better code. Better languages also achieve this goal. let’s look at a simple example and work backwards from that to the importance of languages.
Whereas in Java, you need to define an
(Well, you can forward calls to classes that don't implement interfaces, but you can't make the delegator look like a delegatee. So what you really have is a kind of private inheritence. You cannot, for example, decorate a
Why can’t you just generate the delegation using the IDE’s handy-dandy feature?
The problem is that code generation produces a static change. It’s a one-time thing, like one of those push-button programming wizards from MSFT. If you change interfaces, perhaps you are refactoring something else, your delegation breaks. So you have to write something relatively dynamic.
In Ruby, it looks things up at run time. I’ll settle for something that can reflect at compile time. But either way, you can’t just do it once. You have to have dynamic code handle the forwarding.
(There are exactly similar problems in all sorts of code. If you write some serialization code by hand or with an IDE, you have a problem when you expand an object. The old code doesn’t know how to serialize the new members. You have to make sure your serialization code reflects on the object so it can keep up with all changes.)
I find that IDEs are nice, but even the most sophisticated IDEs are… static. They make one-time changes to code. I’m far more interested in languages that let me write code that writes code, in languages that let me write code that doesn't need a lot of rewriting and fixing across the board when I make a change. (This is just my perspective. I’m still learning a lot about the intersection between theory and practice in programming languages.)
But this is why you’re tilting at windmills when you ask me why I don’t eschew Ruby for Java just so I can “rename class.”
In short, they do a lot of drudge work on the programmer’s behalf. What’s not to like about this? Nothing. These are useful tools. I have been working with Eclipse lately when editing Java, although I really don’t care one way or another. I like what it does but I’m fundamentally indifferent to its existence.
Of course, there are more than a few people who espouse the idea that the most powerful features of these IDEs are only possible with statically typed languages. And that this productivity is so important that it—ummm—eclipses all other considerations such as which language allows you to express a program with the least redundancy.How interesting.
Do all powerful IDEs require static typing?
The idea that IDEs make you more productive is nothing particularly specific to statically typed languages. Have you ever heard of an editor called Emacs? Did you know that some of the hairiest, most dynamic, self-modifying code ever written was once part of Emacs? And that the author of that dynamic, self-modifying code, some fellow named Jim, went on to develop a popular statically typed language?Anyways, lots of people use Emacs for Lisp (don’t you love the symmetry? you need to use Lisp for Emacs). They invest a lot of time making themselves more productive.
So here’s what I see from my chair: There’s this group of star-bellied sneetches who say “the important thing about being a productive programmer is having a tool that makes you productive.” And there’s this group of sneetches who say the same dammned thing. The only difference between them is that the star-bellied sneetches say that the productivity of the tool is all about the special things the tool can do when the programs are written in statically typed languages. The plain sneetches love the tool that they can customize so heavily it feels like a personal extension of their mind.
Hmmm.
Who deserves a title bout?
The thing that strikes me as obvious is that I’m the wrong person to argue with a star-bellied sneetch. I use a fairly nice text editor, TextMate, but it doesn’t do 1% of what Emacs does and while it does a lot more than I ever figured out how to make Eclipse do, it also doesn’t even try to do a lot of the star-bellied stuff like move methods around.The right thing to do is to get two people who both agree that productivity from IDEs is important to debate how to do it. That way there’s one, clear point of debate: does an IDE + static typing beat a programmable editor by an order of magnitude?
Debating with me is going to be entertaining but inconclusive, because we’ll be mixing several arguments up: are IDEs important, and if they are, what’s the best way to make an IDE powerful, and if that is the best way, is it so good that it outweighs other considerations for choosing a programming language? If you stop by my house, that’s a conversation that is going to put a serious dent in my collection of single malts.
The third perspective
I’m not a sneetch in this matter. I spend far, far more time thinking about code than I do editing code. Therefore, I invest more energy looking for ways to improve the way I think about code than I do looking for ways to improve the way I edit code.Kinda something not entirely unlike what happens when you sit down to improve the performance of a program. First you profile. Then you choose what to optimize. I’ve profiled myself, and what I need is more help deciding what to do than I do doing it.
I am also holding on to a somewhat axiomatic (and irrepressibly optimistic) belief that some languages have orders of magnitude higher signal-to-noise ratio than others. This is important for understanding why I am reluctant to pick a language because my IDE likes it.
For starters, one needs to understand the goal of all this refactoring: it’s to have better code. Better languages also achieve this goal. let’s look at a simple example and work backwards from that to the importance of languages.
Delegation and dynamic programming
Delegation in Ruby is easy because you havesend and you have method_missing, so even if you don’t feel like using the built-in SimpleDelegator or Forwardable, you can forward messages whenever and however you like.Whereas in Java, you need to define an
Interface, then make sure that person_doing_the_actual_work implements the interface, then you can use a DynamicProxy to implement delegation. Note that you can’t delegate to a built-in class unless it implements an interface, so you can delegate to ArrayList (because it implements the List interface), but you can’t delegate to String.(Well, you can forward calls to classes that don't implement interfaces, but you can't make the delegator look like a delegatee. So what you really have is a kind of private inheritence. You cannot, for example, decorate a
StringBuffer so that you can set a flag for whether it has been encoded for display in HTML or not, and decorate the append method so that you can throw an exception when an unencoded String is appended to an encoded StringBuffer).Why can’t you just generate the delegation using the IDE’s handy-dandy feature?
The problem is that code generation produces a static change. It’s a one-time thing, like one of those push-button programming wizards from MSFT. If you change interfaces, perhaps you are refactoring something else, your delegation breaks. So you have to write something relatively dynamic.
In Ruby, it looks things up at run time. I’ll settle for something that can reflect at compile time. But either way, you can’t just do it once. You have to have dynamic code handle the forwarding.
(There are exactly similar problems in all sorts of code. If you write some serialization code by hand or with an IDE, you have a problem when you expand an object. The old code doesn’t know how to serialize the new members. You have to make sure your serialization code reflects on the object so it can keep up with all changes.)
I find that IDEs are nice, but even the most sophisticated IDEs are… static. They make one-time changes to code. I’m far more interested in languages that let me write code that writes code, in languages that let me write code that doesn't need a lot of rewriting and fixing across the board when I make a change. (This is just my perspective. I’m still learning a lot about the intersection between theory and practice in programming languages.)
But this is why you’re tilting at windmills when you ask me why I don’t eschew Ruby for Java just so I can “rename class.”
Thursday, October 19, 2006
Would you like a side of referential transparency with your order of static typing?
Recapitulating some of the arguments for and against static typing has been very refreshing. And thanks to everyone who took the time to share their point of view.
Leaving aside the argument that static typing helps your IDE help you, the really big idea behind modern static typing is that because certain properties of variables are invariant, it is tractable to perform a lot of analysis on a program looking for contradictions. For example, we say that
And do you know what? Although I accept that this is true, and even useful, I haven’t personally been swayed by it (I’m just going to give my experience here, not a prescription or advice to others). The problem, as I see it, is that the statically typed languages I’ve used for production work have had such primitive typing systems that I couldn’t use them to solve really important problems.
Errors where I mistakingly try to call
But I had a sudden “Oho, you’re busted!” moment a few days ago. Didn’t I write a nice post explaining why mutable local variables are bad? The gist of my argument was… wait for it… mutable things make it hard to move stuff around, because you don’t have those nice invariants to reason about. Hmmm. Could there be a strong parallel between getting rid of mutable variables and static typing?
Well… I could argue that dynamic meta-programming is worth more to me than the benefits of static typing. They really are worth a lot more than the benefits of the simplistic typing systems you find in popular languages. But are they worth more than the powerful systems in languages like Haskell or ML? Maybe not.
And how dynamic is my meta-programming? I love the fact that I can use constructs like
Does this sounds familiar? I’m confessing that one of the reasons I like Ruby is that it’s easy to write things that are static, that don’t change state. But Ruby is all about having flexible things that change at runtime. This is what you might call a paradigm smell, the paradigm of the language—types change on the fly—is at odds with the kind of programs I try to write in Ruby.
Isn’t that interesting?
p.s. Okay, all of you static typing fans who are rolling up your sleeves to write an “I told you so” comment: before you hit “publish,” ...
Are you curious about what would happen if you turned the static typing knob up to eleven? If you took the Red Pill? Could you use a really powerfully typed language to detect XSS vulnerabilities in the compiler? Could you switch from an impertitive, loop programming style to a functional style and get rid of mutable variables? Could you express your domain logic declaratively in a DSL instead of in procedures and methods? If a little compile-time analysis is good, how much better could a lot of compile-time analysis be?
Leaving aside the argument that static typing helps your IDE help you, the really big idea behind modern static typing is that because certain properties of variables are invariant, it is tractable to perform a lot of analysis on a program looking for contradictions. For example, we say that
foo is an Integer, and then a little later on we call foo.append . Since Integers don’t implement a method for appending, we know that there is an error in the program without having to run the program.Thoſe who would give up Essential Liberty to purchaſe a little Temporary Safety, deſerve neither Liberty nor Safety.
And do you know what? Although I accept that this is true, and even useful, I haven’t personally been swayed by it (I’m just going to give my experience here, not a prescription or advice to others). The problem, as I see it, is that the statically typed languages I’ve used for production work have had such primitive typing systems that I couldn’t use them to solve really important problems.
Errors where I mistakingly try to call
append on a String when I should be calling it on a StringBuffer just don’t make up for all the extra verbiage and the onerous restrictions on meta-programming imposed by popular languages.But I had a sudden “Oho, you’re busted!” moment a few days ago. Didn’t I write a nice post explaining why mutable local variables are bad? The gist of my argument was… wait for it… mutable things make it hard to move stuff around, because you don’t have those nice invariants to reason about. Hmmm. Could there be a strong parallel between getting rid of mutable variables and static typing?
Static typing and stateless programming
Yes, of course there is. It’s about minimizing the state changes. Static typing is about having just one state for each variable’s type. Programming with immutable variables is about having just one value for each variable. I advocate the latter. Why haven’t I embraced the former?Well… I could argue that dynamic meta-programming is worth more to me than the benefits of static typing. They really are worth a lot more than the benefits of the simplistic typing systems you find in popular languages. But are they worth more than the powerful systems in languages like Haskell or ML? Maybe not.
And how dynamic is my meta-programming? I love the fact that I can use constructs like
acts_as_versioned in Ruby, but there are languages that allow static meta-programming (like Scheme’s macros) that would go as far for much of what I do. Much farther than the restrictive straight-jacket of popular languages, anyways.Paradigm smells
This brings me to writing DSLs in Ruby. One of the reasons DSLs are incredibly useful is that they are declarative: The what is cleanly separated from the how. Lots of successful DSLs are “business rules”, they aren’t statements to be executed, they’re constraints on the behaviour of a system. Just like static types are constraints.Does this sounds familiar? I’m confessing that one of the reasons I like Ruby is that it’s easy to write things that are static, that don’t change state. But Ruby is all about having flexible things that change at runtime. This is what you might call a paradigm smell, the paradigm of the language—types change on the fly—is at odds with the kind of programs I try to write in Ruby.
Isn’t that interesting?
p.s. Okay, all of you static typing fans who are rolling up your sleeves to write an “I told you so” comment: before you hit “publish,” ...
Are you curious about what would happen if you turned the static typing knob up to eleven? If you took the Red Pill? Could you use a really powerfully typed language to detect XSS vulnerabilities in the compiler? Could you switch from an impertitive, loop programming style to a functional style and get rid of mutable variables? Could you express your domain logic declaratively in a DSL instead of in procedures and methods? If a little compile-time analysis is good, how much better could a lot of compile-time analysis be?
Good taste for software design
This just landed in my in box:
"Good taste for software design." I love how our industry is shaping up: proud development leaders like Dmitri are not afraid to seek the "quality without a name."
I am looking for a talented .NET developer/architect to drive a
challenging and ambitious project.
Strong hands-on .NET, good taste for software design, agile mindset,
and exposure to Test Driven Development are the must.
The project is long term, therefore permanent is preferred, yet
contract is negotiable for a right candidate.
Please spread the word; if someone is interested, contact Dmitri at
Opalis.com for details.
"Good taste for software design." I love how our industry is shaping up: proud development leaders like Dmitri are not afraid to seek the "quality without a name."
Monday, October 16, 2006
Irony
From time to time people quote this as a justification for staying away from high-level languages like Lisp, Scheme, Python, and Ruby, and not only sticking to popular languages but also staying within the lowest-common denominator when choosing idioms within those popular languages.
Isn’t it interesting that the quote is from a book explaining concepts such as recursion and metalinguistic abstraction? In Scheme?
The full text is on line. If you have yet to read it, I recommend you give it a try.
update: if you enjoyed reading SICP, you may find my list of the first seven books I would buy if my shelves were bare interesting.
Programs must be written for people to read, and only incidentally for machines to execute.
Abelson & Sussman, Structure and Interpretation of Computer Programs
From time to time people quote this as a justification for staying away from high-level languages like Lisp, Scheme, Python, and Ruby, and not only sticking to popular languages but also staying within the lowest-common denominator when choosing idioms within those popular languages.Isn’t it interesting that the quote is from a book explaining concepts such as recursion and metalinguistic abstraction? In Scheme?
The full text is on line. If you have yet to read it, I recommend you give it a try.
update: if you enjoyed reading SICP, you may find my list of the first seven books I would buy if my shelves were bare interesting.
Sunday, October 15, 2006
In praise of informed choices
“You use J, therefore you’re a Blub Programmer.”
“There’s no such thing as Blub, therefore I can’t be a Blub Programmer.”
For the remainder of this post, we are going to discuss things in terms of informed and uninformed choices. This is a functional description and it focuses on a one-time behaviour. If you like the old terms, you know where to find grep.
One argument against the idea that there is one overarching continuum of language or tool “power” is that tools specialize. The most appropriate tool for building CRUD database-backed web applications may not be the most appropriate tool for building interactive games.
The specialization argument is invoked quite often. When I hear it from someone who has taken the time to become comfortable with several different approaches to solving a problem, I give it some credence.
But how often do we hear someone say, “Oh, language X. Yeah, I hear that’s kind of specialized for Artificial Intelligence. And Y is probably terrific if you need to do some startup thing where you need to make a lot of changes as you go. I haven’t tried it, because I don’t do Y-like stuff. But for my purposes Z is the way to go. Each language has its specialization, and Z is best for what I’m doing.”
Are those the words of someone who really understands X or Y and why anyone would need those weird features X and Y programmers boast of? Or what parts of languages X and Y would or wouldn’t apply to the problem at hand? Or is this just a way of sounding Fair and Balanced?
Just because tools may provide specialized benefits depending on the problem at hand, that doesn’t mean that the particular tool a programmer chooses is the best possible or best available. In other words, uninformed choice can be local to a problem domain.
The older a language or tool is, the more likely it is a general-purpose tool. The converse tends to be true: the younger a language or tool is, the more likely it is to be specialized. The latter seem to be true even if the designers and sponsors intend it to be a general-purpose tool, because younger tools have smaller ecosystems.
Take a language with a relatively small ecosystem: it may have been designed as a general-purpose tool, but if the the most popular stable, widely used libraries and frameworks support building Ajax-ified CRUD applications, it is a specialized tool as compared to a language that seems to support building everything up to an including interplanetary navigation.
So It seems that some languages boast flexibility, others specialization. It seems unrealistic to expect that the specialized language can beat the generalized language. Unless you are comparing the languages on the basis of the specialized language’s “home turf.” In a fight between a bear and an alligator, terrain determines the outcome.
Now someone argues, “Hey, you wouldn’t try to fight a fire with a Maserati or drive the Autobahn on a Segway.” The specialization argument. Ok. But is that really the comparison? All too often, I hear that argument, but what I see in the toolbox is a mini-van, the king of popularity, the vehicle that does several different things equally poorly.
Honestly. Are programmers clinging to Haskell, Erlang, and Postscript because their needs are specialized and within their niche these languages are the best possible choice? Or are they vocally defending a language that does an awful lot of things in awkward ways against languages that solve their exact problem in an elegant way?
Java is not an uninformed choice, and neither is Visual Basic
Languages and tools are not uninformed. It’s the breadth of experience applied to making a choice that counts. Making a popular, comfortable, established, unfashionable, safe choice does not make you uninformed. An uninformed choice is made in ignorance of the benefits of making other choices.
If you think that your next CRUD application would need anywhere from one half the code to one tenth the code if you used a specialized meta-programming tool like Ruby on Rails, but you’re sticking with ASP because that’s what the client asked for, you’re making an informed choice.
If you write a new application and make heavy use of functional programming constructs for the first time, good for you. If you choose to leave an existing application more or less as-is and you don’t refactor every method to remove mutable local variables, good for you again. You’re making an informed choice.
Some uninformed choices are conservative choices, but so are are some informed choices. You can’t tell the difference merely by examining the language or tool chosen.
When you see a program written in a conservative, “safe” language, and you want to know whether the programmer is a made a fully informed choice, examine the source code looking for evidence that the programmer has been exposed to other programming paradigms.
Uninformed programmers invariably stick to whatever has been promoted as the sole, orthodox way to write programs for their language. If they need to work around a language limitation, they will do so using well known conventions such as widely disseminated “design patterns.” They can invariable name a dozen or more such patterns, but can never name a single language that handles these problems in a different way.
The informed but conservative programmer has learnt something from other tools, other languages. The principles find their way into her work. A conservative programmer might have extremely pragmatic reasons for doing some work in Javascript. But she uses the Prototype Library to implement higher-order functional programming. Or she might need to build something really big and hairy in C++. But she’ll implement Lisp’s map and reduce as a library so that C++ programs can vectorize on a grid with thousands of computers.
The uninformed programmer may have heard of these things, but not only will they eschew other tools and languages, the eschew the ideas behind them as well.
Although the act of implementing a more powerful language on top of a more conservative language is sometimes frustrating and often derided by purists, it is a sign that the programmer has thought deeply about the best way to solve a particular problem, and is not afraid to port what they like from one environment to another.
What do you think? Do informed but conservative programmers incorporate features and paradigms from other tools and languages into their work?
First, a word from our sponsor
You know, this whole “Blub” thing has too many negative connotations. It has become, like “Star-Bellied Sneetch,” a way of labeling people.“You use J, therefore you’re a Blub Programmer.”
“There’s no such thing as Blub, therefore I can’t be a Blub Programmer.”
For the remainder of this post, we are going to discuss things in terms of informed and uninformed choices. This is a functional description and it focuses on a one-time behaviour. If you like the old terms, you know where to find grep.
One argument against the idea that there is one overarching continuum of language or tool “power” is that tools specialize. The most appropriate tool for building CRUD database-backed web applications may not be the most appropriate tool for building interactive games.
Success through specialization
Within a particular specialization, there is still a continuum of power. There may be a programmer who insists that Visual Basic backed ASP pages provide all the power she will ever need for CRUD web applications, and there is no need to Ruby, ASP.NET, or any of the other advances on the technology she likes. This sounds remarkably like she is making an uninformed choice, doesn’t it?The specialization argument is invoked quite often. When I hear it from someone who has taken the time to become comfortable with several different approaches to solving a problem, I give it some credence.
But how often do we hear someone say, “Oh, language X. Yeah, I hear that’s kind of specialized for Artificial Intelligence. And Y is probably terrific if you need to do some startup thing where you need to make a lot of changes as you go. I haven’t tried it, because I don’t do Y-like stuff. But for my purposes Z is the way to go. Each language has its specialization, and Z is best for what I’m doing.”
Are those the words of someone who really understands X or Y and why anyone would need those weird features X and Y programmers boast of? Or what parts of languages X and Y would or wouldn’t apply to the problem at hand? Or is this just a way of sounding Fair and Balanced?
Just because tools may provide specialized benefits depending on the problem at hand, that doesn’t mean that the particular tool a programmer chooses is the best possible or best available. In other words, uninformed choice can be local to a problem domain.
In a fight between a bear and an alligator
My second conjecture about specialized tool hypothesis is that most tools evolve over time away from specialization and towards generalization. Java, the language often mentioned in the same breath as uninformed choice, began its life as a specialized language for embedded programming. Toasters, set-top boxes, that kind of thing. Other languages have seen the same trend towards trying to solve all problems for all people.The older a language or tool is, the more likely it is a general-purpose tool. The converse tends to be true: the younger a language or tool is, the more likely it is to be specialized. The latter seem to be true even if the designers and sponsors intend it to be a general-purpose tool, because younger tools have smaller ecosystems.
Take a language with a relatively small ecosystem: it may have been designed as a general-purpose tool, but if the the most popular stable, widely used libraries and frameworks support building Ajax-ified CRUD applications, it is a specialized tool as compared to a language that seems to support building everything up to an including interplanetary navigation.
So It seems that some languages boast flexibility, others specialization. It seems unrealistic to expect that the specialized language can beat the generalized language. Unless you are comparing the languages on the basis of the specialized language’s “home turf.” In a fight between a bear and an alligator, terrain determines the outcome.
Now someone argues, “Hey, you wouldn’t try to fight a fire with a Maserati or drive the Autobahn on a Segway.” The specialization argument. Ok. But is that really the comparison? All too often, I hear that argument, but what I see in the toolbox is a mini-van, the king of popularity, the vehicle that does several different things equally poorly.
Honestly. Are programmers clinging to Haskell, Erlang, and Postscript because their needs are specialized and within their niche these languages are the best possible choice? Or are they vocally defending a language that does an awful lot of things in awkward ways against languages that solve their exact problem in an elegant way?
Java is not an uninformed choice, and neither is Visual Basic
Languages and tools are not uninformed. It’s the breadth of experience applied to making a choice that counts. Making a popular, comfortable, established, unfashionable, safe choice does not make you uninformed. An uninformed choice is made in ignorance of the benefits of making other choices.If you think that your next CRUD application would need anywhere from one half the code to one tenth the code if you used a specialized meta-programming tool like Ruby on Rails, but you’re sticking with ASP because that’s what the client asked for, you’re making an informed choice.
If you write a new application and make heavy use of functional programming constructs for the first time, good for you. If you choose to leave an existing application more or less as-is and you don’t refactor every method to remove mutable local variables, good for you again. You’re making an informed choice.
Some uninformed choices are conservative choices, but so are are some informed choices. You can’t tell the difference merely by examining the language or tool chosen.
Conservative? Or just Uninformed?
I have a little litmus test for identifying an informed choice that just happens to be conservative. I don’t claim it’s infallible, and I have done absolutely no research to back it up. But in the interest of keeping the amusement value up, here is my proposal:When you see a program written in a conservative, “safe” language, and you want to know whether the programmer is a made a fully informed choice, examine the source code looking for evidence that the programmer has been exposed to other programming paradigms.
Uninformed programmers invariably stick to whatever has been promoted as the sole, orthodox way to write programs for their language. If they need to work around a language limitation, they will do so using well known conventions such as widely disseminated “design patterns.” They can invariable name a dozen or more such patterns, but can never name a single language that handles these problems in a different way.
The informed but conservative programmer has learnt something from other tools, other languages. The principles find their way into her work. A conservative programmer might have extremely pragmatic reasons for doing some work in Javascript. But she uses the Prototype Library to implement higher-order functional programming. Or she might need to build something really big and hairy in C++. But she’ll implement Lisp’s map and reduce as a library so that C++ programs can vectorize on a grid with thousands of computers.
The uninformed programmer may have heard of these things, but not only will they eschew other tools and languages, the eschew the ideas behind them as well.
Although the act of implementing a more powerful language on top of a more conservative language is sometimes frustrating and often derided by purists, it is a sign that the programmer has thought deeply about the best way to solve a particular problem, and is not afraid to port what they like from one environment to another.
What do you think? Do informed but conservative programmers incorporate features and paradigms from other tools and languages into their work?
Labels: java
Thursday, October 12, 2006
Are we Blub programmers?
From time to time I open my email and find someone asking a question:
Actually, Blub is a hypothetical programming language Paul Graham invented when describing something very interesting: the Blub Paradox:
The interesting things about this paradox is that almost any language could be Blub. The pre-requisites for a language being Blub are (a) there is at least one language less powerful than Blub, (b) there is at least one language more powerful than Blub, and (c) that there be at least one programmer using Blub who accepts (a) but refutes (b) because he or she cannot see how the more powerful language is more powerful. She does not think in the idioms that the more powerful language affords.
The Little MLer introduces ML (and Ocaml) through a series of entertaining and straightforward exercises leading up to the construction of the Y Combinator.
ML and OCaml introduce powerful strong typing and type inference. Both are great languages to learn: you will stretch your understanding of defining types and writing correct programs.
When I use the term, I am thinking of the language and also the programmers around it. Could Java be Blub? Sometimes, possibly often, but only when I'm thinking about Java programmers who dismiss Ruby's features as unnecessary. Could Ruby be Blub? Sometimes, but only when I'm thinking about Ruby programmers who dismiss macros as unimportant.
Could Lisp be Blub? I suspect that Erlang and Haskell programmers might say that it is, provided we can find a Lisp programmer who feels that all progress in programming languages stopped when Common Lisp was standardized.
At the same time, Java is not Blub when I am thinking of programmers who are perfectly aware of its shortcomings and deliberately Greenspun around them for pragmatic reasons. The same goes for any other language: it is sometimes Blub, and sometimes not Blub.
So... I use the term "Blub" to refer to a programming language in the context of intransigent programmers who feel that their chosen tool is the best tool possible.
This provokes a very obvious question: How do we know which things are accidentally difficult and which are actually difficult? Is it only because we haven't discovered the right tool yet?
It's easy to find a Java programmer who believes that all of the Design Patterns in the GoF's book are necessary. She believes that the difficulties of applying those patterns are actual difficulties of programming systems. It is only when she learns a different language that she realizes how the patterns were strongly driven by limitations in Java's object model.
At that point she has an epiphany and understands that what she thought were actual difficulties were merely accidental difficulties. And the line between "accidental" and "actual" moves for her.
No matter how much each us us thinks we know right now, are we nevertheless like this Java programmer, unable to see the difference between accidental and actual differences because we simply haven't discovered a more powerful tool?
Are we Blub programmers?
update: In praise of informed choices and Lisp is not the last word
What the hell is Blub? I take from context that it may be a pejorative generalist term for programming languages that encourage writing pablum instead of programs. Or maybe it really is a new language out there that has a huge following I'm unaware of.
Actually, Blub is a hypothetical programming language Paul Graham invented when describing something very interesting: the Blub Paradox:Blub falls right in the middle of the abstractness continuum... As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages... Blub is good enough for him, because he thinks in Blub.
Paul Graham, Beating the Averages
The interesting things about this paradox is that almost any language could be Blub. The pre-requisites for a language being Blub are (a) there is at least one language less powerful than Blub, (b) there is at least one language more powerful than Blub, and (c) that there be at least one programmer using Blub who accepts (a) but refutes (b) because he or she cannot see how the more powerful language is more powerful. She does not think in the idioms that the more powerful language affords.
The Little MLer introduces ML (and Ocaml) through a series of entertaining and straightforward exercises leading up to the construction of the Y Combinator.
ML and OCaml introduce powerful strong typing and type inference. Both are great languages to learn: you will stretch your understanding of defining types and writing correct programs.
Could Lisp be Blub? I suspect that Erlang and Haskell programmers might say that it is, provided we can find a Lisp programmer who feels that all progress in programming languages stopped when Common Lisp was standardized.
At the same time, Java is not Blub when I am thinking of programmers who are perfectly aware of its shortcomings and deliberately Greenspun around them for pragmatic reasons. The same goes for any other language: it is sometimes Blub, and sometimes not Blub.
So... I use the term "Blub" to refer to a programming language in the context of intransigent programmers who feel that their chosen tool is the best tool possible.
Programming consists of overcoming two things: accidental difficulties, things which are difficult because you happen to be using inadequate programming tools, and things which are actually difficult, which no programming tool or language is going to solve.
This provokes a very obvious question: How do we know which things are accidentally difficult and which are actually difficult? Is it only because we haven't discovered the right tool yet?
It's easy to find a Java programmer who believes that all of the Design Patterns in the GoF's book are necessary. She believes that the difficulties of applying those patterns are actual difficulties of programming systems. It is only when she learns a different language that she realizes how the patterns were strongly driven by limitations in Java's object model.
At that point she has an epiphany and understands that what she thought were actual difficulties were merely accidental difficulties. And the line between "accidental" and "actual" moves for her.
No matter how much each us us thinks we know right now, are we nevertheless like this Java programmer, unable to see the difference between accidental and actual differences because we simply haven't discovered a more powerful tool?
Are we Blub programmers?
update: In praise of informed choices and Lisp is not the last word
Tuesday, October 10, 2006
What colour is your parachute?
You know the old saying:
My World
I write about my experience, what has worked for me and what I think will continue to work for me. I'm not presenting a unified theory of everything. I know for a fact that my world is different from your world.
Case in point: everything I write is from the perspective of small teams of skilled practitioners. Yes, I know that there are people who try to build big pieces of software with one hundred and eighty programmers. I'm not one of them. Furthermore, I have absolutely no interest in trying to find techniques or tools that "scale" to hundreds of programmers.
I assume you have a modicum of intelligence and an open mind. I assume you will temper my blacks and whites, you will think them over, and you will figure out whether or how they apply to you.
I did some programming in Scheme a long time ago. Then I learned Java. One day I was responsible for writing approximately four hundred classes in a Struts application. I wrote a code generator that wrote the classes for me. The code generator actually worked off of an Excel spreadsheet that some business analyst used for writing business rules.
Java and Scheme are different worlds. But if you squint a little, and tilt your head just so, you might see some common ground.
Bippety Bop, Bang!
When I write something and say "this is a guideline, use it in these situations but eschew it in these other situations, and temper it with this technique," everyone reads it, nods, and forgets about it. No impact. No passion.
When I say "this is bad, and that is good," I get fan mail. I get hate mail. I get flowers at work. I get dog poop in a burning paper bag on my door step. (Well, not enough fan mail, no flowers, and hardly any hate mail. I'm not that good at writing.) But I get one more thing that I can't ignore: the posts incite debate. People argue for and against. With themselves. With me. With each other.
And to me, that's the point. If you are motivated enough to say, "Reg, your idea is no damned good because of A, B, and C," I think we're making some progress. We're talking. We're thinking for ourselves. We don't have some corporation with a billion dollar marketing budget telling us how to program and what's important in our lives.
Do you really think there are no mutable local variables in my code? Come on. But... don't you think I sincerely try to use alternatives whenever I can do so without crippling the code? Yes, I use alternatives in most cases.
University Politics
Sometimes the debates over these things get heated. This is a problem. I think it happens because people have difficulty understanding and articulating their own values. So I say something about static and dynamic typing, and I get people saying I'm stupid or a poor poor programmer (both possibly true).
What's going on? I think that people have some personal investment in these things. We're a little tribal. We think that because we have stars on our bellies, we're better than those without stars. Then someone says "stars are no dammed good."
What happens? We get hot and bothered, because we think that person was really saying "you and your entire value system are no dammed good." But we can't say that, because we're pretending this is about productivity, and methodology, and scaling to two hundred programmers.
That's a shame, because getting in touch with what's really important to us and shaping our lives around it is so incredibly important. I really wish more people could be like this fellow a few desks away from me who programs in .NET because that's where he thinks the money is.
Flames
I had a manager once. He was my boss's boss. When someone presented him with an idea, he would ask for proof it would work. If you wanted to try anything new, he demanded that it be risk-free. You could spend months getting the kind of data he demanded to prove that you couldn't fail.
Now, I don't think he was a malicious man. I doubt he even knew that he was the organization's number one idea shooter-downer. Too bad.
Anyways, a lot of people do this. If they see something new, they seize upon its flaws at once. They point out the missing bits. "It's just superstition, there's no proof that Ruby is more productive or that smaller programs are less buggy, or that static typing eliminates critical bugs, or, ..."
Well maybe so, but do you really think the people complaining about Cargo Cult Programming are rigorous scientists when they look at their own practices? No, they have a blind spot. The finicky rejection of ideas only applies to other ideas, new ideas.
And especially ideas that question some of the ideas that they already have.
A mind is like a parachute: it works better when it's open.So: what's with the black-and-white sounding statements I make about subjects like people ("no methodology can save a project staffed with under-performers") or local variables ("bad") or even static typing ("useful, but it doesn't prevent errors that endanger my projects")?
My World
I write about my experience, what has worked for me and what I think will continue to work for me. I'm not presenting a unified theory of everything. I know for a fact that my world is different from your world.
Case in point: everything I write is from the perspective of small teams of skilled practitioners. Yes, I know that there are people who try to build big pieces of software with one hundred and eighty programmers. I'm not one of them. Furthermore, I have absolutely no interest in trying to find techniques or tools that "scale" to hundreds of programmers.
I assume you have a modicum of intelligence and an open mind. I assume you will temper my blacks and whites, you will think them over, and you will figure out whether or how they apply to you.
I did some programming in Scheme a long time ago. Then I learned Java. One day I was responsible for writing approximately four hundred classes in a Struts application. I wrote a code generator that wrote the classes for me. The code generator actually worked off of an Excel spreadsheet that some business analyst used for writing business rules.
Java and Scheme are different worlds. But if you squint a little, and tilt your head just so, you might see some common ground.
Bippety Bop, Bang!
When I write something and say "this is a guideline, use it in these situations but eschew it in these other situations, and temper it with this technique," everyone reads it, nods, and forgets about it. No impact. No passion.
When I say "this is bad, and that is good," I get fan mail. I get hate mail. I get flowers at work. I get dog poop in a burning paper bag on my door step. (Well, not enough fan mail, no flowers, and hardly any hate mail. I'm not that good at writing.) But I get one more thing that I can't ignore: the posts incite debate. People argue for and against. With themselves. With me. With each other.
Because "considered harmful" essays are, by their nature, so incendiary, they are counter-productive both in terms of encouraging open and intelligent debate, and in gathering support for the view they promote. In other words, "considered harmful" essays cause more harm than they do good...
And to me, that's the point. If you are motivated enough to say, "Reg, your idea is no damned good because of A, B, and C," I think we're making some progress. We're talking. We're thinking for ourselves. We don't have some corporation with a billion dollar marketing budget telling us how to program and what's important in our lives.
Do you really think there are no mutable local variables in my code? Come on. But... don't you think I sincerely try to use alternatives whenever I can do so without crippling the code? Yes, I use alternatives in most cases.
University Politics
Sometimes the debates over these things get heated. This is a problem. I think it happens because people have difficulty understanding and articulating their own values. So I say something about static and dynamic typing, and I get people saying I'm stupid or a poor poor programmer (both possibly true).
What's going on? I think that people have some personal investment in these things. We're a little tribal. We think that because we have stars on our bellies, we're better than those without stars. Then someone says "stars are no dammed good."
What happens? We get hot and bothered, because we think that person was really saying "you and your entire value system are no dammed good." But we can't say that, because we're pretending this is about productivity, and methodology, and scaling to two hundred programmers.
That's a shame, because getting in touch with what's really important to us and shaping our lives around it is so incredibly important. I really wish more people could be like this fellow a few desks away from me who programs in .NET because that's where he thinks the money is.
Flames
I had a manager once. He was my boss's boss. When someone presented him with an idea, he would ask for proof it would work. If you wanted to try anything new, he demanded that it be risk-free. You could spend months getting the kind of data he demanded to prove that you couldn't fail.
Now, I don't think he was a malicious man. I doubt he even knew that he was the organization's number one idea shooter-downer. Too bad.
Anyways, a lot of people do this. If they see something new, they seize upon its flaws at once. They point out the missing bits. "It's just superstition, there's no proof that Ruby is more productive or that smaller programs are less buggy, or that static typing eliminates critical bugs, or, ..."
Well maybe so, but do you really think the people complaining about Cargo Cult Programming are rigorous scientists when they look at their own practices? No, they have a blind spot. The finicky rejection of ideas only applies to other ideas, new ideas.
And especially ideas that question some of the ideas that they already have.
Labels: passion
Sunday, October 08, 2006
If Sneetches with Stars use Java, and Sneetches without Stars use Ruby, who uses ML?
ML is a programming language featuring type inference: you don’t have to encumber your code with type declarations, the compiler can figure them out for you. So… are type inference languages like ML for Sneetches with or without stars? Or another kind of Sneetch entirely?
Let’s review. Sneetches with stars like to use a colour-coded label maker to label the drawers, boxes, and files in their office. Once glance at everything and you know what it holds. Sneetches with stars add extra labels even when you don’t need them. For example, if a box is labeled ‘tax receipts’, each piece of paper inside has a post-it note saying tax receipt’, even if it’s obviously a tax receipt and lives inside the tax receipts box.
Let’s dive into this a little deeper. (My apologies to my readers who were actually paying attention to the stuff in first year computer science that isn’t a requirement for getting a job at BigCo.) What does it mean when we say “something can be resolved at compile time”? That expression is laden with implementation details like assuming we’re using a compiler. But it’s a convenient short-hand for saying something about the program that is true every time you run the program.
Consider the
Java can take advantage of this to perform constant propagation. Everywhere you write
Now back to the word stariness. We really mean the amount of stuff about the program that can be resolved at compile time, or if you prefer, the amount of stuff that is true every time you run the program.
In the example above, the compiler can figure out which branch the program will follow at compile time, because that variable is true every time you run the program.
Stuff that is always true is useful. For most programs, we have an idea in our head about “correctness.” What we mean when we talk about a program being correct is that it produces desirable results every time you run the program.
It feels a lot like it, but it isn’t the same thing. The reason it isn’t the same thing is that while its true that a program throwing
Now that we’ve dispatched that logically, let’s be clear about something: just because stariness does not enforce correctness, it doesn’t mean that stariness isn’t useful. Stariness is useful. Period, no debate.
Languages with type inference look like variables can have any type, but the compiler knows better. Remember the labels that the verbose declaration Sneetches with stars love? Type inference languages still have labels, but the labels are hidden inside of the files and boxes where you can’t see them.
Remember when manufacturers used to put their labels inside clothes instead of right across the front? Same thing. The rules for what goes where are strictly enforced, it’s just that if you can figure out what goes where with a bit of common sense, you don’t need a label or a post-it note.
Compare these two snippets of Java:
(Frank Atanassow pointed out that techniques exist for inferring the types of nearly all Java variables through inspection of programs. But this simple case is enough for our purposes.)
So if we take a valid Java program and simply erased type declarations whenever we could logically deduce the type of the variables (using our simple scheme), but left them in whenever we were not sure of the final type of the variables, we would have exactly the same program. Nothing about it has changed except it has fewer symbols. It’s just as starry, it is just as static, it is no more or less correct than it was before we erased some symbols.
And you over there itching to say something about IDE refactorings and auto-completions: None of those go away either. You can rename things and move things and press command-tab to get an object’s methods whenever you like. So… would you agree that type inference of this sort doesn’t change a starry program into a starless program? This isn’t about stariness versus starlessness, it’s about the obsessive-compulsive desire to label everything.
The bottom line: type inference does not change a statically typed language into a dynamically typed language. It’s still starry.
And whenever someone talks about a refactoring IDE or an auto-completing IDE for a dynamic language, they’re talking about performing some type inference on Ruby programs that are written in a static way. So… what’s the holdup? We said we could add type inference to Java in a Summer. Where’s the intern to add it to Ruby?
The problem is that the set of all programs that are "starry" is a subset of the set of all programs that parse correctly. So either not all starless programs are neat, or not all portions of a starless program are neat, or both.
Let’s compare back to our Java snippet. Remember:
For a very large class of programs, we cannot infer the contents of a variable with less runtime complexity than running the program for every possible input. This is why compilers have limitations on the optimizations they can perform, and humans still need to do some thinking about writing fast programs.
This exact same thing happens with types. In statically typed languages, types are never re-assigned. Whether explicitly declared or inferred, they're immutable. But in languages like Ruby where methods can be added and removed dynamically, where messages can be forwarded dynamically, where we can even send messages dynamically, the types of objects are fully mutable.
In starless languges, there is no
The type inference problem in dynamically typed languages is exactly the same as the inferring the possible contents of a variable problem. The inferring the contents of a variable problem is doable for a restricted set of programs. And the way we tell the compiler that a variable is a member of this restricted set is with the
Likewise, the way we tell a compiler that the type of a variable is also restricted is that we use a language where the type of every variable is final. It’s the same thing: we don’t reassign final variables and we don’t change types on the fly.
Starlessness is not about writing programs without labels. Starlessness is when you write dynamic programs. Dynamic doesn’t mean ‘unlabeled’. As I showed above, if the
Truly starless programs have dynamic types: types that change at run time. they are not always one thing or another. For example, what if you write an Object-Relational Mapper (“ORM”) that reflects on the database structure at run time. That is, you can change columns in a database table and you get new getters and setter methods in your program. Without recompiling.
In a fully static language (with or without type inference), you can’t do that. Think of Java’s JDBC: you have to fool around with methods that get values and pass a column name as a parameter. Or maybe you create a hash. And C# is getting this capability, but of you look closely you still have to define the “type” of a query through the LINQ syntax.
But what happens if you write the same thing in a neat program? Aha! a
This is a specific case of Greenspunning. There are some facilities of dynamic languages that you are going to need. If you don’t have them built into your static language, you will build them yourself or use a framework that has already built them for you. Other examples I have seen in Java programs include:
Spring and Hibernate;
Any use of
Any use of dynamic proxies;
In essence, you’re being a Sneetch without a star but twisting your starry language to permit starlessness. And for those portions of the program that are no longer nice, starry bundles that can be examined at compile time for invariant behaviour, you are indeed in dynamic territory and have to live with the risks.
In my experience, all non-trivial starry programs contain this kind of starlessness. To my admittedly inexperienced eyes, starlessness is the hallmark of expert programming in starry languages ("expert" does not necessarily mean "more desirable," especially in the minds of those who believe that programs should be written and maintained by mediocre developers).
Yes, you can say that. And you may be right, for you. The Boo people believe that: their language has a
If you dislike the verbosity of starry languages like Java but like the feeling of safety, try a type inference language. Don’t go to a starless language if you don’t intend to actually write dynamically typed programs.
My experience is that if you are frustrated by the amount of work you have to do to express the algorithms in your head, you should look at a language that removes your frustration. If you're using Java and don't like the verbosity, find a language that celebrates brevity while preserving static typing. But if you're using Java and find yourself pushing more and more logic out of java because its type system is too static or too inflexible, you should consider a language with a different approach to typing.
You can take that statement as an argument in favour of specialized languages for Sneetches without stars or as an argument against them. My guess is that the above statement is true and a Rorschach Inkblot: You will interpret it as confirmation of your existing prejudices.
Now, the Star-Belly Sneetches had bellies with stars.Update: More than a few people have written that Steve Yegge's association of static typing with neatness and dynamic typing with slovenliness runs opposite to their impressions of the kinds of people who strongly prefer one or the other. I used Steve's terms in the original post, partly because I thought people would get the same joke I thought Steve was making. It looks like they don't, nobody wrote to say "LOL." I have changed the terms to something that represents what I think of the cultural divide between programmers who like Java and programmers who like Ruby.
The Plain-Belly Sneetches had none upon thars.
Those stars weren’t so big. They were really so small
You might think such a thing wouldn’t matter at all.
Let’s review. Sneetches with stars like to use a colour-coded label maker to label the drawers, boxes, and files in their office. Once glance at everything and you know what it holds. Sneetches with stars add extra labels even when you don’t need them. For example, if a box is labeled ‘tax receipts’, each piece of paper inside has a post-it note saying tax receipt’, even if it’s obviously a tax receipt and lives inside the tax receipts box.
What is Stariness?
Sneetches with stars like these languages we say are statically typed. What do we mean by the word static? We mean it can be resolved at compile time. Other words for this idea are invariant or constant. Sneetches with stars like languages where the type of each entity can be resolved at compile time.Some people are always critical of vague statements. I tend rather to be critical of precise statements; they are the only ones which can correctly be labelled "wrong."
Raymond Smullyan
Let’s dive into this a little deeper. (My apologies to my readers who were actually paying attention to the stuff in first year computer science that isn’t a requirement for getting a job at BigCo.) What does it mean when we say “something can be resolved at compile time”? That expression is laden with implementation details like assuming we’re using a compiler. But it’s a convenient short-hand for saying something about the program that is true every time you run the program.
Consider the
final declaration in Java. If you write:final String snafu = "situation normal...";
snafu always holds a reference to the constant string "situation normal...". No matter what data you feed to your program and how you mangle it, snafu will always be "situation normal...". Do you agree? (Joe Campbell, put your hand down. Yes, there is a back door way you can change the contents of a String in Java.)Java can take advantage of this to perform constant propagation. Everywhere you write
snafu, Java can substitute "situation normal..." and throw away the variable lookup. To get away from arguing about back doors in the String class, let’s consider one of the primitive types, a boolean. If you write:final boolean foo = true;
// code without assignments to foo
if (foo) {
// do something
}
else {
// do something else
}
if statement? The path through the code is always through the // do something path every time you run the program.Now back to the word stariness. We really mean the amount of stuff about the program that can be resolved at compile time, or if you prefer, the amount of stuff that is true every time you run the program.
In the example above, the compiler can figure out which branch the program will follow at compile time, because that variable is true every time you run the program.
Stuff that is always true is useful. For most programs, we have an idea in our head about “correctness.” What we mean when we talk about a program being correct is that it produces desirable results every time you run the program.
A formalist is one who cannot understand a theory unless it is meaningless.Stariness is thus similar to correctness. And that’s why a lot of people, the Sneetches with stars, are obsessed with it. Being able to “prove” something about their program (“the method call
foo.bar(5) never throws a MethodNotImplemented exception”) feels a lot like being able to prove that their program is correct.It feels a lot like it, but it isn’t the same thing. The reason it isn’t the same thing is that while its true that a program throwing
MethodNotImplemented exceptions is probably not correct, it’s not true that a program that doesn’t throw such exceptions is correct. It just feels, somehow, more likely to be correct because we’ve thrown out one of the infinite ways it can be incorrect.Now that we’ve dispatched that logically, let’s be clear about something: just because stariness does not enforce correctness, it doesn’t mean that stariness isn’t useful. Stariness is useful. Period, no debate.
Back to inferences
Type inference is also for Sneetches with stars. A language with type inference resolves the type of each entity at compile time by inspecting the program and figuring the types out through inspection. It’s a lot like the way a compiler can look at the Java code above and figure out that you always// do something and you never // do something else. The code looks sorta like you could go either way, but the compiler knows better.Languages with type inference look like variables can have any type, but the compiler knows better. Remember the labels that the verbose declaration Sneetches with stars love? Type inference languages still have labels, but the labels are hidden inside of the files and boxes where you can’t see them.Remember when manufacturers used to put their labels inside clothes instead of right across the front? Same thing. The rules for what goes where are strictly enforced, it’s just that if you can figure out what goes where with a bit of common sense, you don’t need a label or a post-it note.
Compare these two snippets of Java:
final String[] words = { "foo", "bar", "blitz" };
final int word_length = words.length;
final String[] anagrams = new String[word_length];
final words = { "foo", "bar", "blitz" };
final word_length = words.length;
final anagrams = new String[word_length];
(Frank Atanassow pointed out that techniques exist for inferring the types of nearly all Java variables through inspection of programs. But this simple case is enough for our purposes.)
So if we take a valid Java program and simply erased type declarations whenever we could logically deduce the type of the variables (using our simple scheme), but left them in whenever we were not sure of the final type of the variables, we would have exactly the same program. Nothing about it has changed except it has fewer symbols. It’s just as starry, it is just as static, it is no more or less correct than it was before we erased some symbols.
And you over there itching to say something about IDE refactorings and auto-completions: None of those go away either. You can rename things and move things and press command-tab to get an object’s methods whenever you like. So… would you agree that type inference of this sort doesn’t change a starry program into a starless program? This isn’t about stariness versus starlessness, it’s about the obsessive-compulsive desire to label everything.
The bottom line: type inference does not change a statically typed language into a dynamically typed language. It’s still starry.
So why can’t the Sneetches without stars use type inference?
Think of types as being like values and objects like variables. A statically typed language is one where there are no type re-assignments. Some languages enforce this. But if you write a program in a static way, you can still reason about it. This is why lots of people think that we can “neaten up” languages like Ruby by adding type inference to the compiler: they're thinking about programs that are neat to begin with, but we happen to have written them in a language for Sneetches without stars.And whenever someone talks about a refactoring IDE or an auto-completing IDE for a dynamic language, they’re talking about performing some type inference on Ruby programs that are written in a static way. So… what’s the holdup? We said we could add type inference to Java in a Summer. Where’s the intern to add it to Ruby?
Programmed. In me somewhere, he thought, there is a matrix fitted in place, a grid screen that cuts me off from certain thoughts, certain actions. And forces me into others. I am not free. I never was, but now I know that; that makes it different.
Philip K. Dick, "The Electric Ant"
The problem is that the set of all programs that are "starry" is a subset of the set of all programs that parse correctly. So either not all starless programs are neat, or not all portions of a starless program are neat, or both.
Let’s compare back to our Java snippet. Remember:
final boolean foo = true;
// code without assignments to foo
if (foo) {
// do something
}
else {
// do something else
}
final keyword as well:boolean foo = true;
// code that might have assignments to foo
if (foo) {
// do something
}
else {
// do something else
}
For a very large class of programs, we cannot infer the contents of a variable with less runtime complexity than running the program for every possible input. This is why compilers have limitations on the optimizations they can perform, and humans still need to do some thinking about writing fast programs.
This exact same thing happens with types. In statically typed languages, types are never re-assigned. Whether explicitly declared or inferred, they're immutable. But in languages like Ruby where methods can be added and removed dynamically, where messages can be forwarded dynamically, where we can even send messages dynamically, the types of objects are fully mutable.
In starless languges, there is no
final keyword on the types of objects. We can no longer infer the type of a variable in any but the simplest, degenerate cases.The type inference problem in dynamically typed languages is exactly the same as the inferring the possible contents of a variable problem. The inferring the contents of a variable problem is doable for a restricted set of programs. And the way we tell the compiler that a variable is a member of this restricted set is with the
final keyword.Likewise, the way we tell a compiler that the type of a variable is also restricted is that we use a language where the type of every variable is final. It’s the same thing: we don’t reassign final variables and we don’t change types on the fly.
Starlessness is not about writing programs without labels. Starlessness is when you write dynamic programs. Dynamic doesn’t mean ‘unlabeled’. As I showed above, if the
final keyword is there, the label is mostly optional. But if you don’t have final, you’re writing dynamic programs.Truly starless programs have dynamic types: types that change at run time. they are not always one thing or another. For example, what if you write an Object-Relational Mapper (“ORM”) that reflects on the database structure at run time. That is, you can change columns in a database table and you get new getters and setter methods in your program. Without recompiling.
In a fully static language (with or without type inference), you can’t do that. Think of Java’s JDBC: you have to fool around with methods that get values and pass a column name as a parameter. Or maybe you create a hash. And C# is getting this capability, but of you look closely you still have to define the “type” of a query through the LINQ syntax.
Are Sneetches with stars ever starless?
A dynamically typed language lets us define an object holding a database row with methods for each column. But we can’t know at compile time whether our program will throw aMethodNotImplemented exception because we don’t know whether someone will monkey with the database structure. That sounds bad.But what happens if you write the same thing in a neat program? Aha! a
SQLException! it seems that there are dynamic things that must be dynamic no matter what you do.This is a specific case of Greenspunning. There are some facilities of dynamic languages that you are going to need. If you don’t have them built into your static language, you will build them yourself or use a framework that has already built them for you. Other examples I have seen in Java programs include:
Spring and Hibernate;
Any use of
Class.forName(...);Any use of dynamic proxies;
In essence, you’re being a Sneetch without a star but twisting your starry language to permit starlessness. And for those portions of the program that are no longer nice, starry bundles that can be examined at compile time for invariant behaviour, you are indeed in dynamic territory and have to live with the risks.
In my experience, all non-trivial starry programs contain this kind of starlessness. To my admittedly inexperienced eyes, starlessness is the hallmark of expert programming in starry languages ("expert" does not necessarily mean "more desirable," especially in the minds of those who believe that programs should be written and maintained by mediocre developers).
Eating cake
So… can we say that since you can write starless programs in neat languages, you can have the useful benefits of stariness when you need it and the flexibility of starlessness when you need that too? Isn’t that better?Yes, you can say that. And you may be right, for you. The Boo people believe that: their language has a
duck keyword for when you feel like a Sneetch without a star. Be aware that at this moment in history, languages designed for Sneetches without stars seem to have much better features for writing starless programs than languages for Sneetches with stars. So my observation is this:If you dislike the verbosity of starry languages like Java but like the feeling of safety, try a type inference language. Don’t go to a starless language if you don’t intend to actually write dynamically typed programs.
My experience is that if you are frustrated by the amount of work you have to do to express the algorithms in your head, you should look at a language that removes your frustration. If you're using Java and don't like the verbosity, find a language that celebrates brevity while preserving static typing. But if you're using Java and find yourself pushing more and more logic out of java because its type system is too static or too inflexible, you should consider a language with a different approach to typing.
Computer languages differ not so much in what they make possible, but in what they make easy.
Larry Wall
Why would the Sneetches without stars use starless languages?
Writing starless programs on top of neat languages is exactly the same thing as writing automatic memory management routines on top of a manually managed programming language or writing functional programs on top of a noun-centric object-oriented language.You can take that statement as an argument in favour of specialized languages for Sneetches without stars or as an argument against them. My guess is that the above statement is true and a Rorschach Inkblot: You will interpret it as confirmation of your existing prejudices.
Wednesday, October 04, 2006
Why are local variables bad?
Steve Yegge wrote a terrific post about refactoring, "Transformation," in March of 2006. My two cent review is that the most valuable part is his discussion of how the Blub community has become so enamored of push-button refactoring at the line of code level that they have lost sight of what refactoring is supposed to accomplish at the design level.
Many Blub programmers won't even try a language that doesn't have so-called refactoring support in their IDE. Given the choice between dynamic metaprogramming and refactoring, they choose refactoring without even knowing exactly what dynamic metaprogramming is.
And they are blissfully unaware that the 'refactorings' in their editors are just a few of the refactorings suggested in Martin Fowler's seminal book on the topic. Furthermore, there is more than one way to accomplish the goal of well-designed software, and some languages, (especially folding languages), provide much more powerful tools for clean software design than micro-refactoring.
All that leads me to a question posed on programming.reddit.com:
Why are local variables considered bad?
I'll simplify things. Mutable local variables are bad. And here's why:
Mutable local variables mean that within a method execution you have changes of state. That makes it very difficult to change anything without breaking things.
For Eclipse fans, consider what happens if you take a random chunk in the middle of a method with mutable local variables and try to use the automatic refactoring to make a new private method.
If you have lots of immutable local variables, they have to be passed to your new method as parameters. That's not so bad. But if you have mutable local variables, you have to do some back flips and sit ups to ensure that any changes to those variables are propagated to the rest of your method.
This is a tip-off that the mutable local variables have introduced a lot of complexity to the structure of the method. Unless they make it dramatically shorter than a variable-free version of the same method, they should be removed.
(Update): What kind of complexity are we talking about? We are talking about lots and lots of dependencies between the lines of code in the method. Those dependencies are what make the code brittle: if you try to move something or change it, you break some other code that is depending on its side effects. The extra dependencies have introduced coupling, which is a code smell.
The book that touched off a revolution in software design. Incredibly, most programmers think that refactoring is just about the dozen or so menu items in their IDE. This book discusses over a hundred refactorings, organized into themes. It goes beyond mere transformations to explain what makes good code and how to recover from bad decisions.The extra coupling manifests itself in many ways, not just in the difficulty of moving code around. You also have problems changing lines of code. This manifests itself in regressions: every time you fix a problem or add some functionality, do you find yourself having to fix a bunch of bugs created by your change?
Let's be clear: we don't say mutable local variables are bad because it's hard to refactor methods with mutable local variables. We say mutable local variables are bad because they intrduce complex dependencies and coupling between the lines of code in a method.
Immutable local variables, such as the local variables you use to cache certain results, are not bad. (I should say, "in my opinion," but this is my blog. Note the words "my" and "blog," establishing that we're talking about me and opinions, not everyone and especially not widely accepted irrefutable facts).
As a matter of fact, I can prove that they are not bad. Okay, "proof" is an ambitious word given that there is no widely accepted proof that 1+1=2. I'll demonstrate. Here's some code with an immutable local variable:
Let's transform that code into:
We've transformed the immutable local variable into a parameter for a lambda (or closure, or anonymous proc object, whichever name you prefer). Parameters are not harmful. And since this transformation can be automatically performed for any immutable local variable, I claim that immutable local variables are also not harmful.
Cool discovery, hunh? I should be famous. Oh wait, someone on the phone from the 1970s. The Lisp people want their let macro back. And they're right. People having been using let for more than thirty years to write concise code without the side effects and state changes associated with mutable local variables. Consider instead:
That's simpler structurally than using one of Ruby's procs. So what do we conclude?
First, refactoring is cool, but it's a tool, not an objective. The important thing is the objective, well-designed software. Second, read the book. Really read it, cover to cover.
Update: This is exactly why languages with more powerful abstractions are more important than adding push-button variable renaming to less powerful languages. Where's the button that refactors a method with lots of mutable variables into one with no mutable variables? No button? Okay, until you invent one, don't you want a language that makes it easy to write methods and functions without mutable local variables? (Attention Java programmers: Yes you do, and I have proof. Do you like the new for syntactic sugar in Java 1.5? Its biggest benefit is that it makes loops that used to have a mutable variable look just like an iterator block with an immutable parameter in Ruby or Smalltalk. Think about it.)
Many Blub programmers won't even try a language that doesn't have so-called refactoring support in their IDE. Given the choice between dynamic metaprogramming and refactoring, they choose refactoring without even knowing exactly what dynamic metaprogramming is.
And they are blissfully unaware that the 'refactorings' in their editors are just a few of the refactorings suggested in Martin Fowler's seminal book on the topic. Furthermore, there is more than one way to accomplish the goal of well-designed software, and some languages, (especially folding languages), provide much more powerful tools for clean software design than micro-refactoring.
All that leads me to a question posed on programming.reddit.com:
Why are local variables considered bad?
I'll simplify things. Mutable local variables are bad. And here's why:
Mutable local variables mean that within a method execution you have changes of state. That makes it very difficult to change anything without breaking things.
Those are my principles, and if you don't like them... well, I have others.
Groucho Marx
For Eclipse fans, consider what happens if you take a random chunk in the middle of a method with mutable local variables and try to use the automatic refactoring to make a new private method.
If you have lots of immutable local variables, they have to be passed to your new method as parameters. That's not so bad. But if you have mutable local variables, you have to do some back flips and sit ups to ensure that any changes to those variables are propagated to the rest of your method.
This is a tip-off that the mutable local variables have introduced a lot of complexity to the structure of the method. Unless they make it dramatically shorter than a variable-free version of the same method, they should be removed.
(Update): What kind of complexity are we talking about? We are talking about lots and lots of dependencies between the lines of code in the method. Those dependencies are what make the code brittle: if you try to move something or change it, you break some other code that is depending on its side effects. The extra dependencies have introduced coupling, which is a code smell.
The book that touched off a revolution in software design. Incredibly, most programmers think that refactoring is just about the dozen or so menu items in their IDE. This book discusses over a hundred refactorings, organized into themes. It goes beyond mere transformations to explain what makes good code and how to recover from bad decisions.
Let's be clear: we don't say mutable local variables are bad because it's hard to refactor methods with mutable local variables. We say mutable local variables are bad because they intrduce complex dependencies and coupling between the lines of code in a method.
Immutable local variables, such as the local variables you use to cache certain results, are not bad. (I should say, "in my opinion," but this is my blog. Note the words "my" and "blog," establishing that we're talking about me and opinions, not everyone and especially not widely accepted irrefutable facts).
As a matter of fact, I can prove that they are not bad. Okay, "proof" is an ambitious word given that there is no widely accepted proof that 1+1=2. I'll demonstrate. Here's some code with an immutable local variable:
my_var = SomeModelClass.find :all
if my_var.empty?
:empty
else
my_var
end
Let's transform that code into:
(lambda do |my_var|
if my_var.empty?
:empty
else
my_var
end).call(SomeModelClass.find :all)
We've transformed the immutable local variable into a parameter for a lambda (or closure, or anonymous proc object, whichever name you prefer). Parameters are not harmful. And since this transformation can be automatically performed for any immutable local variable, I claim that immutable local variables are also not harmful.
Cool discovery, hunh? I should be famous. Oh wait, someone on the phone from the 1970s. The Lisp people want their let macro back. And they're right. People having been using let for more than thirty years to write concise code without the side effects and state changes associated with mutable local variables. Consider instead:
(let ((my_var (find SomeModelClass 'all)))
(if
(empty? my_var)
'empty
my_var))
That's simpler structurally than using one of Ruby's procs. So what do we conclude?
First, refactoring is cool, but it's a tool, not an objective. The important thing is the objective, well-designed software. Second, read the book. Really read it, cover to cover.
Update: This is exactly why languages with more powerful abstractions are more important than adding push-button variable renaming to less powerful languages. Where's the button that refactors a method with lots of mutable variables into one with no mutable variables? No button? Okay, until you invent one, don't you want a language that makes it easy to write methods and functions without mutable local variables? (Attention Java programmers: Yes you do, and I have proof. Do you like the new for syntactic sugar in Java 1.5? Its biggest benefit is that it makes loops that used to have a mutable variable look just like an iterator block with an immutable parameter in Ruby or Smalltalk. Think about it.)
Labels: popular
Sunday, October 01, 2006
Three questions and three answers about Wasabi and Rotten Fish
I have read some questions about my post Wasabi cannot cure rotten fish. Here are the questions and my replies:
Reg, are you really saying "a good team will succeed despite methodological strategy"?
No. I'm saying that a "good" methodology cannot save a bad team. That does not mean that a good team can survive a bad methodology. There are four critical parts of a successful software project, and in my experience you cannot succeed if any of them are missing. People are the first of those, management the last. Methodology is a part of management in my view. In some situations, strong management can impose an informal rather than a formal methodology. But it would be stretching a point to claim that such situations lacked process or had a bad process.
Reg, when you say "the quality of the result is almost entirely driven by the quality of the programmer", are you also saying "the quality of the tool employed is irrelevant to the quality of the result"?
No, although I might have chosen better words for this idea. What I'm saying is that the difference in results between good and bad programmers using the same tool is much, much larger than the difference in results for programmers of equal abilities using good and bad tools. The good tools are wasted on bad programmers, who find ways to write Visual Basic in any language. The bad tools are the source of frustration for good programmers, who waste time and energy "greenspunning" the bad tools into passable tools.
So I am not saying that (good programmer, bad tool) == (good programmer, good tool). I am saying that:
(good programmer, any tool) - (bad programmer, same tool) >
(any programmer, good tool) - (same programmer, bad tool)
So it is to your advantage to combine good programmers with the best tools available to them. It is always possible to cause a project to fail by selecting bad tools. There is still a difference in result between a good programmer with a good tools and a good programmer with a bad tool. Just because it is a much smaller difference than the difference between good and bad programmers doesn't change the fact that there is a difference.
As Paul Graham and Assaf have pointed out, tool selection strongly drives your ability to attract and retain good programmers. So while in theory you may be able to combine good programmers with bad tools, in practice you may have programmers who are experts in buzzword obfuscation and UML refactoring, but they are not competent in actual programming.
Some "bad" tools have such fatal flaws that it is unrealistic to expect even talented teams to overcome the weaknesses in the tool. In that case my expectation is that a good programmer will fail just like a bad programmer, but the good programmer will alert you to the tool's weaknesses much earlier than a bad programmer, perhaps in the exit interview. Good luck with any project where your best people quit because of poor tools, management, or working conditions.
Finally, in some situations management does not restrict themselves to only prescribing a poor tool but compounds the error by proscribing effective use of the poor tool. In plain English, they don't just force their "A" programmers to use a "C" programming language, but they tell them to write software that they think a "C" would be comfortable maintaining.
(Aside: The usual intent is to "make the code easy to read and maintain." But the measure of "easy to read and maintain" is wrongly "would an inexperienced programmer write the code this way." This is not the same thing as "would an inexperienced but motivated programmer be able to study this code, learn from this code, understand this code, and then maintain this code.")
There is a much larger difference between good programmers using good tools and good programmers forced to use bad tools and bad programming practices. There is some debate as to whether my assertion holds for this "worst case scenario" (such organizations often go for broke, treating the Joel Test as a golf score and do things like demand that their programmers wear ties or refrain from using the Internet).
It may be that the formula becomes:
(good programmer, good tool) - (bad programmer, good tool) >
(good programmer, good tool) - (good programmer, bad tool+bad practices) >
(bad programmer, good tool) - (bad programmer, bad tool+bad practices)
Reg, I don't get the connection to the Wasabi DSL!?
The title was actually snarfed from Guy Kawasaki, who wrote this aphorism approximately twenty years ago. It may be traditional Japanese wisdom. Then again, it may not.
Reg, not everyone can hire the top 1%, 5%, 10%, or even 50% of all developers. So isn't your simplistic "hire great developers" suggestion broken?
(Make that four questions and answers)
You know, this kind of question reminds me why I'm prejudiced in favour of working for CEOs that have a background in competitive fields like sales or competitive team sports like Soccer.
In sales, nobody abandons the idea of hiring good salespeople because "not everyone can hire the top salespeople." Imagine being appointed coach of an Ultimate team. Do you abandon the idea of recruiting the most gifted athletes you can find because every team ought to be able to do the same thing and therefore you will have to settle for average players?
No, obviously not. There can only be one top company. There can only be one championship team. To win in a competitive arena, you have to have better people, and better strategy, and you have to out-execute your competitors. Your goal is to be the one company, to be the one team.
Of course your strategy won't work for everyone, it won't work if everybody tries it. By definition, in a competitive (or "zero-sum") game there is no strategy that can help every player win.
That's ok, you are not interested in World Peace, in every company providing nice working conditions and good management for programmers. You are ruthlessly attempting to beat those companies into oblivion. You want them closed, bankrupt. If they have any talented programmers you want to hire those programmers away from them and put them to work building your dream while their former employers mumble platitudes about "methodologies that produce great results from mediocre people".
Consultants can wander around selling utopian visions of methodologies that work for every company and every programmer, no matter how inexperienced or incapable. Language and tool vendors can spend billions convincing everyone that their silver bullet eliminates all of the bugs that cause projects to fail. IDE mavens can tout the code generation and auto-completion bells and whistles that will give every programmer, no matter how mundanely talented, the productivity of a wizard.
These people have a vested interest in maintaining the illusion that software development is a positive sum game where everyone can win, where every project can succeed, where every product will make money.
But if you are working in a start up where you have to compete for talent, compete for funding, and compete for business success, you do not care whether your strategy can work for 100% of the projects in the world. You care whether it works for just one project, yours.
If you are working on an internal project where you compete with the prospect of your work being outsourced, where you compete for funding with other projects, where you compete for executive mind share, you do not care whether your strategy can work for 100% of the projects in the world. You care whether it works for just one project, yours.
Honestly, what matters to me is how I manage my own projects.
post scriptum: Not everyone wants to be the top salesperson, some people play competitive sports for the camaraderie, not to win, and some people go to work for the purpose of collecting nothing more than a pay cheque. If this describes you, you may find a kindred spirit in the "Angry Aussie": In praise of an average career.
Reg, are you really saying "a good team will succeed despite methodological strategy"?
No. I'm saying that a "good" methodology cannot save a bad team. That does not mean that a good team can survive a bad methodology. There are four critical parts of a successful software project, and in my experience you cannot succeed if any of them are missing. People are the first of those, management the last. Methodology is a part of management in my view. In some situations, strong management can impose an informal rather than a formal methodology. But it would be stretching a point to claim that such situations lacked process or had a bad process.
Reg, when you say "the quality of the result is almost entirely driven by the quality of the programmer", are you also saying "the quality of the tool employed is irrelevant to the quality of the result"?
No, although I might have chosen better words for this idea. What I'm saying is that the difference in results between good and bad programmers using the same tool is much, much larger than the difference in results for programmers of equal abilities using good and bad tools. The good tools are wasted on bad programmers, who find ways to write Visual Basic in any language. The bad tools are the source of frustration for good programmers, who waste time and energy "greenspunning" the bad tools into passable tools.
So I am not saying that (good programmer, bad tool) == (good programmer, good tool). I am saying that:
(good programmer, any tool) - (bad programmer, same tool) >
(any programmer, good tool) - (same programmer, bad tool)
So it is to your advantage to combine good programmers with the best tools available to them. It is always possible to cause a project to fail by selecting bad tools. There is still a difference in result between a good programmer with a good tools and a good programmer with a bad tool. Just because it is a much smaller difference than the difference between good and bad programmers doesn't change the fact that there is a difference.
As Paul Graham and Assaf have pointed out, tool selection strongly drives your ability to attract and retain good programmers. So while in theory you may be able to combine good programmers with bad tools, in practice you may have programmers who are experts in buzzword obfuscation and UML refactoring, but they are not competent in actual programming.
Some "bad" tools have such fatal flaws that it is unrealistic to expect even talented teams to overcome the weaknesses in the tool. In that case my expectation is that a good programmer will fail just like a bad programmer, but the good programmer will alert you to the tool's weaknesses much earlier than a bad programmer, perhaps in the exit interview. Good luck with any project where your best people quit because of poor tools, management, or working conditions.
Finally, in some situations management does not restrict themselves to only prescribing a poor tool but compounds the error by proscribing effective use of the poor tool. In plain English, they don't just force their "A" programmers to use a "C" programming language, but they tell them to write software that they think a "C" would be comfortable maintaining.
(Aside: The usual intent is to "make the code easy to read and maintain." But the measure of "easy to read and maintain" is wrongly "would an inexperienced programmer write the code this way." This is not the same thing as "would an inexperienced but motivated programmer be able to study this code, learn from this code, understand this code, and then maintain this code.")
There is a much larger difference between good programmers using good tools and good programmers forced to use bad tools and bad programming practices. There is some debate as to whether my assertion holds for this "worst case scenario" (such organizations often go for broke, treating the Joel Test as a golf score and do things like demand that their programmers wear ties or refrain from using the Internet).
It may be that the formula becomes:
(good programmer, good tool) - (bad programmer, good tool) >
(good programmer, good tool) - (good programmer, bad tool+bad practices) >
(bad programmer, good tool) - (bad programmer, bad tool+bad practices)
Reg, I don't get the connection to the Wasabi DSL!?
The title was actually snarfed from Guy Kawasaki, who wrote this aphorism approximately twenty years ago. It may be traditional Japanese wisdom. Then again, it may not.
Reg, not everyone can hire the top 1%, 5%, 10%, or even 50% of all developers. So isn't your simplistic "hire great developers" suggestion broken?
(Make that four questions and answers)
You know, this kind of question reminds me why I'm prejudiced in favour of working for CEOs that have a background in competitive fields like sales or competitive team sports like Soccer.
In sales, nobody abandons the idea of hiring good salespeople because "not everyone can hire the top salespeople." Imagine being appointed coach of an Ultimate team. Do you abandon the idea of recruiting the most gifted athletes you can find because every team ought to be able to do the same thing and therefore you will have to settle for average players?
No, obviously not. There can only be one top company. There can only be one championship team. To win in a competitive arena, you have to have better people, and better strategy, and you have to out-execute your competitors. Your goal is to be the one company, to be the one team.
Of course your strategy won't work for everyone, it won't work if everybody tries it. By definition, in a competitive (or "zero-sum") game there is no strategy that can help every player win.
That's ok, you are not interested in World Peace, in every company providing nice working conditions and good management for programmers. You are ruthlessly attempting to beat those companies into oblivion. You want them closed, bankrupt. If they have any talented programmers you want to hire those programmers away from them and put them to work building your dream while their former employers mumble platitudes about "methodologies that produce great results from mediocre people".
Consultants can wander around selling utopian visions of methodologies that work for every company and every programmer, no matter how inexperienced or incapable. Language and tool vendors can spend billions convincing everyone that their silver bullet eliminates all of the bugs that cause projects to fail. IDE mavens can tout the code generation and auto-completion bells and whistles that will give every programmer, no matter how mundanely talented, the productivity of a wizard.
These people have a vested interest in maintaining the illusion that software development is a positive sum game where everyone can win, where every project can succeed, where every product will make money.
But if you are working in a start up where you have to compete for talent, compete for funding, and compete for business success, you do not care whether your strategy can work for 100% of the projects in the world. You care whether it works for just one project, yours.
If you are working on an internal project where you compete with the prospect of your work being outsourced, where you compete for funding with other projects, where you compete for executive mind share, you do not care whether your strategy can work for 100% of the projects in the world. You care whether it works for just one project, yours.
Honestly, what matters to me is how I manage my own projects.
post scriptum: Not everyone wants to be the top salesperson, some people play competitive sports for the camaraderie, not to win, and some people go to work for the purpose of collecting nothing more than a pay cheque. If this describes you, you may find a kindred spirit in the "Angry Aussie": In praise of an average career.
Labels: agile
