(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Sunday, October 15, 2006
  In praise of informed choices

First, a word from our sponsor

You know, this whole “Blub” thing has too many negative connotations. It has become, like “Star-Bellied Sneetch,” a way of labeling people.

You use J, therefore you’re a Blub Programmer.”

There’s no such thing as Blub, therefore I can’t be a Blub Programmer.”

For the remainder of this post, we are going to discuss things in terms of informed and uninformed choices. This is a functional description and it focuses on a one-time behaviour. If you like the old terms, you know where to find grep.

One argument against the idea that there is one overarching continuum of language or tool “power” is that tools specialize. The most appropriate tool for building CRUD database-backed web applications may not be the most appropriate tool for building interactive games.

Success through specialization

Within a particular specialization, there is still a continuum of power. There may be a programmer who insists that Visual Basic backed ASP pages provide all the power she will ever need for CRUD web applications, and there is no need to Ruby, ASP.NET, or any of the other advances on the technology she likes. This sounds remarkably like she is making an uninformed choice, doesn’t it?

The specialization argument is invoked quite often. When I hear it from someone who has taken the time to become comfortable with several different approaches to solving a problem, I give it some credence.

But how often do we hear someone say, “Oh, language X. Yeah, I hear that’s kind of specialized for Artificial Intelligence. And Y is probably terrific if you need to do some startup thing where you need to make a lot of changes as you go. I haven’t tried it, because I don’t do Y-like stuff. But for my purposes Z is the way to go. Each language has its specialization, and Z is best for what I’m doing.”

Are those the words of someone who really understands X or Y and why anyone would need those weird features X and Y programmers boast of? Or what parts of languages X and Y would or wouldn’t apply to the problem at hand? Or is this just a way of sounding Fair and Balanced?

Just because tools may provide specialized benefits depending on the problem at hand, that doesn’t mean that the particular tool a programmer chooses is the best possible or best available. In other words, uninformed choice can be local to a problem domain.

In a fight between a bear and an alligator

My second conjecture about specialized tool hypothesis is that most tools evolve over time away from specialization and towards generalization. Java, the language often mentioned in the same breath as uninformed choice, began its life as a specialized language for embedded programming. Toasters, set-top boxes, that kind of thing. Other languages have seen the same trend towards trying to solve all problems for all people.

The older a language or tool is, the more likely it is a general-purpose tool. The converse tends to be true: the younger a language or tool is, the more likely it is to be specialized. The latter seem to be true even if the designers and sponsors intend it to be a general-purpose tool, because younger tools have smaller ecosystems.

Take a language with a relatively small ecosystem: it may have been designed as a general-purpose tool, but if the the most popular stable, widely used libraries and frameworks support building Ajax-ified CRUD applications, it is a specialized tool as compared to a language that seems to support building everything up to an including interplanetary navigation.

So It seems that some languages boast flexibility, others specialization. It seems unrealistic to expect that the specialized language can beat the generalized language. Unless you are comparing the languages on the basis of the specialized language’s “home turf.” In a fight between a bear and an alligator, terrain determines the outcome.

Now someone argues, “Hey, you wouldn’t try to fight a fire with a Maserati or drive the Autobahn on a Segway.” The specialization argument. Ok. But is that really the comparison? All too often, I hear that argument, but what I see in the toolbox is a mini-van, the king of popularity, the vehicle that does several different things equally poorly.

Honestly. Are programmers clinging to Haskell, Erlang, and Postscript because their needs are specialized and within their niche these languages are the best possible choice? Or are they vocally defending a language that does an awful lot of things in awkward ways against languages that solve their exact problem in an elegant way?

Java is not an uninformed choice, and neither is Visual Basic

Languages and tools are not uninformed. It’s the breadth of experience applied to making a choice that counts. Making a popular, comfortable, established, unfashionable, safe choice does not make you uninformed. An uninformed choice is made in ignorance of the benefits of making other choices.

If you think that your next CRUD application would need anywhere from one half the code to one tenth the code if you used a specialized meta-programming tool like Ruby on Rails, but you’re sticking with ASP because that’s what the client asked for, you’re making an informed choice.

If you write a new application and make heavy use of functional programming constructs for the first time, good for you. If you choose to leave an existing application more or less as-is and you don’t refactor every method to remove mutable local variables, good for you again. You’re making an informed choice.

Some uninformed choices are conservative choices, but so are are some informed choices. You can’t tell the difference merely by examining the language or tool chosen.

Conservative? Or just Uninformed?

I have a little litmus test for identifying an informed choice that just happens to be conservative. I don’t claim it’s infallible, and I have done absolutely no research to back it up. But in the interest of keeping the amusement value up, here is my proposal:

When you see a program written in a conservative, “safe” language, and you want to know whether the programmer is a made a fully informed choice, examine the source code looking for evidence that the programmer has been exposed to other programming paradigms.

Uninformed programmers invariably stick to whatever has been promoted as the sole, orthodox way to write programs for their language. If they need to work around a language limitation, they will do so using well known conventions such as widely disseminated “design patterns.” They can invariable name a dozen or more such patterns, but can never name a single language that handles these problems in a different way.

The informed but conservative programmer has learnt something from other tools, other languages. The principles find their way into her work. A conservative programmer might have extremely pragmatic reasons for doing some work in Javascript. But she uses the Prototype Library to implement higher-order functional programming. Or she might need to build something really big and hairy in C++. But she’ll implement Lisp’s map and reduce as a library so that C++ programs can vectorize on a grid with thousands of computers.

The uninformed programmer may have heard of these things, but not only will they eschew other tools and languages, the eschew the ideas behind them as well.

Although the act of implementing a more powerful language on top of a more conservative language is sometimes frustrating and often derided by purists, it is a sign that the programmer has thought deeply about the best way to solve a particular problem, and is not afraid to port what they like from one environment to another.

What do you think? Do informed but conservative programmers incorporate features and paradigms from other tools and languages into their work?



Comments on “In praise of informed choices:
Catch-22. A Blub programmer never knows they're a Blub programmer.

Obviously, they get offended for no reason at all, so there's half your test.

As for the rest, I'm not sure how accurate the test is.

People will lift features from one language to another because they can't afford to switch.

What you're wondering is why they can't afford to switch: are they being conservative or informed? But either way, they'll pick features from the other language.

Remember JAXP? It's an API invented for the sole purpose of keeping Sun's Crimson parser alive. There's a lot more work on "enabling SOA", that's really designed to keep Java relevant in a world where languages don't matter as much.

Some of those decisions would never be made if people said "hell with the language, what works best for our customers?"

I use my gut instinct. Conservative is not a choice of programming language, it's a point of view. Ask enough questions, and you get a definite answer.

Unfortunately, I think conservative will always be better defined and rationalized than informative. Conservative has perspective. Informative requires taking risks.

If you can't trust your gut, you might want to switch to UML architecture diagrams that generate code from your business domain objects. Or hire a consultant :-)
Do informed but conservative programmers incorporate features and paradigms from other tools and languages into their work?

I sure hope so.

Unless you define conservative to mean "incapable of change," conservative programmers, like other programmers, will seek to do more work with less effort. The only difference between the two groups is that conservatives are more averse to risk. Conservative programmers, then, will need more assurance of a worthwhile return before leaving the beaten path to use a less-accepted tool. But they will use less-accepted tools, just like any other sensible programmer, when it makes sense to do so.

Now, here's the thing. The best tools for solving a given problem are often much more effective than other options. Indeed, the best tools frequently offer enough additional reward, when compared to more-accepted tools, to more than offset any perceived off-the-beaten-path risk, for almost any reasonable degree of risk aversion, including "conservative."

When you find programmers who cling to the beaten path, then, it's probably not because they are conservative. It's more likely that they simply don't know that other paths are worth taking. Had they knowledge of the other paths' rewards, they would almost certainly find the occasional detour worthwhile.
"Programs must be written for people to read, and only incidentally for machines to execute."

Do you want to use large words or words/idioms borrowed from other languages, or do you want to be understood in the local dialect?
Software needs philosophers -- Reg, you've been doing a lot lately to push the discussion this way, towards clarity and fact. Nice job.

As an ex-Java neo-Rubyist living in .Net land, I find the thing I miss the most from Ruby is metaprogramming. I get, but don't grok, functional programming -- I don't think in it (yet).
Programs must be written for people to read, and only incidentally for machines to execute.

Thank you.

This is exactly why I choose tools, languages, and idioms that allow me to express ideas concisely and as close to the semantic meaning as possible.

The machine does not care: it does not mind if I write in Assembler or repeat the same tired Singleton pattern dozens of times or if I annotate every variable with type declarations. Any program will do.

But humans care very much if the actual intent of the code is obscured by the work-arounds and leaky abstractions. Humans care very much if the code is full of needless repetition. Such code is unreadable.

As I've said elsewhere:

In some situations management does not restrict themselves to only prescribing a poor tool but compounds the error by proscribing effective use of the poor tool. In plain English, they don't just force their "A" programmers to use a "C" programming language, but they tell them to write software that they think a "C" would be comfortable maintaining.

The usual intent is to "make the code easy to read and maintain." But the measure of "easy to read and maintain" is wrongly Would an inexperienced programmer write the code this way.

This is not the same thing as Would an inexperienced but motivated programmer be able to study this code, learn from this code, understand this code, and then maintain this code.
I'm not sure the heuristic you proposed holds. I'm a pretty informed programmer; C++ is the language of choice where I work, but I'm familiar to some degree or other with Ruby, Perl, Smalltalk, Lisp, Haskell, etc. Does my experience with these languages inform my use of C++? Sure. But I'm not sure if it would be correct to say that I use e.g. Haskell techniques in C++. I try to use the best practices I can while staying with the grain of the language, and often this leads quite naturally to functional-style code. It would be a mistake, though, to say that the functional style is a foreign one: C++ started moving in the functional direction with the introduction of the STL, and C++ programmers have been moving naturally in that direction ever since, because it works.

Likewise, after reading the academic literature about concurrent languages, and reading the industry literature about patterns such as "Reactor" and "Actor", I've come to the conclusion that both have converged on very similar solutions to the problems of concurrency, just from different angles. Is a C++ programmer who uses event-driven programming patterns to manage concurrency writing in an "Erlang style" in C++? Again, no. She's using proven techniques to solve a hard problem in idiomatic C++.

Often if the problem really seems to be leading in the direction of using a foreign programming style, the real answer is to code that part of the application in that other programming language, rather than trying to shoehorn it's style into another language. For instance, I'm currently coding a test framework in good ole' conservative C++; but I opted to embed a TCL interpreter to handle the parsing of test scripts.

I think the informed programmer realizes that even an individual project may call for more than one language to best handle different aspects of the problem. And regardless of which language they are coding in, they use powerful patterns which transcend individual languages - patterns like event-driven programming, map and reduce, referential transparaency, generic algorithms applied to generic data structures - implemented in ways which flow with the grain of the language at hand.
I think what defines a programmer the most is their individual ability to see the nuances - the grey areas as you discussed in your parachute post. Being able to see the grey areas allows them the ability to make the informed choices or at the very least admit that they don't know and allow them to go and find the information that would let them make an informed choice. Seeing in black and white or right and wrong leads to the overly religious, uninformed wars over which IDE, language, GoF is better, bigger, more usful and why X, Y, and Z don't measure up in any given situation. Or worse yet - seeing in black and white leads to the inevitable I am right and you're an asshole argument.

Excellent Post.
Often if the problem really seems to be leading in the direction of using a foreign programming style, the real answer is to code that part of the application in that other programming language, rather than trying to shoehorn it's style into another language.

This is axiomatically an informed choice.

Of course, someone may Greenspun another style into their language because they don't know that another language could solve the problem for them.

Or they might be making an informed by poor choice. Not all informed choices are good choices.
Conservative? Or just Uninformed?

This makes me ponder a possible converse: "Liberal? Or just Malcontent?" A liberal programmer is somebody who makes informed choices and does the best with what they have, while a Malcontent is merely anti-Blub.

Watch out for catchphrases like: "Our device driver wouldn't have memory leaks if we used a language with a REAL garbage collector", "We need to delete the zip libraries from the webserver because it conflicts with my zip code class and PHP doesn't have namespaces", and my personal favorite, "Why does $LANGUAGE have to be so haaaaaard?!?"
"Do you want to use large words or words/idioms borrowed from other languages, or do you want to be understood in the local dialect?"

It's okay to borrow constructs from other languages if that'll result in better code. In fact not only is it okay, it's the right thing to do.
Are programmers clinging to Haskell, Erlang, and Postscript because their needs are specialized and within their niche these languages are the best possible choice?

Based on the content of the mailing-list and the user conferences - yes, most Erlang programmers do seem to be doing telcom applications, and that is the niche the language was designed for.
It's okay to borrow constructs from other languages if that'll result in better code. In fact not only is it okay, it's the right thing to do.

Seconded. It's important to note, however, that Greenspunning for Greenspunning's sake is NOT the right thing to do.

I don't agree with the Python Blubbers that say that TIMTOWTDI is evil. Python's One Right Way is great right up until somebody discovers a better way to do it. This is happening in some areas of Python (like the networking modules) and it's literally a crisis of faith because the religion built around the language has no mechanism for coping with growth.

Neither do I agree with the Perl Blubbers that think TIMTOWTDI is a virtue all by itself. Having seven different hows isn't a feature if none of them reveal the what.

Would I replace an existing idiom with an alien construct? Probably not. I don't like Python's list comprehensions, but I'll use them rather than trying to write my own collect() method.

It's when the language has no good way of expressing an idea that I reach for my polyglot codebag. For example, as far as I'm concerned, C++ STL is ungainly, unwieldy and bizarre, but I love it because map<> lets me reuse all my dirty perl hash tricks. :-)
Programs must be written for people to read, and only incidentally for machines to execute.

okay -- very nice article until the last line:

"Do informed but conservative programmers incorporate features and paradigms from other tools and languages into their work?"

your test implied -- to me at least -- that looking at a piece of code (pretty much any piece of code) from an informed programmer would yield evidence of their informativeness.

But then your concluding question (quoted above) could return an affirmative answer even if they only did this once in their entire career.

personally i find that when you're programming in blub(X) you're going to cause some maintenance nightmares if you use high-faluting tricks from blub(X+1). not that i'd ever resist...

lb (secretGeek.net)

<< Home
Reg Braithwaite

Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /