(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)
Friday, November 30, 2007
The Optimistic View
There is a movement advocating throwing our hands in the air and banning all forms of “advanced” programming idioms in the hope that our code bases will be somehow more accessible. Everyone has horror stories about incompetent and downright dangerous programmers who somehow keep their jobs, and there’s an argument that code bases must cater to them.
But maybe, just maybe, most programmers rise to the occasion when challenged. Maybe instead of catering to the bottom 10th percentile we can be open-minded about what the 50th to 75th percentile are capable of achieving, much less the top quarter.
Maybe Rails and Ocaml and Lisp and Factor and Haskell aren’t really advanced or scary or obtuse. Maybe they are a little unfamiliar at first, but no more unfamiliar than SQL or Regular Expressions or XML or JSON or Javascript. Maybe we should put these tools in people’s hands and let them show us what they can accomplish.
all programmers are equal, but some are more equal than others
There’s always someone out there claiming that closures and expressive typing and meta-programming and all of these other “academic” or “hacker culture” tools are elitist, as if they are restricted to the privileged few that speak cryptically of folds and unfolds being duals in category theory, or who argue whether homoiconicity is a requirement for a self-interpreter to be metacircular.
The title really says it all: Squeak: Learn Programming with Robots. The modern dialect of the original pure-OO language, Smalltalk. Advanced tooling that still causes jaws to drop. And Robots!!!
But maybe that’s a prejudice foisted upon us. Personal computers taught us that anyone can operate a computer. Spreadsheets taught us that anyone can program. Maybe we should open our minds to the possibility that although not everyone does program in this manner today, they are capable of it if we stop telling them that they aren’t.
And when we say “Anybody can program,” we say that without reservation or disclaimer. We don’t mean anybody can program provided they are banned from using certain langauges, or anybody can program provided we hide the details of relational databases behind ORMs, or anybody can program provided they are not required to learn any idioms beyond those authorized by the local High Priest. We mean anybody can program, period, without being lashed to a heavy yoke and driven through the fields under the whip.
Maybe—just maybe—the person advocating the non-mainstream tool isn’t the elitist. Maybe they are the egalitarian, the one who believes everyone can program if given the chance. What if the person saying “No, no, that doesn’t scale to a team of one hundred hum-drum monkeys” is the real elitist? Maybe the elitists are the ones who are convinced that other, lesser programmers cannot be trusted to program.
Maybe all programmers are not created equally. Maybe some over come early disadvantage to make themselves great. Maybe some squander their gift. Maybe it’s just a matter of sweat and toil and dedication. We really don’t know very much about our brains at all.
But what I personally know is this: I have never met someone who desperately wanted to be great but failed to be at least decent. I have never met someone who applies themselves to the obvious means of self-improvement such as learning a new language every year but is still unfit to create a home page in HTML.
Not everyone is motivated to go against the grain, to study subjects that the vendors say you do not need to know, to gain skills that recruiters say are not in demand, to go against the technical leads in their organization who shoot their ideas down.
Why should they be motivated to swim against the current? Why are the incentives in business programming working against people fulfilling their potential? Are we surprised it is a self-fulfilling prophesy?
Maybe we should just stop assuming that any of this is unusual or special or exceptional. Maybe we should just start assuming it’s normal and everyday and part of the professional toolkit and just see what happens.
Maybe we should just try stuff.
We might get a very pleasant surprise.
This post was inspired by Think Big.
I think it’s funny that aspects were invented a while back to shield the masses from the full horror of class level meta-programming, yet with Rails the concepts are becoming commonplace and no one is flinching (yet). It's a measure of progress.
—martoo on programming.reddit.com
There is a movement advocating throwing our hands in the air and banning all forms of “advanced” programming idioms in the hope that our code bases will be somehow more accessible. Everyone has horror stories about incompetent and downright dangerous programmers who somehow keep their jobs, and there’s an argument that code bases must cater to them.
But maybe, just maybe, most programmers rise to the occasion when challenged. Maybe instead of catering to the bottom 10th percentile we can be open-minded about what the 50th to 75th percentile are capable of achieving, much less the top quarter.
Maybe Rails and Ocaml and Lisp and Factor and Haskell aren’t really advanced or scary or obtuse. Maybe they are a little unfamiliar at first, but no more unfamiliar than SQL or Regular Expressions or XML or JSON or Javascript. Maybe we should put these tools in people’s hands and let them show us what they can accomplish.
all programmers are equal, but some are more equal than others
There’s always someone out there claiming that closures and expressive typing and meta-programming and all of these other “academic” or “hacker culture” tools are elitist, as if they are restricted to the privileged few that speak cryptically of folds and unfolds being duals in category theory, or who argue whether homoiconicity is a requirement for a self-interpreter to be metacircular.
The title really says it all: Squeak: Learn Programming with Robots
. The modern dialect of the original pure-OO language, Smalltalk. Advanced tooling that still causes jaws to drop. And Robots!!!And when we say “Anybody can program,” we say that without reservation or disclaimer. We don’t mean anybody can program provided they are banned from using certain langauges, or anybody can program provided we hide the details of relational databases behind ORMs, or anybody can program provided they are not required to learn any idioms beyond those authorized by the local High Priest. We mean anybody can program, period, without being lashed to a heavy yoke and driven through the fields under the whip.
Maybe—just maybe—the person advocating the non-mainstream tool isn’t the elitist. Maybe they are the egalitarian, the one who believes everyone can program if given the chance. What if the person saying “No, no, that doesn’t scale to a team of one hundred hum-drum monkeys” is the real elitist? Maybe the elitists are the ones who are convinced that other, lesser programmers cannot be trusted to program.
I wrote a parser combinator library to support simple parsing and pattern matching tasks, and I used closures to handle constraints. Unfortunately, both ideas were shot down. Here are a few reasons my tech lead gave to me: Closures are just too hard and too complicated; Nobody will understand it. Nobody will understand that functions can be created, passed around, and destroyed like objects.
—name withheld for obvious reasons
Maybe all programmers are not created equally. Maybe some over come early disadvantage to make themselves great. Maybe some squander their gift. Maybe it’s just a matter of sweat and toil and dedication. We really don’t know very much about our brains at all.
But what I personally know is this: I have never met someone who desperately wanted to be great but failed to be at least decent. I have never met someone who applies themselves to the obvious means of self-improvement such as learning a new language every year but is still unfit to create a home page in HTML.
Not everyone is motivated to go against the grain, to study subjects that the vendors say you do not need to know, to gain skills that recruiters say are not in demand, to go against the technical leads in their organization who shoot their ideas down.
Why should they be motivated to swim against the current? Why are the incentives in business programming working against people fulfilling their potential? Are we surprised it is a self-fulfilling prophesy?
Maybe we should just stop assuming that any of this is unusual or special or exceptional. Maybe we should just start assuming it’s normal and everyday and part of the professional toolkit and just see what happens.
Maybe we should just try stuff.
We might get a very pleasant surprise.
This post was inspired by Think Big.
Comments on “The Optimistic View”:
<< Home
I found myself nodding in agreement while reading this.
I am convinced a lot of people can program if they were to put their minds to it. So how can you get them to put their minds to it without making it look like the seemingly unapproachable act of programming?
My perspective changed on this when I started thinking about it as an HCI problem and gave up on the notion that every aspect has to result in something that is "correct". Instead, think of the end product as something that should be correct, but allow for mistakes along the way. Build tools that let you make steps. Those steps may not result in a "correct" program, but maybe the program is another stepping stone towards the goal.
I think the future of interesting programming elements lies in the communication between the compiler/runtime and the programmer and less in what we traditionally see as language constructs.
I am convinced a lot of people can program if they were to put their minds to it. So how can you get them to put their minds to it without making it look like the seemingly unapproachable act of programming?
My perspective changed on this when I started thinking about it as an HCI problem and gave up on the notion that every aspect has to result in something that is "correct". Instead, think of the end product as something that should be correct, but allow for mistakes along the way. Build tools that let you make steps. Those steps may not result in a "correct" program, but maybe the program is another stepping stone towards the goal.
I think the future of interesting programming elements lies in the communication between the compiler/runtime and the programmer and less in what we traditionally see as language constructs.
This is a fantastic post. I will be pointing people at it in online conversations at least until Java 7 arrives. {closures,function types,custom control syntaxes} suck because Joe doesn't understand them, or can do stupid things using them, apparently. This post addresses that beautifully.
Haskell lets me change what 3+4 gives back, but that doesn't mean all Haskell programmers do that, or that Haskell is any worse for that capability.
Haskell lets me change what 3+4 gives back, but that doesn't mean all Haskell programmers do that, or that Haskell is any worse for that capability.
Great article. I agree.
I don't consider myself all that smart, and I didn't understand closures for a long time. It took hacking at Javascript and setTimeout for me to really understand how they worked.
In Ruby, I have no problem calling Proc.new or monkeypatching or lots of other "elitist" things. They're really not that difficult in implementation, they're just hard to explain. I think that's the problem- trying to explain the theory behind closures is much more difficult than teaching the implementation of closures.
Teach theory, and you're branded elitist. Teach implementation, and you're not.
I don't consider myself all that smart, and I didn't understand closures for a long time. It took hacking at Javascript and setTimeout for me to really understand how they worked.
In Ruby, I have no problem calling Proc.new or monkeypatching or lots of other "elitist" things. They're really not that difficult in implementation, they're just hard to explain. I think that's the problem- trying to explain the theory behind closures is much more difficult than teaching the implementation of closures.
Teach theory, and you're branded elitist. Teach implementation, and you're not.
# posted by 
: 10:19 AM
: 10:19 AM
I think most people suck at most things they do - but most really excel at one or two things. But only sometimes, these two things happen to be their main job. We should stop to accept that somebody who works as a programmer sucks at his job. We do not accept this anywhere else, so why do we think its normal for people to be a bad programmer and work as one?
# posted by : 6:36 PM
: 6:36 PM
Thank you for your refreshing optimism.
I have long wanted to lift programmers (and non-programmers) out of the grueling development environments we have now. When I mention meta-programming solutions, and domain specific languages, my friends scoff at the complexity. Scoff not at the complexity itself, but at the complexity for the 'other' programmers. Subconsciously, I started to believe them; thinking my ideas where only for the top 10%.
You have reminded me of my original dreams: To bring programming to the masses, and in forms tailored to how they work.
Thanks again.
I have long wanted to lift programmers (and non-programmers) out of the grueling development environments we have now. When I mention meta-programming solutions, and domain specific languages, my friends scoff at the complexity. Scoff not at the complexity itself, but at the complexity for the 'other' programmers. Subconsciously, I started to believe them; thinking my ideas where only for the top 10%.
You have reminded me of my original dreams: To bring programming to the masses, and in forms tailored to how they work.
Thanks again.
# posted by : 11:18 AM
: 11:18 AM
Such an elitist discussion.
Maybe a big part of the problem is that these "Elite" concepts are not really understood by the so called elite who use them.
You can't teach theory that you don't really understand, you just end up parroting what someone else said.
Maybe you should not be teaching the theory of saw design to someone who just needs a fancy saw. Build him the saw he need and let him carry on with cutting wood.
Maybe a big part of the problem is that these "Elite" concepts are not really understood by the so called elite who use them.
You can't teach theory that you don't really understand, you just end up parroting what someone else said.
Maybe you should not be teaching the theory of saw design to someone who just needs a fancy saw. Build him the saw he need and let him carry on with cutting wood.
# posted by : 12:25 PM
: 12:25 PM
Such an elitist discussion... Maybe a big part of the problem is that these "Elite" concepts are not really understood by the so called elite who use them...
I confess, I am not smart enough to understand what you just wrote. Would you please be kind enough to explain it in simpler terms so that I can get what you are saying?
I confess, I am not smart enough to understand what you just wrote. Would you please be kind enough to explain it in simpler terms so that I can get what you are saying?
I'm not too eager to learn "academic" programming techniques like design by contract and new language features. There are already too many frameworks and libraries that require a lot of time to learn. As a web developer I also need to keep on top of a lot of other technology and even design tools like Photoshop. “Advanced” programming idioms that are hard to explain are often poorly documented and lack practical sample code. For example, you can't find much sample code in Eiffel for many common programming tasks like reading a text file.
I'm not too eager to learn "academic" programming techniques like design by contract and new language features.
Robert, would it be fair to say that for you, “Good Enough” is good enough? That as long as you can find some way to make things work, that’s all you want or need?
It sounds very much like your view is that new idioms, language features, new languages, and new frameworks add work for you without providing any benefit.
Is that the case in your view?
Robert, would it be fair to say that for you, “Good Enough” is good enough? That as long as you can find some way to make things work, that’s all you want or need?
It sounds very much like your view is that new idioms, language features, new languages, and new frameworks add work for you without providing any benefit.
Is that the case in your view?
I think that the issue is not that programmers are unable to learn, its that they are unwilling to learn.
Now there is good unwilling to learn, and bad unwilling to learn. Bad unwilling to learn is the Blub programmer, who sees no value in learning new things.
On the other hand, even for the best of us, intellectual resources are limited. An experienced developer of web apps would be expected to know at least two of [C, C++, Java, C#] well and the other two in passing, SQL, at least two of [Perl, Python, Ruby, bash/sh/csh/ksh] and have some understanding of the others, HTML, JavaScript, CSS, a thorough knowledge with at least one web framework, understand the ORM impedance mismatch, and one application container/webserver.
That sounds like a lot, but we all seem to do it. But now, layer on the business domain knowledge, and eventually our intellectual capacity begins to get scarce. There are plenty of people who realize that learning LISP will make them incrementally better programmers, but that they may have done the calculus, and the the intellectual capacity might be better spent focussed on the problem domain.
Now there is good unwilling to learn, and bad unwilling to learn. Bad unwilling to learn is the Blub programmer, who sees no value in learning new things.
On the other hand, even for the best of us, intellectual resources are limited. An experienced developer of web apps would be expected to know at least two of [C, C++, Java, C#] well and the other two in passing, SQL, at least two of [Perl, Python, Ruby, bash/sh/csh/ksh] and have some understanding of the others, HTML, JavaScript, CSS, a thorough knowledge with at least one web framework, understand the ORM impedance mismatch, and one application container/webserver.
That sounds like a lot, but we all seem to do it. But now, layer on the business domain knowledge, and eventually our intellectual capacity begins to get scarce. There are plenty of people who realize that learning LISP will make them incrementally better programmers, but that they may have done the calculus, and the the intellectual capacity might be better spent focussed on the problem domain.
# posted by : 3:41 PM
: 3:41 PM
Sometimes you have a situation where the other programmers in the group are true masters of real-time hardware oriented code. Or masters of the financial rules of the company's brokerage system. But they might not have seen a parser in a long time, if ever. Not all good programmers come through the CS system.
Putting a sophisticated parser, for example into an environment like that, means that when it comes time to alter, to port, or to fix the parser, they have to hire a consultant they haven't budgeted for.
I've been in plenty of places where somebody who thought they were a hot-shot (and sometimes were) left behind modules that were unreadable or far too complex. Code with closures, continuations, parsers and all that can have that effect. Even if the code is actually quite good.
On other words, code that would be just fine inside of Intel, Microsoft or Google is merely an additional cost inside the Third National Bank or a Hedge Fund or sometimes even NASA.
Shoe factories and steel mills often have their own custom software and coders to keep it all going. What new languages and techniques are you going to drop on them?
And drive-by coders tend to write useless documentation, if any.
Now, if you've got a pure web app that is maintained and updated only by the code-generating organization, or the guts of a compiler which is only distributed in binary. Then go to town with whatever you like.
Putting a sophisticated parser, for example into an environment like that, means that when it comes time to alter, to port, or to fix the parser, they have to hire a consultant they haven't budgeted for.
I've been in plenty of places where somebody who thought they were a hot-shot (and sometimes were) left behind modules that were unreadable or far too complex. Code with closures, continuations, parsers and all that can have that effect. Even if the code is actually quite good.
On other words, code that would be just fine inside of Intel, Microsoft or Google is merely an additional cost inside the Third National Bank or a Hedge Fund or sometimes even NASA.
Shoe factories and steel mills often have their own custom software and coders to keep it all going. What new languages and techniques are you going to drop on them?
And drive-by coders tend to write useless documentation, if any.
Now, if you've got a pure web app that is maintained and updated only by the code-generating organization, or the guts of a compiler which is only distributed in binary. Then go to town with whatever you like.
For example, you can't find much sample code in Eiffel for many common programming tasks like reading a text file.
I think this is the reason why "advanced" programming techniques get a reputation for being rarefied and academic: they come attached to tools that nobody actually uses for anything.
The reason that metaprogramming is catching on now is Ruby. Ruby is useful. It's not a toy. It doesn't live in its own safe little world, like Lisp in its Machine or Emacs Lisp in its all-text universe or Smalltalk in... Smalltalk. Ruby is a social animal, mixing it up with the Unix command line and the Java runtime and the Mac OS in the best tradition of Perl. So you don't have to win an intellectual argument to get people exposed to Ruby. You just have to build something useful and hand it to them. Then they'll probably start poking around inside to figure out how it works.
The secret to teaching someone a complicated thing is to use it in front of them, repeatedly. That's how human language acquisition works - and, despite their complexity, everyone masters at least one human language. If you immerse the students, they will eventually pick up some idioms. Then they will start tinkering with the idioms. Then they will generalize. Soon, lo and behold, they're using closures and writing DSLs - perhaps without even knowing what "closure" means!
If you want to change the programming world, you have to be a doer and not a talker.
I think this is the reason why "advanced" programming techniques get a reputation for being rarefied and academic: they come attached to tools that nobody actually uses for anything.
The reason that metaprogramming is catching on now is Ruby. Ruby is useful. It's not a toy. It doesn't live in its own safe little world, like Lisp in its Machine or Emacs Lisp in its all-text universe or Smalltalk in... Smalltalk. Ruby is a social animal, mixing it up with the Unix command line and the Java runtime and the Mac OS in the best tradition of Perl. So you don't have to win an intellectual argument to get people exposed to Ruby. You just have to build something useful and hand it to them. Then they'll probably start poking around inside to figure out how it works.
The secret to teaching someone a complicated thing is to use it in front of them, repeatedly. That's how human language acquisition works - and, despite their complexity, everyone masters at least one human language. If you immerse the students, they will eventually pick up some idioms. Then they will start tinkering with the idioms. Then they will generalize. Soon, lo and behold, they're using closures and writing DSLs - perhaps without even knowing what "closure" means!
If you want to change the programming world, you have to be a doer and not a talker.
# posted by : 10:47 AM
: 10:47 AM
If you want to change the programming world…
I love everything your wrote here, it is all true up to this point. The people who created and evolved Lisp adn Scheme and Smalltalk were doers as well. The people who are currently nurturing Haskell and Coq are doers.
It’s true that these things are not achieving popularity. But I ask you this: would there be a Ruby if there was no Lisp or no Smalltalk?
Programming languages are no different than a lot of other technologies like Teflon: there is a role for the researcher and another for the popularizer.
I read something: “Your ideas will go further, if you don’t insist on going along with them.”
if you want to go along with your idea, it is better to work on combining existing things into a useful product, as you suggest.
But if you don’t mind others getting the glory, you can take satisfaction in changing the world by a slightly more indirect path.
I love everything your wrote here, it is all true up to this point. The people who created and evolved Lisp adn Scheme and Smalltalk were doers as well. The people who are currently nurturing Haskell and Coq are doers.
It’s true that these things are not achieving popularity. But I ask you this: would there be a Ruby if there was no Lisp or no Smalltalk?
Programming languages are no different than a lot of other technologies like Teflon: there is a role for the researcher and another for the popularizer.
I read something: “Your ideas will go further, if you don’t insist on going along with them.”
if you want to go along with your idea, it is better to work on combining existing things into a useful product, as you suggest.
But if you don’t mind others getting the glory, you can take satisfaction in changing the world by a slightly more indirect path.
<< Home
