raganwald
Monday, July 21, 2008
  "L" is not a code smell
During my presentation at RubyFringe, I shared a question that has been swirling around in my brain for a while: Are IDE features really language smells?

I don’t think it’s an original thought. If nothing else, it’s a corollary to what I believe to be true about many of the GoF design patterns: Many of them are workarounds, ways to Greenspun missing language features. Now, this is probably not the case for all IDE features, and in truth it may be that there are some features which could be implemented in either the language or the IDE, but the IDE may be the best place to put them.

But there is a fairly large class of IDE features that strike me as language workarounds. One of them is definitely the ability to spit out a lot of boilerplate. If you need a lot of code written, you ought to be able to get your programming language to do it for you, not your IDE.

There is room for people to disagree about this. There are some who feel (Strawman alert!) that programs consisting of large numbers of simple elements are easier to understand than programs consisting of a small number of highly abstract elements. Those folks feel an IDE gives you the ease of writing a program quickly plus the ease of reading that same program quickly. They feel that abstractions make the program easier to write but harder to read.

I happen to disagree with this, and if you have been reading this weblog for more than a couple of days you have already read why my experience leads me down a different path. Although in deference to my colleagues with different views, I offer this quote:

All problems can be solved by adding another layer of abstraction, except the problem of having too many layers of abstraction.

Anyone who has dealt with an hammer factory will agree.

So back to “L.”

Two speakers before me, Giles Bowkett gave his excellent Archaeopteryx—um—presentation. I hesitated over that word, because I could just as easily say performance. Performances are terrific entertainment, but they sometimes obscure the message behind them. I want to say outright that while this is true of many other subjects, I felt it worked for Giles because the subject of his presentation was software development as a how rather than a what, and for Giles the “what” is performance.

(Giles got a standing “O,” and many people might be tempted to rush out and make their presentations just as stimulating (400+ in-your-face slides punctuated with loud, driving drum and bass). Be sure that your material matches your presentation style! If not, people may walk away saying “Wow, amazing, but what exactly did she say?” I think it worked for Giles and that’s quite an accomplishment.)

Now really, back to “L.”

Giles is one of the people using closures in Ruby. Meaning, he is passing functions around and storing functions in objects. I am not going to try to say exactly what Archaeopteryx does, so I will describe this style of programming using an imaginary companion program that creates walking bass lines. I will call it Troody.

Let’s simplify things greatly and say that Troody will only ever play in perfect 4-4 time and further that Troody only ever play one of the eight notes in a particular chord’s standard scale. The probability of playing each of those notes on any one “beta” could be represented as an array with eight elements, like this: [.35, .05, .1, .05, .25, .05, .1, .05 ]. You can imagine passing arrays like this around in Troody.

For example, we can pass this array to an object that actually plucks the strings: Plucker.new.start_plucking([ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ]).

Let’s try writing a naïve Troody Plucker:

class Plucker
    def start_plucking(probs)
        while (self.tune.playing)
            if (Metronone.on_the_beat)
                r = rand
                cumulative_probs = probs.inject([]) { |cum, element| 
                    cum + [ cum[-1] && (cum[-1] + element) || element ] 
                }
                notes_to_cumulative_probs = (1..8).zip(cumulative_probs)
                note_to_play = notes_to_cumulative_probs.detect { |note, prob| prob >= r }
                self.pluck(note_to_play)
            end
        end
    end
end

You pass it a set of probabilities, it produces bass notes. But stop, that’s so procedural. Let’s learn from a flying creature, let’s learn from Archaeopteryx. Instead of passing arrays, let’s pass lambdas, like this: lambda { [.35, .05, .1, .05, .25, .05, .1, .05 ] }. Now whenever Troody needs the probability of something, we call the function with .call or Ruby’s [] alternative syntax. So now we write Plucker.new.start_plucking(lambda { [.35, .05, .1, .05, .25, .05, .1, .05 ] })

Our new Plucker code is the same as the old, except we write:

cumulative_probs = probs.call.inject([]) { |cum, element| 
    cum + [ cum[-1] && (cum[-1] + element) || element ] 
}

We now call the probs lambda when we need a note. That’s it, we’ve added a .call call. What does that get us? Well, here’s one thing: If we want the probability to change over time, our function can do that, and we don’t have to rewrite our start_plucking method to handle the idea.

For example, here’s a probability lambda that usually plays the same way but from time to time decides it ought to play pedal notes (refactoring to OO is an optional exercise):

probs = lambda { |bars_of_pedal, beat|
    lambda {
        if beat == 0
            if bars_of_pedal == 0
                bars_of_pedal = 1 if rand < .05
            elsif bars_of_pedal == 5
                if rand < .25
                    bars_of_pedal = 0 
                else
                    bars_of_pedal += 1
                end
            elsif bars_of_pedal == 9
                if rand < .5
                    bars_of_pedal = 0 
                else
                    bars_of_pedal += 1
                end
            elsif bars_of_pedal == 13
                bars_of_pedal = 0
            end
        end
        beat = (beat + 1) % 4
        if bars_of_pedal == 0
            [ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ]
        else
            [ 1.0, 0, 0, 0, 0, 0, 0, 0 ]
        end
    }

}.call(0, 0)

Thanks to the way we’ve separated the probabilities from the plucking, we do not need to subclass Plucker to try a different playing style in Troody.

As Giles pointed out, this is the Strategy Pattern. We are making different kinds of pluckers by encapsulating the logic of what to pluck in something we pass to a plucker. Archaeopteryx appears to do this everywhere. There are lambdas paramaterized by lambdas, lambdas that return lambdas…

This creates a problem. Imagine a programing language where all the keywords are in upper case: IF foo THEN bar ELSE bizzat. Try reading such a program aloud, and you end up shouting the punctuation but speaking the words. This is wrong! We should be shouting the words and whispering the punctuation!

And the problem with Ruby’s lambdas is that if you use a lot of them the word lambda really starts to stand out. So Giles fixed this by aliasing it to L: and using [] instead of .call():

alias :L :lambda

L{ |a| a + a}[5].
    => 10

Much nicer, and as Giles pointed out, this is an example of Ruby’s strength. If you have a program that rarely uses lambdas, you probably want lambdas to stand out when you use them, so you don’t alias lambdas to L and you use the call method, not the square brackets. But if you use lambdas a lot, it’s a win to abbreviate things.

Okay, we’re talking about “L.” Good.

Now in my talk, I said that abbreviating lambda to L was a code smell. I was wrong! Giles, my bad!!

What I actually think is that needing to abbreviate lambda to L is a language smell. Very different. If you show me a Java program and you show me Strategy Pattern, I shouldn’t say it’s a code smell. I should say too bad for you that you need all that boilerplate when Ruby lets you do that with the word lambda and a pair of curly braces.

So now to “L:” Giles, if lambdas are integral to Archaeopteryx, if they are so woven into the fabric of what Archaeopteryx does that you want the keyword “lambda” to fade away, I honestly think this is a place where the language could help you.

For example, what if Ruby had call-by-name semantics? You could write:

Plucker.new.start_plucking([ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ])

# or...

bars_of_pedal, beat = 0, 0
Plucker.new.start_plucking(
    if beat == 0
        if bars_of_pedal == 0
            bars_of_pedal = 1 if rand < .05
        elsif bars_of_pedal == 5
            if rand < .25
                bars_of_pedal = 0 
            else
                bars_of_pedal += 1
            end
        elsif bars_of_pedal == 9
            if rand < .5
                bars_of_pedal = 0 
            else
                bars_of_pedal += 1
            end
        elsif bars_of_pedal == 13
            bars_of_pedal = 0
        end
    end
    beat = (beat + 1) % 4
    if bars_of_pedal == 0
        [ 0.35, 0.05, 0.1, 0.05, 0.25, 0.05, 0.1, 0.05 ]
    else
        [ 1.0, 0, 0, 0, 0, 0, 0, 0 ]
    end
)

And you would get the same behaviour as if you were using lambdas.

That’s it, that’s what I should have said on stage: any time you are working around your language—whether in your IDE, or by modifying open classes, or by abbreviating things—that’s a place where we should step back, where we should ask if our language is missing something.

The answer may very well be “no.” But we ought to at least ask the question.
 

Comments on “"L" is not a code smell:
I think it all comes down to what happens in the future. What happens in eight years, when you've been hired again to come back to this stuff and change it? Will you remember the language? Your specific hacks? The weirdness of the domain? Which IDE version you were dependent on?

What works is what is obvious, even a decade later. We put so much effort into our work, it is a shame if it doesn't stay around long enough to justify itself.

The worst thing that can happen is to depend on some infrastructure tool like an IDE for something critical, like managing the code. If it's not entirely contained in the source code control, then your missing a dependency, aren't you? Opps ...


Paul.
 
Paul:

Oh Lordy, Lordy you are preaching to the choir. A part of me dies when someone is only able to build and run their app in their IDE and can’t replicate production on their development box.

As for making things obvious, that’s the whole debate. Aliasing lambda to L: does it make things less obvious because you have to find the line of code that performs the aliasing?

or does it make the code more obvious by putting the actual logic front and centre?

And of course, the third option is to have well-known language extensions or tools that can make the entire problem go away.
 
It is funny, but for a couple of weeks now I've been flip flopping on this point exactly. I wrote a blog entry, but they bailed in posting it because I wasn't sure I said it well enough.

In the most simplest form, we end up creating a great deal of intrinsic domain specific complexity anyways with our implementations. In that sense, if we leave our custom expressions to be based purely on the underlying language expressions, we are almost doubling up the amount of 'stuff' we need to cope with in order understand what the code does. If on the other hand, the programmers develop a consistent fully encapsulating grammar that mostly hides the underlying language elements, replacing them with their own, an outsider need only understand one simple complete set of expressions.

I find it a lot easier to read C that has heavily customized macros hiding the language ugliness, then I do to read Java that is written in purely primitive arguments. To be very fair, I really really miss the ability in the language to enhance and encapsulate the language itself. Pascal, VB, and Java all have that annoying restrictiveness in common.

Now for the fun part: some programmers make really good use of language extensions, most do not. What I'd like to see is a language where you were free to extend it, but you could also lock it down preventing extensions. In that way, the core underlying developers could set forth a better domain-specific set of primitives, but the bulk of programmers aren't allowed to shoot themselves.


Paul.
http://theprogrammersparadox.blogspot.com
 
Muahahahaha! A retraction! Victory is mine!

OK. I think it's basically true. I'm basically using, in Ruby, JavaScript's model of functions. It isn't patched into the language itself, so it's a convention in the code rather than any actual new syntax. I went with the L{} because I was thinking of what Prototype did with $() - it's actually a very similar problem. There's a piece of syntax you need, you can't alter the language, so you make it as unobtrusive as possible. And as Hampton pointed out, that $() is a smell, not a code smell, not even a language smell, but a (language/context)-mismatch smell.

Sometimes you can smell it, but you're not sure what it is.

Even if I knew C and the Ruby internals very well, I'd probably still go with L{}, on the virtue of its cheapness, but if I were working with a fully-turtles Rubinius, I'd look for a better way. You use $() for the first time, you're like, gawd, what a hack, and then after you've gotten used to it you wonder why it took such a long time for somebody to add it. Same thing, kinda. People who use Perl for a long time and get good at it, the ugliness disappears, replaced with a perception of only the power, but the reality is, the whole win with Ruby is you get power and clarity, so no doubt this is an edge that could be smoothed a bit.

But man, a code smell! I couldn't believe it! I should have shouted and waved my arms and ran up and down the aisles objecting like crazy, but I was completely out of energy. I had loaded up on coffee, timing it so the peak would hit from 12 to 1, and had so much caffeine in my system during the talk that my finger was twitching and my face was red from the heightened blood pressure. After the talk I was zonked.

Also the whole presentation v. performance thing - extremely relevant! Not the first time I performed live drum and bass in front of a group - not even the first time I performed live drum and bass in front of that group that weekend. I basically looked at it as performing a documentary live. I didn't even get to the final two slides, which list contributors, inspirations, and related projects by other programmers in a scrolling top-to-bottom film credits style.

A lot of people who weren't reading my blog last year don't know about my interest in acting, either. If you study acting with real professional actors and structure your programming schedule to allow time to go on auditions, it's safe to say that you'll also be comfortable with public speaking. But that's just the tip of the iceberg. The goal of my acting training was never to become comfortable with public speaking. The goal of my acting training was to get standing ovations.

However the goals of the presentation were to evangelize Archaeopteryx and push everybody away from the 2007 venture capital infatuation back to the 2006 "fight the power" style. Anyway, total threadjack. Thank you for the retraction - I think you're right - it's a smell like the $() in Prototype. A useful smell, I think. But remember the cardinal rule, PDI! Archaeopteryx is on GitHub so there are absolutely zero barriers to entry if anybody would like to hack together an alternate, cleaner approach to closure-centric Ruby. Hint hint!
 
Sorry, I didn't realize I was thanking the Academy quite so much. Also come to think of it the first time I used $() I was like "finally!".
 
> Archaeopteryx is on GitHub so there are absolutely zero barriers to entry

I wish this were true, but last I checked it had a nasty dependency on a bunch of proprietary software. I'd be glad if this were no longer true though.

As for the lambda/L issue, I prefer to just fix it in the rendering layer:

(defun pretty-lambdas ()
(font-lock-add-keywords
nil `(("(?\\(lambda\\>\\)"
(0 (progn (compose-region (match-beginning 1) (match-end 1)
,(make-char 'greek-iso8859-7 107))
nil))))))
 
Phil,

What proprietary software are you referring to? Take another look. I don't think you know what you're talking about.
 




<< Home
Reg Braithwaite


Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

Books
What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

Share
rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

Beauty
IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

Work
The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Management
Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

Notation
The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Opinion
Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

Whimsey
The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

History
06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /