My Mixed Feelings about Ruby

There’s a lot of features of Ruby that I like, but there are some that just drive me nuts like blocks not taking blocks and the ampersand operator. Raganwald did a great job of explaining blocks, procs, and the ampersand in this blog post: Explanation of Ruby’s Unary operator. I came away with the feeling, “Wow! It took that much explanation just to tell how to send blocks around?” If blocks were first-class citizens, Ruby would be more elegant. Raganwald would not be writing huge blog posts on block vs. proc because it would be unneeded.

I feel for Blaine, I really do. When I first looked at Ruby, quite honestly I hated it. Not hated it in a Blubb-dacious “It’s not the same as my popular language” way, but hated it in a “The language is full of Accidental Complexity” way.

To my eyes, Ruby as a language looked a lot like an internal IT app that is built as an aggregation of features. There might be a wonderful, coherent design in the implementation that I can’t see, but the interface I use seems like a bunch on one-off features that don’t play well together.

Ruby definitely isn’t turtles all the way down. And I have never stopped being troubled by this. I lamented the fact that Ruby isn’t written in Ruby. Twice. No, make that three times.

Deep breath. But.

There’s a part of my personality that craves purity and regularity in a system. Purity and regularity have practical benefits, to be sure, but at the same time there are practical benefits to being messy and having special cases optimizing for things people do frequently. This is why design is freakin’ hard. It’s not as easy as following a bunch of rules that produce a system with the smallest possible intellectual surface area, otherwise we’d have stopped with Scheme and its five axiomatic special forms.




A Rube Goldberg contraption does a simple thing (like making ramen) in an extremely complex way. It turns accidental complexity and irregularity into entertainment.
Nor is it as easy as piling more features on regardless of how well they fit or whether people will actually use them. Otherwise Windows would have 97% of the market and OS X 3%. (Oh wait.)

When it comes to design, sometimes you have to experience the result to judge it. You can decide whether a chair is attractive by looking at it, but you really need to sit in it a while to know whether you will feel comfortable using it. Even then, only long experience will tell you whether it is a keeper. Many things that are nice in the showroom are mediocre in day-to-day use.

Design is all about problem solving. What problem do you think programming languages solve as a group? What problem does Lisp solve? Smalltalk? Ruby? For whom?

Matz has said that Ruby is an attempt to solve the problem of making programmers happy. So maybe we aren’t happy with some of the accidental complexity. But can we be happy overall? Can we find a way to program in harmony with Ruby rather than trying to Greenspun it into Lisp?

(I speak as a man working on rewriting code for Ruby: By far the hardest part of this is trying to provide the power of macros in a mechanism that works in harmony with the Ruby Way, rather than bolting Lisp onto Ruby’s side.)

Programming languages also solve the deep problem of helping programmers think about the parts of a program that matter and not clutter their minds up with the parts that don’t matter. This is a very hard problem. Very, very hard. Make things too simple, have too few axioms and abstractions, and you end up with something where each element is extremely simple to understand, but any non-trivial program has too many elements with difficult-to-understand interactions and dependencies so as a whole programs are harder to understand.

Does a regular language help us understand the parts of programs that matter more than an irregular language? That is not an easy question to answer. When I’m struggling with some subtle difference between Proc.new and lambda, I want to shout YES, GODDAMIT, GIVE ME SOME CONSISTENCY.

I’m wary of trying to decide about languages based on infrequent edge cases. With a certain very popular language, I’ve made up my mind that the design choices don’t pay off, that the places where you work hard to express yourself aren’t places where the easy, obvious thing is easy to write and obvious to read. But so far in Ruby, the things that trouble me do seem to have some inherent trade-off merit. I can see how making blocks a special case makes certain eay, simple things easy. So is it overall the best possible design? I don’t know. is it a good design? So far it seems to be a reasonable design.

At my core, I believe axiomatically that there is no one “best” language. That is not an excuse for saying that every language has merit. While I think that there can be more than one good language with different approaches and styles, I do not exclude the possibility that 90% of the programming languages in existence are stinkers through and through.

Is Ruby better for me personally than Lisp or Smalltalk? I don’t know the answer to that question either.

I decided a while back that I would give Ruby an honest try. And while I give myself the freedom to express my misgivings about some of the choices Matz has made, I also try to keep an open mind about them.

It’s really freakin’ (I know, I used this word already) hard for me to to do, but I’m trying to find out if worse might be better. And I don’t mean, “Ruby is worse than Lisp, but it’s better for those n00bs over there.” That’s not embracing change. I mean I am trying to discover if worse might be better for me personally, and it is hard for me to open my mind to that possibility when I’m already invested in “better.”

At this moment in time I have extremely mixed feelings about Ruby. I sorely miss the elegance and purity of languages like Scheme and Smalltalk. But at the same time, I am trying to keep my mind open to some of the ways in which Ruby is a great programming language.  ¶ 10:42 AM 23 comments

  The unary ampersand in Ruby

A marshmallow toasting dragon slayer from Montréal asked about the unary &.

I like this question, because when I saw it, I realized immediately that although I’ve figured out how to use it to accomplish certain things, writing an answer serves the excellent purpose of forcing myself to learn more.

Let’s start with what I do know, and what everyone can figure out from grepping source code: It has something to do with converting things to and from blocks. If you take nothing else away from this, remember that when you see a unary “&” in Ruby, you are looking at making something into a block, or making a block into something.

Now, blocks in Ruby are not first-class entities. You cannot store them in variables. They are not objects. There is no Block.new expression. The only way to make a block in Ruby syntax is to write one as part of a method call, like this:

[1, 2, 3, 4, 5].map { |x| x * x  }
    => [1, 4, 9, 16, 25]

As we said above, you cannot assign a block to a variable:

square_block = { |x| x * x  }
    => syntax error, unexpected '}', expecting $end

What do we do with these blocks? Well, inside of a method, we can yield to a block. Yielding to a block is something very much like calling a function. The value of an expression yielding to a block is the value of the expression in the block, paramaterized by anything you pass in the yield expression.

Oh heck, an example is much better:

def take_a_guess(range) # expects a block testing a guess
    guess = range.begin + rand(range.end - range.begin)
    if yield(guess)
        "Yay, I guessed correctly"
    else
        "Boo hoo, my guess was wrong"
    end
end

take_a_guess(1..10) { |x| x == 6 }
    => "Boo hoo, my guess was wrong"
take_a_guess(1..10) { |x| x == 6 }
    => "Boo hoo, my guess was wrong"
take_a_guess(1..10) { |x| x == 6 }
    => "Yay, I guessed correctly"

This method plays a guessing game with you: it takes a range (“guess a number from one to ten”) and a block for testing whether the guess was correct. It takes a guess and yield the guess to the block. It then exclaims its joy to the world if it guesses correctly.

Notice that there is nothing in the method signature saying it expects a block. There is no name for the block. You have to look for the yield keyword to figure out what is going on if the programmer doesn’t add a comment.

converting blocks to procs

So… Let’s talk conversions. If you want to do anything besides invoke a block with yield, you really want a Proc, not a block. For example:

class Guesser
    attr_reader :range, :tester
    def initialize(range, &tester)
        @range, @tester = range, tester
    end
    def take_a_guess
        guess = range.begin + rand(range.end - range.begin)
        if tester.call(guess)
            "Yay, I guessed #{guess} correctly"
        else
            "Boo hoo, my guess of #{guess} was wrong"
        end
    end
end

foo = Guesser.new(1..10) { |n| n == 6 }
foo.take_a_guess
    => "Boo hoo, my guess of 2 was wrong"
foo.take_a_guess
    => "Boo hoo, my guess of 8 was wrong"
foo.take_a_guess
    => "Yay, I guessed 6 correctly"

We want to store the tester block as an instance variable. So what we do is add a parameter at the very end with an ampersand, and what Ruby does is take the block and convert it to a Proc, which you can pass around as an object. And when you want to use it, you send it the #call method.

Now if you think about this for a second or two, you’ll realize that almost every Proc you ever create works this way: We pass a block to a method, and the method turns it into a proc by having a parameter with an &. Let’s try writing another one:

def our_proc(&proc)
    proc
end

double = our_proc { |n| n * 2}
double.call(7)
    => 14

Not much to it, is there? When you want a Proc, you can create one by calling a method with a block and using &parameter to convert the block to a Proc. There is no other way to convert a block to a proc because the only place blocks exist is in method invocations.

converting procs to blocks

Okay, we know how to make a Proc out of a block. What about going the other way? What if we want to make a block out of a Proc?

Let’s reopen our Guesser class:

class Guesser
    def three_guesses
        guesses = Array.new(3) { range.begin + rand(range.end - range.begin) }
        if guesses.any?(&tester)
            "Yay, #{guesses.join(', ')} includes a correct guess"
        else
            "Boo hoo, my guesses of #{guesses.join(', ')} were wrong"
        end
    end
end

bar = Guesser.new(1..10) { |x| x == 3 }
bar.three_guesses
    => "Yay, 5, 9, 3 includes a correct guess"

What just happened?

For starters, we made an array with three guesses in it. That line of code includes a block, but let’s ignore that as being irrelevant to this particular discussion. The next part is what we’re after:

We then want to call Enumerable#any? to ask the array if any of its members are the correct guess. Now #any? expects a block. But we don’t have a block, we have a Proc. So now we do the reverse of what we did when we wanted to convert a block to a Proc: instead of a method having an extra parameter with an ampersand, we pass a parameter to the method and apply the ampersand to the parameter we are passing.

So “&tester” says to Ruby: “Take this object and pass it to a method as a block.” The #any? method gets a block, it has no idea we are doing any Proc to block conversion shenanigans. We can prove that:

def did_you_pass_me_a_block?
    if block_given?
        yield
    else
        "NO BLOCK"
    end
end

did_you_pass_me_a_block?
    => "NO BLOCK"
did_you_pass_me_a_block? { 'I passed you a block' }
    => "I passed you a block"
proc = Proc.new { 'I passed you a proc' }
did_you_pass_me_a_block?(&proc)
    => "I passed you a proc"

As you can see, our methods don’t really know whether they get a block or a proc passed as a block. They just yield and all is well. (And yes, you can convert a block to a proc and then the method can convert it right back into another proc.)

to_proc shakur

Which leads us to the final piece of the puzzle. How does Ruby convert whatever you pass with “&” into a block? The answer is that if it is not already a Proc, it tries to convert the object to a Proc by calling the object’s #to_proc method, and from there it converts the Proc into a block.

So you can do fun stuff like convert strings into blocks by defining a method that converts a string to a Proc:

(1..5).map &'*2'
    => [2, 4, 6, 8, 10]

Note that this conversion only happens when you try to pass a string as a block to a method with “&.” It is not correct to say that “&” converts an object to a Proc, the #to_proc method does that. It is correct to say that when passing an object to a method, the “&” unary operator tries to convert the object to a block, using the Object’s #to_proc method if need be.

We’ll close with an example, where we decide that a Guesser can be converted to a Proc:

class Guesser
    def to_proc
        Proc.new { |guess|
            "Yay, I guessed #{guess} correctly" if tester.call(guess)
        }
    end
end

foo = Guesser.new(1..10) { |n| n == 8 }
&foo
    => syntax error, unexpected tAMPER
foo.to_proc
    => #<Proc:0x0008cc08@-:26>
(1..100).map(foo).compact.first
    => ArgumentError: wrong number of arguments (1 for 0)
(1..100).map(&foo).compact.first
    => "Yay, I guessed 8 correctly"

So that’s it: When you want to convert a block to a Proc, define an extra parameter for your method and preface it with an ampersand. When the method is called, the parameter will hold a proc.

When you want to convert any object to a block, pass it to a method expecting a block and preface it with an ampersand. Ruby will call the #to_proc method if need be, then convert the Proc into a block.

because it’s friday

I really hate writing anything too serious on a Friday. So if you’re eyes have glazed over and you’ve marked this to be read later, here’s a little diversion for you. I just picked up The Magic Garden of George B. And Other Logic Puzzles by my favourite nonfiction author, Raymond Smullyan.

Right off the bat, the first paragraphs of the preface introduce a neat puzzle:

Here is a remarkable problem: Imagine a garden of magic flowers that can change color from day to day. On any one day, a flower is either blue the entire day or red the entire day, but can change from one day to another. Given any flower A and any flower B, there is a flower C that is red on all and only those days on which A and B are both blue. Also, we are given that for any distinct flowers A and B, there is at least one day on which A and B are of different colors. Now suppose that the number of flowers is somewhere between 200 and 500. How many flowers are in the garden?Amazingly enough, the problem actually has a unique solution! Doesn’t this surprise you?

Have fun!

  ¶ 4:07 PM 22 comments

  A bit of history and an interesting conjecture

We Smalltalkers used to think the advantages of our language were so significant that it would take over the world. We had a huge productivity advantage over C coders. Then C++ came along and gave C coders just enough to let them improve their productivity and their ability to write larger more complex systems. It still wasn’t as good as Smalltalk, but it was better than C, and much more accessible to most programmers than Smalltalk.

C++ eventually sucked up all the oxygen and Smalltalk is now only a language for hobbyists and the occasional programming god. I think this is the most likely threat to the Rails surplus, that C# or Scala or something can do a good enough job that people can double their productivity with far less of a change in mindset or tools, and eventually no one will care about the ten times (or whatever) productivity of Rails.

“Good enough is good enough.”

  ¶ 10:05 PM 3 comments

  Interview questions I have never been asked, Episode I Compare and contrast:

some_array.any? { |n| n > 100 }

And:

!!some_array.detect { |n| n > 100 }

Which do you think is easier to read?

Are classes and modules easier to read and understand when they have lots and lots of specific methods like #any? or are they easier to read and understand with a small number of axiomatic methods that can be composed and combined like #fold and #unfold? When you design a module or class, do you write lots of convenience methods in advance? Or do you refactor code by writing convenience methods when you find yourself repeating the same code?

If you do refactor code to eliminate duplication, is there an amount of duplication that is too small to matter, like "!!"? Or is there an underlying principle of documenting intent that you wish to make explicit?

Is:

do_something() if some_array.detect { |n| n > 100 }

Misleading because it doesn’t actually use the element it detects? Or is it a reasonable idiom to test for an element’s existence without using a specific method like #any or #nil?

Are applications easier to read and understand when they make use of lots and lots of specific methods like #any or are they easier to read and understand when they compose and combine a smaller number of axiomatic methods so that you aren’t constantly looking things up?

Do you think applications should have large or small vocabularies?

This example can be easily translated to the language du jour, the underlying principle applies to programming in general  ¶ 10:45 AM 14 comments

  Macros, Hygiene, and Call By Name in Ruby

Never send a macro to do a function’s job.

Sound advice, however just because functions (or methods) are better than macros for the things they both can do, that doesn’t mean functions can do everything macros can do. Let’s look at andand for a moment. When you write:

foo().andand.bar(blitz())

Using the andand gem, Ruby treats this something like:

temp1 = foo()
temp2 = temp1.andand
temp3 = blitz()
temp2.bar(temp3)

As it happens, if you call nil.andand.bar(blitz()), it will return nil. But it will still evaluate blitz() before returning nil. What I would expect from something named andand is that if foo() is nil, Ruby will never evaluate blitz(). Something like:

temp1 = foo()
if temp1.nil?
    nil
else
    temp2 = blitz()
    temp1.bar(temp2)
end

What we want is that when we pass blitz() to andand, it is not evaluated unless the andand function uses it. The trouble is, you cannot write an andand method in Ruby that delivers these semantics.

Let’s hand wave over the difference between methods and functions for a moment and just look at calling functions. We’ll consider writing “our_and,” a function that emulates the short-circuit evaluation behaviour of Ruby’s “&&” and “and” operators. Ruby (and most other languages in use these days) uses call-by-value when it passes parameters to functions. In other words, when you write:

our_and(foo(), blitz())

Ruby turns that into something that looks like this:

var temp1 = foo()
var temp2 = blitz()
our_and(temp1, temp2)

It doesn’t matter if the function our_and uses blitz() internally or not, it is evaluated before our_and is called and its value is passed to our_and. Whereas our “if” statement in the previous example does not evaluate “blitz()” unless “foo().andand” is not nil.

Well, well, well. The inescapable conclusion is that there are some sequences of expressions in Ruby that cannot be represented as functions or methods. That’s right, functions and methods can’t do everything that Ruby code can do.

Macros and code rewriting can do an awful lot. The implementation of andand in the rewrite gem does rewrite code. When you write:

with(andand) do
    # ...
    foo().andand.bar(blitz())
    # ...
end

Rewrite rewrites your code in place to look something like:

# ...
lambda do |__121414053598468__|
    if __121414053598468__.nil?
        nil
    else
        __121414053598468__.bar(blitz())
    end
end.call(foo)
#...

And for that reason when you write foo().andand.bar(blitz()) using the rewrite gem instead of the andand gem, blitz() is not evaluated if foo() is nil. Big difference!! So it looks like one way to get around call-by-value is to rewrite your Ruby code. Excellent. Or is it?

What’s wrong with rewrite

Right now, the rewrite gem supports writing sexp processors. These are objects that encapsulate a way of transforming sexps. For example, here is the code that transforms expressions like “foo().andand.bar(blitz()):”

  def process_call(exp)
    exp.shift
    receiver_sexp = exp.first
    if matches_andand_invocation(receiver_sexp)
      exp.shift
      mono_parameter = Rewrite.gensym()
      s(:call, 
        s(:iter, 
          s(:fcall, :lambda), 
          s(:dasgn_curr, mono_parameter), 
          s(:if, 
            s(:call, s(:dvar, mono_parameter), :nil?), 
            s(:nil), 
            begin
              s(:call, 
                s(:dvar, mono_parameter), 
                *(exp.map { |inner| process_inner_expr inner })
              )
            ensure
              exp.clear
            end
          )
        ), 
        :call, 
        s(:array, 
          process_inner_expr(receiver_sexp[1])
        )
      )
    else
      begin
        s(:call,
          *(exp.map { |inner| process_inner_expr inner })
        )
      ensure
        exp.clear
      end
    end
  end

And that’s just a third of andand: There is another method that handles expressions like “foo().andand { |x| x.bar(blitz()) }” and a third that handles “foo().andand(&bar_proc).” Brutal.

Now, rewriting code has many other uses. One on my wish list is a rewriter that transforms expressions like: “foo.select { |x| … }.map { |y| … }.inject { |z| … }” into one big inject as an optimization. So I’m not ready to throw rewrite in the trash can just yet. But there’s no way I want to be writing all that out by hand every time I want to implement a function but work around call-by-value semantics.

What about macros?

Why can’t I write:

def_macro our_and(x,y)
    ((temp = x) ? (y) : (temp))
end

…And have it automatically expand my code such that when I write:

# ...
foo = our_and(bar(), blitz())
# ...

The macro expander rewrites it as:

# ...
foo = ((temp = bar() ? blitz() : temp)
# ...

Wouldn’t that work? Maybe. Then again, maybe not.

The problem given above—working around call-by-value—is just one small problem. A macro implementation would solve that problem, but there’s an awful lot of overhead required to make the implementation work, and whatever you do ends up being an incredibly leaky abstraction.

Take our example above. What happens if we have our own variable named temp? Does it get clobbered by expanding our_and? Or do we rename temp? Or do some automagic jigger-pokery with scopes?

Getting macros right is very tricky. I don’t personally plan to try my hand at implementing macros until I’m an expert on the subject of variable capture and can hold forth on the design trade-offs inherent in different schemes for implementing hygienic macros. But that’s just me.

Perhaps there are other ways to solve it without diving into a full-blown macro facility?

Lambdas and blocks

Indeed there are other ways. Ruby already has one-and-a-half of them: blocks and lambdas. Using blocks and lambdas, you can control evaluation precisely. The andand gem actually does support short-circuit semantics using a block. When you write:

nil.andand { |x| x.foo(blitz()) }

It does not evaluate blitz(). This alternate way of using andand supports the semantics we want by explicitly placing the code that should not be eagerly evaluated in a block. Given patience and a taste for squiggly braces, you can create non-standard evaluation without resorting to macros.

We said at the beginning that the reason we cannot use functions and methods to represent everything we can write in code is because Ruby uses call-by-value to pass parameters to functions. One way to work around that is this: instead of passing the value of each expression to a function, we can pass the expression itself, wrapped up in its own lambda.

Then, when the function needs the value, it can call the lambda. This technique has a name: it is called thunking.

We could implement our_and as follows:

our_and = lambda { |x,y|
    if temp = x.call
        y.call
    else
        temp
    end
end

Then when we call it, we could wrap our parameters in lambdas:

our_and.call(
    lambda { a() },
    lambda { b() }
)

Verify for yourself that this produces the behaviour we want, without the worry of our local variables messing things up for the calling code. Let’s go further: we can implement functions with a variable number of arguments using an enumeration of thunks. For example, we could write:

def try_these(*clauses)
    clauses.each { |clause| return clause.call rescue nil }
    nil
end

And call our function like this:

try_these(
    lambda { http_util.fetch(url, :login_as => :anonymous) },
    lambda { http_util.fetch(url, :login_as => ['user', 'password']) },
    lambda { default_value() }
)

We have just implemented the Try.these function from the Prototype Javascript library.

This technique gets us almost all of what we want for this common case of wanting to work around call-by-value semantics. As you can surmise from the fact that it has a name, it is not some newfangled shiny toy idea, it goes back to ALGOL 60, where it was known as call-by-name. (PHP has something called “Call By Name,” but it has a lot more in common with C++ references than it does with ALGOL parameter passing.)

The application of call-by-name as a substitute for full-blown macros isn’t novel either. Joel Klein pointed out that Call by need is a poor man’s macro. Another suggestion along similar lines is to rethink macros in Arc.

thunks: ugly name, ugly code

Our thunking approach solves a lot of our problems, but the implementation severely protrudes into the interface! We could argue that since our call-by-name functions have different behaviour than ordinary functions or methods, they ought to have different syntax.

That’s a reasonable point of view, and that’s exactly how languages like Smalltalk work: everything that involves delaying evaluation in some way uses blocks, even the if statements, which are methods that take blocks as arguments. So in Smalltalk, everything is consistent.

Ruby, OTOH, is not consistent. Operators like “&” and “|” are actually methods with call-by-value semantics, while operators like “&&” and “||” are special forms with call-by-value semantics. Likewise if you only need to delay one expression you can use a block, but if you need to delay two or more, you need at least one lambda. So another reasonable point of view is that we should follow Ruby’s philosophy of making the common case easy to use and not become reductionists trying to build everything out of five axiomatic forms.

So we have one approach—rewriting—that is crazy-hard to write but produces nicely readable code. And we have another approach—thunking—that is easy to write but produces unsightly boilerplate.

Maybe what we want is a rewriter, but we want an easier way to write rewriters for this simple case?

Called by name

Here’s how we could define and use a call-by-name function called “our_and”:

with (
    called_by_name(:our_and) { |x,y|
        if temp = x
            y
        else
            temp
        end
    }
) do
    # ...
    foo = our_and(bar(), blitz()) # method-like syntactic sugar
    # ...
end

What we just did is manufacture a rewriter without any sexps. Instead of getting rid of sexps, we’re treating them like assembler and using a declarative language to write the assembler for us. Our rewriter dutifully rewrites our code to look something like:

our_and = lambda { |x,y|
    if temp = x.call
        y.call
    else
        temp
    end
end
# ...
foo = our_and.call(
    lambda { bar() },
    lambda { blitz() }
)
# ...

We can define a rewriter for functions with splatted parameters too:

with(
    called_by_name(:try_these) { |*clauses|
        clauses.each { |clause| return clause rescue nil }
        nil
    }
) do
    # ...
    try_these(
        http_util.fetch(url, :login_as => :anonymous),
        http_util.fetch(url, :login_as => ['user', 'password']),
        default_value()
    )
    # ...
end

Becomes something like:

try_these = lambda { |*clauses|
    clauses.each { |clause| return clause.call rescue nil }
    nil
}
# ...
try_these.call(
    lambda { http_util.fetch(url, :login_as => :anonymous) },
    lambda { http_util.fetch(url, :login_as => ['user', 'password']) },
    lambda { default_value() }
)
# ...

It goes, boys!

As of now, the rewrite gem supports called_by_name. You can write your own functions with call-by-name semantics using called_by_name just as you see here. As is standard with the rewrite gem, only the code in the do… end block is affected by your change.

call-by-name, in summary

To summarize, with the rewrite gem you can write functions that have call-by-name semantics without wrestling sexps into submission or encumbering your code with a lot of superfluous lambdas and calls:

with(
    called_by_name(:try_these) { |*clauses|
        clauses.each { |clause| return clause rescue nil }
        nil
    },
    called_by_name(:our_and) { |x,y|
        if temp = x
            y
        else
            temp
        end
    }
) do
    # ...
    try_these(
        http_util.fetch(url, :login_as => :anonymous),
        http_util.fetch(url, :login_as => ['user', 'password']),
        default_value()
    )
    # ...
    foo = our_and(bar(), blitz())
    # ...
end

This is a win when you don’t want your code encumbered with more lambdas than business logic. It may be a matter of taste, but part of what I like about Ruby having a special case for blocks is that they act as a huge hint that an expression is temporary: a block after #map suggests we are only using that expression in one place. Whereas when I see “Proc.new” or “lambda,” I expect that the expression will be passed around and used elsewhere.

Functions with call-by-name semantics communicate the same thing as blocks: the expressions are to be consumed by the function. When I see a lambda being passed to a function, I automatically expect it to be saved and possibly used elsewhere. For that reason, I prefer call-by-name semantics when an expression is not meant to be persisted beyond the function invocation.

Now, called_by_name is not a replacement for macros. There are lots of things macros can do that called_by_name cannot do (not to mention that there are lots of things code rewriting can do that macros cannot do). But just as Ruby’s blocks are a deliberate attempt to make a common case for anonymous functions easy to write, called_by_name makes a common case for macros easy to write and safe from variable capture problems.

Of course, called_by_name does so with lots of anonymous functions, and that is a much more expensive implementation than using a hygienic macro to rewrite code inline. But it feels like a move in an interesting direction: if it is a win to sometimes meta-program Ruby’s syntax with DSLs, it ought to also be a win to sometimes meta-program Ruby’s semantics with call-by-name functions.

afterword

So… Is this merely a way to replicate things that are already built into Ruby but do them fifty times slower?

I don’t know how to answer that question. When I heard Matz talk about Ruby at LL1, I didn’t catch the part of his speech where he described how to use metaprogramming to build a really neat web development framework. When you first see a new tool, you naturally start by applying it to problems you already know how to solve with your existing tools in the same way you have always solved such problems.

Only later, after this tool becomes perfectly natural to you, do you start to think of entirely new ways to use the tool. I’m not there yet, but my experience tells me that it’s always a win to have more freedom, to have fewer things you can’s do with a language.

If just one person—maybe it’s me, maybe it’s somebody else—leans forward one day and sees a new way of solving a problem with call-by-name semantics, I’ll consider working on this feature time well spent.

It probably won’t be something trivial like replicating short-circuit boolean operators. But it will be interesting, and I’m looking forward to finding out what it is.  ¶ 9:25 PM 6 comments

  A little something about dumbing code down to the lowest common denominator

When the productive have to ask permission from the unproductive in order to produce, then you may know your culture is doomed.

---

update: Chalain nails it, especially when he writes “… she's right about those two little words: ask permission. And not about dumbing down code.” Although the examples he gives are nothing like the examples that crossed my mind.

I have been happiest when I have worked with people much, much smarter than I am, people who forced me to raise my game. I recall that when I worked on JProbe Threadalyzer, the three key people on my team were Steve Rosenberg, Christian Jaekl, and John MacMillan. Steve was my manager, and he forced me to raise my game as a team lead. Christian was our domain expert, he had written his Masters Thesis on analyzing the behaviour of concurrent software, and he forced me to immerse myself deeply in concurrency just to understand what he was designing. And John was a C++ Wizard. I recall him being extremely patient with me a I got up to speed writing production C++ code.

All three were patient with my shortcomings, but our culture was to help me be more productive, to help me understand their ideas. Not to hold them back so that I could stay in my comfort zone.  ¶ 7:42 AM 13 comments

  Not going dark Rubyforge is now hosting an “initial pre-release of a preview of an alpha of an undocumented proof-of-concept” of the rewrite gem. More on this presently, but the important (and really the only thing of interest) about rewrite is that when you use rewrite to write:

Person.find_by_last_name("Braithwaite").andand.first_name

...You are not opening the Object, Person or Nil classes to add an andand method, nor are you creating a weird temporary BlankSlate object. Rewrite does its thing by rewriting Ruby code: it performs syntactic metaprogramming, much as Lisp macros rewrite Lisp code.

what problem does rewrite solve?

Recall that when you use the “standard” implementation of things like andand or try, you are openly modifying core classes like Object.

Therefore, you are reaching out and touching every line of code in your project. You probably aren’t breaking everything, but even if the chance of introducing a bug by adopting something like “try” is infinitesimal for each source code file in your project, the chance grows greater and greater as your application grows.

The problem is that you are introducing a change on Object, and everything depends on Object. This is very different than introducing a change in your code. In that case, only the other bits of code that directly depend on your code are at risk.

Also, imagine if you introduce try and are careful not to break anything. Now somebody else wakes up one day and decides they need a method that works like Prototype’s Try.these. They call it “try.” They just broke your code, dude! Not only are you making everything dependant upon your version of try, but your code is dependent upon everyone else not breaking try as well. It’s a train-wreck waiting to happen.

Rewrite restricts things like andand or try to your code and your code alone. Sure, if you introduce a bug in your code, you may break things that directly depend on your code. But if you introduce “try” using rewrite instead of modifying Object, you will not reach out across your project and break something entirely unrelated that happens to have defined its own version of try in a completely different way.

how does it work?

Rewrite takes your code, converts it to an sexp with Parse Tree, then rewrites the sexp using one or more rewriters you specify. Finally, it converts the sexp back to Ruby with Ruby2Ruby and evals it. It does this when the code is first read, not every time it is invoked, so we mitigate the “do not use andand in a tight loop” problem.

For example, rewrite converts this:

emails.find_by_email(email).try(:destroy)

Into:

lambda { |receiver, method|
   receiver.send(method) if receiver.respond_to? method
 }.call(emails.find_by_email(email), :destroy)

And this:

 numbers.andand.inject(base_sum()) { |total, number| total + number }

Into:

 lambda { |__1234567890__|
   if __1234567890__.nil?
     nil
   else
     __1234567890__.inject(base_sum()) { |total, number| total + number }
   end
 }.call(numbers)

Note that with the examples, the names “andand” and “try” completely go away. If someone else defines a try method elsewhere, it will not affect your code because your code never executes a method called try.

the most important open problem to solve in this area

At the moment that is a huge PITA when creating new rewriters. I don’t think that it's OK that a language makes it harder for a library creator than for an application developer, so I am working on making it easy to write things like andand or try.

puzzles for language weenies

Note that the andand example above uses gemsym for its parameter while the try example does not. Why? What could break if it used a name like “receiver?” If you can figure out the salient difference between these two example rewrites, you can probably explain why rewriting try produces exactly the same semantics as the original open classes implementation, but rewriting andand produces a subtle change in semantics.  ¶ 11:30 PM 19 comments

  Today's public key is { :n => 46, :e => 5 }

def rsa_encrypt(message, public_key)
  n = public_key[:n]
  e = public_key[:e]
  n_arr = message.downcase.scan(/[a-z]| /).map { |character|
    "abcdefghijklmnopqrstuvwxyz ".index(character)
  }
  n_arr.map { |m|
    (m ** e).modulo n
  }
end

def rsa_decrypt(ciphertext, private_key)
  n = private_key[:n]
  d = private_key[:d]
  n_arr = ciphertext.map { |c| 
    (c ** d).modulo n
  }
  n_arr.map { |n| "abcdefghijklmnopqrstuvwxyz "[n,1] }.join('')
end

And the secret message is: [11, 38, 13, 0, 24, 36, 16, 26, 36, 18, 24, 36, 1, 16, 21, 11, 17, 13, 0, 24]  ¶ 7:15 AM 9 comments

  Bitchiness also sucks

“This framework I never use in a programming language I also never use is designed in a way that I don’t like, so I’m going to raise a ruckus even though I’m never going to use it and there are other frameworks that meet my design standards better” just seems like a dumb thing to write a bitchy blog post about.

Chuck, thank you for saying what needs to be said. Bitchy blog posts purport to describe shortcomings, but are they anything more than thinly veiled chest thumping from an author who feels he is the smartest person in the room?

p.s. Question: Are bitchy blog posts about bitchy blog posts (like this one!) still bitchy blog posts?  ¶ 7:06 AM 6 comments

  Skirting the edge of topicality: Five days left to register for RubyFringe Pete Forde wrote to say that there are only 5 days left to register for RubyFringe, and currently fewer than 20 tickets left.

Coïncidentally, I have submitted my title and abstract:

Ruby.rewrite(Ruby)

An introduction to writing Ruby that reads, writes, and rewrites Ruby. In an extremely short period of time we will extend the Ruby language to include new conditional expressions, add new forms of evaluation such as call-by-name and call-by-need, and if time permits we’ll define new recursive combinators.

In other words we’ll practice the truest form of constructive criticism: Instead of complaining about missing language features, we’ll implement them.

Luckily for you, the other speakers are people with actual talent and a knack for entertaining people. What are you waiting for? Register!  ¶ 12:59 PM 0 comments