raganwald
(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Friday, June 27, 2008
  The unary ampersand in Ruby


A marshmallow toasting dragon slayer from Montréal asked about the unary &.

I like this question, because when I saw it, I realized immediately that although I’ve figured out how to use it to accomplish certain things, writing an answer serves the excellent purpose of forcing myself to learn more.

Let’s start with what I do know, and what everyone can figure out from grepping source code: It has something to do with converting things to and from blocks. If you take nothing else away from this, remember that when you see a unary “&” in Ruby, you are looking at making something into a block, or making a block into something.

Now, blocks in Ruby are not first-class entities. You cannot store them in variables. They are not objects. There is no Block.new expression. The only way to make a block in Ruby syntax is to write one as part of a method call, like this:


[1, 2, 3, 4, 5].map { |x| x * x  }
    => [1, 4, 9, 16, 25]

As we said above, you cannot assign a block to a variable:


square_block = { |x| x * x  }
    => syntax error, unexpected '}', expecting $end

What do we do with these blocks? Well, inside of a method, we can yield to a block. Yielding to a block is something very much like calling a function. The value of an expression yielding to a block is the value of the expression in the block, paramaterized by anything you pass in the yield expression.

Oh heck, an example is much better:


def take_a_guess(range) # expects a block testing a guess
    guess = range.begin + rand(range.end - range.begin)
    if yield(guess)
        "Yay, I guessed correctly"
    else
        "Boo hoo, my guess was wrong"
    end
end

take_a_guess(1..10) { |x| x == 6 }
    => "Boo hoo, my guess was wrong"
take_a_guess(1..10) { |x| x == 6 }
    => "Boo hoo, my guess was wrong"
take_a_guess(1..10) { |x| x == 6 }
    => "Yay, I guessed correctly"

This method plays a guessing game with you: it takes a range (“guess a number from one to ten”) and a block for testing whether the guess was correct. It takes a guess and yield the guess to the block. It then exclaims its joy to the world if it guesses correctly.

Notice that there is nothing in the method signature saying it expects a block. There is no name for the block. You have to look for the yield keyword to figure out what is going on if the programmer doesn’t add a comment.

converting blocks to procs

So… Let’s talk conversions. If you want to do anything besides invoke a block with yield, you really want a Proc, not a block. For example:


class Guesser
    attr_reader :range, :tester
    def initialize(range, &tester)
        @range, @tester = range, tester
    end
    def take_a_guess
        guess = range.begin + rand(range.end - range.begin)
        if tester.call(guess)
            "Yay, I guessed #{guess} correctly"
        else
            "Boo hoo, my guess of #{guess} was wrong"
        end
    end
end

foo = Guesser.new(1..10) { |n| n == 6 }
foo.take_a_guess
    => "Boo hoo, my guess of 2 was wrong"
foo.take_a_guess
    => "Boo hoo, my guess of 8 was wrong"
foo.take_a_guess
    => "Yay, I guessed 6 correctly"

We want to store the tester block as an instance variable. So what we do is add a parameter at the very end with an ampersand, and what Ruby does is take the block and convert it to a Proc, which you can pass around as an object. And when you want to use it, you send it the #call method.

Now if you think about this for a second or two, you’ll realize that almost every Proc you ever create works this way: We pass a block to a method, and the method turns it into a proc by having a parameter with an &. Let’s try writing another one:


def our_proc(&proc)
    proc
end

double = our_proc { |n| n * 2}
double.call(7)
    => 14

Not much to it, is there? When you want a Proc, you can create one by calling a method with a block and using &parameter to convert the block to a Proc. There is no other way to convert a block to a proc because the only place blocks exist is in method invocations.

converting procs to blocks

Okay, we know how to make a Proc out of a block. What about going the other way? What if we want to make a block out of a Proc?

Let’s reopen our Guesser class:


class Guesser
    def three_guesses
        guesses = Array.new(3) { range.begin + rand(range.end - range.begin) }
        if guesses.any?(&tester)
            "Yay, #{guesses.join(', ')} includes a correct guess"
        else
            "Boo hoo, my guesses of #{guesses.join(', ')} were wrong"
        end
    end
end

bar = Guesser.new(1..10) { |x| x == 3 }
bar.three_guesses
    => "Yay, 5, 9, 3 includes a correct guess"

What just happened?

For starters, we made an array with three guesses in it. That line of code includes a block, but let’s ignore that as being irrelevant to this particular discussion. The next part is what we’re after:

We then want to call Enumerable#any? to ask the array if any of its members are the correct guess. Now #any? expects a block. But we don’t have a block, we have a Proc. So now we do the reverse of what we did when we wanted to convert a block to a Proc: instead of a method having an extra parameter with an ampersand, we pass a parameter to the method and apply the ampersand to the parameter we are passing.

So “&tester” says to Ruby: “Take this object and pass it to a method as a block.” The #any? method gets a block, it has no idea we are doing any Proc to block conversion shenanigans. We can prove that:


def did_you_pass_me_a_block?
    if block_given?
        yield
    else
        "NO BLOCK"
    end
end

did_you_pass_me_a_block?
    => "NO BLOCK"
did_you_pass_me_a_block? { 'I passed you a block' }
    => "I passed you a block"
proc = Proc.new { 'I passed you a proc' }
did_you_pass_me_a_block?(&proc)
    => "I passed you a proc"

As you can see, our methods don’t really know whether they get a block or a proc passed as a block. They just yield and all is well. (And yes, you can convert a block to a proc and then the method can convert it right back into another proc.)

to_proc shakur

Which leads us to the final piece of the puzzle. How does Ruby convert whatever you pass with “&” into a block? The answer is that if it is not already a Proc, it tries to convert the object to a Proc by calling the object’s #to_proc method, and from there it converts the Proc into a block.

So you can do fun stuff like convert strings into blocks by defining a method that converts a string to a Proc:


(1..5).map &'*2'
    => [2, 4, 6, 8, 10]

Note that this conversion only happens when you try to pass a string as a block to a method with “&.” It is not correct to say that “&” converts an object to a Proc, the #to_proc method does that. It is correct to say that when passing an object to a method, the “&” unary operator tries to convert the object to a block, using the Object’s #to_proc method if need be.

We’ll close with an example, where we decide that a Guesser can be converted to a Proc:


class Guesser
    def to_proc
        Proc.new { |guess|
            "Yay, I guessed #{guess} correctly" if tester.call(guess)
        }
    end
end

foo = Guesser.new(1..10) { |n| n == 8 }
&foo
    => syntax error, unexpected tAMPER
foo.to_proc
    => #<Proc:0x0008cc08@-:26>
(1..100).map(foo).compact.first
    => ArgumentError: wrong number of arguments (1 for 0)
(1..100).map(&foo).compact.first
    => "Yay, I guessed 8 correctly"

So that’s it: When you want to convert a block to a Proc, define an extra parameter for your method and preface it with an ampersand. When the method is called, the parameter will hold a proc.

When you want to convert any object to a block, pass it to a method expecting a block and preface it with an ampersand. Ruby will call the #to_proc method if need be, then convert the Proc into a block.

because it’s friday

I really hate writing anything too serious on a Friday. So if you’re eyes have glazed over and you’ve marked this to be read later, here’s a little diversion for you. I just picked up The Magic Garden of George B. And Other Logic Puzzles by my favourite nonfiction author, Raymond Smullyan.

Right off the bat, the first paragraphs of the preface introduce a neat puzzle:

Here is a remarkable problem: Imagine a garden of magic flowers that can change color from day to day. On any one day, a flower is either blue the entire day or red the entire day, but can change from one day to another. Given any flower A and any flower B, there is a flower C that is red on all and only those days on which A and B are both blue. Also, we are given that for any distinct flowers A and B, there is at least one day on which A and B are of different colors. Now suppose that the number of flowers is somewhere between 200 and 500. How many flowers are in the garden?Amazingly enough, the problem actually has a unique solution! Doesn’t this surprise you?

Have fun!

 

Comments on “The unary ampersand in Ruby:
Interesting as always, thanks. I guess most ruby coders are like me, and just leave the &/Proc things in a corner of their heads to use only if necessary.

After reading this, I can’t help but ask: why? Why aren’t blocks first-class? Why can’t I do x = {…}?
 
Good summary.
Now if only you'd posted it two days ago, I wouldn't have spent as much time researching it :-/
That said, when I went over it I came to the same conclusion as leoboiko.
If you can do:
proc = &block
eval(&proc)
(just as an example)
Then what's the point in having both procs & blocks? I understand that they fill different use-cases, but if they're so easily interchangeable......?
 
Thanks for questions about procs and blocks.

The truth is, I glossed over something deep when I discussed converting between procs and blocks. My intention is/was to write another post called “Cage Match: Procs vs. Blocks” to discuss how things chaneg when you convert between a proc and a block.

To give you the very simplest explanation, the current context is quite different when yielding to a block or calling a proc.

Matz wanted to make blocks behave like a syntactic abstraction. So blocks execute entirely in the caller's context.

This matters when you think about things like: What happens if you use the return keyword in a block? Do you return a value back to the yielding method or do you return from the method where the block was defined?

With blocks being like syntax, return returns a value from the calling method. Whereas when a block is converted to a proc, return returns a value from the proc.
 
The wording of the problem is quite interesting, but I'm somewhat disappointed with the results. I expected something less trivial, really. Well, maybe the surprising part is that I didn't see it coming at all...
 
I'm a big fan of this exploration of Ruby, closures, and the like.

http://www.innig.net/software/ruby/closures-in-ruby.rb
 
"to_proc shakur"? how very PUNny of you; its clever though. and a nice article too.
 
I don't agree with all of Paul’s editorializing but liked the thoroughness of his exploration.

For example, methods only taking zero or one block is a deliberate design choice, not an accident of history. Paul is thinking ina Schem-ish way, where the "least surprising thing" is the most elegant thing.

Matz designed Ruby with a different design aesthetic, where you optimize for common cases. Allowing multiple blocks would have made the single block case less easy, and eliminating blocks in favour of making everything a proc would have made certain other things—like the return stement—suprising in a different way.

I like Scheme and Smalltalk as well (to name two languages that use one entirely consistent mechanism), but it’s perfectly valid to design a language that is neither Scheme nor Smalltalk. Otherwise, honestly, why do we need a different language?
 
Hmm. I can't figure out the puzzle. It seems to me that the conditions are respected if you have at least one red flower and one blue flower and the sequence of colors rotates each day. For example, for 5 flowers any of these sequences seem to be solutions:

[01] -> rrrrb -> rrrbr -> rrbrr -> rbrrr -> brrrr
[02] -> rrrbb -> rrbbr -> rbbrr -> bbrrr -> brrrb
[03] -> rrbbb -> rbbbr -> bbbrr -> bbrrb -> brrbb
[04] -> rbbbb -> bbbbr -> bbbrb -> bbrbb -> brbbb

What am I missing?
 
Oopsie. "For any distinct flowers A and B, there is at least one day on which A and B are of different colors" -- both must change colors, not just one. Missed that, back to the drawing board :)
 
Ok, "A and B are of different colors" means A != B, not A[day1] != A[day2] && B[day1] != B[day2]. Damn. I'll be hiding in my corner now.
 
@reginald - So I understand the desire to "keep things simple and readable for the most common use case", but the number of ways that ruby deals with functions is really astounding (it makes it /very/ hard for those of us coming from other backgrounds).

We have Procs, Blocks and Methods. All of wich are slightly different and behave in different ways. They are all functional abstractions, just different ones.

Personally I'm waiting for the time when Blocks become syntactic sugar for Procs. But if you're right and Procs yield back to where they were defined and not back to the calling function (which would make Procs not closed in my opionion)...ack...

Right now, blocks act /exactly/ like anonymous functions in javascript, and minus the weird edge case of return like functions in scheme or lisp. Why we would need to clutter the language with a completely new syntax for using functions is beyond me (yeild is just a broken .call).

Anyway
 
I was going to give the puzzle a try..

import List

allHaveC f = True

complete f = all (\(x:xs) -> any (\c -> not $ c == x) xs) (transpose f)

data Color = Red | Blue

generate n d | n == 0 = d
| True = generate (n-1) (Red:d) ? generate (n-1) (Blue:d)

makeDay n [] = generate n []
makeDay n f | generate n [] =:= d &
elem d f =:= False &
allHaveC (d:f) =:= True = d
where d free

makeFlowers n f = if (not (null f)) && (complete f) then f
else makeFlowers n ((makeDay n f):f)

flowers n = makeFlowers n []

goal = [200..500] =:= x++n:xs & (null (flowers n)) =:= False where n, x, xs free


Run here: http://www-ps.informatik.uni-kiel.de/~mh/pakcs/curryinput_c2p.cgi

allHaveC needs to be implemented, and even so it can't run beyond 8, much less 200-500, in the site's timeout period. And I don't have it installed anywhere to let it run for few hours.. or days. :(
 
Am I being too dumb, but the last sentences say "Now suppose that the number of flowers is somewhere between 200 and 500. How many flowers are in the garden?"
They tell us there a re between 200 and 500 flowers in the garden and then ask us how many flowers are there?!? 200 to 500?

Regards
 
if you're right and Procs yield back to where they were defined and not back to the calling function (which would make Procs not closed in my opionion)...ack...

Procs, methods and lambdas return to the context that called them. Blocks return to the context that called the method where they were defined.
 
They tell us there a re between 200 and 500 flowers in the garden and then ask us how many flowers are there?!? 200 to 500?

try being more specific: how many flowers exactly?
 
So I understand the desire to "keep things simple and readable for the most common use case", but the number of ways that ruby deals with functions is really astounding (it makes it /very/ hard for those of us coming from other backgrounds).

My point is neither to endorse nor to condemn Matz’s choice9s) but to point out that there is a design trade-off involved and to suggest that there was conscious thought put into making the decision(s).

I consider The Ruby Way in this area to be a valid design choice. Is it what I personally want? That is a different question. But I don’t think it’s something that just happened.
 
"try being more specific: how many flowers exactly?"

All I can think of here is that the answer is a power of some small base - 2, 3, 4 have unique powers in that range. However, I still have no idea *why* this would be the case... it was the only thing I could come up with that would respect the "unique" constraint.
 
P.S.

allHaveC f = allPairs (\(x, y, z) ->
any (\cs ->
all (\(a:b:c) -> getC a b == last c)
(transpose (x:y:[cs])))
z)
(transpose f)
where
allPairs f l = all f $ findall(\p -> p =:= findPairs l)

findPairs (h++x:xs++y:ys) = (x, y, (h++xs++ys))

getC a b | a == Red || b == Red = Blue
| True = Red


Result for 3: [[Red,Blue,Blue],[Blue,Red,Blue],[Blue,Blue,Red]]

Unfortunately too slow to check higher than 4. Marcel, that power of 3 thing seems like a good bet.
 
Thanks for the great explanation!

> With blocks being like syntax, return returns a value from the calling method. Whereas when a block is converted to a proc, return returns a value from the proc.

Matz is one smart cookie. After reading this article I can't help thinking: I'd either really hate or really love to write a Ruby parser. I guess it wouldn't be as bad as a Perl parser though.

> Right now, blocks act /exactly/ like anonymous functions in javascript

Not true, in JavaScript you can go
var x = function() {
document.writeln("Hello, world!");
};
x();

In Ruby you'd have to stick a lambda in there, which converts it to a proc:
x = lambda { puts "Hello, world!" }
x.call
Think what you meant to say is lambdas are exactly like anonymous functions in JavaScript. I always thought "anonymous function" was a synonym for "lambda", although an anonymous function tends to imply it is in an imperative language.
 
Procs, methods and lambdas return to the context that called them. Blocks return to the context that called the method where they were defined.

Unless Procs are created with Proc.new, in which case they behave like blocks. At least on 1.8.6.

Brennan, i think your solution for 3 is not correct:

Given any flower A and any flower B, there is a flower C that is red on all and only those days on which A and B are both blue.

A and B don't have to be distinct flowers. Therefore we have that for every flower A there is a flower B that has the opposite color of A on every day, where the opposite color of red is blue and vice versa.
Unless I misunderstood you, your solution violates that constraint.
 
Unless Procs are created with Proc.new, in which case they behave like blocks. At least on 1.8.6.

As I said above, there are a number of issues surrounding blocks and procs that I deliberately omitted in order for the post to maintain its focus on the ampersand.
 
IllegalCharacter: I work for MS on IronRuby, and constantly hear from my co-workers how complex Ruby is, to parse and to run, when you consider all of the corner cases.
 




<< Home
Reg Braithwaite


Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

Books
What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

Share
rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

Beauty
IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

Work
The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Management
Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

Notation
The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Opinion
Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

Whimsey
The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

History
06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /