David Heinemeier Hansson step aside, Paul Graham is next to be voted off the island Well, you recall my mentioning how real life is a lot like high school. The latest manifestation of this is an “Arc backlash.” In case you haven’t seen it, Arc is a programming language designed by Paul Graham and (I think) Robert Morris. Actually, it’s a Lisp dialect, so in some respect, you can probably claim that it isn’t a new language at all, just Paul and Robert doing what Lisp programmers have always done.

Now technically, there is not much to Arc at the moment. The language’s own site admits as much:

The current version compiles into MzScheme and structurally is as much a skin on MzScheme as a separate language.

So, what we have here is someone who promotes the language in his essays, then releases a version that fails to transmute lead into gold. And immediately, there was much rejoicing. Seriously, I said rejoicing. There is a distinct tone of schadenfreude to the commentary that is going around. There is much talk of people being “disappointed,” but I have a felling these folks complaining would have had no interest in trying Arc no matter what form it took. I have read the argument that expectations for Arc were high. What does that mean, exactly? That there were people holding off on trying a new language because they were waiting for Arc? Has Paul spread FUD to prevent people from using Scala or Haskell?

Why aren’t people shrugging their shoulders? Honestly, if you don’t have any career or business capital invested in Arc, what could you possibly care about whether it is more or less than what you imagined it might be?

The response to Arc reminds me very much of the alleged “Ruby on Rails backlash.” Quite a bit of the energy behind the RoR backlash is resentment against David Heinemeier Hansson for daring to promote his web development framework. Stepping forward and aggressively attempting to change the world is something we humans claim to admire, but we have complicated rules for how you are supposed to go about it. We want more Steve Wozniak, less Steve Jobs. We want people who blink in surprise at the TV cameras and the limelight of fame, not people who seek it out.

It’s easy to understand why. Although we seem to have complex motivations—some of us after technical mastery, some of us after money, some of us after sexual partners, some of us after land holdings—most of what we do can be traced to standing within our social group. We are social animals, and almost everything we do is connected to the complexities of aligning ourselves with some folks and distancing ourselves from others in our tribes. Although I dislike the show, I would offer Survivor as being a true mirror of human nature. We spend most of our time voting each other off the island.

Live the early stages of Survivor, we play a complex game where we have to think about both our overall welfare and also about our standing within the tribe. The shy, quiet genius who invents things we can use without promoting himself is an asset to us. He helps our tribe in battles against other tribes without jeopardizing our own standing in the tribe. Whereas the self-appointed hero may indeed help our tribe, but in doing so he vaults ahead of us.

That’s why Matz is so well-regarded: he is quiet and unassuming. His behaviour poses no threat to our own place on the ladder. Whereas David is eager to step forward and change the world. His technical contribution to the world of Ruby is at least as valuable as the creation of the language itself, and it is not obvious to me that the world of Ruby would benefit if David hadn’t vigourously promoted Rails. But nevertheless, we are eager to see David “cut down to size,” just as we criticize Steve Jobs while praising Woz.

Which brings me back round to Arc. Paul’s behaviour is now drawing the inevitable response. I personally think it is almost irrelevant how much technical strength the language may have had, there has been a noticeable anti-Paul tone on many programmer forums for quite some time. And what is at the root of it all? Resentment against what some people think is relentless self-promotion.¹ Resentment against his occupying a place above us on the nerd social ladder.

I don’t think there’s anything wrong with feeling that way, if you choose. We are humans, and it is folly to pretend otherwise. Social behaviour matters. But if that’s what it is, let’s “Call a spade a spade” and not pretend that we are disappointed with Arc. Let’s be up front and say that we are angry with Paul for daring to elbow us aside on his climb up the ladder. Let’s admit that we want to be the admired one.

And let’s admit that even if Arc had cured cancer, we would still resent Paul. It’s just that we would have been forced to bide our time, to wait for him to stumble on something else. Just as we are still waiting for Apple’s stock to fall and for Rails to fade into obscurity.

---

1. That sentance originally read “Paul’s relentless self-promotion.” I wrote that as a reflection on what people perceived to be the case, but I should have been more explicit, thus my edit. Paul challenged this. Paul is an essayist, he distributes his ideas and, in a sense, promotes the ideas and not himself. That is very true. It is also true that David promotes Rails and Steve Jobs promotes iPods. The perception of a large number of people—and this article is about those people, not about Paul specifically— is that promoting your ideas is exactly the same thing as promoting yourself. These things are not particularly rational.

  ¶ 5:41 PM 28 comments

  Apple is in the hardware business I admit, I don’t fully understand where Jeff is coming from when he says “When you buy a new Mac, you're buying a giant hardware dongle that allows you to run OS X software.”

Apple is not in the operating system business. Apple is in the hardware business. Steve Jobs famously said that Apple views itself as a software company. And indeed, they do a great job of writing software. But nevertheless, they are in the hardware business: that’s where they get the overwhelming majority of their revenues.

Put in that light, the licensing restrictions on the operating systems that come pre-installed on their hardware are exactly the same as the licensing restrictions on the operating systems that come pre-installed in Windows PCs from manufacturers like Dell and Sony.

It is appropriate to lambaste Apple for selling their product—hardware—with a non-free OS pre-installed. However, they offer the exact same Freedom Zero as the PCs you can buy from Dell.

Arguing that we ought to be able to install OS X on a white box PC is analogous to arguing that we ought to be able to purchase a Sony Vaio, then take the OEM copy of Windows off the laptop and install it on a white box laptop. Or that we ought to be able to purchase an upgrade copy of Windows and install it on a white box PC. I say upgrade copy, because the standalone copies of OS X for sale are licensed for upgrades only, not for new machines.

People who think of the argument as being Apple vs. Microsoft think of the two companies as being in the OS business. But Apple is not in the OS business, and hasn’t been in that business since Jobs returned and quickly killed the OEM Macintosh market. Think of an iPod touch. Would anybody argue that when you buy an iPod you are buying a copy of OS X with a dongle that happens to play music attached to it? Seriously? is anybody complaining that they can’t transfer their copy of OS X from an iPod touch to whatever lame music player Microsoft is peddling this week?

A dongle exists to prevent you from using a piece of software twice without paying for it twice. Apple is hardly even trying to prevent you from using OS X twice without paying for it twice. I can demonstrate this for you: I have a copy of Leopard. There is no copy protection on it. It does not phone home. Why is that? Ah! It is because it is only supposed to run on Apple hardware. Exactly. If Apple lose a few sales of Leopard to Macintosh owners that illegally copy Leopard, that is too bad but it is no big deal to them. Of course, if you hack it to run on non-Apple hardware, you are going to wind up on a first-name basis with Apple’s lawyers. That’s because—are you tired of hearing me say this?—they are in the hardware business, not the operating system business.

In that light, it is appropriate to lambaste them for selling their product—hardware—with a non-free OS pre-installed. However, they offer the exact same Freedom Zero as the PCs you can buy from Dell. You can modify them without incurring Apple’s wrath. You can install another operating system on them, even Windows. It’s true that their hardware is bundled with non-free software, and that’s too bad. But the product they sell—hardware—is free of encumbrances. If you want to compare and contrast them, compare them to another hardware company Dell: In addition to selling hardware with non-free operating systems, Dell also sells hardware with Linux pre-installed.

I could end this with a cheap shot about Microsoft only selling non-free products, but I won’t. I can’t. The point here is that comparing Apple to Microsoft is almost completely meaningless in this context, because Apple is a company in the hardware business that happens develop and bundle an insanely great—but non-free—OS with their hardware.  ¶ 12:08 PM 9 comments

  Lazy Evaluation in Java Thanks for your call. I can pull your interview evaluation right up on my screen. Let me see now, home…, candidates…, search candidates…, click the last name field…, clickity clickety…, search. Sorry, wrong, I put your full name in the last name field, let me try again…, search candidates…, click the last name field…, clickity clickety…, search. Sorry about the wait, it’s too bad finding someone in our application is so clunky. Ok, here’s your evaluation, I’m looking at it right now.

I’m sorry, I don’t have good news abut the job opening here at BigCo. We have elected to pass on bringing you in for another interview.

Yes, I can appreciate that you are disappointed. You’re right, you have the right kind of degree from the right kind of University, and you have excellent grades. But still, we are going to bring some of the other candidates in for another round of talks.

We know that you will never need to understand recursion to do things here. In fact, we will probably fire anyone who writes higher-order functional code in our Java applications.

Why? Well, you seem like a nice person, so I will break our policy and tell you the actual reason. It was because you blew the recursion and data structures code tests. That’s it, really. You passed the database stuff and the loop stuff and the web architecture stuff just fine. As well as anybody might. But the recursion and data structures tests are kind of important. If we didn’t think so, we wouldn’t ask those questions.

Yes, we know that recursion doesn’t come up often in business applications. It’s been a while since I needed it for anything, these days we use XPath and SQL and let libraries do all of the dirty work for us. That’s true. And you’re right, we always use libraries and never write our own data structures. This is BigCo, not some sushi-gobbling start up. But we still ask you these questions, and we still expect you to get them right if you want to work here.

Are you asking why again? I feel like I’m talking to my three year-old here. But you obviously put a lot of work into researching our company before coming in for the interviews, you really stood out from the crowd of new graduates, and nobody else had the gumption to follow up with a phone call, I feel I owe you one.

The reason we expect you to know these things is that they were subjects you claim to have studied when you got your Computer Science degree. Don’t argue with me, I know that Universities are getting a lot more “practical” these days, we like that. I know that you learned your stuff in Java instead of in Scheme or—and this would make you instantly unemployable here at BigCo—in Haskell. But they still teach you about recursion. They still teach you about data structures. You are supposed to graduate knowing how to reverse a linked list, how to find the shortest distance between two nodes in a graph, how to search a tree, and what’s involved in finding cycles in a graph.

I know, if it were up to us they wouldn’t waste time teaching you stuff we don’t want you to use every day. But until we finish making programming a level playing field with the Third World, they are going to keep teaching some of that stuff and you are supposed to learn it. We didn’t ask you anything really tricky, we just asked you to regurgitate stuff you ought to have learned.

Laziness

Harumph. really? You really knew it but forgot it because you didn’t think you’d need it here? Well, let me be blunt: you were lazy. You tried to optimize your time to spend as little as possible on the things that you thought we wouldn’t ask you to know. You tried to maximize your grades by taking the easiest courses possible, so you didn’t get much of that stuff.

You were lazy. Now, laziness is said to be a virtue in a programmer. And if you want to be one of those super-smart programmers who can tell me why the only birds you need are Starlings and Kestrels, you might start a company that we buy for big bucks. But quite honestly, we don’t appreciate laziness in someone we are hiring to toil away in the bowels of our enterprise application code.

Quite honestly, we don’t appreciate laziness in someone we are hiring to toil away in the bowels of our enterprise application code.

Ok, stop, stop, I am not finished. Of course we know that you will almost never need to deeply understand recursion to do things here. And if you need it, you can Google around for some code to cut and paste. In fact, we will probably fire anyone who writes a lot of that higher-order functional code in our Java applications. We know that. But you know what else we know? We know you took shortcuts in school, you did just enough to get through without actually learning anything about the stuff you figured you didn’t need to get a job here.

Which makes us worried about what else you skipped to get here. We can’t test everything, we have to assume that if you skipped that stuff, you might have skipped something else. We have to assume that if you took short cuts to get here, you’ll take short cuts after we hire you. Maybe you’ll decide to push code into production without testing it thoroughly. Maybe you’ll decide to pretend to review someone else’s code but you won’t find an obvious error because you were too lazy to read it carefully.

There you go interrupting me again.

Yes of course we sometimes push code into production without as much testing as we like. But you know what? We want to make those decisions. We don’t want our newly hired, wet-behind-the-ears juniors being lazy and making those decisions for us. We want to hire people that are thorough and meticulous and always do everything they are told by us, and by their professors in University.

The bottom line is this: we appreciate your interest, but there is no room for laziness in our evaluation. We’re looking for people who put their heads down and do what they’re told without trying to game the system. We play the game. You are supposed to be the pawn, and if we let you survive to reach the eighth rank, you might get a chance to play along. But nobody needs a lazy pawn, sorry.

Best of luck with your job search, better luck next time.

---

Update: Finally, someone sided with the hapless and much-maligned candidate

To expect someone to know a skill which is not going to be needed on the job is unfair.

First, remember that an employer is talking to dozens or even hundreds of candidates. This one knows everything they anticipate is needed for the job, that one knows all of those things and more: which one should the employer hire?

Second, forget a Computer Science degree for a moment. You are interviewing a programmer who has just graduated with a degree in something unrelated, say English. You ask him whether he studied Shakespeare. He did. Does he remember any of it? Why no, he was just taking the course to get a degree, he didn’t pay attention beyond passing his examination. Is it unfair to pass over this person because they didn’t actually do any learning in College, just doing whatever they needed to do to pass courses and accumulate credits? I think this is a behaviour that—if repeated on the job— is deleterious.

As an employer, I do not select a candidate’s University. I do not select their courses. I do not choose the coursework and study material. The candidate does all of that. Am I unfair to say to the candidate that whatever they choose to study, they ought to actually learn it?

Putting it in harsher terms, when a candidate puts a degree on their résumé, they are making a claim. Are they claiming that they did just enough to obtain the required marks and no more, or are they claiming to know the material that was taught to them? If it is the latter, then as an employer I am being perfectly fair in testing their claims, whether or not the material has direct relevance to their initial job assignment.  ¶ 6:52 PM 21 comments

  ruby.reddit.com Reddit is now experimenting with user-generated reddits, and one of the new ones is reddit for ruby hackers. For the three people who have never heard of reddit, this is a link aggregation site where you can vote on ruby articles, something like Digg and Dzone.



The Ruby reddit is important, and if you patronize link aggregators like this, I encourage you to please post good Ruby links, vote for the links you like, and perhaps even participate in the comments.

There is already a programming subreddit, but the Ruby reddit is appears to be an entirely distinct “space.” Posts can be in programming.reddit com and in ruby.reddit.com, and there are separate posts and comments. What this means is that Ruby programmers can vote for links and exchange comments without getting into tiresome flame-wars with people who do not care for Ruby.

Of course, you are still free to defend Ruby against Smug Haskell Weenies over in programming.reddit.com. It isn’t an all-or-nothing choice. I look forward to seeing you there!

Labels: ruby

  ¶ 12:39 PM 0 comments

  Trevor Howard on having passion for your profession

Good God, some of the new young actors say they don’t know whether they wanted to be actors or not! I cannot understand this. To me, it is like saying you can’t make up your mind whether or not you love a certain woman. If you don’t then take a walk.

In acting, as in love, there is no place for indifference.

  ¶ 12:31 PM 1 comments

  Object#andand & Object#me in Ruby At Mobile Commons, we have to write a lot of code under time constraints. We also work on a fairly distributed basis, which means that writing readable code is not a luxury, it’s a necessity. We can’t afford to be clever for clever’s sake, but if a new idiom helps make our code easier for our colleagues to grasp in a hurry, we embrace it. Here’re some idioms we’ve recently created:

Object#andand

Ruby programmers are familiar with the two guarded assignment operators &&= and ||=. The typical use for them is when you have a variable that might be nil. For example:

@first_name &&= @first_name.trim
@phone ||= '612-777-9311'

You are trimming the first name provided it isn’t nil, and you are assigning ‘612-777-9311’ to the phone if it is nil (or false, but that isn’t important right now). The other day we were discussing the guards and we agreed that we wished there was a guarded method invocation operator. Here’s an example of when you would use it:

@phone = Location.find(:first, ...elided... )&&.phone

Meaning, search the location table for the first record matching some criteria, and if you find a location, get its phone. If you don’t, get nil. (Groovy provides this exact functionality, although Groovy uses ?. instead of &&.) However, &&. won’t work because &&. is not a real Ruby operator. So what do we write instead?

A feature is “powerful” when at least one of the following holds:

1. It can be used to implement something trivial in an pointlessly complicated way.
2. It can cause a lot of damage.

Seriously, it seems like 85% percents of the contexts where something is called “powerful,” it really means “useless and dangerous.”

How about:

@phone = (loc = Location.find(:first, ...elided... )) && loc.phone

Note that we need a local variable to make it work without extending the Object class. Note further that although we only need the local variable for one line of code, it is in method scope and we must be careful that we don’t clobber an existing loc variable. And when reading this code, we now have to look and see whether loc is used again to understand what this line does.

This is a scoping problem. if you are going to use variables, you want to restrict their scope as much as possible. It feels like half of what we do when programming is manage side effects like overwriting a variable you are using elsewhere, and the more you can restrict those side effects, the easier programming becomes. Java supports block scoping natively, and languages like Scheme solve this by providing a block scope macro, let. And you can roll your own in languages like Javascript that provide anonymous functions. But this seems to get the natural left-to-right order wrong:

@phone = lambda { |loc| loc && loc.phone }.call(Location.find(:first, ...elided... ))

Well, we are tinkerers and toolsmiths. So let’s roll our own. We can’t use &&. without rewriting the Ruby parser, and that seems drastic. So let’s use “andand” in the hope that our non-English-speaking colleagues will forgive us. We want to write:

@phone = Location.find(:first, ...elided... ).andand.phone

And get the same effect as @phone = ->(loc){ loc && loc.phone }.call(Location.find(:first, ...elided... )). This would be bloody trivial if we have a macro facility, but “If wishes were horses then beggars would ride.” Instead, we need to write a method that does some conditional evaluation for us. If the receiver (in this case the result of Location.find(:first, ...elided... )) is truthy, like an ActiveRecord model, we want andand to return it so we can use it. But if it is falsy, we want andand to somehow return something that takes any message you send it and returns itself without doing anything. In other words, we want a mock object of sorts. So that’s what we’ll do:

class Object
    def andand (p = nil)
      self or MockReturningMe.new(self)
    end
end

class MockReturningMe
  instance_methods.reject { |m| m =~ /^__/ }.each { |m| undef_method m }
  def initialize(me)
    @me = me
  end
  def method_missing(*args)
    @me
  end
end

Actually, there is more to it than this, for example the MockReturningMe class is enclosed in a module to minimize namespace pollution. Consult the source and RSpec (links below) for the latest particulars.

Note that because you accept any method using Ruby’s method invocation syntax, you can accept methods with parameters and/or blocks:

list_of_lists.detect { ...elided... }.andand.inject(42) { ...elided ... }

Some of the other solutions to this problem that use send change Ruby’s syntax. This solution emphasizes syntactic regularity: the goal was to make an “&&.” operation that worked like &&=. &&= looks just like normal assignment, you can use any expression on the RHS, only the semantics are different. The andand method also works just like a normal method invocation, only the semantics are modified.

Enhanced Object#tap

(Formerly called Object#me)

Ruby 1.9 includes Object#tap:

Location.find(:first, ...elided... ) do |location|
    puts location.inspect
end
   => location

Ruby On Rails includes a similar feature, the returning idiom:

returning(Location.find(:first, ...elided... )) do |location|
    ...elided...
end
   => location

Both return the location, not whatever happens inside the block. This is very useful, especially when chaining methods in a pipeline. Sometimes you want to do something for side effects, but continue on with the chain. For example:

returning([1, 2, 3, 4, 5]) do |arr|
    arr.pop
end.map { |n| n * 2 }
    => [2, 4, 6, 8]

There are going to be times that using a method instead of a function makes the code more consistent. So let’s imagine what that would look like:

Location.find(:first, ...elided... ).tap do |location|
    ...elided...
end…

This is much nicer when you are “injecting” some extra code into the middle of an expression, such as when adding some poor-man’s-debugging puts statements.

Would we ever want to go beyond a block and write Location.find(:first, ...elided... ).tap.some_method instead of Location.find(:first, ...elided... ).tap do ...elided... end? Of course we would! Quite often, we want to send several methods to the same receiver. In Smalltalk, the semicolon does this explicitly. In Ruby, we have to be very careful to make sure our methods return self to enable chaining. For example:

[1, 2, 3] << 4 << 5
    => [1, 2, 3, 4, 5]

But if a method doesn’t return the receiver, chaining doesn’t work. That’s why we had to use “returning” in the array example above:

[1, 2, 3, 4, 5].pop.map { |n| n * 2 }
    => NoMethodError: undefined method `map' for 5:Fixnum

That’s because [1, 2, 3, 4, 5].pop => 5, not [1, 2, 3, 4]. So instead, we write:

[1, 2, 3, 4, 5].tap.pop.map { |n| n * 2 }
    => [2, 4, 6, 8]     # or we write...
[1, 2, 3, 4, 5].tap { |arr| arr.pop }.map { |n| n * 2 }
    => [2, 4, 6, 8]     # both work

When you examine the source, you’ll see that tap looks a lot like andand. No surprise, we are doing similar things with slightly different semantics.

This implementation of Object#tap has two advantages over the implementation in Ruby 1.9: first, it works in Ruby 1.8. Second, it adds the ability to call another method (like pop) and not have to include a block.

Summary: andand & enhanced tap

andand and tap both provide the same two benefits: First, they either eliminate (in the case of a method call) or limit in scope (in the case of a block) a local variable. Second, they let you maintain a natural left-to-right order when writing pipelined expressions. They are simple and obvious enough that I feel any unfamiliarity for the reader will be more than outweighed by the cleaner, easier-to-digest code that results from their liberal use.

My suggestion is that these two benefits make it worth your while to add these methods to the Object class and use them regularly in your code.

Installation

sudo gem install andand. For the source and more information, http://andand.rubyforge.org.

---

Similar solutions:

1. Kernel#ergo in Ruby Facets
2. send_with_default
3. maybe
4. _?
5. if_not_nil (via François Beausoleil)
6. Groovy’s Safe Navigation Operators via Call by Name: "Yo Elvis"
7. try() and A better try

---

postscript: Inline Rescues

I have now seen two people asking about using an inline rescue instead of andand:

Location.find( …elided… ).phone rescue nil

Also, one of the alternate solutions seems to use this technique. Obviously, it works for many of the cases you want to try, and you don’t need to add a new method to Object. However, it is—to paraphrase Chalain—“Sneaking up on the Interpreter.”

It achieves what we want almost by accident, namely that it rescues NoMethodError, which by coïncidence gives us the result we want. However, it does not communicate our intent. It says that we want to transmute all exceptions into a value of nil. When someone glances at the code, will they think of NoMethodError? Or will they assume that we are trying to handle exceptions that the .phone method might throw?

And what if .phone actually throws something serious? In Rails, what happens if a migration is screwed up and there is no phone column? I think we actually want a NoMethodError in that case. The rescue clause will accidentally swallow the exceptions we are trying to catch.

I think if you are absolutely certain that you do not care about other exceptions and also if your team uses this idiom extensively so that it does not miscommunicate its intent, you can live a long and happy life using it. However, I would be hesitant to recommend it as a general-purpose solution. andand and some of the other solutions obviously require the code reader to learn a new method name, but after that they offer the exact functionality we need and clearly communicate their intent.

Labels: ruby

  ¶ 9:09 PM 25 comments

  It's fundamental: You are a programmer if you... Write code that works.

That’s all, you write working code. And it doesn’t matter what degree you have, what courses you took, or whose blogs you read. It’s all about the code.

Joel, I hear what you’re saying about knowing what a pointer is and understanding the byzantine Windows API. But still, if someone write a program, I say he’s a programmer.

Steve, you’re preaching to the choir. Yeah boy, programming without understanding how programming languages really work is like painting with the brush held between the toes of one foot. But I think it’s still programming.

And you over there, the one who just wants a job as a programmer, the one who wants to do whatever everybody is doing and wants to learn as little as possible to get things done. You’re a programmer, don’t let any Chablis-swilling pretentious art-house blogger tell you otherwise between their rants about why we should still be using Lisp or Smalltalk.

All these other things, be they type theory, agile practices, compilers, pointers, DSLs, or anything else… These are all tools. Ignorance of those tools is not helpful. I think you will be even better with more knowledge. I think you will be happier if you learn how to “cut with the grain” and work with the principles of software development instead of against them. But even if you decide you can’t be bothered with anything hard, the important thing is that you write programs, therefore you are a programmer.

I don’t want to wake up one day and find there’s a required reading course for being a programmer (even if my blog is on the list). I don’t want anyone deciding who is—and who isn’t—fit to be a programmer. I want everyone to be free to program. The price of that freedom is that yes, some people are going to do things I don’t like. But you can’t build a world with room for Ruby unless there’s room for PHP. You cannot build a world with room for Scala unless you build a world with room for Java.

Now, does that mean that when Steve Yegge talks about why you ought to know how compilers work, everyone ought to shout him down and say that compiler architecture and/or programming language implementation is bullshit? No. I will be the first to say that compilers and language implementation are important. I will rant.

I support your individual choice of what to learn and what to skip as a programmer. Agree or disagree with me, I say you’re a programmer and I measure how good you are by what you do with the knowledge you decide to acquire. But I still believe that there are such things as fundamentals. There are things that every programmer would do well to know. There are things that would make everybody’s lives easier if every programmer knew them.

Fundamentals matter, because although you are a programmer without them, you could be a much better programmer with them

I do say that you can program without the fundamentals, but I will never say that they are irrelevant, that we have passed the day when fundamentals matter. They do matter, because although you are a programmer without them, you could be a much better programmer with them.

But you will still be a programmer.

---

Elsewhere, I see some folks are arguing whether writing HTML is programming or not. To questions like that, I have but one response: give me an objective definition of what is and what isn’t a program, and I’ll give you an objective definition of what is and isn’t programming.  ¶ 2:57 PM 28 comments

  Are you an Engineer?

If you don’t use mathematics in your day to day work, you aren’t an engineer. All engineers (say those who build bridges, or space craft, or cars) make heavy use of mathematics and/or hard sciences like Physics on a regular basis…

The title “Software Engineer” is (most of the time) a particularly deceptive one. To be accurate it should [be] something like “Software Maintenance Worker” or “Software Handyman,” but I guess it is easier to hire someone if his job title is “Rodent Officer” instead of "Ratcatcher.”

Provocative.

I’m not sure that is an unassailable definition of Engineering, however it seems close enough that quibbling over the definitions of modeling and mathematics wouldn’t add a lot of value. The discussion of appropriate job titles for enterprise software development didn’t really resonate with me (although I quoted them for shock value). Instead, I was moved by the question of which parts of software development are Engineering? Which parts are Science?

In my own career, I have actually been most interested in Industrial Design. Programming Languages have scope for Engineering: look at formal languages like Haskell and Coq for examples of languages designed by people making day to day use of Science. Programming languages also have huge scope for Design: designing a notation for programs is an exercise in applied psychology, an exercise in influencing the thoughts and behaviour of the programmer.

A recent article accused the MapReduce algorithm of lacking novelty, of using forty year-old ideas. For a Scientist, this is a problem: Scientists wake up in the morning trying to move the world forward. But for a software developer, using a forty year-old idea ought to be the normal case. Is it Engineering? Is it Science? Maybe it is Damn Useful, and maybe the non-Engineer, non-Scientist measures themselves against that yardstick.

I don’t have a tidy conclusion for you: just a suggestion that you read the linked post and ask yourself what you really want to do with your talent and your life, be it Engineering, Science, Design, or making software that enriches people’s lives.

Then go out and do it.

Update: Are you a Doctor?

To come up with a proper treatment, doctors need to have an exact diagnosis. To have an exact diagnosis, doctors need to know exactly what is going on in the body. And the only way they can do that is to have a total and complete understanding of how the body works. They can’t just skimp out and decide “Oh, the circulatory system’s not that important.” They can’t just say “Heart trouble? Here, have some aspirin.” Sure, the aspirin might stop the pain temporarily, but in the long run, the patient will suffer even more, and probably die.

  ¶ 9:25 AM 28 comments

  Ruby is soooooo 2002 I first heard about Ruby when Matz spoke at LL2 in November, 2002. I experimented with it on and off over the next few years, and I liked it. I also experimented with C#, Groovy, and a few others, but Ruby reminded me of Smalltalk, a language I used in the 80s and loved, so I stuck with it even though I seemed to be one of three people using Ruby without using Rails.

Lately, the internet echo chamber is souring on Ruby and moving from embracing it to hating it. I suspect that although people are quoting all sorts of technical reasons for their dislike, it comes down to cultural forces. Shrug.

I am reminded of the time I showed up in high school one September on a skateboard. To give you an idea of where skateboards were in those days, my model had clay wheels designed for wood-rink roller-skates, not the polyurethane we have today. Any ways, everyone laughed at me. The next Summer, everyone had one, and they were laughing at the fact that I didn't ride the latest model, even though I was a more accomplished free-styler and slalom rider. Then, the fad passed, and I was the iconoclast again, still riding my board while everyone embraced mopeds.

Guess what? There is no villain in the story. I was happy riding with my friends. The people who jumped on and off fads were happy, and they were learning important life skills about participating in human culture. And the fad was a big win: Early boards really weren’t very good, and the popularity brought money and manufacturing scale to skateboarding that improved the technology.

Paul Graham has said that the real world is nothing like high school. I’m not so sure.

---

I often use Ruby for code examples. What do you think: Should I continue using Ruby, or should I use another language for examples? If I continue using Ruby, should I try to show idiomatic Ruby or should I try to focus on general ideas that can be applied elsewhere?

Code examples should be:
	Ruby-specific
	Anything but Ruby
	Ruby, but general
	A mix of specific and general ideas
pollcode.com free polls

Labels: ruby

  ¶ 11:12 AM 18 comments

  No Disrespect

It’s like if you walked into a painting class, told everyone that learning to paint was too hard, and then gave everyone a camera and told them that photography was the same thing as painting, only a lot easier to do. No disrespect towards photographers intended.

I hear an argument that business programming isn’t hard. It does not require understanding combinatorial logic and fixed points. There is no value in knowing how to program in OCaml (or why that might be a good idea). Business programming is practical, and business programmers need practical training to do a practical job.

Ok, I’ll bite. Really. As Keith would say, I’ll climb down from my former position on this. Fine. So here’s my question to you, Mr. “An undergraduate Computer Science program ought to be as easy as business programming:”

Why don’t businesses advertise for vocational college graduates?

You are describing a vocational job to me. The rote application of practical principles is nothing more and nothing less. How is what you’re describing any different than a job as an accounts receivable clerk or a dental technician? Or a land surveyor? Or a architectural draftsperson?

If your comp-sci course is easy then you’re either a genius or you’re wasting your money. The odds on that are not in your favour.

These are all vocational jobs with a well-established and perfectly functional system for training clerks, technicians, surveyors, and draftspersons. Well? Why won’t businesses come clean with everyone and say that’s what they want?

Let me tell you the cold, hard, truth. You aren’t going to like this, but I ask you to believe me when I say that I am telling you this for your own good:

There is a culture of pretending business programming is more than it is. Some of you calling for more Java in University may take false hope that I am on your side. You may think that the people arguing for Scheme, Haskell, and OCaml are elitists. Wrong. They do not have a problem. You are the one with a problem because you don’t want to tell all your friends you have a job as a clerk.

So you want to be a clerk

You do a clerk’s job, you settle for a clerk’s working conditions and wages, but you take solace in the thought that you are somehow more than a clerk, because you have a university degree and the dental technician who cleans your teeth doesn’t.

Only everyone knows it’s a sham, especially the hiring manager who puts “University degree required” in the job advertisement. He wants to hire a clerk, someone who will work long hours doing as they’re told in a top-down, hierarchal command structure. Does that job sound like there is any Science involved? Of course not, everyone knows that, it’s why the industry is trying to weed all of the Science out of a Computer Science degree.

And you’re falling for it, hook, line, and sinker.

The university just wants your money, they’re selling you a gown and mortar for the low, low price of $5,836 a year (books, room, board, and other expenses not included). Business just wants to let you have your little fantasy of being a cut above the X-Ray Technician who does the same, repetitive thing all day working with a big flat screen and data that goes on a hard drive ($45,950 median salary).

The sooner you figure out the game, the sooner you can start playing instead of being played.

So we dumb down the curriculum. Who loses? Please don’t tell me the University loses its reputation. They don’t care, their budget is based on how many people like you they can con into getting a piece of paper that isn’t really any better than a vocational college certificate. If they need any more prestige, they’ll take your fees and give them to someone who will write a paper.

You didn’t write any papers, did you? The one thing that is really, really useful for getting ahead in business is learning to write, learning to speak, learning to persuade. If a Computer Science degree was really meant to give you an advantage in a business environment, it would involve a lot more writing English than writing Java. You would know that if you went on to get an MBA.

Anyways, the university is doing just fine, thank you for helping to pay for a fancy new building. How about businesses? Are they losing because they're paying university degree wages for clerks? No, because the secret of business is that the market works out the right price unless you can monopolize supply.

And you, my friend, are not monopolizing supply. Did your university have a strict cap on the number of undergraduates entering the CompSci program? No? Then your degree has absolutely zero leverage. Wait, I could be wrong. Do businesses refuse to hire workers with foreign credentials? They do hire workers with foreign credentials? Well, there goes that component of value.

Let me tell you, and I speak as a hiring manager. We lie to you. We write in articles and books and on blogs how much we value a degree. Joel Spolsky argues that there ought to be good degrees. But I will bet anyone $100 that he will hire a high school graduate if that graduate is smart and can get things done.

Beyond your actual ability to write programs, a degree is only as meaningful as its scarcity. If degrees are easy to get, they mean squat. Sure, when I’m hiring I might choose to toss all the no-experience resumés without degrees. I’m still left with a pile of two hundred people to interview. Do you think I’m paying any of you top dollar when I have 199 more people to see?

The reality is that your degree is only a pacifier, a way to make you feel good about yourself. The industry is selling you the illusion of respect.

I’m telling you this because the sooner you figure out the game, the sooner you can start playing instead of being played. If you really want to be more than a clerk, you can pay more attention to what is to be done and how much freedom you have to do it and less attention to whether there is a title or a degree involved.

You can be respected for your job. But it has nothing to do with whether we pretend a university degree is required for the position. Everybody knows it isn’t: if it was, there would be no need to dumb down the program to suit the job. Respect comes from what you accomplish, and where programs are involved, it comes from writing programs, not from a title or a piece of a paper (nor from a book or a blog). Nobody on Earth can stop you from writing software and earning genuine respect.

So do yourself a favour. Don’t let them play this game at your expense. Don’t allow them to disrespect you so transparently.

You deserve more respect than that.

p.s. Look, there is nothing wrong with being a clerk as long as you not in denial about your job. There is nothing wrong with educational institutions helping to fulfill a demand for clerks. And although I don’t think it is in a business’s best interests to treat programmers as clerks, there is nothing wrong with a business making that choice.

That being said, there are institutions that offer much more than vocational training in their undergraduate programs. Here’s why it’s in your best interests—not business’s best interests, but your best interests—to get a degree involving actual Computer Science, not vocational training: What good is a CS degree? And furthermore, there are plenty of programming careers that are an alternative to clerking. I am not going to say better or worse, just different. You don’t have to follow that road, no matter how many others are going that way.

The point of this post is that you need to have your eyes wide open when choosing what type of education you need and what type of career to pursue.  ¶ 6:47 PM 22 comments

  No Detail Too Small Expression-oriented programming (also known as functional or side-effect-free programming, although the three things are related, not synonymous) is a wonderful way to make calculations easier to understand and maintain. However, sometimes deeply nested function calls or mixing function calls with method invocations can make an expression difficult to understand at a glance. Here is a tip for refactoring your expressions so they are easier to read.

Expressions naturally form a tree, with values at the leaves and function calls or method invocations at each node. In this post, I’ll be talking about the simplest form of expression, a pipeline. A pipeline is an expression that does not branch: a value (or often collection of values) is transformed by two or more function calls or method invocations in succession. Here’s a slightly obfuscated example of a pipeline working with collections from one of our Rails applications:

widget.fizz.fizz_buzzes.select { |fizz_buzz| 
 fizz_buzz.widgets_column_name =~ /^special_.*/ 
}.map { |fizz_buzz|
 widget.attribute_present?(fizz_buzz.widgets_column_name) &&
   { fizz_buzz.label => widget.send(fizz_buzz.widgets_column_name) } ||
   {}
}.inject({}, &:merge)

While the details don’t make much sense out of context, the overall pattern ought to be familiar as an example of the MapReduce pattern (without the distributed processing, of course).

Pipelines read from right-to-left, left-to-right, or both. For example, this set of three nested function calls reads from right-to-left:

sum_numbers.call(square_numbers.call(odd_numbers.call(1..100)))

If I try to read it from left-to-right, it’s sounds like a caricature of speech: “The sum of the squares of the odd numbers from one to one hundred.” You can’t figure it out unless you build an abstract syntax tree in your head and then evaluate it with a stack machine. Having to emulate a computer to figure out what something means is not a good sign. it reads much easier from right-to-left: “Take the numbers from one to one hundred. Select the odd ones. Square them. And finally, take the sum.”

Popular languages like Ruby make it easy to write expressions that read from left-to-right directly: here’s an example from Ruby 1.9 (or with Symbol#to_proc):

(1..100).select(&:odd?).map { |n| n*n }.inject(&:+)
    => 166650

Object orientation’s emphasis on nouns at the expense of verbs has its issues. But when a computation really is a step-wise transformation of data, I find that chaining methods makes code a lot easier to understand than nesting functions. On the other hand, I prefer nesting functions when the expression has more of a tree form.

But whichever direction you prefer, I find it very difficult to read code that mixes directions in the same expression:

square_elements = lambda { ... } # content elided
square_elements.call((1..100).select(&:odd?)).inject(&:+)

You go left-to-right to select odd members, then back left to square them, then right to sum them. I find this much more confusing than either the nested functions or the chained method calls.

Object#into

A little while ago, I saw John Carter define factorial in Ruby as a method in the Integer class:

class Integer
  def factorial
    return 1 if self <= 1
    self * (self-1).factorial
  end
end

My first reaction was to think that adding factorial as a method was an idea from another planet:¹ why should integers know how to answer their own factorials? This seemed like a classic case of a function that should not be an object method. But nevertheless, having calculations be methods instead of functions lets you write a certain type of expression consistently from left-to-right (5.succ.factorial.succ.odd) instead of mixing directions (factorial.call(5.succ).succ.odd?).

All the same, there are good reasons why we don’t overload numeric classes with every possible calculation and formula. So what can we do? How about:

class Object
    def into expr = nil
        expr.nil? ? yield(self) : expr.to_proc.call(self)
    end
end

Now, snarfing Charles Duan’s code, we can write:

y = proc { |generator|
    proc { |x|
        proc { |*args|
            generator.call(x.call(x)).call(*args)
        }
    }.call(proc { |x|
        proc { |*args|
            generator.call(x.call(x)).call(*args)
        }
    })
}

factorial = y.call(proc { |callback|
    proc { |arg|
        if arg.zero?  then 1
        else          arg * callback.call(arg - 1)
        end
    }
})

Which lets us write:

5.succ.into(factorial).succ.odd?
    => true

I read this as “Start with five, get its successor, put that into the factorial proc, take the result’s successor, and answer whether it is odd.” The whole thing reads in one consistent style, you aren’t mixing left-to-right method chaining with right-to-left nesting functions. I wouldn’t go crazy with Object#into in a program, but if you have an expression that is predominately chaining methods, Object#into can make it consistent and improve its readability.

Function Composition

There is more than one way to skin a cat. If f(g(h(value))) is too constricting, we can compose functions instead of nesting them. So we can write:

class Proc
  def self.compose(f, g)
    lambda { |*args| f[g[*args]] }
  end
  def *(g) # Tom's origional composition operator
    Proc.compose(self, g)
  end
  def |(g) # The reverse composition operator, mimicing a pipe
    Proc.compose(g, self)
  end
end

plus1 = lambda { |n| n + 1 }
squared = lambda { |n| n * n }
minus1 = lambda { |n| n - 1 }

This allows us to write (minus1 | squared | plus1).call(5), which puts almost everything in left-to-right order. Hey, remember Object#into? Why don’t we try it?

5.into(minus1 | squared | plus1)

That saves us from writing 5.into(minus1).into(squared).into(plus1) if we find three instances of “into” a little noisy. Composing functions using * lets us maintain right-to-left order and composing functions with | lets us create left-to-right order when we are making a “pipeline” of expressions.

Summary

In the end, this is a very trivial idea: When an expression can be written so that it reads consistently from left-to-right or consistently from right-to-left, do so. The code will be easier to read.

---

1. Uh, yes, I am familiar with Smalltalk. I’m thinking that my opinion of my ability to make a joke far exceeds my actual ability: the phrase is meant as a pun on Edgar Rice Burroughs’s Barsoomian Tales, featuring the Warlord John Carter. But all that being said, regardless of how OO you want to get, I am not convinced that objects are responsible for every operation that can possibly be performed on them.

  ¶ 8:50 AM 12 comments

  The challenge facing programming language designers

Programmers are in a race with the Universe to create bigger and better idiot-proof programs, while the Universe is trying to create bigger and better idiots. So far the Universe is winning.

And the corollary:

Programming language designers are in a race with the Universe to create bigger and better idiot-proof programing languages, while the Universe is trying to create bigger and better programming idiots. So far the Universe is winning.

  ¶ 2:20 PM 6 comments

  There's no shame in being a gear head

You guys are TECHNOLOGY JOURNALISTS. You are GEAR HEADS. There is no shame in this, but, come on, recognize that what you think is cool is NOT what my mom thinks is cool, or what an executive thinks is cool, or what a lawyer who just wants to write a deposition on her laptop thinks is cool.

When you stop laughing (or frowning), take these words to heart. They apply to bloggers, programmers, even product managers thumbing emails into their crack berries. What you think is cool doesn’t count unless your market is composed strictly of people just like you. And those people just like you? They are outnumbered 1,000 to one by people unlike you.

All software developers should be hailing the advent of the computer-as-appliance, because it means we'll be reaching into markets that are afraid of self-service machines.

That is an idea I can savor.  ¶ 8:44 AM 6 comments

  Informed Choice I found some folks arguing about whether developers should write messy code. Having observed our amazing midwives “deliver” for the second time, I believe it is often an issue of Informed Choice.

Presuming that the client or stake holder bears the entire “cost” of the decision, they deserve the last word. I know more than a few people who argue with this, saying that writing clean code is in the client’s best interests. Well yes, of course it is. And in my experience, so are a number of other practices such as managing a project using Theory P, using small teams, and customizing a design for the specific project. But let’s make this really simple: the client is asking for something which—in your professional opinion—is not in their best interests but also is not so ruinously bad that the entire project will absolutely, positively fail.

There can be a set of options that are all somewhat reasonable, with some better or much better than others, but none being outright wrong. In midwifery, there is a choice between giving birth in the hospital or at home. There is a choice between having an epidural or going natural. There are choices about what to administer to the newborn as a prophylactic against infection. The mother makes an informed decision on each of these subjects with the assistance of the midwife, and the midwife supports here even if it is a decision that the midwife feels is statistically less than optimum.

In software development, a client may wish to design everything up front. I believe with all of my heart this is not the best way to go, but let me tell you: I have managed projects on this basis and succeeded. I think I have done a better job managing projects another way, but I can’t tell you that a project absolutely has to fail if you start this way. Likewise large teams deliver software, and some of the things I have built have had a very high buzzword-to-value ratio.

I would argue against these practices in many cases, but if the client is fully informed of the possible outcomes, they must be permitted to control their own destiny.

I know there’s a grey area here. If you believe the client wants things so messy and so unhygienic that you don’t have confidence in the project, I will support your decision to say no to the client. I applaud that. But if it’s in that grey area where we are arguing the difference between good and mediocre but serviceable from the client’s perspective, I believe that my responsibility is to provide clients with the information they need to make the decision.

So for example, if they demand BDUF, I have to tell them how I think the project will actually go if we walk that line. I need them to “sign a consent form” explaining that they understand the risk that we will deliver something that will meet the written requirements but will not meet their updated, educated requirements once the project is underway. They must be clear on the fact that this choice will be a loss for them if and when they realize that the origional direction was imperfect. I need them looking me straight in the eye and saying “I understand what you are saying, but I am willing to take that chance.”

If they argue with me and tell me I’m wrong, that BDUF is always better than iterative, then I know that I am not the right man for them. I know that they are not making an informed choice. And likewise, if asked to do something quick and dirty, I will say “no” if the client tells me they believe it is the best way. But if they understand technical debt, if they know they are making a trade off of now versus later, who am I to reach into their pocket and spend their money as I see fit?

Our midwives know much more about giving birth than we did. And they advised us accordingly. But we bore the responsibility for certain decisions, and they ceded those decisions to us, even when they had strong opinions about what was the best thing to do. I think as a software developer I must follow the same path. A midwife will not allow a client to perform a home Caesarian section. There are limits, there are things we know mean instant and irrevocable failure. But where judgment is required, where there is a decision between mediocre, good, and better outcomes, they advise and then allow the client to make the final say.

I believe we in software must follow the same path.  ¶ 9:02 PM 8 comments

  Off Topic: Clara Maude Braithwaite Arrived this morning at 7:52. Mother and baby are doing fine, brother Thomas is delighted.  ¶ 9:24 AM 24 comments

  Off-Topic: Why did he say she? Someone raised the question of whether it was appropriate to use the pronoun “she” in a recent anecdote about envelopes. The question included some irrelevant anti-diversity ranting, but the part that was worth consideration questioned whether the generic personal pronoun should be “he” or “she.”

The rule in English is clear: if the person is of unknown gender, use he. For example:

The rule is that when a writer makes a statement about an unknown person, he should use “he.” He should not use “she” unless he is making a statement about a person who is female.

Therefore, if I am writing about an unknown person and I use the word “he,” I am not asserting that the person is male. It isn’t even the masculine pronoun in this context.

However, that has nothing to do with the anecdote. The reason why it has nothing to do with the anecdote is that the anecdote was not about an unknown person. It was about a fictitious person who was female.

Once upon a time there was an author who was trying to decide which pronoun to use. She was told that when she was making a statement about an unknown person, she should use “he.” She was told that she should not use “she” unless she was making a statement about a person who was female.

In this example, I am not talking about an unknown author, I am asking you to consider one specific fictitious author whom I have decided is female. There is no debate about that: I am entitled to write about women, Canadians, Scots, and anyone else I fancy.

There is a question as to whether the manager’s gender is relevant. Including details like this helps you see in your mind’s eye the person I saw when I wrote it. It is not obvious whether this improves the anecdote or whether the anecdote would be better if the gender was omitted.

But I think it is an appropriate use of the word “she,” and I stand by its use in the anecdote.

Update: The difference between a specific pronoun and a generic pronoun is clear when you contrast the story about a fictitious manager with this sentence from Whatever happened to code reviews?:

If someone writes code in that language, they are absolved from all responsibility for its style. And so is their manager: the programmer’s code compiled, it is demonstrably free of bad style, what can we demand of their manager other than to see to it that they use the language?

In that post, I used they are instead of he is, and perhaps this is wrong, but this is unmistakably generic: I am not making a statement about some programmer, I am making a statement about all programmers who use a certain type of programming language. This is very different.

Elsewhere, Kate Rhodes bought up the point that choosing to depict women in programming stories encourages women to enter the profession. This may be true, but I confess it is not why I do it from time to time. In actual fact, I know a number of very capable women who manage software development, so when I sometimes choose to write about a fictitious man and I sometimes choose to write about a fictitious woman, I am simply reflecting my own experience.

If your experience does not resemble mine, that is good: what is the point of reading if not to discover new things?  ¶ 10:39 PM 9 comments

  Whatever happened to code reviews? I have read an argument that a programming language can enforce good programming style by restricting programmers to constructions and mechanisms that are deemed worthy, or perhaps are restricted to a small set so that there is an enforced consistency in a code base. (I am not talking about using compilers to detect bugs or memory management being used to prevent bugs, I am talking about programming languages that do not permit certain kinds of abstractions outright).

The premise of the argument is that if programmers are constrained to a single, universal “good style,” good programs will result.

responsibility and authority cannot be decoupled from each other

In many other forms of management, people have attempted to automate the manager out of a job. There is an idealistic notion that if we write just the right series of instructions for people, if we build the perfect instruction manual, if we detail every possible case and script every possible behaviour, we can build a team that always does the right thing. We see that in highly industrialized occupations like preparing fast food: there is a detailed, step-by-step instruction for everything, and employees simply follow the rules.

Many things have been written about whether that works or doesn’t work for software developers. I am now talking about why that doesn’t work for teams, for team leaders, and for development managers. Every thing you put in place in an attempt to enforce good behaviour takes something away from the team’s responsibility to manage the process using judgement.

a false sense of security

We joke about a compiler generating a false sense of security: it compiles, so it must work. But that is exactly the mind set of choosing a language on the basis of it actively preventing poor style: we are choosing it because it will relieve us of the work of manually reviewing code and analysing whether it is of good style or not. It will relieve us of the work of setting a good example and evangelizing good code. It will relieve us of the work of teaching, of leading, or managing.

In the fast food example, we know that the “managers” in the restaurants are not true managers. They have little or no authority to deviate from the rules. And for that reason, they have little or no accountability for what results: you can fire them for failing to follow the rules, but if they follow the rules and bad things happen, it is the rules that are at fault, not the manager.

And so it is with enforcing “good style” with a programming language. If someone writes code in that language, they are absolved from all responsibility for its style. And so is their manager: the programmer’s code compiled, it is demonstrably free of bad style, what can we demand of their manager other than to see to it that they use the language?

Software development is a young profession. We don’t know an awful lot about what works and what doesn’t work. but I’ll take a stand here: We humans have an awful lot of experience with the relationship between authority and responsibility. And what we have learned is that if we remove an individual’s authority to choose, we absolve them of responsibility for the result, and ultimately performance suffers greatly even if their skills are objectively high.

automatons on the style council

Does this mean that automation is bad? Why do we use high level languages and compilers? Why do we use libraries and frameworks? Don’t those tools also relieve us of responsibility?

Well yes, of course they do. They relive us of responsibility for things we have deemed not to matter. For the vast majority of lines of code, the exact translation to CPU op codes is unimportant, as long as it works. Nobody cares to look under the hood. Compilation is the process of automating the accidental complexity of our CPU.

If you choose to use a programming language with lazy evaluation, you have decided that evaluation order is an accidental complexity that no longer matters to you, and you want the language to sweep it out of sight. What we choose to automate speaks to what we consider irrelevant. As long as it works, we don’t care how it’s done.

So when we automate style, we are deciding that style dosn’t matter. As long as it passes the compiler, it is good style.¹

Automating good style ultimately weakens the notion of individual and shared code ownership, it emphasizes the notion that if it passes the gate, it must be good. That, in turn, creates the illusion that no further inspection is required. If it functions as we require, and it is written in the language that enforces good style, why do we need to review it?

We can talk about why role of a programming language should be to enable good style, and have a terrific debate about what that means and which languages do a good job with it. But it is more important to keep our eye on the ball. The most important factor for the success of software projects is still the people. And what makes people produce good software? Taking responsibility for their work. Ownership and accountability.

Ownership and accountability is weakened when you attempt to constrain the programmer and it is enhanced when you provide a programmer with freedom and follow up with review. Thus, my question:

Whatever happened to code reviews?

---

1. Python’s significant whitespace is a good example of this. By enforcing a standard indentation, Python says that indentation isn’t worth thinking about: everybody does it one way, and that’s a benefit for readability. Compare and contrast this to a hypothetical language Imperitivity. Imperitivity does not have anonymous functions or closures as first-class objects, and many Imperitivity programmers argue that they should not be added to the language because that would be bad style, or it would promote inconsistent style with some people using closures and some not.
   
   Would we seriously equate the two positions? I think not: whitespace really is pretty trivial: the fact that we can write a script or IDE plug-in to “fix” indentation to match a style guide is evidence that it should not take up valuable team and management attention.
   
   However, the question of whether a code base should include closures, should not include closures, or ought to enforce a consistent approach to closures is important. Using a language to settle the question is an abrogation of responsibility: the matter should be decide by the team or technical leadership and enforced through review, mentoring, training, example code, and so forth.
   
   The salient difference is that while consistent indentation is useful, indentation itself is unimportant in comparison to other technical choices the team must make. Important choices, such as the appropriate paradigms or idioms to use, ought to be decided by the team and enforced through review.

  ¶ 1:38 PM 17 comments

  Billy Martin's Technique for Managing his Manager A software manager is brought into a doomed project. It is hopelessly behind, the requirements have changed so often they are now kept on a white board instead of in a document, and the office is wallpapered with Dilbert cartoons. She’s replacing the previous manager, who has resigned.

When she gets to her new office, her predecessor is clearing out her desk. “I’ll only be a minute. By the way, I left you something” says the old manager before leaving. After the predecessor leaves, the new manager has a look around, and discovers two envelopes in the top drawer of the desk. The first envelope is labeled “Open when in trouble.” The second is labeled “Open when in even more trouble.”

Well, she researches anti-patterns in software project management, has a look around, talks to everyone, identifies the key players, and goes to management with a clear explanation of what the problems are, what’s realistic to achieve, and what needs to be done. To her surprise, management seems to go along with things and tells her to make whatever changes she needs. She institutes daily builds, stand up meetings, rapid iterations, and starts tracking velocity. Two months later, at the quarterly review, she presents her progress. The team is producing at 200% of the previous rate, morale is up, and they’re going to get 80% of the previous functionality done with less than 90 days of slip off the impossible schedule.

There is silence, then management tells her that this is unacceptable. Sure, they agreed she could institute change. That’s her job. However they did not approve any compromise in scope or delivery date. This is a competitive business, and there are thousands of qualified managers looking for a job who can come in and get things done. Does she want to admit she can’t do her job?

We have assigned a product manager to discuss IBM WebSphere Portal integration with your technical lead. And the VP of Marketing needs to talk to you. We have made a commitment to all-singing, all-dancing web services and he needs a forty-minute presentation explaining how your project will be implemented with web services throughout. You look pale. Is there a problem?

She goes back to her office in a quandary, then remembers the first envelope. Wordlessly, she opens it, and inside there is a three by five card inscribed: Blame Me.

She goes back to management a day later with a powerpoint emphasizing how the project was way off track, how the architecture was fubar, how morale was poor, and how requirements were incorrectly documented. She doesn’t blame the previous manager directly, but she does identify various management techniques and “best practices” that were not enforced prior to her joining the project. She says she will work the team to hit the date, but she demonstrates how commitments made by the previous manager were done without actually checking to see whether the team could deliver or making any changes to actually hit the new dates. After some huffing and puffing, management buys off on delivering 80% of the promised functionality with a 45 day slip in the delivery date. Whew!

Well, things go along quite well until two weeks before the original due date. Management calls her in and asks her to sit on the release planning board, there will be a major customer release in two weeks. She reminds them of the 45 day slip they negotiated.

“No,” they tell her baldly, “We agreed on delivering 80% of the functionality on the original date and the remaining 20% of the functionality 45 days later in release 1.1. Furthermore, we are entering into a strategic agreement with IBM and have assigned a product manager to discuss IBM WebSphere Portal integration with your technical lead. And the VP of Marketing needs to talk to you. We have made a strategic corporate commitment to all-singing, all-dancing web services and he needs a forty-minute presentation explaining how your project will be implemented with web services throughout. You look pale. Is there a problem?” She leaves the meeting aghast.

Well, it’s back to her office and she opens the second envelope with shaking hands. The advice on the card is of little comfort: Cercner gjb rairybcrf.

This story is often told about baseball manager Billy Martin. He managed the Yankees five different times, because he was prepared to be fired when things didn’t work out. The moral of the story: Do the very best you can. Do so because YOU want to do so, regardless of whether you will succeed or not. Regardless of how dysfunctional your organization is, hold your head up and stay true to your values. Give them at least two chances to change. Do what you can to help them change. But if they do not change, move on with dignity.

Update: Someone suggested that the ability to quit your job is a luxury. I don’t like to use that word, because it suggests that the job itself is a luxury. It is possible that the pay cheque is a luxury, but for people who are engaged in their careers, the job is a necessity because their self-actualization depends on doing a good job. When you put it that way, the question becomes whether people have the luxury of remaining in a job where they don’t feel good about themselves and what they do.

p.s. Yes, she’s actually a woman. Which is why I used the word “she.”  ¶ 7:59 AM 5 comments

  A Familiar Quote

A programming language that doesn’t change the way you think about programming is not worth knowing.

  ¶ 9:01 PM 3 comments

  The word "intuitive"... isn't. First off, thanks to everyone who wrote in with different interpretations of what the word “intuitive” really means. I agree that Jef Raskin did not have the final say.

Intuitive does not only mean familiar. It can also mean consistent, stable, discoverable, and exhibiting affordances. And some other things. The reason it means different things is that “intuitive” is not a concrete or objective property of software. It is the result of a good combination of objective properties. “Intuitive” is the result of good design, not the cause.

And this is why it is not a very helpful word. Like good design, it means different things to different people. So saying that such-and-such a programming language is “intuitive” is far less meaningful than saying it is consistent, or stable, or discoverable, or “familiar to C++ programmers.” Intuitive is a word people use to describe their subjective impression of whether something is well designed. Someone can always argue with the word “intuitive,” but we can all agree on objective characteristics like whether Java or PHP are consistent. Or we can compare the intellectual surface area of Haskell to Ruby.

So on to why a programming language cannot be better without being unintuitive:

All programming languages are equally powerful. So what does it mean to be “better”? Better is a subjective word, a result, just like intuitive. The sentence is consistent, because it compares two subjective things to each other. And obviously it makes a lot of sense if you accept Raskin’s rather narrow explanation that intuitive equals familiar. (Although narrow, it fits my experience when talking about how non-professionals view end-user software: In a recent project, clients were still claiming that search boxes for each field in a table were more “intuitive” than a single full-text search. Naturally, these were older managers who had spent their formative years working with ASCII applications that worked on that principle.)

So for the narrow explanation that intuitive springs from familiarity, a language cannot be better without being unfamiliar. Well, how can it be “better” if they’re all Turing Equivalent? Only in a subjective way; only by making you better. And in that sense, a language is only better if the process of using it makes you better as a programmer. Which comes from being moved out of your comfort zone and into a place where you must learn, and where the things you learn are productive.

Not rote memorization of when to use .length, .length(), .size(), or .getLength(). But rather when you learn new ways of thinking about the problem to be solved. Not every unfamiliar language is better, but every better language is unfamiliar.

Now what about the other ways you can arrive at software that feels “intuitive”? What about a language that is unfamiliar but consistent, stable, and discoverable? Won’t it be better? Well, yes, of course. In fact, it will be a better better than one that isn’t. You will learn more from functional programming in Haskell than functional programming in Ruby.

But what if a language was familiar but improved on what you already use by being more discoverable, more consistent, more stable, and so forth? How much better could it really be? In the sense of improving you as a programmer, how much personal value can we extract from a language that improves upon the familiar while still being familiar? Raskin suggests we will obtain at best minor improvements. Is Java really only a little better than C++, if at all? In the sense of making you a better programmer, what does Java teach you if you are already a competent C++ programmer?

(Of course, a straw man interlocutor can ask, “But what if I don’t already know C++? Isn’t Java a better language for me to learn first?” Ys, it may be a little better to learn. Perhaps. But in that case, the premise of the post is assured since neither language is familiar to the beginner.)

Unfamiliarity is the driving force behind being better in the sense of a programming language. And “intuitive” isn’t a real property of programming languages, it’s the result of good design.  ¶ 10:34 AM 14 comments

  A programming language cannot be better without being unintuitive

Optimising your notation to not confuse people in the first 10 minutes of seeing it but to hinder readability ever after is a really bad mistake.

Or as Jef Raskin put it:

The impression that the phrase “this interface feature is intuitive” leaves is that the interface works the way the user does, that normal human “intuition” suffices to use it, that neither training nor rational thought is necessary, and that it will feel “natural.” We are said to “intuit” a concept when we seem to suddenly understand it without any apparent effort or previous exposure to the idea.

It is clear that a user interface feature is “intuitive” insofar as it resembles or is identical to something the user has already learned. In short, “intuitive” in this context is an almost exact synonym of “familiar.”

The term “intuitive” is associated with approval when applied to an interface, but this association and the magazines’ rating systems raise the issue of the tension between improvement and familiarity. As an interface designer I am often asked to design a “better” interface to some product. Usually one can be designed such that, in terms of learning time, eventual speed of operation (productivity), decreased error rates, and ease of implementation it is superior to competing or the client’s own products. Even where my proposals are seen as significant improvements, they are often rejected nonetheless on the grounds that they are not intuitive.

It is a classic “catch 22.” The client wants something that is significantly superior to the competition. But if superior, it cannot be the same, so it must be different (typically the greater the improvement, the greater the difference). Therefore it cannot be intuitive, that is, familiar. What the client usually wants is an interface with at most marginal differences that, somehow, makes a major improvement. This can be achieved only on the rare occasions where the original interface has some major flaw that is remedied by a minor fix.

Complaining that a programming language “Violates the Principle of Least Surprise” is a complaint that it fails to appeal at first glance, without any training or investigation. But if a new language is “intuitive,” it can only offer you a marginal improvement over the languages you already know well.

You needn’t look at functional langauges like Scala to see this at work. Is Ruby’s for loop an improvement over Java? By how much? Ruby’s big win over Java in that regard is the ease with which you can use Enumerable’s collect, select, detect, and inject methods. Which, of course, are not familiar to the programmer with a grounding in for loops. They require study to understand. But once understood, they make code easier to read thereafter.

---

post scriptum:

As Justin points out in the comments, the real meaning of the “Principle of Least Surprise” is that languages should be consistent. The principle applies to how the language behaves when you already know something about it. So if you see (_+_), the questions is whether this makes sense if you already know that underscore is a function parameter.

And more: The word "intuitive"... isn't.

And even more, from Matz in Japanese: http://www.rubyist.net/~matz/20080108.html#p02  ¶ 10:51 AM 21 comments

  Another problem with problems with design patterns

The documentation for a design pattern describes the context in which the pattern is used, the forces within the context that the pattern seeks to resolve, and the suggested solution. [For example:] The visitor design pattern is a way of separating an algorithm from an object structure. A practical result of this separation is the ability to add new operations to existing object structures without modifying those structures.

I have read the view that design patterns represent deficiencies in a programming language. This argument stems from the observation that the specific code examples for half or more of the patterns in the GoF Book are "superfluous" in languages that provide abstractions above and beyond basic OO. The Visitor pattern can be said (depending on which Golden Hammer you are holding) to represent a way to brew your own multiple dispatch, open classes, or pattern matching.

Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it.

Does this mean that given a sufficiently expressive language there will be no design patterns? Certainly not. Each design pattern organizes a problem and one suggested solution in a structured way, giving the solution a name so that programmers may enjoy a common vocabulary. The pattern would only go away if the problem goes away.

But why would a more expressive language have this better feature, if not to solve the problem? Clearly the problem doesn’t go away, it is just that instead of writing our own incomplete and buggy mechanisms for multiple dispatch we can now use the language’s complete, tested, and “free” multiple dispatch feature to solve the problem.

If you consider the problem to be “How do I implement my own multiple dispatch,” then the Visitor pattern is obsolete if you are using a language like Common Lisp. However, if you consider the problem to be “How do I separate algorithms from data structures,” then the only thing obsolete in the originally publicized pattern are the language-specific details. The high-level approach can still be relevant.

A programming language is low level when its programs require attention to the irrelevant.

Some programming languages accomplish the same task with a declaration or a singe line of code. From this, some draw the conclusion that design patterns exist because popular programming languages lack certain abstractions or they lack the meta-abstractions permitting you to build your own abstractions cleanly. The underlying assumption here is that the purpose of the pattern is to explain specifically how to write the code.

This is clearly not the case. The Ruby language implements the Singleton pattern with a single abstraction, the Singleton module. include Singleton is all you need to implement the pattern. Yet, would we say that in Ruby there are no Singletons? No, we would say that Ruby makes them easy.

I always felt the beauty of Java was that there is only one ‘standard’ way of handling most problems. I know it might not be ‘fun’, ‘smart’ or ‘elegant’ in most people’s views, but most software factories don’t want people to write elegant code. They want standardized maintainable stuff that people can understand ten years from now because it’s written in the same fashion [as] everything else out there…

I really think that great code is easy to understand and follows patterns that people are used to and comfortable with… I read someone’s code I just want to be able to understand it as quickly as possible, find the spot where the changes go and be able to figure out how said changes should fit in the general design.

One argument is that implementing code a certain way is what makes it easy to read, what makes it familiar. This line of thinking is grounded in the idea that the valuable part of the patterns is the suggested solution at a detailed level, because that is what will enable