raganwald

Why Apple is more expensive than Amazon

2008-05-06T10:56:00.020-04:00

Let’s say you would like to listen to one of the most amazing performances ever made, Glenn Gould’s 1981 recording of Bach’s Goldberg Variations. And let’s say you have figured out the DRM is a really terrible idea, so you would like the hear it on high-quality MP3s.

The good news is that there are a number of places to legally purchase the right to download DRM-free MP3s. Lots of music from the major labels is available on Amazon, and it costs the same or less per track than Apple’s iTunes Music Store (“iTMS”), and you can find more DRM-free tracks on Amazon than on iTMS.

For example, Bach: Goldberg Variations, BWV 988 on 256-bit DRM-free MP3 is just $9.99 from Amazon. The same album is also $9.99 from Apple, but you get DRM. And there are tons of tracks on Amazon that are actually less expensive than on iTMS, so you get better music for less money without the DRM hassle. So is Apple screwing the customers?

In a word, no.

The reason you can find more music on Amazon at a lower price is that the Record Labels want it that way. Do you think they charge Apple and Amazon the same price for each track and Apple simply charges you more and pockets the difference as a higher markup? The labels would like you to think that, but they actually charge Amazon less for each track, and that’s how Amazon can charge you less.

Do you think Apple insists on the DRM but Amazon has the vision to see that the future of music is DRM-free? Do you think Jeff Bezos is a better negotiator and he was able to get a better price per track than Steve Jobs? Without putting up with DRM?

Really? How does that work, when iTMS is now the third-largest music retailer in the United States? How does the lower-volume store get better selection with fewer hassles and negotiate lower prices from its suppliers? When Steve Ballmer—not an easy man to intimidate—asked them for music to put in his Zune marketplace, these nice people extorted a $5 tax on each Zune sold from him. So how did Amazon get such a sweet deal?

Price fixing is how.

The major labels want nothing more than to break Apple’s dominance of the digital music business. They spin it as a good thing. More retailers means more competition, which is good for consumers. But let me ask you: if Amazon selling music for 89 cents a track is good for consumers, why isn’t iTMS also selling music for 89 cents a track good for consumers?

It would be good for consumers, but it wouldn’t be good for the music labels. The reason it wouldn’t be good for the music labels is that they really don’t want consumers paying 89 cents a track for DRM-free music. They want consumers to stop using iPods and start using devices with DRM that the labels can control.

They want consumers using devices in proprietary silos like old-fashioned cell phones, where you pay for the track, you pay for the bits transferred over the air, and then you pay all over again when you want to use a few seconds of the track as a ring tone.

As soon as they can break this pesky iPod-iTMS-iPhone nonsense, the labels want to get back to dictating what you pay and how often you pay. The labels want to do business with people like Microsoft. Microsoft gets it: all the people who bought music using MSN music? They can buy it all over again at the Zune store.

Do you think that the MP3 genie is out of the bottle and will never go back? You haven’t been paying attention to digital video media. The movie folks have succeeded in making DRM a standard for movies on DVD, and it is even worse on BlueRay. Have you seen the hoops a Microsoft PC has to go through to play video these days? Each piece of hardware has to promise to be a good boy and not let you actually do anything with the bits without permission.

The movies people have imposed harsher DRM on movies, and the music labels like the look of that.

The only—I repeat only—reason the labels allow competing stores to have DRM-free tracks is that it’s the only way to get music onto an iPod. Think about that for a moment: Apple’s dominance of the music player business is the actual reason you can buy a DRM-free track from Amazon. If anybody else had a substantial chunk of the player market, the labels would be busy trying to make the other player’s DRM the standard.

Imagine a world where every online store sells DRM-free music encoded in open licensable formats. In such a world, any player can play music purchased from any store, and any store can sell music which is playable on all players. This is clearly the best alternative for consumers, and Apple would embrace it in a heartbeat.

If the big four music companies would license Apple their music without the requirement that it be protected with a DRM, we would switch to selling only DRM-free music on our iTunes store. Every iPod ever made will play this DRM-free music.

—Steve Jobs, Thoughts on Music

I know, you’re going to bring up Apple’s DRM. Let me ask you a question: who do you think wanted DRM on iPods? Apple? Don’t make me laugh. Apple built a nice little business selling iMacs using the slogan “Rip. Mix. Burn.” They would love to keep building their nice little iPod business on the back of DRM-free music.

Think about this: If you buy a 32 GB Apple iPod touch from Amazon for $472.54, do you think Steve Jobs stays awake all night worrying about whether you will buy tracks from Apple or from Amazon?

If you answered yes, then you understand that Apple would be delighted to offer DRM-free music for the same price as Amazon. Or even less, since the profit selling you an iPod can subsidize their on-line store.

I personally answered “no,” because in my opinion, there are only three threats to Apple”s iPod business: Someone will actually out-design a music player and make that is actually better designed, cell phones will become the dominant music players, or someone else establishes a DRM standard that locks Apple out of playing downloaded music. I am not going to worry about anyone making nicer music players, I think we see what Apple is doing about the cell phone business, and now we understand what Apple is doing about the DRM business: If nobody can establish a DRM business, if nobody can lock you into one player, then nobody can take the music business away from Apple simply by getting into a back room and doing secret, monopolistic deals.

I’m looking at you, Sony and Microsoft. If the whole world has DRM-free music, you have to compete with iPod by building better music players. Sony, at least, has a chance. And if the whole world can sell DRM-free music, then Amazon and the like would have to compete with iTMS by building a better music store. Except, of course, they don’t have to compete with iTMS because the labels are colluding to place Apple at a disadvantage.

It’s delightful that for the moment, we have a choice of where to buy DRM-free music. It’s wonderful that today, we can legally purchase the right to listen to a lot of major label music on DRM-free MP3s. But let’s not kid ourselves. If the labels break Apple, this will not last. The genie will go right back into the bottle.

Buy what you like, where you like. But remember why things are the way they are. Apple is more expensive than Amazon because the labels want you listening to music on a Zune.

Update: Thanks for the links, John and Jeff. I spotted an interesting idea in the comments on Jeff’s post: the labels may not succeed in breaking iPod, but if they can break iTMS, they might be able to salvage the ability to set arbitrary prices.

One of the things they really hate about iTMS (alongside letting you burn your own CDs, transfer the music between computers, and share your music over a LAN) is that all music costs exactly the same. Breaking iTMS will restore the biggest weapons they have against artists. That has been discussed before, I recall Joel Spolsky’s Price as Signal.

I have a truly marvellous title of this post which this field is too narrow to contain

2008-05-01T13:29:00.008-04:00

Management saw this general-purpose mail reader and said, “since this mail reader is popular with normal people, we must now pimp it out to ‘The Enterprise’, call it Groupware, and try to compete with Lotus Notes!”

To do this, they bought a company called Collabra who had tried (and, mostly, failed) to do something similar to what we had accomplished. They bought this company and spliced 4 layers of management in above us. Somehow, Collabra managed to completely take control of Netscape: it was like Netscape had gotten acquired instead of the other way around.

“Groupware” is all about things like “workflow”, which means, “the chairman of the committee has emailed me this checklist, and I’m done with item 3, so I want to check off item 3, so this document must be sent back to my supervisor to approve the fact that item 3 is changing from ‘unchecked’ to ‘checked’, and once he does that, it can be directed back to committee for review.”

Nobody cares about that shit. Nobody you’d want to talk to, anyway

—Jamie Zawinski, Groupware Bad (2005)

The hallmark of an architecture astronaut is that they don’t solve an actual problem.

Groove had some early success selling secure networks to the military-industrial complex, but didn’t make much of a ripple outside that niche. Their real success was in getting bought by Microsoft, which brought Groove’s designer and chief architecture-astronaut Ray Ozzie to the role of “Chief Software Architect” at Microsoft…

Ray Ozzie just can’t stop rewriting this damn app, again and again and again, and taking 5-7 years each time. And the fact that customers never asked for this feature and none of the earlier versions really took off as huge platforms doesn’t stop him.

—Joel Spolsky, Architecture astronauts take over (2008)

And my title?

Buying a company that makes shit that nobody cares about—nobody you’d want to talk to, anyway—just so you can replace your product leadership and vision with theirs and so that you can stop building your products that people actually care about and start building shit that nobody will care about…

Still bad.

The single most important thing you must do to improve your programming career

2008-04-27T08:02:00.012-04:00

Yes, I said improve your programming career, not your programming prowess. And by career, I mean all of it in any form, whether your career is working for BigCo, launching a start up, consulting, whatever.

Let’s assume you’re already as smart as Steve Wozniak, the über-engineer. Do you want to hack part-time for the rest of your life or do you want to change the world? To change the world, you must create the new, but you must also show people why it matters. And that’s why…

The single most important thing you must do to improve your programming career is improve your ability to communicate.

To program, you must elicit ideas from other people and share your ideas with them. Sharing ideas comes in many forms: explaining how you did something. Suggesting a new practice for the team. Demonstrating how something works. Convincing everyone to switch programming languages. Persuading a brilliant engineer to join your team. Persuading your manager to get out of the way and let you do your thing.

Advancing your career is entirely about communicating. Getting a job. Turning down a job. Asking for a promotion. Turning down a promotion. Getting onto a good team. Politely extricating yourself from a good team. Persuading a brilliant engineer to co-found a company. Helping a brilliant engineer understand why co-founding a company isn’t the right thing to do. Asking for funding. Turning down funding. Getting clients. Turning down clients.

It’s all communication. Now you know, I have a background in Sales and Marketing. So I worry that I am holding a Golden Hammer and suggesting to you that your career is all about nailing things. But here is the short and obvious truth: Human beings are all about physical communication. It’s a huge, huge part of what makes us human, this talking and listening jibber-jabbering thing.

Frankly, there is no substitute for actual speech with visible body language. We do not react to the written word the same way that we react to a speech where we can see the speaker. So let’s skip right past learning to write well. It’s phenomenally important, but learning to speak well trumps everything. At Mobile Commons we talk about technical things all the time, and the remote members of the team switch out of Campfire into iChat or Skype to hear each other’s voices and see each other’s faces at least once a day.

If you take just one thing from this post, let it be this: To improve your programming career, the single most important thing you must do is improve your ability to communicate your ideas face to face.

What to do right now

Would you like to improve? I have deliberately avoided suggesting what specifically you might want to improve: My Latin teacher said we crave the three Ps: Power, Prestige, and Pecuniam. But the nuänces of aspiration are many. You have ideas you want us to understand. You have people you want to associate with. You have things you want to build and you need to harness other people’s efforts to build them.

Here is what to do today to shorten the line between where you are and where you need to be: Volunteer to give a presentation that you can entirely control, and invest the time and effort to do the best possible job.

Improving your ability to present takes practice. But it also takes study. Start with Presentation Zen. Presentation Zen is not about how to make slides. It’s about how to think deeply about your ideas and how to transform them into a presentation.

And that’s why it matters: Thinking deeply about your ideas from a new perspective engages your brain and makes you smarter.

Now, you may already be giving presentations. Maybe it’s part of your job already. But I am still recommending you add one to your workload, one that you entirely control. This is crucial: if you don’t control the subject, the content, the audience, everything, it may not be the very best way for you to improve.

I recommend you find a venue that is begging for presentations—local programming users groups are ideal—and sign up to give a talk. Pick a topic. If you don’t have a burning desire to shout something across the roof tops, you can always fall back on giving an “Experience Report.” That’s where you discuss something you did or tried to do and how it worked out.

Or go with one of the ideas I suggested for blog posts: “What I learned from Language X that makes me a better programmer when I use Language Y,” “Something surprising that you probably wouldn’t guess about Language X from reading blog posts,” or “My personal transformation about Idea X.”

By the way: If you already have a certain degree of comfort with presenting to an audience and you really want to ignite your career, try doing a “Fish out of Water” presentation: Present to a group with whom you have very little in common. For example, give a presentation about how to work with programmers to a business networking breakfast club. Learning to present ideas to people who are not just like you will make you a superstar.

Here is what to do today to shorten the line between where you are and where you need to be: Volunteer to give a presentation that you can entirely control, and invest the time and effort to do the best possible job.

Now put together a presentation deck¹ for your talk. Do not give a talk without a deck, you will lose valuable experience giving presentations and this is about your experience. Furthermore, presentations are tougher than speeches because the presentation medium gives you many, many opportunities to screw it up (like having what you say and what’s on the screen be the same thing).

Then give the talk. And then give another one. And keep going. And never stop, because presenting well is a skill that helps you from where you are now right up to when you are presenting the numbers to the Board of Directors or presenting a new product to a hall full of cheering customers or, or, or anything and everything.

See you there!

That would be a “deck of slides.” Nowadays it’s a keynote, open office, or that-other-thing file, but once upon a time it was a set of 35mm transparencies you would load into a carousel.

Stacked up, they looked a little like a deck of cards, thus the nickname “deck.” You can probably find one (and someone old enough to remember when they were called decks) in a museum somewhere.

[back]

Why we are the biggest obstacles to our own growth

2008-04-25T11:11:00.005-04:00

When you’re old and everyone around you is doing or buying something you don’t understand, you think they’re assholes; when you’re young, you do it/buy one, too.

—John Gruber, talking about Apple’s strength

This statement is intriguing. I recall Michael Lewis making a similar point in his book The New New Thing: A Silicon Valley Story. Michael pointed out that change was being driven by the young because they didn’t have any sense of self to hang on to.

This is crucial. If you think of yourself as a—for example—Lisp Programmer, learning Factor and trying it on a project is not just an intellectual exercise, it’s a shedding of a little of who you think you are. If you have built software—successfully—using large teams with meticulous planning, working with a small team in a less structured setting is again more than an experiment, it’s walking away from part of what you are.

When you are young, you don’t have that sense of self to protect. You’re driven by a need to find out who you are, to turn the pages of your biography and see how the story turns out. If people around you are doing something you don’t understand, you assume the problem is your inexperience and you go to work trying to understand it.

But when you are old, when you know who you are, everything is different. When people around you are doing something you don’t understand, you have no trouble at all explaining why they are ~~assholes~~ mistaken. Let me give you an real-life example.

[] is Ruby for “Swiss Army Knife”

One of the examples I give for using andand is with regular expression matching. I had a bunch of code that looked like this:


begin
  md = 'The song "777-9311" is by The Time'.match(/"(\d{3}-?\d{4})"/)
  md[1] if md
end

And being the type of person who dislikes the proliferation of mutable variables, I rewrote my code to look like this:


'The song "777-9311" is by The Time'.match(/"(\d{3}-?\d{4})"/).andand[1]

This week, I received a very tactful email from Cornelius Mika pointing out that Ruby has this use case covered:


'The song "777-9311" is by The Time'[/"(\d{3}-?\d{4})"/,1]

My first reaction was rather unprintable. I have read the documentation for the String class many times, but for some reason this never stuck. My brain refused to parse the fact that [] means all of the following:


'The song "777-9311" is by The Time'[7]
=> 103
'The song "777-9311" is by The Time'[4..7]
=> "song"
'The song "777-9311" is by The Time'[4,4]
=> "song"
'The song "777-9311" is by The Time'['The Time']
=> "The Time"
'The song "777-9311" is by The Time'['The Tyme']
=> nil
'The song "777-9311" is by The Time'[/"(\d{3}-?\d{4})"/]
=> "\"777-9311\""
'The song "777-9311" is by The Time'[/"(\d{7})"/]
=> nil
'The song "777-9311" is by The Time'[/"(\d{3}-?\d{4})"/,1]
=> "777-9311"
'The song "777-9311" is by The Time'[/"(\d{7})"/,1]
=> nil

Quite obviously, my brain is hanging onto a bunch of stuff about regularity, and discoverability, and languages consisting of a small number of features that interact in powerful ways. I don’t understand why [] means index sometimes but search some others. Why does 'string'[5] return a character, 'string'[2,3] return a substring (but not the same substring as 'string'[2..3]) and 'string'[regexp,1] look kinda sorta like but 'string'[regexp][1] but with maybe Monad semantics thrown in? I am confused, and that is before thinking about the fact that [] also means “call” for lambdas!

As you can see I was mightily resisting the idea that sometimes, a useful tool consists of a collection of things you need, put within easy reach. I know that I need the semantics that both my andand example and 'string'[regexp,1] provide, and I simply need to embrace the Ruby way of doing it for a while.

Recognizing that we need to let go of ourselves in order to learn and grow is difficult. Part of my job is to bring my experience to the table, to guide and grow the team using the things I have learned. But I also have to accept that there are things I still must learn. And because I am a fully formed person, to make room for a new idea I must be willing to let go of an old one, I must be willing to chuck a piece of myself overboard.

How do I know when to hold fast and when to try something new? I don’t. Sometimes when a bunch of people are doing something, and my gut tells me they’re mistaken, I override my gut and try it, I go along with what everybody else is doing.

If you want a new idea, you have to silence your inner critic. Your sense of right and wrong, of smart and stupid works by comparing new ideas to what you already know. Your sense of what would be a good fit for you works by comparing new things to who you already are. To learn and grow, you must let go of you, you must be young again, you must accept that you don’t understand and seek to understand rather than explaining why it doesn’t make any sense.

post scriptum: An interesting distinction.

Are we building Universities or Amphitheaters?

2008-04-24T10:04:00.013-04:00

Advertising-supported media gains much more from your attention than it does from your edification. This is not coincidence. It is fundamental and inherent. There is a huge economic incentive to grab your attention, and there is no economic incentive at all to give you something worth reading.

—Giles Bowkett

This explains why so many “social news aggregators” head downhill towards idiocrity so quickly that they produce small sonic booms.

For the site owners, the money is in flame wars and troll-fests. Such things quickly drive out all useful information, but they bring slavering crowds into the amphitheater to watch the Christians battle the Lions. And as far as an advertisement for little blue pills is concerned, a slavering non compos mentis is actually worth more than a single, thoughtful programmer seeking self-improvement. Such folks often block advertisements, and even if they see them they won’t buy products just because of an animated, glitzy pitch.

We know a little something about preventing trolls and encouraging thoughtful discussion on the web. But what I have observed is this: we only see lip service of these ideas in sites that are built for making money through traffic. We only see a serious attempt to maintain a site’s value and character when the site owner has an ulterior motive for maintaining a quality social experience.

For example, Hacker News and The Joel on Software Discussion Group. YCombinator profits from having quality developers pitch their ideas to YCombinator for launching. FogCreek profits from having quality developers pitch FogCreek’s software and job board service to their employers. Neither profit from doubling their readership at the expense of driving their core audience away. Both have founders with strong opinions about how to maintain a social site (here and here).

At their best, such sites resemble our rosy view of Universities of old: places of learning where people shared and debated ideas for the purpose of advancing knowledge.I say at their best. Not always, of course, but far more frequently than the traffic-motivated sites, and that’s good enough for me.

I think Giles has nailed it neatly in that paragraph. If you are interested in edification, you have to spend your time in places where people stand to benefit from your edification, not where people stand to gain from your hanging around clicking on things.

post scriptum:

This post does not say anything about whether Universities are better than Amphitheaters, or whether people who build Universities are somehow more noble than people who build Amphitheaters. I mentioned YCombinator. I have read that they give their investees t-shirts reading “Make something people want.” Statistically speaking, people want amphitheaters. The people who build them are making something people want.

This post is not about whether Amphitheaters are reprehensible in some grand, social-engineering sense. Or about the nobility of founding a University. It is simply a speculation about what sorts of factors conspire to make one site behave like an Amphitheater and another site behave like a University. And also a far smaller thing that is of little importance to anyone except you and I, namely, a suggestion as to where should we spend our time learning how to make what people want.

Good sense

2008-04-23T10:23:00.005-04:00

Good sense is the most equitably distributed of all things because no matter how much or little a person has, everyone feels so abundantly provided with good sense that he feels no desire for more than he already possesses.

—René Descartes, via David Weiss’ excellent essay, Metacognitive Miscalibration

I regularly feel like I don’t know anything. I have been working as a software developer for around 8.5 years and I feel like a green newbie who just came out of University. The internet serves as the greatest asset to my professional life but also the greatest ego bruiser I have ever known.

—Bill, A Confessional

Experience can be a handicap… At forty-two (or three!), it's easy to think you know things. You’re at incredible risk of thinking you know things when you’ve achieved some measure of success, no matter how modest. You become “unconsciously incompetent.” You don’t know, but you don’t know you don’t know.

—I’m not young enough to know everything

Dear Raganwald: How do I...

2008-04-15T12:09:00.010-04:00

Dear Raganwald:

I love the andand gem you posted, and I have been using it on all my projects. It really does help make the code easier to read because it focuses on what I am actually trying to do and not on the bookkeeping of writing extra if statements and superfluous local variables just to handle nil checking when it is really an edge case.

How can any one argue with Person.find(...).andand.name? Fabulous!!!

And when I am writing my own project, I simply laugh at the doom-and-gloom naysayers who howl about open classes leading to conflicts. andand doesn’t change any object’s existing behaviour, it extends it, in the sense that your post about Strict Liskov Equivalence talks about subclasses extending superclass behaviour and not redefining it. Cool. It’s easy to use something like andand on a project and not screw things up. It just takes a modicum of common sense rather than a slavish fear of what-we-do-not-understand.

Ok, now I have one question. I am sorry to write, but I couldn’t find the answer in the docs. Quite simply, how do I use andand to write a gem, or a library, or a Rails Plugin, or anything else that I have to share with other people?

How do I use andand to write a gem, or a library, or a Rails Plugin, or anything else that I have to share with other people?

—Stan

You see, I like andand but I don’t want to force my opinions on the people using my stuff. So for example, I have this great idea for a Ruby Gem that adds syntactic metaprogramming so that I have an answer for the Ocaml and Lisp folks when they ~~sneer at~~ question Ruby. But while I desperately want to use andand to write the gem, I am worried that perhaps the people who use my gem don’t want to install andand into all of their work? Perhaps they will be annoyed if they discover that I have extended the Object class?

You know that this is innocuous, and I know that this is innocuous, but perhaps they want to pick and choose their own way of handling nils? Perhaps—misguided though this may be—they don’t want Object altered in any way, no matter how much safer it is to extend rather than modify it?

Or maybe they like andand. A lot. And maybe it’s worth it to them to figure out what it does and decide it’s a great addition. But if every gem they install installs one, or two, or even three things like andand, at some point they will throw their hands up in the air and yell, “Enough!” Decisions about projects should always be made with a certain perspective, I think you have said that yourself in some of your essays about time management. It seems wrong to force the people who use my code to decide whether they want andand in their project when they have to decide whether to install its dependencies.

So. How do I use something like andand without forcing it downstream on the people who use my code?

Thanks in advance,

Signed,
Your biggest fan,
Stan.

p.s. I hope you can answer this, I’m sure the answer will also be interesting to people working on large teams who want to use something like andand but do not want to go through a big bureaucratic process getting the entire team to agree that it should be pervasive across the project. If you can use something like andand in one place without changing Ruby globally, that will make a lot of people happy. —Stan

"Stuffy" Dowding

2008-04-11T09:12:00.014-04:00

The other night I popped The Battle of Britain into my MacBook.

Early in the movie (where “the movie” is understood to mean “the highly sensationalized dramatic adaptation of a highly fictionalized book recounting the version of history written by the winners of WWII”)… Where was I? Oh yes, early in the movie there is a very telling scene between Harry Andrews, playing a senior civil servant, and Sir Laurence Olivier, playing Air Chief Marshall “Stuffy” Dowding.

Both are anticipating an assault on Britain by the Luftwaffe, partly to beat it into submission, partly to damage its war capabilities for strategic reasons, and most urgently to destroy Britain’s air force so that the Germans may enjoy air superiority, which would allow them to launch an invasion.

Olivier has stirred up a hornet’s nest by writing a report saying outright that the RAF is outnumbered and has too many inexperienced pilots. Andrews is pressuring him to give a more optimistic review of the situation. He explains that the report conflicts with what the Minister has told Churchill. There are hints that if Olivier does not buck up and go along with the way the rest of the chaps are running the war, he will be sacked.

In real life, Dowding was sidelined during the First World War for insisting that pilots needed rest to be effective. In WWII, he had realized that France was a lost cause, concluding that sending men and machines to their death in France would leave Britain completely undefended, although he did provide air cover for the evacuation from Dunquerque. He exercised prudence and caution religiously, as befits a commander outnumbered 4-1, and in the end his methods were vindicated by success in defending Britain.

So, an interesting semi-fictional anecdote. And?

Managing Software Development

Managers have this incredibly hard problem: they have to push (or pull, or prod, or horsewhip, or cajole, or whatever) their teams into producing the optimum performance. In most cases, they rely on the team for information about what is to be done, how it is to be done, and when it can be done. In software, this is especially true: the people doing the work have the most expertise in determining what can be done, by when.

It is almost impossible to be fired if you (a) Go along with the official plan cheerfully, (b) Are seen to work long hours, and (c) Can give plausible explanations for why people or circumstances unrelated to management are responsible for things not working out.

And then we add to the mix a very difficult social problem: The members of the team usually need the approval of the manager to keep their jobs. It is very, very difficult to have a relationship between a manager and a subordinate where the manager actively dislikes the subordinate. In almost all cases, one or the other will eventually have to go.

So most subordinates have a great deal of incentive to tell the manager what he wants to hear, whether it’s true or not. To this, you may add the fact that it’s relatively easy to explain away failure in software development after the fact, provided you are seen to have worked your ass off. “We worked around the clock, but there were too many bugs in the FizzBuzz library…”

Seriously, it is almost impossible to be fired as a programmer on a team if you (a) Go along with the official plan cheerfully, (b) Are seen to work long hours, and (c) Can give plausible explanations for why people or circumstances unrelated to management are responsible for things not working out. It is much easier to find yourself looking for work if you (a) Dispute the feasibility of the official plan, and then (b) Blame poor planning and management for the team’s failure.

So there is a problem: there is an incentive for subordinates to give falsely optimistic information to managers. And guess what? In many organizations, managers have managers too. So they take the optimistic information and make it even more optimistic when reporting to their managers. Astronomy buffs will be amused to hear that this effect has a name: Green Shift.

Information gradually changes in hue from red to orange to yellow to green on its way up the pyramid.

The Two Kinds of Managerial Behaviour

Obviously this is an oversimplification, but I am going to posit that managers exhibit two kinds of behaviour around asking the team for estimates, progress reports, feasibility checks, and any other kind of information that validates or invalidates the “official plan.”

Behaviour number one is to seek the most accurate information. The manager may still press the team to try to do difficult things. For example, the team may say that they have a lot of confidence something can be shipped by September 15th, and the manager may overrule them and insist on September 1st. Or may laugh at their optimism and put October 15th down. But either way, the manager wants to know what the team really thinks.

Behaviour number two is to manipulate the team into saying what the manager wants them to say. At KL Group, we used to call this the “Guess the date I’m think of” game. A certain manager would ask for an estimate. Then, he would reject it and ask the subordinate to think again, harder. This would go on until the subordinate gave the answer the manager wanted, and from then on the subordinate had committed to the date.

The manager, of course, could blame the subordinate for any failure: Hadn’t the subordinate volunteered the date?

This latter behaviour is quite common, and one of my epiphanies in development project management practices was realizing that there is no practice, no methodology, no tool that can fix this problem. It is a social problem that requires a social fix. For example, FogBugz has a very interesting feature called Evidence-Based Scheduling. In short, Evidence-Based Scheduling takes your estimate and combines it with reality to produce a delivery date.

For example, if Fred Fast is conservative on estimates but always delivers on time, Evidence-Based Scheduling doesn’t change his estimate. But if Sally Slow is always late, Evidence-Based Scheduling adjusts her estimate accordingly. Terrific!

But here is the question: does it solve the problem of Maligna Manager badgering her team into giving unrealistic estimates?

You might think “Yes!” You might think that if Maligna pressures her people into giving unrealistically optimistic estimates, Evidence-Based Scheduling will adjust for that and fix the estimates. That is actually true if Maligna’s own manager uses Evidence-Based Scheduling and doesn’t tell her.

If you are the CIO of BigCo, you can work on igniting positive change in your culture, incentivising your management structure to increase visibility into their process, perhaps leveraging the competencies of a strategic value-added think tank. In other words, hire consultants.

But Maligna is fully aware that the company is using Evidence-Based Scheduling, and what actually happens is this: When Maligna wants things done on September 1st, she badgers Fred into giving September 1st as his date to be done, and she badgers Sally into giving August 1st as her date to be done. Maligna doesn’t view Evidence-Based Scheduling as a system for finding out when things will actually be done: Maligna views Evidence-Based Scheduling as a system for finding out what estimates Fred and Sally need to give her to get the result she wants.

Evidence-Based Scheduling is like a formula with an unknown team estimate on the left side and an unknown delivery date on the right side. One kind of manager finds the team’s estimate, plugs it into the left side, and out pops the most likely delivery date. The other, Maligna, plugs her desired delivery date into the right side and out pops the estimates she demands from the team.

To fix this, you have to change Maligna’s motivation. There is no tweak to the algorithm that can prevent her from gaming it to get the result she believes is in her best interests.

You Own Your Own Estimates

Well, if you are the CIO of BigCo, you can work on igniting positive change in your culture, incentivising your management structure to increase visibility into their process, perhaps leveraging the competencies of a strategic value-added think tank. In other words, hire consultants to tell you what a strategic thinker you are.

The rest of us are a little like Stuffy Dowding. It’s our job to stand our ground and just say no when people pressure us to pretend we can warp the laws of space and time. I wish it were more complex than that. I would like to write what most people would like to hear about selling your boss. I would like to write an article about the three sure-fire ways to manage your manager into making realistic plans.

But reality goes a little like this: If you “play the game,” you will obtain personal reward even though some or many of your projects will suffer. If you stand your ground and politely but firmly insist on telling what you believe to be true about what will be done by when, your career will go a little like Dowding’s: you will suffer some setbacks, such as being sidelined during World War I. And you will, from time to time, save the country, as he did in World War II.

It’s really up to you. My old (and terrific) manager Steve Rosenberg used to say “We own the compiler,” meaning that no matter what product management demanded, we had the final say on what code was written. And so it is with estimates. No matter how it is couched, you own your own estimates. Your manager can write any plan, but you and you alone decide whether you agree or disagree. You and you alone answer questions like “Do you think you can accomplish X by Y?”

Managers will try every trick. One President I recall used to come up with all sorts of scenarios, “But what if this turns out to be easy? What if there aren’t that many bugs?” And I used to say, “Then we will have good news for you in a few months. But today, this is my opinion of how things will go based on my experience and what we know to be true right now.”

No matter what, I would not allow him to get me to change my estimate. He had, of course, the final say on the plan. As he should. And it was my job to execute the plan as best I could, which meant getting the team of 23 or so people to execute the plan as best they could. That’s the job, to do your best once the decision is made. Just as Dowding’s job was to execute the plan that Churchill approved.

But nothing can force you to fabricate false optimism or to allow your manager to tell you your opinion of the situation. You own your own estimates.

Stuffy, Again

It is easy to blame management, or blame certain kinds of corporate cultures for problems with unrealistic plans. That is very nice to think about, it relives us of responsibility for what happens: “I had to go along, that’s the way things work around here.”

This was Stuffy’s problem, too. His management were pressuring him to do unrealistic things like save France, or engage the Luftwaffe head on, or to promise things the RAF couldn’t deliver. There were many things the RAF had to do to win, like use Radar effectively, and take advantage of intelligence, and yes, to out-fly the enemy in combat.

The Battle of Britain was not won in that office on that day, of course. But it could very easily have been lost there.

Raganwald goes back to U(niversity)

2008-04-09T11:24:00.002-04:00

I am delighted to announce that I will be presenting a workshop (title and subject T.B.A.) at the upcoming meshU in Toronto on May 20, 2008. I am especially flattered to have been selected to present alongside people like Daniel Burka, Avi Bryant, John Resig, Ryan Carson, and Leah Culver.

There will be just twelve workshops, roughly streamed into design, implementation, and management/inspiration/cat-herding tracks. I understand registration is just $239, a bargain for a full day of high-bandwidth, real-time learning.

See you there!

Design Matters

2008-04-09T07:48:00.008-04:00

The recent hoo-haw over whether the Do-No-Evil Empire launched an Attack of the Clones gives us a great opportunity to think about design and its value:

[Some people are] more or less arguing that it’s ridiculous to claim that 37signals somehow owns the concept of web-based group chat. Which is stupid, because no one made that argument. (Does Artie MacStrawman have a web-app cousin?)

Consider, say, Movable Type and WordPress. WordPress came along with a free (beer and freedom) package that does the same basic thing as Movable Type and took a big chunk of the market away. But I’ve never seen anyone call WordPress a rip-off or clone of Movable Type. Why? Because it isn’t. It’s an original implementation of the same basic idea.

A new implementation of the same concept is competition; a clone of an existing implementation is a rip-off.

—John Gruber

As is often the case, Gruber’s analysis of the difference between the concept and the implementation is spot-on. The distinction is important to John: he’s deeply involved in the design community, where people care very much about the value one can add by coming up with a better implementation of the same concept.

We saw a variation of this thinking when Rails was released: more than a few people claimed there was nothing new in Rails, it was “just” a framework for building CRUD applications, and a lightweight framework by the measure of how many new things you could do with it.

But the value in Rails is its design, in the way you put those CRUD applications together. Love it or hate it, Rails is an exercise in design, an exercise in user interfaces for programmers. At the time, it was a unique implementation of an existing and well-known concept.

Don’t we agree, for example, that Grails is a clone of Rails while Camping is not? One clones the implementation on a different platform, one does not.

People claim that certain ideas are “obvious” and therefore should be copied at will. I find it easy to think that something is obvious after you’ve looked at it and studied it. But the true test of “obviousness” is to go into a clean room, having never seen the original, and come up with what you imagine is an original design that does a good job of fulfilling the requirements of the concept.

If you arrive at the same design independently, you have a decent argument that the design is obvious.

What really matters

I’m going to change tacks here. If you are sitting down to make something, don’t clone an existing thing. Why? Because you can do better!

Before Rails was released, I’m quite certain that 999 out of 1,000 web developers would have told you that CRUD development was “mature” and “done.” If you wanted to create a new framework, most people would expect you to clone existing ideas.

But guess what? It wasn’t done, there was still value to be had doing the old thing in a new way. Likewise with music players and… must I make a list of 10,000 things where people thought the concept was mature and devoid of innovation, then someone found a new way to execute the existing concept, but their design was better and they reinvented the whole thing without adding new features?

You can do better. And that’s the point. Not whether the party of the first part should have done better, or whether the party of the second part should have complained about whether the party of the first part actually did better or not. I don’t care who did what, where it was hosted, when the concept was invented, and why it has generated so much debate. And in a few days, neither will you.

But what will always matter is what you and I choose to do with our very limited time while we are here. Not what they choose to do or not do with their time and their talents.

I hope you will alway choose to strive to create something new, to never take it for granted that we can’t do better with even the most allegedly mundane problems to solve. Like chat line, CRUD frameworks, design patterns, programming languages, choosing our team mates, or mobile telephones.

IS-STRICTLY-EQUIVALENT-TO-A

2008-04-07T07:54:00.012-04:00

Favor object composition over class inheritance.

—Chapter one of Design Patterns: Elements of Reusable Object-Oriented Software

Maxims like this drive me crazy. On the one hand, there in one sentence is some of the best advice I can imagine for designing Object-Oriented software. On the other hand, following this advice without any idea why one favors one over the other, without any idea when class inheritance is the right thing to do…

As Basho said, “Do not seek to follow in the footsteps of the men of old; seek what they sought.”

Perhaps of the maxim “Favor object composition over class inheritance” is what happens when we seek what they sought, when we question the fundamentals, when we ask ourselves, “What does inheritance really mean?

So: What does it mean when we go to the design white board (or pick up a 3x5 card in a design session) and boldly write:


Manager IS-AN Employee

???

Commenting on IS-A IS-A HAS-A, Robert Fisher pointed out that there is no definitive meaning to inheritance in programming languages. Another reader reminded him that Bertrand Meyer’s Object-Oriented Software Construction identifies no fewer than twelve(!) kinds of inheritance. I took that to support Robert’s point: if there really are twelve different kinds of inheritance, it is easy to see how two reasonable, intelligent people could each write “A IS-A B” and mean two completely different things.

Should the compiler always win?

There is one argument—the compiler always wins—where the only thing that matters is what a program does. A program’s code means whatever effect the program has when it runs. So when you organize your code around a result like the fastest possible performance, or the smallest possible program, and in doing so you use features like inheritance strictly because they help your technical objective and not because you think they carry any meaning.

A Little Java, a Few Patterns: The authors of The Little Schemer and The Little MLer bring their deep and important insights about programming to the Java language. This post will help explain why I never say never use inheritance.

This practice is programming for the compiler. Smeone who is programming for the compiler doesn’t even think in terms of “IS-A,”, they think directly in their language’s features—like interfaces, abstract classes, or modules.

I appreciate that point of view, but I believe that we can produce even better software if we decide that language features like inheritance have a semantic meaning above and beyond what the compiler does with them. Especially when we are considering projects that have multiple programmers working on them over a substantial period of time. Such projects must produce sustainable code, not just code that works today.

When we have a shared understanding of inheritance’s semantic meaning, our code more closely resembles the problem it solves. If it is just an engineering tool, a statement like Manager IS-AN Employee may have no parallel in the real world. Are managers really employees? When we declare other relationships such as HAS-MANY, we work very hard to make our code model the real world. Why would IS-AN be any different?

That being said, there seem to be as many opinions of what class inheritance means as there are programmers. Do any two programmers—much less a full team— agree on exactly what “Manager IS-AN Employee” means if Employee and Manager are both classes?

All I can do is share is a meaning that has worked well for me: Strict Equivalence. I’m going to describe what I mean and illustrate some of the consequences of adopting this as the standard meaning of “IS-A” on your project.

(I am not trying to say that Strict Equivalence is the only or even best way to build OO systems: I have worked on many projects with a different understanding, and they worked out just fine.)

Strict Equivalence

You know the Liskov Substitution Principle, of course. In short, if X IS-A Y, then anywhere you could use a Y, you can substitute an X and things should Just Work. So if every Manager IS-AN Employee, every piece of code that operates on Employees can operate on Managers and Just Work.

What I just said is the Liskov Substitution Principle, but there is a lot of room for discussion over what ‘just works’ means. So here is something much more specific, Strict Equivalence.

A client is a piece of code that uses an entity. Strict Equivalence is the following property: X IS-A Y if and only if it is the case that for every client that uses an object of class Y, you can substitute an object of class X and the results from the perspective of the client are indistinguishable.

Note that clients expecting an X can presumably distinguish an X from a Y, but that’s because they know about the differences between X and Y. Note also, and with emphasis, that we are talking about classes. Specifically, concrete classes. If X and Y are Java interfaces or C++ abstract base classes, there is no difference between Strict Equivalence and Liskov Substitutability, which leads to a completely different maxim, “program to interfaces and not implementations,” or my preferred interpretation—“Favour exposing interfaces over exposing implementations.”

This is not the same as Liskov Substitutability. With Liskov Substitutability, X IS-A Y if and only if it is the case that for every client that uses an object of class Y, you can substitute an object of class X and the results will be semantically valid although they may differ from the perspective of the client. Again, we are talking about a case where the client is using a Y, not using an interface. For example:


public class OurClient {
  public void consumeY(final Y someY) {
    ...
  }
}

This code is a client that uses and depends on objects of class Y, not an interface that Y implements.

Strict Equivalence is a specialization of Liskov Substitutability

One of the nice properties of Strict Equivalence is that it also guarantees semantic validity: It guarantees that you can substitute an X for a Y and since the results are indistinguishable to a client expecting a Y, the results will be semantically valid as well. Therefore, Strict Equivalence is a specialization of Liskov Substitutability.

So if you like Liskov Substitutability, you ought to also like Strict Equivalence. But it is still a specialization, and a very restricted specialization at that. Let’s see how.

Consider something very basic. Overriding a method. Isn’t that what you’re supposed to do? You have a method in a superclass, but you need a different, more specialized behaviour in the subclass. This is OOP 101. And it’s usually wrong. Okay, wrong is a strong word. It’s usually a violation of Strict Equivalence. Let’s have a look at why.

Take our Employees and Managers, where every Manager IS-AN Employee. We want to have a calculate_annual_bonus method for employees. You can imagine writing it, blah-blah personal goals set in performance review, blah-blah overall division profit.

Now what happens with managers? Perhaps the business rule is that managers get 10% of the bonus of each of their reports, like an MLM plan. So you write something like:


class Employee
  def calculate_annual_bonus
    personal_goals.select(&:achieved).map(&:incentive).inject(&:+) +
      self.division.declared_profit_share_amount
  end
end

class Manager < Employee
  def calculate_annual_bonus
    reports.map(&:calculate_annual_bonus).inject(&:+) * 0.1
  end
end

(You can easily imagine the Java equivalent, starting with public class Manager extends Employee…)

With Liskov Substitutability, everything is fine: you have some code that transfers a bonus to an employee’s bank account. It asks the employee object to calculate the annual bonus, and it transfers that amount. Nice. But Strict Equivalence prohibits this simple and common construction.

The reason this is prohibited is that if we allow a subclass to override the bonus calculation, we have effectively declared that the amount of annual bonus is arbitrary for employees. Our code says that the correct calculation is personal_goals.select(&:achieved).map(&:incentive).inject(&:+) + self.division.declared_profit_share_amount, but in reality when we have an employee object, that may not be true.

Overriding the annual bonus calculation breaks the declared behaviour of the Employee class. It is no different than if some code in your Manager class reached out like a ninja and redefined the calculate_annual_bonus method in the employee class. From the perspective of a client using an employee object, how is that different?

With Strict Equivalence, you may only override the annual bonus calculation if client code like pay_annual_bonuses cannot tell the difference. So when I am looking at the Employee class, I can’t tell whether someone has written a Manager class. But I do have confidence that if they do write a Manager class, they are not to override the calculate_annual_bonus calculation in a way that breaks my code: whatever they do, it must be indistinguishable to me.

But… but… but…

Strict Equivalence seems at first to violate the whole point of polymorphism. Why should a client know or care what a method does? Why should it rely on a what an Employee defines as the correct calculation for an annual bonus?

One argument in favour of overriding is to talk about the separation of interface and implementation. Orthodox OO teaches that a class’s interface is its collection of method signatures, and its implementation is what those methods happen to do. The notion is that clients should program to the interface only: the implementation is the class’s private business.

This practice is reified in C++, where the public interface of a class is found in a separate header file from the implementation. Sounds good. Now what happens with Java? In Java, a class’s interface and its implementation are in the same file. But if you want to give someone just the collection of method signatures, Java provides interfaces. So if you as the author wish to have clients program strictly to method signatures, you can give them an interface.

The point here is that Strict Equivalence is not in conflict with orthodox interface-oriented design: they can live happily side-by-side if you simply use interfaces when you mean interfaces, abstract methods when you mean abstract methods, and if you think twice before providing a concrete method in a class your provide to clients.

The difference comes up when you ask a client to use a class and provide an actual working method. In our example, we deliberately put behaviour in the Employee class, even though we have alternatives. In Ruby, we could borrow an idiom from Smalltalk and write:


def calculate_annual_bonus
  raise 'implemented by subclass'
end

In Java, we could declare it to be an abstract method:


abstract class Employee {
  abstract public Money calculateAnnualBonus();
  protected final Money baseAnnualBonus() {
    // helper calculation
  }
}

public class Contributor extends Employee
  public Money calculateAnnualBonus() {
    return baseAnnualBonus();
  }
}

public class Manager extends Employee
  public Money calculateAnnualBonus() {
    return ...
  }
}

Is that so hard? This conforms perfectly to Strict Equivalence. The Employee class guarantees that there is a calculateAnnualBonus method, but it doesn’t guarantee what it will be. There is a protected helper method so that each subclass doesn’t have to repeat the calculation. And you can see that both the Contributor and Manager classes guarantee the exact calculation.

We chose to provide clients with an Employee class that defined the exact calculation for the annual bonus. Given that we could have written our code another way, why shouldn’t a client assume that we deliberately wanted them to depend on the calculation? Why shouldn’t a client infer that the annual bonus calculation is part of the definition of what it means to be an employee?

Theories X, Y, and Law Enforcement

In Management Science, a Theory X manager believes “workers need to be closely supervised and comprehensive systems of controls developed.” A Theory Y manager “will try to remove the barriers that prevent workers from fully actualizing themselves.”

Some languages provide tools for enforcing some of the semantics of Liskov Equivalence. I personally have mixed feelings about this. I actually like this idea of the compiler noticing that I am trying to call the non-existent annual_bonus method of an Employee object. And enforcing Strict Equivalence would be terrific.

But what I don’t like is actually not the language’s fault. I don’t like it when I have to deal with a code base where people assume that as long as they are following the language’s rules, the result must be well-designed OO software. So net-net, I like languages providing mechanisms for enforcing obvious things about inheritance. But philosophically, I want to seek what the language designers sought and design to the principle of the law, not the letter of the law.

Let’s talk about Java first. Java provides two tools—the abstract and final keywords—for partially enforcing Strict Equivalence. As shown above, an abstract method lets us declare that all Employees have a calculate_annual_bonus method without making any guarantee as to what it will return. A final method lets us declare that its calculation is invariant. If we write:


class Employee {
  final public Money calculateAnnualBonus() {
    // calculation...
  }
}

Then we know that all Employees have the same calculation. It is illegal to implement a different calculation for the Manager subclass. And by extension, if we fail to mark a method final, we know that its implementation is not guaranteed. A non-final method in a non-final Java class is unreliable, it’s just like functionality without a test in a test-infected program. It is not part of a class’ contract with its clients.

I conjecture that this is the basis of the suggestion that methods should be final by default. If a class is not final and it has a non-final method, you can override it and break Strict Equivalence. So with Java, if you want a method’s behaviour to be part of a class’ contract with its clients, you have to make it final.

There is a shortcoming to this system. It is possible to override a method without breaking Strict Equivalence.

The limitation of using the final keyword (or whatever tool your compiler gives you to prevent overriding) is that it prohibits a very large class of specializations that are useful and do not violate Strict Equivalence. It is as if the Police Chief decided to fight crime by imposing a curfew on the entire town.

It is not that all ways of overriding a method break Strict Equivalence. Let’s go back to our employees for a moment:


class Employee
  attr_accessor :manager

  def something_serious
    # ... just do it
  end
end

class ProbationaryEmployee < Employee
  def something_serious
    returning(super) do
        self.manager.notify(self, 'did something serious')
    end
  end
end

This is a little contrived, but the point here is that although a ProbationaryEmployee overrides the something_serious method, it does so in a way that is presumably indistinguishable from the perspective of client code. It does something serious and returns the same result, but there is an extra side-effect that matters to probationary employees and their managers.

This conforms to Strict Equivalence because to a client method, the results are identical. A ProbationaryEmployee will pass the same tests and you can insert assertions as you see fit.

The only problem is, how do we know that ProbationaryEmployee adheres to Strict Equivalence and Manager does not? How can programmers write code like something_serious? Just writing any old method, calling super, and hoping that you haven’t broken the Employee class is rather like tossing salt over your shoulder to ward off evil spirits. And never overriding Employee methods is rather drastic. What to do?

When the going gets tough, the tough redecorate

Solving problems like this is a lot of the motivation behind Aspect-Oriented Programming systems like Lisp’s Flavors & CLOS, the Decorator Pattern, and method chaining in Rails.

The principle is that a subclass doesn’t really override a superclass’ method, it extends its functionality. Meaning, it typically does something before it, something after it, or something around it. It decorates the method, it doesn’t replace it.

In the example above, it is not really obvious that we are decorating the Employee#something_serious method. That happens to be a side-effect of what we are doing, but it is not obvious at all. Compare and contrast with a little easy AOP magic based on Rails’ object life-cycle style of coding:


class ProbationaryEmployee < Employee
  include EasyAOP

  after :something_serious do
    self.manager.notify(self, 'did something serious') 
  end
end

(No, there is no “EasyAOP” module. If you are interested in adding AOP to Ruby, consider using the Aquarium gem).

This style of coding makes it obvious that we are attempting to maintain Strict Equivalence. Of course, we can abuse AOP and write advice that breaks our equivalence. 100% enforcement of Strict Equivalence while permitting things like AOP is probably impossible in the compiler. But we can make languages that encourage good programming. I honestly believe that if I have a powerful way to write advice—like CLOS or AOP—I could live with never overriding methods. Forget methods being final by default, I think I would be happy with methods always being final if I can have subclasses decorate them.

The Art of the Metaobject Protocol is one of the most important books ever written on the subject of building abstractions with objects. The Metaobject Protocol is a system for defining your own object semantics, such as the simple aspects described here. While you’re learning how to make your programs better in your language, you just might pick up a little Common Lisp. Highly recommended.

Ruby, of course, does not enforce anything. Java provides some enforcement with its final keyword, but at a cost: subclasses cannot decorate final methods. On the whole, Java is not worse than Ruby for enforcement, because if you don’t declare a method final, you are no worse off than in Ruby. In Ruby’s favour, you can actually write things like EasyAOP in a few minutes. Better still, you can tweak it to suit your project and its domain. For Java, you probably need to use a heavyweight AOP framework to do the same thing.¹

Or maybe you can chuck all of that and use CLOS. You could do a lot worse.

A Promotion: Manager WAS-AN Employee

Wow, that’s a lot of flummery over the statement “Manager IS-AN Employee.” As we have seen, there are a lot of guarantees that statement makes, especially if you adopt Strict Equivalence. You may not want to make those guarantees. You may not really mean that a manager is an employee, you may mean that a manager has a lot in common with an employee. So what to do?

Well, all this trouble started because we said that a Manager IS-AN Employee. What if it isn’t? What we want is a relationship often called WAS-A. Or it’s technical name, implementation inheritance. We want to say that a Manager uses the code of an Employee, but it isn’t to be used where you expect an employee.

In C++, you can use private inheritance. The Manager class would obtain all of an Employee’s members and methods, and it could expose the ones it wanted as part of its own interface:


class Manager : private Employee { ... }

A Manager is not an employee. It is its own thing that happens to behave a lot like an employee. In languages like Java that do not provide a WAS-A construct, you can achieve the same thing with delegation. You can say a Manager HAS-AN Employee. It doesn’t really have one in the sense of a manager having an executive assistant, but it uses an Employee object internally to handle some of its work.

This is very much like a manager who has two ‘hats.’ It works well if you have been careful to separate things like the implementation of database persistence from the semantic hierarchy of public inheritance. In other words, you didn’t decide that an Employee IS-AN ActiveRecord::Base.

In other words… If a Manager is not an employee in the true sense of the word but is simply like an employee, we can use object composition to achieve code re-use and communicate to our fellow programmers what managers and employees have in common, without muddying the meaning of IS-A.

What they sought

So what have we seen? First, that if you adopt Strict Equivalence, standard inheritance carries a great deal of meaning with it. You can infer that the code in a class describes the behaviour of that class and all of its subclasses: The Employee class describes all employees, even probationary employees and managers. Subclasses extend functionality but never break it.

Second, that there are many alternatives to breaking Strict Equivalence, including refactoring the class hierarchy when you need polymorphism and using object composition when you do not. This means that we never need to use inheritance, we only use it when we choose to make guarantees about equivalence.

In Programming conventions as signals, I advanced the view that when there was more than one way to do something, each should carry a different meaning. And furthermore, when one way is the obvious way to do it, using the non-obvious way was a signal that there was something non-obvious going on.

This obviously applies to choosing between class inheritance and object composition. Both can be used to express the idea that two classes or objects have something in common. Class inheritance carries heavyweight baggage along with it about the semantics of IS-A. Object composition does not.

Therefore, class inheritance is the more “expensive” of the two to maintain, because it has the most serious implications and constraints on code. Object composition is cheaper because it makes no guarantees, it is a convenience mechanism.

If class inheritance required a pattern and some boilerplate to establish, while object composition was as simple as writing:


delegate :calculate_annual_bonus, :something_serious, :to => @employee_helper

…then there would have been no need for the maxim. Unluckily for us, the popular OO languages make class inheritance cheap to write in code even though it is expensive to maintain, while object composition is expensive to write in code while being cheap to maintain.

Thus, we need a maxim to remind ourselves that while class inheritance is seductively easy to write, it should be reserved for those cases where it expresses our actual semantic meaning and where imposing its implications and constraints on our code is going to make things cheaper and not more expensive to maintain.

Favor object composition over class inheritance. Now we understand a little of where those six words are heading.

Digression: Test-Driven Equivalence

Consider the requirement that a user of the code be unable to distinguish a subclass from a superclass in a Test-Infected project. You have dozens or even hundreds of tests for each major class. What about all of those unit tests. Should a unit test for the Employee class work when given Manager objects?

Hmmm. If a Manager IS-AN Employee, it follows that a Manager can be substituted for an Employee and the tests will pass. If an Employee test breaks when given a manager, we must pick one of two possibilities: First, it is possible that we were wrong to state that a Manager IS-AN Employee. We should have chosen a different structure for our classes.

In a test-infected project, the meaning of “Manager IS-AN Employee” is external to the Employee class, it is in the tests.

The second possibility is that the test is inappropriate. We might say that the bonus calculation for an employee is uncertain, that the exact amount of the bonus is not part of the ‘spec’ for the Employee class. Therefore, we shouldn’t have a test that validates the amount of an employee’s bonus. I can address this point easily: if we can’t test it, it doesn’t work, therefore if there shouldn’t be a test that works for every employee, then there shouldn’t be code associated with every employee.

Object Design: Roles, Responsibilities, and Collaborations focuses on the practice of designing objects as integral members of a community where each object has specific roles and responsibilities. The authors present the latest practices and techniques of Responsibility-Driven Design and show how you can apply them as you develop modern object-based applications.

So above we stated that according to Strict Equivalence, the code in the Employee class makes a guarantee about all employees. However in a test-infected project, we can take another line. When I say test-infected, I mean specifically that the team has made a commitment to 100% functional coverage in the project, and the team considers the tests to be a form of executable documentation for the code.

If this is the case for a project, it follows that the tests for a class must reflect the contractual obligations of that class to its clients. If we write a test that only passes when an Employee’s annual bonus is such-and-such an amount, that is the amount of bonus for all employees. If we want subclasses of Employee to have their own bonus calculations, we are free to provide a default or base implementation, but we are prohibited from writing a unit test for it.

You may think “That’s ok, my unit test directly creates an Employee object, it doesn’t create a Manager object, so the test is not making a guarantee about Managers.” If you take that line, you have removed all possibility of using the tests to make guarantees about Employees. You cannot say anything about objects that are Employees, because your tests don’t apply to all employees. This effectively eliminates the IS-A relationship in your test-infected project, because when I am looking at a Manager class, the fact that it derives from Employee tells me nothing, because I can’t trust any of the Employee tests to tell me something about Managers.

Therefore, if you believe that an Employee test does not constrain the behaviour of a Manager object, you are required to duplicate all of the Employee tests that apply to Manager instances in Manager tests. You have moved the problem of what is inherited and what is not inherited from the Employee and Manager classes into your test classes.

In a test-infected project, the meaning of “Manager IS-AN Employee” is external to the Employee class, it is in the tests. It also conforms to the Liskov Substitution Principle: it demands that whatever the superclass provides its clients, all of its subclasses must also provide. However, it is more relaxed than Strict Equivalence.

There are other approaches: Eiffel’s Design by Contract implementation provides preconditions and post-conditions for methods. As long as a subclass conforms to or narrows its superclass’ post-conditions, and conforms to or widens its superclass’ preconditions, it may override a method.

Design by Contract takes the view that the pre-conditions and post-conditions are the contract for a method. Given that understanding of Liskov Equivalence, the language is provides both compiler and run-time enforcement options. Code that adds notification or logging to a superclass’ methods obviously conforms to its post-conditions, so Eiffel permits it without arguing about whether the method should be final or not.

Design by Contract also lies in between Plain and Strict Equivalence: Although the Employee class contains code, you cannot rely on it as a guide to what an Employee does. In Eiffel, you can rely on the post-conditions to be enforced, which are typically looser than the method implementation but tighter than a method signature without conditions.

[back]

javascript at reddit.com

2008-04-06T09:34:00.003-04:00

javascript.reddit.com

It’s pretty quiet at the moment, which means it’s a blank canvas. Go wild, post your favourite Javascript articles, up- and down-mod the articles already posted.

The great thing about a new site is that you can have such a huge influence over its direction. Once there are thousands of people following it, your vote will be a drop in the ocean. But right now, you can set the direction, a direction that will establish its character forever.

So what are you waiting for? Off you go!

What is a Specialist's Skill Set?

2008-04-03T19:29:00.005-04:00

My son Thomas had his tonsils and adenoids out today. Huge thanks to his Surgeon, Dr. Chiodo, and the pediatric staff at Toronto Eastern General Hospital. He’s recovering pretty much as can be expected, thank you very much for asking.

One thing I noticed about the process. It all started with a referral from a General Practitioner, in this case our Family Physician Dr. Kohut. She analyzed the situation and determined that an opinion from a specialist was appropriate. This being socialized medicine, you couldn’t pull out our wallet and go straight to the specialist, we had to have our GP do a little qualifying first.

But the interesting thing is what happened next. We went to see Dr. Chiodo, he examined Thomas, and after discussing our options and the risks associated with the procedure with us, he booked Thomas for surgery.

Then today, Dr. Chiodo removed his tonsils and adenoids. This is standard procedure, and I want to mention it to compare it to programming, of course. You notice that although Dr. Chiodo is a very busy specialist, he does his own diagnoses. There is no division of labour where one doctor determines that the tonsils need to come out and the first time the surgeon sees the patient is when the anesthetist signals it’s ok to proceed.

The person performing the procedure is the most qualified to diagnose the patient and also—mark my words carefully—the person who discusses the options and risks with the patient. This is sometimes the case in programming and sometimes not.

Right now, I don’t feel like talking about what kinds of organizations divide things up so that one person sells the service to the client, another writes down the requirements, a third person estimates when it will be done and manages the process, a fourth person turns the requirements into a design and a specification, and finally the hapless programmer, the “specialist,” performs the procedure without any meaningful influence over the diagnosis or the decision-making process. Let’s skip that rant.

Let’s also skip a rant about how many programmers have the skills to diagnose the patient, by which I mean, the business acumen to discuss what a client actually needs to accomplish in business terms. Let’s not argue about how many programmers have the communication skills to present the options to a client in such a way as to guide a client to making a smart decision while still ensuring that the programmer’s business profits. (This is relevant even when programmer and client are employed by the same organization).

Let’s talk about you. Do you have the specialist’s skill set? Can you diagnose, inform, and operate?

If not, why not?

Raganwald in disagreement

2008-04-01T12:19:00.007-04:00

Relax, this is not one of the 17,322 April Fool’s posts to arrive in your feed reader today. Quite seriously, I just discovered that I am inconsistent, and that is very interesting in a Raymond Smullyan kind of way.

Here’s the thing: Many of my posts extol the virtues of programmers to producing programs. Not everyone agrees, of course, many people claim what you need are these “engineering principles” and even complete idiots will magically produce working software with terrific ROI over the complete software lifetime. Naturally, when these principles are elucidated, they are quickly diminished, because if you can name one such principle, you can also find someone who tried it and failed.

Of course, the same goes for smart people: it’s easy to hand wave and say, “You need smart people who can get things done.” But that’s begging the question: looking at a successful project and declaring that the people were smart and did get things done is easy. But coming up with an unambiguous and objective measure for deciding who is smart and who will get things done is hard.

Any blogger can rant (as I rant) about smart people, but the easy reply is to ask for a repeatable process for selecting such people. What’s on their résumé? What questions do you ask in the interview? How many references do you check? What do you look for in their portfolio? What do you expect to see in their code samples?

Not so easy, is it?

Any ways, let’s accept that I rant about programmers being important without having any sort of objective, reproducible process for selecting such programmers. A scientist would laugh at me, but then again a scientist would laugh at everyone mumbling about software project management and the ROI from using Java instead of Lisp—such claims are equally unproven.

However, let’s be charitable and chalk it up to optimism. Fine.

But what’s with my posts extolling the virtues of programming languages and suggesting they are important? If a better language (whatever that means) can make a good programmer better, why can’t it make a mediocre programmer acceptably average?

And if good programmers are “All that and a fresh bag of chips,” why can’t they do just fine with Java or Visual Basic? Why do they need these so-called better languages?

Well? What’s the deal? Is it the programmer? Or is it the tool?

The only workable system for generating interesting, cool, and relevant software

2008-03-29T11:03:00.004-04:00

The only workable system for generating interesting, cool, and relevant software is well-known. Find a bunch of really smart programmers and point them at a large problem space for which there are actual users, and for which new solutions are unconstrained by old designs.

—Mencius Moldbug, What’s Wrong with CS Research

Note the desired outcome: Interesting, cool, and relevant software. Do not become mired in empty debate over the phrase “smart programmers.” Instead, ponder the other three preconditions: a large problem space, actual users and most especially unconstrained by old designs.

Alan Kay is purported to have said, “A fresh perspective is worth 80 IQ points.” You cannot have a fresh perspective when you are encumbered by backwards compatibility.