(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Wednesday, July 04, 2007
  Certification? Bring It On!

Not too far in the distant past, I was persuaded to give my résumé to a recruiter. He was trying to place a development manager for a growing company, and they wanted heavy Agile experience, deep management experience, and someone with some technical chops. Well, I figured that twenty-plus year of experience, with something like eight years of legitimate Agile, leading a team of twenty-plus, producing a product that won several Jolt awards and a JDJ Editor’s Choice award… I thought I was a lock for an interview.

But an email came right back: The client is wondering where you got your degree. Twenty years of experience, and they want to know how I spent my time in the mid-eighties.

This got me thinking about certification. It’s another long-running debate. And something funny has happened to me. I’ve switched sides. I’m actually in favour of certifying software developers. Yes, I am in favour of disqualifying intelligent programmers from professional employment if they do not possess a little piece of paper from a certifying agency.

Deep breath. Wait for the room to stop spinning. Or is my head spinning around on my shoulders (heh)?

the catch

Like everyone else in favour of certification, I have my own ides about what skills and knowledge you need to demonstrate to get your certification. Unlike everyone else, I think I would fail my own certification if I didn’t do a whole lot of studying. That’s because I think our industry is undemanding, very undemanding. I know a few people who would pass my certification without studying. But only a few.

Before I tell you what’s on the final examination, let’s talk about what isn’t:

By now, you are thinking, “Raganwald, this certification is worthless. You are excluding just about everything we know about writing great software. What’s the point?”

Let me explain. My certification does not say you are any good at coding software. Let me repeat. My certification does not say that you are any good at coding software. I’ll let the marketplace decide. I am not telling businesses, “Hire certified programmers, they are great coders.”

Why should I? Business is perfectly happy to hire programmers with Comp. Sci. degrees, and there seems to be little or no evidence that a Comp. Sci. degree says anything about your ability do deliver working software. So why should my certification raise that bar?

what’s on the exam

Now let’s talk about what is on the exam. Just one subject, but the exam goes into excruciating detail about that subject. If you don’t know this subject cold, I am not going to certify you. Period. No debate, no negotiation, no “equivalent experience.”

The one subject? Testing and Quality Control. That’s right. All I care about is that if you are asked to make bulletproof software, you know how. I’m going to ask you about:

And more:

And a whole lot more on top of that. There is room for debate about whether to have separate testers or whether programmers should test themselves. You are not getting certified with me unless you know how to do it both ways, and can write a comprehensible essay describing the relative advantages and disadvantages of both. I am not going to require you to know the latest programming frameworks, but you are not getting certified unless you demonstrate up-to-date knowledge of the latest continuous integration tools.

Test Driven Development: By Example is THE book that ignited a revolution in software development practices. Whether you are developing in an Agile environment or working from a telephone book specification in a Waterfall project, Test-Driven Development will show you how to write automated tests that work to actually shorten your development time and clarify your code.

And best of all, this book uses actual projects as an example. This is not an exercise in theory, this is a practical tome full of practical advice. You’re already familiar with the concepts, reading this book will dive into the details that will make your coding more effective.

I don’t care if you know how to write a great architecture document. But I will fail you if you can’t write a good code review. The same goes for everything else. I will not demand that you do it a specific way, but you will prove that you have state-of-the art knowledge of how to ensure that software is solid and does what you expect it to do, no more, no less.

And you know what? After you get your piece of paper, you’ll need to work for at least a year under the supervision of a certified leader to get your upgraded “practitioner” certification. I want you to practise continuous education.

Does this mean that I am going to certify all of the QA Analysts in the industry while barring the programmers from work? Well, let me ask you: do most, a few, or any of the QA Analysts you know have a deep knowledge of software quality and methodologies? Can they write an essay describing the cost of bug fixing comparing early vs. late detection? Can they talk about various unit testing tools? Can they measure code coverage? Can they look at a piece of code with 93% coverage and tell whether the missing 7% has one or more crucial cases missing?

If so, I want to certify them. If not, they need to hit the books with me.

where did I get this crazy idea?

In our industry we have wasted millions of person-years debating the relationship between software development and architecture/engineering. Last weekend I went to see Pixar’s Ratatouille with my son. And it struck me: we should be like chefs.

Do I demand that the chef in a restaurant use a certain kind of stove? Cook a certain kind of food? Manage her kitchen a certain way? Non! The marketplace decides all these things. And the marketplace works for this. What do we demand? What do we require of restaurants?

Safety. We demand that if they serve the public, that they have certain fire safety standards. That they have certain food cleanliness standards. That they know enough about food not to poison us by accident. Of course, a cook learns to cook when getting their designation. But the thing we really demand of them is that they keep us safe!

If we order Chicken, we do not want the Fish to come out and put us into Anaphylactic Shock. Cooks know that mixing up Chicken and Fish is fatal, while mixing up Basil and Oregano is not. So they have a different protocol for handing the two kinds of food. They keep us safe.

And if I am placed in charge of certification one day, that’s what I will demand. Keep us safe. Don’t leave back doors and XSS vulnerabilities in your code. Don’t store our passwords in the database. Don’t deliver code that is full of undiscovered bugs.

If someone can be relied upon to write software that is safe, I will not dictate how they do it, or how long it takes to do it. The marketplace can decide whether they are employable, much as a restaurant can decide whether to hire someone whose food is bland and unappetizing.

I am not telling businesses that they can’t ship software full of bugs. You have Product Managers, they can decide whether it is more important to build new features or fix the old ones. Microsoft, do your thing! But business can’t make those decisions unless they have an accurate view of what the software actually does, of which parts are solid and which are brittle.

And I am not telling managers how to run projects. But I do expect that a certified manager understands the trade-offs when she chooses BDUF over Agile, Theory D over Theory P. She can do as she please, provided her eyes are wide open to the consequences.

Well, that’s my thought about certification. I’m all for it, don’t let anyone say otherwise. And like everyone else, I want it to reflect what my experience tells me is important about software development in the commercial environment, namely safety and a clear view of what works and what doesn’t.

This is obviously a pipe dream, the product of Dark Horse Café’s “Ruby’s Pride” French Press coffee. But what do you think? Would such a certification be useful?



Comments on “Certification? Bring It On!:
Wow, are you trying to take that 25% off Steve Yegge? :-)
The marketplace metaphor is not good when looking for prospective employees. Your certification process will result in a safe employee; but not necessarily an efficient employee.

You and I have many instances of hunger where going to a restaurant will satisfy our needs: The amount we invest in a meal is quite small; if it turns out to be swill (but technically edible) we can choose to never come back. The free market selection process only works when consumers can make many small purchases. The free market selection process may be a good macro economic principal, but it requires some consumers get screwed on the micro-economic scale.

The free market does not work for employee selection; employing a person is a big investment, so information is necessary to limit potential losses. The number of times your company can make employee choices is limited, so they had better be good.

To stretch your analogy; suppose humans only get hungry every 6 months, and a meal costs $5000. Your certification process will result in an excellent restaurant manager; food will be safe to eat, but he need not be able to cook. Now if I was to invest $5000 in my semi annual meal, I am NOT going to be happy with mealworms and ketchup with a side of burnt toast. Faced with the prospect of truly terrible food, I will have to interview the restaurant to see what food they do make, and HOW they make it; I may even risk food poisoning for truly wonderful food.

Your certification process will result in a good manager of code, but not necessarily a producer of code. Like you admitted, a QA person will pass the certification, but your company might run out money by the time the QA person finished developing code. There was a guy I worked with that could pass your certification process; but depend on him to build functionality and he could sink a company.
Your certification process will result in a good manager of code, but not necessarily a producer of code.

I am not trying to solve that problem. I agree that the marketplace does a fairly terrible job with that in some sectors, and a good job with that in others.

Do you really think that managers in BigCo are walking around wondering why they can't hire people who are actually any good at programming? I think they know darn well that their pay rates and HR process and company culture and everything else force them to enjoy the taste of citrus fruit.

If they cared a whit about a certification process that would result in good producers of code, they would force universities to turn out good programmers.

I think the free market works perfectly in the area of programmer selection. I notice that YCombinator has absolutely no problem funding young hackers that can actually write code.

I am not claiming such a certification would be the only requirement for employment. Do you think a restaurant would hire anybody with their Chef's Papers? Well, some might, but some would want to know who you've worked with, some would demand that you cook them a meal.

And so it is with certification. I suggest such a thing be considered necessary but not sufficient.
Sign me up.
Another vote for "there's more to softwae development than safety": Adequacy is inadequate.
I love it. As a baseline criteria for certification, I'd support this (while at the same time having to go back and study I fear). I like that this formulation is easy to layer upon as needs require.

On top of this - and perhaps this is obvious and thus redundant - would then come the consideration as to whether or not somone can actually write code (cook), write good code (cook well), loves writing code, works well in a team, fits with the company, and the such (cooks well enough to work in an n-star restaurant).

Once again you give me another aspect to consider when hiring developers. Much appreciated my friend.
Interesting idea, although i would be weary about just plonking everything on a final set of exam papers, then deriving a final "pass / fail" certification based on what you got right.

After all, if you end up just ticking the right boxes, all it says is that you are good at remembering things. Applying the knowledge in a real-world situation requires more. E.g. problem solving skills.
Here's an alternative suggestion: just build something cool. Nine times out of ten, the companies that care about your degree or non-degree aren't worth working for. I don't just mean in a rah-rah sense or an indie-is-always-better sense. The company that balks on that has low expectations. If you've ever worked for people who hired you with low expectations, you know, it's basically better to be unemployed. Low expectations in this context doesn't just mean a lame job; it means they want to know about your degree because they don't trust your blog or your experience. They suspect you of lying on your resume.

The thing is, although a certain amount of filtering and screening is absolutely necessary, you should be very cautious about companies which are themselves very cautious.

Literally every single time I've worked for a company which wanted to verify every single word on my resume, I've ended up cursing myself for not thinking clearly enough to verify every single word in the job description.

Honest people don't suspect everyone they meet of being a liar.
Nine times out of ten, the companies that care about your degree or non-degree aren't worth working for.

I'll generalize: nine times out of ten, companies aren't worth working for. It's like love: the odds are stacked incredibly high against you, but you only need one, so it's no big deal.

Good points, Giles, but I'm not too concerned. I've built some cool stuff, some boring stuff, I've worked with some cool people, I've worked with some boring people.

I have a gut feeling that most people don't have any trouble figuring out who I am and what I can and cannot do.

So... no problemo. The incident was valuable in that it got me thinking about something else. But I'm not laying awake worrying about it!
There is one certification that covers about half of what you are looking for: ISTQB. ISTQB however isn't really designed for programmers but full time testers, but covers vital testing strategies such as boundary value analysis, pairwise testing (in no way related to pair programming) and a testing methodology called the "V-Model".
There is one certification that covers about half of what you are looking for: ISTQB. ISTQB however isn't really designed for programmers but full time testers

My point here is that it's tough to call yourself a "professional software developer" if you aren't intimately familiar with the processes by which we evaluate the result of your work.

Imagine walking up to a Structural Engineer and talking about stress-testing materials. Wouldn't you be worried if they shrugged and said, "that's testing, I just design stuff"?
A pipe dream this may ultimately be.

But if you were interested in starting this up, I bet you could get further than you'd expect nowadays with nothing invested but some time.

Get some of those "few people who would pass the test" together with a wiki and start writing the (first) test. I wouldn't make it open to the public at large, but you would need to have enough interested parties to get some momentum and diverse viewpoints.

Certifications are ultimately nothing more than some group of people saying "You meet this standard"; it doesn't intrinsically have to involve money to start with.

And even if nothing "happened", you might produce some interesting writing artifacts to share, so the worst case scenario still isn't "total waste of time".

This could be interesting.
Your view of certification is right on. My personal experience only supports it.
It would be good to have such a certification process in place.
Maybe in 10 years from now, we will.
Hahahahahahaha. Thanks Reg. Penetrating insights, like this one, always make me laugh with joy. Pursuing how to improve the quality of software leads you through all of the aspects of software, and the basics of what you learn are testable. Hmmm, one might speculate that you are a reincarnation of Archimedes, for giving us a place to stand with respect to software certification, and Panini for your love and use of metarules, transformations, and recursion in your approach.

I would support this type of certification, but suggest that you consider these aspects as well:

1) Formal and informal(Dijkstra) proofs of correctness.
2) The Demming notion of building quality in, such as using Poke Yoke, TDD, and other techniques.
3) Different types of testing needed for the different axes: problem domain, problem, solution and implementation.
4) The cost and benefits of quality, see The ROI from Software Quality.
nine times out of ten, companies aren't worth working for

at least!
It seems to me that some developers are singular specialists (excel at usually only one aspect) and some are generalists (jack-of-all-trades knowledge), although a smaller percentage of them happen to be both (capable of just about everything but specialize in something particular).

I believe those that blog about stuff like this post are more this way. I have seen many that specialize in certain aspects (like top-notch coding) but can't interact with a user to save their lives (obviously the top-notch coding and interacting with a user are interchangeable).

I'd be happy to see more get a solid background for all aspects of the development INCLUDING testing, and from there developers should aim at improving skills in whatever area they so choose.

This at least would be a baseline in my mind since so many don't seem to have it.

I see so many that have no knowledge whatsoever of testing. Unfortunately, at almost every company I have worked at they have balked at any in-depth testing procedures.

Towards that end, do you (or anyone )have any recommendations on where one can actually take some courses (or something other than just reading some books) to improve this skill set? I can get the books, but my work would rather I take a course... I think Jean-Paul Boodhoo might cover this at least in some way that I'd benefit from. Ironically, I found him around a month after he was in my area (Richmond, VA)...
I really wish we had a "profession".

Seems that you see adverts for "no previous knowledge necessary", "study while in your existing job" adverts in the media. They emphasise how short and easy the course is and how much money you can make from qualifying.

I find this very hard to accept having done 2 computer science degrees. I'm left wondering what's the point? This sort of thing is proof we don't have a profession. Those "fast-track to a lucrative career" adverts don't seem to offer courses in medicine, law or structural engineering....

Seems like just about anybody that can give a confident, convincing account of themselves to a non-technical hiring manager can 'blag' their way in to our business.

Another trend I've noticed is that pay seems to bear absolutely no relation to results, ability or qualification. The best developer I know is also one of the least well paid, whilst another relatively young developer I know earns more than one of our local consultant neurosurgeons specialising in paediatrics.

... I don't think the medical or legal profession would use a non-technical hiring manager ;-)
But an email came right back: The client is wondering where you got your degree. Twenty years of experience, and they want to know how I spent my time in the mid-eighties.

I see where you're coming from, but wanting to know someone's educational pedigree is a standard background question as far as I'm concerned. Just like it's helpful to know where my (medical) doctor got his degree. Even if he has 20 years of experience.

It's not a deal breaker but it's a key and important part of someone's background, and thus fair game.

I always ask about formal education in interviews, and I certainly take it into account when deciding whether to interview people with very little experience.

For this particular situation, it was a deal breaker, just as a medical degree is a deal breaker for a Doctor.

No degree == no interview with that firm. And no, they were not interested in what I was doing instead, it was a simple boolean check box. You can decide for yourself whether this is a good idea, I personally think it's perfectly ok to use this kind of filter, even if I use a different set of criteria to evaluate candidates.

This is exactly what got me thinking about certification, I was asking myself, "what is there that would serve as an absolute deal breaker if I was in charge and could impose anything I wanted on the industry?"

Notice that just because that's how I would define certification doesn't mean that's how I evaluate candidates today.
This is like the Fireman Chit and Whittlin' Chit in Boy Scouts. It doesn't say you actually know how to build a fire or use a knife, it just says that you know the principles behind to _not_ burning down the forest and _not_ cutting your fingers off.

<< Home
Reg Braithwaite

Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /