Wednesday, February 22, 2006
Better Testing Presentation
Here's a brief presentation I gave for the Toronto chapter of the Software Process Improvement Network ("SPIN").
Download "Better Testing" (584Kb, PowerPoint).
It emphasizes principles rather than giving specific tips, how-tos, or examples. This was written for the cerebral QA or development manager who is seeking some insight into why some things work well and others don't, rather than a prescription for an inexperienced manager looking for specific direction.
I started developing the presentation with a completely different plan for what it would contain and communicate. I really wanted to talk about burn-down charts and talk about various metrics I have personally found to be useful.
I imagined a kind of diagnosis model where the presentation would list specific project problems and talk about what you can empirically measure to confirm the problem. However, as I wrote out explanations for each metric, I noticed a problem.
Every time I'd explain a metric, I found myself writing out various caveats and disclaimers. The trouble is that metrics can be completely misused, causing teams to chase the numbers rather than develop software. So with each metric there are good things you can discover with measurement, but there are also pathological problems you can experience if you focus on the metric to the exclusion of the underlying principle.
After a while, I realized that this problem seemed more interesting to me than the metrics themselves. This is a cliché for programmers. We start developing a to-do list application and end up with an entire web development framework because we find the underlying common problem more interesting than the application.
So, this presentation is really about what makes some tests better than others, rather than specific tests. I'd appreciate any and all feedback, especially suggestions for improvement.
Download "Better Testing" (584Kb, PowerPoint).
It emphasizes principles rather than giving specific tips, how-tos, or examples. This was written for the cerebral QA or development manager who is seeking some insight into why some things work well and others don't, rather than a prescription for an inexperienced manager looking for specific direction.
I started developing the presentation with a completely different plan for what it would contain and communicate. I really wanted to talk about burn-down charts and talk about various metrics I have personally found to be useful.
I imagined a kind of diagnosis model where the presentation would list specific project problems and talk about what you can empirically measure to confirm the problem. However, as I wrote out explanations for each metric, I noticed a problem.
Every time I'd explain a metric, I found myself writing out various caveats and disclaimers. The trouble is that metrics can be completely misused, causing teams to chase the numbers rather than develop software. So with each metric there are good things you can discover with measurement, but there are also pathological problems you can experience if you focus on the metric to the exclusion of the underlying principle.
After a while, I realized that this problem seemed more interesting to me than the metrics themselves. This is a cliché for programmers. We start developing a to-do list application and end up with an entire web development framework because we find the underlying common problem more interesting than the application.
So, this presentation is really about what makes some tests better than others, rather than specific tests. I'd appreciate any and all feedback, especially suggestions for improvement.
Labels: agile
