(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)
Tuesday, January 15, 2008
Informed Choice
I found some folks arguing about whether developers should write messy code. Having observed our amazing midwives “deliver” for the second time, I believe it is often an issue of Informed Choice.
Presuming that the client or stake holder bears the entire “cost” of the decision, they deserve the last word. I know more than a few people who argue with this, saying that writing clean code is in the client’s best interests. Well yes, of course it is. And in my experience, so are a number of other practices such as managing a project using Theory P, using small teams, and customizing a design for the specific project. But let’s make this really simple: the client is asking for something which—in your professional opinion—is not in their best interests but also is not so ruinously bad that the entire project will absolutely, positively fail.
There can be a set of options that are all somewhat reasonable, with some better or much better than others, but none being outright wrong. In midwifery, there is a choice between giving birth in the hospital or at home. There is a choice between having an epidural or going natural. There are choices about what to administer to the newborn as a prophylactic against infection. The mother makes an informed decision on each of these subjects with the assistance of the midwife, and the midwife supports here even if it is a decision that the midwife feels is statistically less than optimum.
In software development, a client may wish to design everything up front. I believe with all of my heart this is not the best way to go, but let me tell you: I have managed projects on this basis and succeeded. I think I have done a better job managing projects another way, but I can’t tell you that a project absolutely has to fail if you start this way. Likewise large teams deliver software, and some of the things I have built have had a very high buzzword-to-value ratio.
I would argue against these practices in many cases, but if the client is fully informed of the possible outcomes, they must be permitted to control their own destiny.
Better: A Surgeon's Notes on Performance is a terrific book, especially for software developers. Issues of efficiency, ethics, and improving personal performance are all critical to software professionals. “Better” addresses them with insights that forced me to re-examine some of my own ideas and practices. Highly recommended.
I know there’s a grey area here. If you believe the client wants things so messy and so unhygienic that you don’t have confidence in the project, I will support your decision to say no to the client. I applaud that. But if it’s in that grey area where we are arguing the difference between good and mediocre but serviceable from the client’s perspective, I believe that my responsibility is to provide clients with the information they need to make the decision.
So for example, if they demand BDUF, I have to tell them how I think the project will actually go if we walk that line. I need them to “sign a consent form” explaining that they understand the risk that we will deliver something that will meet the written requirements but will not meet their updated, educated requirements once the project is underway. They must be clear on the fact that this choice will be a loss for them if and when they realize that the origional direction was imperfect. I need them looking me straight in the eye and saying “I understand what you are saying, but I am willing to take that chance.”
If they argue with me and tell me I’m wrong, that BDUF is always better than iterative, then I know that I am not the right man for them. I know that they are not making an informed choice. And likewise, if asked to do something quick and dirty, I will say “no” if the client tells me they believe it is the best way. But if they understand technical debt, if they know they are making a trade off of now versus later, who am I to reach into their pocket and spend their money as I see fit?
Our midwives know much more about giving birth than we did. And they advised us accordingly. But we bore the responsibility for certain decisions, and they ceded those decisions to us, even when they had strong opinions about what was the best thing to do. I think as a software developer I must follow the same path. A midwife will not allow a client to perform a home Caesarian section. There are limits, there are things we know mean instant and irrevocable failure. But where judgment is required, where there is a decision between mediocre, good, and better outcomes, they advise and then allow the client to make the final say.
I believe we in software must follow the same path.
I found some folks arguing about whether developers should write messy code. Having observed our amazing midwives “deliver” for the second time, I believe it is often an issue of Informed Choice.
Presuming that the client or stake holder bears the entire “cost” of the decision, they deserve the last word. I know more than a few people who argue with this, saying that writing clean code is in the client’s best interests. Well yes, of course it is. And in my experience, so are a number of other practices such as managing a project using Theory P, using small teams, and customizing a design for the specific project. But let’s make this really simple: the client is asking for something which—in your professional opinion—is not in their best interests but also is not so ruinously bad that the entire project will absolutely, positively fail.
There can be a set of options that are all somewhat reasonable, with some better or much better than others, but none being outright wrong. In midwifery, there is a choice between giving birth in the hospital or at home. There is a choice between having an epidural or going natural. There are choices about what to administer to the newborn as a prophylactic against infection. The mother makes an informed decision on each of these subjects with the assistance of the midwife, and the midwife supports here even if it is a decision that the midwife feels is statistically less than optimum.
In software development, a client may wish to design everything up front. I believe with all of my heart this is not the best way to go, but let me tell you: I have managed projects on this basis and succeeded. I think I have done a better job managing projects another way, but I can’t tell you that a project absolutely has to fail if you start this way. Likewise large teams deliver software, and some of the things I have built have had a very high buzzword-to-value ratio.
I would argue against these practices in many cases, but if the client is fully informed of the possible outcomes, they must be permitted to control their own destiny.
Better: A Surgeon's Notes on Performance
is a terrific book, especially for software developers. Issues of efficiency, ethics, and improving personal performance are all critical to software professionals. “Better” addresses them with insights that forced me to re-examine some of my own ideas and practices. Highly recommended.So for example, if they demand BDUF, I have to tell them how I think the project will actually go if we walk that line. I need them to “sign a consent form” explaining that they understand the risk that we will deliver something that will meet the written requirements but will not meet their updated, educated requirements once the project is underway. They must be clear on the fact that this choice will be a loss for them if and when they realize that the origional direction was imperfect. I need them looking me straight in the eye and saying “I understand what you are saying, but I am willing to take that chance.”
If they argue with me and tell me I’m wrong, that BDUF is always better than iterative, then I know that I am not the right man for them. I know that they are not making an informed choice. And likewise, if asked to do something quick and dirty, I will say “no” if the client tells me they believe it is the best way. But if they understand technical debt, if they know they are making a trade off of now versus later, who am I to reach into their pocket and spend their money as I see fit?
Our midwives know much more about giving birth than we did. And they advised us accordingly. But we bore the responsibility for certain decisions, and they ceded those decisions to us, even when they had strong opinions about what was the best thing to do. I think as a software developer I must follow the same path. A midwife will not allow a client to perform a home Caesarian section. There are limits, there are things we know mean instant and irrevocable failure. But where judgment is required, where there is a decision between mediocre, good, and better outcomes, they advise and then allow the client to make the final say.
I believe we in software must follow the same path.
Comments on “Informed Choice”:
<< Home
(This is a copy of the response I posted to InfoQ.)
Reg, you make some interesting points. I also think you're addressing a broader point than I am, which confuses the issue a bit.
So let's look at the specific case of deliberately incurring technical debt in order to meet a deadline. More specifically, we've got an agile team that has provided an estimate for a story, but now is being pressured by its on-site customers or product owner to take shortcuts in order to reduce that estimate.
The mistake I see teams make, and the mistake I think you're making, is to assume that the on-site customers and product owners are in charge. On an agile team, that shouldn't be true. It's a team, not a heirarchy. The whole team should be working together to create a good plan. If that's not happening--if anybody, programmers or customers, is bullying or making unilateral decisions--then something is wrong.
The team, not the customers, not the programmers, but the team is responsible for figuring out how to best meet organizational goals.
Now, if executive sponsor who's funding the project says, "thou shalt use punch cards and COBOL," or anything similarly silly*, that's fine. That's what you do, after informing her of the ramifications of her decision (and starting your job search). But on-site customers and product owners are not executive sponsors. They're part of the team. They're an equal voice, not the last word.
PS: I'm the first to admit that it's sometimes appropriate to incur technical debt to meet a deadline. There are consequences--serious consequences--and it's not something to do lightly. Again, the decision to incur technical debt should be a team decision, one that's made with eyes open and that balances business rewards with technical costs and risks.
*PPS: To be totally fair, I imagine there are times when punch cards and COBOL might actually be appropriate. Maybe.
Reg, you make some interesting points. I also think you're addressing a broader point than I am, which confuses the issue a bit.
So let's look at the specific case of deliberately incurring technical debt in order to meet a deadline. More specifically, we've got an agile team that has provided an estimate for a story, but now is being pressured by its on-site customers or product owner to take shortcuts in order to reduce that estimate.
The mistake I see teams make, and the mistake I think you're making, is to assume that the on-site customers and product owners are in charge. On an agile team, that shouldn't be true. It's a team, not a heirarchy. The whole team should be working together to create a good plan. If that's not happening--if anybody, programmers or customers, is bullying or making unilateral decisions--then something is wrong.
The team, not the customers, not the programmers, but the team is responsible for figuring out how to best meet organizational goals.
Now, if executive sponsor who's funding the project says, "thou shalt use punch cards and COBOL," or anything similarly silly*, that's fine. That's what you do, after informing her of the ramifications of her decision (and starting your job search). But on-site customers and product owners are not executive sponsors. They're part of the team. They're an equal voice, not the last word.
PS: I'm the first to admit that it's sometimes appropriate to incur technical debt to meet a deadline. There are consequences--serious consequences--and it's not something to do lightly. Again, the decision to incur technical debt should be a team decision, one that's made with eyes open and that balances business rewards with technical costs and risks.
*PPS: To be totally fair, I imagine there are times when punch cards and COBOL might actually be appropriate. Maybe.
Well James, perhaps we are talking about different things. I am using words like stake holder, client, and (in a related post) employer to describe a very specific person, the person who bears the consequences of the decision. I think you use the term Executive Sponsor in a similar vein.
if anybody, programmers or customers, is bullying or making unilateral decisions--then something is wrong.
I am not talking about bullying. We had midwives. We didn't bully them, that simply wasn't possible. And you are right, there are certain things you can direct me to do, but you cannot bully me into doing them. For example, for low-risk mothers, statistics suggest they are safer giving birth at home than in the hospital.
However, the hospital is the only accepted place for birth in high-risk pregnancies. Although the midwives and the parents are a team, the decision on this subject works like this: if the pregnancy is high-risk, the midwives will walk away if you decide to give birth at home. They will not assist you.
If the birth is low-risk, the midwives advise you of your options and let you choose.
I believe there are many decisions like this in software development. If you ask me to estimate how long something will take, I can give an estimate (or set of likely to unlikely outcomes). If asked "what about incurring technical debt by cutting corners," I can respond that there's a very small chance of coming out ahead if nothing goes wrong but that the most likely outcome is falling very far behind.
Just as you advise, I cannot be bullied into changing my view of the outcome, although if you tell me you understand but wish to "roll the dice," and if you are the true responsible person, the Executive Sponsor, perhaps, then maybe I can try my best if there is a reasonable chance of an acceptable outcome. So perhaps we agree, this privilege does not extend to every employee or nominee of a client organization, just to the person who signs the cheques and carries the responsibility for the project on their back.
p.s. Punch cards and COBOL sounds exactly like something that would cause me to walk away, but I can name something else that is against my beliefs but within the realm of reason: I could be asked to build something in Java using a medium-to-large team of relatively inexperienced developers relying on me as an architect.
At this time, I don't believe this best meets organizational goals for my clients and/or employers, but I also don't believe it is an instant source of failure, so I am prepared to support that decision if—as you suggest—the executive sponsor understands the risks and serious consequences of the decision.
if anybody, programmers or customers, is bullying or making unilateral decisions--then something is wrong.
I am not talking about bullying. We had midwives. We didn't bully them, that simply wasn't possible. And you are right, there are certain things you can direct me to do, but you cannot bully me into doing them. For example, for low-risk mothers, statistics suggest they are safer giving birth at home than in the hospital.
However, the hospital is the only accepted place for birth in high-risk pregnancies. Although the midwives and the parents are a team, the decision on this subject works like this: if the pregnancy is high-risk, the midwives will walk away if you decide to give birth at home. They will not assist you.
If the birth is low-risk, the midwives advise you of your options and let you choose.
I believe there are many decisions like this in software development. If you ask me to estimate how long something will take, I can give an estimate (or set of likely to unlikely outcomes). If asked "what about incurring technical debt by cutting corners," I can respond that there's a very small chance of coming out ahead if nothing goes wrong but that the most likely outcome is falling very far behind.
Just as you advise, I cannot be bullied into changing my view of the outcome, although if you tell me you understand but wish to "roll the dice," and if you are the true responsible person, the Executive Sponsor, perhaps, then maybe I can try my best if there is a reasonable chance of an acceptable outcome. So perhaps we agree, this privilege does not extend to every employee or nominee of a client organization, just to the person who signs the cheques and carries the responsibility for the project on their back.
p.s. Punch cards and COBOL sounds exactly like something that would cause me to walk away, but I can name something else that is against my beliefs but within the realm of reason: I could be asked to build something in Java using a medium-to-large team of relatively inexperienced developers relying on me as an architect.
At this time, I don't believe this best meets organizational goals for my clients and/or employers, but I also don't believe it is an instant source of failure, so I am prepared to support that decision if—as you suggest—the executive sponsor understands the risks and serious consequences of the decision.
I once worked on a project that was a BDUF. For a new system, not a replacement, in fact the client's first attempt at major EIS. And it used new technology just out of beta. Staffing was problematic, politically (client mandated) and economically (cost cutting). The project manager was fresh. And once delivered, the client's eyes opened and they realized they wanted something else. Except, it was deliver on time, under budget, and that something else was "more of the same, and we'll pay double".
Sometimes your initial pessimistic instinct can be wrong, even if most projects like that do end up in miserable failure.
I think you first need to answer a simple question. What's your stake in the project? To get paid? Have a referenceable success story? Boost your ego? Make a happy client? Answer that, and everything else falls into place.
Assuming what the client needs is presumptuous. It might be a political play, when near failure is considered a success. Other times, a small difference in quality has significant affect on the bottomline. Sometimes it's best done quick, sometimes best done right. There is no clear cut answer to all situations.
But there are two things I learned that always hold true:
1. Make sure the client's desired outcome is aligned with yours.
2. And if not, that you can walk away.
Sometimes your initial pessimistic instinct can be wrong, even if most projects like that do end up in miserable failure.
I think you first need to answer a simple question. What's your stake in the project? To get paid? Have a referenceable success story? Boost your ego? Make a happy client? Answer that, and everything else falls into place.
Assuming what the client needs is presumptuous. It might be a political play, when near failure is considered a success. Other times, a small difference in quality has significant affect on the bottomline. Sometimes it's best done quick, sometimes best done right. There is no clear cut answer to all situations.
But there are two things I learned that always hold true:
1. Make sure the client's desired outcome is aligned with yours.
2. And if not, that you can walk away.
Sarah:
Thanks for your comment. Could you provide some more feedback? Is this analogy funny because it is odd? or is it laughable in the deeply unsound sense?
Also, thanks for your blog. I found last night’s entry on the risks of Caesarian Sections particularly interesting: like software development, there is obviously a wide range of opinions on which procedures are in the patient’s best interests and which are not.
Thanks for your comment. Could you provide some more feedback? Is this analogy funny because it is odd? or is it laughable in the deeply unsound sense?
Also, thanks for your blog. I found last night’s entry on the risks of Caesarian Sections particularly interesting: like software development, there is obviously a wide range of opinions on which procedures are in the patient’s best interests and which are not.
Presuming that the client or stake holder bears the entire “cost” of the decision, they deserve the last word.
Presuming that, and also presuming that there isn't some aspect of the domain that could cause harm to uninvolved parties (e.g. avionics, medical equipment, reactor controls, traffic signals, etc.), I would agree.
It's not always a reasonable presumption that the client is the one bearing the cost, though. When the project is internal to a company, rather than on a contract/consulting basis, you're effectively the one bearing the cost if stakeholders pressure you to abandon good process, and it's worth pushing back hard at that.
Presuming that, and also presuming that there isn't some aspect of the domain that could cause harm to uninvolved parties (e.g. avionics, medical equipment, reactor controls, traffic signals, etc.), I would agree.
It's not always a reasonable presumption that the client is the one bearing the cost, though. When the project is internal to a company, rather than on a contract/consulting basis, you're effectively the one bearing the cost if stakeholders pressure you to abandon good process, and it's worth pushing back hard at that.
Susan:
presuming that there isn't some aspect of the domain that could cause harm to uninvolved parties (e.g. avionics, medical equipment, reactor controls, traffic signals, etc.)
I'm such an enthusiast that I wrote a post on that subject.
And yes, there are more complex situations where you bear some of the cost. In many organizations, the department requesting the work gets the credit for functionality but the development team gets the blame for defects. In such situations, you are entitled to share the last word on process!
This is why informed consent might be a good model. For serious decisions, patients actually sign a consent form. A similar document could be fashioned indicating that the department requesting the slapdash process was warned of the consequences.
I have used this technique. In a past life, I worked with Investment Bankers. Ever prospectus as a "Risk Factors" section designed to protect them against liability law suits. A lot of it is boilerplate, but it is crucial to include as many risks as you can identify and how you plan to manage them.
I have put a "Risk Factors" section in project plans. Most recently, I was told that the organization would "hire up" to the needs of the project. I take a dim view of planning a project without knowing who will be carrying out the work, much less asking them to estimate the work. I included the personnel situation as a risk factor.
presuming that there isn't some aspect of the domain that could cause harm to uninvolved parties (e.g. avionics, medical equipment, reactor controls, traffic signals, etc.)
I'm such an enthusiast that I wrote a post on that subject.
And yes, there are more complex situations where you bear some of the cost. In many organizations, the department requesting the work gets the credit for functionality but the development team gets the blame for defects. In such situations, you are entitled to share the last word on process!
This is why informed consent might be a good model. For serious decisions, patients actually sign a consent form. A similar document could be fashioned indicating that the department requesting the slapdash process was warned of the consequences.
I have used this technique. In a past life, I worked with Investment Bankers. Ever prospectus as a "Risk Factors" section designed to protect them against liability law suits. A lot of it is boilerplate, but it is crucial to include as many risks as you can identify and how you plan to manage them.
I have put a "Risk Factors" section in project plans. Most recently, I was told that the organization would "hire up" to the needs of the project. I take a dim view of planning a project without knowing who will be carrying out the work, much less asking them to estimate the work. I included the personnel situation as a risk factor.
Reg, I think you're right: we do agree. We're talking about different aspects of professionalism, both of which are important. And I agree that the client/executive sponsor gets to override our decisions, and that we get to protest strenuously and/or find another job if it's that important.
I've brought the two different approaches to professionalism together in a new blog entry.
I've brought the two different approaches to professionalism together in a new blog entry.
<< Home
