Wednesday, December 27, 2006
Relational Calculus is about Relations, not Rows
A post on Enfranchised Mind (fabulous blog name!) nails the ORM problem:
What strikes me about this quote is the real value in learning new programming paradigms—like functional programming—where stuff other than Plain Old Objects (“POO”) is raised to first class status. When functions can be passed functions as parameters and return functions from functions, we start to think about manipulating the functions themselves, rather than manipulating dumb data.
The same goes for continuations, environments, super-strong typing, and a host of other ideas that have been known to the programming community for decades. Even if they aren’t a formal part of your current language, they are a valuable part of your perspective on how to use your language to solve problems.
In the case of modeling SQL, there is no reason why standard OO cannot model joins, selections, and projections with ease. All it takes is the flash of insight that it’s important to think about a relational database that way.
Update: Here’s your homework: compare and contrast the Relational Algebra with the Relational Calculus. Does SQL implement the algebra or the calculus? If your answer is Algebra, propose an OO model of the Calculus; if your answer is Calculus, propose an OO model of the Algebra.
The core of relational calculus operates on relations,aka tables. There are three main operators, all of which work on relations/tables: join, which takes two relations and creates a third relation which consists of all possible pairings of the rows of the two original relations; selection, which takes a relation and creates a new relation with only a subset of rows of the original relation; and projection, which takes a relation and creates a new relation with a subset of the columns of the old relation
These three operations are the core, the heart and soul, of relational calculus—which is the core of SQL. To faithfully model SQL we must, on some level, faithfully model the relational calculus. And this is where I think the Object Oriented programmers go astray in trying to interface to SQL. In their hurry to make things into objects, they immediately (and without fail) declare the rows to be objects—and thus miss the fact that relational calculus and thus SQL is about relations, not rows.
They’re abstracting at the wrong level.
What strikes me about this quote is the real value in learning new programming paradigms—like functional programming—where stuff other than Plain Old Objects (“POO”) is raised to first class status. When functions can be passed functions as parameters and return functions from functions, we start to think about manipulating the functions themselves, rather than manipulating dumb data.
The same goes for continuations, environments, super-strong typing, and a host of other ideas that have been known to the programming community for decades. Even if they aren’t a formal part of your current language, they are a valuable part of your perspective on how to use your language to solve problems.
In the case of modeling SQL, there is no reason why standard OO cannot model joins, selections, and projections with ease. All it takes is the flash of insight that it’s important to think about a relational database that way.
Update: Here’s your homework: compare and contrast the Relational Algebra with the Relational Calculus. Does SQL implement the algebra or the calculus? If your answer is Algebra, propose an OO model of the Calculus; if your answer is Calculus, propose an OO model of the Algebra.
Economizing can be penny-wise and pound foolish
or,
Reinier Zwitserloot said something interesting:
The one factor that has risen above all others for these tasks is the signal to noise ratio in code. This idea has been far, far better explained as the difference between “accidental” and “inherent” (or “actual”) difficulty in programming.
Pick up the source code for a program and three highlighter markers, one green, one yellow, and one red. Go through the source. If you don’t understand what something does, mark it in red. If you understand it, but it has everything to do with the accidental difficulty of the implementation and nothing to do with the inherent difficulty of the problem, mark it in yellow. And finally, if something seems to express the problem and its solution fairly directly in an manner you understand, mark it in green.
What could be simpler? We want more green, less yellow, and absolutely no red in our programs. We can’t escape the yellow marker: if we like (I only say if we like) Ruby on Rails, we like the fact that our code has lots of green, because it’s written in a DSL aimed at CRUD applications on the web. But if we look inside Rails, we see lots of stuff that is yellow or—horribly for us—red. The yellow under the covers makes the green in our code possible.
(Ad: Raymond Smullyan introduces Godel's incompleteness theorems and some interesting related results though puzzles about knights, knaves, and their ability to become self-aware.)(I dare say that this was once the philosophy behind enterprisey stuff like SOAP and J2EE. Somebody, somewhere felt that these frameworks could help programmers write stuff that focused on the task at hand and left the accidental difficulties, the “gory details” as they were, to the framework.)
Concentration
This is the principle behind structured programming and every programming language where we can build our own abstractions.
It’s not that we eliminate the yellow. Rather, we shift it away from the green so that there's one nice place where everything is green: we say that this is our solution, this is our idea. And there's this other mass of yellow that makes the green work, we say this is our infrastructure, this is the hardest working code in show business.
The opposite of this is where the green and yellow is all mixed up. This is why Design Patterns are a sign of weakness, not of strength. With a pattern, you have yellow code intermingled with your green code. You may be so used to it you think you don’t see the yellow, but nevertheless your Visitor or your Singleton or your Adapter is there taking up space and diluting your ideas.
At a finer level, this is why iteration—especially when we maintain our own index variables—is weaker code than mapping and folding. Yellow code like
When the green is all in once place, we can look at it and verify it and think about it with our Inherent Difficulty hat on. And by segregating all of the yellow code into its own place, we can look at it and verify it and think about it with our Accidental Difficulty or Implementation hat on.
Further to that, my experience is that our green code sometimes changes in really fundamental ways. Requirements change, our understanding changes, and we must make drastic changes to the fundamental logic behind our programs. When the green is in one place, you change it and—cross your fingers—you’re done.
The yellow code is as obstinate and coupled and messy as ever, but you don’t touch it. When your code is intermingles, the slightest change to your program logic breaks all of your infrastructure and patterns in unpredictable ways. Coupling in yellow code is bad, but coupling yellow code to green code is hideous.
That's the appeal of Domain-Specific Languages and Metaprogramming. When done right, there's no less implementation or accidentally difficult code, but it’s sharply separated from the code that solves the inherent difficulties of our problem.
So, does size matter?
I’m very comfortable with the idea that some languages enable more green and less yellow through sheer economy of expression. But I’m also comfortable with the idea that other languages may offer more green and less yellow or more separation of green and yellow through different mechanisms.
If two programs do much the same thing and one is smaller than the other, I’m not going to jump right up and say that the former is necessarily better or that the former’s language/framework/libraries are necessarily better.
At the moment, I want to look at them both and start colouring. If the longer program has more green code, or has it all in one place, or if the shorter program is marred by red… I’m inclined to say the short one isn’t the best we can do.
So… if expressiveness is measured by economy of expression, I don’t think it is always better. And should we pursue economy at the expense of separating the solution to inherent and accidental difficulties, I think we make our code less worse, not better.
“How I learned to stop worrying and love the separation of accidental and actual complexity in programs.”
Reinier Zwitserloot said something interesting:
What’s worth more? The macro facilities or the provably linear-time chasability of code identifiers and the like? All I know is: I have never read a satisfactory answer to the dilemma, merely a boatload of silent assumptions that the expressive power must be better… somehow.I’m a simple man, and what matters to me is the amount of time it takes me to design code, to write code, to debug code, to understand the code that others have written, and to write code that others will understand.
The one factor that has risen above all others for these tasks is the signal to noise ratio in code. This idea has been far, far better explained as the difference between “accidental” and “inherent” (or “actual”) difficulty in programming.
Pick up the source code for a program and three highlighter markers, one green, one yellow, and one red. Go through the source. If you don’t understand what something does, mark it in red. If you understand it, but it has everything to do with the accidental difficulty of the implementation and nothing to do with the inherent difficulty of the problem, mark it in yellow. And finally, if something seems to express the problem and its solution fairly directly in an manner you understand, mark it in green.
What could be simpler? We want more green, less yellow, and absolutely no red in our programs. We can’t escape the yellow marker: if we like (I only say if we like) Ruby on Rails, we like the fact that our code has lots of green, because it’s written in a DSL aimed at CRUD applications on the web. But if we look inside Rails, we see lots of stuff that is yellow or—horribly for us—red. The yellow under the covers makes the green in our code possible.
(Ad: Raymond Smullyan introduces Godel's incompleteness theorems and some interesting related results though puzzles about knights, knaves, and their ability to become self-aware.)
Concentration
This is the principle behind structured programming and every programming language where we can build our own abstractions.
It’s not that we eliminate the yellow. Rather, we shift it away from the green so that there's one nice place where everything is green: we say that this is our solution, this is our idea. And there's this other mass of yellow that makes the green work, we say this is our infrastructure, this is the hardest working code in show business.
The opposite of this is where the green and yellow is all mixed up. This is why Design Patterns are a sign of weakness, not of strength. With a pattern, you have yellow code intermingled with your green code. You may be so used to it you think you don’t see the yellow, but nevertheless your Visitor or your Singleton or your Adapter is there taking up space and diluting your ideas.
At a finer level, this is why iteration—especially when we maintain our own index variables—is weaker code than mapping and folding. Yellow code like
++i; is sprayed all over our problem logic.When the green is all in once place, we can look at it and verify it and think about it with our Inherent Difficulty hat on. And by segregating all of the yellow code into its own place, we can look at it and verify it and think about it with our Accidental Difficulty or Implementation hat on.
Further to that, my experience is that our green code sometimes changes in really fundamental ways. Requirements change, our understanding changes, and we must make drastic changes to the fundamental logic behind our programs. When the green is in one place, you change it and—cross your fingers—you’re done.
The yellow code is as obstinate and coupled and messy as ever, but you don’t touch it. When your code is intermingles, the slightest change to your program logic breaks all of your infrastructure and patterns in unpredictable ways. Coupling in yellow code is bad, but coupling yellow code to green code is hideous.
That's the appeal of Domain-Specific Languages and Metaprogramming. When done right, there's no less implementation or accidentally difficult code, but it’s sharply separated from the code that solves the inherent difficulties of our problem.
So, does size matter?
I’m very comfortable with the idea that some languages enable more green and less yellow through sheer economy of expression. But I’m also comfortable with the idea that other languages may offer more green and less yellow or more separation of green and yellow through different mechanisms.
If two programs do much the same thing and one is smaller than the other, I’m not going to jump right up and say that the former is necessarily better or that the former’s language/framework/libraries are necessarily better.
At the moment, I want to look at them both and start colouring. If the longer program has more green code, or has it all in one place, or if the shorter program is marred by red… I’m inclined to say the short one isn’t the best we can do.
So… if expressiveness is measured by economy of expression, I don’t think it is always better. And should we pursue economy at the expense of separating the solution to inherent and accidental difficulties, I think we make our code less worse, not better.
Why Philip is a Member of the Group of Seven
Dossy was kind enough to link to this weblog. I tried posting a comment in reply to his post, but—alas—the OpenID login didn’t work and I couldn’t get past his weblog’s captcha.
(Ad: The most beautiful book ever written about writing web applications. Also available online for free.)Here’s what I wanted to say:
(Ad: The most beautiful book ever written about writing web applications. Also available online for free.)
Thanks for the link to The first seven books I would buy if my shelves were bare.
I’m curious: why are you surprised that Phil’s book is one of the seven?
I’ll try to explain my reasoning. I deliberately restricted my choices to books that I have returned to re-read many times over the years. I have flipped through “Philip and Alex’s Guide to Web Publishing” more than a few times since I first read it.
The technology choices are no longer popular, true. But the message, about simplicity and reliability, still resonates with me. I especially like his diatribe against “JunkWare”. In this era of enterprisey stacks like SOAP, perhaps his message is even more relevant than ever?
Wednesday, December 20, 2006
Giles Bowkett turns language elitism upside down
Isn’t the idea that languages like Scheme and SmallTalk are easier to use than languages like C# and Java very interesting? Is this true? How could this be? The conventional wisdom is that languages like Scheme are too abstract. How could so-called novices grasp concepts like recursion, mapping, and folding when they have trouble remembering that the
The authors of The Little Schemer and The Little MLer bring deep and important insights to the Java language. Every serious Java programmer should own a copy.Could it be that more abstract languages are smaller languages? With a more abstract language, there are obviously fewer things you need to use to write even complex functions. That’s the expressiveness people talk about: everyone grasps the idea that some languages let you express an algorithm in fewer bits of information. So with these languages, you need to use fewer bits of information to express each idea.
Could it be that these same languages require you to learn fewer bits of information as well?
I am reminded of one of those silly “Men are from Mars, degrees are from mail-order stores” aphorisms. The idea was that every bit of kindness is worth one point, no more no less. One rose? One point. A dozen roses? The same one point.
We talk about some ideas (like mapping and folding) as if they are intrinsically harder to grasp and learn than other ideas (like iteration). But maybe it doesn’t require more bits to learn? Maybe map is one point and iteration is one point, even though we want to think that map is twelve points and iteration is two points?
If that’s the case, could a language with fewer ideas that can be combined in succinct ways be easier to learn than a language with lots of different weak ideas?
To say that Lisp is a language of the gods, and Java is a language for mortals, that could in fact be getting everything entirely backwards. Scheme is actually easier to use than Java. So the easier language is the language of the gods, and the hard one is the one for regular people? How does that make sense?
—Giles Bowkett, What If There Are No Gods?
Isn’t the idea that languages like Scheme and SmallTalk are easier to use than languages like C# and Java very interesting? Is this true? How could this be? The conventional wisdom is that languages like Scheme are too abstract. How could so-called novices grasp concepts like recursion, mapping, and folding when they have trouble remembering that the
main method must be static?The authors of The Little Schemer and The Little MLer bring deep and important insights to the Java language. Every serious Java programmer should own a copy.
Could it be that these same languages require you to learn fewer bits of information as well?
I am reminded of one of those silly “Men are from Mars, degrees are from mail-order stores” aphorisms. The idea was that every bit of kindness is worth one point, no more no less. One rose? One point. A dozen roses? The same one point.
We talk about some ideas (like mapping and folding) as if they are intrinsically harder to grasp and learn than other ideas (like iteration). But maybe it doesn’t require more bits to learn? Maybe map is one point and iteration is one point, even though we want to think that map is twelve points and iteration is two points?
If that’s the case, could a language with fewer ideas that can be combined in succinct ways be easier to learn than a language with lots of different weak ideas?
Labels: lispy
Tuesday, December 19, 2006
Just because we can
The following program can be compiled in ten different programming languages—and it prints something different in all of them. The languages: C, C++, Perl, TeX, LaTeX, PostScript, sh, bash, zsh and Prolog:
Snarfed (with awe and respect) from pts oldalai.
%:/*:if 0;"true" +s ||true<</;#|+q|*/include<stdio.h>/*\_/
{\if(%)}newpath/Times-Roman findfont 20 scalefont setfont(
%%)pop 72 72 moveto(Just another PostScript hacker,)show((
t)}. t:-write('Just another Prolog hacker,'),nl,halt. :-t.
:-initialization(t). end_of_file. %)pop pop showpage(-: */
int main(){return 0&printf("Just another C%s hacker,\n",1%
sizeof'2'*2+"++");}/*\fi}\csname @gobble\endcsname{\egroup
\let\LaTeX\TeX\ifx}\if00\documentclass{article}\begin{doc%
ument}\fi Just another \LaTeX\ hacker,\end{document}|if 0;
/(J.*)\$sh(.*)"/,print"$1Perl$2$/"if$_.=q # hack the lang!
/
sh=sh;test $BASH_VERSION &&sh=bash;test $POSIXLY_CORRECT&&
sh=sh;test $ZSH_VERSION && sh=zsh;awk 'BEGIN{x="%c[A%c[K"
printf(x,27,27)}';echo "Just another $sh hacker," #)pop%*/
Snarfed (with awe and respect) from pts oldalai.
Monday, December 18, 2006
Never stop learning
I think every program you write should be the hardest you’ve ever written.
Labels: passion
Saturday, December 16, 2006
Lisp is not the last word
Ken Tilton asked: What is up the power continuum from Lisp?
I don’t have a ready answer. However, just because I don’t have an answer doesn’t mean I don’t believe there’s an answer. It could be that Lisp is a little like Democracy. It could be the least powerful programming language possible, excepting all of the others invented so far. But you know what? I have faith we can do better.
Ken doesn’t say there isn’t a language up the power continuum. And I won’t say we have already invented one: like Ken, I’ll pose a question: what law of computer science places a limit on the power continuum at Lisp?
G.J. Chaitin explains his proofs of Kurt Godel’s incompleteness theorem and Alan Turing’s “halting problem” in computation. Chaitin’s creative use of Lisp in mathematics and fervent belief that no theorem is proof against new analysis are welcome shots of espresso.Human history is chock-a-block full of inventions and practices that were considered for decades or even centuries to be the final word, the ultimate expression and implementation of ideas. And then someone came along and demolished everything. Geocentricity. Heliocentricity. Newtonian celestial mechanics. Light as a wave. Light as a particle. Three dimensions. Uniform space. Euclidian geometry.
Some of these new ideas took years to take root while the establishment derided them as “not even wrong.” Others were so obviously right they immediately displaced what had come before. We might now have invented a more powerful language. Or we might have invented one but not realize it yet. But who can say that we haven’t invented a more powerful language and will never do so?
If you believe there is a power continuum, if you are not so obsessed with Turing Completeness and theoretical equivalence, what is the argument that it has any limit whatsoever, let alone that its limit is Lisp?
I believe that the only language that is affixed to the top of the power continuum is Blub. For everyone whose imagination soars above the ceiling of their laboratory, Lisp is not the last word.
I don’t have a ready answer. However, just because I don’t have an answer doesn’t mean I don’t believe there’s an answer. It could be that Lisp is a little like Democracy. It could be the least powerful programming language possible, excepting all of the others invented so far. But you know what? I have faith we can do better.
Ken doesn’t say there isn’t a language up the power continuum. And I won’t say we have already invented one: like Ken, I’ll pose a question: what law of computer science places a limit on the power continuum at Lisp?
G.J. Chaitin explains his proofs of Kurt Godel’s incompleteness theorem and Alan Turing’s “halting problem” in computation. Chaitin’s creative use of Lisp in mathematics and fervent belief that no theorem is proof against new analysis are welcome shots of espresso.
Some of these new ideas took years to take root while the establishment derided them as “not even wrong.” Others were so obviously right they immediately displaced what had come before. We might now have invented a more powerful language. Or we might have invented one but not realize it yet. But who can say that we haven’t invented a more powerful language and will never do so?
If you believe there is a power continuum, if you are not so obsessed with Turing Completeness and theoretical equivalence, what is the argument that it has any limit whatsoever, let alone that its limit is Lisp?
I believe that the only language that is affixed to the top of the power continuum is Blub. For everyone whose imagination soars above the ceiling of their laboratory, Lisp is not the last word.
Tuesday, December 12, 2006
Does the term "Exponential Growth" mean anything to you?
Assaf laid the tag-smack on me, so here are five little known facts about me. This blog has a strict theme, so all five are “nerdy”:
Lucas, Chris, Reinier, Rusty, and Dharmesh, you’re it. The ‘rules’ appear to be: five personal trivialities and tag five more victims. Good luck finding five who haven’t already been tagged by this generation of growth!
- “My” first computer was the High Speed Job Stream in the Sanford Fleming building at the University of Toronto. It was an IBM something-or-other, but there was no restriction on running punch card jobs in SNOBOL, LISP, or WATFOR. I was twelve at the time, so please don’t be shocked when I say that it took me a few weeks before I became bored of printing calendars with line-printer art of playboy models and progressed to using Fortran to print the multiplication table and Fibonacci series.
My second computer was a Nova 1220. I wrote BASIC programs for it and especially my own highly customized Star Trek game. It needed all 16K of RAM, it was so big. The listing was fourteen feet long. More than a few people tell me with pride that they work on Enterprise software that would probably be fourteen miles long if anyone ever printed all of the source. I change the subject by telling a joke about a truck. Yes, those are front-panel switches for bootstrapping the CPU. Yes, I learned the three 16 bit words needed to obtain an audience with the demon. - I grew up in the sixties, when there were still race riots, when companies routinely discriminated against women because “they would just quit to start a family,” and when Toronto hosted a great number of young American men who didn’t want to fight in Vietnam. I helped my parents prepare for protests by colouring placards. I lived on a commune for a short while and I went to “Young Socialists” camp in the Summers.
Although this seems unrelated to nerdy things, I think anyone reading what I have to say about coding as a form of self-expression and about the importance of the programmer in the production of software will see the unbroken thread that has run through my life. - My first job was in “Mr. Gamesway’s Ark,” a games and puzzles superstore way ahead of its time. My second was as a haberdasher. my career has oscillated in the same fashion ever since: the pursuit of beauty for its own sake (in math or in dress), the social side of nerdery, and sales. Software development in a startup environment seems to deliver all of the goods in one load. And when someone gives me a friendly tip to “wear a blue suit to the presentation,” it’s always nice to be able to ask in return “the Chalk Stripe or the Plain Navy?”
- I dream in black and white, although from time to time I am aware of a colour in my head: I know in the dream that something has a colour, but I cannot see the colour and this seems very normal. Like most of humanity, I remember having flying and levitation dreams as well as helplessness dreams.
- I am an autodidact. This is probably obvious to anyone with a real (as opposed to JavaSchool) education in Computer Science.
Lucas, Chris, Reinier, Rusty, and Dharmesh, you’re it. The ‘rules’ appear to be: five personal trivialities and tag five more victims. Good luck finding five who haven’t already been tagged by this generation of growth!
Labels: passion
Monday, December 11, 2006
You call this progress?
I can’t emphasize this point strongly enough. In a mini-presentation on connecting outcomes to project success, I suggested that when measuring progress, evidence of achieving the “ends” is more valuable than evidence of achieving the “means.”
In other words, evidence of completed features, things of use and value, is more useful than evidence that we are “on plan” or “in compliance with the methodology.”
Always.
One of the things most people want to measure is progress towards project completion. But you can’t measure project completion progress unless you have completed features: developed, integrated, and tested features. A completed feature is done enough for someone to use.
Johanna Rothman, Measuring Project Completion Progress
I can’t emphasize this point strongly enough. In a mini-presentation on connecting outcomes to project success, I suggested that when measuring progress, evidence of achieving the “ends” is more valuable than evidence of achieving the “means.”
In other words, evidence of completed features, things of use and value, is more useful than evidence that we are “on plan” or “in compliance with the methodology.”
Always.
Labels: agile
Friday, December 08, 2006
Trick or Bleep?
I like to post fun stuff on Fridays. I also repost old stuff from time to time when my brain is page faulting or when something I’m reading reminds me of an old post. Today both were true: I spotted this post about interviews going wrong and I remembered an interview where I was the hapless victim.
There has been some refreshing commentary on the post and also on reddit. I want to give my thoughts on one issue. Was the interviewer posing a “trick question,” trying to find out how I negotiate conflict? This subject has attracted a lot of attention since Joel Spolsky recently raised the issue, and while it didn’t cross my mind at the time, it’s worth considering.
The question posed to me was, list all of the ways to call instance and class methods in Java. I listed a bunch of things, and at the end, I included:
If this was a trick question, ok, you had me! I am not afraid to argue points where I believe there is a direct connection between the issue and the outcome of software development projects. If I really believe it is irrelevant, I don’t argue even if I have a personal preference. So I’ll happily go along with any particular coding style you advocate (including none), and if you insist that I use Eclipse for Java instead of TextMate, fine.
The problem with that question (let’s be clear, it’s my problem, not the interviewer’s problem), is that I don’t really care whether Java lets you call a static method via an instance of a class. The reason I don’t care, as I explained to the interviewer, is that I will never allow that code to get into the repository.
For those who are too busy with Lisp to know all of Java’s warts, the problem is that static or class methods are never polymorphic. So if you write:
Back to the “trick question.” My problem is that since I never have and never will use this, I’m not going to argue whether it’s legal or not. It will never have the slightest impact on any project I’m on. If you want an argument, let’s discuss why you think we should use this idiom in production code.
I honestly don’t think it was a trick question. If the interviewer knew it was legal and wanted an argument, he could have (and I suggest should have) chosen a different thing to argue, something a little more obvious, such as telling me it is legal and polymorphic, and then seeing whether I (a) know that it is not polymorphic, and (b) argue with him.
I don’t use questions like this, but I can see the value of this second question: a really experienced person knows exactly why it isn’t polymorphic and can discuss the kinds of bugs you will get if you try to write code depending on it.
What do you think? Was this really a trick question? If so, should I have been more aggressive in arguing the subtle points of Java law?
p.s. The interviewer later told me that he did not like my answer, and indeed I did not get an offer: one benefit of much water having passed under the bridge is a more balanced perspective. I no longer assume that this one question is the only reason the interviewer decided I was not appropriate.
“Blink” examines why people make unconscious decisions. It's very relevant to interviews!I may have been overqualified, too talkative, too reflective, or even too meticulous (when given the programming challenge of permuting a string, I was apparently the only candidate to ask what the output should be when the string contained duplicate letters). Perhaps the interviewer had already decided the interview outcome before this question was discussed.
And do you know what? I’m not so sure I wouldn’t have reacted the same way as the interviewer. One of the big “No Hire” flags for me is a candidate who doesn’t fundamentally “get” a programming language. Anyone can “Learn Blub in 21 Days,” memorizing code snippets and patterns by rote. But such candidates often have no clue how a language works, how its features interact with each other to make those memorized code snippets work.
Let’s say that I’m interviewing and I don’t know that Java (which isn’t particularly OO in theory or in practice) allows this bug of calling a static ‘method’ via an instance of a class. Is that so bad? Of course not, it’s a wart on the language. It’s not like knowing that it is allowed reveals a deep understanding of how Java works. So not knowing this isn’t a fundamental flaw in a Java developer or interviewer.
Now a candidate tells me that this thing works. Well, I might jump to the conclusion that the candidate really doesn’t get the difference between static and instance methods, that the candidate will have trouble understanding the various scoping issues involved with instance members and static members. I could easily believe that this candidate will draw a complete blank when working with inner classes.
An interviewer who didn’t happen to know this bit of Java trivia might reasonably infer that thinking it works betrays a superficial knowledge of Java.
There has been some refreshing commentary on the post and also on reddit. I want to give my thoughts on one issue. Was the interviewer posing a “trick question,” trying to find out how I negotiate conflict? This subject has attracted a lot of attention since Joel Spolsky recently raised the issue, and while it didn’t cross my mind at the time, it’s worth considering.
The question posed to me was, list all of the ways to call instance and class methods in Java. I listed a bunch of things, and at the end, I included:
someInstanceVariable.staticMethod();
// calls the staticMethod of someInstancevariable's declared class
If this was a trick question, ok, you had me! I am not afraid to argue points where I believe there is a direct connection between the issue and the outcome of software development projects. If I really believe it is irrelevant, I don’t argue even if I have a personal preference. So I’ll happily go along with any particular coding style you advocate (including none), and if you insist that I use Eclipse for Java instead of TextMate, fine.
For those who are too busy with Lisp to know all of Java’s warts, the problem is that static or class methods are never polymorphic. So if you write:
// (yes, I know about public vs. private. These are obviously in 'package scope')
class House {
static String doorAction () { return "closes and locks"; }
}
class S___House extends House {
static String doorAction () { return "bangs"; }
}
final House structure = new S___House();
System.out.println(structure.doorAction());
closes and locks instead of bangs. Always. This is not what happens with instance methods, and that’s because so-called ‘static methods’ in Java are really global functions that live in class namespaces. The compiler resolves the function doorAction by looking up the declared type of structure at compile time. Its actual type at runtime is irrelevant. C++ programmers recognize this as the difference between virtual and non-virtual functions.Back to the “trick question.” My problem is that since I never have and never will use this, I’m not going to argue whether it’s legal or not. It will never have the slightest impact on any project I’m on. If you want an argument, let’s discuss why you think we should use this idiom in production code.
I honestly don’t think it was a trick question. If the interviewer knew it was legal and wanted an argument, he could have (and I suggest should have) chosen a different thing to argue, something a little more obvious, such as telling me it is legal and polymorphic, and then seeing whether I (a) know that it is not polymorphic, and (b) argue with him.
I don’t use questions like this, but I can see the value of this second question: a really experienced person knows exactly why it isn’t polymorphic and can discuss the kinds of bugs you will get if you try to write code depending on it.
What do you think? Was this really a trick question? If so, should I have been more aggressive in arguing the subtle points of Java law?
p.s. The interviewer later told me that he did not like my answer, and indeed I did not get an offer: one benefit of much water having passed under the bridge is a more balanced perspective. I no longer assume that this one question is the only reason the interviewer decided I was not appropriate.
“Blink” examines why people make unconscious decisions. It's very relevant to interviews!
And do you know what? I’m not so sure I wouldn’t have reacted the same way as the interviewer. One of the big “No Hire” flags for me is a candidate who doesn’t fundamentally “get” a programming language. Anyone can “Learn Blub in 21 Days,” memorizing code snippets and patterns by rote. But such candidates often have no clue how a language works, how its features interact with each other to make those memorized code snippets work.
Let’s say that I’m interviewing and I don’t know that Java (which isn’t particularly OO in theory or in practice) allows this bug of calling a static ‘method’ via an instance of a class. Is that so bad? Of course not, it’s a wart on the language. It’s not like knowing that it is allowed reveals a deep understanding of how Java works. So not knowing this isn’t a fundamental flaw in a Java developer or interviewer.
Now a candidate tells me that this thing works. Well, I might jump to the conclusion that the candidate really doesn’t get the difference between static and instance methods, that the candidate will have trouble understanding the various scoping issues involved with instance members and static members. I could easily believe that this candidate will draw a complete blank when working with inner classes.
An interviewer who didn’t happen to know this bit of Java trivia might reasonably infer that thinking it works betrays a superficial knowledge of Java.
Thursday, December 07, 2006
Grady Booch called from 2004. He wants his ideas back.
Grady was talking about software factories, components, and yes, even domain-specific languages. Or rather, he was talking about their limitations. There is No Silver Bullet (the one essay is available for free here, but the entire book is outstanding).
This does not mean that we can’t improve the way we write well-understood kinds of software, or that we can’t write new kinds of software that would have been impossible to write with older ideas and tools. Or that we can’t write software with smaller teams and less overhead than with older processes and methodologies.
Here’s the deal:
Writing a program requires two activities. One is to understand the problem and generate an ideal solution in the mind or minds of the programmers. The other is to translate the ideal solution into a language that the computer understands (hint: the computer’s favourite ‘words’ are one and zero).
The translation activity is overcoming the impedance mismatch between our thoughts and the machine. Every step we take with tools and languages and processes removes another portion of this “accidental difficulty”. But it does nothing for the fundamental hardness of understanding problems and formulating the solutions.
As many people have pointed out, building software is exactly like building a building, only: We are always building a building that has never been built before, with materials that have only just been introduced to the market, on a scale that has never been attempted by our team before, and finally we have completely automated the construction process with our marvelous compilers, so all we have to do is come up with a perfect set of blueprints, and did we mention that the client has never used a building for this exact purpose before, and therefore they have very little idea what they will need until the building has been completed and they can try it out?
But that’s ok. Honestly, if it isn’t hard, why bother? The point of our languages, our tools, even of sharing our ideas so we can improve is so that we can spend more time on the problems of the problem, rather than the problems of the tools.
Software development has been, is, and will remain fundamentally hard.
Grady Booch, factories help you produce lots of bad software
Grady was talking about software factories, components, and yes, even domain-specific languages. Or rather, he was talking about their limitations. There is No Silver Bullet (the one essay is available for free here, but the entire book is outstanding).
This does not mean that we can’t improve the way we write well-understood kinds of software, or that we can’t write new kinds of software that would have been impossible to write with older ideas and tools. Or that we can’t write software with smaller teams and less overhead than with older processes and methodologies.
Here’s the deal:
Writing a program requires two activities. One is to understand the problem and generate an ideal solution in the mind or minds of the programmers. The other is to translate the ideal solution into a language that the computer understands (hint: the computer’s favourite ‘words’ are one and zero).
The translation activity is overcoming the impedance mismatch between our thoughts and the machine. Every step we take with tools and languages and processes removes another portion of this “accidental difficulty”. But it does nothing for the fundamental hardness of understanding problems and formulating the solutions.
I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation.
Fred P. Brooks, No Silver Bullet
As many people have pointed out, building software is exactly like building a building, only: We are always building a building that has never been built before, with materials that have only just been introduced to the market, on a scale that has never been attempted by our team before, and finally we have completely automated the construction process with our marvelous compilers, so all we have to do is come up with a perfect set of blueprints, and did we mention that the client has never used a building for this exact purpose before, and therefore they have very little idea what they will need until the building has been completed and they can try it out?
But that’s ok. Honestly, if it isn’t hard, why bother? The point of our languages, our tools, even of sharing our ideas so we can improve is so that we can spend more time on the problems of the problem, rather than the problems of the tools.
In the end, the overall “productivity” of the system, the fact that it came into being at all, was the handiwork not of tools that sought to make programming seem easy, but the work of engineers who had no fear of “hard.”
Ellen Ullman, The dumbing-down of programming, part two
Wednesday, December 06, 2006
Repost: How to Manage an Iteration with a Cork Board and a Stack of Three by Five Cards
On a recent project, we had very good results managing the work within iterations with a giant cork board. The board was divided into columns.
To begin, stories were written by a program manager. He owned the absolute right to define “done-ness” for a story. The stories were written on three by five cards and placed in the first column, “story”.
Programmers were assigned to work with the program manager to size stories. They would usually, but not always, end up working on the code. But first, they negotiated acceptance tests with the program manager. Both sides had a veto on the test suite (an acceptance test is not exactly the same thing as a unit test: see How to Use Acceptance Tests to Ship Software).
When they agreed on a set of tests, the tests were documented (preferably on the card) and the card moved to the next column, “accept-able”.
The programmers were blocked from coding or estimating until this point. Very important! Now they negotiated estimates. When an estimate was done, the card advanced to “sized”. But no programming started.
Based on estimates for most or all of the stories, the stories were prioritized and some could be dropped or delayed. The stories that would be in the iteration stayed in “sized” but now would have names attached to them. By convention, we placed the assigned stories in the lower half of the board.
Now a date was set for the entire iteration and coding would begin.
When a story was ready for testing, it advanced to the next column, “test-able”. The program manager would validate the story, and it would advance again to “accepted”.
If it didn’t validate (boo) it bounced back to “sized”: we had no column for “work-in-progress” (the assumption was that if a story was sized and assigned to someone, they were thinking and/or coding).
When all the stories were “accepted,” we’d do a big acceptance test again, catching any regressions. Some of the stories would bounce to a “penalty box” in the lower right hand corner of the board. We felt it was important to highlight the shame of something being “accepted” but breaking.
The iteration was done when all of the stories were simultaneously “accepted.”
So the columns were:
The Benefits
This process was really light weight. That made it possible for us to have more iterations within the project; we worked on one and two week iterations.
The visibility of the cork board was invaluable: everyone knew where we were, every day. This created a sense of urgency right from the start of each iteration: “we have to get all this done for next Friday? let’s get those acceptance tests defined right now!”.
The process was rigid where we needed it to be rigid (acceptance tests before estimates), but flexible where we needed to be flexible (negotiating acceptance tests, locking down the set of stories after estimates)
Putting the entire team into binary, project-wide iterations really lowered our integration grief.
The Fine Print
Other methods were used for choosing themes for iterations and managing the project as a whole. This is just how we worked within an iteration. In fact, by strict definition the cork board didn’t manage the iteration so much as help us understand where we were and help us follow a set process for movement: I call this Administration rather than Management.
“Now a date was set for the entire iteration and coding would begin.”: That’s a lot of hand waving! Deciding what to include and what to drop is a really complex weaving a deep understanding of the project risk factors, a desire to show constant visible progress (“Demo or Die”), negotiation with stakeholders, negotiation with the developers… A very hard problem.
(Many thanks to Eric Torreborre, whose post Writing on the Walls reminded me how important the physical manifestation of a project is to getting the project done.)
To begin, stories were written by a program manager. He owned the absolute right to define “done-ness” for a story. The stories were written on three by five cards and placed in the first column, “story”.
Programmers were assigned to work with the program manager to size stories. They would usually, but not always, end up working on the code. But first, they negotiated acceptance tests with the program manager. Both sides had a veto on the test suite (an acceptance test is not exactly the same thing as a unit test: see How to Use Acceptance Tests to Ship Software).
When they agreed on a set of tests, the tests were documented (preferably on the card) and the card moved to the next column, “accept-able”.
The programmers were blocked from coding or estimating until this point. Very important! Now they negotiated estimates. When an estimate was done, the card advanced to “sized”. But no programming started.
Based on estimates for most or all of the stories, the stories were prioritized and some could be dropped or delayed. The stories that would be in the iteration stayed in “sized” but now would have names attached to them. By convention, we placed the assigned stories in the lower half of the board.
Now a date was set for the entire iteration and coding would begin.
When a story was ready for testing, it advanced to the next column, “test-able”. The program manager would validate the story, and it would advance again to “accepted”.
If it didn’t validate (boo) it bounced back to “sized”: we had no column for “work-in-progress” (the assumption was that if a story was sized and assigned to someone, they were thinking and/or coding).
When all the stories were “accepted,” we’d do a big acceptance test again, catching any regressions. Some of the stories would bounce to a “penalty box” in the lower right hand corner of the board. We felt it was important to highlight the shame of something being “accepted” but breaking.
The iteration was done when all of the stories were simultaneously “accepted.”
So the columns were:
- Story
- Accept-able
- Sized:
- (upper) Not Assigned
- (lower) Assigned
- Test-able
- Acceptance:
- (upper) Accepted
- (lower) Penalty Box
The Benefits
This process was really light weight. That made it possible for us to have more iterations within the project; we worked on one and two week iterations.
The visibility of the cork board was invaluable: everyone knew where we were, every day. This created a sense of urgency right from the start of each iteration: “we have to get all this done for next Friday? let’s get those acceptance tests defined right now!”.
The process was rigid where we needed it to be rigid (acceptance tests before estimates), but flexible where we needed to be flexible (negotiating acceptance tests, locking down the set of stories after estimates)
Putting the entire team into binary, project-wide iterations really lowered our integration grief.
The Fine Print
Other methods were used for choosing themes for iterations and managing the project as a whole. This is just how we worked within an iteration. In fact, by strict definition the cork board didn’t manage the iteration so much as help us understand where we were and help us follow a set process for movement: I call this Administration rather than Management.
“Now a date was set for the entire iteration and coding would begin.”: That’s a lot of hand waving! Deciding what to include and what to drop is a really complex weaving a deep understanding of the project risk factors, a desire to show constant visible progress (“Demo or Die”), negotiation with stakeholders, negotiation with the developers… A very hard problem.
(Many thanks to Eric Torreborre, whose post Writing on the Walls reminded me how important the physical manifestation of a project is to getting the project done.)
Labels: agile
Tuesday, December 05, 2006
Five things customers want to hear from you
When we’re in front of executives, we quickly learned to talk about only five things:When we work with teams, we teach them to follow the money and look for the pain. Somewhere in your organization, someone is feeling pain because they aren’t getting the answers they want to one of the questions above.
- How do we increase revenue?
- How do we reduce expenses?
- How do we bring in more customers?
- How do we get more business out of each existing customer?
- How do we increase shareholder value?
Labels: agile
