(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)
Thursday, January 18, 2007
Business programming standards have become higher in 2007. Learn to love it.
From time to time people suggest that fundamental computer science familiarity is irrelevant to the work of a business programmer. I am talking about a knowledge of recursion, operations on data structures, code generation, and other topics that are often derided as being “unnecessary” in a business programming context.
Hmmm.
I think this is wrong in 2007. It may not have been wrong in 2002, perhaps such knowledge was a bonus but not a basic requirement. But today, I think you need to have it. And I don’t mean, you need to have it on your resumé. I mean, you will use it on your job from time to time.
Now, I know that some readers are shaking their heads, no. And some are nodding their heads, yes. It’s easy to think this is all about culture, and some kind of weird hacker fraternity, or whatever. It’s especially easy to dismiss stuff you never use: if you never needed it before, why would you need it now?
That’s the Blub talking. Your toolbox is good enough because you've never needed anything else to do a job... up to now.
No matter what you think of Lisp, Google-style interviews (do you remember when they were “Microsoft-style interviews”?), or optimizing code, please put that aside and try to read this post as objectively as possible. I’ll lay out my thinking for you.
When you say “these things are not relevant for the job,” how do you know? Ok, you have twenty years of experience. And you’ve never used recursion or you’ve only used it once or twice. So you won’t need it now, you’ll find a way to use iteration. And who cares what a suffix array is, you’ll look it up in wikipedia if you need to implement one.
That’s what people have done for quite a while: wire existing frameworks together. What programmers need to know is how to Google stuff, and what a programmer need on her desktop is an IDE that auto-completes stuff so she’ll know what the methods are called. And of course, static typing to make sure she gets the method parameters right. Good to go.
Well, there are jobs out there like that. Last year’s jobs. But how do you know the one I’m filling in 2007 is one of those? Because I told you that for this position we are working with one of the world’s largest financial institutions on a public-facing J2EE application that has been in service for more than five years?
Now, I agree that you have figured out 98% of what we do. Most of the time, we mess with XSLT, message queues, JDBC through a DAL, and other buzzword compliant tasks. The keys to success for those items is less about programming brilliance than around discipline, process, requirements management, and the other stuff the “no hard CS” folks want to talk about in the interview.
And of course, we care about skills in those areas. We have to, we can’t hire someone who can distribute data sets across a grid but is unable to negotiate requirements effectively with a business analyst. But let’s talk about the other 2%.
The top two percent
From time to time we get some challenges. Here are some recent examples:
As part of a refactoring effort last year, we wrote some Java that used Reflection and Dynamic Proxies to replace an entire layer of the application that used to include extensive hand-coding of stuff that was repetitive and error-prone. This saves us 80-90% of the code in that layer when we add new stuff to the application. A testing utility the year before used Reflection to automatically write a certain JUnit suite.
We’re working on something right now that is highly performant. We have a seven-figure user base, and peak loads are intense. You know how bit-twiddling in Java is irrelevant because you’re waiting for the database any ways? Well, we can’t afford to wait for this particular function, it’s one of those AJAX-y things that happens in real time. We can’t wait to go back to the database. So we have to load something into memory on the server, build a compact data structure, and traverse it quickly. And oh yes, we can’t have a lot of layers of crap, we need to get a response back to the browser with every key press.
A Little Java, a Few Patterns: The authors of The Little Schemer and The Little MLer bring deep and important insights to the Java language. Every serious Java programmer should own a copy.For another client, we had to build a task dispatching system. It was like building a piece of a very lightweight fault-tolerant operating system. That operated across data centres in three different cities, moving jobs around from centre to centre. If you were in an interview and someone posed one of those hypothetical “how would you design a …” problems, would you think they stole the problem from some Amazon programmer’s weblog? Would you think “you don’t need that for business programming?” Well, we built that. For an ordinary, brick and mortar business that makes physical stuff.
That 2% figure? That was in 2006. In 2007, it will be way more. Standards are rising. We’re doing more and more work that steps outside of the usual CRUD development.
Here’s the big reason why. Have you read people grousing about interviews where they’re asked about how to implement a search? And about what a waste it is because 99% of the time the database does it, and the rest of the time they stick it in a hash table? Well, in 2007 search matters. The database is a big part of that, but it’s not as easy as
The Google Effect
Google has become the “start page of the internet.” As a result, everyone now thinks that the way to find stuff is to do a full text search. Everyone thinks that relevant results should be first. And I mean everyone, not just your “early adopter” users, you now have Joe Average calling your customer support hotline and complaining that the search page on your application—the one with a different field for each column in each table—is too hard and why can’t he just type something and get an answer?
Just how do you plan to implement full text search? Buy it from Google or Oracle? And do you think you can do the “Google Suggest” thing with the drop down? In real time?
Users now love having a single search box. They don’t want to have one box for searching on product SKU and a drop-down for searching on supplier name. One box. And if you want to make searching for supplier name easy, give them an auto-complete. Users don’t like to be given an application that basically has the implementation protruding into the interface.
They don’t care that you store first name and last name in separate columns, they want to search for “Reg Braithwaite” and find him, even if “Reginald” is what’s stored in the first name column and there are 3,215 Braithwaites in the table. You figure it out, possibly by word stemming, possibly by statistical analysis. Or maybe you’re just doing some untrained bayesian classification to cluster the “Reginald Braithwaite” record with some other things the user is looking at right now so you put that record at the top of the list you returned.
Hmmm, we’re not in Kansas any more. It isn’t all about
This stuff isn’t rocket science. And you don’t need Common Lisp or Haskell to pull it off. You can do it in Java or C#: we do it and there are thousands of places just like ours where people just like us are doing it every day.
But in 2007, you do need to let go of the idea that all we’re doing with “business programming” is building web applications that are replacing the client-server applications of the eighties and nineties that themselves replaced the green screen terminal applications of the seventies and eighties.
The “leading edge” interface and ideas employed by Google, Amazon, eBay, and Yahoo! are suffusing our culture to become the standard user interface of web applications. And programming the standard user interface is a basic job requirement. Learn to love it.
Do you love applications like Google Mail? Would you like to write stuff like this, even if it’s less than 100% of the time? But are you looking for a stable company working on stuff you can explain to your neighbors? Michael Lucas is hiring intermediate and senior developers for positions in Toronto, Canada. To be considered for a position, please send Michael an email with your answer to the following question:
From time to time people suggest that fundamental computer science familiarity is irrelevant to the work of a business programmer. I am talking about a knowledge of recursion, operations on data structures, code generation, and other topics that are often derided as being “unnecessary” in a business programming context.
Hmmm.
I think this is wrong in 2007. It may not have been wrong in 2002, perhaps such knowledge was a bonus but not a basic requirement. But today, I think you need to have it. And I don’t mean, you need to have it on your resumé. I mean, you will use it on your job from time to time.
Now, I know that some readers are shaking their heads, no. And some are nodding their heads, yes. It’s easy to think this is all about culture, and some kind of weird hacker fraternity, or whatever. It’s especially easy to dismiss stuff you never use: if you never needed it before, why would you need it now?
That’s the Blub talking. Your toolbox is good enough because you've never needed anything else to do a job... up to now.
No matter what you think of Lisp, Google-style interviews (do you remember when they were “Microsoft-style interviews”?), or optimizing code, please put that aside and try to read this post as objectively as possible. I’ll lay out my thinking for you.
When you say “these things are not relevant for the job,” how do you know? Ok, you have twenty years of experience. And you’ve never used recursion or you’ve only used it once or twice. So you won’t need it now, you’ll find a way to use iteration. And who cares what a suffix array is, you’ll look it up in wikipedia if you need to implement one.
That’s what people have done for quite a while: wire existing frameworks together. What programmers need to know is how to Google stuff, and what a programmer need on her desktop is an IDE that auto-completes stuff so she’ll know what the methods are called. And of course, static typing to make sure she gets the method parameters right. Good to go.
There are jobs out there like that. Last year’s jobs.
Well, there are jobs out there like that. Last year’s jobs. But how do you know the one I’m filling in 2007 is one of those? Because I told you that for this position we are working with one of the world’s largest financial institutions on a public-facing J2EE application that has been in service for more than five years?
Now, I agree that you have figured out 98% of what we do. Most of the time, we mess with XSLT, message queues, JDBC through a DAL, and other buzzword compliant tasks. The keys to success for those items is less about programming brilliance than around discipline, process, requirements management, and the other stuff the “no hard CS” folks want to talk about in the interview.
And of course, we care about skills in those areas. We have to, we can’t hire someone who can distribute data sets across a grid but is unable to negotiate requirements effectively with a business analyst. But let’s talk about the other 2%.
The top two percent
From time to time we get some challenges. Here are some recent examples:
As part of a refactoring effort last year, we wrote some Java that used Reflection and Dynamic Proxies to replace an entire layer of the application that used to include extensive hand-coding of stuff that was repetitive and error-prone. This saves us 80-90% of the code in that layer when we add new stuff to the application. A testing utility the year before used Reflection to automatically write a certain JUnit suite.
You know how bit-twiddling in Java is irrelevant because you’re waiting for the database any ways? Well, we can’t afford to wait for this particular function, it’s one of those AJAX-y things that happens in real time. We can’t wait to go back to the database.
We’re working on something right now that is highly performant. We have a seven-figure user base, and peak loads are intense. You know how bit-twiddling in Java is irrelevant because you’re waiting for the database any ways? Well, we can’t afford to wait for this particular function, it’s one of those AJAX-y things that happens in real time. We can’t wait to go back to the database. So we have to load something into memory on the server, build a compact data structure, and traverse it quickly. And oh yes, we can’t have a lot of layers of crap, we need to get a response back to the browser with every key press.
A Little Java, a Few Patterns: The authors of The Little Schemer and The Little MLer bring deep and important insights to the Java language. Every serious Java programmer should own a copy.
That 2% figure? That was in 2006. In 2007, it will be way more. Standards are rising. We’re doing more and more work that steps outside of the usual CRUD development.
Here’s the big reason why. Have you read people grousing about interviews where they’re asked about how to implement a search? And about what a waste it is because 99% of the time the database does it, and the rest of the time they stick it in a hash table? Well, in 2007 search matters. The database is a big part of that, but it’s not as easy as
SELECT foo.* FROM bar WHERE ... any more.The Google Effect
Google has become the “start page of the internet.” As a result, everyone now thinks that the way to find stuff is to do a full text search. Everyone thinks that relevant results should be first. And I mean everyone, not just your “early adopter” users, you now have Joe Average calling your customer support hotline and complaining that the search page on your application—the one with a different field for each column in each table—is too hard and why can’t he just type something and get an answer?
This stuff isn’t rocket science. And you don’t need Common Lisp or Haskell to pull it off. You can do it in Java or C#.
Just how do you plan to implement full text search? Buy it from Google or Oracle? And do you think you can do the “Google Suggest” thing with the drop down? In real time?
Users now love having a single search box. They don’t want to have one box for searching on product SKU and a drop-down for searching on supplier name. One box. And if you want to make searching for supplier name easy, give them an auto-complete. Users don’t like to be given an application that basically has the implementation protruding into the interface.
They don’t care that you store first name and last name in separate columns, they want to search for “Reg Braithwaite” and find him, even if “Reginald” is what’s stored in the first name column and there are 3,215 Braithwaites in the table. You figure it out, possibly by word stemming, possibly by statistical analysis. Or maybe you’re just doing some untrained bayesian classification to cluster the “Reginald Braithwaite” record with some other things the user is looking at right now so you put that record at the top of the list you returned.
Hmmm, we’re not in Kansas any more. It isn’t all about
has_one, has_many, or has_and_belongs_to_many, and you don’t have to be a high-profile start up to care. Jane Average uses stuff like this when she reads her mail and books her vacation. But does her office HR support application work half as well? Why not?This stuff isn’t rocket science. And you don’t need Common Lisp or Haskell to pull it off. You can do it in Java or C#: we do it and there are thousands of places just like ours where people just like us are doing it every day.
But in 2007, you do need to let go of the idea that all we’re doing with “business programming” is building web applications that are replacing the client-server applications of the eighties and nineties that themselves replaced the green screen terminal applications of the seventies and eighties.
The “leading edge” interface and ideas employed by Google, Amazon, eBay, and Yahoo! are suffusing our culture to become the standard user interface of web applications. And programming the standard user interface is a basic job requirement. Learn to love it.
Do you love applications like Google Mail? Would you like to write stuff like this, even if it’s less than 100% of the time? But are you looking for a stable company working on stuff you can explain to your neighbors? Michael Lucas is hiring intermediate and senior developers for positions in Toronto, Canada. To be considered for a position, please send Michael an email with your answer to the following question:
Name three features from public web ‘sites’ like Google, Amazon, and YouTube (you can pick any site or sites you like) that will make the jump to business applications in 2007.
Comments on “Business programming standards have become higher in 2007. Learn to love it.”:
<< Home
What a stack of self wankery.
Knowing how to write cooler software is good. Uh, yes. It is. That's not much of a point, though.
Purposefully making your code 'clever' and hard to work with without understanding it deeply - that's just stupid. Memories of "write a cool one-liner in BASIC" come to mind. Pointless.
That random jab at static typing somewhere in the middle was hypocritical to absurd, in fact: If you're going to jump into the deep end and actually write interesting code, why in hell's often busied name would you want to ignore or even marginalize the benefits of static typing?
You're going to have to fit all that tricky code into a larger framework. You're not the only one hacking away at this giant J2EE thing, and it won't be a one month job where you can keep the relevant details in your brain for the full course of the project. Static typing is key... and it's an entirely separate issue from something like recursion.
I seriously don't get it. Why IS there such a 'pick one or the other' mentality here? You can have your cake and you can eat it too.
Just yesterday I spent some time brainstorming on an issue (and ended up with a cool solution): The ONLY point to the session was to move some sanity check away from a runtime check and into compiler check territory. That's 2 hours very very well spent. We eliminated a boatload of documentation (because now auto-complete and on-the-fly compiler errors automatically force you into doing the right thing with it) and this thing will be used for years to come.
In fact, in order to make it work I had to dive down into some fancy footwork, well removed from the usual simplistic territory you seem to dislike so much. I do too. That's not the point.
The point is: You can have em both. Better yet: You must have them both if you want to get away with fancy code.
“Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?” -Brian Kernighan
Oh, won't someone think of the debugging?
Knowing how to write cooler software is good. Uh, yes. It is. That's not much of a point, though.
Purposefully making your code 'clever' and hard to work with without understanding it deeply - that's just stupid. Memories of "write a cool one-liner in BASIC" come to mind. Pointless.
That random jab at static typing somewhere in the middle was hypocritical to absurd, in fact: If you're going to jump into the deep end and actually write interesting code, why in hell's often busied name would you want to ignore or even marginalize the benefits of static typing?
You're going to have to fit all that tricky code into a larger framework. You're not the only one hacking away at this giant J2EE thing, and it won't be a one month job where you can keep the relevant details in your brain for the full course of the project. Static typing is key... and it's an entirely separate issue from something like recursion.
I seriously don't get it. Why IS there such a 'pick one or the other' mentality here? You can have your cake and you can eat it too.
Just yesterday I spent some time brainstorming on an issue (and ended up with a cool solution): The ONLY point to the session was to move some sanity check away from a runtime check and into compiler check territory. That's 2 hours very very well spent. We eliminated a boatload of documentation (because now auto-complete and on-the-fly compiler errors automatically force you into doing the right thing with it) and this thing will be used for years to come.
In fact, in order to make it work I had to dive down into some fancy footwork, well removed from the usual simplistic territory you seem to dislike so much. I do too. That's not the point.
The point is: You can have em both. Better yet: You must have them both if you want to get away with fancy code.
“Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?” -Brian Kernighan
Oh, won't someone think of the debugging?
Reiner:
Thank you for your candid—as usual—response. It is very valuable, because it shows that the initial reaction to any "out of the box" thinking is usually "hey, those people are playing with themselves."
But the fact is, we are working to what our clients need. And theyin turn are working to what the market expects. I Don't know if you missed the whole message: we don't invent this stuff, we are merely keeping up with rising expectations.
To address the very reasonable question about maintenance, the motivation for the reflective code was to make the code easier to maintain. A small bit of Java metaprogramming eliminated a situation where five or six classes were all tightly coupled for each bit of stuff we have in the application.
This made the code easier to read. Oh sure, the reflective stuff takes a little head scratching the first time. But it isn't anything that would trouble anyone on the team or reading this blog.
The very point of this article is that this stuff is not cool: it's business as usual, circa 2007. I said outright: it is not rocket science. I said outright: thousands of shops are doing this stuff. In Java. And C#. People just like you and I and all of the other readers of this blog.
So the message is: This is the reality of business programming. STOP thinking of it as wankery. START thinking of it as part of the job.
And yes, I often think of the debugging and of how to do what we used to call "type torture" in my C++ days to get the compiler's type checking system to perform constraint checking above and beyond simple interface compatibility.
And no, I don't dislike the day-to-day stuff that hasn't changed much since I used B-trees with Borland Pascal to write business applications. It's just that there's very little of it that would make a blog interesting, is there?
Now, to respond to the rather emotional portion of your comment:
Perhaps I was not clear about something? This code base is J2EE, as in Java, as in statically typed Java.
Not Java plus cool stuff, or cool stuff instead of Java. Just Java. Straight out of the box. It has static typing. So what jab are you thinking of?
Was I unclear? Did I say that we build this stuff in Ruby or Scheme or Python because we don't know how to build it with Java?
Look, I'll say it again. This is nothing special. It's not "we're bleeding edge." It's a proposition that we're just like you and everyone else and this is the job in 2007.
Not because it tickles my fancy—Ruby tickles my fanacy a lot more than Java—but because Google, Amazon, and whomever else out there with the genius IQ programmers are training millions of your users and our users to expect something better than what we've delivered over the last decade.
Pax.
Thank you for your candid—as usual—response. It is very valuable, because it shows that the initial reaction to any "out of the box" thinking is usually "hey, those people are playing with themselves."
But the fact is, we are working to what our clients need. And theyin turn are working to what the market expects. I Don't know if you missed the whole message: we don't invent this stuff, we are merely keeping up with rising expectations.
To address the very reasonable question about maintenance, the motivation for the reflective code was to make the code easier to maintain. A small bit of Java metaprogramming eliminated a situation where five or six classes were all tightly coupled for each bit of stuff we have in the application.
This made the code easier to read. Oh sure, the reflective stuff takes a little head scratching the first time. But it isn't anything that would trouble anyone on the team or reading this blog.
The very point of this article is that this stuff is not cool: it's business as usual, circa 2007. I said outright: it is not rocket science. I said outright: thousands of shops are doing this stuff. In Java. And C#. People just like you and I and all of the other readers of this blog.
So the message is: This is the reality of business programming. STOP thinking of it as wankery. START thinking of it as part of the job.
And yes, I often think of the debugging and of how to do what we used to call "type torture" in my C++ days to get the compiler's type checking system to perform constraint checking above and beyond simple interface compatibility.
And no, I don't dislike the day-to-day stuff that hasn't changed much since I used B-trees with Borland Pascal to write business applications. It's just that there's very little of it that would make a blog interesting, is there?
Now, to respond to the rather emotional portion of your comment:
Perhaps I was not clear about something? This code base is J2EE, as in Java, as in statically typed Java.
Not Java plus cool stuff, or cool stuff instead of Java. Just Java. Straight out of the box. It has static typing. So what jab are you thinking of?
Was I unclear? Did I say that we build this stuff in Ruby or Scheme or Python because we don't know how to build it with Java?
Look, I'll say it again. This is nothing special. It's not "we're bleeding edge." It's a proposition that we're just like you and everyone else and this is the job in 2007.
Not because it tickles my fancy—Ruby tickles my fanacy a lot more than Java—but because Google, Amazon, and whomever else out there with the genius IQ programmers are training millions of your users and our users to expect something better than what we've delivered over the last decade.
Pax.
In 2002, using recursion was seen as "irrelevant" in a business context?
Opinions do vary. Without dragging my own opinion into it, let's play the "JavaSchools" test. Here's how it works:
If you can get a job in the industry while openly stating that you never use X, and that while X is nifty, you can always find a work-around for X, X is irrelevant.
You aren't rude about X, you don't sneer at X users, you just don't use it.
I think in those terms, something like thin client programming slash browser-based application programming was irrelevant for a while, but then became a part of the core business requirements for a programmer.
My guess is that even today you can find and keep a job while stating that you haven't touched recursion since doing some University coursework.
Am I unrealistic?
p.s. The JavaSchools test does not imply that knowledge of a particular technique or tool is not useful or even powerful.
Opinions do vary. Without dragging my own opinion into it, let's play the "JavaSchools" test. Here's how it works:
If you can get a job in the industry while openly stating that you never use X, and that while X is nifty, you can always find a work-around for X, X is irrelevant.
You aren't rude about X, you don't sneer at X users, you just don't use it.
I think in those terms, something like thin client programming slash browser-based application programming was irrelevant for a while, but then became a part of the core business requirements for a programmer.
My guess is that even today you can find and keep a job while stating that you haven't touched recursion since doing some University coursework.
Am I unrealistic?
p.s. The JavaSchools test does not imply that knowledge of a particular technique or tool is not useful or even powerful.
The thing about recursion is if you know what it is, you typically know how to use it. So, that rules out the bright people who haven't used it in years and just leaves the people who never knew how to use it in the first place. I wouldn't hire them.
I would definitely excuse being rusty in other areas, though, if I was confident that the person could relearn the material.
I would definitely excuse being rusty in other areas, though, if I was confident that the person could relearn the material.
# posted by 
: 10:27 PM
: 10:27 PM
I was referring to:
What programmers need to know is how to Google stuff, and what a programmer need on her desktop is an IDE that auto-completes stuff so she’ll know what the methods are called. And of course, static typing to make sure she gets the method parameters right. Good to go.
This post still counts as self-wankery because, while the revelation that cooler code* is now, apparently, part of the job, is certainly post-worthy, the entire remainder breaks down in -admittedly eloquent, but nonetheless- petty namecalling.
'blub', 'you think it's good enough', 'you shake your head no', 'you never needed anything else', 'no matter what you think of lisp'... need I go on? that's just from 4 sentences.
There's absolutely no snowball's chance in hell this will convince anyone who ascribes to the theory that recursion is pointless. If those people even exist. I have no problem with the message. I have a problem with how you preached it.
Where's the actual anecdotal evidence** to support your theory that you need fancy footwork these days? Fortunately at least, it's there, but you have to hit page down a couple of times to even get there. You've lost the ones you appear to be preaching to by then.
Which means.. it's wankery. It's a lot of people reading this and vehemently agreeing with each other, which is the exact, and only, qualification a post needs to qualify as wankery.
*) I'd still like some actual proof, or even just observations to support this theory, beyond the observation that global text search is hard (I don't think so, that's a matter of someone writing a library). That would have been a cool post.
**) I know anecdotal evidence doesn't work because you can find cases to support anything, but it can be fun to read, and it can make you think.
What programmers need to know is how to Google stuff, and what a programmer need on her desktop is an IDE that auto-completes stuff so she’ll know what the methods are called. And of course, static typing to make sure she gets the method parameters right. Good to go.
This post still counts as self-wankery because, while the revelation that cooler code* is now, apparently, part of the job, is certainly post-worthy, the entire remainder breaks down in -admittedly eloquent, but nonetheless- petty namecalling.
'blub', 'you think it's good enough', 'you shake your head no', 'you never needed anything else', 'no matter what you think of lisp'... need I go on? that's just from 4 sentences.
There's absolutely no snowball's chance in hell this will convince anyone who ascribes to the theory that recursion is pointless. If those people even exist. I have no problem with the message. I have a problem with how you preached it.
Where's the actual anecdotal evidence** to support your theory that you need fancy footwork these days? Fortunately at least, it's there, but you have to hit page down a couple of times to even get there. You've lost the ones you appear to be preaching to by then.
Which means.. it's wankery. It's a lot of people reading this and vehemently agreeing with each other, which is the exact, and only, qualification a post needs to qualify as wankery.
*) I'd still like some actual proof, or even just observations to support this theory, beyond the observation that global text search is hard (I don't think so, that's a matter of someone writing a library). That would have been a cool post.
**) I know anecdotal evidence doesn't work because you can find cases to support anything, but it can be fun to read, and it can make you think.
Reinier:
I think you're getting all worked up over very little. It's just an observation that the bar is rising. It isn't an insult.
I don't know why the word "blub" seems to raise your hackles so much. I have chosen to write Java code for a profession.
I don't suggest that all Java programmers are programming in blub. In fact, I said just the opposite. And although I love Scheme, I have dared to suggest that Lisp could be Blub.
As for it being about cooler code, I say definitely no, because the definition of cool is that everybody isn't doing it.
This post is saying that everybody is doing it, or will be doing it in 2007.
I think what we are looking at is something a little like the Moore's "crossing the chasm" adoption model. What I'm saying is that technologies and approaches that were "early adopter" stuff five years ago are going mainstream this year, mostly because of Google.
I could have made the same point in a sober, Gartner Group-ish pseudo-academic and completely impersonal way. I chose to write in a personal way.
You seem to agree somewhat with the content but do not care for the way it is presented. Is that right?
If so, I would say this is the normal state for writing of this nature. I am not writing a marketing piece, or a business report presented to a client for sober review. It's a weblog!
As a blogger yourself, you must write plenty of things that your readers would have written themselves in a different style.
I think you're getting all worked up over very little. It's just an observation that the bar is rising. It isn't an insult.
I don't know why the word "blub" seems to raise your hackles so much. I have chosen to write Java code for a profession.
I don't suggest that all Java programmers are programming in blub. In fact, I said just the opposite. And although I love Scheme, I have dared to suggest that Lisp could be Blub.
As for it being about cooler code, I say definitely no, because the definition of cool is that everybody isn't doing it.
This post is saying that everybody is doing it, or will be doing it in 2007.
I think what we are looking at is something a little like the Moore's "crossing the chasm" adoption model. What I'm saying is that technologies and approaches that were "early adopter" stuff five years ago are going mainstream this year, mostly because of Google.
I could have made the same point in a sober, Gartner Group-ish pseudo-academic and completely impersonal way. I chose to write in a personal way.
You seem to agree somewhat with the content but do not care for the way it is presented. Is that right?
If so, I would say this is the normal state for writing of this nature. I am not writing a marketing piece, or a business report presented to a client for sober review. It's a weblog!
As a blogger yourself, you must write plenty of things that your readers would have written themselves in a different style.
I've always had a certain fondness for hard problems, because the payoff for solving hard problems is often higher than the payoff for solving easy ones.
- Solving a hard problem can help someone make a lot of money--it can provide a competitive advantage to a business.
- Solving a hard problem can teach me something.
- Solving one of today's hard problems often gives me some insight into tomorrow's easy problems.
(It's also worth remembering that solving an easy problem really well is typically a hard problem!)
So I'm pleased that the average business application is getting harder: We're digging deeper to help our users work faster, to help them remain in a flow state, and to allow them to treat our software as a extension of their thinking process.
But just for a moment, let's stop thinking about 2007, and let's start thinking about 2012. What issues will we need to deal with next? Here are a few which leap to mind:
a) Google does lots of cool stuff: everything from Bayesian analysis, to clustering, to natural language parsing, to linear algebra.
Today, these are difficult techniques to apply: Most involve math, and all of them involve piles of yellow code. What would it look like if these techniques were "green code"--something we could express concisely, with no implementation crud? What would it look like?
b) Given that clock speeds have stabilized, but also that transistor counts keep climbing exponentially, we'll be looking at 80 hardware threads all too soon. We're so not ready for this world. The current toolbox of locks and semaphores and threads is way too fiddly, and it only scales to 100 cores with major effort.
Again, what would we have to change for us to be happy in such a world? Do we need to rethink our notion of mutable state? Do we need ways to trivially guarantee the absence of deadlocks and race conditions?
Strangely, Lisp doesn't offer me much help in answering these questions. Sure, I can build anything with macros, but what's the right thing to build? And given that Lisp has exactly one kind of mutable state (with no restrictions), how do I even begin to prove a Lisp program free of race conditions?
I need new languages to think in--not languages that express my current ideas exceedingly well, or even languages that could express any idea in theory, but languages that bend my thoughts into strange new shapes.
- Solving a hard problem can help someone make a lot of money--it can provide a competitive advantage to a business.
- Solving a hard problem can teach me something.
- Solving one of today's hard problems often gives me some insight into tomorrow's easy problems.
(It's also worth remembering that solving an easy problem really well is typically a hard problem!)
So I'm pleased that the average business application is getting harder: We're digging deeper to help our users work faster, to help them remain in a flow state, and to allow them to treat our software as a extension of their thinking process.
But just for a moment, let's stop thinking about 2007, and let's start thinking about 2012. What issues will we need to deal with next? Here are a few which leap to mind:
a) Google does lots of cool stuff: everything from Bayesian analysis, to clustering, to natural language parsing, to linear algebra.
Today, these are difficult techniques to apply: Most involve math, and all of them involve piles of yellow code. What would it look like if these techniques were "green code"--something we could express concisely, with no implementation crud? What would it look like?
b) Given that clock speeds have stabilized, but also that transistor counts keep climbing exponentially, we'll be looking at 80 hardware threads all too soon. We're so not ready for this world. The current toolbox of locks and semaphores and threads is way too fiddly, and it only scales to 100 cores with major effort.
Again, what would we have to change for us to be happy in such a world? Do we need to rethink our notion of mutable state? Do we need ways to trivially guarantee the absence of deadlocks and race conditions?
Strangely, Lisp doesn't offer me much help in answering these questions. Sure, I can build anything with macros, but what's the right thing to build? And given that Lisp has exactly one kind of mutable state (with no restrictions), how do I even begin to prove a Lisp program free of race conditions?
I need new languages to think in--not languages that express my current ideas exceedingly well, or even languages that could express any idea in theory, but languages that bend my thoughts into strange new shapes.
Hi, Neel! It's been a while.
The nested data parallel constructs look really interesting, and I need to spend some time studying them in detail. Thanks for the pointer!
But on another level, I don't think that there's any single answer for achieving large-scale concurrency. Tim Sweeney, for example, argues that he needs multiple models of concurrency, ranging from massive data parallelism to 10,000+ heterogeneous objects with lots of state.
But I'm starting to veer off-topic. :-) Reginald is right that hard problems are starting to crop of everywhere, and that we'll see even more in the future. It's fun (i.e., scary and exhilarating) to imagine a world where today's hard problems are routine.
The nested data parallel constructs look really interesting, and I need to spend some time studying them in detail. Thanks for the pointer!
But on another level, I don't think that there's any single answer for achieving large-scale concurrency. Tim Sweeney, for example, argues that he needs multiple models of concurrency, ranging from massive data parallelism to 10,000+ heterogeneous objects with lots of state.
But I'm starting to veer off-topic. :-) Reginald is right that hard problems are starting to crop of everywhere, and that we'll see even more in the future. It's fun (i.e., scary and exhilarating) to imagine a world where today's hard problems are routine.
Hi Eric!
Yeah, I don't think that there is one solution, either, but nested data parallelism has an amazing power-to-weight ratio. I found out about it January of last year, over a lunch conversation, and it's now something that I believe I must keep in mind when designing a new language. It won't necessarily go in version 1, or even 2.0, but I better make sure I don't make adding it impossible.The Fortress people are apparently thinking hard about adding support for it, too, which makes sense given their problem space.
But I suspect we're all going to be in that world really soon, too -- keeping a hundred cores occupied without allowing it to diver too much attention from the real problems looks like the big difficulty.
Yeah, I don't think that there is one solution, either, but nested data parallelism has an amazing power-to-weight ratio. I found out about it January of last year, over a lunch conversation, and it's now something that I believe I must keep in mind when designing a new language. It won't necessarily go in version 1, or even 2.0, but I better make sure I don't make adding it impossible.The Fortress people are apparently thinking hard about adding support for it, too, which makes sense given their problem space.
But I suspect we're all going to be in that world really soon, too -- keeping a hundred cores occupied without allowing it to diver too much attention from the real problems looks like the big difficulty.
Reg, any other blogger, sure. But you've been raising my expectations which means this kind of thing can't stand without a few pointed comments to show you that you've missed one.
The problem with the presentation of your point is that there's nothing particularly interesting in it. No revelations. Nothing to make me think. It drowns in the sea of 'look at me, Im'a doing the new and hip thing!' posts that abound on the web (those annoy me, which is probably why you picked up a defensive tone in my previous comments).
I'm actually tempted to hang the other way, even: Because google exists, it's much easier to find the one that has gone before you, and you can just copy their code and modify it to your needs.
The problem with the presentation of your point is that there's nothing particularly interesting in it. No revelations. Nothing to make me think. It drowns in the sea of 'look at me, Im'a doing the new and hip thing!' posts that abound on the web (those annoy me, which is probably why you picked up a defensive tone in my previous comments).
I'm actually tempted to hang the other way, even: Because google exists, it's much easier to find the one that has gone before you, and you can just copy their code and modify it to your needs.
It drowns in the sea of 'look at me, Im'a doing the new and hip thing!'
Well, there's something I can refute with relish: what makes you think that I consider the kinds of things I mentioned in this post to be either new or hip?
;-)
Sure, in 2003 or 2004 these things were new and worthy of admiration. But today? Not particularly difficult, and I don't think anyone with a real education in Computer Science would have trouble with the algorithms.
I agree 100% that there's nothing new or challenging in 2007: My agenda with this post was to try to explain why I consider these kinds of things the minimal competency level for an intermediate position on our team.
I don't expect anything from someone with your experience other than "well, duh." If you want to hold me to a higher standard of content, I am flattered.
Because google exists, it's much easier to find the one that has gone before you, and you can just copy their code and modify it to your needs.
I don't care where the code that a programmer checks into svn comes from, but surely we can agree she has to understand it?
I wouldn't want to ask "why did you do such-and-such" only to hear I have no idea, that's how the code was written when I copied it?
Well, there's something I can refute with relish: what makes you think that I consider the kinds of things I mentioned in this post to be either new or hip?
;-)
Sure, in 2003 or 2004 these things were new and worthy of admiration. But today? Not particularly difficult, and I don't think anyone with a real education in Computer Science would have trouble with the algorithms.
I agree 100% that there's nothing new or challenging in 2007: My agenda with this post was to try to explain why I consider these kinds of things the minimal competency level for an intermediate position on our team.
I don't expect anything from someone with your experience other than "well, duh." If you want to hold me to a higher standard of content, I am flattered.
Because google exists, it's much easier to find the one that has gone before you, and you can just copy their code and modify it to your needs.
I don't care where the code that a programmer checks into svn comes from, but surely we can agree she has to understand it?
I wouldn't want to ask "why did you do such-and-such" only to hear I have no idea, that's how the code was written when I copied it?
Well, depends. If something is nicely wrapped in a library I honestly don't need to know how it works. A generic single text box search plugin of sorts is a textbook example of something that you don't need to grok, to use.
Modularization is nice.
I think it's commendable that your business is moving to a place where you're so innovative that you need good programmers.
However, saying that your average run of the mill business programmer needs to be a programmer wizard is true - but has always been true. I don't think anything changed in 2007.
Live is far easier if your entire programming staff is an expert and motivated at their game. This is a symbotic relationship between those two concepts: Nice code is motivating, and motivated programmers write nice code.
Modularization is nice.
I think it's commendable that your business is moving to a place where you're so innovative that you need good programmers.
However, saying that your average run of the mill business programmer needs to be a programmer wizard is true - but has always been true. I don't think anything changed in 2007.
Live is far easier if your entire programming staff is an expert and motivated at their game. This is a symbotic relationship between those two concepts: Nice code is motivating, and motivated programmers write nice code.
"Business programming" is as meaningless a phrase as "enterprise programming", and to make the distinction between business programming and consumer programming is very narrow minded.
And that interview question made me violently ill.
If you're asking questions like "Name three features from public web ‘sites’ like Google, Amazon, and YouTube that will make the jump to business applications in 2007." you're looking for a marketing person, not a developer.
If you ask questions like that, expect to only get applicants who use "online" as a noun. (We're putting a lot of resources into online.)
And that interview question made me violently ill.
If you're asking questions like "Name three features from public web ‘sites’ like Google, Amazon, and YouTube that will make the jump to business applications in 2007." you're looking for a marketing person, not a developer.
If you ask questions like that, expect to only get applicants who use "online" as a noun. (We're putting a lot of resources into online.)
It's a good article I understand the business ideas is more useful to people If you are interesting. Who are the world’s most influential business thinkers alive today. Who are interesting visit the site business ideas
""Business programming" is as meaningless a phrase as "enterprise programming", and to make the distinction between business programming and consumer programming is very narrow minded."
i agree that in essence they are both software that people use, and therefore similar. but the fact remains that there is a rift between how people making "business software" think and how people making "people software" think.
and i think that usually comes down to when an engineer asks themselves "would i ever use this software?". if the answer is yes, then the quality and innovation shown from the program will be much higher, since the developer takes emotional ownership of the product.
and when i say they ask themselves if they would ever use it, i don't mean putting themselves in somebody elses shoes. i mean would they, as a professional software developer, ever desire to use what they are creating.
i agree that in essence they are both software that people use, and therefore similar. but the fact remains that there is a rift between how people making "business software" think and how people making "people software" think.
and i think that usually comes down to when an engineer asks themselves "would i ever use this software?". if the answer is yes, then the quality and innovation shown from the program will be much higher, since the developer takes emotional ownership of the product.
and when i say they ask themselves if they would ever use it, i don't mean putting themselves in somebody elses shoes. i mean would they, as a professional software developer, ever desire to use what they are creating.
<< Home
