(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)
Tuesday, September 05, 2006
Business programming simply isn't that hard
Business programming simply isn't that hard, right?
Seriously, how hard can it be to wire a SQL database to some basic business rules ("employee HAS-A manager, employee.manager_id != employee.id") and add some AJAXy goodness on top like edit_in_place? Business applications are a simple pyramid with the frameworks doing the heavy lifting at the bottom and the AJAXy goodness on the top providing the sizzle that makes users say "wow." Right?
And needless to say, business applications are borrrring! Nothing about the programming is stimulating. It's all a question of managing requirements, priorities, and figuring out how to cram ninety hours worth of work into a seventy hour week to stave off the outsourcing.
But... Could there be interesting problems to seek out in business? Try a couple of these on for size:
Real time risk management
You have an online banking application. Your goal is to produce a system that can alert a customer service rep when suspicious activity is detected so that they can telephone the customer before the customer has left the web site. How can you detect suspicious activities in real time?
For an example of how this works, consider credit cards. I often make very large computer purchases, and they never bat an eyelash. But when I purchase jewelry for my wife, I get a call. A friend swears she gets the same pattern, but they called her when she purchased an iPod because she doesn't usually purchase electronics.
Your system will notice weird patterns, like a customer who never banks internationally suddenly trying to transfer money to Russia. But take heed: you don't want to be bothering the fur importer who regularly does business in the "Wild Wild East." You need to look for suspicious patterns. And you need to explain it simply enough that a customer service rep knows how to verify the transaction with the customer.
Did I meantion in real time? Visa normally calls me within a few hours. Your goal is to make the call while the customer is still on line.
Let's get our hands dirty
Concrete is funny stuff. From the moment you start mixing it, you have a limited amount of time it can sit in the mixer truck before it has to get poured. Construction sites need so much an hour, no more and no less. A cement company can't send a customer one hundred trucks simultaneously and have them idling on site, they have to arrive just as they're needed.
In major cities, traffic is a problem. So build a system that can dispatch and direct the trucks in real time. Your input will be demand from customers plus real time traffic and road construction reports, which your system will use to update a graph representation of the road network. Naturally, it will search the network in real time for apropriate paths for the trucks. They're outfitted with GPS units, so you will have real time locations for your trucks.
Don't forget to manage the return of the empty trucks to your site. Concrete is a low margin business, so operational efficiency is a competitive advantage. Do a great job, and your company could become the dominant supplier.
update: This isn't just finding the fastest way to get trucks to their destinations. That is not a simple problem, but you must also solve the flow problem: when do the trucks have to leave your yard in order to arrive at each customer's site on schedule? If a customer is ninety minutes away and traffic is building towards ruch hour, you have to leave earlier. How much earlier? Do you send two trucks by different paths at certain times and reroute one of the trucks if the other arrives ahead of schedule? When an accident suddenly clogs the freeway, is your system able to predict how long the freeway will be closed and what the effects will be on alternate routes?
To do this job well, you're going to have to have a really good model of traffic in the city you serve. Don't forget to build the baseball, football, hockey, and basketball schedules into your system, along with big ticket concerts :-)
...his lips are moving, that's how
Consider any large organization that makes big-ticket sales to walk-in prospects. Let's choose car sales. Some prospects will walk in and buy right away, others will never buy. Some will buy, but only if you follow up with a call after they've been in for a test drive.
When's the right time to call them back? If salespeople call too soon, they waste time, make the prospects feel pressured, and their quote ends up being fodder for the competition to beat and steal the business. But of they call too late, the prospect will have bought from someone else.
Build a system that learns over time when to call each prospect. For starters, you'll have access to a survey team that will call a sample of prospects that didn't buy. They will fill out a form about the prospect (age, gender, household income, zip code, car they owned, car they bought, anything else you can gather), along with when they bought.
This becomes your training corpus. Now build a system that the salespeople use to manage callbacks. Your system must suggest when to call the prospect in order to have the best chance of closing the deal.
Yes, your system must be able to train itself. Yes, your system must do "meta-training": if the zip code isn't significant but the age of their existing car is significant, your system must recommend dropping the former from the survey and promoting the latter to the top of the form. And you ought to add new questions from time to time and test them for relevance.
Oh yes, you'll have to infer distributions like the Erlang, Normal, or Binomial for purchase times: it should figure out the most appropriate model for lag between test drive and purchase.
Did I mention the scale? This system should be rolled out to every dealer in North America, and every salesperson on duty will be using it. How many dealers are there? I think you answered that question in your job interview, didn't you?
Well?
Does business programming have to suck?
Business programming simply isn't that hard, right?
Seriously, how hard can it be to wire a SQL database to some basic business rules ("employee HAS-A manager, employee.manager_id != employee.id") and add some AJAXy goodness on top like edit_in_place? Business applications are a simple pyramid with the frameworks doing the heavy lifting at the bottom and the AJAXy goodness on the top providing the sizzle that makes users say "wow." Right?
And needless to say, business applications are borrrring! Nothing about the programming is stimulating. It's all a question of managing requirements, priorities, and figuring out how to cram ninety hours worth of work into a seventy hour week to stave off the outsourcing.
But... Could there be interesting problems to seek out in business? Try a couple of these on for size:
Real time risk management
You have an online banking application. Your goal is to produce a system that can alert a customer service rep when suspicious activity is detected so that they can telephone the customer before the customer has left the web site. How can you detect suspicious activities in real time?
For an example of how this works, consider credit cards. I often make very large computer purchases, and they never bat an eyelash. But when I purchase jewelry for my wife, I get a call. A friend swears she gets the same pattern, but they called her when she purchased an iPod because she doesn't usually purchase electronics.
Your system will notice weird patterns, like a customer who never banks internationally suddenly trying to transfer money to Russia. But take heed: you don't want to be bothering the fur importer who regularly does business in the "Wild Wild East." You need to look for suspicious patterns. And you need to explain it simply enough that a customer service rep knows how to verify the transaction with the customer.
Did I meantion in real time? Visa normally calls me within a few hours. Your goal is to make the call while the customer is still on line.
Let's get our hands dirty
Concrete is funny stuff. From the moment you start mixing it, you have a limited amount of time it can sit in the mixer truck before it has to get poured. Construction sites need so much an hour, no more and no less. A cement company can't send a customer one hundred trucks simultaneously and have them idling on site, they have to arrive just as they're needed.
In major cities, traffic is a problem. So build a system that can dispatch and direct the trucks in real time. Your input will be demand from customers plus real time traffic and road construction reports, which your system will use to update a graph representation of the road network. Naturally, it will search the network in real time for apropriate paths for the trucks. They're outfitted with GPS units, so you will have real time locations for your trucks.
Don't forget to manage the return of the empty trucks to your site. Concrete is a low margin business, so operational efficiency is a competitive advantage. Do a great job, and your company could become the dominant supplier.
update: This isn't just finding the fastest way to get trucks to their destinations. That is not a simple problem, but you must also solve the flow problem: when do the trucks have to leave your yard in order to arrive at each customer's site on schedule? If a customer is ninety minutes away and traffic is building towards ruch hour, you have to leave earlier. How much earlier? Do you send two trucks by different paths at certain times and reroute one of the trucks if the other arrives ahead of schedule? When an accident suddenly clogs the freeway, is your system able to predict how long the freeway will be closed and what the effects will be on alternate routes?
To do this job well, you're going to have to have a really good model of traffic in the city you serve. Don't forget to build the baseball, football, hockey, and basketball schedules into your system, along with big ticket concerts :-)
...his lips are moving, that's how
Consider any large organization that makes big-ticket sales to walk-in prospects. Let's choose car sales. Some prospects will walk in and buy right away, others will never buy. Some will buy, but only if you follow up with a call after they've been in for a test drive.
When's the right time to call them back? If salespeople call too soon, they waste time, make the prospects feel pressured, and their quote ends up being fodder for the competition to beat and steal the business. But of they call too late, the prospect will have bought from someone else.
Build a system that learns over time when to call each prospect. For starters, you'll have access to a survey team that will call a sample of prospects that didn't buy. They will fill out a form about the prospect (age, gender, household income, zip code, car they owned, car they bought, anything else you can gather), along with when they bought.
This becomes your training corpus. Now build a system that the salespeople use to manage callbacks. Your system must suggest when to call the prospect in order to have the best chance of closing the deal.
Yes, your system must be able to train itself. Yes, your system must do "meta-training": if the zip code isn't significant but the age of their existing car is significant, your system must recommend dropping the former from the survey and promoting the latter to the top of the form. And you ought to add new questions from time to time and test them for relevance.
Oh yes, you'll have to infer distributions like the Erlang, Normal, or Binomial for purchase times: it should figure out the most appropriate model for lag between test drive and purchase.
Did I mention the scale? This system should be rolled out to every dealer in North America, and every salesperson on duty will be using it. How many dealers are there? I think you answered that question in your job interview, didn't you?
Well?
Does business programming have to suck?
Labels: popular
Comments on “Business programming simply isn't that hard”:
<< Home
Yes i think it still sucks.
Case 1: This is all about the patterns. Real time notification is simple. So most of the time will be spend bitching what is or is not a "suspicious activity"
Case 2: Route planning is relativly interesting. The only problem here is making it more efficient to make it real time. So just an optimization job.
Case 3: The product will sucks anyways, because it will be too complicated for the sales people to be used efficiently. They are better of with using PostIts and their intuition.
Case 1: This is all about the patterns. Real time notification is simple. So most of the time will be spend bitching what is or is not a "suspicious activity"
Case 2: Route planning is relativly interesting. The only problem here is making it more efficient to make it real time. So just an optimization job.
Case 3: The product will sucks anyways, because it will be too complicated for the sales people to be used efficiently. They are better of with using PostIts and their intuition.
# posted by 
: 6:37 AM
: 6:37 AM
This is all about the patterns.
Obviously.
most of the time will be spend bitching what is or is not a "suspicious activity"
Ummm... Do people sit around a conference table debating a set of rules? Or does an engine infer the patterns from previous activities that were known to be fraudulant?
And again, you can't just 'decide' what is a suspicious activity on a global basis. What is suspicious for one client is normal for another.
And hey, what's the plan for pattern matching? When looking at a history of transactions, you are looking for subsequences, not substrings. It isn't one of the NP-hard problems in subsequence matching, but it isn't trivial to implement.
Route planning is relativly interesting. The only problem here is making it more efficient to make it real time. So just an optimization job
Help me understand what you mean by "just an optimization job." Do you mean, write an algorithm out and then fiddle with the program expression using coding tricks to speed things up?
Or do you mean find ways to distribute your shortest path algorithm over a grid of commodity computers?
Don't forget, you have hundreds of trucks to manage in real time. This is the real world, not some simple programming interview question.
You know, "just an optimization job" is a meaningless statement in Computer Science. Research in algorithms is dominated by optimizing stuff from brute force to something practical for real world data sets.
it will be too complicated for the sales people to be used efficiently
I am aware of applications just like this that are used by sales teams in the real world; it is not at all clear that we do not know how to design software that is simple enough for salespeople to use.
Like cement supply, sales is a ruthless business. Any, I mean any tool that provides a competitive advantage, no matter how small, will be adopted and the adopters will drive the laggards out of business.
The bottom line is that if your algorithm works to provide higher closing rates on callbacks, salespeople will use it. Period.
Obviously.
most of the time will be spend bitching what is or is not a "suspicious activity"
Ummm... Do people sit around a conference table debating a set of rules? Or does an engine infer the patterns from previous activities that were known to be fraudulant?
And again, you can't just 'decide' what is a suspicious activity on a global basis. What is suspicious for one client is normal for another.
And hey, what's the plan for pattern matching? When looking at a history of transactions, you are looking for subsequences, not substrings. It isn't one of the NP-hard problems in subsequence matching, but it isn't trivial to implement.
Route planning is relativly interesting. The only problem here is making it more efficient to make it real time. So just an optimization job
Help me understand what you mean by "just an optimization job." Do you mean, write an algorithm out and then fiddle with the program expression using coding tricks to speed things up?
Or do you mean find ways to distribute your shortest path algorithm over a grid of commodity computers?
Don't forget, you have hundreds of trucks to manage in real time. This is the real world, not some simple programming interview question.
You know, "just an optimization job" is a meaningless statement in Computer Science. Research in algorithms is dominated by optimizing stuff from brute force to something practical for real world data sets.
it will be too complicated for the sales people to be used efficiently
I am aware of applications just like this that are used by sales teams in the real world; it is not at all clear that we do not know how to design software that is simple enough for salespeople to use.
Like cement supply, sales is a ruthless business. Any, I mean any tool that provides a competitive advantage, no matter how small, will be adopted and the adopters will drive the laggards out of business.
The bottom line is that if your algorithm works to provide higher closing rates on callbacks, salespeople will use it. Period.
just work so that you can afford to go out and play ultimate
Why not try to have both stimulating work and recreational activities that please you?
If (and I say if) you're going to work anyways, shouldn't it be interesting and challenging?
Why not try to have both stimulating work and recreational activities that please you?
If (and I say if) you're going to work anyways, shouldn't it be interesting and challenging?
So, I have a bit of background in pattern recognition. Banks have neural networks deployed on fucked-up old mainframes with awful floating-point precision. Visa still uses neural nets as well. At least in the 90s and stuff, whether they thought your transaction(s) is(are) suspicious or not is a completely mechanical, mathematical calculation.
Fraud detection is too hard a problem for a typical "business programmer" who hates math and having to write code that runs as screamingly fast as possible. You train an NN with a given data set and you have basically infinite computational demands.
Machine learning is a huge, fascinating area where all the most interesting practical knowledge is locked away in places like FairIsaac.
Fraud detection is too hard a problem for a typical "business programmer" who hates math and having to write code that runs as screamingly fast as possible. You train an NN with a given data set and you have basically infinite computational demands.
Machine learning is a huge, fascinating area where all the most interesting practical knowledge is locked away in places like FairIsaac.
# posted by : 1:41 PM
: 1:41 PM
Fraud detection is too hard a problem for a typical "business programmer" who hates math and having to write code that runs as screamingly fast as possible.
Luckily for me, my readership is small enough that I do not worry about whether their interests or aptitudes are "typical" :-)
Mechanical or not, neural networks are still far more interesting than vanilla database-backed web sites!
Luckily for me, my readership is small enough that I do not worry about whether their interests or aptitudes are "typical" :-)
Mechanical or not, neural networks are still far more interesting than vanilla database-backed web sites!
Interesting... Good examples.
Anyway, I think this just shows a few interesting 'heavy lifter' libs you can write.
The rest is still boring fluff.
Anyway, I think this just shows a few interesting 'heavy lifter' libs you can write.
The rest is still boring fluff.
# posted by : 10:23 PM
: 10:23 PM
Again, as in the previous post, you have to like what you do. You can look at this in two ways:
1) I can accept that what I do in my day to day sucks and SULK and WALLOW in that fact
OR
2) I can accept that my day to day sucks and attempt to locate the things in my day to day where I can make it not suck.
Number two above is what I would attempt to do in most cases. A given job that sucks might be made more interesting if you focus on the things that need fixing that are interesting in addition to that things you are asked to fix that are boring. I guess that makes me a glass half full type of person.
1) I can accept that what I do in my day to day sucks and SULK and WALLOW in that fact
OR
2) I can accept that my day to day sucks and attempt to locate the things in my day to day where I can make it not suck.
Number two above is what I would attempt to do in most cases. A given job that sucks might be made more interesting if you focus on the things that need fixing that are interesting in addition to that things you are asked to fix that are boring. I guess that makes me a glass half full type of person.
# posted by : 8:54 AM
: 8:54 AM
What the hell is wrong with you people? Business programming doesn't suck, day-to-day gruntwork doesn't suck, and "all the other programming" you have to do doesn't suck... unless you let it.
I'll say it again: if you aren't having fun at your job, you're doing it wrong.
Reg's post nails the fact that, if you boil an 8-to-6 cubefarm coding job down to its constituent ingredients, programming is ultimately, no matter the field, about doing cool stuff with computers.
I respect those of you who look at programming as something to be gotten out of the way quickly so you can enjoy a game of frisbee. This implies to me that you have a life. I've read blogs on the internet about those and I hear it's healthy to have one. I am happy for you, if somewhat intellectually and not out of any sense of real understanding.
But know that your disdain for your job is your own choosing. This post isn't about patterns, not directly. It's about the fact that there is really cool stuff going on in the machine all the time. There are reasons your boss wants those features, reasons your company wants them, reasons your customer wants them.
I am working in a niche of the industry that really doesn't interest me, to be honest. But the people in that niche have problems with their software, and that does. They have data here, but need it over there. They need their data stored in 78 different tables that can't be pulled up in less than 32 SQL queries, their dialup connections need 1500ms per query, and they need all this data retrieved in less than 500ms.
I don't go home thinking how boring my job or my company or my industry is. I go home hopped up like a monkey on crack that I figured out how to turn a caching algorithm inside out and turned "the impossible 48-second barrier" into "the 5-second barrier, but we're going to have Dave look at that before we call it impossible."
And the fact that the customers use that part of the software live while talking to their customers means the problem I solved is making people money somewhere. You're asking how can I find this work so interesting, but I'm boggling right back at you: How do you not throw your hands up and sing the "I Rule" song at the top of your lungs when you make the compiler sit up and beg?
Business programming isn't boring. It's programming, and that's not boring. If you're not having fun, you're doing it wrong.
I'll say it again: if you aren't having fun at your job, you're doing it wrong.
Reg's post nails the fact that, if you boil an 8-to-6 cubefarm coding job down to its constituent ingredients, programming is ultimately, no matter the field, about doing cool stuff with computers.
I respect those of you who look at programming as something to be gotten out of the way quickly so you can enjoy a game of frisbee. This implies to me that you have a life. I've read blogs on the internet about those and I hear it's healthy to have one. I am happy for you, if somewhat intellectually and not out of any sense of real understanding.
But know that your disdain for your job is your own choosing. This post isn't about patterns, not directly. It's about the fact that there is really cool stuff going on in the machine all the time. There are reasons your boss wants those features, reasons your company wants them, reasons your customer wants them.
I am working in a niche of the industry that really doesn't interest me, to be honest. But the people in that niche have problems with their software, and that does. They have data here, but need it over there. They need their data stored in 78 different tables that can't be pulled up in less than 32 SQL queries, their dialup connections need 1500ms per query, and they need all this data retrieved in less than 500ms.
I don't go home thinking how boring my job or my company or my industry is. I go home hopped up like a monkey on crack that I figured out how to turn a caching algorithm inside out and turned "the impossible 48-second barrier" into "the 5-second barrier, but we're going to have Dave look at that before we call it impossible."
And the fact that the customers use that part of the software live while talking to their customers means the problem I solved is making people money somewhere. You're asking how can I find this work so interesting, but I'm boggling right back at you: How do you not throw your hands up and sing the "I Rule" song at the top of your lungs when you make the compiler sit up and beg?
Business programming isn't boring. It's programming, and that's not boring. If you're not having fun, you're doing it wrong.
The non-Math, non-Science aspects of Computer Science or Information Systems/Technology aren't really that hard!
How hard is it for someone to learn point and click operations? Seriously, anyone can learn the non-Math, non-Science based aspects of Programming easily. Why do you think that everything (almost) has been outsourced?
If you are looking for something hard, than please don't consider business programming, or becoming a Microsoft Systems "Engineer", etc...
haha, anyhow, I loved your post! Good one!
the irony: I am an IS student, because I love the stuff, somewhat.
How hard is it for someone to learn point and click operations? Seriously, anyone can learn the non-Math, non-Science based aspects of Programming easily. Why do you think that everything (almost) has been outsourced?
If you are looking for something hard, than please don't consider business programming, or becoming a Microsoft Systems "Engineer", etc...
haha, anyhow, I loved your post! Good one!
the irony: I am an IS student, because I love the stuff, somewhat.
# posted by : 7:22 PM
: 7:22 PM
There are no uninteresting problems to solve. At least not if you are solving them correctly.
If you are faced with a problem that seems to involve some brainless work - you should look into solving the "parent problem" - it is probably much more interesting and (if done correctly) may solve a lot more problems than the one you are facing!
In fact, this is quite often the early beginnings of a new business or product that makes you rich and famous!
Ultimately, humans should only be doing creative work! And the only limitation to how creative your work can be is your own creative limitations.
Peace!
If you are faced with a problem that seems to involve some brainless work - you should look into solving the "parent problem" - it is probably much more interesting and (if done correctly) may solve a lot more problems than the one you are facing!
In fact, this is quite often the early beginnings of a new business or product that makes you rich and famous!
Ultimately, humans should only be doing creative work! And the only limitation to how creative your work can be is your own creative limitations.
Peace!
you make me want to go back to development :)
and this time i won't join a sucky company like the one where i worked as a dev for two years, before switching to a SALES job !!!
and this time i won't join a sucky company like the one where i worked as a dev for two years, before switching to a SALES job !!!
# posted by : 6:36 AM
: 6:36 AM<< Home
