raganwald
Thursday, February 21, 2008
  The Mouse Trap

In the board game Mouse Trap, players build an elaborate Rube Goldberg machine. Wikipedia explains: The player turns the crank (A) which rotates the gears (B) causing the lever (C) to move and push the stop sign against the shoe (D), which tips the bucket holding the metal ball (E) which rolls down the stairs (F) and into the pipe (G) which leads it to hit the rod held by the hands (H), causing the bowling ball (I) to fall from the top of the rod, roll down the groove (J), fall into and then out of the bottom of the bathtub (K), landing on the diving board (L). The weight of the bowling ball catapults the diver (M) through the air and right into the bucket (N), causing the cage (O) to fall from the top of the post (P) and trap the unsuspecting mouse (i.e. the player who occupies the spot on the board at that time).

Software sometimes suffers from a Mouse Trap Architecture, it becomes a chain of fundamentally incompatible components used for purposes far removed from their core competencies, incomprehensibly connected to each other with brittle technologies. And here is the tale of how one such system came about.

The project was originally designed by a Business Analyst who had been a DBA in her younger days. One of the key requirements of the system was that it be completely flexible: it was designed to accommodate almost any change in business requirements without reprogramming. Her approach to designing a program that would be very nearly Turing-complete was to make a data-driven design: nearly none of the business logic was to be written in code, it was to exist in various tables in the database. Changes to business logic would be accomplished by updating the database.

Of course, the application would just be an empty shell, so the actual business analysis would consist of gathering requirements and generating data for the tables. The program was obviously trivial and could be generated in parallel with the business analysis, delivering a huge time saving over her company’s traditional waterfall model where analysis was completed in its entirety before the first line of code was written.

Delighted with this breakthrough methodology, all parties signed off on a seven figure contract and she started by building a large Excel workbook with the business rules, one sheet per table. The plan was to simply export her workbook to CSVs and import them into the database when they were ready to deploy the finished application. And in the mean time, the customer could sign off on the business rules by inspecting the Excel workbook.

Meanwhile, the trivial job of designing a web application for two hundred internal users plus a substantial public site to handle millions of users with unpredictable peak loads was handed off to the Architect. While her Golden Hammer was the database, his was XML and Java. His first order of business was to whistle up a Visual Basic for Applications script that would export the workbook to XML. From there, he wrote another script that would generate the appropriate configuration files for Struts in Java. His work done, he moved along to another project leaving some impressive presentations that delighted the customer immensely.

Implementation being an obvious slam dunk, the company put a few people on the project more-or-less part time while they completed the final easy stages of other delivery projects. Thanks to the company’s signature up-front analysis and rigid waterfall model, they were confident that customer UAT and delivery into production on other projects would not generate any meaningful bugs or requirements changes, so the resources would have plenty of time for this new project.

The New Guy

But just to be sure, they hired the New Guy (not to be confused with the New Girl). The New Guy had a lot of New Ideas. Generally, his ideas fit into one of two categories: Some were sound but unworkable in the company’s environment, and the others were unsound and still unworkable in the company’s environment. His early days were marked by attempts to hook up his own wifi so he could surf on his shiny new Tablet PC during meetings, attempts to disconnect the loud pager that would interrupt all programming in the cubicle farm whenever the receptionist was paging a salesperson, and attempts to get the project to fix all bugs on completed features before moving on to write new features.

When he saw the design of the system, he immediately grasped its deepest flaw: Changes to business requirements in the Excel workbook could cause problems at run time. For example, what if some business logic in Java was written for a Struts action that vaporized when a business rule was rewritten?

Today, we can sympathize with his obsession. He was deeply discouraged by the company’s insistence that development run at full speed developing new features with the actual business of making the features work deferred to UAT at the end of the project. One developer claimed that she had a working dynamic web page switching back and forth between English and Inuktitut, but the English version was generated by a JSP backed by a stub class and the Inuit version was actually a static HTML page. Management gave this a green light to be considered “feature complete” after the customer failed to ask any pertinent questions during the Friday-afternoon-after-a-heavy-steak-lunch-and-before-a-long-weekend demonstration

Depressed and quite pessimistic about the team’s ability to orchestrate Java development in parallel with the rapid changes to the workbook, he came up with the solution: a series of XSLT files that would automatically build Java classes to handle the Struts actions defined by the XML that was built by visual basic from the workbook that was written in Excel. Then, any changes that were not properly synchronized with the Java code would cause a compiler error, and the team would be forced to update the Java code immediately instead of hand waving things.

Excel >> VBA >> XML >> XSLT >> Java!

The New Guy ripped his phone out of its socket, ignored all emails, and worked around the clock. When management came looking for him, he hid in vacant conference rooms, feverishly tapping away on his tablet. A few days later, he emerged triumphantly with his working build system. Saving a change to the excel workbook automatically generated the XML, which in turn automatically generated the Java classes and rebuilt the entire application, along with regenerating the database tables in a style what would presage Rails migrations.

He presented this nightmare of dependencies and fragile scripts to management and waited for the response. They had shot down every single one of his ideas without exception, and now he was promoting a daring scheme that owed its existence to the proposition that their management style was craptacular. But he was a man of principle, and was committed to do the right thing in the face of any opposition.

Management wasted no time in responding. Brilliant! He was obviously far too valuable a resource to waste time on implementation. He was promoted to a junior Architect role where he could deliver demonstrations to clients. His very first job? To write a white paper explaining the new technology.

Expecting to get the axe, he was shocked by their warm reception. He had failed to realize that management was indifferent to the idea’s actual development value, but had a keen sense of what played well with clients in Enterprise environments. These companies lived and breathed integration between wildly disparate technologies, many of which didn’t work, had never worked, and never would work.

I suspect this is typical of Mouse Trap architectures everywhere. Built with the best of intentions, they survive for reasons their creators could never have anticipated.

Epilogue

The company adopted the new build scripts immediately and assigned a junior programmer who was working on several other projects to maintain them. Within months they had been dismantled, in no little part because the team hated the idea that every time the business analyst changed the business rules, their Java code that was carefully constructed for a previous version would stop compiling.

The New Guy lasted a few months longer before realizing that his sudden accord with management was illusory, and that nothing really had changed. He has now forsworn his love for static typing and is now wandering a production Ruby code base muttering about software development the way the demented wander the streets muttering about government conspiracies.

All that remains of his work are a few XSL files somewhere, like old game pieces that are rolling around the bottom of a drawer at the cottage hoping that someone will open a bottle of wine and call for a game of Mouse Trap to pass the time.

The story depicted here is 100% true. Any similarities in tone and style to The Daily WTF are intentional.

Update: Never assume that you are alone in your madness.
 

Comments on “The Mouse Trap:
Of course, the real WTF is the forum software.

(Do they still say that?)
 
Hm... This article made me think (how rare these days) - we can tweak a bit Greenspun's tenth rule to say something like "Every sophisticatedly large enterprise architecture contains an ad-hoc, informally specified semi-compiler".
 
This was a great post. But you know what was even better? You linked to the 'New Girl' - one I had not read - and THAT was a great read!

So you get double bonus points for (a) a great post and (b) linking to an even better post. Five stars!
 
New Girl is an amazing story with a happy ending for Joey, he married a fellow blogger called "The Redhead."
 
Hey! we had that game when I was a kid, forty (oh shit) years ago. It was great.
 
the url for TheDailyWTF is wrong
 
Thanks a lot, now I'm going to have nightmares. Almost this exact same thing happened to... someone I know... a few years ago. Yet another case of a ridiculously normalized database containing business logic that was mapped to objects using xml which were read by skeleton java generated by an xml schema, whose output was xml translated by xsl into html. I almost want to bash myself in the head just thinking about it. Java, web services, and "enterprise" almost made me leave the programming industry altogether, at least until I started playing with ruby and began to like programming again.
 




<< Home
Reg Braithwaite


Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

Books
What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

Share
rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

Beauty
IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

Work
The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Management
Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

Notation
The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Opinion
Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

Whimsey
The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

History
06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /