Roman Wagon Wheels

It's said that the gauge (width between tracks) of American railroads can be traced from bureaucracy to bureaucracy back to the width of Roman war chariots.  It turns out that this is just another urban legend, but if you've ever worked in an organization of any size, you've experienced the organizational inertia that makes this legend so plausible.

I ran into a great one today, for instance.  I'm doing some work in a place where I don't get to set the standards (yes, it's a government agency).  One of the real winners is a standard that mandates that all SQL queries for an application be stored in an XML file, with the queries and their corresponding parameters specified.

I actually learned about this one quite some time ago, and of course, I asked this requirement existed.  This is a proprietary structure, and none of the modern conveniences of .Net development know a thing about this file, so it really slows things down.  The file also tends to grow large and unruly over time, and it's difficult to find anything in it because, again, .Net doesn't know anything about its syntax.

The reason, I was told, is that the DBA's want to be sure that all of the queries for the entire system could be found in a single place so they'd be easy to review and tune.  I bought this, hook, line, and sinker.  I may also be interested in a bridge if you have one for sale.  After paying close attention to the ongoing flurry of intense DBA activity that consistently failed to materialize around this file, though, I began to have my doubts.

So today, I was debugging a really nasty instance of this mess -- a search query.  The "advanced search" screen has a collection of fields that the user can search, and like most advanced search screens, the user only fills in the fields she wants to search - the rest are left blank.  When we search, we query on the fields containing data and ignore the rest.

But that's not how our search works.  Since we've got to store our queries in XML and substitute parameters to alter the queries, we have a huge, awful SQL statement that joins all of the tables that might possibly be used in a search.  Not very efficient, and not certainly not easy for developers.  This mechanism has been difficult, slow, and error-prone since day one.

On top of everything else, the production Oracle servers have really been struggling lately, and the DBA's are looking under rocks for any performance improvements they can find.  Since this bungled search technique is really, really inefficient in terms of database execution speed (in addition to being a gaping black void of development time), I thought I'd run this by the DBA's again.  After all, they'd probably seen the "normal" queries as they're encoded in XML, but maybe they'd never seen one of these search monstrosities, so I asked.  "What's up with the XML file?" I asked.  "Haven't you seen what that does to these search queries?  Haven't you seen all the extra joins we don't really need?  Wouldn't you like those database cycles back?"

Do you know what the DBA said?  "Tain't me that's forcing the XML thing - go find another tree to bark up."

Now, that means that the "architecture people" are blaming the use of this file on the "database people", and the "database people" are blaming it on the "architecture people".

Sigh.

Yes, this is a government agency, but I'd bet you'll run into this in some other places, too, because this is one of the universal truths of organizational dynamics.  I haven't run this one all the way to ground yet, but I clearly should have known better.  I don't fall for "that's the way we've always done it" anymore, but I'm going to have to work on my screwy-reason radar so I don't let another one like this by me.  Pay attention so you don't buy any bridges (or wagon wheels), either.

Just for kicks, by the way, here's a version of the original "chariot wheels" story that claims that the railroads gauge is more than just a coincidence.  You can draw your own conclusions - the XML query file still stinks.