Month: May 2008

It's a popular misconception that if you throw some SOAP, WCF, or J2EE service layers on top of an application, it's automatically easy to integrate.  I place the blame for this misinformation squarely on PHB's and Gartner conferences, because this is simply a case of management by buzzwords.

A good friend of mine used to tell me all the time that the reason we're entertained by a dancing bear isn't that the bear dances well - it's that the bear dances at all.  There once was a time when just being able to talk to a mainframe routine from Java, or talk to Java from .Net, or talk to a routine on one of your partners' computers was a major technical triumph.  Web services made the bear dance, it's true.

Now that we've seen the bear dance, though, we're looking for a few new steps and a little more rhythm.  We want the bear to dance well.  We want quick integration.  We want security.  We want reuse across our enterprises, and yes, Virginia, we want mashups like those kids are doing on face-space, or my-book, or whatever the hell it is they're using.  We drunk the Gartner kool-aid, and we want some of that Web 2.0 stuff, too.

But sadly, integration isn't something you pull out of your Web 2.0 Conference swag bag like a door prize.  You will, in fact, have to know what you're doing to achieve any really meaningful results.  Here are a couple of places where conference-oriented development gets tripped up.

Misconception #1: If we're using web services, no additional effort is needed to make our systems integrate well.

Even though nobody actually makes this claim in black and white, lots of executives (and even some software architects) seem to believe it at some level.  Even Garter will tell you that SOA is a lot of work, but somehow, that's not the message that people seem to internalize.

The truth is that integration on an Enterprise scale is still hard work, and your results are still dependent on planning and execution.  It's true that web services make integration easier than it was in days gone by, but you won't end up on the cover of CIO magazine just because you can spell WSDL.

This may not be a popular point of view, but I believe you've really got to treat integration as a product in its own right in order for it to truly be successful.  The API's you create have users, and that means you've got to consider the usability of your API's just as seriously as you'd consider the usability of a flagship application.  I won't go into detail here on what usability means for an API, but it's really comprised of exactly the same factors as application usability.  If you don't pay attention to the people who are going to use your API, your integration isn't going to be much more successful as an application that all your users hate.

Misconception #2: We can mask our business problems by using web services.

Sorry again.  I've seen this a bunch of times, and it never ceases to amaze me.  If you've got a business problem you can't seem to solve without the benefit of a web service, it's unlikely that a web service (by itself) is going to fix your problem.  Data problems?  If you've got data integrity rules to enforce, a web service is great at enforcing them, but if your data is already messed up and you don't have clear algorithmic rules to fix the data, a web service isn't going to help you very much.

Similarly, if you've got a well-defined business process, a web service may be just the thing to expose that process to other applicaions.  If your business process isn't well-defined, or (worse) if your business users can't agree on a process or other requirements, a web service won't bridge that gap.

Business issues like these really have to be raised up and dealt with in cooperation (at least) with your business leaders.

So embrace SOA and web services, yes.  But don't blindly trust that they're going to fix your problems for you.

It's rare these days that a Web 2.0 startup lands a round of financing, and the funding is completely overshadowed with bad news.  Twitter isn't just shooting itself in the foot - it's mowing itself down with a chain gun.

Problem #1 is uptime, or lack thereof.  Anyone who's been on Twitter over the last month or so has experienced a *severe* up-again, down-again roller coaster ride at Twitter.  Every day, it seems like there's an outage, and some of them have lasted for hours.  I've seen by far more blog traffic about "Twitter's down again" than I've seen talking up the service itself, and that's not good.  Let's not forget that Twitter is still solidly in "early adopter" territory, and they're not going to attract too many mainstream users if the site's down.

Twitter has taken some heat for lack of transparency during this episode, and even that has been entertaining.  They're blogging about what's going on (to some degree), but as a commenter pointed out, they're Twitter for Pete's sake -- Twitter about it!! It's slightly disturbing that this didn't come as second nature to these guys, since this is what they do.  I've also noticed that the manner in which they "splat" is singularly ungraceful.  When Twitter crashes, I see (on my mobile browser) the PHP equivalent to a blue-screen, not a buttoned-down page explaining that there's a problem and someone's working on it.  Here's another take on their error reporting.

Now, I understand that one of the purposes of this round of funding they just landed is to shore up their infrastructure and get some of this stuff ironed out, but I wonder if it'll happen in time.  Fixes for problems like these, in my experience, are a long time in coming, and an infusion of cash today is still months away from paying dividends once you use the money to hire help, buy servers, configure the servers, install the servers, fix the software, test the software -- you get the picture.

But as bad as this all sounds, uptime may not be Twitter's biggest problem.

Read this.

From everything I've seen so far, Ariel's account of events is undisputed and accurate.  Twitter's response is pathetic.  After sandbagging for months, they're now hiding behind the "we're an infrastructure company, not a content regulator" argument.  Totally lame, and totally inadequate.

I'm not aware of anyone who's learned about this and hasn't been outraged, and I'm not aware of anyone who's defending Twitter's response to-date.  Sometimes, stories like this have a grey area where you could argue either side of the story, but that's not true in this case.  Twitter is simply alienating users as fast as this news travels, and it's traveling fast.

The only good part about this story (for Twitter) is that it's taking attention away from their downtime.  As far as silver linings go, that's pretty lousy.

Aside from just condemning Twitter, though, why would we, as software professionals, care about Twitter's agony?  That's easy - we want to learn from their mistakes.

The Ariel Waldman mistake is big and obvious, and the lesson is pretty simple, too: Don't be a jackass.  If there's a problem, acknowledge the problem and meet it head-on.  Don't lube up and try to slither past the problem, because it's going to keep haunting you until you kill it dead.  The stall tactics, the wordsmithing, the "we're just a humble little infrastructure company" line are all just ways to bide time in the hopes that the story will just go away on its own.  How's that working out, Twitter?

The second lesson seems just as simple, but I don't believe it really is.  "Don't build a crummy infrastructure," you say.  Sure, but how?  The data model Twitter provides isn't as easy to model as you might initially believe.  Here's an excellent discussion of what their architecture might look like.  So it's very possible that they're facing some architectural challenges far beyond those most of us see in our careers.

Add to that the startup culture of "features first, then users, then scaling and monitization," and you can end up with a code base that works really well at low volumes, but just breaks apart when you start putting a real load on it.  A friend of mine used to tell me all the time that "the thing that makes you successful today will kill you tomorrow."  In this case, that means that our architectures have to be able to absorb change on a wholesale scale.

I really hope Twitter figures this out and pulls through.  They've got a lot going for them, and they're close to making the leap to the next level, but they need to get with the program in a big way, like *right now* if they're going to pull out of this nosedive.

It's said that the gauge (width between tracks) of American railroads can be traced from bureaucracy to bureaucracy back to the width of Roman war chariots.  It turns out that this is just another urban legend, but if you've ever worked in an organization of any size, you've experienced the organizational inertia that makes this legend so plausible.

I ran into a great one today, for instance.  I'm doing some work in a place where I don't get to set the standards (yes, it's a government agency).  One of the real winners is a standard that mandates that all SQL queries for an application be stored in an XML file, with the queries and their corresponding parameters specified.

I actually learned about this one quite some time ago, and of course, I asked this requirement existed.  This is a proprietary structure, and none of the modern conveniences of .Net development know a thing about this file, so it really slows things down.  The file also tends to grow large and unruly over time, and it's difficult to find anything in it because, again, .Net doesn't know anything about its syntax.

The reason, I was told, is that the DBA's want to be sure that all of the queries for the entire system could be found in a single place so they'd be easy to review and tune.  I bought this, hook, line, and sinker.  I may also be interested in a bridge if you have one for sale.  After paying close attention to the ongoing flurry of intense DBA activity that consistently failed to materialize around this file, though, I began to have my doubts.

So today, I was debugging a really nasty instance of this mess -- a search query.  The "advanced search" screen has a collection of fields that the user can search, and like most advanced search screens, the user only fills in the fields she wants to search - the rest are left blank.  When we search, we query on the fields containing data and ignore the rest.

But that's not how our search works.  Since we've got to store our queries in XML and substitute parameters to alter the queries, we have a huge, awful SQL statement that joins all of the tables that might possibly be used in a search.  Not very efficient, and not certainly not easy for developers.  This mechanism has been difficult, slow, and error-prone since day one.

On top of everything else, the production Oracle servers have really been struggling lately, and the DBA's are looking under rocks for any performance improvements they can find.  Since this bungled search technique is really, really inefficient in terms of database execution speed (in addition to being a gaping black void of development time), I thought I'd run this by the DBA's again.  After all, they'd probably seen the "normal" queries as they're encoded in XML, but maybe they'd never seen one of these search monstrosities, so I asked.  "What's up with the XML file?" I asked.  "Haven't you seen what that does to these search queries?  Haven't you seen all the extra joins we don't really need?  Wouldn't you like those database cycles back?"

Do you know what the DBA said?  "Tain't me that's forcing the XML thing - go find another tree to bark up."

Now, that means that the "architecture people" are blaming the use of this file on the "database people", and the "database people" are blaming it on the "architecture people".

Sigh.

Yes, this is a government agency, but I'd bet you'll run into this in some other places, too, because this is one of the universal truths of organizational dynamics.  I haven't run this one all the way to ground yet, but I clearly should have known better.  I don't fall for "that's the way we've always done it" anymore, but I'm going to have to work on my screwy-reason radar so I don't let another one like this by me.  Pay attention so you don't buy any bridges (or wagon wheels), either.

Just for kicks, by the way, here's a version of the original "chariot wheels" story that claims that the railroads gauge is more than just a coincidence.  You can draw your own conclusions - the XML query file still stinks.