Category: Uncategorized

I just found a site called highscalability.com via feedly.  There's some great stuff in here, including the article I read first: Stack Overflow Architecture.  If you're building big apps or apps that you hope will be big someday, you're going to want to add this blog to your reader.

Related articles by Zemanta

• Learning from StackOverflow.com (redditech.blogspot.com)
• Google Reader Gets A Social Makeover (techie-buzz.com)
• What should a developer know before building a public web site? - Stack Overflow (redditech.wordpress.com)

Back in February, amidst the news about Microsoft's new Azure platform, I asked why the concept of Private Clouds seemed to be either dismissed or ignored as a viable Enterprise strategy (see also "PDC Reactions" and "Could Azure be self-hosted?").

Yesterday, I learned that there's hope, after all.

I happened to be in the office for one of our internal MS development "user group" meetings, and we were lucky enough to have Brian Prince stop in to talk to us about Azure.  The talk was great - he demo'ed a super-simple sample app that really helped make some of the Azure concepts real, but one of the things that caught my attention was only mentioned in passing: Microsoft is bringing Azure technology to private data centers.

Point #1 -- boot from VHD.  This is a cool little novelty item in Windows 7, but it turns out that its origin is really the Azure team.  This is how they spin up instances for you, and it was a cool enough idea that the Windows team built it into Win 7.

Point #2 -- Yes, Microsoft is going to power private clouds.  Mark my words: this is going to be big.  In fact, why don't you leave yourself a reminder to come back here in about five years so I can say, "I told you so."  This is absolutely one of those technical announcements that's completely underwhelming at introduction because it's so disruptive that people don't know how to deal with it.

When you read about MS's Private Cloud products, watch for the bit about chargebacks.  You'll probably miss this the first time you read it.  Again, this line-item doesn't mean too much until you consider that this technology is coming from Azure.  You know -- Azure -- the platform where you can rent capacity and Microsoft sends you a bill at the end of the month.  How'd you like to be able to do that for your internal customers?  I thought so.

So what's an architect to do if this technology won't be fully appreciated for a number of years?

Simple.  Learn the basics of the technology and start architecting for the cloud even if your app and the cloud aren't quite ready for one another.  Specifically,

• Look at how cloud apps scale.  The way storage and processing scale in the cloud is different than what you're used to.  Figure out what that means, and design to be compatible with this.
• Revisit your storage assumptions.  Azure is going to support SQL Server, but it's clear that SQL Server isn't the 100% brain-dead automatic end-all-be-all storage choice that it once was.
• Get comfortable with messaging and queuing.  This is how cloud components are connected.
• Get comfortable with threads.  This should be on your list anyway, because of the rise of multi-core processors, but if you understand thread management, you'll be more comfortable with processing threads distributed across the cloud.
• Think about chargebacks.  If you're suddenly able to track processing and storage utilization with great ease and accuracy, what does that mean for your shared applications?  How would you split up costs for services that are shared across departments?

There are some really powerful concepts here that are just starting to emerge, and now is the time to start wrapping your brain around them.

What can your enterprise do with private clouds?

Related articles by Zemanta

• SQL Services is now SQL Azure (blogs.msdn.com)
• Exploring Scalability, Performance And Deployment (slideshare.net)
• Talking about Microsoft's Windows Azure with Amitabh Srivastava (cloudofdata.com)
• Microsoft preps massive cloud expansion (theregister.co.uk)

I love these dialog boxes:

How hard, exactly, would it be to figure out that a process that had been running at a rate of "x" had suddenly dropped to an entirely unreasonable speed -- so slow, for instance, that perhaps it had actually stopped, and thus, would very likely either fail or resume operating at a reasonable speed again?

Apparently, it's pretty difficult.

I recently read a review of a new Lotus (the car, not the spreadsheet), and the reviewer recalled the defining design tenant of Colin Chapman, who explained that his goal was always to "add lightness".

This isn't as straightforward as one might imagine in car design -- we don't easily give up all the conveniences we've come to expect over the years (radios, antilock brakes, windshields, etc.).  Similarly, government regulations insist that our cars meet certain safety standards.

These are certainly good things for us and our automobiles, but they've resulted in a substantial bloating of our automobiles over the years.  It's simply easier to make a car safer by adding more steel, for instance.  When the performance of the car suffers, it's far easier to bore out the motor or add a couple more cylinders than it is to find and trim excess weight.

A typical sports car has a power-to-weight ratio of 1:10 or better (or 10 lbs. per horsepower).  This means that if you want to boost the performance of your car by either adding power or reducing weight, this ratio suggests that adding 10hp would be roughly equivalent to dropping 100lbs.  Which do you think is easier?  How about if we make it 50hp or 500lbs?  There's simply no way you can remove 500lbs from a modern car without making some compromises, and/or incurring extra costs by employing exotic materials.

So why bother "adding lightness"?  Why not just bulk up with a bigger motor?  Part of the answer is the feedback loop you start.  A bigger motor will itself weigh more, so part of its additional power is offset by its own weight.  You'll need bigger brakes to slow this extra weight down.  Bigger brakes will call for larger wheels to clear the discs.  Maybe a bulkier frame to accommodate the power.  Beefier suspension pieces to deal with the extra weight.  Pretty soon you've eaten up a fair bit of the power you just added, and the weight of your car is up dramatically.

The rest of the answer, though, can only be appreciated by driving a light car.  Rather than rely on huge tires to grip the road and a huge engine to move you, a light sports car is quick because it's light.  A Lotus is more nimble, tossable, and agile than its competitors in large part because it doesn't carry as much mass.  There's less inertia to overcome when changing directions, and the driver feels much more connected to the road.

There's got to be a software lesson here, doesn't there?

Indeed, there is.  In our world, we add lightness by adding simplicity.  Just as we don't want to live without antilock brakes, nobody really wants to go back to running DOS or green-screen applications, but where are the places where we've accrued bulk unintentionally?

Do you remember your reaction the first time you saw Ruby-On-Rails, or maybe even MS Access?  A simple approach to software can make us question how we ended up with all the bulk we see in many of our applications.  How many lines of code exist simply because "that's the way we always do it"?  If you start simpler, can you do without some of the layers we take for granted?

How do you add lightness to your software?

Developers are familiar with "code smells" --the little signs you see upon superficial examination of code that lead you to fear deeper pathalogical problems.  Over time, many developers become pretty good at spotting these signs, and volumes have been written about how to address these problems once they're detected.

But code smells aren't the only signs that problems are lurking in your system.  Most systems with even moderately complex data models can hide all sorts of problems in their data.

A good system, of course, will be coded defensively, such that it can tolerate, or maybe even fix bad data.  This is feasible and practical in small to mid-sized systems, but it becomes increasingly difficult as systems become larger and more complicated.  In all but trivially-small applications, bad data is a very real problem.

Like bad code, bad data is sometimes bad in very subtle ways.  Database constraints can (and should) be used to prevent obvious problems with things like unique id's, foreign-key references, required values, and so on.  This is a minimal requirement, but it won't help you deal with data that violates complex business rules (ex: an order must have an associated invoice if the status of the order is "placed order").

Typically, you'll find examples of data rule violations when you're diagnosing error, or maybe when you're doing reporting or data analysis.  When an instance of bad data is discovered, you've really got only two ways to deal with the problem:  Fix the code, or fix the data.

Fixing the code is often our first reaction, since we're (generally) more comfortable working in code than data in the first place.  We'll often go to the source of the error, and we'll change the code to tolerate this particular class of bad data, but it's important for us to ask ourselves if this is truly a fix for the problem:

• Just because we fix the code in one place, how can we know we won't blow up somewhere else because of the same bad data?
• If there's really a business rule governing this data, how are we helping by tolerating violations to those rules?
• Are the business rules governing this data known at all (I know this sounds silly, but it's going to be a valid question more often than you might think)?
• How did the bad data get into the system to begin with (is there a real bug upstream that's allowing bad data to be created)?

So in some cases, at least part of the problem is to fix the bad data.  Again, there are some questions you should pose of your system before you dive headlong into SQL:

• Are there other instances of this data corruption?  How many?
• What are the circumstances of the problem?  Is there a way to predict the scope or context of the problem?  Perhaps the context can lead you to the source of the data corruption.
• Can the data be fixed at all?  Sometimes, the damage is irreversible, and repairs can be quite difficult.

As you may have gathered by now, the sooner these issues can be nipped in the bud, the better off you'll be.  I'll cover some strategies to help you with this in a future post.

I just saw a blog post from Bill Sempf describing a book he'd written for Microsoft to help them explain licensing for Windows Server 2008.  At first, I read right past a key metric, but I doubled back and read it again -- the book is 86 pages long.

Eighty-Six pages?? Really???

Now, don't get me wrong.  I have every faith that Bill's done a fine job of documenting the licensing requirements in the simplest fashion possible.  I don't mean to bash the book; I mean to bash the licensing requirements.

Do you think there's a chance that the real problem here isn't the fact that nobody liked reading licensing whitepapers?  Maybe the real problem is that the licensing model takes 86 pages to explain in a "Dummies" book.  How long is the "Licensing Unleashed" book going to be??

I'm not exactly sure how you fill up 86 pages with licensing guidelines, but I have to guess you're going to see chapters like this:

• What's it going to take to put you in a new OS today?
• If you have to ask, you can't afford it.
• Feeding Ballmer's Ellison-envy since 1998.
• If you think the licensing rules are complicated, you should see our commission calculations.
• This is going to hurt me more than it hurts you.
• Hang on a second while I go talk to our General Manager.
• How much did you say your budget was again?

Enjoy the read, though.

Logging is one of those "system" components that always seems to either be left out or way over-engineered (glares at Microsoft's Enterprise Application Blocks). Today, I'd like to introduce you to a logging framework that's everything it needs to be and nothing it doesn't.

The .Net Logging Framework from The Object Guy is powerful enough to handle any of your logging needs, but simple and painless to use.  Here's a relatively complicated example -- we're going to log to three logging sources to demonstrate how easy it is to set up.  In most cases, of course, you'll log to only one or two sources:

/* first instantiate some basic loggers */
Logger consoleLogger = TextWriterLogger.NewConsoleLogger();
Logger fileLogger = new FileLogger("unit_test_results.log");
Logger socketLogger = new SerialSocketLogger("localhost", 12345);

/* now instantiate a CompositeLogger */
logger = new CompositeLogger();

/* add the basic loggers to the CompositeLogger */
logger.AddLogger("console", consoleLogger);
logger.AddLogger("file", fileLogger);
logger.AddLogger("socket", socketLogger);

/* now all logs to logger will automatically be sent
to the contained loggers as well */

/* logging is a one-liner */
logger.LogDebug("Logging initialized.");

When you download this logger, you'll get all the source code, including a socket reader to catch the logs thrown by the socketLogger in the example above.  Extending the logger is a piece of cake, too, so you could build yourself a WCF Logger, for instance, in no time flat.

You'll note the lack of config file-driven settings in the example above -- this is purely intentional.  You can decide if you want to make any of these settings configurable, and do so in the format you're comfortable with, so you don't need to try to get your config files to conform to whatever format your logger insists on using.  This small simplification can be a big time-saver for simple apps, debugging / test harness apps, and so on.

Introduction

There's no question that Visual Studio is a class-leading tool for building large applications.  The IDE is incredibly helpful to coders, and the .Net framework lends itself to managing dependencies among components and classes in large applications.  In addition, Visual Studio is designed to be extended by third-party tools that can make it even better.  NDepend is one of these tools; its purpose is to analyze large applications and expose information that's typically hidden deep inside your code.

Installation and getting started

The NDepend web site shows some great screen shots with all manner of graphs and charts and reports, so naturally, you want to see that stuff for your code, too, right?  Good news: installation is a piece of cake.  Just unzip into a directory, add the license file, and you're ready to start your first analysis.

When you start the NDepend Startup shows a screen reminiscent of Visual Studio (start screen).  Create a new project, point it at a Visual Studio solution file, and let NDepend do its thing.  Zero to more graphs than you can shake a stick at in about four minutes:

Continue reading "NDepend review"

Over the last couple days, I've seen a small firestorm erupt over Microsoft's Kobe MVC sample project.  First out of the gates was Karl Seguin, with a rant about all sorts of coding ills found in the source.  Then, today, I saw another great post that breaks down some specific problems with duplicate lines and cyclomatic complexity.

My first reaction was to share in the shock and horror of it all -- after all, how could Microsoft issue a sample app so obviously riddled with problems, but as I thought about it some more, the real issue became obvious:

Real-world apps deal with this sort of coding all the time.

You see, very few apps evolve in environments that are conducive to crafting museum-quality code throughout the entire application.  Very few software teams, in fact, are comprised entirely of developers who are capable of crafting this sort of code.  Our development teams have imperfections, and so do our systems.

Maybe Microsoft should have been held to a higher standard because (a) they're Microsoft, and (b) this is a sample application where the primary deliverable is the source code itself.  That's valid, but what's more interesting to me is that this brand-spanking-new framework is so damned easy to mangle and abuse!

As architects and managers, one of the most valuable long-term effects we have on the software we develop is to leverage good ideas across our teams and our code.  It's not sufficient that we can dream up the perfect architecture and the perfect solution if it stays in our heads.  The real measure of effectiveness is how well we can project these good ideas across the entire breadth of code in all of our systems, and this is where our current tools fall short.

Developers love C# because you can do anything with it.  It's a fabulously-flexible tool.  Architects try to impose order by mandating the use of standards, frameworks, and constraints.  It's a perpetual tug of war.  We're supposed to be working together like an orchestra, but in most cases, we're still a bunch of soloists who just happen to be on the same stage.

In Kobe's case, the mere fact that people are opening up this code and recoiling in shock because the code doesn't look the way they expected is proof that we're not where we need to be  -- on many levels.

Related articles by Zemanta

• Avoiding the high cost of bad code (infoworld.com)
• Microsoft Embracing Open Source? (solutionset.com)
• Drastically leverage your Code Reviews (codebetter.com)
• Introduction to the ASP.NET MVC Framework (codebetter.com)

In the early days of shrink-wrapped PC software, I used to buy a software title and not expect to pay anything more for that software until I decided to upgrade it. I might upgrade when the publisher released a new title, or I might skip a release -- it was up to me to decide.

Pretty soon, though, software companies discovered that it was expensive to staff a help desk to support customers, and then they started to discover that it was painful to have customers working with software that was several versions out-of-date. The solution: software subscriptions and maintenance plans.  Enterprise software companies had already been doing this for years; it lets the software company generate revenue from support areas, and smooths their revenue stream (so it's not clustered around new releases).

Although enterprise customers had been paying maintenance for years, consumers have been wary of subscriptions.  They want to know what they're getting for their money, and they want to be able to decide when to upgrade.

In a recent blog entry (Software maintenance pricing - Fair or out of control?), Scott Lowe shows that this sentiment affects the enterprise customer, too.  Especially in these times of constrained budgets, enterprises aren't too excited about big increases in software maintenance prices without a whole lot of additional perceived value.

If you're a software developer, you need to understand that you can't get away with milking your existing customers just because they decided to buy your software years ago.  This should go without saying, but if you sell subscription-based support, make sure you provide upgrades that are worth the cost your customers are paying.  In a recent Joel on Software thread, a developer sounded off against Lowe's article, but he completely missed Lowe's point:  the prices of software maintenance are going up, but value isn't.

If you're a customer, you hope your vendors are committed to providing value for your support dollars, but this won't always be true.  If you've ever felt locked into a vendor, you know this is no fun at all.  When you're faced with a vendor who's got you over a barrel, you can feel like your organization is being held for ransom, and you're powerless to extract yourself.

As a manager or an architect, part of your job is to manage vendor risk. I've got some thoughts on this, and I'll share them in another post.

Let me know what you're thinking - what are you doing to manage vendor lock-in in your organization?

Related articles by Zemanta

• When You Treat Your Customers Like Criminals, Don't Be Surprised When They Go To Different Suppliers (techdirt.com)
• Penny wise, pound foolish (meganmcardle.theatlantic.com)
• Another Take on Enterprise Open Source (cazh1.com)