The Hidden Costs of Technical Debt

by on February 3, 2011 · 5 comments

Recently, I had the unpleasant responsibility of stopping our development line. And it wasn’t just for a day or two. It was for a couple of months. The complexity of the codebase made every simple feature we added take weeks to implement, and I realized we were getting a negative return on our efforts. How the hell did things get this bad?

Over designed

We all know the allure of a new project. The sky is the limit, our ideas unbelievably awesome – we’re sure to take over the world! The gushing adrenaline on the team convinces everyone we’re utterly infallible as we sit down to begin the design.

“We’ll need this when …” – the most evilly utterable phrase in software design. Let’s dissect:
1. “We” – who the hell is this? You? Me? That dude over there?
Probably the most critical question you can ask in design is who you’re building this for. Get some [Personas] created and figure out exactly what types of users need which features.

2. “will need this” – really? Can you see into the future?
Focus on what [Sally the photographer] needs right now. If you’re actually listening to your users, you’ll find more than enough to do.

3. “when” – and what if that ‘when’ never happens?
There’s an awful lot of things that will happen on the way to that ‘when’. So many things, in fact, that ‘when’ will probably never happen in the way you planned.

I’m not saying this phrase should never be uttered, but when you hear it be very, very alert. Move the discussion away from the abstract (fluffy, feel-good, we’re all just God’s creatures) back to the concrete. Too many of these phrases gets us to our next problem.

Over architected

The gauntlet has been thrown, and your team is confident they can implement every grandiose idea outlined above. In fact, they have some ideas of their own to ensure the success (scalability, performance, etc.) of the project. They’ll use the best software patterns and engineering at their disposal. They’ll incorporate the coolest libraries and code to all the latest social media APIs available.

Voila! You’ve just created technical debt and you don’t have a single registered user! And exactly therein lies the problem. Your team has been coding like crazy for months and you have 0 users!

Time to refactor

A year later, the ‘new project’ has thousands of users – the “we’s” have become concrete user types, “will need when’s” have become “needed yesterday”. It’s much easier to design new features now because you must be concrete and focus on the details. Those design specifications written last year have long been recycled into toilet paper. But what about the codebase? If you didn’t continuously “recycle” it to evolve with the emerging realities of the growing platform, you’ve got a big problem.

Suddenly, you’re presented with a bill of the steady drip-drop of technical debts that have made their way into your code over a year. It’s a pretty damn big bill and you furtively eye the door wondering if you should just make a break for it and get the hell out of there.

Ok, settle down. We can do this. How? Stop the line. Stop the development and refactor the codebase. Over simplify down to what the platform needs right now. This will probably be one of the most difficult (and costly) decisions that you make, but it will make the difference between a prolonged death and new life for the project.

Keep it clean

How can we avoid these vicious cycles? First of all, read Robert C. Martin’s book Clean Code. Second, … go back and read the book again. If you did, you’ll have your answer. The short of it is – take care of your code, because if you don’t, no one else will. It’s yours.

If you spend just 30 minutes per day refactoring, you’ll save yourself weeks (if not months) of painful “overhauls”. Don’t race to grab that next user story. Take 30 minutes and refactor what you just wrote. Keep it clean and debt free! Keep it real and avoid the dreaded relaunch.

Did you enjoy this article? Get new articles for free by email:

Comments

  1. says

    I like the post however wonder whether a random 30 minutes of refactoring each day will be effective or whether it will cause further sub-optimization? Rather would it make more sense to systematically approach the refactoring ( target the worst over design or other form of technical debt area). I think the idea of a targeted 30 minutes makes even more sense.

  2. says

    Good point, Tom. Perhaps keeping a running, prioritized list of technical cruft and the commitment to spend at least 30 minutes a day on one of these is better time spent.

  3. says

    Over engineering applications is something that doesn’t get a lot of attention in technical debt discussions but is something I have run into quite a bit. When developers start “gold plating,” complexity goes up and agility goes down. I won’t mention any names but we were a long time user of a .NET third party component suite and had to switch because the controls became overly complex. This was a difficult choice given the investment we had made but the controls were essentially a technical debt. We were paying interest in the form of productivity losses and the principal was the additional investment in time to get up to speed with another suite. It turned out to be a very good investment.

  4. says

    Nice post. I agree with the fact that over-engineering is a problem itself. However, sometimes you’re presented with future ideas that the code needs to be ready for. But then, in reality, the new features do not come. Instead, the Business has other new features in mind and your design is already out-dated, introducing technical debt with the first release.

    Knowing this, the inevitable changes that are coming, your code should be flexible. This does not mean you should over-engineer, but you should be watching the responsibilities of your classes for instance.

    I’ve read the book Clean Code, refactoring should be part of your process all the time. Like TDD – Red Green Refactor -. Especially the later one is forgotten once the feature is complete. Even after I’ve completed my change code-wise (ie, it works). I then go back and review my code and make it even better. Not because I have enough time on my hands, but because I know that other developers will come here and they need to know what is happening without reading the ‘how it is done’ lines of code. I call this ‘seperating the what from how’.

    Either way, nice post!

Trackbacks