How to Avoid Technical Debt and Increase Systems Flexibility

Accrued and compounded over time, technical debt increases with rigid IT architecture, dependent satellite applications and staff attrition. Custom architecture is common in just about every industry, and is powerful when first put online. Over time, though, the systems used become outdated and do not support newer technologies.

What Exactly Is Technical Debt?

What is technical debt? Ward Cunningham, the developer of the Wiki concept and co-inventor of the foundations of Extreme Programming (XP), describes technical debt as a wonderful metaphor for having done things the quick but dirty way, setting yourself up with a technical debt to have to deal with later – a concept which is similar to a financial debt. Consultant Martin Fowler also likens technical debt to a financial debt – it incurs interest payments over time. This interest comes in the form of all the extra effort that is needed in future development due to the earlier quick-but-dirty design choice.

For example, if your core architecture is legacy-driven and you want to launch an e-commerce service, the IT department has to bend the core architecture to work with the newer technology. Legacy systems weren’t intended to “bend” in this manner, so now you have a custom code “bridge.” The new Web-based application is dependent on the core IT systems as a gateway to populate, authenticate and update key information for the company, customer and various financial bodies. All of these key connections require custom-coded “bridges.”

Long-term Ramifications

Extrapolate this one example over five years, and your IT department has “bent” the core architecture perhaps 50 times - creating thousands of these custom coded “bridges.” If all of this work was done internally, how many of the original developers are still with the company now? Did they leave easy-to-follow documentation for their replacements to follow? Chances are, you’re going to be left with one or two “heroes” - those needed employees who are the only ones in the company who can fix any sort of problem with the entire IT system. This can increase service outage and create a bottleneck for new service rollout.

With each application dependent on the core architecture, a seemingly unconnected application error somewhere on the network can impact many other services. This can take some offline and deny access for countless customers. Now that thousands of custom-coded “bridges” exist, finding the source of the problem can be time-consuming and difficult to pinpoint. This can prevent services from being restored quickly. With everything interconnected now, the core IT architecture is rigid and brittle.

Why Does Technical Debt Hit the Bottom Line?

IT system performance is paramount to any company today. Whether it is new service rollouts, fixing system failures, staffing costs or security and compliance-related processes, legacy IT systems impact the bottom line. Going back to the “heroes” concern, that singular developer resource is now a threat to the performance of the company. If that person leaves, you’re in trouble. If they stay, you may be forced to pay far more than the individual’s talent level would ordinarily warrant.

Many companies install policies that require all IT work to comply with fixed, corporate release windows. This rigid IT development window impacts a company’s ability to stay competitive. For example, the recent lending crisis has created a major revenue stream for financial institutions to tap into the “down market” opportunities of medium- or small-cap investors.

Institutional financial companies are losing major amounts of money by not being able to offer these customers services. In return, they cannot benefit from the millions of daily transactions for which these markets account. If the corporate IT release window doesn’t open up for three more months, a more agile competitor may swoop in and steal the market. At this point, mounting technical debt will have forced you to forgo millions of dollars in revenues.

What Is a Solution?

A common solution to technical debt is to literally throw everything away and start fresh. There’s some validity to this strategy. However, two very important “bottom-line” issues remain. First, the cost alone can be monumental - not to mention the staff’s learning curve, rollout glitches and unexpected downtime that accompanies any major IT system replacement. Second, if technical debt is not kept top of mind, you might be climbing out of the existing hole just to begin digging another one.

How Do Agile Practices Remove Technical Debt?  

Understanding what Agile application development really means is important in order to get out of debt. In a simplistic view, Agile provides the “gold seal of approval” which guarantees that a set of best practices are being followed during the planning, development and testing phases of a project. The following four Agile practices are key to relieving technical debt while creating a road map to keep your institution from accruing even more debt:

1.  Decoupling applications and creating independent applications

2.  Refactoring and simplifying both new code and any code “touched”

3.  Establishing repeatable automated unit and functional testing

4.  Creating documentation for only required business needs

It is possible to eliminate the current technical debt without the assurance of Agile. However, the debt cycle may begin again. Application development with Agile starts by analyzing the existing IT structure, determining where the business value is strongest, and then working on the most important coding first. To do this, dependent applications must be decoupled and made to stand on their own - independent from the core IT architecture. This saves overall project time and provides usable applications while others are “under construction.”

Automated testing can cause huge overhead, and be a massive undertaking, if a re-development project involves a large amount of legacy code. It is important to balance the effort required to do automated testing against the amount of regression testing that is being done every time the new solution is worked on. 

Similarly, refactoring legacy code can be an equally voluminous task. The business value for having a simple code base that can be easily maintained by your developers is clear. But while working with legacy applications, a balance needs to be made. A development team should be looking to refactor everything they touch during the development lifecycle, including the legacy code. 

With an Agile project, you’re assured the development team is working on key functionality - not putting effort into unnecessary, bloated documentation. Keeping the developers focused on writing code, rather than writing sentences, will result in more business value in less time - as well as a reduction in rework on low-priority documentation.

 David Webb currently serves as VP, Professional Services, for Exigen Services. He has been working with distributed Agile teams for more than five years, establishing practices and metrics to effectively manage and execute offshore Agile projects. David lives in London, England. He can be reached at david.webb@exigenservices.com.