Technical Debt is a term which captures sloppy code, unmaintainable architecture, clumsy user experience, cluttered visual layout, bloated feature-sets, etc. My stance is that the term, Technical Debt, includes all the problems which occur when people defer professional discipline — regarding any/every technical practice such as product management, visual and UX design, or code.
I assert that the change we need to catalyze in the business community is larger than any one discipline and I am worried that I have seen an increase in blog articles in recent years about concepts like “Design Debt”, “UX Debt”, “Experience Debt” — these articles unfortunately are not helping and have served only to divide the community. They are divisive, not because we shouldn’t be discussing the discreet facets in which Technical Debt can manifest, but because authors often take a decidedly combative approach in their writing. Take these phrases for example:
“Product Design Debt Versus Technical Debt” written by Andrew Chen
“User Experience Debt: Technical debt is only half the battle” written by Clinton Christian
“Design debt is more dangerous because…” written by James Engwall.
I agree with Andrew Chen that Product Design Debt is a problem — I just don’t like that he chose to impose a dichotomy where there is none. Why must he argue one “versus” another? Clinton Christian has implied that we’re in a “battle”. James Engwall has compared the “danger” of Design Debt relative to Technical Debt. These words are damaging, I argue, because they divert attention to symptoms and away from root causes.
The root cause of Technical Debt is that people forget this simple principle of the Agile Manifesto: “Continuous attention to technical excellence and good design enhances agility.”
The root solution to Technical Debt — all of its forms — is to help business leaders realize there is a difference between “incremental” development and “iterative” development so they may understand the ROI of refactoring. No technical expert should ever have to justify the business case for feature-pruning, refreshing a user interface, refactoring code, prioritizing defects. Every business leader should trust that their technical staff are disciplined and excellent.
Yes, please blog about UX Debt and Product Development Debt, etc. But please do so in a way that encourages cohesion and unity within the Product Development community.
Whenever I run a Certified Scrum Product Owner training session, one concept stands out as critical for participants: the relationship of the Product Owner to the technical demands of the work being done by the Scrum team.
The Product Owner is responsible for prioritizing the Product Backlog. This responsibility is, of course, also matched by their authority to do so. When the Product Owner collaborates with the team in the process of prioritization, there may be ways which the team “pushes back”. There are two possible reasons for push-back. One is good, one is bad.
Bad Technical Push-Back
The team may look at a product backlog item or a user story and say “O gosh! There’s a lot there to think about! We have to build this fully-architected infrastructure before we can implement that story.” This is old waterfall thinking. It is bad. The team should always be thinking (and doing) YAGNI and KISS. Technical challenges should be solved in the simplest responsible way. Features should be implemented with the simplest technical solution that actually works.
As a Product Owner, one technique that you can use to help teams with this is that when the team asks questions, that you aggressively keep the user story as simple as possible. The questions that are asked may lead to the creation of new stories, or splitting the existing story. Here is an example…
Suppose the story is “As a job seeker I can post my resume to the web site…” If the technical team makes certain assumptions, they may create a complex system that allows resumes to be uploaded in multiple formats with automatic keyword extraction, and even beyond that, they may anticipate that the code needs to be ready for edge cases like WordPerfect format. The technical team might also assume that the system needs a database schema that includes users, login credentials, one-to-many relationships with resumes, detailed structures about jobs, organizations, positions, dates, educational institutions, etc. The team might insist that creating a login screen in the UI is an essential prerequisite to allowing a user to upload their resume. And as for business logic, they might decide that in order to implement all this, they need some sort of standard intermediate XML format that all resumes will be translated into so that searching features are easier to implement in the future.
Because that’s not what the Product Owner asked for. The thing that’s really difficult for a team of techies to get with Scrum is that software is to be built incrementally. The very first feature built is built in the simplest responsible way without assuming anything about future features. In other words, build it like it is the last feature you will build, not the first. In the Agile Manifesto this is described as:
Simplicity, the art of maximizing the amount of work not done, is essential.
The second feature the team builds should only add exactly what the Product Owner asks for. Again, as if it was going to be the last feature built. Every single feature (User Story / Product Backlog Item) is treated the same way. Whenever the team starts to anticipate the business in any of these three ways, the team is wrong:
Building a feature because the team thinks the Product Owner will want it.
Building a feature because the Product Owner has put it later on the Product Backlog.
Building a technical aspect of the system to support either of the first types of anticipation, even if the team doesn’t actually build the feature they are anticipating.
Okay, but what about architecture? Fire your architects. No kidding.¹
Good Technical Push-Back
Sometimes stuff gets non-simple: complicated, messy, hard to understand, hard to change. This happens despite us techies all being super-smart. Sometimes, in order to implement a new feature, we have to clean up what is already there. The Product Owner might ask the Scrum Team to build this Product Backlog Item next and the team says something like: “yes, but it will take twice as long as we initially estimated, because we have to clean things up.” This can be greatly disappointing for the Product Owner. But, this is actually the kind of push-back a Product Owner wants. Why? In order to avoid destroying your business! (Yup, that serious.)
This is called “Refactoring” and it is one of the critical Agile Engineering practices. Martin Fowler wrote a great book about this about 15 years ago. Refactoring is, simply, improving the design of your system without changing it’s business behaviour. A simple example is changing a set of 3 radio buttons in the UI to a drop-down box with 3 options… so that later, the Product Owner can add 27 more options. Refactoring at the level of code is often described as removing duplication. But some types of refactoring are large: replacing a relational database with a NoSQL database, moving from Java to Python for a significant component of your system, doing a full UX re-design on your web application. All of these are changes to the technical attributes of your system that are driven by an immediate need to add a new feature (or feature set) that is not supported by the current technology.
The Product Owner has asked for a new feature, now, and the team has decided that in order to build it, the existing system needs refactoring. To be clear: the team is not anticipating that the Product Owner wants some feature in the future; it’s the very next feature that the team needs to build.
This all relates to another two principles from the Agile Manifesto:
Continuous attention to technical excellence and good design enhances agility.
The best architectures, requirements, and designs emerge from self-organizing teams.
In this case, the responsibilities of the team for technical excellence and creating the best system possible override the short-term (and short-sighted) desire of the business to trade off quality in order to get speed. That trade-off always bites you in the end! Why? Because of the cost of fixing quality problems increases exponentially as time passes from when they were introduced.
Refactoring is not a bad word.
Keep your code clean.
Let your team keep its code clean.
Oh. And fire your architects.
Update Sep. 8, 2015: Check out this YouTube video on the closely related topic of who has authority over the Product Backlog and why developers should not set the order of PBIs:
¹ I used to be a senior architect reporting directly to the CTO of Charles Schwab. Effectively, I fired myself and launched an incredibly successful enterprise architecture re-write project… with no up-front architecture plan. Really… fire your architects. Everything they do is pure waste and overhead. Someday I’ll write that article
Solving technical problems is the job of the product developers on the Scrum Team, not the Product Owner. The Product Owner is responsible for the product from a business and user perspective and has authority over the team only in this limited realm. By overstepping the bounds of authority in this way, the Product Owner becomes an obstacle for the team by reducing its ability to self-organize. A Product Owner who is part of a team that has reached a high-performance state may be able to safely make technical suggestions, but should always be careful not to push the team to accept those suggestions.
The Product Owner’s main responsibility is to maintain the Product Backlog through direct communication with the users and stakeholders. To do this well it will take most of his time and effort to be effective. Hands-on technical is done by the Team Members not the Product Owner since this is not the Product Owner’s strength or area of expertise. If the Product Owner refrains from doing the hands-on technical work, then he is able to provide the vision and share the “what” that the team needs to accomplish. If not, he will be bogged down by the tasks and lose the time and ability to provide product guidance and connect with the stakeholders.
Solving technical problems is the job of the product developers on the Scrum Team, not the ScrumMaster. The ScrumMaster is responsible for the Scrum process and has authority over the team only in this limited realm. By overstepping the bounds of authority in this way, the ScrumMaster becomes an obstacle for the team by reducing its ability to self-organize. A ScrumMaster who is part of a team that has reached a high-performance state may be able to safely make technical suggestions, but should always be careful not to push the team to accept those suggestions.