Wow, what a weird feeling. The authors of this article seem like super smart and experienced guys, but the article itself is introduced with incorrect statements in the very first two sentences and slides downhill from there.
"There is a kind of design distortion that happens when a team chooses to build iteratively instead of looking at all of the requirements at once. Ward Cunningham coined the term technical debt to describe those design distortions."
1) There isn't an option to build software by "looking at all of the requirements at once." The requirements of any (sufficiently complex) project will emerge over the course of development, regardless of whether it's built in an iterative style, or with a much larger investment in planning and design up front. We often work iteratively because we acknowledge this particular aspect of reality and it helps everyone when we work closer to reality, rather than fighting it.
2) Technical Debt does not describe design distortions that arise due to "iterative" development, it describes design problems that arise in every project.
Quantizing the choices for dealing with tech debt into 4 buckets doesn't feel right either. Changing the design of a running system happens along a continuum and the need and benefit vary widely over the course of any given software lifecycle and surrounding (business or other) environment.
Maybe I'm just grumpy this morning, and I think reasonable folks could disagree, but the metaphor at the center of the thesis (tech debt ≅ soon-to-be-deallocated-memory) doesn't hold up for me at all.
Tech debt is sometimes left in place, intentionally or not, for many years. Sometimes we chip away at it, sometimes we stop the world and push on it. Sometimes it's bad enough to throw the whole system away and start over. It's like debt for businesses. It can be valuable to take on some debt if it lets you stay alive long enough to pay it back.
Finally, I'm struggling with this article because tech debt is already a metaphor, that includes information about how, and when to take it on and pay it down.
It's not helpful to layer another, unrelated metaphor on top of it.
Ward Cunningham's original idea of tech debt (see [1], a beauty of concision at just 300 words) is that iterative development distorts your code because you start writing code before you know the requirements, but even so, it's better than waterfall. "The traditional waterfall development cycle has endeavored to avoid programming catastrophy by working out a program in detail before programming begins. We ... [instead use] the alternative, incremental growth ..."
Today, the term "tech debt" includes sloppy code, shortcuts, novice code -- really any kind of bad code. The original conception of tech debt is more limited and tied to waterfall vs iterative process choices. [2]
I would argue that there is the option to look at all the requirements at once. The disconnect with your mental model is in the definition of the piece of software being designed. You’re considering all iterations to be the same piece of software, despite ostensibly having different sets of features and functionality (e.g the ones added over time).
Consider the alternative perspective that each new feature actually makes it a new piece of software. You were able to fully design and specify the original piece up front, thus avoiding technical debt on that piece of software.
I don't think that perspective really holds up to practical scrutiny. The most common sources of tech debt in the real world are when new features are tacked onto existing software without integrating with them fully/smoothly.
As a relatable example, think about how the Control Panel in Windows still uses a 2000s-era UI, even as the Settings menu, and most of the rest of the OS, uses a new UI. The likely reason for this is that it was faster to tack on a new UI and leave the Control Panel as-is. It would've taken more time to refactor the Control Panel from the ground up to accomodate a new UI.
The end result is that there are now two separate settings interfaces, and there is probably some ongoing engineering effort required in maintaining both and keeping them coherent with one another. That's a classic example of tech debt - save time now, but it may cost you later.
But, by your definition, the old UI and the new UI are separate pieces of software, therefore there is no debt. How does that track?
There are some bits of software that are simple enough to describe first and then implement, but just about any software that has more than one person working on it is more complex than that (IME), and for those cases, one might as well just implement it anyway.
There are varying degrees to which a team or business may invest in collecting and negotiating requirements before starting development, and there is obviously some amount of this work that is essential and valuable.
That said, as this planning investment grows beyond some reasonable point, and moves deeper into technical details, it will be composed of more waste and this waste will only become apparent as the people executing discover the real technical and product requirements.
The technical debt we're discussing will become an option immediately as the engineering work begins, and it's up to the team to decide how much, if any debt they are interested in taking on at whatever point in time.
I'm not convinced that it's helpful to conceive of the thing we're all working on and becoming more and more intimately familiar with as a new thing over some randomly selected interval.
There is no reality in which we can fully design or specify everything up front. If the specification were comprehensive, it would be the solution.
In practice, *practice*, we use the term iterative pretty loosely. In fact sort of like this IEEE article, it feels dated now, and was never really true in my experience.
It seemed like the word iterative became a euphemism for “dont worry about it”. I have worked across the entire industry for 30 years, and I have never seen a “true iteration”. On any feature or application. Anything at the end of the sprint that doesnt work is just a bug. Thats not iteration.
Iterative development was just a powerpoint slide for the Scrum Industrial Complex which was hard selling itself and it worked!
This article which seems even more antiquated than Id like to describe, seems to confuse iterative with “having all the requirements at once”. Iteration and the quality of requirements, and the bandwidth to analyze them and design a system dont really have much to do with each other.
And thats why iterative came to be a stand in for “dont worry about it”.
I personally feel that the creators of Scrum or some responsible group of modern architects, should audit the results of supposed iterative development, and do an honest assessment of what it actually is in practice. Because its not what we were sold, and its not iterative. It does result in awfully bad software but there have been other shifts in development practice that have accelerated the awfulness of delivered work in the industry, and Ill just leave it at that.
Please resist the temptation to comment, “because your not doing scrum right”. That was the genius of the Scrum Industrial Complex, to suggest that somehow this simple thing is really magical and complex and keep repeating the same things over and over. Its not magical or complex, its just bad.
As far as the article goes, I have been on projects that did spend over a year ingesting requirements and doing design before starting to code. Its true they usually fail. But thats a money/commitment/time to market problem.
Starting to “code” right away creates some better optics and fewer arguments in meetings. The one who pays the piper calls the tune as always.
"There is a kind of design distortion that happens when a team chooses to build iteratively instead of looking at all of the requirements at once. Ward Cunningham coined the term technical debt to describe those design distortions."
1) There isn't an option to build software by "looking at all of the requirements at once." The requirements of any (sufficiently complex) project will emerge over the course of development, regardless of whether it's built in an iterative style, or with a much larger investment in planning and design up front. We often work iteratively because we acknowledge this particular aspect of reality and it helps everyone when we work closer to reality, rather than fighting it.
2) Technical Debt does not describe design distortions that arise due to "iterative" development, it describes design problems that arise in every project.
Quantizing the choices for dealing with tech debt into 4 buckets doesn't feel right either. Changing the design of a running system happens along a continuum and the need and benefit vary widely over the course of any given software lifecycle and surrounding (business or other) environment.
Maybe I'm just grumpy this morning, and I think reasonable folks could disagree, but the metaphor at the center of the thesis (tech debt ≅ soon-to-be-deallocated-memory) doesn't hold up for me at all.
Tech debt is sometimes left in place, intentionally or not, for many years. Sometimes we chip away at it, sometimes we stop the world and push on it. Sometimes it's bad enough to throw the whole system away and start over. It's like debt for businesses. It can be valuable to take on some debt if it lets you stay alive long enough to pay it back.
Finally, I'm struggling with this article because tech debt is already a metaphor, that includes information about how, and when to take it on and pay it down.
It's not helpful to layer another, unrelated metaphor on top of it.