For years the team was told to deliver features, that the upgrades to node dependencies weren't of a high enough priority. Now some of the dependencies are so old they're complicating upgrading past node 12.

Yet the cybersecurity people are demanding that vulnerabilities be fixed.

Thankfully we have a stronger product owner now, and spent most of the last 6 months getting things updated. Yet there are still those in senior leadership who openly question why our team hasn't delivered the features they want.

We're working on educating them. And something pithy and catchy is useful, so I'm definitely going to mention this in future: "fixing tech debt has a price, not fixing it has a cost."

It's the kind of thing that if you can get a "cool" or influential senior leader to repeat, the rest will fall in line.

And don't get me started on the Cold Fusion code I wrote in 1997 that probably still is buried somewhere in the Epic software that generates mycharts for HIPAA compliant doctor/patient reporting. Last year I was almost arrested for hacking when my real name triggered a security flag that I might be doing something evil when I really just needed a colonoscopy.

Think the internet is forever? Try being scowled at for innocent code you wrote as a kid.

And stay off my lawn...

I think what kills me most is when you encounter these codebases in the real world and you're like "wait … this entire startup is younger than Python 3 significantly", and younger than the point where there was good support from the ecosystem…

Double bonus points these days if younger than the Python 2 deprecation date…

(And yet every security team lists some variant of "keeping stuff updated" in their process docs, somehow.)

If you work at tech companies led by tech people who think of their companies as tech companies, yah, you'll probably be on new(ish) versions.

Most software isn't written at companies like that; it's written at boring business companies (like banks and utilities) that are led by business people who say "what do you mean to want to update java? Isn't it working fine? That would cost us a huge amount of money and be super risky and it's not even end-of-life. Go write more Java 8."

> "... it's not even end-of-life. Go write more Java 8."

Sounds like Oracle is enabling this response by not EOL'ing it. Just like IE didn't die completely until Microsoft stopped supporting it.

Source: trust me, bro

Talk to them. They may have your back. It was always a happy day when the security audit we were going through at that moment wanted us to attest that all our dependencies were sufficiently updated. That meant we had an external mandate to do the work we'd been wanting to do for ages anyway.

When I was a CISO, I would've been glad to "order" one of my colleagues to do that if they asked me to.

Is this _really_ so hard for stakeholders to understand? I've found some limited success by dual-boating -- upgrading as features are created. "We can't build this feature without an upgrade" carries a lot of weight -- but sometimes you just have to bite the bullet and demand an upgrade.

Are stakeholders completely oblivious to the existential threat to their bonus if their company is hacked and driven to bankruptcy in lawsuits? Sure some bigs can avoid this, but not the mids and smalls.

for decades now, every generation of programmers has been larger than the previous one.

every generation of engineering managers has probably grown in a proportionate way.

so, for decades now, every generation of engineering managers has been learning the same basic "how software differs from manufacturing" lessons over and over again.

(partly because business schools don't seem to have learned this lesson yet even once.)

Can't find the sources for the OG argument about programmers, but it does sound plausible.

Most of what people see in open are companies that get hacked, share price dips for a month or two if it is software company, if it is non software company share price does not even dip.

Okta went down quite a lot since 2022 but still far away from bankrupt and latest "hack" did not move it that much really.

Where in my opinion (as a tech person) they should be out of business already even after slightest "hack".

Microsoft Azure latest hack did not really affect their stock...

Not sure if the actual cause of bankruptcy was lawsuits, action by authorities, or just loss of business caused by the publicity.

*In April 2023, Helsinki District Court did sentence the ex-CEO of Vastaamo, Ville Tapio, to a three-month suspended sentence. It found him guilty of a data protection crime mandated in the General Data Protection Regulation (GDPR).*

In my experience in the medium term (~2-5 years) it doesn't even slow down feature delivery since high technical debt can have an absolutely catastrophic effect on development speed.

At the previous company I worked in I'd say we delivered work at about 50% of the potential rate because the infrastructure they'd developed was so janky. A big Python/YAML/Makefile mess with zero static types. CI time was over 2 hours and used hundreds of CPU-hours, even for fixing a typo.

sympathetic-lol at the desperate copium.

There's 2 "cool" directors who were tech people but are also influential in that peer group. You can't trust them to represent you fairly, but you can trust that they'll listen to your argument.

The 40% copium stems from knowing that some of the "only features have business value" group are stubborn and think everyone should just work more.

Good work takes time.

But I think it is an optimization problem. Rather, that different people are optimizing different things. Your team is optimizing long term profits. The senior leadership who's frustrated is optimizing short term profits. I think one way to handle these situations is to clarify which is the more important optimization problem. That makes points like "opportunity costs vs marginal cost" discussions. The same way we turn down immediate profits to go to school, because in the long run we make more money (and for other reasons). But our systems often push us to focus on short term goals because these are far easier to measure.

I don't think we as engineers can ethically ever let management optimize our work for the short term. They will ALWAYS want the short term improvement and by even giving them that decision we abdicate our own judgement. That abdication is totally unethical.

We have an application that runs on Java 6.

Looking at the historical maintenance cost of the stacks when choosing them and prioritizing ones with a better history of backwards and forwards compatibility in their ecosystem does far more to prevent tech debt pile up than trying to reach a strict habit of bending over backwards to update dependencies.

One refactored everything endlessly to a fault, wrote tests for everything and emulators for every external piece of hardware.

The other rarely/never refactored and was focused on ends-justify-means functionality.

They hated each other and progress was very slow because they could not work together at all.

My two cents is that all products of engineering are temporary. Software (and hardware) are not eternal. What you write today you will eventually become obsolete.

Your entire app will eventually be re-written. If not by you then by your competitor who takes you out.

There is no such thing as perfect or optimal code because what is great for now won't be great for tomorrow.

With that in mind, good refactoring is like good math. Taking a long and complex system of code and replacing it with something functionally better and far simpler and easy to understand.

Bad refactoring is just moving things around for "understandability" or in a lot of cases one developer rewriting something in their own idiosyncratic style because they don't understand the style of the programmer before them.

And the final point is don't be afraid to rewrite the entire system if you have that ability.

Since the development process is about defining functionality as much as enacting it, once you have a finished application, you can re-create it much faster and simpler than it was originally written.

The Apollo missions got to the moon and back with 145000 lines of code you don't need 20 million + for your web app.

Been there. And often gets worse when the "ends-justify-means" camp (1) collects velocity points (that's the most important thing, they decided to gain the system that way) and people who care about velocity (typically management) are happy but... (2) leave a path of destruction in their wake that slows everyone else down, decreasing the productivity of everyone

With refactoring, consider 2 things (1) How bad / ripe for simplification is the code (2) How much is it changing

If its bad stuff but nobody needs to deal with it, I say leave it unless it has dependencies (those sneak up on you too and prevent perhaps a library upgrade that would benefit the rest of the app)

No... you should absolutely be afraid of that.

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

Rewriting your entire system is almost always the wrong thing to do and has been the end of plenty of products and companies. I think Joel is spot on here. It's much more difficult to read code than to write code, so new developers always think existing code bases are junk and need to be scrapped while ignoring the value that platform has delivered over years if not decades. Developers aren't paid to be craftsmen. They are paid to deliver a product that provides business value.

Too often developers lose sight of the business value and want to write code for the sake of the craft. You're working for Ikea and complain that you can't use your fancy woodworking tools and techniques and have to stick to dowel based assembly instead of that fancy joinery you'd rather be working with. Those fancy joints are stronger and more stable after all! And what the fuck is up with all the particle board?! All real woodworkers know the stuff is junk and will fall apart as soon as it gets wet. We should be working with mahogany and walnut with cocobolo inlays!

There are places where your fancy joinery skills are appreciated and none of those expenses are spared. But most developers are working at an Ikea or equivalent where the predictability of cost and process is well understood and a reliable product can be delivered at a reasonable price. If you want to be a craftsman, you need to work at the places who are looking for those skills and not just the places who want flat pack furniture quickly and cheaply to keep delivering value to their customers.

"If you have that ability"

I think I specifically called out the danger of the average developer rewriting code because they don't understand it.

But If you DO understand the code base and you do have the time/resources/vision you can rewrite as a form of radical refactoring, taking the lessons learned over the original development and compressing them into a newer system that allows some kind of business expansion/pivot/innovation.

Yes all change requires risk but there is risk in not changing as well.

Companies go under just as often from NOT rewriting or NOT building something new/better.

If you don't truly understand the whole system and you don't have a business case for what your new system would do then DON'T rewrite.

On the flip side, if you DO understand a 20million+ line codebase of your company's web app and you know you could rewrite it all yourself/with a small team and make it 10x better then definitely don't waste time arguing with your boss about whether refactoring is a good idea.

Quit and start your own company because you've identified a massive inefficiency in the market to exploit.

True, but "eventually" can vary wildly.

Web app front-end written in framework-du-jour? Yeah it will probably be gone in a few years.

OS kernel functions? May still be running unchanged 30 years later.

How much to invest in design vs how fast can things be done w/o such an investment is a difficult problem. Very similar in nature to https://en.wikipedia.org/wiki/Multi-armed_bandit .

Sometimes the program you are writing is so worthless, it'll be thrown out tomorrow. Sometimes it may outlive you. The later is rare, but not impossible. Eg. my former boss has some of its code flying on an satellite, circling the Earth some 30 odd years.

My personal oldest code that is still alive (that I'm aware of) is some minor stuff in cl-gd library (Common Lisp bindings to GD library). It's about 15 years old now.

For my day job, I work for a company whose codebase was started some time in late 90's. While things have changed over time, some, and especially design decisions are still there. A lot of them are painfully wrong, and have been a burden on this company for the last 20+ years.

Oh, and should I mention the 32-bit sized IPv4 addresses?

----

How much should one invest into design? how often should cornerstone ideas be revisited? -- Definitely not nothing and definitely not never. But how to know when it's the right amount? -- I don't believe anyone has a strategy for finding out. Good (or lucky) project managers will have a good guess about the timing.

Maintenance mode products aren’t growing revenue. They might get a few new features but the main goal is to not mess with the cash cow unless you need to patch or make a special customer a new feature. It will fade into irrelevance with time as something new replaces it.

Projects that are growing need to scale and adapt to new environments and use cases. If you treat these with disrespect then your business will suffer and you’re less likely to make that new feature or big pivot needed to become a cash cow.

As a further elaboration, I'm trying out that this gets you:

- Better position. Code's better, so product is better (more stable, with better features, etc) - Higher velocity. Each time you do this, the overall codebase gets easier to change, because you're continually making it easier to change - High acceleration, because you're giving your devs a way to stretch their capabilities and grow their skills.

Definitely gonna see how "Refactoring has a price. Not refactoring has a cost. Either way, you pay" plays with folks!

PS -

AFAICT, future proofing is bad when you build stuff to aim it, mostly because you probably won't end up with the problem you think you will, so you're not actually able to build it correctly now.

But!

You can build in gaps, for where that new stuff can go. You can future proof pretty much only by writing your code today with eye towards making unknown changes later - ye olde "the only architecture that matters is the one that lets you change".

Note I say factor rather than refactor since in most cases the existing code wasn't factored in the first place. It's also a reminder that we should know what that factor is before making changes. In some cases there is a new reason that changes which to factor but is less common IMO.

The design was made without a diligent machine like a computer with compiler checking it. Design reviews (by people) are boring and it hurts when someone pokes holes in one's design that has come to them after a few days and or nights of thinking. Also it is often based on unclear requirements.

So that's how code is born, which is based on suboptimal or even backward designs.

As soon as there is need for refactoring, usually only the code itself (a whole of tiny design decisions) is revamped, keeping the initial design alive.

BUT I do think at times we need to slow down. Too much "move fast and break things" and sprints have many opportunity costs. Long term optimization is a hard task, so people typically focus on short term gains, which we should be aware can leave a lot of money on the table. So don't confuse moving fast with maximizing profits (e.g. we invest in ourselves by going to school turning away immediate profits for higher future profits. But this is harder for businesses to do). A good manager/CEO should be one that is learning and focusing on this optimization problem, not the next quarter or ill defined metric.

> Design reviews (by people) are boring and it hurts when someone pokes holes in one's design

I think there's a lot to be said here too. A not uncommon gauntlet many scientists have to go through is getting their work trashed on. With many lab groups purposefully holding mock presentations and being overly critical to help train the junior scientist's confidence. We should do this in review too, making sure to do it with juniors. It's slower, but pays high dividends.

But review, in any form, has another problem. The reviewers themselves. Paid or unpaid, they often frame their job as being to find flaws in a work. Instead, their actual job is to make the work the highest quality it can be. If you frame the reviewer reviewee relationship as adversarial then you're (in general) degrading the system. Rather this always needs to be framed as cooperative, where you're working together to make the work better. This means that it is perfectly okay for a work to be accepted with no comments, and in fact that should be the goal of any QA system. That your job often is closer to a teacher and your success is reducing your work. But if we frame adversarially you're going to use metrics like the number of QA comments and rejects to determine how good your reviewers are. In turn, this will be a cobra effect and people will purposefully create low hanging but easy to fix mistakes so that reviewers can point to something (demonstrating they did their job and their need) but reviewees can fix the problems and everyone is happy. But that's just Goodhart's law in action and isn't optimizing the product/profit/work/whatever. This does need to be kept in mind since many metrics are nowhere near aligned with the actual thing being desired to be optimized (and there's environmental drifts too).

All of these are balancing acts. It's never 'refactor or not', it's how much to refactor and what to let go? These are strategic issues and they go hand-in-hand with business goals and knowledge about the longer term roadmap. For instance it would be pointless to refactor a codebase that will end up being supplanted by another product or for which a strategic acquisition is planned of a company with a technically superior product.

That means that refactoring has an increasing cost as well (with increased frequency, the total money spent on refactoring increases).

The question is more about complexity and interactions. Does a specific subsystem interact with N modules, or with 1. If N+1 module interacts with N, then the cost of maintaining that system will explode very quickly. OTOH, if the system is revisited after that test goes green to increase modularity, reduce dependencies, then complexity is now potentially growing sub-linearly to the number of modules rather than O(n!)

The best testing is battle testing - if it is working well after plenty of contact with the customer then leave the code alone.

Bitter experience has taught me this.

---

I'm going to expand on my comment a bit - we need to be clearer on what (production) code actually is. We believe it is the definition of the runtime behavior of a given system (and of course it is exactly that).

However, for any reasonably complex system, a complete understanding of the definition/behavior is beyond any single person. Thus code is in fact a recording of the social and technical process of producing the system, and as such embeds decisions and solutions that may not be evident to anyone just reading the code.

When viewed in this light, it is easy to see how dangerous refactoring can be.

If you write some code and you don't know how it works, it's most likely not correct in some edge cases.

Code that is written to be so simple it's obviously doing the right thing is better than battle tested mess of if statements.

I've given interviews of fizzbuzz and even though developers usually recognize they need a separate case for 15, they don't always do the else ifs right (maybe test for the 15 case last when you've already hit the divisible by 3 case first, or similar silly mistakes). In that case, doing the string accumulation refactor makes it clear that the code has no mistakes because it for sure adds a Fizz first and a Buzz second in the 15 case. It's also a good refactor for extending to Bazz (divisible by 7) as well, so it's something you will want to do when you want to add that feature

Code that is written to be so simple it's obviously doing the right thing is better than battle tested mess of if statements."

Only if it actually do the right thing in ALL cases. In a lot of refactors I have seen (and done myself) that has not been true.

If you don't understand the problem enough you may hack at it in another wrong way.

If a more advanced question is posed, then the assumption is that the requirement is for the prospective employee to accomplish even more advanced tasks in practice.

I see no value in the "gotcha" of you made a logic mistake since we all make them and there's a reason why we test code

I wrote if (n % 15) print "fizzbuzz" ...

I got away with it because my interviewer assumed it was some kind of shell which means 0 is true

I should have written n % 15 == 0 to be absolutely clear what I'm doing

note that in Rust this won't even compile:

    3 |     if n % 15 {
      |        ^^^^^^ expected `bool`, found integer

I don't think I've ever interviewed anyone like this, so far?

The comment is bit too bizarre to puzzle out.

The most foolproof way to avoid car accidents is to avoid ever being around a car.

Surely that's not the only objective, right?

Some people tinker with adding mods and things to their cars which increase the probability of the problem.

Would you rather fly on a plane that has been used daily for the same flight for the past 10 years? Or fly on a plane that’s either never flown, just got out of the shop, or hasn’t been flown in years?

But you also can't go and redesign the entire thing.

Classically speaking, refactoring is only for code that has thorough coverage. Refactoring code that is not under test might be better worded as rewriting.

Obviously, rewriting critical and complex code that has evolved over time with a variety of fixes and random features bolted on with random conditionals - obviously that is risky

Except that isn't even remotely true. Perhaps if you system has a purely machine interface, with a limited 'dictionary' of interactions, then a test suite could replicate deployment to a very large degree, but otherwise at best it can give only an indication of how use-worthy the system is.

Take a (reasonably complex, with human interface) production system, implement as much testing as you like, and I can guarantee any significant refactoring will introduce a regression, and quite possibly a serious one.

The problem with testing is that you need to know what to test. So unless you have some method of capturing every possible way a system is actually used, you will always run into the 'unknown unknowns' that you didn't realize even needed testing.

Refactoring suffers from having almost no meaning. I would interpret what you are saying as re-architecture and/or update of system design. That is what most people mean by refactoring today. The article laments that people skip the part of "make it clean" in the "make it work, make it clean, make it fast, in that order"

In theory refactoring is a safe activity to do because you are only updating something that has a defined API which is tested.

> Except that isn't even remotely true.

This is hyperbole. How long before a system is actually fully and truly battle tested? 1 month, 1 year, 5 years? I'm making up numbers; but you could think of it in terms of iterations and code paths. How many executions of a block of logic before you start to trust it more? The passage of time adds more certainty because conditions vary over time. Without any testing, that production release is going to be iteration number one in the journey of battle testing that updated (which can be called re-written) code. All I'm saying is running that code through a battery of tests fast-forwards (some) of that battle testing time.

As an analogy, car tires - testing is like running a car tire for a few thousand miles before giving it to the customer. When getting a new one (ie: any update to the code at all), the customer is not getting a new tire with zero miles on it, where perhaps the rubber is bad and that would have been discovered easily after about 10 miles. That car tire might still have issues after 20k miles that are abnormal, but nonetheless the testing released a car tire that had more confidence compared to one with no testing at all.

I think my favorite analogy would be more RPG & DnD like games. Testing is like having your resurrected characters start at level 3 instead of level 1. Are they level 9? No, but at the same time it's far better than starting at scratch every time. Every time any modification is done to the code, you don't have to start back at level 1 on your journey to get back to battle tested.

>> Code that has been thoroughly tested is akin to a running system for a year

> Except that isn't even remotely true. Perhaps if you system has a purely machine interface, with a limited 'dictionary' of interactions, then a test suite could replicate deployment to a very large degree, but otherwise at best it can give only an indication of how use-worthy the system is

The last sentence there, I would state the exact same about battle testing. Both testing and 'battle testing' is just executing code, no real difference from that perspective. The difference comes in the form and variety of scenarios.

I'm making up the duration amount for the sake of examples, and clearly that can vary based on the system and the efficacy of the testing.

My assertion is that testing is akin to running code in production for 'n' units of time. Perhaps it's exactly the same as running that code for one second in production. Does one second in production give much more than an indication? Not really, but testing & battle testing are in the same boat.

> The problem with testing is that you need to know what to test. So unless you have some method of capturing every possible way a system is actually used, you will always run into the 'unknown unknowns' that you didn't realize even needed testing.

This is just a problem with building systems. This line of reasoning can become a bit nihilistic (why do any testing at all, manual or automated?)

Let's say a system has 10 major permutations, 2 of them are unknown unknowns. Let's say further that it takes 10 seconds running in production to be aware of half of those cases. Applying tests that cover those 5 scenarios is then akin to running the system for 10 seconds. If i have 100 modules, I change one, and there is a test suite that gives me the confidence of having run all 100 for at least 10 seconds (in varying combinations of together, in isolation, in shadow environments, etc).. then I will have effectively gained 10 seconds of battle testing before having released anything.

The difference is that one is real, whereas the other is simulated. It is very hard indeed often impossible to perfectly simulate the actual use of a system. Thus the simulation will be incomplete, and quite possibly in fundamental ways that aren't appreciated until problems occur. It doesn't matter how many tests there are or long they are run for - they can only be at best an indication of how the system will actually behave when confronted by its 'battle' environment.

Currently dealing with some 13 year old objective C code for a mobile app that’s gonna require a major rewrite cause it looks like it was written by someone who should have never been writing code in the first place

https://en.wikipedia.org/wiki/Simulated_annealing

The idea is code goes through development cycles. You have the most leeway to make changes in the first few development cycles. But less and less as time goes on. And the beginning of a development cycle you can make larger changes than at the end. Between cycles the code is frozen and you never touch it unless you can prove a change is needed. And development cycles are triggered by the need to for gross functional changes or interface changes.

Lehman's laws of software evolution say so.

https://en.m.wikipedia.org/wiki/Lehman%27s_laws_of_software_...

> embeds decisions and solutions that may not be evident to anyone just reading the code.

Strongly agree! Good code communicates this to other programmers (including yourself in 3-6 months). If the code doesn't do a good job of this, that alone can not only be worth a refactor, and when you refactor while "what it took to understand what this does" is fresh in your mind, you can write the new version that much more effective at communicating those learnings.

I agree that one should not take something like this lightly. Far, far too many "refactorings" really just boil down to hasty rage-coding at some predecessor who had differing stylistic opinions, or keeners who've never heard of Chesterton's Fence getting frustrated with some annoying design element they don't understand.

But also, sometimes the code really is tangled and difficult to maintain, such that it's difficult to make any modifications without introducing regression defects.

Concrete example: I recently did a refactoring of a piece of code my team maintained that was so extensive that, by the final iteration, it was arguably a rewrite. The old code was sprawling and complex, could basically only be comprehended by the staff engineers, and tended to be ground zero for defects. The new code is much smaller and simpler, and easier to modify, and every single member of the team is comfortable reading it.

- but -

I only did this after months of careful observation, quietly building a case for the change and a plan for how to do it. This process included interviewing everyone I thought might know anything about this module, including folks who had moved on to different teams, to make sure I understood as much as possible of what was there and why it was built that way. I discussed my plans with them and sought (and, ultimately always got) their blessing. And then I spent more time bolstering the test coverage, including hunting for edge and corner cases that the existing test suite had missed, when necessary, adding additional tests that did not depend on this module's implementation details so that they would not be broken by the refactor. And then, when I finally did do the actual refactor, I didn't do it all at once. I broke the work into many small, carefully controlled iterations.

But the problem is, even if code is battle-tested, someone always wants it to do something more. To get it to do more means changing it. And when you've changed it, they'll want another change. And then another.

So you either don't make changes (even bug fixes!) or you do. If you do, you either refactor the code as you change it, or you don't. If you don't, the code becomes more and more brittle, that is, harder and harder to change without breaking something.

So I think you are drawing wrong conclusions from true observations. You are correct that you can break code when you change it, even by refactoring it. Keep it in a state where the probability of that is minimized - where it's clear what the code is doing, and how, and why. To do that, you need to keep refactoring.

And if you are actively developing then refactoring along the way is generally better than if you wait until the very end.

People have the wrong reflexes where being overly conservative and fearful to change a working system eventually leads to a system that becomes so hard to fix that people stop bothering to even try and just replace or retire it in the end. Which is very expensive and destructive of course. Usually such decisions become a reality when people leave or change teams and it turns out there's nobody around that even has a clue where to begin fixing the old thing. Getting rid of the thing then becomes the logical choice.

Calling it technical debt is a way of talking yourself into believing you can deal with it later. Quite often that doesn't work out that way. Another fallacy is believing the debt is something with a fixed/known price that you can pay off at the time of your choosing. You can't; it's literally risk that accumulates that you haven't properly analyzed and estimated yet. Some of it might be fixable, some of it might not be. You won't know for sure until you sit down and do the work. Deferring the process of finding out is what gets teams into trouble. The less you know what you can still fix the more likely it is that the cost has become stupendously high.

Half the work is just knowing what needs doing and how it needs to be done. That doesn't mean you do the work right away. But spending a lot of time chin stroking over stuff you could do without actually doing stuff (aka. analysis paralysis) is also not productive. Finding a balance between knowing what needs doing and then just balancing new stuff vs. making sure that the cost of that new stuff isn't overly burdened by broken old stuff is the name of the game. Having a lot of technical debt usually translates into work that ought to be quick, simple, and easy being anything but those things. So, the technical debt metaphor means that you burden the cost of future essential work with that extra burden. Until you fix it, it only gets worse and worse. You pay the interest on every change until you fix the problem. And you won't know how expensive that is until you do it.

That's why you should refactor a little bit all the time. Low cost, it lowers your risk, and it keeps you aware of all the pain points and weak points in your system.

  But one day all the cruft calcifies, and the numerous incompatible changes and features grind our progress to a halt. It is suddenly impossible to add new features without breaking something else

Refactoring and paying constant attention supposed code perfection has a huge unknown price that you have to pay very soon, whereas not refactoring has a future cost. You should use the money saved not refactoring right now to make something better later once the tech debt is too high but you know you have a good working software.

In any case, this kind of article needs tangible examples to illustrate...

This is why, often, when you refactor code it doesn't make it better. You don't fully know if it's better. It's just another way. So even when you do this refactor stuff often you find yourself in the SAME situation even if you didn't touch it.

Think about it. You follow best practices, you try to write code as beautifully as possible. That doesn't actually prevent a new feature from having you to gut a huge portion of your code to accommodate it.

But developers don't realize this. They think the extra time spent refactoring puts them in a better place then before. But really they just can't see the counter factual. Did it really?

I've been in places that emphasize on good code quality and put lots of good practices in place but they end up with "tech debt" anyway. Then when clean up day comes they blame it on bad practices and shortcuts but they're blind to the fact that the they were one of the most rigorous companies in terms of code quality.

Refactoring happens because the model of the world that the software works with has changed since the software has been implemented. Maybe the way users and roles are represented was kinda OK 5 years ago, but the world (== business environment, understanding, requirements) has changed since. So the code that has been OK 5 years ago is painfully mismatched with the present.

That's when refactoring is needed.

Also. Tech debt is usually (not always) the marginal cost of the next feature. If new features cost X, and engineers are convinced that features could cost much less if the software was organised differently, then it might be useful to calculate the cost of a refactor.

Totally agreed. This is a practical reason for it. But much of refactoring doesn't happen because of this. The article says you should refactor all the time for "better" code. It's not talking about your use case where you have to update dependencies, security features and things like that.

This is absolutely false. This talk: https://www.youtube.com/watch?v=3OgbQOsW61Y does an excellent job of introducing some basic theory on good design, with solid (heh) justification for each principle.

You can see plenty of companies that follow those principles and still end up in bad places.

Theory is like number theory. You need axioms and theorems. Just because you assign acronyms and big words to things doesn't make it a "theory". You need proof or empirical justification.

Solid is like scrum, someone made it up. It was pulled out of someone's ass. Doesn't mean it's bad, but it's not categorically correct, it's more of someone's opinion.

Something that is categorically correct is complexity theory for algorithms. You can definitively say Algorithm A is better than algorithm B from a performance perspective. But for program design and organization? Is one design better than another? How Solid is one design vs. another design? Can you concretely even define Solid in formal terms? No.

I say again there is no theory on software code organization. None.

I want to have a better idea of the goal posts that you’re setting.

https://www.wikiwand.com/en/Formal_theory

The first three definitions fit.

Anything that's called "Design" usually means there's no "theory" around it. It's all ad-hoc gut feelings and made up. Things like UI or art typically have a lot of "design" associated with it.

Usually a "theory" is a very concrete thing. You don't "design" the shortest distance between two points. That's more of a calculation. A calculation is available because we have a formal theory called geometry.

We can calculate which algorithm is better because we have a formal theory around algorithm speed and memory.

But program organization? No. No theory. Just rules of thumbs and made up acronyms and big words masquerading as theory. We don't truly even have a formal definition of "good program design" or "technical debt" It's these nebulous terms and the culture shifts and changes with new fads all the time.

OOP used to be big with OOP design patterns as a big thing. Now OOP is on it's way out... Do you see that with geometry? Can the theory of geometry be a fad? You may not know it, but even SOLID is more religion than it is formal theory.

The closest thing to a formal theory of program organization I've seen is within functional programming and category theory. To get a formal theory for program organization we have to come of with formal definitions. What is good design? What is modularity? How do we compare the modularity of one program to another? None of these things have been defined yet but it looks like category theory and functional programming has sort of the primitives to build upon to make these concepts concrete. We're still far away from something formal but if we ever develop something along these lines it will likely arise from Functional programming or category theory.

Check this resource if you're interested: https://www4.di.uminho.pt/~jno/ps/pdbc.pdf

He calls it "program design by calculation" but it's really just program design using algebras. He can't prove why it's better because, again, lack of theory but it looks like this approach is generally the right direction.

If I'm understanding all this correctly (and I was never all that good at this level of math), what you're saying is that we don't have a way to, essentially, calculate the "goodness" of code using math. Or, in a practical sense, your linter can't tell you how much tech debt you've got.

And - yeah, I think you've convinced me that we don't have that. Not as a complete theory like geometry, anyway.

...but! I do think we've got some solid steps in that direction. There's a lot of things we can calculate - LoC, symbol counts, "cyclomatic complexity" - so then as I see it, the gap is connecting those metrics to "goodness" in a solid enough way that it'd cross the threshold into "formal theory". Which might not be possible, if "goodness" itself can't be nailed down.

One of the big reasons I like the talk I linked above is that I think it takes a solid (heh) step towards making that connection. You can use the techniques within to measure "how much change was used to accomplish something" (although there's issues with that measurement, from "what's the unit?" to "how do you normalize that against, say, the size of the something?" and "that's how much was used, but could less have been used?").

I'd then propose that "goodness" is something like a weighted sum of a few metrics, one of which is that "how much change was used", and further, that there are measurable things that can tell us something about how much change might be needed - although to your point, none of it a degree of completeness that'd constitute a theory.

I guess, at the end of the day - I think we do have enough to propose some basic theories about program "goodness". SOLID could be one such theory, if it were rendered into much more formalized language.

Lines of code? So less lines of code is better than more lines of code? So If I fit my program into one line it's better? That's not a good metric.

But then you say add it up with symbol counts and cyclomatic complexity and the average of the three should counterweight any inaccuracies that stem from each individual metric.... but does any of these metrics have anything to do with good design as our intuition defines it? I would say No.

The problem is our intuition of good design encompasses more than what all these things measure. The core principle that most people agree on that is fundamental to good design in programming is modularity. How do I design code so that any modification to the application is as much as possible more a reconfiguration of existing primitives rather then a rewrite or re-architecture? I want to design my coding primitives to be legos that fit together perfectly and any change is just reconfiguring those legos. However, in bad designs I find that my primitives don't act like legos. A new feature may often necessitate rebuilding and refactoring previous primitives and changing the way things work in order to get that feature in. Essentially new lego pieces are needed and editing lego pieces and changing the way lego pieces connect points to mistakes made in the initial "bad" design.

Note that None of the above has anything to do with cyclomatic complexity or symbol counts. You can have highly modular code with high cyclomatic complexity, you can have highly modular code with many LOC, and you can have highly modular code with high symbol counts. That's just one problem with your proposal.

This is the other problem:

Your "metric" is just a "metric". Theories don't deal with metrics as metrics are statistical measures... like how an average that hides the individual values. This approach is less "theory" and more "scientific" I would say. It has some validity but it treats the system as more of a black box and uses a very blurry metric to essentially guess how good a program is. But this is no theory.

>I guess, at the end of the day - I think we do have enough to propose some basic theories about program "goodness". SOLID could be one such theory, if it were rendered into much more formalized language.

It can't. SOLID is like color theory. It's made up. You know why color theory is bullshit right? It states that there are three primary colors, red, blue and yellow and all colors stem from those three primary colors when in reality color is a wavelength and our eyes only have cones for RGB. Yet people from both within art and outside of art fall for this theory because it presents itself in a beautiful way with the color wheel, secondary colors and tertiary colors. I mean somebody just made this shit up and it became convention in the field of art. Believe it or not much of computer programming outside of academia is full of color theory-esq bullshit. Take Solid:

The whole acronym SOLID is just as gimmicky. Someone chose certain words and decided to order those words so it spells "SOLID"... I mean that's more marketing then an actual attempt at a formal theory. It's just staring you in the face. It's not speaking to some foundational aspect of software design it's something someone pulled out of their ass. Like color theory it actually somewhat works, but also like color theory there are aspects of it that won't work and are utter bullshit.

You've probably been a staunch believer of SOLID for a while now. Impossible for people to change a long held view from logic alone. Such change takes time. I'm not saying SOLID doesn't work, it could be quite practical. I'm saying SOLID isn't anywhere near a formal theory nor does it have any chance of becoming an actual theory, it's like color theory... just rules of thumb for an artist and like color theory it's very very incomplete and likely not accurate to what's really going on with a "bad design".

The sad part here is that there's no alternative to turn to. Since there is no formal theory, bullshit stuff like SOLID or GOF Design patterns are much more likely to exist because we have no solid foundation to compare these things to.

Woah woah woah - I'm not saying that LoC directly measures quality. I'm only saying that it's a concrete, measurable thing. It can then be used as an input to any theory of code quality, not as a direct measure of code quality.

"But then you say add it up with symbol counts and cyclomatic complexity and the average of the three should counterweight any inaccuracies that stem from each individual metric.... but does any of these metrics have anything to do with good design as our intuition defines it? I would say No."

I understood your objective here. But my counter to it is much longer than the first paragraph. You'll have to read the whole thing carefully. There are several reasons why your approach doesn't qualify as theory and why it has problems. It's more of a statistical or analytical average then it is a formal theory.

Subjectively, Rubocop pushing me to have smaller everything by LoC made my code better. As has reducing function cardinality.

Both of those are as straightforward a metric as “angle of incidence” or “length of a side”; and I can’t help but imagine there’s theorems that can be made from them akin to the Pythagorean. Although whether it’s possible to go from there to “code quality “ - yeah, might not be possible.

Doesn't matter how you aggregate these metrics. In statistics you have all kinds of arbitrary aggregations that attempt to form a blurry "summary" of what's going on. These "aggregations" like the mean, median, mode, standard deviation, weighted average... etc... are made up as well. It's arbitrary and essentially just summaries of what's going on. You're just proposing another aggregation and obviously a blurry one too.

You're not picking out the "angle of incidence" or the "length of a side". More like out of 1000 triangles you're picking out all the sides and averaging them to get some summary of the length of sides of all triangles. You can see how this statistical metric is different from say euclidean geometry right?

A concrete formal theory involves definitions. What is the formal definition of a module? What are the formal rules for how modules compose? Can we put a formal algebra together that allows us to implement these rules and for theorems around these things? We want a formal theory of program organization that is akin to euclidean geometry, not some data sciencey aggregation.

It can be part of an explanatory model of code quality once you have a concrete metric for code quality. That is, if you can measure code quality independently of features of design, then youc an build a theory of design using measurable features of design/implementation like LoC that explains how thost features determine quality.

But otherwise your "theory" is not a predictive/empirical theory and something more like various schools of literary theory.

There's a magic to slowing things down, too - require reviews on every change, require tests to run and pass, run UI tests on each PR. Let people discuss good and bad code they see. Turn every little thing people mention, (like "man, this view is difficult to reason about", "the entire xyz class is a mess", "does anyone know why we do X here?") into a Refactor issue, discuss it, plan it, do it. You can do this during a big feature sprint, you dont need to wait until your product is done (then its too late).

Refactor, let people say "yes this issue isnt high prio, but its important to me". Embrace when people want to make code better.

Problem with this is your best devs will soon become your best former devs.

Becoming a "mandatory reviewer" sucks. Especially at scale. You're a bottleneck, you get pressured by other devs and occasionally management to let code that you don't like through, and if things go wrong you get assigned at least partial blame for not catching it during the review process.

I'm speaking from a background of seeing this implemented at a major AWS service with designated "component owners", who were 1-3 developers with a final (and mandatory) review for all code merged into subcomponents. This was for a service with 400+ developers working on it.

> There's a magic to slowing things down, too - require reviews on every change, require tests to run and pass, run UI tests on each PR. Let people discuss good and bad code they see.

Yeah this is the way to do it. Automate everything. Unfortunately there's no real "good code" vs "bad code" easy metric, but you should at least have automated testing, linting, compiling, etc to ensure people aren't merging total garbage to shared branches.

Sounds like a good idea, but no one wants to do this. People would rather quit.

In the playoffs you don't tell your star player to stop shooting and pass more. In my experience, software projects are always lead to accomplish a goal that is out of reach unless the stars bust their ass.

If your company isn't budgeting/estimating development effort appropriately, then it's going to affect a lot more things then just code reviews: writing good planning documents, adding tests, Q&A, fixing bugs, design polish, small touches, etc. are all going to suffer if the organization is broken in this way.

I’ve seen estimates that it takes 1h to properly review 400 LOC and the reviewer then needs a long break after that time regardless of whether they’re done reviewing or not.

Anything less than that is basically just checking code style or egregious errors.

Whenever there's continuous refactoring, the factors aren't so much known and decided at a high level, it's typically more an exercise of deduplication. If you can't name the thing (or why) you're factoring it's blind deduplication, which still has some value but is much lower. If someone says they want to refactor, I'll ask what's the factor and why. A system separated along well chosen seams allows things to naturally and easily fit in its design.

For instance monitoring has a price to set it, configure it... You can avoid part of the price by setting up a simple monitoring It has a cost as you're not going to catch early on issues in prod. And the latter you realize something is no going well, the higher the cost will be to fix it, loss clients...

And we can go on with many things as well, UX has a price, testing has a price, backups have a price...

Hopefully the writer don't think he has found the final reason to justify why refactoring is a must as there are many other best practices were low price will imply higher cost at the end. Wasn't that what we used to call tech debt? XD

ideally you're investing into creating shareable code libraries and into infrastructure that makes the process of maintaining/referencing shared components easy enough for engineers to want to dedicate their precious sprint time to it.

My fizzbuzz:

https://bitbucket.org/iopq/fizzbuzz-in-rust/src/master/src/l...

there's just no more refactors I can do because of the limitations of Rust

The condition in reality actually lives on a gradient and it very much influences everyone to varying degrees. It's only labeled ocd once it crosses a threshold. It no doubt exists in programming. Much of refactoring is done for the same mechanisms that cause ocd and zero practical reasoning.

I have ocd but I'm able to differentiate my compulsions from rational reasoning. Thus I can see how irrational my actions are.

Do what you can, as if you were going to be the next person. For one reason, because anything else is irresponsible laziness. For another, you may in fact be the next person - you may not leave in time for someone else to inherit the mess. So do what you can now to minimize it.

- I don't even know you

- 'Irresponsible laziness' could also be replaced with 'ducks in space'

- I can leave any time. This is not slavery.

So its a method of work - maximize the input, minimize the loss.

You would be surprised. I actually know people like that, more then 1.

Refactoring has both

Price is pretty unambiguous, cost is very ambiguous. I've never seen anybody make this distinction, but IMO, it's clear enough.

(1) Before you work on a feature, if the existing code has some tradeoffs or outright deficiencies which make the new feature hard to add, start by fixing the work site, and then add the feature. The benefits from this snowball quickly on code of any age or complexity, and since the assumption is that we're only doing "extra" work when a feature was going to be complicated and that the "extra" work makes the feature simple, it doesn't actually add a meaningful amount of time to your tickets (i.e., throughput slows down a bit at the beginning but not enough for management to notice or care, and it ramps up over time).

(2) A fair number of developers seem to "think" in code -- incrementally building out the solution in code to learn how to solve it and build appropriate mental models. If this describes you, throw away the first draft and start over. V2 takes a fraction of the time to write as V1 since you now know what you're doing, and the second draft will be substantially better. If this gets pushback, never describe V1 as "working" or a "prototype" or anything vaguely implying that enough duct tape and unpaid overtime would allow it to hobble along in production. I find that "brainstorming" often elicits the correct response.

(3) The article calls out that after building a feature "correctly" it might not fit perfectly into the existing code. I'm not necessarily convinced that the right time to refactor this is immediately (since it's so hard to predict which abstractions you might need (ignoring rules of thumb like "separating payments and entitlements", which represent hard-won tribal knowledge)), but if you're able to come up with a measurably better solution then the best time to implement it is definitely while everything is fresh in your mind and before you bug your coworkers for a review.

(4) The article _also_ calls out that cruft has probably accumulated over time, which might need to be handled incrementally or with a rewrite. A lot of that is often addressable via (1), but partial rewrites _can_ be cheaper than you think. If you happen to notice a particularly thorny problem afflicting an organization, can confine it to under a few thousand lines of code, and have a decent idea for how you will make it better, spending a couple days to rip it out is time well spent. I've worked on a lot bigger projects before than those sorts of quick rewrites, but those small changes have had a comparatively outsized ROI. They're fast projects that nobody else had any good ideas for fixing (or they would have done so rather than bitch about how much of a hassle the thing is) that can immediately save 1+ developer days per calendar day or 10 GPU years per calendar day or whatever. Your work pays for itself quickly, and since it only took a small fraction of your time you still have nearly identical throughput on whatever your main job is.

(4a) No longer job-related, but a similar idea manifests itself in ordinary household functions as well. If you have a low-risk, low-cost method for achieving 10%+ annual returns, that's often worth considering. Consider buying flour or soap or something in greater quantities than normal. It's not uncommon to pay 2x the price for 5x the quantity. If you go through the item every 2.4 months (illustrative, to make the rest of the math easy), consider the relative effects on your wallet from buying in bulk. You spent 2p (for some unit price p) immediately (and couldn't invest it or do anything else; it's gone), but at the end of the year you have 3p more cash than you would have had you bought smaller units. I.e., your 2p investment grew 50% to 3p. That's an amazing annual return that you couldn't easily replicate elsewhere, even after accounting for tax-deferred benefits or other such effects. Similarly, paying some portion off a 20% interest loan (try to never get those...) has, ignoring taxes and bankruptcy and other such details, exactly the same effect on your net worth as getting 20% returns in some investment account. Moreover, it's nearly "risk-free", whereas such a high yield investment account would not be.

(4b) Individual circumstances vary. Try to interpret (4a) in a charitable light; if you don't bake much, don't buy 10yrs worth of flour. If you have to buy extra square footage to plan to buy in bulk that's maybe a problem. If you require a ton of time to avoid scams or quasi-scams (like stores pricing bulk items with a higher per-unit cost than non-bulk items) then that's potentially a cost that keeps it from being worth it. The point though is that on the scale of a single person going through a single life, there are lots of small things you can do which have outsized returns (education and learning fall into that as well IMO, but this comment is long enough), far beyond the strategies which work at the scale of a retirement fund.

Other industries went the way of producing good quality products whereas IT seems to use the approach to test the quality into the products and to fix, quite often when product is already at customer. This normally turns out be more expensive and have less quality in the products at the same time.

Hypothesis: software market has near zero barriers to entry, so while other industries are protected from would-be competitors by large up-front expenses, for most purely-software products anyone can rapidly build a half-assed clone of whatever you're doing and start eating into your customer base - making software companies obsess over velocity / feature delivery rate, and/or seeking all kinds of non-software barriers to entry (e.g. network effects, or content deals - like, it wouldn't be hard to make a better Spotify client, but good luck replicating the deals Spotify has with recording labels).

Hypothesis 2: this is why you have a lot of anti-capitalistic talk among tech folk.

The whole concept of "anyone can replicate my product, but I got here first, so i have to use lock-ins and other ethically questionable things if I want to keep my spot" is antithetical to the goals of capitalism i.e companies out-competing each other constantly, leading to better and better products.

The equilibrium with monopolies that is possible under capitalism is reached really, really easily in tech. Because being a monopoly/locking-in, not open sourcing, etc, is the only way you can be a big tech company.

In my opinion, industries with low barriers of entry are fundamentally incompatible with - atleast the current flavor of - capitalism, if you use user harm as the metric.

Agreed. Tech is exemplary here, and so is all media since it went digital: when natural barriers to entry are low, the first thing companies do is to establish artificial ones, whether by abusing the legal system, or by piling up so much marketing bullshit it creates new barriers ex nihilo, right there in the minds of consumers.