I've been at a company where we had two leading developers.
One refactored everything endlessly to a fault, wrote tests for everything and emulators for every external piece of hardware.
The other rarely/never refactored and was focused on ends-justify-means functionality.
They hated each other and progress was very slow because they could not work together at all.
My two cents is that all products of engineering are temporary. Software (and hardware) are not eternal. What you write today you will eventually become obsolete.
Your entire app will eventually be re-written. If not by you then by your competitor who takes you out.
There is no such thing as perfect or optimal code because what is great for now won't be great for tomorrow.
With that in mind, good refactoring is like good math. Taking a long and complex system of code and replacing it with something functionally better and far simpler and easy to understand.
Bad refactoring is just moving things around for "understandability" or in a lot of cases one developer rewriting something in their own idiosyncratic style because they don't understand the style of the programmer before them.
And the final point is don't be afraid to rewrite the entire system if you have that ability.
Since the development process is about defining functionality as much as enacting it, once you have a finished application, you can re-create it much faster and simpler than it was originally written.
The Apollo missions got to the moon and back with 145000 lines of code you don't need 20 million + for your web app.
> They hated each other and progress was very slow because they could not work together at all.
Been there. And often gets worse when the "ends-justify-means" camp
(1) collects velocity points (that's the most important thing, they decided to gain the system that way) and people who care about velocity (typically management) are happy but...
(2) leave a path of destruction in their wake that slows everyone else down, decreasing the productivity of everyone
With refactoring, consider 2 things
(1) How bad / ripe for simplification is the code
(2) How much is it changing
If its bad stuff but nobody needs to deal with it, I say leave it unless it has dependencies (those sneak up on you too and prevent perhaps a library upgrade that would benefit the rest of the app)
Rewriting your entire system is almost always the wrong thing to do and has been the end of plenty of products and companies. I think Joel is spot on here. It's much more difficult to read code than to write code, so new developers always think existing code bases are junk and need to be scrapped while ignoring the value that platform has delivered over years if not decades. Developers aren't paid to be craftsmen. They are paid to deliver a product that provides business value.
Too often developers lose sight of the business value and want to write code for the sake of the craft. You're working for Ikea and complain that you can't use your fancy woodworking tools and techniques and have to stick to dowel based assembly instead of that fancy joinery you'd rather be working with. Those fancy joints are stronger and more stable after all! And what the fuck is up with all the particle board?! All real woodworkers know the stuff is junk and will fall apart as soon as it gets wet. We should be working with mahogany and walnut with cocobolo inlays!
There are places where your fancy joinery skills are appreciated and none of those expenses are spared. But most developers are working at an Ikea or equivalent where the predictability of cost and process is well understood and a reliable product can be delivered at a reasonable price. If you want to be a craftsman, you need to work at the places who are looking for those skills and not just the places who want flat pack furniture quickly and cheaply to keep delivering value to their customers.
I agree with you on the value proposition consideration.
"If you have that ability"
I think I specifically called out the danger of the average developer rewriting code because they don't understand it.
But If you DO understand the code base and you do have the time/resources/vision you can rewrite as a form of radical refactoring, taking the lessons learned over the original development and compressing them into a newer system that allows some kind of business expansion/pivot/innovation.
Yes all change requires risk but there is risk in not changing as well.
Companies go under just as often from NOT rewriting or NOT building something new/better.
If you don't truly understand the whole system and you don't have a business case for what your new system would do then DON'T rewrite.
On the flip side, if you DO understand a 20million+ line codebase of your company's web app and you know you could rewrite it all yourself/with a small team and make it 10x better then definitely don't waste time arguing with your boss about whether refactoring is a good idea.
Quit and start your own company because you've identified a massive inefficiency in the market to exploit.
You are forgetting to measure. You are saying something to the effect of "if I fry potatoes for too long, I get coal, therefore it's best to eat potatoes raw".
Sometimes the program you are writing is so worthless, it'll be thrown out tomorrow. Sometimes it may outlive you. The later is rare, but not impossible. Eg. my former boss has some of its code flying on an satellite, circling the Earth some 30 odd years.
My personal oldest code that is still alive (that I'm aware of) is some minor stuff in cl-gd library (Common Lisp bindings to GD library). It's about 15 years old now.
For my day job, I work for a company whose codebase was started some time in late 90's. While things have changed over time, some, and especially design decisions are still there. A lot of them are painfully wrong, and have been a burden on this company for the last 20+ years.
Oh, and should I mention the 32-bit sized IPv4 addresses?
----
How much should one invest into design? how often should cornerstone ideas be revisited? -- Definitely not nothing and definitely not never. But how to know when it's the right amount? -- I don't believe anyone has a strategy for finding out. Good (or lucky) project managers will have a good guess about the timing.
There two kinds of phases a product lives under: growing and maintenance.
Maintenance mode products aren’t growing revenue. They might get a few new features but the main goal is to not mess with the cash cow unless you need to patch or make a special customer a new feature. It will fade into irrelevance with time as something new replaces it.
Projects that are growing need to scale and adapt to new environments and use cases. If you treat these with disrespect then your business will suffer and you’re less likely to make that new feature or big pivot needed to become a cash cow.
One refactored everything endlessly to a fault, wrote tests for everything and emulators for every external piece of hardware.
The other rarely/never refactored and was focused on ends-justify-means functionality.
They hated each other and progress was very slow because they could not work together at all.
My two cents is that all products of engineering are temporary. Software (and hardware) are not eternal. What you write today you will eventually become obsolete.
Your entire app will eventually be re-written. If not by you then by your competitor who takes you out.
There is no such thing as perfect or optimal code because what is great for now won't be great for tomorrow.
With that in mind, good refactoring is like good math. Taking a long and complex system of code and replacing it with something functionally better and far simpler and easy to understand.
Bad refactoring is just moving things around for "understandability" or in a lot of cases one developer rewriting something in their own idiosyncratic style because they don't understand the style of the programmer before them.
And the final point is don't be afraid to rewrite the entire system if you have that ability.
Since the development process is about defining functionality as much as enacting it, once you have a finished application, you can re-create it much faster and simpler than it was originally written.
The Apollo missions got to the moon and back with 145000 lines of code you don't need 20 million + for your web app.