I've had the opposite happen every time the team I've been on decided to refactor a large portion (or even the entire code base). Every time, what was a source of constant bugs (i.e., X bugs per week, every week, never lessening), became tractable and moved to stable post the rewrite (X bugs first week, .7x bugs second week, etc, until finally we're encountering the odd bug only once every few months, if at all).

I'm not sure what the differentiator is. I'd be curious if others have ideas. I think part of it is that in both cases it was a small team, who caught the issues early enough that it hadn't gotten too bad yet, but late enough that the right direction to move in was clear.

I was talking about enterprise projects that had had years of development, always changing personnel and had to follow complex and changing business rules. These tend to be ugly and difficult to work with after a few years of development. In my view the only way to deal with these is to break them down into smaller components and then refactor. But that turns then into a political issue because the managers (and a lot of developers) don't see the need.

Yeah, I could see that. I've been in those environments too; I have no data points from that, because getting the okay to refactor was so hard it never happened while I was on the team (I'd been on projects that claimed to have refactored the code, but then it was mixed as to whether people claimed it was a success or a waste).

I always tell the younger guys not to try to get an explicit OK to refactor but just add 20% to all estimates and use that for continuous refactoring without asking. It's just a regular part of professional work like writing code, pull requests and testing. This also has the advantage that refactors are relatively small so you can rollback if it turns out that the idea for the refactor was wrong (yes, this happens:-) ).

Even single man-month long "refactors" are something I'm wary of. Smaller changes are easier to test, easier to review, easier to merge, easier to verify are actually improving the state of things and heading in the right direction, easier to pause when your priorities unexpectedly shift midway through cleanup without leaving a terrible mess...

I'm okay with the occasional week-long rewrite of a subsystem, but usually only after I've spent some time coming to grips with exactly why the old one is terrible and have a firm grip of exactly how the new one will be better.