Persistent refactoring is one of those things where the cost is clear and well known (expensive developer time) while the payoff is big, but very diffuse, very hard to explain and often gets attributed to something else - sometimes even by the developers who did it.
It's not the only thing like that. Plenty of companies skimp on cost centers and fail spectacularly as a result - sometimes without even ever realizing that skimping on the relevant cost centres was the reason why they failed.
Refactoring only matters if decision makers care about long term consequences and value. One thing that makes it difficult to make these kinds of decisions wisely is that no one knows how long a piece of software will live in production.
If I could know for sure that a piece of software would exist for 50 years in production I could make a better case for refactoring. Likewise, if I knew for sure that a piece of software would get completely replaced in 1 year I would not bother with refactoring.
Never in my career have I been able to accurately predict how long software would live in production. I have had throwaway code live for years and things I thought would last forever get replaced almost instantly.
Refactoring doesn't just benefit you in 50 years it can benefit within weeks.
I agree that it's a waste on throwaway code, throwaway code is common and what ends up being throwaway code is hard to predict, but I don't think it's impossible to get a sense of the likelihood of the code being tossed, and that sense can be used to tweak your refactoring budget.
But if I spend a month refactoring and only get a weeks worth a benefit before the project is scrapped it wasn't worth it. If I get a month back before the project is scrapped it wasn't worth it (I could have been working on something else instead). Unless/until we know how to quantify how much gain will be got back, and how long it will take to get that gain back and so we cannot do a proper economic analysis if it is worth it - but that is what we need.
> Refactoring doesn't just benefit you in 50 years it can benefit within weeks.
Proving the value is the hard problem. It's almost impossible to prove the value of small changes today. Product wants things to do X and they decided the value tradeoff was good, not the engineers who are tasked with making it so.
Thats why I just do it. If somebody wants proof that my years of experience applied in a particular way will provide a quantifiable benefit that is impossible I will just avoid doing that thing.
Or more likely, move somewhere which can appreciate those non quantifiable benefits.
> Or more likely, move somewhere which can appreciate those non quantifiable benefits.
How can you find such a place? This definitely resonates with me.
There's so much value in doing things right, but it's impossible to prove. I can do things that make my team go faster, but there's no way to say that having a X% better tests resulted in us being able to release Y% more features.
Try to give off trustworthy vibes and pick customers/clients/employers who will trust you.
I'm not going to pretend it always worked for me, but it certainly got easier once I could demonstrate a wealth of experience.
There's a lot of luck, too.
In terms of refactoring, I have an iron rule that I never ask for permission. If I think it needs doing I do it. I don't justify it to product or ask for their opinion. Technical issues and priorities are on a need to know basis and they don't need to know. All the work gets baked into feature work and bugfixes.
Persistent refactoring is one of those things where the cost is clear and well known (expensive developer time) while the payoff is big, but very diffuse, very hard to explain and often gets attributed to something else - sometimes even by the developers who did it.
It's not the only thing like that. Plenty of companies skimp on cost centers and fail spectacularly as a result - sometimes without even ever realizing that skimping on the relevant cost centres was the reason why they failed.