It's not a phenomenon specific to refactoring. Humans just arent good at appreciating abstract gains or abstract risks.

The proliferation of technically clueless managers of tech projects and their "control systems" such as scrum has exacerbated the problem.

I met some doctors recently that made some rhyming complaints when non-medical mba management took over a practice.