Refactoring, like any other activity, must have a clear goal defined for it. Once that goal is clear, you would consider the current project status and life cycle stage. For a development project that is 80% complete, 30% behind schedule you should justify refactoring effort based on the goal set earlier. In this example, if the code pieces were unit tested and is working fine in a development environment, it is hard to justify refactoring.
The fact that 40 developers left may not be as dramatic as it sounds. I'd expect that those developers have delivered working code that was reviewed and tested. So, unless there are known problems in this code, I'd leave it as is. The idea is that in a large project like yours, I'd expect that there were standards and procedures and that the code is not a complete mess.
Remember that refactoring will cause many if not all tests that has been made to be repeated. Also, since the refactoring of this size can't be done by a one or two senior members, the refactoring may introduce problems that did not exist. This is a risk that should not be neglected.
Having said that, it is not unusual to add tasks to a project when the unforeseen happens. So, if the developers disappeared for some reason, that that would be considered an event of a special nature and whatever actions to remedy the situation must be taken. It would be treated like a fire or an earthquake, etc.
In summary, I would not refactor large working code in a large project for no good solid technical reason, specially that we all know that most project are usually in late status.