The problem with doing a major refactoring is that you can and will sometimes follow a path that leads you to realize that you've bitten off more than you can chew. Giant refactorings are a mistake. If the system design is flawed in the first place, then refactoring may only take you so far before you need to make a hard decision. Either leave the system as it is and work around it, or plan to redesign and make some major changes.
There is however another way. The real benefit of refactoring code is to make things simpler, easier to read, and even easier to maintain. Where you approach a problem that you have uncertainty about, you spike a change, go so far to see where it might lead in order to learn more about the problem, then throw away the spike, and apply a new refactoring based on what the spike taught you. The thing is, you can really only improve your code with certainty if the steps are small and your refactoring efforts don't overrun your ability to write your tests first. The temptation is to write a test, then code, then code some more because a solution may seem obvious, but soon you realise that your change will change many more tests, so you need to be careful to only change one thing at a time.
The answer therefore is to never make your refactoring a major one. Baby steps. Start by extracting methods, then look to removing duplication. Then move to extracting classes. Each in tiny steps one minor change at a time. IF you're extracting code, write a test first. If you are removing code, then remove it and run your tests, and decide if any of the broken tests will be needed any more. One tiny baby step at a time. It seems like it will take longer, but will actually shorten your refactoring time considerably.
The reality is however, that every spike is seemingly a potential waste of effort. Code changes sometimes go nowhere, and you find yourself restoring your code from your vcs. This is just a reality of what we do from day to day. Every spike that fails is not wasted however, if it teaches you something. Every refactoring effort that fails will teach you that you are either trying to do too much too quickly, or that your approach may be wrong. That too is not a waste of time if you learn something from it. The more you do this stuff, the more you learn and the more efficient you will become at it. My advice is to just wear it for now, learn to do more by doing less, and accept that this is just the way things probably need to be until you get better at identifying how far to take a spike before it leads you nowhere.