how to manage such a complicated setup?
You need two things which are extremely important:
1. Good developers
We are talking reverse engineering here.
There are lots of programmers who can follow a tutorial or copy-paste code they barely understand, or follow the company's training program and roll out working code, but for this kind of situation you need people to do the opposite. Instead of building something, you have to tare something apart and see how it works so that you can later build on it, maintain it, evolve it etc.
Skills are necessary for this, so you need experienced developers. Juniors won't do! Even if they are good developers and hard working people they "haven't yet seen the horrors of war".... if you understand what I mean.
2. Support from management
This is even more important than number one. If you have lots of frameworks, it's a huge project, it's not documented, maybe the business domain is new to you etc, then to get up to speed and be able to enhance the system you need time. You can't just roll up your sleeves and start coding the new features. You must learn, dissect, analyze change a little, test a little etc.
Does management realize this? Are they perfectly aware of the risks that such uncertainty generates? Are they supporting you (they must be the ones to buy you this time, projects don't have the luxury to go on forever) or are they putting pressure because "everything takes to long"?
It's never easy to manage a situation like this, but people are the ones to put your bets on, not tools or UML diagrams.
Best of luck to you!