Centralize the Side Effects
The difficulty of reasoning about the correctness of event-driven systems for me, and of eventually running into that "series of unfortunate events", is the level of decentralization of code resulting from it combined with side effects. Ideally the code executed when an event triggered has hardly any risk of ever doing something wrong, or at the wrong time.
When you try to visualize the control flow of an event-driven system, the code doesn't even tell you what the paths of execution are, since what functions are called as a result of an event being triggered are often largely unknown (and subject to change at runtime), unless you trace through it. The mental image might be something like this:
... but worse (simplified diagram), and this would only be a snapshot in time since what functions are called by an event will change as the system changes state. The difficulty here is the unpredictable control flow combined with the fact that each event is doing something "complex".
Is the event even being triggered from a particular thread? In some systems, that too might not even be clear. When/where/how becomes a total mystery in an event-driven system.
"Complex" to me practically always implies some form of external state changes. An event might occur which then resizes a widget which then triggers a cascade of subsequent events on the descendants of that widget. An event might occur which removes an item from a scene graph with another that reorders the hierarchy... Things of this sort.
Combining something that complex, resulting from state changes, with the unpredictable nature of the control flow of event-handling is a recipe for systems whose correctness are really, really difficult to reason about, even when all of our tests pass. I'm talking about the ease of comprehending our systems, and how they will respond to changes in advance. Tests are a critical part of ensuring reliability which is a critical step to not being at the mercy of our systems, but I'm talking about staying in the driver's seat and not becoming a passenger, to stay in total control of our systems, to shape it to our will.
The problem is that each event triggered forms like a little "branch" of functionality. If you have a boatload of little branches of functionality that each do something potentially complicated inside such as changing external, fragile state (and I'm not talking about global variables but simply calling methods which modify a widely-accessible object central to the software, e.g.), then we're in for potentially a lot of pain.
The solution to me: centralize and limit the number of places in your codebase where these complex state changes can occur.
One strategy to centralize the complexity is to simply have all these disparate events push to a central event queue, like so:
What happens in this case is that all this maze of winding and twisting event-triggered functionality is no longer causing any external side effects besides pushing to some central queue to be processed later. This is very simple, symmetrical behavior for each event and never bound to fail no matter how you shuffle up the order in which events occur, since each one is just immediately pushing to a central queue.
This "Central Event Processor" then pops from the queue and does the actual critical and error-prone work of updating this kind of central, external state and causing side effects.
While this might just seem like it's transferring complexity to a central place, that's kind of the whole point. It's hard to reason about software if complex things are happening in all kinds of event-triggered functions which are called "who knows when?", "from who knows where?", and "who knows how?".
This central event processor is easy to understand in terms of when it is executed, and how. It also becomes easier to thread both safely and efficiently, to avoid doing redundant updates of data for two or more events which could be consolidated to yield a single update of data in response, etc. etc. It just becomes easier in general for humans to reason about exactly what is happening, when, and how.
It also becomes easier to reason about the correctness of this central state when there's only one entire place in the system that can modify it (the central event processor).
As much as it's generally true that many simpler things are easier to comprehend that one complex thing, if the decision is between many disparate places in your codebase that can cause complex side effects vs. one central place that can do that, the central place generally wins if you want an easier software to maintain and reason about. You want to limit the number of places that can mutate external state to an absolute minimum. We might not be able to use pure functional programming for everything, but we can definitely limit the number of places in the system that can trigger complex side effects, and centralize them and make it easy to understand exactly when/where/how those side effects are caused.
There are other strategies to centralize the processing that become even simpler than an event queue if the order of processing events doesn't have to be in the incoming order, and your events are being generated from entities (in which case your design starts to take on an entity-component system with systems processing local state encapsulated by components).
In any case, the broad strategy, when applicable, is minimize the places in your codebase where complex side effects can occur. Event-driven systems without special care, and in very large-scale and complex codebases, can really exacerbate that and maximize the number of places where side effects can occur while simultaneously making it difficult to predict when/where/how those side effects will occur. The easiest way to fight against that is to then defer the processing once more to a centralized place.