As with most cases, it depends (relation, project, etc.).
If the customer wants the slow performance to be fixed, you can send a bill in my opinion. It's your fault the performance is this slow, true, but every developer makes mistakes and customers most of the time pay for it.
They could have hired a big company with lots of (experienced?) developers who probably wouldn't have made this mistake, but the initial costs would have been much higher. Those devs would have made other mistakes and the customer would also have to suffer for that.
You've delivered a piece of software and they have probably agreed on it.
Now they want it changed, well, that's a request for change, which will cost money. If you initially hadn't used these (bloated) frameworks, the job would have taken a lot more time/money.
If it's a fixed price project, it'll be a fixed in a new (minor) version. Depending on the contract, people can pay for upgrades.
In a different scenario/example, you've chosen a certain IoC/DI framework in your project. After a few months you discover it's performing very slow compared to a different framework. What would you do? I'd let the customer know that I can speed up the system by changing some internals of the system which will cost X dollar. They can decide if they want it or not.
In short: I'd say they have to pay for upgrades on the system, but it really depends on the situation.