I'll direct my answer more to what comes after an exception: what's it good for and how should software behave, what should your users do with the exception? A great technique I came across early in my career was to always report problems and errors in 3 parts: context, problem & solution. Using this dicipline changes error handling enormously and makes the software vastly better for the operators to use.
Here's a few examples.
Context: Saving connection pooling configuration changes to disk.
Problem: Write permission denied on file '/xxx/yyy'.
Solution: Grant write permission to the file.
In this case, the operator knows exactly what to do and to which file must be affected. They also know that the connection pooling changes didn't take and should be repeated.
Context: Sending email to 'email@example.com' regarding 'Blah'.
Problem: SMTP connection refused by server 'mail.xyz.com'.
Solution: Contact the mail server administrator to report a service problem. The email will be sent later. You may want to tell 'firstname.lastname@example.org' about this problem.
I write server side systems and my operators are generally tech savvy first line support. I would write the messages differently for desktop software that have a different audience but include the same information.
Several wonderful things happen if one uses this technique. The software developer is often best placed to know how to solve the problems in their own code so encoding solutions in this way as you write the code is of massive benefit to end users who are at a disadvantage finding solutions since they are often missing information about what exactly the software was doing. Anyone who has ever read an Oracle error message will know what I mean.
The second wonderful thing that comes to mind is when you find yourself trying to describe a solution in your exception and you're writing "Check X and if A then B else C". This is a very clear and obvious sign that your exception is being checked in the wrong place. You the programmer have the capacity to compare things in code so "if" statements should be run in code, why involve the user in something that can be automated? Chances are it's from deeper in the code and someone has done the lazy thing and thrown IOException from any number of methods and caught potential errors from all of them in a block of calling code that cannot adequately describe what went wrong, what the specific context is and how to fix it. This encourages you to write finer grain errors, catch and handle them in the right place in your code so that you can articulate properly the steps the operator should take.
At one company we had top notch operators who got to know the software really well and kept their own "run book" that augmented our error reporting and suggested solutions. To recognise this the software started including wiki links to the run book in exceptions so that a basic explanation was available as well as links to more advanced discussion and observations by the operators over time.
If you've had the dicipline to try this technique, it becomes much more obvious what you should name your exceptions in code when creating your own. NonRecoverableConfigurationReadFailedException becomes a bit of shorthand for what you're about to describe more fully to the operator. I like being verbose and I think that will be easier for the next developer who touches my code to interpret.