I'm writing from a Java perspective (to save me having to put 'from a Java perspective' everywhere).
Carefully designing an API is about more than just "organisation". Many of the public classes I write are used by a lot of clients and by using the lowest access modifier possible I am free to change the inner workings of my classes without breaking their code (and thereby getting nasty phone calls and emails!).
tl;dr: Encapsulation/information hiding, you can change your representation and inner workings without breaking client code, public mutable fields are not thread-safe, promote immutability.
A module that is well-designed will hide its internal data and implementation details from other modules and cleanly separates its API from its implementation. This means that communication between modules is done only via their APIs and they are oblivious to the inner workings of each other.
The importance of this is that it decouples modules from each other allowing them to be developed, tested, used, optimised and modified in isolation and they can be developed in parallel. It also increases reuse because modules that aren't tightly coupled can potentially be used in other contexts.
In Joshua Bloch's book "Effective Java" he states that the rule of thumb is:
make each class or member as inaccessible as possible. In other words, use the lowest possible access level consistent with the proper functioning of the software that you are writing.
After designing your public API your instinct should be to make everything else private, and only if another class in the same package really needs to access a member should you make it package-private (remove the modifier). The need for protected members in a public class should be relatively rare as a protected member becomes part of the class's exported API.
When you have everything as public it means that the data fields of the class are accessed directly and you cannot change their representation without changing the API. You also generally cannot change the inner workings of your class as you have the potential to break the code of whoever is using your class. This issue/risk is multiplied by the number of your clients as your class may be used all over the place.
If a field is non-final or is a final reference to a mutable object (e.g. Date) and is
public then you give up the ability to limit what is stored in the field or enforce invariants on the field. So if you wanted to limit an
int to a certain value range, having the field
public means that you no longer have control over what is stored in the field. You also lose the ability to take any action when a field is modified, which means that classes with public mutable fields are not thread-safe.
Public final fields with a mutable object in them have all of the disadvantages of a non-final field too. Whilst you cannot change the object reference, the object can still be modified which may break the inner workings of your class and have unintended and disastrous results.
By making fields private you also minimise mutability of your class. Immutable classes are generally easier to design, implement and use than mutable classes are. Private fields prevent clients of your class from obtaining access to mutable objects and modifying the objects directly. You generally also never give a client references to the mutable objects either (you give them copies).
As your class minimises mutability or becomes completely immutable it also becomes inherently thread-safe and requires no synchronisation. They cannot become corrupt by multiple threads accessing them concurrently. A huge functional gain of immutability is it is one of the easiest ways of achieving thread safety.