As you're being very high level and not specifying what technologies you're using I can't point you to a definitive online source, however I wil give you some suggestions for your design.
You will need a service bus of sorts. The purpose of the service bus is to allow your application and other plugins to communicate with each other.
Work out whether you're going to support certain types of extensions (i.e. document helpers, toolbar addons, etc) and if so whether it is appropriate to define interfaces for individual extensions.
Consider writing as much of your core functionality as possible as extensions / plugins. This will help you to iron out any kinks in your architecture.
You will then need to work out how you are going to load them, unload them and manage them.
Frameworks such as the Managed Extension Framework (MEF) in .NET, or OSGi or JPF for Java will help you with common tasks.
For the maximum level of extensibility you will need well defined interfaces that are accessible to external libraries / assemblies / packages for implementation.