When you link your application to a library, you are making a derivative work of that library. Therefore, under the terms of the GPL (under which the library is licensed), you must distribute the application source code with the application (if you distribute it). And it is a derivative work whether you just link to it or if you alter the source code as well.
Where this gets fuzzy is with dynamically linked libraries. If your software can dynamically link a GPL'ed library (e.g. .dll or .so), but could also run without that library, then you could distribute your software without the library, but then tell the distributee how to run the software with the library. Because the software can run without the library, you can distribute working software without the GPL'ed library and without the application source code, and still not be violating the GPL. But it is still fuzzy.
One trick I saw a big company use a few years ago was to put a shim between the proprietary application and the GPLed library. The shim was a functionally equivalent wrapper around the GPLed library, and it was itself implemented as a dynamic library (.so). The shim was licensed with LGPLv2, so if the application linked to the shim, the application source code didn't have to be distributed. The source code of the shim "worked" without the GPLed library too, so it could be distributed under any license, so they were able to choose LGPLv2 instead of GPL. Still fuzzy.
Aparently, dynamic loading is how the one can get proprietary drivers to work on Linux.
The key in all of this is that if the entity who received the working application is able to reassemble a working application to use a GPLed library he acquired himself, then the copyright owner of the working application does not have to distribute the source code because the derivative work was made after the distribution of the application, and not before.
Still fuzzy... but I have given you more ways to look at the problem.