Your best option would be to structure / package your library such that users of your library would not be bound by the GPL. The FSF FAQ describes a few means of how this can be achieved.
This is the "viral" aspect of GPL, and yes, you would potentially be exposing your future users to legal issues. More importantly, most users wouldn't / shouldn't use your package for risk of violating the GPL.
the copyright holders of the software are the ones who have the power to enforce the GPL
You stated that you are working on a forked project that is GPL'd, so you are not the sole copyright owner of the product. So while you may rescind your right to enforce the copyright, you cannot make that claim for all of the other copyright holders unless they have also made that claim. At which point, the project is public domain and not GPL but that's off topic.
If your users only use your library for internal, never-distributed-to-the-public applications, then they would be okay with using your libraries. I suspect that's not your target audience though.
As I alluded to, there are several options that you can consider depending upon which version of the GPL is in effect. For example, if your users can call your library / use its output after a fork or an exec, then your library is not considered part of their application. You may be able to relicense your fork of the project to a more compatible GPL version. Check out the following links to see what's going to work for you.
Their FAQ is well worth the read and you will discover quite a few options to work with.