Requiring the files to be signed could help somewhat.
Here is a wikipedia article about JAR signing.
Here's the section from the wikipedia article that is significant...
Developers can digitally sign JAR files. In that case, the signature
information becomes part of the embedded manifest file. The JAR itself
is not signed, but instead every file inside the archive is listed
along with its checksum; it is these checksums that are signed.
Multiple entities may sign the JAR file, changing the JAR file itself
with each signing, although the signed files themselves remain valid.
When the Java runtime loads signed JAR files, it can validate the
signatures and refuse to load classes that do not match the signature.
It can also support 'sealed' packages, in which the Classloader will
only permit Java classes to be loaded into the same package if they
are all signed by the same entities. This prevents malicious code from
being inserted into an existing package, and so gaining access to
package-scoped classes and data.
Developers can obfuscate JAR files so that a user of the JAR file
doesn't get much information regarding the code it contains, or to
reduce its size, which is useful in mobile phone application
I normally wouldn't just link and quote Wikipedia. No-one had mentioned this solution yet, but it's not in my area of expertise. If someone with a bit more Java experience would give a detailed answer, drop me a comment and I'll remove this answer.