Here is what that developer is talking about. See here and here for a list of vulnerabilities in the run-time engine itself. If you scroll to the bottom of this page and on to the next page you'll find more vulnerabilities that might be relevant, depending on what is in your software stack.
Well first you have to understand that Java is both a server and a client. As a client, it tries to run in a carefully sandboxed way where mutually untrusted code can run in the same space. This is a very hard problem, and it is used in enough "highly interesting targets" that a lot of effort has gone into breaking it. The result is that most browsers will give you big security alerts before enabling Java. It is simply too easy for an attacker to send you something which, when run by the JRE, will do Officially Bad Stuff that the security model said it shouldn't do.
People generally do not run untrusted code on a server, so most of that does not apply to you.
Of course there are still vulnerabilities that do apply. But here we have a more complex question. Is it better to have vulnerabilities that apply to you, but which you find out about? Or to not know about vulnerabilities that apply to you?
Java is a high profile target, used in high profile applications. So a lot of effort goes into finding bugs. There are a lot of public bugs. That go into automated attack tools. That any kid can use, and any decent admin can defend against. But a penetration tester will have a harder time finding a new one that nobody else has already found.
If you use a less known software stack, there are likely to be similar numbers of bugs, but less effort has been put into finding them. That means that a random kid with an attack tool that knows a list of bugs is less likely to find something. But any penetration tester examining your software stack will have a field day finding attacks that you have no way of defending against because you don't know about them.
If you're writing a high profile application (banking software, etc), you have to assume that competent attackers are a fact of life. Therefore something like Java makes sense there. If you're a small startup, Java might not make as much sense, but honestly for a startup, how vulnerable your language is is one of your least important problems. Getting to be big enough that someone would want to attack you would be a Good Problem to have.
And, truth be told, anyone who wants to break into your site will first look around for the obvious SQL injections and the like. Most of which will be your fault, and not the fault of your vendor. Really, your language is not going to be your top source of security problems. Your programming is. Nor is it your second top source of security problems. Your misconfiguration of your systems are.
In short, there are a lot of reasons to use or not use Java. (I personally avoid the language.) But security isn't a good reason to stay away, despite the large number of known security problems.