Likely the most practical languages for you to learn would be Ruby and Python. These languages will be fulfilling in all three points you mentioned above. (They are in high demand, they will challenge you by adding more conceptual tools to your toolbelt, and they will be smooth transition. (Ruby probably wins #1 - though depends on the job, they tie on #2, and Python may have a slight edge at #3).
Ruby is a bit more popular than Python in terms of web application development, but Python will remain very useful as a utility language (scripting, etc). Both languages also have large (full-stack), well established frameworks (Rails & Django). Although both languages may challenge your programming paradigms by adding additional concepts and syntax for functional programming, they are still very well suited for classical OOP. Obviously Java has a heavy classical design. The transition from classical to functional can be rather shocking, so these choices may be helpful to ease that learning curve.
Conversely, Clojure has developed an interesting following because of its very functional (lisp-derivative) nature. And although it does leverage the JVM as its runtime, it is pretty far from Java development. While interesting, this language generally fails to meet most of your points above.
You mentioned Groovy, but I would avoid that because there doesn't seem to be as much retaining community (as other languages mentioned). In my research, it seemed to be just an experiment of porting Ruby-esc syntax back into Java. (Similarly Grails attempts to be the Rails of Groovy). It will maintain a lot of familiarity for you, but will likely fail miserably at #1 & #2.
I will skip Scala (which you also mentioned) as I have not done enough to fairly judge it. I will say that I did evaluate learning Play which is a Java/Scala framework, although that really doesn't help answer your question ;)
I don't necessarily want to pressure your decision by suggesting what works for me will work for you, but I was recently leading a similar transition while starting a new web application at work. (We have an enterprise Java app and wanted to evaluate other languages & frameworks). We eventually chose Ruby+Rails.