If you want to do Java-based web programming, you need to learn the Servlet API and learn it well.
99% of all Java-based web frameworks build on top of this API, so you need to know what is going on. Additionally it allows you to build simple applications where servlets you write based on core Java classes can do "real work" and generate HTML as a response (or images or PDF's or whatever else you can generate a byte stream for). This is a good starting point and allow you to create working stuff relatively quickly.
If you do not know HTML then learn it too. Do not consider the advanced new things in HTML5 yet because there is plenty in HTML4 to master first.
I would consider two good starter things to do:
- Generate a table in a servlet containing some kind of information based on something the user entered. For instance a multiplication table or so.
- Generate a GIF in a servlet which shows a rainbow in the browser for a given URL.
Please do not underestimate the amount of knowledge you will need to learn to truly master this area too, but do not become discouraged either. You can relatively fast learn enough to produce quite interesting things.