Swing is platform-dependent to a certain extent. In addition to the widget themes being different, as you noted, we've had Swing-based applications where the layout is different from platform to platform. There was also another group I worked in that had some widgets not even show up on certain platforms.
I'm not sure how big a problem this is though, because I'm not sure how much Swing is used anymore. Eclipse's SWT seems to be a lot more platform-independent, so it seems that Java + GUI doesn't necessarily imply non-portability.
Also, any Java code that uses the operating system or makes assumptions about the underlying platform will naturally be less portable, for reasons including (but not limited to):
- case (in)sensitivity of the filesystem
- newline characters
(Many of these can be easily handled by Java APIs --
As a counterpart to the saying you mentioned, some of the more experienced programmers in my group say that Java is "write once, test everywhere".