I'd like to say that as a programmer you shouldn't be creating the visual design at all, instead handing that off to UI and graphic designers, but unfortunately I know all too well that most companies aren't willing to invest any money in that unless it's going to directly generate revenue.
So the next best thing is to wait as long as possible before getting into that. You can hash out most of what the UI is going to do with tools like Balsamiq or the free Pencil, and if you're using a good presentation architecture (MVC, MVP, MVVM, etc.) then you should have no trouble at all testing the UI functions without an actual UI.
The more complete your GUI design is, the more everybody else who sees it is going to think that you're "almost done". You don't want this. Back when I did mostly Winforms development, I tended to start with the UI first, but I've since learned my lesson, and now it's always the last thing I do, after the controller or view-model unit tests.
The other issue is that it's simply a distraction and requires you to approach the problem from a completely different perspective. If you spend too much time working on the GUI when the library/model/business/domain code's not done, you're going to hurt your own productivity from the constant context-switching. People aren't very good at multitasking; don't do it if you can avoid it.