If you change occasionally, say typing on QWERTY every few weeks or months, you can expect a 2-10 minute "warm up" period where you are slow. Perhaps you will need to look down, but you can type with 10 fingers, and look away after a few minutes.
If you change frequently, every few days or every few hours, then you should have no problem and should be able to switch instantly, full speed.
I do find that there is a visual element, so if you can only type QWERTY or Dvorak in certain specific graphical or environmental conditions, it will help with the switch. What I mean by this is if you want to type QWERTY and Dvorak in this same text box, your brain and muscles need to remember two full keyboard layouts with zero external support. If you type QWERTY at work on Mac with desktop background A while listening to music B, and then type Dvorak at home on a Windows box in a different environment, you'll have an easier time.
Another example: I can only recall certain passwords when I am looking at the page where I need to enter them. I cannot recall them typing in a blank text document without the small peripheral graphical cues.