Speed is one of the factors I take in account when interviewing people.
This factor is more or less important depending on the questions.
1. Basic technical questions
For example, when I ask:
What is the difference between Stack and Queue?
Personally, I expect a nearly-immediate answer.
- Either you know the answer and can immediately talk about FIFO/LIFO,
- or you are unsure, in which case it's not what I expect for a question so simple,
- or you don't know the answer.
On the other hand, as mentioned by Konrad Rudolph, it's "very easy to make mistakes on trivial questions when under stress [...] so this is even more of a reason to take my time".
If you answer immediately, you may make a mistake. If you spend too much time thinking about an answer, the interviewer may believe that you don't remember the answer well.
2. Problem solving
When I ask to solve a problem which requires thinking, the time it takes you to solve it is really not important at all, given the stressful context.
3. Advanced technical questions
Some questions even require thinking. Hurrying things will not help the candidate, since I expect a clear and detailed answer, not the first thing which comes into his mind.
Not only you should take time understanding such question, but also finding a correct solution among others which may come to mind, and finally presenting an answer in an easy to understand way.
That's what you see on Stack Overflow: some questions can be answered in a few seconds and in a few words. Others expect long answers which take time to elaborate.
Notes
Note that if the interview is made by phone, it will hurt you, since the interviewer may think that you're taking time searching for an answer on the internet or in your notes or books or, as suggested by Jeff O, being helped by somebody through Skype.
To conclude, if such interviews are an issue for you, try to show that you have a solid profile in a different way, for example by publishing lots of your source code, explaining in details your personal projects related to software development and showing your interest in them, etc.