Of course you can't ask only puzzles, and "gotchas" that revolve around language trivia are not very useful. But some kind of programming exercise is a virtual necessity if you use a staffing agency or recruiters, or if you keep hiring people who are good at faking their level of competence.
Asking the right kinds of questions can help gauge how familiar programmers are with common algorithms and how to apply them to problems. It can also tell you whether they look for multiple approaches and choose the best one, or just pick one that they know will work. In the latter case they may be aware that there is a better way but would need to look it up to get it right, which is fair enough -- working, if less than maximally efficient, code is better than code that doesn't work, and just knowing there's a better way is more than half the battle.
An example of a decent place to start is a problem I had in a programming competition back in 1985-ish, when I was a young Applesoft BASIC hacker. The problem was to print all perfect squares between 1 and 1000. Of course, there are two approaches to solving this:
- Iterate through all the integers between 1 and 1000. Take the square root of each and see if this was equal to the truncated-to-integer result of the same calculation. (No modulo operator in Applesoft BASIC!) If so, print the number.
- Iterate through all the integers between 1 and sqrt(1000). Print the square of each.
Obviously the second is far more efficient than the first, and as a bonus it will print all the right answers, while the first, due to rounding errors in an Apple's floating-point implementation, actually missed one or two. However, most of the solutions turned in used approach #1. (These were high school students, after all.)
A discussion around a modern-day problem of this sort could reveal whether a candidate is a aware that there are two approaches, can tell which is better, and understands the limitations of binary floating-point math. In fact, I would say that someone who couldn't tell you why the second approach is better would be a poor candidate -- basically on the level of a high school student from 1985. This problem is only slightly more complicated than "fizzbuzz" -- and much less complicated than anything they're likely to actually be working on!
My point is to know what you expect to learn from such a question, and to not set them as pass/fail questions but to use them as jumping-off points for probing the limits of a candidate's knowledge. You can ask a series of increasingly difficult questions and see where they begin to have trouble.
You might ask, how do I tell from a series of programming exercises whether someone is smart enough to work on my team? And the answer is: you don't, you rank them in order of smartness and hire the smartest you can afford, assuming he or she is smart enough.