Consider an alternative approach if you're able to
Requiring non-technical recruiters to select quality technical candidates is difficult. You may find that using an online code submission process with an automated unit test would be more effective. Candidates login and are given a random extremely simple programming task (FizzBuzz, numerical sequences with limits etc). They post their code which is executed against a unit test (in a sandboxed environment obviously). This should be sufficient to screen out the non-starters automatically.
But I digress.
Recruiters want keywords
In general, the recruiter will not be able to understand half of what the candidate is burbling on about, but they can listen out for certain keywords. The purpose here is to identify candidates who appear to:
- know what they are talking about
- can communicate this knowledge effectively in the required natural language (English, Spanish etc)
- can demonstrate enthusiasm for the subject.
So, leading questions are the order of the day - and these are questions that recruiters love to ask. Ideally, they want to be able to count up the keywords and apply a subjective measure to identify the candidate that is most likely to result in a successful placement for them. You want to reduce the number of time-wasters that you're going to be talking to in the phone interview.
To make sure that the recruiters are not feeding their candidates answers, get them to provide a short summary of how the candidate performed.
Explain in detail how a browser gets a generated page from a web server
Listen out for HTTP, request, response. Extra points for session, connectionless, stateless, socket, port 80. And so on... you get the picture.
How would you attempt to stop spam in your email or blog?
Listen out for keyword filtering, blacklist, whitelist. Extra points for Bayesian filters, early manual feedback and so on.
Overall, spend a bit of time exploring Programmers to identify some interesting questions and see if you can incorporate them into your question bank.