For hiring new engineers at our company, we strongly believe in testing whether someone can actually write meaningful, usable, reasonable-quality code. Have been trying to devise a test for this for some time. The qualities i am looking for in this test are:
- It should not take less than 3-4 hours of work which is the allotted time for the test.
- It should be language and technology independent.
- It should not be a trick question, if you can code well in your language of choice, you should be able to do it easily with a bit of googling.
- The exact answer code should not be available from google itself.
- It should consistently be of the same difficulty level so that we are always comparing apples to apples. At the same time, one candidate walking out of the door after attempting the test should not be able to give any hint or answers to the next candidate.
Given all these considerations, the one test we are considering is to have a very large number of URLs pointing to news stories on the web and the idea is to give each candidate a random URL from this large set and ask her to scrape the news story, given a language of her choice, and save it in a structured way (json, xml) such that we have all required fields properly extracted. For example story title, date, author, text etc. Candidates are free to use internet and use whatever help they would otherwise have available when attempting a real programming problem in a job-setting.
What does the community think of this way of testing? What biases do you think exist in this test? Also Id love to hear other people testing programmers with similar testing methods. Would also like to hear suggestions for similar/better test questions.
EDIT: I realize that the question we devised is not ideal, in fact far from it. So while, your critique is welcome, you should also suggest some questions which are language-neutral, domain-neutral, require non-trivial coding and dont have answers already available on the internet.
EDIT: Clearly people think its actually offensive for programmers to call their friends if they are stuck with a problem so i am taking that out from the process. (But, ironically asking someone on stackoverflow is ok)