Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We've been interviewing on and off for about two years, and I've run this by about 30 candidates. I wrote some 20 lines of code which perform a relatively basic operation, but has a bunch of errors in it. None of them are syntax errors. Errors range from bad memory management, incorrect data types, multithreading problems, misuse of keywords, and a couple functional errors that would give undesired output.

People get really nervous during this question. Is it a fair question to be asking? The goal is to get an idea of their comfort level with the language (Objective C has some very strange behaviors, and most applicants we get are relatively inexperienced in ObjC specifically as compared to other languages), as well as see their ability to recognize logical problems in otherwise foreign code.

Once a candidate has made a first pass, I often will point to specific lines that have remaining errors on them and see how much hand-holding it takes to get a correct identification and/or fix from them.

At our company, this is a real world scenario, as programmers would be thrust straight into a 100k line app and given bugs to fix, and we need people that are comfortable with working with the code of 6 other programmers.

However, today, another of our interviewers told me he hated this problem (I didn't quite understand what his reason was, admittedly). Is this something I should continue asking, or should I look for alternate avenues?

share|improve this question

closed as primarily opinion-based by GlenH7, MichaelT, gnat, jwenting, Bart van Ingen Schenau May 3 '14 at 14:12

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

Are you using any libraries in the test code other than basic standard ones? – Daenyth May 1 '14 at 21:26
Only standard libraries, and all standard code. Well, there is one fairly obscure enumeration method in there, but I leave it in to see if candidates will either a) already be familiar with it (good sign), or b) inquire as to what it does or figure it out and just roll with it. – puzzl May 1 '14 at 21:28
How many people do well on this code challenge? – FrustratedWithFormsDesigner May 1 '14 at 21:36
@puzzl: Maybe run it by other people not in your company who are familiar with Objective-C? Maybe it needs to be simplified a bit, or broken into separate smaller, independent challenges - some more difficult than others. – FrustratedWithFormsDesigner May 1 '14 at 21:45
I think the word "fair" in terms of interview questions is not quite right. It is better for both the company and the prospective employee that only the right matches are made. Keeping this in mind may result in very different questions than what candidates find comfortable. It's better to have people go away thinking a question was "unfair" than to hire them because you avoided hard questions and then fire them six months later. – Steven Burnap May 1 '14 at 22:24

This is perfectly fair. Reading code is inherently easier than writing code, so checking if they can do the most basic of professional tasks is a pretty important part of an interview. For junior candidates it may be tough to expect them to debug something on paper, as they're likely reliant on debuggers still. If you could have the broken program on a computer, that might be better.

I don't buy the whole "pressure situation is unfair" argument either. Clients and/or bosses will inevitably be breathing down your candidate's throat when some critical issue is found. How they work under pressure is part of evaluating how they'll do at the job.

All that said, some good people won't do well at this sort of thing. They're not critical, or they are more deliberate (or collaborative), or they're simply inexperienced with your particular problem (simple or not).

That doesn't make it less fair - you just need to be sure to account for other things as well.

share|improve this answer
I have a coworker who asks this sort of question, and says "let me act as your debugger". He'll tell them any state information they want to know about how the code executes. – Steven Burnap May 1 '14 at 22:20
Actually, I've found that reading code tends to be much more difficult than writing it, except in some few cases where the code is very elegant and clean. But as you say, reading code is an important task that you'll want a candidate to be able to do. – Allan May 2 '14 at 3:25
+1 for everything except "Reading code is inherently easier than writing code" - Boy, i've had to read and "decrypt" terrible-terrible code time and again. Reading code is not easier than writing code. Unless of course the code in question is either extremely well constructed, is too simple and isn't performance optimized. – techfoobar May 2 '14 at 4:01
@RobbieDee as an interview question it's not necessarily about finding all the issues or not, win or lose. It's also about how one approaches the task, their thinking process, their strategy etc. – Konrad Morawski May 2 '14 at 11:14
@techfoobar (et al) - Enh. If you give me some language I've never used before and tell me to fix this bug, I'm probably going to be able to step through and fix the bug. If you give me a language I've never used before and tell me to implement something, the latter is always going to be tougher to accomplish. Not only do you need to write the code, but then you get to fix the bugs too. – Telastyn May 2 '14 at 12:24

As an aside, I'd be interested to hear why the other interviewer thought the problem was inherently unfair, but on to the meat of your question:

It rather depends on what you're trying to achieve. If your goal is to recruit an Objective C guru, then the test is clearly fair.

However, if you're just looking for a journeyman Objective C programmer then perhaps not. Is the code typical of what they'll find in your organisation or purely an academic exercise? Certain types of code problems sometimes vary from organisation to organisation.

Also, how the test is staged could be a factor:

  • Are they told how many issues are in the code?
  • Are they timed?
  • Are the interviewers sat in the room or are they left alone?
  • Are they allowed to ask questions?

Whilst you point out that they'll be expected to fix such problems day to day, they won't be doing this from a source code print out.

The acid test of course is whether you're getting the right sort of staff once they jump through the hoop. If you are, then I'd be inclined to stick with it.

share|improve this answer

The question is not whether it is "fair" or not but whether the test a) is eliminating candidates who do not have the skills needed for the job and b) letting through those candidates who do. As long as the test does this job well, you should continue asking it.

The details of the test depend on the needs you have and the standards you want to maintain. When I show to a candidate a piece of code where a database API call could raise an exception if the record is absent but the code that uses the API call is written to treat all exceptions raised by the call to mean "record is absent", I'd expect any candidate fit for any job I offer to point out that the code is swallowing exceptions. I don't have a position for someone who cannot see this problem.

One way to lessen the opportunity for uneasiness could be to just have an extremely simple coding test before having the "find the errors" test. Once candidates pass the initial documentary check (i.e. their résumé looks good), I find that most candidates that don't make it through are failing at the simple coding test stage.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.