Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Based on my experience in technical interviews, I have found that most tend to be subjective because the interviewer already has his own answer. Even though the answer from the candidate is correct, because the interviewer was not prepared for that answer, the candidate does not get the job.

In an interview recently I said something about using an AVL tree algorithm to solve a specific problem that was asked. The interviewer responded: "What's an AVL tree?". Another example is anything around syntax; I've encountered this mostly in interviews that require Ruby code because there are many ways to implement a solution to a given problem. A very common ones is problems around Object Oriented designs.

In this situation, there's no way for the interviewee to succeed. Has anybody else felt this way too or is it just me? If it's not just me, how can we make technical interviews better?

share|improve this question

closed as primarily opinion-based by MichaelT, GlenH7, Ampt, gnat, BЈовић Jun 17 at 8:11

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

9  
I think you'd be hard-pressed to find an interview situation in any industry that isn't somehow subjective. It's the nature of the beast. Unless you could come up with some internationally accepted rubric for grading programming interviews, and even that could have a bias, then I don't think there's much you can do. –  jonsca Oct 12 '11 at 14:41
8  
Maybe he knew what an AVL tree is and was just testing you. –  Pastronio Faruglio Oct 12 '11 at 15:03
2  
Yes, they are. They're not meant to be 100% objective. –  quant_dev Oct 12 '11 at 16:57
1  
The goal of an interview is determine if you'll be a good match/fit for the team; it is not to assess your technical skills (although assessing them is part of determining if you're a good fit). For example there are people (including me) who dislike programmers that focus on design patterns, or that keep mentioning weird three letter acronyms I've never even heard of (not talking about AVL here). In that case they may be very good but they are not a good match and they won't work in my team, otherwise it will be bad for everyone involved. –  Andreas Bonini Oct 12 '11 at 17:32
1  
At least you were asked a technical question. I was recently asked what year I graduated from college. –  JeffO Oct 12 '11 at 18:03

10 Answers 10

I think your own blind spots are leading you to an assumption that is invalid as to why you don't get the job.

It is possible to answer everything correctly in a job interview and not get the job because you are competing against other people who may also have gotten everything correct. When you only have 1 job and three people you think can do the job, you still can only hire one. It's a tough market out there and lots of experienced people are looking. You never know just how well your competition did in their interviews.

Yes interviews are subjective, they are looking for people who not only have technical skills but who will fit in the existing group. Just keep trying eventually if you are capable, you will find the right place and they will hire you.

share|improve this answer
    
(+1) There is a lot of technical rockstars, that doesn't "play well with others". –  umlcat Oct 12 '11 at 15:40
    
It's always interesting that "have X great candidates, can only choose 1" usually applies to new hires but not to things like clients or contracts. –  joshin4colours Oct 12 '11 at 16:54
    
I am talking about technical interviews, not just interviews. Technical interviews should be objective right? Because they want to see how you can solve the problem that is why they did a technical interview. –  jpartogi Oct 12 '11 at 20:43
    
No, technical interviews are subjective. They are judging more than just your technical skill but how well they think you will fit in, how you respond to stress, how argumentative you are. And sometimes the approach you take can tell you how well this person is likely to interact with the current code base. For instance if you tell them strongly you would never use... and ... is all through their code base in such a way that it would be a major effort to get rid of it, they know you would be unhappy and thus a poor fit. And you still compete with what everyone else said. –  HLGEM Oct 12 '11 at 20:51
    
You are right if the company is looking for a certain number of people. But most technical interviews that I have been are with companies that are always looking for programmers. And usually the interviewer always give me feedback a few days later about my code design. From there I conclude that eventhough my code runs perfectly, the interviewer has his own taste. –  jpartogi Oct 12 '11 at 21:02

Any interviewer who dismissed a candidate simply because they didn't provide the expected answer is a poor interviewer. If a company encourages this and I get that feeling in an interview, that's a serious red flag about the team and company interviewing me.

If an interviewee provided me with an answer that I didn't think of, I'd expect him to be able to explain what the solution used is and the advantages/disadvantages of it. Like you said, there are often multiple solutions, ranging from choosing an algorithm to the syntactical structures used, and almost always there are tradeoffs involved. I'd even probe the interviewee for alternative solutions to see if they can come up with others, especially if it's an obvious solution, and ask for the advantages/disadvantages of each.

Regardless of how you run the interview, it's going to be subjective. Every interviewer, as a person, will have biases. As an interviewee, you can just do the best you can, be thorough (but not overly verbose) when answering questions, and explain your answers and how you got there. That'll take you a long way.

share|improve this answer

One that I found recently is when I said something about using an AVL tree algorithm to solve a specific problem that was asked. The interviewer asked me back: "What's an AVL tree?".

As an interviewer, I would ask that, too, even if I knew everything about AVL trees (I don't), to find out how much the candidate knows. Faking ignorance can be a trick to see if the candidate can explain well. But obviously, if you do come up with an algorithm/data structure that solves the problem, that I don't know, and that you can explain properly, I would hire you. Otherwise, shame on me. Not hiring people because they're smarter than yourself are is not a good hiring strategy.

But generally speaking: yes, technical interviews are always subjective. And they have to be. If you're going to spend 8 hours+ everyday with a person, it's wouldn't be reasonable to pick someone who manages to drive you crazy in a 60-minute interview.

share|improve this answer
    
Thanks for this. I was too naive to think that no interviewers would fake ignorance. –  jpartogi Oct 12 '11 at 20:56
3  
@jpartogi: They wouldn't if interviewees would never imply they know more than they do. The interviewer asked how you'd solve a problem, and you said "With an AVL tree". The interviewer now has to find out whether you picked an AVL tree because it was a good fit and you know how to use it, or whether you said "AVL tree" because you heard the phrase somewhere and thought it might be appropriate. –  David Thornley Oct 12 '11 at 21:24

Trivia Questions are anti-patterns to interviews. If you find yourself stuck in such an interview, try to direct them back to what you know. Concentrate on your resume. If this doesn't work...well consider looking elsewhere.

Interviewers should ask open-ended questions related to your Resume. While it's possible to get a feel for a person's communication skills, it's difficult to gauge a person's development abilities by simply asking questions. That's partly why so many people (including Joel on software) recommend, no require interviewees write code (and jump through several other hoops for that matter).

The fact remains that software development is still mostly about solving an unknown set of problems in an unknown amount of time. There's no set of tests that prove, ok this guy can build "a bridge". We are getting better at turning software creation into a more defined engineering process, but we aren't there yet.

share|improve this answer

The "real" technical interviews I had were all like: "Here is the problem, use these technologies, you have 3 hours". After that we did a code review and talked about why I did what I did. This way he saw what I already know and where I lack knowledgde.

Then we talked a bit about technologies and my goals in general and that was it. The interviews are, in my opinion, designed to get a feel for the candidate. You can not test everything. It takes a long time to get to know someone pretty well so that, again in my opinion, you should focus on the things that are important for you project and see how people handle this issues. The rest comes with experience over time

share|improve this answer

Im afraid that that the subjective part cannot be removed, only diminish. I frecuently have technical interviews, where the interviewer "pollutes" the interview with its own subjective ideas.

One simple example, as you mention, is that the interviewer wants an answer very close to its own answer. Another example is, when an interviewer its more interested to probe he/she knows more than the candidate. And another simple one, is when the interviewer, wants the candidate to know the specific menu location for an operation of a programming I.D.E., (Visual Studio, Eclipse, NetBeans).

I have been on several like that, and its very frustrating.

And, of course, there is also the hidden discrimination thing, where the interviewer doesn't want the candidate, from been poor, gender, political ideas, race, school, whatever you like. And its looking for any excuse to reject the candidate.

The weirdest part, is that, been an C.S. graduate, I know some psychology, (was going to study that profession), and sometimes, detect a lot of the subjective "interference". Or, when the interviewer goes to the office next door, and explicitly tells their coworkers, he / she doesn't wants to hire a candidate, for a subjective reason.

Something important, to consider, is that many Universities, or Companies, teach I.T. / Technical people how to do job interviews, and that teaching include both the technical & human factors, not just technical.

share|improve this answer

Here's how I do it:

  • First I ask the candidate to solve a realistic problem. Usually this is a problem I can solve easily; usually in a lot less than the alloted time.

  • Once the candidate is done; I become the student. I have the candidate talk and point through their solution and show me how they solved it. I ask questions about their ideas, and if they know of alternative solutions, and why they chose this solution over that.

Here's what I'm looking for.

  • Just because I happen to know a good solution to the problem doesn't mean I'm looking for that solution. Any valid solution within the initial constraints is acceptable. I'm interested in how they planned their solution, how they used the provided tools. I'm interested in what shortcomings they are able to identify in their own solutions.

  • I'm also interested in how well they listen and communicate. Were they able to understand and implement all of the initial requirements? How well did they explain the way their solution works?

At the end of all this, once I've got a good idea what the candidate is actually bringing to the opening, I might offer some of the points from my own solution, where I think my choices would be better.

If the candidate says something like "That's a good idea, I wish I thought of that," or "Oh, I didn't know about that technique," or even "I thought of that, but I used this instead" and supplies a sensible reason like "I understand this method better" or "I thought you were looking for this kind of solution", these all count strongly in favor of the candidate.

If it turns out the candidate chose differently from me, It might mean that I am wrong. Or if I am right, and the candidate is open enough to discuss the differences in solutions, it's a good sign that the candidate can be nudged quite easily to make better choices.

Here's How I know a candidate is a poor choice:

  • the candidate tries to fish for an expected solution from the interviewer.
  • the provided solution doesn't solve the stated problem, or somehow violates some of the stated constraints in a way the candidate cannot explain away.
  • the candidate argues with the constraints to try to change the problem he's expected to solve (remember that this is a problem for which I have a solution that can be implemented in much less time than alloted).
  • the candidate gets "stuck", and spends much of the allotted time doing nothing. Ideally a candidate would attack the problem from another angle or seek the interviewer to know what he should do when he cannot proceed.
  • the candidate doesn't know "why" he used a technique, even if he doesn't know of alternatives. Answers like "I don't know any other tool that does this" or "I don't know how this tool works, but it does" are fine, but "because I did" are questionable.
share|improve this answer
    
What would a sample problem be? –  Job Oct 13 '11 at 2:26
    
Try to pick something that agrees with the candidates resume. A resume that shows knowledge of web programming topics might suggest a "contact-us" form web page. –  TokenMacGuy Oct 13 '11 at 11:57

Do you think that, as an interviewee, your acceptance of a job is not subjective, to a certain extent? Recruitment is a two way process.

As an interviewer, if I and my colleagues “click” with a candidate, they have a good chance of being in the running for the job, assuming that other factors such as a home-coding exercise and whiteboard problems go well. To “click” is to be on my wavelength, to have a fruitful conversation, to share common values about the development process. How objective can this be?

On the question of the AVL tree I would be interested in the candidate’s explaining how it works and providing insights into properties and use. To do better than the explanation on Wikipedia. Bear in mind your audience may be someone in a corporate enterprise environment where the last reference to O(log n) in a technical discussion was basically... never.

As an interviewer I want to give a candidate every opportunity to shine. I imagine this would apply to any organisation you would want to work for.

share|improve this answer

A good interviewer won't ask a question where they expect a specific answer, unless the question is something trivially simple where there's only one correct answer. A good interviewer will see how you approach a problem, how you think through it, and how you arrive at your answer - if your answer is valid they should not hold it against you because you didn't do things the way they would do it.

It sounds like you have just been subjected to lousy interviewers more concerned about showing their "superiority" to the candidate than actually gauging technical skill.

share|improve this answer

A very important skill when you work in a team is to be able to justify your own solution and fairly evaluate someone else's. You need to be able to both learn from your colleagues and teach them.

If you want to use an AVL tree to solve a problem, and the other members of your team remember them vaguely from college and haven't thought about it since, you had better be able to explain its advantages to them. If someone presents a solution you don't understand, you had better be willing to ask questions until you do. If someone presents a clearly inferior solution, you had better be able to articulate why. If someone presents a superior solution, you had better be able to recognize that and set your ego aside.

Those are the skills I look for when I ask a technical question at an interview. Whether they came up with the "correct" answer off the top of their head is largely irrelevant. That's only important in school.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.