Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

What I will say might be a tad controversial in nature but I am very disheartened today - and so I will ask this.

I just had an interview with a major tech firm for an internship position, where I was asked a lot of typical algorithm oriented interview questions. Now, given my background, I consider myself to be strong in algorithms (I have also got good grades in graduate level algorithms -stuff involving NP-completeness and beyond (approximation and randomized algorithms), but unfortunately I flunked the interview. I could not think of a very efficient method of solving a string problem in approximately ~10 minutes. Once the interview was over, I had a glass of water, ate a banana and relaxed for a while and tried the problem again. And vola! there is the answer I could arrive in under 5 minutes. And the worst of it all - I was actually on that track and the interviewer did hint about it, but too much pressure cooked me. My entire experience got me thinking about tech interviews. I had some questions and I wanted to pose them in this forum -

  1. Is it really possible to judge someone's technical ability in half an hour? Honestly? Or is it just a throw of dice?

  2. Do technical interview questions measure problem solving ability? This point is very debatable? As a PhD student I know that Mathematical problem solving involves solving something that you have never heard about before. On the other hand questions like - merging two linked lists in sorted order, or printing all the elements of a binary tree in the kth level become "mere exercises" once someone has seen the solution or solved the problem beforehand?

  3. Do people who come out with flying colors in these interview go on to become great programmers? Do they go on and design a sleek game engines, graphics libraries, write fast fork-join frameworks? Is there any evidence to point to a positive co-relation between doing well in technical interviews and actual programming ability? Or are these interviews more geared towards finding "getting things done" type of person (Spolsky)?

I can bet that lots of academics publishing ground breaking ideas in - ICML, VLDB, Mobicom - will flunk these interviews. But I can assure you that they are some of the smartest people you will find on this planet.

I am mainly in academia (grad student) - so I will greatly appreciate some perceptive from someone in the other side of the fence. Someone who actually conducts these interviews?

[Ok everyone. Thanks for all the nice and thoughtful responses. Since I do not want to ask another question, I will ask you to answer this question for me.

Suppose candidate X has a good public portfolio of works where he has contributed to some known open source project where - you can actually go and verify his patches, verify the bugs he has closed and take a look at the designs he has created. In that case, the question is how much weightage are you willing to give to his publicly available/verifiable work versus how well he does in answering some very contrived binary tree interview question in under 15 minutes?]

share|improve this question
10  
When I interview people for jobs, I'm not looking for good programmers. I'm looking for good ENGINEERS. A programmer takes a look at a problem and codes a solution. A good ENGINEER looks at a problem and asked "is this the real problem I need to solve?" and if it isn't, figures out what the right problem is, designs a solution that balances all of the factors in play (schedule, money, ability) and then hands it off to a programmer to code it up. –  PlayDeezGames Feb 29 '12 at 1:39
1  
Hmm, makes sense. Anyone can pick up raw coding skills in C++/Java/python in 6-12 months. But to actually design stuff that require very careful understanding of performance, thread, memory issues and trade offs are the key. I agree. Sorry! but I can't up vote you, don't have enough reps. –  user396089 Feb 29 '12 at 1:51

5 Answers 5

Keep in mind that...

  1. The main goal of an interview process is not to arrive at absolute truth about each interviewees personal ability but to select few candidates from a pool of many.
  2. It is MUCH, MUCH more expensive to hire a bad developer than it is to pass on a good one.

So a lot of times when interviewing, imperfect shortcuts (such as technical quizzes) are taken because as you mentioned, there's no perfect process for being able to judge a person in 30 minutes. But since most don't have the luxury of working with you side-by-side, you should just accept that it is what it is, an odds game that everyone plays.

share|improve this answer
    
"It is MUCH, MUCH more expensive to hire a bad developer than it is to pass on a good one" - yeah I understand the argument here which seems to be from a purely utilitarian point of view, as even if the probability of hiring a bad developer is next to zero the negative cost associated with it can drag down the total expected utility value from positive to negative. –  user396089 Feb 29 '12 at 5:00
4  
@user396089, in academia people are at least going to pay lip service to providing a service to you: training, mentorship, personal growth, etc. At least nominally, it's part of their mission. In the private sector the relationship is 97% utilitarian. Companies are only interested in providing you an opportunity insofar as there is a benefit to them. –  Charles E. Grant Feb 29 '12 at 6:41
    
To add to what Charles said, when you walk through the door, you are, just like 50 other candidates, a complete stranger to the company and to the hiring manager. So you may be the best person ever, on professional and personal levels, but given the circumstances, the company's goals at that point really are to maximize their future value, at the same time minimizing search costs while looking for that maximization. –  DXM Feb 29 '12 at 7:01
    
You know it is funny that I am thinking about the simple string problem which I could not solve today during the interview but solved it later; but, still after trying for almost 3 weeks I am yet to solve a problem related to one of my hobby projects. The details of which can be found here - stackoverflow.com/questions/9056108/… –  user396089 Feb 29 '12 at 7:58
1  
@Antonio2011a: "how valid" is a funny term and it's not black and white. Different people use different tactics to judge the quality of potential employees. Every one of those tactics is some kind of shortcut because the only true test is to let person work with you and allow for ramp up time. So how valid this question? or how valid to ask about big O, or how valid to ask about what he worked on. These are all just gray areas and none of them are perfect. –  DXM Feb 29 '12 at 15:06

Is it really possible to judge someone's technical ability in half and hour?

Not exactly. It is possible to weed out the folks who can't program at all, and those who can't explain the stuff on their CV. Beyond that I'm usually just trying to gauge general intelligence and interest in the field. It's hard to comment on your situation further without knowing the problem you were asked to solve.

Do technical interview questions measure problem solving ability?

That's not the goal of my technical questions. Instead I am trying to discover whether the candidate has a grasp of the fundamentals of computer science. To gauge problem-solving ability, I ask a candidate to tell about an interesting problem they worked on.

Do people who come out with flying colors in these interview go on to become great programmers? Or are these interviews more geared towards finding "getting things done" type of person (Spolsky)?

That's about it. Greatness is rare. I'm happy to find someone competent.

share|improve this answer
    
The problem was a variation of KMP (as and when you make one pass through the string you store the number of occurrences of some characters which you had seen previously). At the end depending upon the length of the string and the count you have - you can answer the actual question. All in one pass - O(n) time and using constant space O(1) –  user396089 Feb 29 '12 at 2:00
2  
"I'm happy to find someone competent." –  AProgrammer Feb 29 '12 at 6:27
1  
Passing an interview is due to (at least!) as much luck as greatness. Once you’ve attained a certain level of knowledge, chance plays a much (!) bigger role than almost all people realise. Reading suggestion: Mlodinow’s The Drunkard’s Walk. –  Konrad Rudolph Feb 29 '12 at 11:09
    
Yes, I have read that book and also the even more interesting and sarcastic "Fooled by Randomness" by Nassim Nicholas Taleb. Needless to say I went on a depression for a while after reading the books - if you understand the basic premise of the two books and that is - "yes, its all luck! luck! and all about luck!" The World is random and we have to deal with it. –  user396089 Feb 29 '12 at 11:26
  1. Yes. Yes, honestly. This is not to say that 30 minutes is enough time to get a complete picture of what a person is capable of. The interviewer's job is to do there best to get a feel for who the interviewee is and what they can do. Finding the people that put something on their resume that they didn't really know is straight forward. Finding out exactly what the interviewee can do, including all their skills and strengths is not.

  2. Yes. They let you see how the interviewee thinks. Getting the correct answer is not always necessary. Seeing someone deal with a problem they struggle with is just as beneficial as seeing someone answer a problem correctly without much trouble. I always ask a question that I think the interviewee might not be able to answer.

  3. I don't know of any empirical proof that says technical interviews find the best candidates, but it is the best thing I have tried. Different people are good at different things and getting technical answers correct is not the only thing that decides if someone get hired. Being a good fit for the company is also very important.

Based on your question, I think the biggest thing you need to understand is that the days clearly defined feedback like a letter grade are over. Getting the right answer is only part of the result. Writing good code, communicating your ideas and listening to others is just as important. However, these areas are more subjective and need to be treated as such.

Just like how you solved the problem, take a step back and clear your head. A lot goes into an interview and the interviewer knows that it is not a perfect system. They are trying to do the best they can for the company, just like you are trying to do the best for you.

share|improve this answer
2  
That's a great point @unholysampler - "Getting the right answer is only part of the result". It doesn't matter one bit if you deliver a perfect online sales portal one day AFTER christmas. Too late. Time is just as important as having the right answer. –  jasonk Feb 29 '12 at 1:57
    
+1 "They are trying to do the best they can for the company, just like you are trying to do the best for you." –  Burhan Ali Feb 29 '12 at 17:40

In a half hour you can judge a person's technical ability is at least as good as x. That is, their ability might be higher than they demonstrated, but you can be fairly sure it's not worse. If x is higher than your requirements, you hire them. Yes, it's unfair if you demonstrate below your actual ability. All I can say is get better at demonstrating.

If someone rattled off a practically memorized answer, I would give them a harder problem. In academia memorization is an important skill. In a job, it's mostly redundant when you have google right there.

Seeing how most great programmers are employed, it's safe to assume they passed a job interview at some point. No, you can't tell if someone will be a great programmer from a half hour interview. Fortunately, most companies don't need great programmers. They get along just fine with solid, reliable programmers who aren't afraid to tackle tasks just outside their comfort zone.

The reason ground-breaking thinkers might flunk job interviews is that opportunities to solve flashy, ground-breaking problems are very rare, and you'll only be trusted to solve them after you've proven you can solve mundane, yet difficult problems your colleagues can't, like why the software crashes when it gets above 16% load.

share|improve this answer
2  
+1 "...mundane, yet difficult problems your colleagues can't, like why the software crashes when it gets above 16% load." –  Jaydee Feb 29 '12 at 9:45

Very few people know whether their interview techniques really work, because they never learn more about those who failed. But when nothing disproves their hypothesis they take it as confirmation that they are true. So there is a definite element of throwing the dice.

If the economic theories saying markets are rational and efficient were true, then companies would look to hire people that were bad an interviewing in ways that would not make them bad at their job. So someone who wore an ill-fitting suit, sweated, appeared very nervous, and had trouble doing technical problems under the pressure of an interview - if there was outside evidence they could otherwise do technical problems and the pressures of the job were not like those of an interview (as they generally are not) - would represent a company's best chance of getting a candidate better than it could otherwise attract and it would act accordingly. (Think Money-ball for the corporate world). Eventually interview biases would vanish.

In practice this goes against human nature. Interviewers like confident extroverts, no matter what the position, and even if the technical interview provides worse information than the resume and references (as perhaps is true in your case) they will take it into account.

As to your questions:

  1. Somewhat. The tests have a poor signal to noise ratio but can be tuned to weed out the completely unqualified at the expense of the qualified-but-bad-at-technical-interviews.

  2. See #1. Part of the noise comes from variation in whether people have seen the problem before.

  3. See #1. There is probably a tiny positive correlation between these tests and becoming a great programmer.

My advice - Study some sample technical questions to get a basic familiarity. Remember it's a bit of a roll of the dice, so during the interview try to be confident. If you can explain what you are thinking without distracting yourself from actually thinking then do so. If it's really going badly, admit you get nervous at interviews and let them know it seems like something you normally wouldn't have trouble with - possibly giving an example of a similar but harder problem you did solve.

share|improve this answer
1  
"Interviewers like confident extroverts" ...this is little disconcerting to me, my Myers Briggs type is INTJ (assuming MB is a little representative of the actual personality type). –  user396089 Feb 29 '12 at 2:46
2  
Many software developers are introverts like me. Consequently, so are many interviewers. Sure, confidence helps. But sensible answers help a lot more. –  kevin cline Feb 29 '12 at 7:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.