Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I've applied for a junior development role, or rather been found by a recruiter looking for a developer. In order to get to a telephone interview stage I've been asked to sit one of those online coding assessments.

This wasn't quite what I expected. I consider myself a fairly good developer for my age and experience, but I've no illusions about being Don Knuth or anything. The test was a series of incredibly obtuse questions asking about the results of various obscure evaluations. About 30 minutes in I was thinking to myself I hadn't intended to enter an obfuscated code contest/code golf exercise.

After my last telephone interview I was asked to build something. I did. That seemed fair. Go away and work this out is more my in office experience of programming than "please evaluate this combination of lambdas, filters, maps, lists, tuples etc".

So I'm a little put off, to be honest. I never claimed to know the language inside out or all the little corner cases. My questions, then:

  1. Should I be put off? Why? Why not?
  2. Are these kinds of tests what I should be expecting for junior roles?
  3. Should I learn stuff exam style? That seems to be the objective of these tests, for which you are timed and not supposed to use references or books? Normally, in the course of development I have a fairly good idea of basic types, rules, flow control and whatever. Occasionally I'll come up on something I need to use a regex for and have to go and remind myself of the exact piece of syntax I need if trying what I think should work doesn't. Or I'll come up against a module I've not used before and go and look it up. For example, if I wanted to write a server using sockets in C right now, I'd probably check the last piece of code I wrote doing that (and or the various books I have) and work from there. Chances are I probably couldn't do it exactly from scratch and from memory, although I can tell you you'd need a socket(), bind(), listen() and accept() call and you might also want select() depending on whether you intend to pthread_create or not. So I know what the calls are, but not their specific parameter list.
  4. What are your experiences if you are a recruiting manager? Are you after programmers who can quote you the API or do you not mind if your programmers have a few books on their desk and google function calls every so often?
share|improve this question
    
Joel Spolsky thinks it is important to have the brain power to muscle through those. –  Job Mar 10 '11 at 22:24
4  
@Job: Joel Spolsky eats Wasabi for breakfast. –  Robert Harvey Mar 10 '11 at 22:32
2  
One thing to bear in mind is that the test isn't necessarily "pass this with flying colours or forget about getting hired." It makes sense to set a test that is deliberately too hard for the target audience so you can see how well they do: if the decent, good and excellent candidates are all smashing the test out of the park, it's a bit hard to distinguish between them. –  Carson63000 Mar 10 '11 at 22:38
    
I suspect that point 3 selects for programmers who cheat, but not too much. I wouldn't be surprised to see the output of that filter be close to noise, possibly biased a little towards "can evaluate what they find on google". –  Мסž Mar 11 '11 at 0:50
    
@moz that's what made me think. Had it been a test of what I can evaluate on google and the python interpreter (in this case), well, I'd probably have passed it 100%, because I could have filled in the gaps. Those things I use regularly I don't need to look up (Django, for example, the C standard lib, a few other APIs, a fair bit of Java) but I'm kinda stuffed if I'm expected to implicitly know the difference between partition and rpartition. I know what partition does but I don't think I've ever had cause to use rpartition. –  Ninefingers Mar 11 '11 at 11:33

6 Answers 6

up vote 3 down vote accepted

I wouldn't necessarily be put off. It's entirely possible that the company has a single "language FOO" assessment that they give to everyone regardless of the level of the position they're applying for. But they probably also expect lower scores if they're looking to fill a junior-level position. Most likely, the intention is to determine whether the candidate actually has any knowledge of a particular language and the company really only cares whether you hit some relatively low threshold score.

It's hard to come up with an objective exam for a programming language that doesn't, at least at points, devolve into a "what does this bit of obscure code do" test where you have to systematically break down the problem and apply the rules you know. If you look at various certification exams, for example, you see a lot of this sort of thing as well-- it's hard to determine how much you know about Java on a multiple choice quiz without using a lot of relatively tricky syntax.

In the real world, of course, it's almost always a good idea to use the language documentation or previous code to figure out how to implement things. No one knows the entire class library of a modern language or remembers the order of parameters to pass to every method without looking them up. To the extent that exams like this are testing something meaningful,

  • They are testing your ability to decompose a complicated problem into simpler components that you can address. In this case, they're testing your ability to untangle complex syntax with the hope that if you can do that, you'll be able to untangle complex statements in a requirements document.
  • They are testing your familiarity with the language you'll be working with. Now, we all know that a good developer can move from one language to another, but if an organization is going out to hire a Java developer, they're going to prefer someone who knows at least enough Java syntax from self-study to pass this sort of exam.
share|improve this answer
    
That's a good set of points. Combined with @Robert Harvey's answer, I guess I'll take away that idea, have a practise and patch up the weak areas and see what happens in terms of the result/when I apply for another role that does this. –  Ninefingers Mar 10 '11 at 23:25

You mentioned this was through a recruiter. Recruiters often require you to do a test ("Prove IT" or similar) so they have your results on file and can show it to the hiring company. It's extremely annoying, and usually a waste of time, but I would consider it a necessary evil of dealing with recruiters and not something that should automatically put you off from a role.

share|improve this answer

Well, there are two ways of looking at this:

  1. The coding assessment wasn't really obtuse. Rather, you lacked the nuts-and-bolts knowledge that you needed to complete it, thereby rendering the assessment obtuse to you, or

  2. The recruiting manager/HR department is clueless, and doesn't have the first idea of how to effectively identify those people that might be good candidates.

Either way, I think the outcome is the correct one: you shouldn't be working there.

Employers should be looking for people who can solve problems, not those who can memorize corner cases. That said, finding out what corner cases you know can be a good way to determine your depth of knowledge. But I would expect that depth of knowledge from a senior developer, not a junior one.

share|improve this answer
    
I guess that's fair enough. This isn't my strongest language and I'm well aware I'm not as up to speed on it as I am with other languages. I guess I'll take it as a learning exercise, research the areas that seemed awkward and go from there. –  Ninefingers Mar 10 '11 at 23:22

My guess is that the company uses this as a standard test for all its applicants, not just for junior roles. So don't be too put-off by some questions you may or may not know the answer to.

I personally don't like these tests. As you said, usually developers often reference how they once coded something out rather than trying to code something cold from memory. Our brains can only hold so many random facts.

Usually I've seen this with companies that have tons of applicants, where most might not be qualified for the job. As a very general example, a lot of engineers want to work at well-known gaming companies as their first jobs, and a lot aren't qualified, so the companies decide to use these tests rather than interviewing all the applicants.

I think they miss out on some quality candidates, as not everyone (including me) feels like jumping through all the hoops, but maybe it's a way to find people who are really passionate about the company. In my experience, tests like these are not the norm, but this might depend on the industry.

share|improve this answer
  1. Should I be put off? Why? Why not?

Shrug, you should but that's how it works anyway. They have to weed people out, otherwise they'll be interviewing everybody and wasting their time.

  1. Are these kinds of tests what I should be expecting for junior roles?

I had five interviews with tests kinda like what you've described. The worst was the stupid function memorization ones. Like dude there are 700+ built-in functions in PHP, I am not a god damn walking reference book. Heck, I had a 3 questions online coding test, one of them is a clever disguised problem that was actually a pumping lemma with a stack. Which seems trivial to some people but man I thought it was cute. Parse parentheses and see if it close correctly or not.

  1. Should I learn stuff exam style? That seems to be the objective of these tests, for which you are timed and not supposed to use references or books? Normally, in the course of development I have a fairly good idea of basic types, rules, flow control and whatever. Occasionally I'll come up on something I need to use a regex for and have to go and remind myself of the exact piece of syntax I need if trying what I think should work doesn't. Or I'll come up against a module I've not used before and go and look it up. For example, if I wanted to write a server using sockets in C right now, I'd probably check the last piece of code I wrote doing that (and or the various books I have) and work from there. Chances are I probably couldn't do it exactly from scratch and from memory, although I can tell you you'd need a socket(), bind(), listen() and accept() call and you might also want select() depending on whether you intend to pthread_create or not. So I know what the calls are, but not their specific parameter list.

Shrug, no clue. Just do what you love to do and find answer to your curiousity. Eventually you know a little of everything of your chosen field. My last interview they asked me a lot of web tech and since I love reading up on latest web technologies I was able to answer all of them. It doesn't mean that I have experiences with SOAP or SAAP or REST but I do know of them.

share|improve this answer

1) I wouldn't be put off, I've been asked to take a few silly tests for various jobs. My first role out of college I had to take the Raven's Progressive Matrices and another math related exam with "susan is 1/5th kates age, but in 4 years..." type questions.

These sorts of things were thought up by HR. Most developers just cracked jokes about it. It didn't affect the way I felt about the company one way or the other.

2) HR is clueless as to the needs of the IT department. I've also had to take programming related tests like the one you describe. Just ignore it and move on. Focus on whether you liked the environment, the people, etc. The person to person interview will give you much better clues.

3) NO, the things on the test are not important.

4) I've seen more and more companies claim to be "language agnostic". A good developer can be a good developer in other languages. There are far more important skills to posses such as communication and problem solving (using google).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.