Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Possible Duplicate:
Good interview programming projects

We are looking to hire some more Java developers onto our team, and plan to test their coding abilities with a test. We currently use a web-based Java test that automatically compiles and runs the code, but it is very flaky and we're having problems with our candidates losing their work on this site. Not only is this frustrating for everyone, it makes us look like we don't know what we're doing.

Is there a popular testing suite out there? What do you use?

I'm not interested in dogmatic arguments on whether or not I should be testing my candidates in this way, I'm looking for a tool that will help me do it.

share|improve this question

migrated from Apr 5 '12 at 0:44

This question came from our site for professional and enthusiast programmers.

marked as duplicate by Caleb, Walter, Morons, gnat, Bill the Lizard Apr 5 '12 at 14:40

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

Maybe have the candidates fix the system you're using? Or have them develop a new one? – David Apr 5 '12 at 0:42
Ask them for their TopCoder handle name and rating, and take a look at their submissions online. An entry-level candidate worth talking to should be able to achieve a rating of above 700; anyone with rating above 1200 should be taken seriously. If you manage to get a 1500+ to talk to your company, make an effort to hire her or him: you wouldn't see too many of them, unless you're Google :P – dasblinkenlight Apr 5 '12 at 1:25
But topcoder can be rigged by using multiple accounts; one to read the question, work on it in your own time, and then use another to copy & paste the solution for maximum points. – James Apr 5 '12 at 1:39
@dasblinkenlight : Your comment assumes the candidates would be using TopCoder. It would be pretty silly to exclude candidates simply because they don't float in the same internet circles as you do. – James P. Wright Apr 5 '12 at 3:36
@JamesP.Wright Or because they were working on actual projects instead of random internet coding competitions... – Fomite Apr 5 '12 at 4:11

Why not do this on a local machine with the main IDEs that you provide for your employees installed (or just the ones used in your company)?

My advice is to give them a (partial) snapshot of a previous project and a bug report/feature request.

Have them fix the bug/implement the feature and see how they do it (how they familiarize themselves with the existing code, how fast they tracked down the bug/where to implement the feature, whether they created unit tests (and which ones), how maintainable the new code is, ...).

This is what they be doing 90% of the time they are working for you. It requires a tester to evaluate the created code after it's done.

Having the test be on a unique platform is confusing and a website is not a IDE. Don't use it as such.

share|improve this answer

I personally don't like beating around the bush at an interview. If you want me to show you my programming and problem solving skills, give me something you would actually use to do the job and have me use it to demonstrate my knowledge and abilities. No trick questions, no whiteboards, no pens, no paper, no hand gestures. Put me in front of a typical development machine and watch me do what you hope I can do: write good code.

share|improve this answer
This would be nice but the point of these question isn't to see how quickly you solve the problem, but to see how you tackle the problem.. how your brain works while you seek a solution and how it takes you to realise that you may be on the wrong path. For financial-house interview they will keep on probing your answers until you can''t answer something. It's not a great feeling but that's how it works. – James Apr 5 '12 at 1:43
@James: I realize that it's like as you describe, from my experience alone, however it seems silly to ask me something when I can demonstrate working knowledge of what is being asked. – Bernard Apr 5 '12 at 2:22
  • Give the candidate a simple problem. Real world in 95% simple problems. I mean a really simple, textbook task, like parsing a trivial config file or finding a transitive closure of a graph.
  • Let the candidate use his preferred tools, any internet sites, etc — exactly what a real developer would.
  • Let the candidate work on code and comment his/her actions. Just make him/her post his code frequently for you to see (or share the screen). The code must finally run and pass tests.
  • Together with the candidate look through the working code and discuss any shortcomings [s]he can find. It can be anything, from a suboptimal algorithm to inadequate variable names and missing comments.
  • Once the code passes tests you have prepared / invented on the spot, change the requirements slightly. Make the config file use sections, or variable interpolation, make the graph oriented, etc. Real world work is full of such small changes.
  • Let the candidate cope with the change. That's where the really interesting things begin. Discuss the ways to restructure code, preserve / break backwards compatibility, etc.
  • If you have time and stamina, repeat the last 3 steps: discuss shortcomings, change requirements, implement.

For this, you won't need any advanced collaboration tools; any shared whiteboard (web-based, jabber/gtalk/skype chat, netmeeting) will do. Keep voice contact; add video if you like. You don't need to control the environment: an impostor won't be able to fake code discussion and the change of requirements, especially talking to you by voice.

share|improve this answer

Not really a suite, rather a service. It's called Codility, it has been described by Joel Spolsky in one of StackOverflow podcasts (transcript).

Spolsky: [...] You go there and it makes a little programming test, and you can tell it what language you speak, so it has English, Chinese and, I don't know, Hindi? I am not really sure, sorry... Polish, I think. [click] "Take a free test". Ah yes, English, Polish and Chinese. [...] You are given a programming problem, you can do it in Java, C++, C#, C, Pascal, Python and PHP, which is pretty cool, and you have 30 minutes. And it gives you an editor in a webpage. And you've got to just start typing your code. And it's going to time you, basically you have to do it in a certain amount of time. And it actually runs your code and determines the performance characteristics of your code.

share|improve this answer

You might want to check out Jon Jagger's fantastic Cyber-Dojo.

It's a web based integrated environment designed for doing deliberate practice of Test Driven Development and learning about team dynamics. It has lots of small programming tasks (kata's) and supports a range of languages, from Python and Ruby to Java and C++.

Unlike IDE's designed for productivity, there is no code-completion, syntax highlighting or auto-refactoring, so you get to see what your interviewee can do without these.

The best thing is, after doing a kata you can then go back and look at the red/green progression (or maybe not if they don't do TDD *8') of each of kata. Every compile/test commits the changes to a git repository along with the results of the test.

I think using this for interview coding tests could tell you a lot about not only a candidates ability to solve a problem, but also their approach to problem solving and the process they use when not constrained by external factors.

If you want your own CyberDojo server, the whole project can be found at github and there is even a Turnkey Linux appliance virtual machine linked from there, which means that assuming you already have VMware player or VirtualBox installed, you can be up and running within a few minutes of downloading the appliance!

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.