Kind of project for pair programming interview

Question

As discussed in the SO question Pair Programming for a job interview, there are mixed feelings on the usefulness of this approach. For those that have used this approach, what kind of projects are useful? From my limited experience with this as an interviewee, a key seems to be a well chosen task that the candidate can understand quickly and be able to contribute to (and the latter may depend somewhat on the candidate's background).

I've gone through a couple days of pairing interviews and found some periods to seem like a few hours were wasted: one because the developer wasn't working on something that lent itself to pairing (messy code that the developer wasn't very familiar with and was already interacting with someone about) and involved a tech I wasn't very familiar with. Another case was because an unexpected build issue prevented much useful work for a long while.

One shop trying this approach wasn't sure if they should have someone outside the company work on a customer's project - any thoughts on this? They also worried that explaining the domain and system would take too long, though without that the candidate may not be able to contribute much. So they chose an open source project the employee was working on, though that's likely not an option for everyone.

Recently, there have been a lot of folks working on coding katas for various things. I'm wondering if this would be a useful thing to use. Some sites even have them in multiple languages and multiple skill levels (e.g. http://codingkata.org), which could be useful for candidates and/or employees with different backgrounds.

[I guess coding katas could be used even for coding exercises in non-pairing interviews - I'll ask a separate question on that: What coding katas are good for interview exercises]

Answer 1

Atlassian makes use of a pair-programming exercise as a standard part of their technical interview process. The way they do it, they have a dedicated sample project (or in practice, they probably have a few different sample projects and they pick one at random for each candidate) that the candidate works through with a developer. The project is all preconfigured and ready to go, and starts out with some basic code (which generally contains a variety of intentionally placed errors/bugs/deficiencies).

Anyways, the developer sort of gives tasks to the candidate, which they work on together. The tasks are structured and sequenced so that they flow into each other, and so that the level of technical difficulty increases with each one. For instance, the first task may be something like "read the Javadoc to determine how this interface is supposed to behave, and then add a unit test to verify that it is working correctly", and then the next task might be "oh look, your unit test is failing, there must be a bug somewhere in the implementation, let's find it and fix it". The developer keeps handing out new tasks to the candidate until they reach some problem that the candidate is unable to solve, at which point the pair-programming exercise stops.

In general, I think this approach is much better than haphazardly pairing your candidate with an engineer to work on whatever that engineer happens to be working on at that moment in time. Most serious engineering projects are going to involve a code-base that is far too large for the candidate to become familiar with in such a short span of time, and may involve tools and libraries that the candidate is not familiar with as well.

Having a canned project for people to work on is better because you can pick something that is small enough for the candidate to understand quickly, that only requires standard libraries and tools to build, but that can also be engineered so that the tasks given to the candidate can test a broad range of competencies. Also it allows you to compare different candidates more objectively, because everyone starts from the same place and you don't get situations where a candidate may be hamstrung by some obscure build issue that was not their fault.

I was fairly happy with the quality of Atlassian's pair-programming exercise when I interviewed there. However, had they done the lazy thing and just stuck me with some random engineer to work on whatever bit of code he happened to be working on my opinion would probably be very different. So I think carefully pick the project, the tasks, and what specifically the candidate is expected to do, and you should be set.