Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

So we're having an internal debate on whether the time restriction that we have in place for our technical test a good thing. I thought I would open it out to the community. So what are your thoughts on this restriction, do people accept it as standard practice or is it an artificial restriction?

A little background, for all our positions, we expect candidates to complete a coding exercise and we set a time limit which we think is reasonable to at least layout the design (we don't expect a fully working solution), and it is this time limit which is under debate. The current time limit is 2 hours (which I frankly think is generous.)

share|improve this question
    
What kind of time limit are you talking about? Hours? Days? Weeks? –  Oded Mar 8 '12 at 11:55
    
@Oded, updated the question... –  Nim Mar 8 '12 at 12:00
    
When you state you "we don't expect a fully working solution" are you telling that to the coder, since if you're not, or only giving that information if asked, it will effect the results. –  blunders Mar 8 '12 at 12:42
    
Bear in mind that you always have the option of not mentioning the time limit and just letting someone get on with it. Whether they went over or stayed under the limit can simply be a factor in assessing what they came up with. –  Marjan Venema Mar 8 '12 at 13:51
    
@blunders, as I mentioned in a comment below, we state that in the spec, "we don't expect a fully working solution", just your thoughts... –  Nim Mar 8 '12 at 16:00
show 1 more comment

6 Answers

up vote 5 down vote accepted

One way you might evaluate the legitimacy of the time limit and the questions involved is to give the same exercise to a sampling of staff that are already employees. Otherwise you're literally guessing based on a preconceived idea about what someone "should" be able to do.

By doing small technical exercises in a short time you are evaluating a narrow indicator of performance on a very artificial situation. The results of such a test have to be taken in context with other factors. Probably not a good idea to use the results as a strict "go / no go" filter.

share|improve this answer
    
Funnily enough, all of the folks here have passed that test (aside from one or two founders!) ;) –  Nim Mar 8 '12 at 15:59
add comment

There is no project that I know of that has been running indefinitely, such a thing is an utopia.

Since the prospective hire is going to be under permanent stress and his tasks will always be o a time limit, it comes naturally that he should be subjected to a test with a time limit.

But, be careful, him being able to finish the test under your time limit does not automatically make him competent (or the other way around) if you do not set the limit properly.

share|improve this answer
1  
Some U.S. federal government software projects approach indefinitely. Utopia is not a word that comes to mind when thinking about those projects. –  Gilbert Le Blanc Mar 8 '12 at 13:51
    
While I do understand the concept of very large and long lasting projects, I am more inclined to think that the projects you are talking about are actually successive projects either trying to achieve the same goal or even with different goals. But I am genuinely curious as to what projects you are talking about. –  Andrei G Mar 8 '12 at 14:18
    
Modernization of the IRS systems. The project has lasted so long, CTG built a building next to the IRS offices in New Carrollton. Modernization of the FBI case management system. I'm sure Google will provide details. –  Gilbert Le Blanc Mar 8 '12 at 14:28
add comment

Time limits give you an out. A competent dev should be able to come up with at least a first approximation solution, even if its just pseudocode or interfaces. Someone who is not competent will immediately start coding, and will tangle themselves up.

I haven't done that many interviews (< 50 ) in my career so far, but I have found that candidates who finish within the time period are generally competent, whereas those who go over the time-limit are either detail-or-analysis-obsessed or incompetent, and further questions about their design can weed this out.

share|improve this answer
    
detail-or-analysis-obsessed should not be such a bad thing, should it? –  Andrei G Mar 8 '12 at 13:06
    
It is when it leads to analysis paralysis. If you fail to deliver a project on time because you spent too much time in analysis, it's as much a failure as if you spent too little time in analysis. Realistically, the sort of problems set during interviews are things like 'Count the number of words in this file on the filesystem', not 'Write us a fully-functional Document management system'. In my experience, anyway :) –  mcfinnigan Mar 8 '12 at 13:11
    
I like "analysis paralysis"... :) –  Nim Mar 8 '12 at 16:03
add comment

You seem to have the process in place for some times. I'd suggest to analyze its results. How do the candidates behave? How did the candidates you ended up by hiring behaved at the test? Were they satisfying? Then you know if the current process is too lenient or not, and what are the problems it doesn't detect. (Sadly it is difficult to know if and why it rejected persons who would have been good hires).

share|improve this answer
    
That's the counter argument to the time limit, by placing this artificial limit, are we also rejecting good but slow candidates.. I guess there is no real win... –  Nim Mar 8 '12 at 16:02
1  
@Nim: at some point, isn't slowness disqualifying? –  kevin cline Mar 8 '12 at 16:31
    
@kevincline, I totally agree, I just wanted to canvass some opinions to see what the general thinking is. The 2 hour time limit is more than generous for anyone of any experience to come up with some sort of design. If they cannot, that in itself is a no-no for me (implies they cannot really think on their feet/adapt/react quickly...) –  Nim Mar 8 '12 at 16:39
    
Knowing if an hiring process is too strict is difficult. But as long as it doesn't prevent to hire timely and isn't too resource consuming, that is not really a problem from the company POV -- the homogeneity of the hired people has both advantages and drawbacks. But if too many employers are biased in the same way, it is a problem for those looking for a job. –  AProgrammer Mar 8 '12 at 16:50
add comment

When it comes to interviews it makes sense to have time limits - otherwise you can end up interviewing all the time...

For coding exercises, the time limit should reflect expectations from the candidate - if you believe the exercise can't be finished in the allotted time you should communicate this to the candidate (otherwise they might believe they are expected to finish the whole thing and get discouraged and not do their best). You should also communicate what the candidate is expected to accomplish.

Even though this is an artificial limit, it needs to be in place - candidates know how long they have to spend on the exercise and it also tests how they self manage their time and how they self organize.

share|improve this answer
    
We do communicate exactly that (i.e. we don't expect a fully working solution, pseudocode with some descriptions is sufficient), we just want to see their thinking, hence why I am in favour of the time limit. –  Nim Mar 8 '12 at 12:05
add comment

I think, in general they are a good idea. It allows you to draw on another comparison between two candidates.

For example, if Candidate A completes the task in an hour and scores 80%, Candidate B completes the task in one hour and half and scores 80%. Naturally you would go for Candidate A.

Every other variants on this depends on your preference. Do you want someone that takes longer to do things but also does them very well. Or someone that does it quickly and does good enough.

It will also show you how the candidate reacts to a pressurised time limit (2 hours is far too long). If they rush, their code will be sh*te but others will take their time, even knowing they won't finish within their time limit.

It's entirely up to you essentially.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.