Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

My boss wants to recruit testers based on their testing efficiency (number of bugs identified). So, he's shortlisted 5 people and I need to give them an app full of bugs and see how they fare in reporting obvious bugs, and hidden bugs. I know.... it kind of sounds weird. I guess, this is just like the coding world, where you hire a programmer by assessing his/her programming ability (which is a little easier). Once hired, these testers would be testing a java swing app, so their familiarity of testing frameworks/tools is not really required.

So, my question here is - How do I go about finding buggy apps (web/non-web), preferably java ones, that I can have the shortlisted testers have a go at? How would you go about this task if your boss asks you to do so? I am kind of clueless at this point - I googled a bit, thought about finding new apps on sourceforge with lots of bugs, but both approaches didn't work for me.

share|improve this question
    
If you practice RDD, the number of bugs isn't critical. It's more critical to find hard-to-fix bugs first, which is very app-specific. Depending on your development methodology and goals, you could have different tests on a buggy application, which would reveal the skills of the testers (or show their absence). –  superM Sep 11 '12 at 12:40
add comment

3 Answers

up vote 15 down vote accepted

I actually did this. I created a buggy calculator. It's just HTML and JavaScript. I created it originally to help my father (a QA professional) practice testing. And because it was fun. I then used it when interviewing.

I don't just look for # bugs found though. I look for:

  • What bugs are found
  • How methodical is the person in testing
  • How clearly are the defects communicated
  • What questions does the tester ask
share|improve this answer
    
cool. Would you please share this buggy calculator with me, if you still have it? I would really appreciate your help –  Jay Sep 11 '12 at 2:57
2  
I'd rather not as I still use it at interviews. I think the key was that it wasn't hard to write. Think about the things that can go wrong when coding. And how you'd want a tester to go about testing them. –  Jeanne Boyarsky Sep 11 '12 at 2:59
1  
Great idea. Calculators are quite easy to create, but it's also easy to create buggy corner conditions, etc. It might be fun to throw in a deliberate rounding error so that 5.5000000 rounds down to 5, rather than up to 6. Sort of like Easter Eggs, but with bugs! –  Peter Rowell Sep 11 '12 at 3:43
    
@JeanneBoyarsky What kind of bugs would you add to your calculator? I found one here javascriptkit.com/script/cut18.shtml It is javascript, so changing source should be easy, I guess, but I not sure about what kind of bugs should I add for them to identify. –  Jay Sep 11 '12 at 5:04
    
@JeanneBoyarsky what can go wrong in coding a simple calculator? I am having to think that I will have to randomly fail it - for instance incorrect operations being randomly performed. But, if I were to code a calculator, why would I randomly fail it? Isn't calculator too simple an example? –  Jay Sep 11 '12 at 10:17
show 3 more comments

This should be easy. You have one or more applications already, right? And you used source control, I hope. And you should have a record of historical bugs. And you really ought to be able to easily build from old versions in source control.

So see how well they can find the bugs you actually had. If that isn't a realistic test I don't know what is.

share|improve this answer
add comment

If you want to do this, create your own buggy app. When I was hired into my current position, I was given an app to test and find bugs that was based on actual software the company worked on. Some of the bugs were pretty obvious, some were more subtle, but mostly here were plenty of bugs or arguably buggy behaviour. My two cents: while this can be fun, it doesn't necessarily make for a good screen of one's testing abilities. Finding obvious bugs in a simple and small application isn't generally too challenging.

If you want to step things up a notch, both for yourselves and the applicants, find real apps and webpages and ask candidates to look for bugs within them. Not only will bugs be usually less obvious, but it may also lead to some enlightening conversations that give some insight into the applicant's thought processes and evaluation skills. It may also force you and your colleagues to up your testing abilities and bug finding skills. But remember, even real-life apps have bugs and issues with them, so finding buggy apps shouldn't be too tall a task.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.