Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I work for a small company (less than 200 employees) whose software group only makes up a small part of our staff (4 employees, occasionally with a few contractors). The four of us have been making strides in transitioning to better practices, and one of the next logical steps is to improve our testing.

As anyone who has done any meaningful tests knows, testing takes a lot of time - and at my company, it takes too much time to justify to management, so we generally do what little we do on the sly. I don't think this is serving us well, as we keep coming up against otherwise avoidable problems when we ship under-tested software.

I would like to be able to come to management with a justification for hiring a dedicated software test engineer (someone who can both write automated tests and perform manual ones). Are there any good published studies that show the benefits of adding such a position to a small company? Where can I find information about costs associated with the position? I plan on doing a little number crunching on our own history, but having some external sources to point to would help bolster my case.

share|improve this question
7  
Number crunching on your own history will be better - if you can show that lack of testing has cost the company $X,000 that much more impressive than a bunch of studies on the internet. –  ChrisF Oct 21 '11 at 14:40
    
When a customer sues the company for supplying buggy software, perhaps they will see the light. –  Oded Oct 21 '11 at 14:50
    
I did this once. I don't recall the studies I referenced at the moment (it was 2 years ago - will look later) but the metric was, with testing, increased upfront dev time by 40% and reduced overall time to market by 80%+ (with drastically fewer critical bugs - IIRC somewhere in the 90% range). –  Steve Evers Oct 21 '11 at 15:35
    
@SnOrfus - any chance you had the opportunity to dig up those studies? –  Nate Oct 24 '11 at 11:39
    
@Nate: I think that it was contained in "Best Kept Secretes of Peer Code Review" which sites the original study. Didn't find the book on the weekend. Will look some more this evening. –  Steve Evers Oct 24 '11 at 13:01
show 1 more comment

4 Answers 4

I believe that you should convince your management not by using numbers, but by explaining the rational.

Joel Spolsky makes some very smart points on the benefits of hiring testers in his article Top Five (Wrong) Reasons You Don't Have Testers, that may help convincing management.

Also here:

If your team doesn't have dedicated testers, at least one for every two or three programmers, you are either shipping buggy products, or you're wasting money by having $100/hour programmers do work that can be done by $30/hour testers. Skimping on testers is such an outrageous false economy that I'm simply blown away that more people don't recognize it.

Some more pearls of wisdom here:

There is no better way to improve a programmer’s morale, happiness, and subjective sense of well-being than to have dedicated testers who get frequent releases from the developers, try them out, and give negative and positive feedback. Otherwise it’s depressing to be a programmer. Here I am, typing away, writing all this awesome code, and nobody cares. Boo hoo.

share|improve this answer
3  
"A programmer’s morale, happiness, and subjective sense of well-being "... Please point me toward a manager who cared about any of those attributes; and where I can send my resume! –  Sheldon Warkentin Oct 21 '11 at 14:56
    
@Sheldon Warkentin: This is why so many of us find it so frustrating to read Joel. But, eventually, this kind of thinking is common in places that attract good programmers. –  Lior Kogan Oct 21 '11 at 15:00
    
@Sheldon they do exist, don't stop looking :) –  RYFN Oct 21 '11 at 15:00
1  
@Sheldon Warkentin: Much of Joel Spolsky's fame comes from promoting the idea that caring about these attributes enables a company to hire and keep highly qualified developers and allow them to work at their peak productivity, ultimately producing more value for the same money as a larger number of lowly-paid unhappy developers with a high rate of attrition. –  Michael Borgwardt Oct 21 '11 at 15:06
    
You beat me to linking to Joels list :) –  Carra Oct 21 '11 at 15:51
add comment

Don't waste time on number crunching. It's not compelling. Everyone disputes the details.

Instead, focus on specific risk scenarios.

Poor (or no) testing == failure in production == loss of revenue (or worse). Follow this line of reasoning with specific losses that can accrue from software failures.

Be detailed and focused so that the logic of "loss comes from bad software comes from bad testing" is inescapable.

What happens is -- of course -- focus on the specific risk scenarios. But that's better than no focus at all.

Very, very specific stories are easier to use than general studies and summarized numbers. It's even better when you can tie historical events, specific customers or products, or other factual details to your risk scenario.

"Remember when we spent 18 hours trying to fix the [X] product software? How much revenue did we lose from cancelled orders? And help desk calls?"

share|improve this answer
6  
While crunched numbers can be disputed, having no numbers simply invites the response that there are no "facts" being presented. Combining crunched numbers with specific risk scenarios could be more compelling. –  Mark Bannister Oct 21 '11 at 16:10
add comment

If you don't test enough, you lose profit to pay to fix buggy software in the field. If you test a lot and deliver late, you lose revenues because software was delivered late.

From a business leader's point of view, the former is usually preferable because delivered software is actually getting used, was probably paid for, and it is therefore easier to justify bug-fixing expenses after it was deployed and paid for.

If you want better testing, you need to show how testing will help the business leader get the software to market faster, within the same budget. In a large-ish software project, time-to market is enhanced when test coverage is good at the lowest layers of the system (e.g. OS, data layer) because that means that the folks developing on top of that don't have to fix their bugs and the lower layer bugs. It also helps to ensure good testing of success paths of key use cases.

You may want to consider outsourcing some testing to a shop in India or Vietnam. They do good work, and their loaded labour rates are much less than those in N. America. And they'll test while you sleep.

share|improve this answer
add comment

It sounds like you have a critical first step:

As anyone who has done any meaningful tests knows, testing takes a lot of time - and at my company, it takes too much time to justify to management, so we generally do what little we do on the sly. I don't think this is serving us well, as we keep coming up against otherwise avoidable problems when we ship under-tested software.

It sounds like you need to make the business case for testing at all and embedding into the schedule before you do anything else.

Here's how I did something similar:

  1. Projects were using classic Big Bang testing. There are plenty of resources showing that this method is the least reliable, most costly and (therefore) most likely to be cut for the purposes of "savings."

  2. In order to "hide" testing inside the project (rather than at the end), I dedicated every fourth week to testing and planning. This means that no developer was ever more than three weeks from something that they'd been working on. It also meant that every task was tested more than zero.

  3. Then I made the business case to the project managers. My justification was that (a) people have to test just to know if they've actually finished their tasking and (b) with early testing, I could easily claim a net cost savings of at least 50% savings.

That last point was a bit dramatic for some people. Considering that post-release bug fixes are considered to be at least ten times more expensive to find and fix, it was a fairly easy argument to win.

The four of us have been making strides in transitioning to better practices, and one of the next logical steps is to improve our testing.

I agree but I don't agree that the first step is to hire. You're talking about an increase in your group's salary budget of 20% without first demonstrating the return on investment. Invest in the testing capabilities and resources of the people that you have now. Demonstrate the value of testing as a core development practice. Then start working on the business case for a new hire when you have the history to back you up.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.