Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

First off I have seen this post which is slightly similar to my question. :

What can you do to decrease the number of deployment bugs of a live website?

Let me layout the situation for you. The team of programmers that I belong to have metrics associated with our code. Over the last several months our errors in our live system have increased by a large amount. We require that our updates to applications be tested by at least one other programmer prior to going live. I personally am completely against this as I think that applications should be tested by end users as end users are much better testers than programmers, I am not against programmers testing, obviously programmers need to test code, but they are most of the times too close to the code. The reason I specify that I think end users should test in our scenario is due to the fact that we don't have business analysts, we just have programmers. I come from a background where BAs took care of all the testing once programmers checked off it was ready to go live.

We do have a staging environment in place that is a clone of the live environment that we use to ensure that we don't have issues between development and live environments this does catch some bugs.

We don't do end user testing really at all, I should say we don't really have anyone testing our code except programmers, which I think gets us into this mess (Ideally, we would have BAs or QA or professional testers test). We don't have a QA team or anything of that nature. We don't have test cases for our projects that are fully laid out.

Ok, I am just a peon programmer at the bottom of the rung, but I am probably more tired of these issues than the managers complaining about them. So, I don't have the ability to tell them you are doing it all wrong.....I have tried gentle pushes in the correct direction.

Any advice or suggestions on how to alleviate this issue ?

share|improve this question
1  
You have to give the Powers That Be a reason (that's relevant to them) to change. If you can associate a dollar-amount in savings (in the long-run) with proper testing, then you can probably convince them to move to a better testing methodology, maybe even hire some full-time testers! –  FrustratedWithFormsDesigner Oct 9 '12 at 17:54
    
If you think "end users" are better testers does that mean you don't believe in testing anything before it gets to them. if thats the case its no wonder the number of bugs have increased. It sounds like your problems are a quality problem, the only way to increase quality, is hire better programmers. –  Ramhound Oct 10 '12 at 11:46
    
Not at all I just think programs need to be tested by someone else other than a programmer. The reason I say they are better testers is they tend to find bugs because they don't know how the application works if you haven't experienced this maybe you aren't a programmer. –  User Smith Oct 10 '12 at 12:06
add comment

5 Answers

up vote 7 down vote accepted

We require that our updates to applications be tested by at least one other programmer prior to going live.

That's a start, but you are writing here because you have proven this to be insufficient.

Applications should be tested by end users

We used to joke that we'd let end-users test our code. The point of testing is to find bugs before the end-users do. That way, end-users have a good experience, and recommend, or buy more of the software.

not [tested by] programmers

What's wrong with programmers testing? I think it's an important part of being a good programmer. Some schools of thought recommend that programmers write tests before even writing code! I think the problem that you are getting at is only having programmers do testing.

staging environment in place that is a clone of the live environment

This is very good. But if it is truly a clone, then the learning you gain from installing from development to staging is not getting transferred from staging to production. One way or another, you aren't getting the full benefit of this system. Do you do your testing in the staging environment?

We don't do end user testing really at all.

End user testing is usually about acceptance of new features, ease of use, and meeting the users needs. Not about finding bugs. End-users finding bugs = lost customers.

We don't have a QA team or anything of that nature. We don't have test cases for our projects that are fully laid out.

Eek! This is the root cause of your problems. A QA team would be fantastic, but even one eager and capable full-time tester would be an improvement. You could get a short-term contractor, but having someone on-board long-term who can weave testing into your whole development cycle would be much better.

I work in a small company and we have a written regression test that we go through for every release. It is not fun, but it catches bugs. We take turns and just do it.

Maybe you mean something different when you talk about end-users finding bugs. If so, you need to find a different way of expressing it, because that's a losing proposition if I've ever heard one. Specifically, it sounds like it loses customers. Advocate instead for:

  • A professional tester
  • A written regression test
  • Test-driven development
  • Establishing a culture of Quality.

Each of those is a time-honored industry-standard solution to a real issue you are experiencing that is affecting your bottom line. If you keep metrics, then you should be able to estimate a dollar figure to the number of users who leave because of bugs, and the negative advertising the bugs generate. There is also an opportunity cost, where customers who don't find bugs are happier and more likely to recommend your product. If 80% of the total cost of the bugs to the company is enough to hire a tester, that's your whole business case right there. A slam-dunk really. Good luck!

share|improve this answer
    
Thanks Glen probably the most concrete answer out of the bunch. I will definitely bring up your thoughts in the next meeting we have. –  User Smith Oct 10 '12 at 2:10
add comment

I personally am completely against this as I think that applications should be tested by end users and not programmers as end users are much better testers than programmers.

This is not a good practice. Your application functionality should be tested by programmer before handling it to the testing stage. QA should be responsible for testing of functionality as well as related modules in your application. After it past QA, your client users (not end-users) should have a UAT (user acceptance testing). Once it passes UAT then you deploy it to production.

However, if your programmers perform testing through writing unit tests, then this is is a great practice to follow.

share|improve this answer
    
Already updated question with response to this obviously programmers need to test. Quote taken out of context. –  User Smith Oct 10 '12 at 11:42
add comment

Developers should be doing real unit and integration testing that can be automated. Ideally after every check in/commit (or at least nightly) all of those tests should be run and pass before the code is released. There are a number of continuous integration tools out there can do this for you (e.g. http://jenkins-ci.org/).

When defining the tasks for your project, make sure your time estimates include developer time for building these tests. NOTE: Usually you will spend more time writing tests than you will writing the actual application code.

Whenever a bug is reported, start by adding a test that reproduces it. Then fix to the test. This will prevent the same bug from coming back.

As for changing management's outlook you can try the following: * Track every hour that is spent resolving bugs * Highlight the bugs that have come back repeatedly * Show them how much time is being wasted fixing bugs rather than improving and adding new features * If you have some downtime start setting up these processes yourself, so at least you know your code is good If you can show them in numbers the cost to the company it may change their outlook.

share|improve this answer
add comment

Manual testing by developers beyond 'yep, the app starts' etc., is a thorough waste of time. Automatic testing is worth doing, as is examining the causes of failures to see if you can reasonably prevent them happening again either in the same code or in other code.

It's ok to have end users test if you have nobody else. Ideally have that structured so that end users can choose to test or to use a more stable version, and make sure you have a good way of getting information back in the event of a failure.

End users might be the only people who could possibly think of, say, testing what happens when you're in a locale where converting "I" to lowercase doesn't give "i" (Turkish, and this bit me), or running the application from a DVD drive, or without any network connected. The trick is to involve them, actually respond to what they report and make them feel like they're contributing instead of being victimised.

Also, in certain environments you might pick a 'friendly' end user to test certain features/fixes. Especially if you can watch them use the program, you can learn a lot that way.

share|improve this answer
add comment

There are a lot of issues here.

I personally am completely against this as I think that applications should be tested by end users and not programmers as end users are much better testers than programmers.

This is exactly the wrong attitude. You programmers are the first line of defense against garbage leaking out. Your end users are the last ones you want doing your testing. For one thing, it's too late. Those bugs should have been detected and fixed long before the product went out the door. For another, your company is risking having those end users becoming ex-users when they finally get so sick of having a buggy product that they go elsewhere.

We don't do end user testing really at all. We don't have a QA team or anything of that nature. We don't have test cases for our projects that are fully laid out.

You may not need a QA team, but that doesn't mean you can forgo quality. All of you need to act like you are members of the QA team. You need to do end-to-end testing. Unit testing isn't enough. You need to take on the attitude that bugs shouldn't creep in. Nip them in the bud.

Ok, I am just a peon programmer at the bottom of the rung, but I am probably more tired of these issues than the managers complaining about them. So, I don't have the ability to tell them you are doing it all wrong.....I have tried gentle pushes in the correct direction.

There apparently is a problem with management here, but you are pushing exactly in the wrong direction. I suspect your management has a "get 'er done" kind of mindset is only paying lip service to quality. This needs to change. Your competition is going to steal your customers away from you with a better and cheaper product if it doesn't.

share|improve this answer
    
I updated my question per your comment regarding programmers not testing obviously programmers need to test code. As far as MGMT is concerned, yes very get er done and out the door kind of mind set that continually bites us in the rear. –  User Smith Oct 9 '12 at 18:35
    
So you should advocate for more time for testing by you programmers, not none. –  David Hammen Oct 9 '12 at 19:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.