10

Our developers demand that someone external to the development team completes the functional testing. Their argument is that they are biased and cannot test their own code as they are too close to it. Our last round of testing is with the client in the form of UAT.

To be clear by 'functional testing' I mean testing the code against the specs, eg. when you click on 'submit' the index page should load. By UAT I mean the client comes in and runs through use cases on the release candidate.

While I understand this argument of bias, is it reasonable for the developers to demand someone else complete their functional testing or is this just a way to avoid doing their job?

4
  • 2
    Is this a question of "either/or" or "and"?
    – Chance
    Aug 12, 2011 at 15:45
  • 3
    Yes functional testing MUST be done by a third-party to avoid bias. And YES THE DEVELOPER MUST TEST THE FEATURE before hand off to the third-party.
    – maple_shaft
    Aug 12, 2011 at 19:20
  • 5
    Functional testing must absolutely be performed by unbiased QA. Developers are responsible for making sure that their unit tests pass requirements. Then there is a level of integration testing and acceptance testing. If you are managing/supervising a team of developers, you should already know this. Developers are typically not responsible for writing the requirements, nor the acceptance tests. Don't be cheap, hire some QA personnel!
    – IAbstract
    Aug 13, 2011 at 14:00
  • 1
    @bobbyz, If you are a manager and asking this question, then you do not know what you are doing. Yes, testing is best done by someone with a different mindset.
    – Job
    Aug 13, 2011 at 14:25

8 Answers 8

16

It is true that developers have a bias when testing their own code. This is both a positive and a negative. For one, they know where the tricky parts are and therefore can do a good job writing tests that target them. On the other hand, they can have blind spots to other areas of the code. It's very easy to fall into the trap of thinking "this path through the code can't possibly break" or "nobody would ever use the tool this way, why test it?".

Testing requires a pragmatic approach. If you can afford it, yes, you should have people other than the developers that wrote the code do some testing. Not all teams can afford such a luxury.

To answer your specific question, then, the answer is yes. It's quite reasonable for them to make such demands.

8

Not only is it a reasonable demand, but I think a necessity for creating a good product. The end user should be testing to ensure not only that the product works, but that the product does what they need it to do. A good relationship between the developer and a user testing in goodwill creates a far superior product than anything created and tested by only the developer.

That said...

  • A developer should test all the code he has written.
  • A developer should test all the code he has written again.
  • If possible, a developer should find another developer to test his code.

The end user should be running through the product before a final release to ensure it works well. They have a better understanding of the nuances of the data and in all likelihood will find a small detail that they forgot to tell you or something you just missed.

In summary, the developer should test his product well so when the user gets it there are no or very few obvious errors. However, an end-user should test the product to catch more of the non-trivial bugs and of course any obvious bugs that were missed. This is especially important in mission critical and/or large/complicated projects.

1
  • 1
    +1 Well said! Developers MUST test their own code, unit and functional testing, however a third-party not involved in the feature should qualify it. This is basic software development.
    – maple_shaft
    Aug 12, 2011 at 19:18
4

Well the way you describe it sounds to me like developers are rather trying to help you get them more productive.

In my experience, the border between developer and external testing is quite fuzzy and flexible and much depends on a concrete project details. I for one have been working in projects with dev:tester ratio varying from 10:1 to 1:2 and all of them went well and fine. The only one that sucked was where there were no testers at all - and it sucked really bad. :)

That border I mention above gets even fuzzier if you take into account that there might (maybe even should) be close collaboration between developers and testers. Eg, developers might contribute quite a lot into product testability while testers contributing into getting specs and design right at the early stages.

It's hard to tell beforehand what amount of testing would be most productive to "offload" to external party in your case, but your note about ...testing the code against the specs, eg. when you click on 'submit' the index page should load... makes me feel like your team can gain some significant performance boost if you wisely hire one or two professional testers or QA consultants. If you get these guys right, you'll not only get immediate gain but also will get someone to help you tune the right dev:testers balance in your team.

  • By the way, before diving deeper into these tricky matters I would recommend you to do some preliminary study. For that, you can eg simply study job descriptions in vacancies from successful software companies (there's plenty available in the web). Find out which are core competencies considered for testers and which are ones typically expected from developers, try to "sense" the differences and similarities. This study may help you make more informed decisions further down the road.

PS. developers are rather trying to help you get them more productive -- for the sake of completeness have to note their argumentation does not look very compelling. The bias they refer to is a real and serious issue but I think it can also be addressed without involving third party - eg by making them cross-test each other. In my experience this trick helped to tame bias to tolerable levels (though have to admit it pales in comparison to "surgical" objective analysis done by qualified professional tester). I believe that most critical issues solved by employing third party testers are more related to matters of professional focus and productivity, division of labour... stuff like that.

0
2

Usually once the initial developer finishes writing the software AND testing it, another developer should look over the code again and retest it. Once 2 developers have looked at the code then it would be reasonable to pass it to a QAer for use case testing. But it IS the developers' responsibility to test the functionality of their own code.

2

I think you absolutely have to do both. Programmers have to do functional testing because functional testing is always part of programming. It would be extremely inefficient for a programmer to release code to someone else if he hadn't already made sure it actually functioned. (Sometimes you have no choice, such as when the programmer can't replicate a problem. This almost always makes things much slower.)

However, the complaint is legitimate. Programmers naturally do things the way they programmed the computer to expect because that's how they expected humans to do it. As a result, functional issues that anyone but the programmer would hit are missed by the programmers. I can't tell you how many times someone complained that my code was broken and didn't do what they expected, and my reaction was "Why would anyone do that?" Yet to many non-programmers, it was the most natural way to request the specified function.

Programmers must always do functional testing when it is possible for them to do so because that is the most efficient way to catch non-functional code and non-functional code is common. However, functionality also must always be tested by someone more like the expected user to ensure the code functions as the user expects and to ensure the programmers correctly understood the specification.

2

The way for you to "smell test" this request is determine how much of their own testing the developers have done. Did they do unit tests? Did they document them? Can they show how the work was tested against the initial requirements? What evidence is there that the work they have done is "ready" for formal review?

This will help you understand if they're just trying to throw it over the wall or if they've given it all the scrutiny they can and want a second set of eyes.

1

At my job we (formally, specifics may vary) use two levels of testing before a product reaches a customer:

  • Developers test their own code using Unit Tests: test class invariants and functions contracts, ensure maximum code coverage. I'd love to push for fuzzy testing, but it's hard to setup in general...
  • Business Analysts (the guys who wrote the specifications) validate the code (once) and create Non-regression scenarios. This is done both for changes and fixes.

This means that we (developers) test the implementation specific details, while the Business Analysts test the functional details.

It would be great if BA could provide the functional tests before we code (TDD), however my team has the peculiarity of working on web sites, so it's much easier to work/test the produce iteratively until we reach what we want, and then record the actions to create a test case.

1

One thing to keep in mind, especially as a manager: testers typically command somewhat lower salaries than developers. So on top of the bias argument, there are also economic benefits in hiring dedicated testers if there is enough testing to be done to occupy at least one person full-time.

3
  • 1
    I'd say that was a bit of a false economy - in my experience, hiring "cheap" for testers can work out pretty expensive. Testing is one of those professions where the very bottom end of the market is absolutely useless - there's a huge gulf between the unskilled commodity "warm bodies" some agencies will send you, and skilled candidates.
    – testerab
    Dec 8, 2011 at 1:16
  • @testerab: the very bottom end of the market is used in pretty much all professions. But even highly skilled testers will still command a lower salary than skilled developers. Dec 8, 2011 at 8:34
  • 1
    @michael_borgwardt For all I know, that may be true in your location/industry. But I know it's not true everywhere. I'm in the UK. Some companies pay testers less, but others have the same pay ladder for testers and devs of equivalent skill/experience. (Anecdotal: in one team I worked in, the highest paid tester's salary (wasn't me!) = 150% of highest paid dev, with roughly equivalent experience & responsibility. Several times I've discovered I was earning more than devs with same exp - I never told them though!)
    – testerab
    Nov 3, 2012 at 11:23