Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Having read a lot about this topic --- such as on this Software Testing Fundamentals site on verification and validation and Software Testing and Quality Assurance: Theory and Practice by Naik and Tripathy --- I still do not get it. Verification should prove that you are building the product right, while validation proves that you built the right product. But only static techniques (code reviews, requirements checks...) are mentioned as being verification methods. How can you say if its implemented correctly if you do not test it? It is said that verification ensures that the product meet specified requirements. Again, if the function is specified to work somehow, only by testing I can say that it does.

Could anyone explain this to me please?

share|improve this question
1  
Where are you reading this? Usually, verification and validation is treated as a singular activity that includes everything that goes into product quality. Both static and dynamic techniques are part of V&V. –  Thomas Owens Sep 10 '12 at 14:35
    
Please let use know where you are reading this. It is just plain wrong. See, for example, this link. –  Peter K. Sep 10 '12 at 14:49
    
For example here: softwaretestingfundamentals.com/verification-vs-validation Also a book I have says the same, that verification is only done by static analysis, while validation by dynamic (actual testing) –  user970696 Sep 10 '12 at 14:56
1  
@user970696 the book you have - would you mind telling its title and author? also, you mention "Wiki" - what's this? –  gnat Sep 10 '12 at 15:56
1  
@user970696 I don't have the book, but I have their slides from here. They don't appear to make the distinction you're talking about. For example, Ch3 of the slides talks about "Dynamic Unit Testing" which is certainly verification, not validation (because the whole system is not available). –  Peter K. Sep 11 '12 at 1:17

3 Answers 3

up vote 13 down vote accepted

As this question created some controversy, let me start off this answer with my background: Apart from being exposed to V&V in daily project work, I worked for several years in the software engineering department of my alma mater and am a lecturer for software engineering. While this does not guarantee that anything I say is correct, I hope it at least gives me the benefit of the doubt that there might be some truth in this answer.

Let me assure you, that you are not as confused as you may believe you are. What you have stated in your question is as much true as it is misleading. Let me first point out, what you have correctly stated:

  • Verification = build the product right vs validation = build the right product
  • Static techniques are part of verification - mainly because they take your program and some formal input derived from the requirements and evaluate them against each other.
  • Verification ensures correct implementation of the requirements (i.e. that you have built it the right way)

Now let me clean up the confusion about testing. First, as many comments have stated before, dynamic testing of code via automated tests (unit, integration, ...) is indeed part of verification. What causes most of the confusion, however, is that people in validation will talk about testing as well, yet mean something different: in validation, testing usually involves a person using the application for its intended purpose. In the optimal case, this is the customer him-/herself.

However, the "errors" [1] found by testing in verification and in validation differ fundamentally:

  • verification testing errors: these are errors that violate your requirements in one way or another.
  • validation testing errors: these are errors with the very product you have built, not its functionality, hence, they reveal errors within the requirements.

For most people, it helps to look at concrete examples of different V&V cases. The following are extreme examples of errors:

  • You have a low-level requirement that states "f(x) should return x+1" and your implementation of "f" always returns the constant 2. You may find this error by several different verification approaches, but your customer probably won't find it during validation, because you are building an e-shopping site and he neither knows nor cares about "f".

  • You have a requirement that states "The system should be able to handle 1000 requests per second at a CPU load maximum of 80%". Again, validation will have a hard time, just as much as most static techniques. In fact, the simplest way to verify this is to dynamically test your application by hammering it with requests and monitoring your CPU load during that time.

  • Consider the above requirement for "f" once more, this time with a "correct" implementation. All your reviews, static analysis and dynamic tests will report a success, but then your customer tests your software and tells you that he misses the shopping cart icon on the webpage. No amount of verification will be able to find this error, as you have made it during the requirements phase.

As you can see, "testing" - if not defined more precisely - is part of both verification and validation, and in fact, "testing" should be performed for both.

[1] "error" is used colloquially here, so as to avoid the distinction between error, failure, mistake, fault, ...

share|improve this answer
    
That was really great! Leaving to cinema, I would really clarify: also both black box or test case based testing are part of verification proces, apart from UAT testing by users. Am I right here? –  user970696 Sep 10 '12 at 18:46
    
But I must add, that ISO 12207:2008 does state only static techniques for verification :/ –  user970696 Sep 10 '12 at 18:50
    
Unfortunately many (most) developers do not work to requirements, so they get confused about the difference between Verification and Validation, hence the pervasive though V&V are the same and even worse, Testing == QA. –  mattnz Sep 10 '12 at 21:12
    
@Frank: Well, in the ISO I mentioned, it specifically mentiones that testing is conducted as a part of validation (7.2.5.3.2.1 and ..2). How come there is so much of dispute :( –  user970696 Sep 10 '12 at 22:06
1  
@user970696: Sure, those sections explicitly mention testing against user needs. The two sections I mentioned (implicitly: 7.2.4.3.2.3 and 7.2.4.3.2.4) state The code implements proper event sequence, consistent interfaces, correct data and control flow, completeness, appropriate allocation timing and sizing budgets, and error definition, isolation, and recovery. and The software components and units of each software item have been completely and correctly integrated into the software item --- how would you do either of those things without testing? –  Peter K. Sep 10 '12 at 23:26

You're jumping to conclusions. "If the function is specified to work somehow, only by testing I can say that it does" is not necessarily true. It is possible to have a specification that is accurate enough, and a platform that is trusted enough, that proving the computation to be correct is all that is needed or practical. For instance, consider a brute-force algorithm to find an optimal traveling salesman circuit; you can prove that it does what the problem definition says, but you can't possibly test it by running because it would take decades!

To be sure, almost always the specifications aren't accurate enough, and the platform you're using isn't trusted enough, to get by without trying the code out. That is why verification is of little importance in the industry, while testing becomes ever more important. But that doesn't mean that verification is flawed, or even a bad idea.

share|improve this answer
4  
That is why verification is of little importance in the industry - I'm not sure in what industry you're working in, but verification is pretty important here. –  Thomas Owens Sep 10 '12 at 14:38

Indeed: verification proves that you are building the product right, while validation you build the right product.

Therefore

  • verification is proving of the process... that you have followed the required standards and procedures, usually checked by static analysis and code review.
  • validation is proving of the resultant product... that the code does what is required, usually checked by dynamic analysis and testing, to show that specified inputs result in specified outputs

I don't usually cite Wikipedia, but in this case, it's helpful...

A more detailed explaination of the two processes can be found in ISO 12207 Software Life Cycle Processes (one of the other answers posts a link to an uncontrolled copy):

7.2.4.3.2 Verification Tasks

  • Requirements Verification
  • Design Verification
  • Code Verification
  • Integration Verification
  • Documentation Verification

7.2.5.3.2 Validation Tasks

  • Prepare Test Requirements, Cases and Specifications
  • Conduct the Tests

Now, the opening question asked why verification does not include testing. And despite some of the other answers, by P.SE members of high reputation, I put it to you that that is not the point of the verification activity, but of the validation activity.

Some answers suggest that you have to test to verify code or integration. No - that activity is to prove the modularity and the interfaces, etc, not the execution.

Utimately, what this discussion shows is that there is a lot of confusion as to what is verification and what is validation, and this is not helped by the boundary being a grey area, and defined slightly differently in different standards.

What is important is that both parts of V&V should be covered, and as long as that is the case then semantically, it doesn't really matter whether it's V or V... just V and V.

share|improve this answer
    
Thanks but now I am really confused. The comments to the question shows that verification is the testing phase, you say the opposite. –  user970696 Sep 10 '12 at 17:21
    
I see I've now got two down-votes... I'd be curious to know why people think my answer is wrong? –  Andrew Sep 10 '12 at 17:37
    
Well IMHO as stated in the comments, verification is the actual testing, while valiadion is more about user acceptance of the final product. –  user970696 Sep 10 '12 at 17:39
    
I absolutely disagree with that summary. In fact, that is plain wrong. Verification is nothing to do with the testing –  Andrew Sep 10 '12 at 17:40
    
Well, look at the links provided in the comments. Looks like its comomonly accepted the way I said, yet I too read different opinions. But reading university texts, there is eg.: Verification approaches attempt to idenfity product faults or errors.." –  user970696 Sep 10 '12 at 17:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.