Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm freshly out of college, and starting university somewhere next week. We've seen unit tests, but we kinda not used them much; and everyone talks about them, so I figured maybe I should do some.

The problem is, I don't know what to test. Should I test the common case? The edge case? How do I know that a function is adequately covered?

I always have the terrible feeling that while a test will prove that a function works for a certain case, it's utterly useless to prove that the function works, period.

share|improve this question
    
Take a look at Roy Osherove's Blog. There is plenty of information on unit testing there including videos. He has also written a book, "The art of Unit Testing" which is very good. –  Piers Myers Sep 12 '10 at 0:43

7 Answers 7

My personal philosophy has thusfar been:

  1. Test the common case of everything you can. This will tell you when that code breaks after you make some change (which is, in my opinion, the single greatest benefit of automated unit testing).
  2. Test the edge cases of a few unusually complex code that you think will probably have errors.
  3. Whenever you find a bug, write a test case to cover it before fixing it
  4. Add edge-case tests to less critical code whenever someone has time to kill.
share|improve this answer
    
Thanks for this, I was floundering over here with the same questions as the OP. –  Stephen Nov 27 '10 at 2:13
    
+1, though I would also test the edge cases of any library/utility-type functions to ensure you have a logical API. e.g. what happens when a null is passed? what about empty input? This will help ensure your design is logical and document the corner case behaviour. –  mikera Jun 30 '12 at 10:49

If you are testing first with Test Driven Development, then your coverage is going to be up in the 90% range or higher, because you won't be adding functionality without first writing a failing unit test for it.

If you are adding tests after the fact, then I cannot recommend enough that you get a copy of Working Effectively With Legacy Code by Michael Feathers and take a look at some of the techniques for both adding tests to your code and ways of refactoring your code to make it more testable.

share|improve this answer
    
How do you calculate that coverage percentage? What does it mean to cover 90% of your code anyways? –  zneak Sep 3 '10 at 19:29
1  
@zneak: there are code coverage tools that will calculate them for you. A quick google for "code coverage" should bring up a number of them. The tool tracks the lines of code that are executed while running the tests, and bases that agianst the total lines of code in the assembly(ies) to come up with the coverage percentage. –  Steve Evers Sep 24 '10 at 22:04
    
+1 on testing first –  Ryszard Szopa Oct 12 '10 at 23:41

The stock answer is to "test everything that could possibly break".

What's too simple to break? Data fields, brain-dead property accessors, and similar boilerplate overhead. Anything else probably implements some identifiable portion of a requirement, and may benefit from being tested.

Of course, your mileage -- and your working environment's practices -- may vary.

share|improve this answer
    
Okay. So which cases should I test? The "normal" case? The edge case? –  zneak Sep 25 '10 at 18:35
1  
Rule of thumb? One or two right up the middle of the golden path, and just inside and just outside any edges. –  Jeffrey Hantin Sep 27 '10 at 21:45

If you start following Test Driven Development practices, they will sort guide you through the process and knowing what to test will come naturally. Some places to start:

Tests come first

Never, ever write code before writing the tests. See Red-Green-Refactor-Repeat for an explanation.

Write regression tests

Whenever you encounter a bug, write a testcase, and make sure it fails. Unless you can reproduce a bug through a failing testcase, you haven't really found it.

Red-Green-Refactor-Repeat

Red: Start by writing a most basic test for the behavior that you are trying to implement. Think of this step as of writing some example code that uses the class or function that you are working on. Make sure it compiles/has no syntax errors and that it fails. This should be obvious: you haven't written any code, so it must fail, right? The important thing to learn here is that unless you see the test fail at least once, you can never be sure that if it passes, it does it because of something that you've done of because of some bogus reason.

Green: Write the most simple and stupid code that actually makes the test pass. Don't try to be smart. Even if you see that there's an obvious edge case but the test take in account, don't write code to handle it (but don't forget about the edge case: you'll need it later). The idea is that every piece of code yo write, every if, every try: ... except: ... should be justified by a test case. The code doesn't have to be elegant, fast or optimized. You just want the test to pass.

Refactor: Clean up your code, get the method names right. See if the test is still passing. Optimize. Run the test again.

Repeat: You remember the edge case that the test didn't cover, right? So, now it's its big moment. Write a testcase that covers that situation, watch it fail, write some code, see it pass, refactor.

Test your code

You are working on some specific piece of code, and this is exactly what you want to test. This means that you should not be testing library functions, the standard library or your compiler. Also, try to avoid testing the "world". This includes: calling external web APIs, some database intensive stuff, etc. Whenever you can try to mock it up (make an object that follows the same interface, but returns static, predefined data).

share|improve this answer
    
Assuming I already have an existing and (as far as I can see) working code base, what do I do? –  zneak Oct 13 '10 at 1:25
    
That may be slightly more difficult (depending on how the code is written). Start with regression tests (they always make sense), then you can try writing unit tests to prove to yourself that you understand what the code is doing. It is easy to get overwhelmed by the amount of work that is (seemingly) to do, but: some tests is always better than no tests at all. –  Ryszard Szopa Oct 13 '10 at 8:01
    
+1 Good explanation of Red,Green,Refactor,Repeat –  RationalGeek Feb 3 '11 at 14:06
    
-1 I don't think it's a very good answer for this question. The question isn't about TDD, it is asking about what to test when writing unit tests. I think a good answer to the actual question should apply to a non-TDD methodology. –  Bryan Oakley Aug 21 '11 at 22:22

Probably my opinion is not too popular. But I suggest that you to be economical with unit tests. If you have too many unit tests you can easily end up spending half of your time or more with maintaining tests rather than actual coding.

I suggest you to write tests for things you have a bad feeling in your gut or things that are very crucial and/or elementary. IMHO unit tests are not a replacement for good engineering and defensive coding. Currently I work on a project that is more or less unusuable. It is really stable but a pain to refactor. In fact nobody has touched this code in one year and the software stack it's based on is 4 years old. Why? Because it's cluttered with unit tests, to be precise: Unit tests and automatized integration tests. (Ever heard of cucumber and the like?) And here is the best part: This (yet) unusuable piece of software has been developed by a company whose employees are pioneers in the test-driven development scene. :D

So my suggestion is:

  • Start writing tests after you developed the basic skeleton, otherwise refactoring can be painful. As a developer who develops for others you never get the requirements right at the start.

  • Make sure your unit tests can be performed quickly. If you have integration tests (like cucumber) it's ok if they take a bit longer. But long running tests are no fun, believe me. (People forget all reasons why C++ has become less popular...)

  • Leave this TDD stuff to the TDD-experts.

  • And yes, sometimes you concentrate on the edge cases, sometimes on the common cases, depending where you expect the unexpected. Though if you always expect the unexpected, you should really rethink you workflow and discipline. ;-)

share|improve this answer
    
Can you give more detail as to why the tests make this software a pain to refactor? –  Mike Partridge Aug 21 '11 at 18:17
3  
Big +1. Having walls of unit tests that test the implementation instead of the rules makes any change require 2-3x as many –  TheLQ Aug 22 '11 at 0:00
    
Like poorly written production code, poorly written unit tests are hard to maintain. "Too many unit tests" sounds like a failure to stay DRY; each test should tackle/prove a specific part of the system. –  Allan Nov 22 '13 at 6:20
    
Each unit test should check one thing, so there are not too many unit tests but, missing tests. If your unit tests are complex, that's another issue. –  graffic Apr 6 at 13:41

For unit tests, start with testing that it does what it is designed to do. That should be the very first case you write. If part of the design is "it should throw an exception if you pass in junk", test that too since that is part of the design.

Start with that. As you get experience with doing that most basic of testing you'll begin to learn whether or not that is sufficient, and start to see other aspects of your code that need testing.

share|improve this answer

Among the plethora of answers thusfar no one has touched upon equivalence partitioning and boundary value analysis, vital considerations in the answer to the question at hand. All of the other answers, while useful, are qualitative but it is possible--and preferable--to be quantitative here. @fishtoaster provides some concrete guidelines, just peeking under the covers of test quantification, but equivalence partitioning and boundary value analysis allow us to do better.

In equivalence partitioning, you divide the set of all possible inputs into groups based on expected outcomes. Any input from one group will yield equivalent results, thus such groups are called equivalence classes. (Note that equivalent results does not mean identical results.)

As a simple example consider a program that should transform lowercase ASCII characters to uppercase characters. Other characters should undergo an identity transformation, i.e. remain unchanged. Here is one possible breakdown into equivalence classes:

| # |  Equivalence class    | Input        | Output       | # test cases |
+------------------------------------------------------------------------+
| 1 | Lowercase letter      | a - z        | A - Z        | 26           |
| 2 | Uppercase letter      | A - Z        | A - Z        | 26           |
| 3 | Non-alphabetic chars  | 0-9!@#,/"... | 0-9!@#,/"... | 42           |
| 4 | Non-printable chars   | ^C,^S,TAB... | ^C,^S,TAB... | 34           |

The last column reports the number of test cases if you enumerate all of them. Technically, by @fishtoaster's rule 1 you would include 52 test cases--all of those for the first two rows given above fall under the "common case". @fishtoaster's rule 2 would add some or all from rows 3 and 4 above as well. But with equivalence partitioning testing any one test case in each equivalence class is sufficient. If you pick "a" or "g" or "w" you are testing the same code path. Thus, you have a total of 4 test cases instead of 52+.

Boundary value analysis recommends a slight refinement: essentially it suggests that not every member of an equivalence class is, well, equivalent. That is, values at boundaries should also be considered worthy of a test case in their own right. (One easy justification for this is the infamous off-by-one error!) Thus, for each equivalence class you could have 3 test inputs. Looking at the input domain above--and with some knowledge of ASCII values--I might come up with these test case inputs:

| # | Input                | # test cases |
| 1 | a, w, z              | 3            |
| 2 | A, E, Z              | 3            |
| 3 | 0, 5, 9, !, @, *, ~  | 7            |
| 4 | nul, esc, space, del | 4            |

(As soon as you get more than 3 boundary values that suggests you might want to rethink your original equivalence class delineations, but this was simple enough that I did not go back to revise them.) Thus, boundary value analysis brings us up to just 17 test cases -- with a high confidence of complete coverage -- compared to 128 test cases to do exhaustive testing. (Not to mention that combinatorics dictate that exhaustive testing is simply infeasible for any real-world application!)

share|improve this answer
    
+1 That's exactly how I intuitively write my tests. Now I can put a name on it :) Thanks for sharing that. –  guillaume31 Nov 22 '13 at 14:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.