Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Both asserts and unit tests serve as documentation for a codebase, and a means of discovering bugs. The main differences are that asserts function as sanity checks and see real inputs, whereas unit tests run on specific simulated inputs and are tests against a single well-defined "right answer". What are the relative merits of using asserts vs. unit tests as the main means of verifying correctness? Which do you believe should be emphasized more heavily?

share|improve this question
2  
I like this idea a lot... so much so that I'm making the topic an assignment for my software testing and quality assurance course. :-) –  Macneil Nov 11 '10 at 19:26
    
Another question is: should you unit-test your asserts? ;) –  mojuba Nov 11 '10 at 20:04
add comment

3 Answers

Asserts are useful for telling you about the internal state of the program. For example, that your data structures have a valid state, e.g., that a Time data structure won't hold the value of 25:61:61. The conditions checked by asserts are:

  • Preconditions, which assure that the caller keeps its contract,

  • Postconditions, which assure that the callee keeps its contract, and

  • Invariants, which assure that the data structure always holds some property after the function returns. An invariant is a condition that is a precondition and a postcondition.

Unit tests are useful for telling you about the external behavior of the module. Your Stack may have a consistent state after the push() method is called, but if the size of the stack doesn't increase by three after it is called three times, then that is an error. (For example, the trivial case where the incorrect push() implementation only checks the asserts and exits.)

Strictly speaking, the major difference between asserts and unit tests is that unit tests have test data (values to get the program to run), while asserts do not. That is, you can execute your unit tests automatically, while you cannot say the same for assertions. For the sake of this discussion I've assumed that you are talking about executing the program in the context of higher-order function tests (which execute the whole program, and do not drive modules like unit tests). If you are not talking about automated function tests as the means to "see real inputs", then clearly the value lies in automation, and thus the unit tests would win. If you are talking about this in the context of (automated) function tests, then see below.

There can be some overlap in what is being tested. For example, a Stack's postcondition may actually assert that the stack size increases by one. But there are limits to what can be performed in that assert: Should it also check that the top element is what was just added?

For both, the goal is to increase quality. For unit testing, the goal is to find bugs. For assertions, the goal is to make debugging easier by observing invalid program states as soon as they occur.

Note that neither technique verifies correctness. In fact, if you conduct unit testing with the goal to verify the program is correct, you will likely come up with uninteresting test that you know will work. It's a psychological effect: you'll do whatever it is to meet your goal. If your goal is to find bugs, your activities will reflect that.

Both are important, and have their own purposes.

[As a final note about assertions: To get the most value, you need to use them at all critical points in your program, and not a few key functions. Otherwise, the original source of the problem might have been masked and hard to detect without hours of debugging.]

share|improve this answer
add comment

They are both tools used to help improve the overall quality of the system you are building. A lot depends on the language your using, they type of application you are building, and where your time is best spent. Not to mention you have a few schools of thought about it.

To start, if you are using a language without an "assert" keyword, you can't use asserts (at least not in the way we are talking here). For a long time, Java did not have an assert keyword, and a number of languages still don't. Unit testing then becomes quite a bit more important. In some languages the assertions are only run when a flag is set (again with Java here). When the protections are not always there, it's not a very useful feature.

There is a school of thought that says if you are "asserting" something, you might as well write an if/throw meaningful exception block. This thought process comes from many asserts placed in the beginning of a method to ensure all values are in bounds. Testing your preconditions is a very important part of having an expected postcondition.

Unit testing is extra code that has to be written and maintained. To many this is a drawback. However, with the current crop of unit test frameworks out there, you can generate a larger number of test conditions with comparatively little code. Parameterized tests and "theories" will perform the same test with a large number of data samples which can uncover some hard to find bugs.

I personally find I get more mileage with unit testing than sprinkling asserts, but that is because of the platforms I develop on most of the time (Java/C#). Other languages have more robust assertion support, and even "Design by Contract" (see below) to provide even more guarantees. If I were working with one of these languages, I might use the DBC more than the unit testing.

http://en.wikipedia.org/wiki/Design_by_contract#Languages_with_native_support

share|improve this answer
add comment

When talking about assertions, keep in mind that they can be turned off at the flick of a switch.

Example of a very bad assertion:

char *c = malloc(1024);
assert(c != NULL);

Why is this bad? Because no error checking is done if that assertion is skipped due to something like NDEBUG being defined.

A unit test would (likely) just segfault on the code above. Sure, it did its job by telling you that something went wrong, or did it? How likely is malloc() to fail under the test?

Assertions are for debugging purposes when a programmer needs to imply that no 'normal' event would cause the assertion to fire. malloc() failing is, indeed a normal event, thus should never be asserted.

There are many other cases where assertions are used instead of adequately handling things that might go wrong. This is why assertions get a bad reputation, and why languages like Go don't include them.

Unit tests are designed to tell you when something you changed broke something else. They are designed to save you (in most cases) from going through every feature of the program (however, testers are important for releases) every time you make a build.

There really isn't any distinct correlation between the two, other than both of them telling you that something went wrong. Think of an assert as a breakpoint in something you are working on, without having to use a debugger. Think of a unit test as something that tells you if you broke something that you aren't working on.

share|improve this answer
    
That's why assertions are for always supposed to be true statements, not error testing. –  DominicMcDonnell Nov 11 '10 at 5:08
    
@DominicMcDonnell Well, 'should be true' statements. I sometimes assert to get around compiler quirks, such as certain versions of gcc that have a buggy abs() built in. The important thing to remember is production builds should have them turned off anyway. –  Tim Post Nov 11 '10 at 5:18
    
And here I think production code's the place that needs the asserts the most, because it's in production where you will get inputs that you didn't think possible. Production's the place that drives all the hardest bugs out. –  Frank Shearar Nov 11 '10 at 7:45
    
@Frank Shearar, very true. You still should be failing hard at the earliest detected error state in production. Sure the users are going to complain, but that's the only way you are going to make sure that the bugs are going to get fixed. And it's a whole lot better to get a blah was 0 than a memory exception for dereferencing null a few function calls later. –  DominicMcDonnell Nov 11 '10 at 8:42
    
why is it not better to handle typical but often uncommon (in the eyes of the user) issues rather than assert that they should not ever happen? I fail to realize this wisdom. Sure, assert that a prime generator returns a prime, which takes a bit of extra work, which is expected in debug builds. Asserting something that a language can natively test for is just, well, stupid. Not wrapping those tests in another switch that can be turned off a few months after a release, even more stupid. –  Tim Post Nov 11 '10 at 9:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.