What's a good name for a third type of testing?

Question

I generally have different types of tests that I work with - the standard types: Unit Tests (testing basic functionality of classes, methods etc) and Integration Tests (testing how multiple components interact etc).

However I also have a third type of test. It's basically manual tests where I use the unit testing framework and test runner to try out functionality (some people use console apps for this). I typically don't have any asserts in these tests, instead I use the debugger or Debug.Write:s. I often use these types of tests when exploring legacy systems.

Is there an established name for these types of tests? (I'm thinking along the lines of "Exploratory Tests" or "Manual Tests" but these are already established.)

Answer 1

I know the type of "test" you mean and do something similar, but I would not call them tests at all. I would call that "exploratory code".

Usually when I am doing this in a structured way I do two additional things:

1. I start writing documentation for the code I am exploring to avoid having to do it again in a few months time (and I document the exploratory code itself as well).
2. If applicable, I actually do add test assertions to the exploratory code whenever something isn't blatantly obvious. That way my exploratory code actually becomes a unit test during the course of an exploration.

Answer 2

The use of debug.write makes it sound like manual white-box testing, which is basically testing the internals of the code rather than its interface.

Answer 3

I've always referred to this sort of thing as Visual Checks. I kind of liken it to what a pilot does when he wanders around the outside of his airplane, kicking the tires, checking there aren't any birds nesting in the engine intakes, pulling bits of bird out of the various tubes from the last time you hit one... that sort of thing. The pilot has no real idea if his plane is entirely airworthy, but he does at least have an understanding that nothing obvious has been missed and he should be ok to at least turn the engine over and go through his pre-flight.

Visual Checking of your software is much the same. It's not really going to tell you if your code is actually working, but a quick debug through a problem area of code might help you spot something obviously wrong as a kind of a sanity check before you go and tell your boss what a great job you've done of your latest task. It certainly doesn't hurt, and can provide a good starting point for a major debugging exercise when you know you have a serious bug in your code that you should walk through, however for the most part you don't gain anything in terms of testing by relying too heavily on visual checks.

Answer 4

I would call this an "ad hoc" test.

Answer 5

I have never come across the proper/best name. I typically preface them with exercise_. For example "exercise_action_foo" or "exercise_bar_dialog". When working w/ WinForms projects, I typically do this for dialogs. Additionally, I add an [Ignore] attribute so they don't run on the build server. This might be 100% 'correct' but can be a time saver during development.

Answer 6

You can call these "test harnesses". These do not contain asserts, but exist to help you exercise functionality for the purposes of experimentation.

Answer 7

I like the term Eyeball Test for a manually-verified unit test. Basically I setup everything in a test fixture and try to get it to launch quickly, only I verify its correctness with my eyeballs, and possibly interacting with a control. This is pretty much the only practical way to unit test GUI layouts and can easily be incorporated into a TDD/test-first style.

Answer 8

I don't think there's a good, standardized name for this type of 'testing', but I've always called it Smoke Testing or Sanity Testing.