What do you call the test suite developers run before checkin?

Question

I'm working on a project where we're adding automated testing to an existing project. We're starting with a central component, and setting up both unit tests at the code module level, and tests that run on the whole component, albeit in the developer environment. The intent is that this test suite must pass before code check-in, and will be run by a continuous integration system on each development branch.

What should we call this? Right now we're calling it "dev unit tests", but the pedantic side of me says that's not quite right, because it contains more than unit tests. And our vision is that this suite will grow over time to include full-product acceptance tests.

Any input here? Or should we stop arguing about names and go write tests?

Update

Everyone, thanks for good discussion! I think the conclusion I'm coming to is that there's not really a "common" definition - each project/team comes up with names that make the most sense for that project, depending on the technologies (Java, C#, ruby, etc) and methodologies (old-skool waterfall, Scrum, XP, etc) in use.

Answer 1

They are known as "Gated check-ins" in MSFT TFS. http://blogs.msdn.com/b/patcarna/archive/2009/06/29/an-introduction-to-gated-check-in.aspx

Answer 2

Call them whatever you want, it doesn't matter. What does matter is the quality of the tests and that they are run.

Our tests are Automated Acceptance Tests but we just call them "tests" here at my work. As in:

• fix your "tests"
• you broke the "tests"
• did you write a "test"
• this "test" sucks, so I rewrote it

Answer 3

Automated Unit Tests is what I have always called them.

Answer 4

I'm not sure if there is an official term or not but I would call them Automated Tests. I agree with you the word unit being in there is not accurate.

Answer 5

In my organization we call them "unit tests" before the build. They're called "integration tests" after the build when all the pieces are tested together. They're called "system integration tests" when the build package goes to the QA team for testing (after integration testing has finished successfully).

Answer 6

I call them "the tests" with the implication that I'm talking about the "automated unit tests" and with the corollary implication that these are fast (a few seconds to a few minutes being what I've experienced more frequently.

Sometimes there will be other tests that run automatically on a check-in, or once at night for example. These tend to take longer. Something between 30 minutes and a few hours. Typically we call those "the functional tests", or "the regression tests" or even "the slow tests".

Answer 7

The term "Automated Tests" cover the spectrum from Unit Tests, Regressions Tests, Integration Tests, Acceptance Tests, etc.

Answer 8

We usually call them smoke tests, sanity tests or priority 1 tests. Priority 2 tests happen after a checkin and then priority 3 tests run on the scheduled builds. Our final set of tests, priority 4 tests, happen when we have a special build that occurs over a weekend - as they take about that long to run.

Answer 9

From my experiences:

A unit test is a test or set of tests that covers a particular module. In OO programming, a module is typically a function or class. Unit tests are grouped into test suites. Unit tests are the building blocks of regression and smoke tests, and in some cases can serve as documentation for the intended behavior of a system or pieces of a system.

Regression testing is the act of running most or all of the unit tests on a changed module. This ensures that the changed module continues to work as expected following the changes.

Smoke testing is the act of running a (small - you want smoke testing to be fairly quick) number of unit tests on unchanged modules to ensure that a change to a module did not have any unintended side effects on other modules. The focus is typically on classes that have associations with the changed class as well as the modules that provide key functionality to the application.

Depending on your build environment, any of these might be automated or executed by the developer. A committed change set should be small enough such that regression tests don't take an unacceptably long amount of time, and smoke testing is designed to be fairly quick. Typically, I run at least regression and smoke tests on code before it even gets checked in so I don't break the build. I've usually seen the build system be designed to execute and report on the status of all test cases on a regular basis, ranging from daily to weekly depending on the rate of development and time to build and execute the tests.

Answer 10

If you run it across the code base before a check in, it's a regression test.

if you just test the bit you've changed it's a unit test.

When you add an test for a system-level bug, then fix it, that's a regression test.

Its tests if you are regressing: going backwards.

It's worth doing regression tests because bugs tend to cluster in certain places, and some bugs recur.

If you can stand the discipline of having system level tests, adding regression tests there pays off. Any brittle sub-systems end up cocooned in regression tests.

True story.