32

Guava has unit test cases automatically generated:

Guava has staggering numbers of unit tests: as of July 2012, the guava-tests package includes over 286,000 individual test cases. Most of these are automatically generated, not written by hand, but Guava's test coverage is extremely thorough, especially for com.google.common.collect.

How they were generated? What techniques and technologies were used to design and generate them?

5
  • I remember seeing a talk by some google dude that touched this subject. No clue about the name though, the talk was at some java convention I think
    – Zavior
    Nov 1, 2012 at 15:08
  • 3
    package com.google.common.collect.testing has a lot of classes with "Generator" in their names - making it look like a framework for tests generation. There are also sub-packages with classes documented as "skeletons" or "base classes" for tests...
    – gnat
    Nov 1, 2012 at 16:01
  • 1
    @gnat Yes, I was sure I've seen it somewhere. com.google.common.collect.testing.features for instance shows tfeatures/constrains a collection class should satisfy, and a test case is a combination of them. This way they can parametrize testing
    – dzieciou
    Nov 1, 2012 at 16:44
  • You may have more luck on Software Quality Assurance and Testing.SX Nov 2, 2012 at 18:57
  • The question got much attention from the community but no reasonable answer so far, so I've followed Martin's suggestion and put it here: sqa.stackexchange.com/questions/5214/….
    – dzieciou
    Nov 15, 2012 at 8:30

2 Answers 2

8

A large part of this mass of tests is for the Guava collection implementations. They've written generic tests that exhaustively test the collection interfaces, and this generates a suite per implementation. See, for example, classes called CollectionAddAllTester, ListIndexOfTester.

This is all backed by a library called testlib, which ships as part of Guava. This is quite generic. It supports writing generic tests for any interface (not just collections). You can specify Features of possible implementations and test those (e.g. if your set is unmodifiable you expect a different outcome from set.add()), and when you run the tests you specify which features your implementation supports.

It's based on JUnit 3, not 4. Normally, you have a class extending TestCase full of methods named testSomething(), and JUnit runs them reflectively. The testlib library hooks into the running of these tests so that the lifecycle looks like this:

  • For each implementation you want to test
  • For each (applicable) test method
  • Create the TestCase instance
  • Initialise the TestSubjectGenerator – this is the testlib interface that you extend where you actually create the test subject
  • Reflectively run the test method. During this method, getSubjectGenerator() gives access to the test subject

The key bit is the extra initialisation step that allows them to inject a specific test subject into the generic test case.

I wrote a post on how to write testlib generating suites for your own interfaces.

(Also posted to the same question on the sqa site.)

6

There are unit test generators. For example, in the .NET world, something like Microsoft Pex could do this.

For example, Microsoft Pex tries based on code analysis all possible values as arguments for a method. Some arguments are expected to let the method throw an exception. Such things can automatically tests created for. Static values like an empty string that is returned in certain cases can also be automatically be tested.

2
  • 3
    That's a random testing useful only for negative path testing (exceptions, invalid input, crash-es, timeout). I believe they generated tests also for happy path, and this requires more design, not just launching static analysis tool.
    – dzieciou
    Nov 1, 2012 at 13:18
  • And I know there are ways and tools for generating test for happy path (e.g., see this answer), but I'm interested how this was done in particular case of Guava
    – dzieciou
    Nov 1, 2012 at 13:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.