# Is anyone doing “real” TDD with Visual-C++, and if yes, how do they do it?

Test Driven Development implies writing the test before the code and following a certain cycle:

• Write Test
• Check Test (run)
• Write Production Code
• Check Test (run)
• Clean up Production Code
• Check test (run)

As far as I'm concerned, this is only possible if your development solution allows you to very quickly switch between the production and test code, and to execute the test for a certain production code part extremely quickly.

Now, while there exist lots of Unit Testing Frameworks for C++ (I'm using Bost.Test atm.) it does seem that there doesn't really exist any decent (for native C++) Visual Studio (Plugin) solution that makes the TDD cycle bearable regardless of framework used.

"Bearable" means that it's a one-click action to run a test for a certain cpp file without having to manually set up a separate testing project etc. "Bearable" also means that a simple test starts (linking!) and runs very quickly.

So, what tools (plugins) and techniques are out there that make the TDD cycle possible for native C++ development with Visual Studio?

Note: I'm fine with free or "commercial" tools.

Please: No framework recommendations. (Unless the framework has a dedicated Visual Studio plugin and you want to recommend the plugin.)

Edit Note: The answers so far have provided links on how to integrate a Unit Testing framework into Visual Studio. The resources more or less describe how to get the UT framework to compile and get your first Tests running. This is not what this question is about. I'm of the opinion that to really work productively, having the Unit Tests in a manually maintained(!), separate vcproj from your production classes will add so much overhead that TDD "isn't possible". As far as I am aware, you do not add extra "projects" to a Java or C# thing to enable Unit Tests and TDD, and for a good reason. This should be possible with C++ given the right tools, but it seems (this question is about) that there are very little tools for TDD/C++/VS.

Googling around, I've found one tool, VisualAssert, that seems to aim in the right direction. However, afaiks, it doesn't seem to be in widespread use (compared to CppUnit, Boost.Test etc.).

Edit: I would like to add a comment to the context for this question. I think it does a good summary of outlining (part of) the problem: (comment by Billy ONeal)

Visual Studio does not use "build scripts" that are reasonably editable by the user. One project produces one binary. Moreover, Java has the property that Java never builds a complete binary -- the binary you build is just a ZIP of the class files. Therefore it's possible to compile separately then JAR together manually (using e.g. 7z). C++ and C# both actually link their binaries, so generally speaking you can't write a script like that. The closest you can get is to compile everything separately and then do two linkings (one for production, one for testing).

-
As far as I am aware, you do not add extra "projects" to a Java or C# thing to enable Unit Tests and TDD, <-- I don't think this is correct. You typically have multiple projects in C# as well; you don't want to ship your test code in your production binary. –  Billy ONeal Sep 8 '11 at 14:43
@Billy - It's not about having the test code in the actual same project. Do you typically have to maintain the test-project(-file) yourself in C#, or is this handled by the framework? –  Martin Ba Sep 9 '11 at 7:30
I've never seen that handled by the framework. Generation of a binary requires a project. You want two binaries; one with test code and one with production code. You therefore need two projects. There's no way around that. –  Billy ONeal Sep 9 '11 at 18:29
@Robert: Visual Studio does not use "build scripts" that are reasonably editable by the user. One project produces one binary. Moreover, Java has the property that Java never builds a complete binary -- the binary you build is just a ZIP of the class files. Therefore it's possible to compile separately then JAR together manually (using e.g. 7z). C++ and C# both actually link their binaries, so generally speaking you can't write a script like that. The closest you can get is to compile everything separately and then do two linkings (one for production, one for testing). –  Billy ONeal Oct 13 '11 at 14:20
Seriously? "Tip: Each test should contain one main function and generate one executable." This sounds ridiculously slow for any reasonable amount of tests. Even if they mean only one test fixture per executable, its still silly advice IMO. Try doing that with thousands of tests (for a medium sized project) or hundreds of thousands of tests (for a large project) and you'll most definitely go insane. –  legalize Nov 15 '11 at 23:11

Well, don't know if this helps, but there are some great videos about TDD from Brett L. Schuchert. Unfortunately, he does not show the combination "C++" and "VS", but

TDD with C# and VS: http://vimeo.com/album/210446

TDD with C++ and Eclipse: http://vimeo.com/13240481

Perhaps you can work it out from those two.

EDIT: the C++ video is about using the CppUTest testing framwork with Eclipse, of course. When I posted it I thought it should be easily adopted for using in VS. So I googled a little and found this:

http://schuchert.wikispaces.com/tdd.cpp.NotesOnCppUTest

which gives you information how to use CppUTest in Visual Studio.

-
Doc - I've (only) looked at the TDD/Eclipse video but I'm gonna downvote this one. The video is exactly showing what I'm not interested in, namely how to write Unit Test code. The problem (of this question) is not writing Unit Tests, it's how to properly integrate them with your C++ production development and I don't see how these videos help here. –  Martin Ba Sep 5 '11 at 15:19
While I downvoted this answer in the context of this question, I would like to add that the videos are quite nice. I found the Eclipse/TDD/C++ interesting to watch. It just doesn't help here :-) –  Martin Ba Sep 6 '11 at 5:46
@Martin: see my edit. –  Doc Brown Sep 6 '11 at 5:52
Your effort is appreciated and while I do not think that this additional link is really helpful in the context of this question, I think I'll need to do some editing myself. –  Martin Ba Sep 6 '11 at 5:58
@Martin: ok, I read your question again and your definition of "bearable", but don't you expect a little too much? Setting up a unit testing project is not a "one click" solution, but the effort to write the actual unit tests outweighs that by orders of magnitude, whichever framework you are using. –  Doc Brown Sep 6 '11 at 6:06

You don't need a plugin, the test is just another target. There aren't plugins to generate test with c++, evne if you could it would be testing pointless stuff like assignments

-
"even if you could it would be testing pointless stuff like assignments" -- what's that supposed to mean? Do you really think that better IDE support for Unit Tests in C++ is worthless?? –  Martin Ba Sep 6 '11 at 5:49
I mean on a language like c++ the system can't automatically create tests for anything other than obvious statements –  Martin Beckett Sep 6 '11 at 6:00
Why not? What's hindering a plugin to auto-generate a vcproj file on the fly, pulling in he test file file I've written and the referenced production file and trying to run that? (Just dreaming, but it could be made to work.) –  Martin Ba Sep 6 '11 at 6:07
I'm not sure I can follow. obviously I have to write the Tests myself. But running them could be made a lot easier that having to manually setup (and maintain!) separate test-project-files. –  Martin Ba Sep 6 '11 at 6:19
No, I did not refer to the test-code, but rather to the project/compiler scaffolding needed to get the test-code plus "it's" production code to run. –  Martin Ba Sep 7 '11 at 7:31

Can't comment on C++ tools as I've not touched in about 20 years (.NET dev these days) and I guess most of the tools these days are for managed code but as for techniques...

As others have mentioned, test code is always in a completely different project/assembly than production code and yes you usually have to maintain that project yourself, although certainly in the VS IDE this isn't a big problem as you often have several projects as part of your solution anyway.

Production code is and has to be written a little different for TDD. As you write the tests first you end up having to design your code to be testable. This is whole other subject in itself but can take time and seem very frustrating at first especially if your IDE/tools don't give you quick feedback, shelling out to run command-line tools to run tests is just to disruptive.

There are lots of specific techniques for making code testable, but most of them break down in to making small objects that don't do much so you can test them in isolation and being able to inject a test version of some behaviour in to more complex objects. IOC frameworks can help a lot here.

A book you may find useful is; Michael Feathers, Working Effectively with Legacy Code. This uses multiple languages in its examples and may help you identify specific approaches to safely adapt code/techniques that were not originally designed to be testable.

Small caveat: I drunk from the Agile Kool-Aid years ago :D

-
Note: I already have Working Effectively with Legacy Codeon my desk :-) –  Martin Ba Sep 19 '11 at 12:51

Maven is not used widely in C++ (yet, it is mainly used for Java but is language agnostic) but it is a very powerful tool and it allows you to keep everything in the one project (including tests, in fact, that is the recommended approach with Maven). I only suggest it now since from the answers so far it looks like an alternative with a VS plugin may not exist.

Searching for plugins I found:

http://incubator.apache.org/npanday/

but it does not look to be very mature yet. With Maven setup all you would need to do to run the tests is run mvn test in the command line.

If you're interested you can learn about it here and (one of) the C++ supporting plugins here (Maven has a plugin architecture so everything is a plugin).

-

I am not using Visual-C++, but I am performing TDD with C++, using googletest and googlemock, with QtCreator as my IDE. Years ago I had a similar setup with Visual-C++ but using a different unit test framework.

What I have found useful is to separate the project into a few sub-projects.

1. A static or dynamic library that contains 99% of the actual project source code.
2. A project that consists mostly of a main() method to run the normal program.
3. A test project that contains a main() to run my test framework and lots of files containing tests and mock objects.

With this setup, my IDE takes care of adding files to various projects and if dependencies are being correctly determined, I can run all of my unit tests with a partial rebuild. I even have it currently setup to run all of my tests immediately after I build. Jenkins, the CI I'm currently using, also runs and provides test results and coverage data.

It may be possible to add a custom launcher in your IDE for a file to run the unit tests for file Foo.cpp if you happened to name all of the unit tests for Foo under the test fixture TestFoo. How to set this up precisely for Visual-C++ I'm not sure but I do think it's possible.

-
Useful info, although I'd go so far as to call it the "common advice" :-) ... also it ain't really feasible for our legacy stuff, but then adding tests to the legacy stuff is a pain anyway (and I would have liked to tat least make adding the test rigging easy). –  Martin Ba Oct 14 '11 at 7:31

I've written a 5-part blog series on doing TDD with C++ and Visual Studio: part 1, part 2, part 3, part 4, part 5.

I'm not sure why you say that you don't make extra projects in C# to do TDD, because that's what I've always done with NUnit and it seems typical of what other people do with NUnit as well. The reason is simple: always keep test code separate from production code. For C++ with Visual Studio, this means separate projects just like it does for C# and NUnit. From what I know of the Java world, this is also common there.

Obviously, everyone has different ideas of what is "bearable" for doing TDD. I've been practicing the method I outline in my blog post for severeal years and I find it very bearable. C++ is a compiled language and compiling stuff can be slow when the system under test is highly coupled. There just isn't any getting away from that without refactoring to a more loosely coupled design.

My "one click action" is "Build Solution". If that's building too much you can always unload irrelevant projects while you're working and then Build Solution will build only the minimal subset of projects neeeded to be updated as a result of your changes.

Granted, the compile time nature of C++ does make this take a little longer in each cycle of the TDD process than it did with NUnit and C#, but the confidence I get from my well-tested C++ code is worth it. Otherwise I'm going to be spending lots more time in the debugger. I would caution to be sparing in your use of gmock as it can add substantially to test compile time. I've so far mostly gotten away with lightweight "fake" objects and rarely need the full-blown functionality of mocks. The mocking frameworks for C++ are heavily template based and this can increase compile time significantly, so they should be reserved for where you really need a mock and a fake just won't do.

I have considered creating a Boost.Test unit test project wizard to automate some of the boiler plate nature of creating the test project for production code, but after you've done it a couple times its really not that hard to do manually.

As for solutions with many (150?) projects, there are ways to cope with that as well. One obvious one is to find groups of related projects and bundle them together and start consuming them/publishing them as a unit. If you really need to rebuild/touch all 150 projects for small changes you're making during a TDD cycle, then your code is so highly coupled anyway that unit tests aren't likely to make much of a difference.

Looking at the netbeans IDE link, I find the eye candy of having something that parses the test output and shows a little test row in a window with a green or red symbol next to it something that I thought I would miss having come from NUnit, but didn't actually miss. I found it more useful for the build to simply just fail and then I could double-click in the errors window to place the cursor at the location of the failed assertion.

-
"... not sure why you say that you don't make extra projects in C# ... From what I know of the Java world, this is also common there ..." May have been a misconcpetion on my part. (For .NET at least, because you need an executable for .NET -- for java you just need the class files, so I don't quite see where the extra project would fit in.) –  Martin Ba Nov 16 '11 at 7:30
I'm not sure how Java works with projects; I have limited experience there. From what little I know, I understand the projects to be an artifact of the IDE and not the language. (Strictly speaking, this is also true for C#, but I don't know anyone that just uses the command-line compiler for anything other than short blog articles or demonstrations.) However, even in Java you definitely keep the test code separate from the production code, which is what separate projects are doing for you. I never recommend conditionally compiling C++ to separate production and test code. –  legalize Nov 16 '11 at 16:10
"keep the test code separate from the production code, which is what separate projects are doing for you" -- aha! Well, not really, IMHO. The separate "project" is a necessity for C++ and .NET, because both need to create an executable to run anything and to create an (one) executable you need a (one) project. I'm fine with keeping the test code apart (or not) from the production code, but I find having to add a (redundant) "project" to generate the test executable annoying. :-) –  Martin Ba Nov 16 '11 at 16:23
You need two built products: production code (static library, shared library, executable, etc) and a test executable. In Visual Studio, each built product corresponds to a project, so you need two projects. It really isn't any more complicated than that. The unit test project is NOT redundant. –  legalize Nov 16 '11 at 20:12

I use MSTest for testing native C++ code.

Yes, there will be at least two projects - one for application itself, one for tests.
Instead of making third project with static library, I just add application's source to test project, so solution looks like this:

[-] Solution 'Foo'      Foo\Foo.sln
|-[-] Foo              Foo\Foo\Foo.vcproj
|  |-[-] include
|  |  |- foo.h         Foo\Foo\foo.h
|  |  |- bar.h         Foo\Foo\bar.h
|  |
|  |-[-] source
|     |- foo.cpp       Foo\Foo\foo.cpp
|
|-[-] Foo.Tests        Foo\Foo.Tests\Foo.Tests.vcproj
|-[-] include
|  |- FakeBar.h     Foo\Foo.Tests\FakeBar.h
|
|-[-] source
|-[-] app
|  |- foo.cpp    Foo\Foo\foo.cpp    -- not in Foo.Tests\
|
|-[-] unit-tests
|- foo_Tests.cpp   Foo\Foo.Tests\foo_Tests.cpp
|- bar_Tests.cpp   Foo\Foo.Tests\bar_Tests.cpp

-
I find that using the C++/CLI stuff for unit testing just muddies the waters when testing pure native C++ code. I have used NUnit to test C++/CLI application code, however. I wrote my tests in C# and that worked just fine. (The existing code base was C++/CLI and I didn't want to port it to C#.) –  legalize Nov 16 '11 at 18:57
In addition, if your tests are in C++/CLI, then you can't run them on other platforms. Most places where I've used C++ they needed the cross-platform compilation abiility. Of course you can't reuse the VS project on other platforms, but it isn't a big deal to have Makefiles or SConscripts for it. –  legalize Nov 16 '11 at 19:03
@legalize We can't reuse WinAPI (and COM and other Windows-specific technologies) on non-Windows platforms also. –  Abyx Nov 17 '11 at 9:31
Yes, of course you can't use Windows specific technologies on non-Windows platforms. My observation was simply that if you have platform agnostic code then you don't want to tie your unit tests to a particular platform. At a previous employer, we avaluated a large number of unit test frameworks and methods. We chose Boost.Test because it was cross platform and if anything was ever to end up in the C++ standard library regarding unit testing, it would most likely be Boost.Test. –  legalize Nov 17 '11 at 18:16

Framework recommendation: At our office, we use TestDriven.NET which integrates with Visual Studio. The unittest classes are written in C++/CLI, which can then call out to exercise whatever native code you need to test. Yes, the C++/CLI classes go into their own assembly, thus a "testing" project added to the solution(s).

-

Maybe a bit late in the day, but if I read your question correctly, you're looking for techniques to improve the TDD cycle? It's not been mentioned here, but have you looked at post-build events in VS?

Our solutions are typically organised (with Project Dependencies shown)...

MAIN-APP > LIB1, LIB2, UNIT-TEST-APP
UNIT-TEST-LIB1 > LIB1
UNIT-TEST-LIB2 > LIB2
UNIT-TEST-APP > UNIT-TEST-LIB1, UNIT-TEST-LIB2


The post-build event of MAIN-APP will run UNIT-TEST-APP

The post-build event of UNIT-TEST-APP will run itself (just put '\$(TargetPath)' as the command to run in post-build event).

(This does mean that when building MAIN-APP, unit tests can run twice, but that's not really been a problem in our case!)

As mentioned though, yes, there is a bit of effort in setting up this structure, but once it's there, adding tests is simple.

So all you have to do is build the main app and the unit tests will run automatically!

-