Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

According to rules of TDD unit tests are written before production code, but what about Integration tests that exercises interaction between concrete (non mocks) wired objects?

Should they be written before unit tests or after production code just to test the "wiring" ?

Note that I'm not talking about Acceptance or functional tests but Lower level integration tests.

share|improve this question
add comment

7 Answers

up vote 24 down vote accepted

The Rspec Book, among other BDD resources, suggests a cycle like this:

enter image description here

In essence, the process is:

While behaviour required
    Write an integration test for a specific behaviour
    While integration test failing
        Write a unit test to fulfil partial behavior
        While unit test failing
            Write code to make unit test pass
        Commit
        While refactoring can be done
            Refactor
            While unit test failing
                Write code to make unit test pass
            Commit
    Push

Disclaimer: There's no doubt in my mind that this leads to the best code and product, but it can be time-consuming. There are all sorts of difficulties around data and determinism, when it comes to saying that integration tests should always pass. It's not appropriate in all circumstances; sometimes you just have to get stuff out of the door.

That said, having an ideal process in mind is great. It gives you a point from which to compromise.

share|improve this answer
2  
+1 for a picture. –  Mathew Foscarini Aug 16 '13 at 9:52
2  
Thanks @pdr but I specified that I was not talking about Acceptance tests which are written before/at the beginning of an iteration, I'm interested in lower level integration tests. –  Chedy2149 Aug 16 '13 at 10:11
    
@chedy2149: Akkh. Missed that comment. Before I remove my answer, I think you should be more specific about what you mean by "lower-level" in the context of integration tests. –  pdr Aug 16 '13 at 10:18
    
Lower-level : which behavior are not specified by users or clients and which are used to test classes/components interactions has expected by developers. –  Chedy2149 Aug 16 '13 at 10:33
1  
Actually, I don't think it makes a difference if you put "acceptance test" or "integration test" in that picture - it is an idealized view for any kind of tests on different abstraction levels. But IMHO the real problem in is not that this may be "time consuming" - the real problem is that writing integration tests "beforehand" against a public interface which is still beeing designed with the help of TDD is like shooting at a "moving target". –  Doc Brown Aug 16 '13 at 11:03
show 2 more comments

Real project showed to me that it's not possible to write unit tests and than integration and even opposite direction is wrong :-) So, I usualy write unit tests together with integration ones.

Why? Let me to write how I see both kinds of tests:

  1. Unit tests - In addition to wikipedia and all know information, unit tests help you to narrow your design, improve your model, relations. The flow is simple: once you start to type new project/new component, most of time you are making some kind of PoC. When you are done, you always have long methods, long classes, non-coherent methods and classes, etc.

    Unit tests help you to remove these issues as when you do real unit testing using mocks (w/o dependency on other component) classes described above are un-testable. Basic sign of untestable code is large mocking part of tests because you are forced to mock many dependencies (or situations)

  2. Integration tests - correct and working tests say to you that your new component (or components) work together or with other components - this is usual definition. I've found that integration tests mostly help you to define flow how to use your component from consumer side.

    This is really important as it sometimes says to you that your API does not make sense from outside.

Well, what happen once I wrote unit tests and integration tests later?

I got nice classes, clear design, good constructor, short and coherent methods, IoC ready etc. Once I give my class/API to some consumer, e.g. developer from integration or GUI team, he was unable to use my API as it seems unlogic, weird. He was just confused. So I repaired API according to his point of view but it also required to rewrite many tests because I was pushed to change methods and sometimes even the flow how to use the API.

Well, what happen once I wrote integration tests and unit tests later?

I got exact flow, good usability. What I also have are large classes, non-coherent code, no logging, long methods. Spaghetti code

What's my advice?

I've learned following flow:

  1. Develop basic skeleton of your code
  2. Write integration tests which say whether it make sense from consumer point of view. Basic use-case is enough for now. The test obviously does not work.
  3. Write code along with unit tests for each class.
  4. Write the rest/missing of integration tests. It would be better to implement these test within #3 how you are improving your code.

Note that I've made small presentation about unit/integration testing, see slide #21 where the skeleton is described.

share|improve this answer
add comment

Unit Tests are used to test the smallest possible testable bit of software in an application and to test it's functionality. Each unit is tested separately before amalgamating them into parts or larger components of the application.

That's where Integration Tests come in:
They test these newly created parts that consist of the previously tested units during fitting these parts together. Best case would be to write the tests at this point while writing the application itself.

share|improve this answer
    
So your answer is after production code? –  Chedy2149 Aug 16 '13 at 9:23
    
This doesn't answer the question. He's asking if production code is written after integration tests are written. Your answer can be taken either way. –  Mathew Foscarini Aug 16 '13 at 9:50
1  
@MathewFoscarini - updated answer. Hope it becomes clearer now. –  Ben McDougall Aug 16 '13 at 10:13
    
With regard to unit tests, I would take issue with "smallest possible bit of software". Test what is in the contract (e.g. an object's public methods, a library's exported functions) because the contract defines what has to work. Other things are testable, yetdoing so is not only a waste of time but counterproductive. –  itsbruce Aug 19 '13 at 13:13
add comment

Aside from acceptance tests, I tend to only write integration tests at the boundaries of an application, to verify that it integrates well with third-party systems or components.

The idea is to create adapter objects that translate from how the third party speaks to what your application needs, and test these translators against the real external system. Whether you do that test-first or test-last is I think less important than with your regular unit tests because

  • The design insight provided by TDD doesn't matter as much here since design is pretty much known in advance and there's typically nothing terribly complex involved, you just map things from one system to another.

  • Depending on the module/system you want to tackle, it can require a lot of exploration, configuration tinkering, sample data preparation, which takes time and doesn't fit really well in a short TDD feedback loop.

However, if you really feel more comfortable building your adapter incrementally in little secure steps, I'd definitely recommend going test-first, it can't hurt.

You can find examples of this approach here : http://davesquared.net/2011/04/dont-mock-types-you-dont-own.html (6th paragraph) http://blog.8thlight.com/eric-smith/2011/10/27/thats-not-yours.html

share|improve this answer
    
here you are talking about integration tests that verify interactions between "our system" and 3rd party libraries, how about testing interactions against a platform while developing a plug-in for example? –  Chedy2149 Aug 16 '13 at 11:32
    
Though I have little experience with plugin development, I guess they might be different since they are by nature tightly coupled with the host application, so you may want to fully embrace that integration and decide you don't need an adapter layer. In that case I'd be really careful about test performance though - depending on the host application, calling its API directly in a large number of your tests can be very slow. If you're afraid of that, you can always resort to the "additional layer of abstraction" approach and use mocks + integration tests on adapters. –  guillaume31 Aug 16 '13 at 11:51
add comment

I tend to view integration tests as very similar to unit tests. In that I am treating a subset of the code as a black box. So integration tests are just a bigger box.

I prefer to write them before the production code. This has the advantage of helping me remember which pieces I haven't wired up yet or that I changed a detail in the objects interaction slightly.

share|improve this answer
add comment

So I was going to accept the first answer but it has been deleted.
To sum it up
In a given iteration :

  1. Write unit-test
  2. Write production code
  3. Write Integration tests to test interactions

Keep in mind integration testing while 1 and 2 to guarantee testability at the integration level.

Integration tests are not necessarily written end to end at step 3 they may be partially written between step 1 and 2.

share|improve this answer
3  
This summing up completely ignores the iterative nature of the process. Once you have the API of your production code stable to some degree, you can start write integration tests.Then you can work on your production code again, and probably change or extend your integration tests. So in most cases you don't "write integration tests after production code", you typically do both to some degree in parallel. And actually, this depends also heavily on what kind of software you are working. "Black-and-white" thinking does not bring you further. –  Doc Brown Aug 16 '13 at 12:36
    
Good point.The answer seem to ignore the iterative nature of design through refactoring. –  Chedy2149 Aug 16 '13 at 13:46
add comment

Unit tests test discrete blocks of code within your project.
Integration tests test how your code interfaces with other code: in other words, they test the interface of your code.

Write unit tests when developing code behind an interface.
Write integration tests when developing the interface or any code that implements the interface.

This means that you will sometimes write integration tests very late in a project, because the majority of the work is behind the interface: for example, a compiler, a particular webservice that implements several layers of logic or .. something that involves a lot of internal logic.

However, if you're implementing a set of REST services or refactoring the data model and adding support for XA transactions, then you're going to start developing integration tests almost immediately, because most of your work is centered around the interface, whether it's the REST API or how the program uses the data model.

share|improve this answer
    
Would you agree to say that unit tests are whitebox testing and integration tests are blackbox testing? –  Chedy2149 Aug 22 '13 at 8:06
    
Unfortunately, that depends. Technologies for integration testing have made enormous improvements in recent years (at least in the Java world) so that I can test 1 class -- but on an application server. The question then becomes, where's the line between unit tests and integration tests? Is an integration test when you test your code interfacing with other technologies, or is an integration test when you test your whole application -- but not necessarily in the environment that it's meant to run in? –  Marco Aug 22 '13 at 8:43
    
In short, in some cases, sure integration tests are black box testing -- but not in all cases. –  Marco Aug 22 '13 at 8:44
    
FYI: Wiki defines integraion testing as "the phase in software testing in which individual software modules are combined and tested as a group" –  Marco Aug 22 '13 at 8:55
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.