Hot answers tagged

200

Surely, by the time something gets committed to master, a developer has already run all the unit tests before and fixed any errors that might've occurred with their new code. Or not. There can be many reasons why this can happen: The developer doesn't have the discipline to do that They have forgotten They didn't commit everything and pushed an ...


82

Your coworker is right that everything that can be unit-tested should be unit-tested, and you're right that unit tests will take you only so far and no further, particularly when writing simple wrappers around complex external services. A common way of thinking about testing is as a testing pyramid. It's a concept frequently connected with Agile, and many ...


77

One of my co-workers maintains that integration tests are all kinds of bad and wrong - everything must be unit-tested, That's a little like saying that antibiotics are bad - everything should be cured with vitamins. Unit tests can't catch everything - they only test how a component works in a controlled environment. Integration tests verify that ...


70

As a developer who doesn't run all the integration and unit tests before making a commit to source control, I'll offer up my defense here. I would have to build, test and verify that an application runs correctly on: Microsoft Windows XP and Vista with Visual Studio 2008 compiler. Microsoft Windows 7 with Visual Studio 2010 compiler. Oh, and the MSI ...


60

No, integration tests should not just duplicate the coverage of unit tests. They may duplicate some coverage, but that's not the point. The point of a unit test is to ensure that a specific small bit of functionality works exactly and completely as intended. A unit test for am_i_old_enough would test data with different ages, certainly the ones near the ...


50

I tend to side with your friend because all too often, unit tests are testing the wrong things. Unit tests are not inherently bad. But they often test the implementation details rather than the input/output flow. You end up with completely pointless tests when this happens. My own rule is that a good unit test tells you that you just broke something; a bad ...


37

Abandon them. I know it's hard to let go of something that was clearly a lot of effort to produce, but the tests aren't working for you, they're working against you. A test suite is supposed to give you confidence that the system does what it's supposed to do. If it doesn't do that, it's a liability instead of an asset. It doesn't matter whether the system ...


32

Broad categories to my mind would be: Black box testing. You don't get to see the code and are just testing blindly to some extent as what is in the application or system is hidden from you. Thus in this case people don't know all the error cases and have to guess with various boundary conditions that may or may not be obvious to find all the cases. ...


32

Continuous integration as a term refers to two distinct ideas. The first is a workflow: instead of everyone in a team working on their own branch and then after a couple of weeks of programming try to merge their changes into the mainline, that changes are integrated (nearly) continuously. This allows problems to surface early, and avoids incompatible ...


31

The Rspec Book, among other BDD resources, suggests a cycle like this: In essence, the process is: While behaviour required Write an integration test for a specific behaviour While integration test failing Write a unit test to fulfil partial behavior While unit test failing Write code to make unit test pass ...


30

It's comparing oranges and apples. Integration tests, acceptance tests, unit tests, behaviour tests - they are all tests and they will all help you improve your code but they are also quite different. I'm going to go over each of the different tests in my opinion and hopefully explain why you need a blend of all of them: Integration tests: Simply, test ...


29

Go and fix the tests. Your biggest mistake is that you allowed tests to fail, and you obviously ignored it for a while. What you have is not "legacy tests" - you are working on a legacy code. And I consider every code written without tests to be legacy. Verification of the test failure - if it's problem in product or in test, takes months. We cannot ...


28

TL;DR: As long as it meets your needs, yes. I've been doing Acceptance Test Driven Development (ATDD) development for many years now. It can be very successful. There are a few things to be aware of. Unit tests really do help enforce IOC. Without unit tests the onus is on the developers to make sure they meet the requirements of well written code (in so ...


27

In my opinion you should mock the webservice calls if this is a unit test, as opposed to an integration test. Your unit test should not test whether the external webservice is working, or whether your integration with it is correct. Without getting too dogmatic about TDD, note that a side effect of turning your unit test into an integration test is that it'...


26

When major code changes occur, unit tests tend to be commented out rather than reworked. With an undisciplined bunch of cowboy coders who think all tests getting "green" is fulfilled when you comment all existing tests out: of course. Reworking unit tests takes the same amount of discipline as writing them at the first hand, so what you have to work on ...


24

True, you do not have particular need of a CI system to perform builds and check that those builds are correct, but that is only part of what CI is about. The purpose of CI is to detect errors as soon as possible, because generally speaking, the earlier an error is caught the cheaper it is to fix. To that end, in the case where a build step is not necessary,...


22

You're going to find yourself writing a lot more tests, of much more complicated, interesting, and useful behavior, if you can do so simply. So the option that involves var input = new Parser().ParseStatement("x = 2 + 3 * a"); is quite valid. It does depend on another component. But everything depends on dozens of other components. If you mock something ...


22

Tests are valuable. At the very least, they record that somebody considered that they should spend time writing them, so presumably they had some value to somebody once. With luck, they will contain a complete record of all the features and bugs that the team has ever worked on – though they may also just have been a way to hit some arbitrary test coverage ...


22

You'd think so wouldn't you - but developers are human and they sometimes forget. Also, developers often fail to pull the latest code. Their latest tests might run fine then at the point of check-in, someone else commits a breaking change. Your tests may also rely on a local (unchecked-in) resource. Something that your local unit tests wouldn't pick up. ...


22

Apart from the excellent Oded answer: You test the code from the repository. It may work on your machine with your files... that you forgot to commit. It may depend on a new table that does not have the creation script (In liquibase for example), some configuration data or properties files. You avoid code integration problems. One developer downloads the ...


21

When doing unit tests the "proper" way, i.e. stubbing every public call and return preset values or mocks, I feel like I'm not actually testing anything. I'm literally looking at my code and creating examples based on the flow of logic through my public methods. This sounds like the method you are testing needs several other class instances (which ...


20

What's the decisive advantage of unit testing vs integration testing? That's a false dichotomy. Unit testing and integration testing serve two similar, but different purposes. The purpose of unit testing is to make sure your code does what it is supposed. In practical terms, the unit tests make sure that the code fulfills the contract outlined by the ...


19

For me unit tests should not deal with the database, integration tests deal with the database. Integration tests that deal with the database should in practice have a empty database with a tear up and tear down approach, using a transaction based approach is quite a good way to go (i.e. create a transaction on setup and rollback on tear down). What your ...


19

My first recommendation would be to not mock types you don't own. You mentioned HTable being a real pain to mock - maybe you should wrap it instead in an Adapter that exposes the 20% of HTable's features you need, and mock the wrapper where needed. That being said, let's assume we're talking about types you all own. If your mock-based tests are focused on ...


19

When your dev team and your QA team don't not talk to each other, there is a certain risk that some tests are unnecessarily done twice, and some others are forgotten. One worst case scenario is when your dev team has implemented some nice automatic integration tests, which run in a few minutes or hours, and your QA people tests the same things manually, ...


17

The easiest metric is to ask, "when was the last time this integration test legitimately failed?" If it has been a long time (there have been a lot of changes) since the integration test failed, then the unit tests are probably doing a good enough job. If the integration test has failed recently, then there was a defect that was not caught by the unit tests. ...


17

First, lets talk about what your goals are: you obviously don't want to test "file formats" - you want to to test your different FileReader implementations you want to find as many different types of errors as possible by automatic tests To reach that goal in full, IMHO you have to combine different strategies: first, real unit testing: your ...


16

Integration vs. unit tests You should keep your unit tests and your integration tests completely separated. Your unit tests should test one thing and one thing only and in complete isolation of the rest of your system. A unit is loosely defined but it usually boils down to a method or a function. It makes sense to have tests for each unit so you know ...


16

You've laid out good arguments for and against unit testing. So you have to ask yourself, "Do I see value in the positive arguments that outweigh the costs in the negative ones?" I certainly do: Small-and-fast is a nice aspect of unit testing, although by no means the most important. Locating-bug[s]-easier is extremely valuable. Many studies of ...


16

There's an awesome book that covers what you're doing in great depth. Working Effectively with Legacy Code describes a number of techniques for doing just what you're doing. The table of contents is available here. It had been recommended to me for years, and I only recently picked it up. Basically, yes you're correct. Test for the functionality that the ...



Only top voted, non community-wiki answers of a minimum length are eligible