Hot answers tagged

190

Surely, by the time something gets committed to master, a developer has already run all the unit tests before and fixed any errors that might've occurred with their new code. Or not. There can be many reasons why this can happen: The developer doesn't have the discipline to do that They have forgotten They didn't commit everything and pushed an ...


68

As a developer who doesn't run all the integration and unit tests before making a commit to source control, I'll offer up my defense here. I would have to build, test and verify that an application runs correctly on: Microsoft Windows XP and Vista with Visual Studio 2008 compiler. Microsoft Windows 7 with Visual Studio 2010 compiler. Oh, and the MSI ...


22

Apart from the excellent Oded answer: You test the code from the repository. It may work on your machine with your files... that you forgot to commit. It may depend on a new table that does not have the creation script (In liquibase for example), some configuration data or properties files. You avoid code integration problems. One developer downloads the ...


22

You'd think so wouldn't you - but developers are human and they sometimes forget. Also, developers often fail to pull the latest code. Their latest tests might run fine then at the point of check-in, someone else commits a breaking change. Your tests may also rely on a local (unchecked-in) resource. Something that your local unit tests wouldn't pick up. ...


14

by the time something gets committed to master I usually set up my CI to run on every single commit. Branches don't get merged into master until the branch has been tested. If you're relying on running tests on master, then that opens a window for the build to be broken. Running the tests on a CI machine is about reproducible results. Because the CI ...


8

If company A develops a individual software component for company B, part of the contract is typically that B defines or develops a list of acceptance tests, and B will only pay A if the test do not show any issues. That is perfectly fine. However, if some of the tests fail, B must report them to A, and A gets a chance to add corrections. And since every ...


7

Wow! They're contracting you to do their testing for them. The best solution is to implement an SLA that gets you out of this relationship. Your contract should penalize them financially for bugs you discover that impede integration of their product. I really don't have anything to add on what is "best practice" when working within some relationship where ...


7

The point that you should be aiming for with such tests is that as many of them as possible should be interacting with a mock of the database, rather than the database itself. The standard way to achieve this is to inject a DB access layer into the logic you are testing here, using interfaces. That way, the test code can create in-memory data sets prior to ...


6

Besides the fact that this is an integration test as opposed to a unit test, the operations you describe typically go in Setup and/or Teardown methods. Frameworks like nUnit allow one to decorate class methods with these attributes to indicate whether the method is a setup method or teardown method. Then your tests should become cleaner and smaller as the ...


6

The basic workflow for TDD is commonly known as "Red; Green; Refactor": Red: Write a failing test Green: Modify the code to make that test pass (without any existing tests failing) Refactor: Tidy up the code to better incorporate the change. There are numerous resources that explain this process in details, eg The Cycles of TDD It's unclear to me as to ...


5

This is not so much an indicator of bad practices as a strict test of your deployment process. Obviously it is very bad if debugging controls, content, backdoors etc. get into production environments. But if they help you achieve the same business goal faster, they save the company real money. Therefore the crucial factor is to ensure that they cannot get ...


5

TDD revolves mostly around unit testing, this answer is going to cover that. Why do you make applications? Do you make them to see see that C# works well, or do you make them to solve a problem presented by your client (be aware, sometimes the client may as well be you)? Unless you are a .NET developer working for the Microsoft company and are actually ...


4

When you change the behaviour of a component in your framework, then just unit-testing that component is not sufficient. You must also re-run the integration- and higher-level tests that involve that component to verify what effect your changes have on the rest of the system. If you find that the effect of your change is very large, the either you should ...


4

The big problem with databases and (unit-)tests is that databases are so darn good at persisting stuff. The usual solution is to not use an actual database in your unit-tests, but instead mock the database or use an in-memory database that can easily be wiped completely in-between tests. Only when testing the code that directly interacts with the database, ...


4

By the time something gets committed to master, a developer should have already run all the unit tests ... but what if they haven't? If you don't run the unit tests on the CI server, you'll not know until someone else pulls the changes to their machine and discovers the tests just broke on them. In addition, the developer may have made a mistake and ...


3

The ideal way to test this without creating any load on the target site would be to create a duplicate of that target web site on your own web server. The duplicate should be a simulation which behaves as similar as possible as the target web site itself. However, if the target site is a complex third party site, this may be impractical or nearly impossible. ...


3

You will normally keep your tests in a separate folder hierarchy as they usually aren't part of the binary you ship, and it's just simpler to not mix them up, but you keep them in the same repo and branch because they belong together in source. As for where you work, TDD means Test Driven Development, which means you first write a (small) test for the ...


3

Unit tests are supposed to be for external functions only. Nevertheless, if you have a lot of "internal code", it should be tested too right? So how? Make smaller functions, separated from you code (another .c and .h), and use those functions as a library that can be unit tested. Then you should be able to test those functions that are used by the internal ...


3

Assuming (contrary to other answers) that developers are quite disciplined and do run unit tests before committing, there can be several reasons : running unit tests can take long for some special set up. For example, running unit tests with memory checker (like valgrind) can take much longer. Although all unit tests are passing, memory check can fail. the ...


3

It is possible to imagine cases when the change A does not break the test, and change B does not break the test, but A and B together do. If A and B are made by different developers, only CI server will detect the new bug. A and B may even be two parts of the same longer sentence. Imagine a train driven by the two locomotives A and B. Maybe one is more than ...


2

Working on a C# Server with SQL Server and PetaPoco, this is the approach we took to clean-up data in Unit Tests. A typical unit test would have Setup and Teardown as follows: [TestFixture] internal class PlatformDataObjectTests { private IDatabaseConfiguration _dbConfig; private Database _pocoDatabase; private PlatformDataObject _platformDto; ...


2

Let's ask an equivalent question: Why would you build the code on a CI server? Surely, by the time something gets committed to master, a developer has already built the code before and fixed any errors that might've occurred with their new code. Isn't that the point of building code? Otherwise they've just committed broken code. The are ...


2

Quis custodiet ipsos custodes? Your test framework had bugs in it, which means that your tests were buggy. This is the problem with complex tests and complex testing systems. One way to deal with this is to create unit tests for the complex parts, and then confirm that they behave correctly. In other words, it's just software, and if it isn't dead ...


2

The tests aren't there to ensure you write the code you want to write. They are there to ensure that three years from now, you don't accidentally change the way the code works through a seemingly unrelated change, causing unintelligible defects. Tests are insurance against the future. Never remove the tests.


2

I don't know about other organizations, but in mine they love waterfall development. The testers make the test scripts during the development. And then during the test phase the developers spend almost all their time helping the testers figure out whether a test worked and if not why it failed. If a test fails there is no way the testers can figure out: ...


2

Have you considered putting the functions into the module's header file, but putting them in a conditionally compiled block that is only enabled in build configurations that are tested? It would be analogous to having a C++ class with public and private functions. However, that's not a good option if you're shipping the header file to users, and want to ...


2

Debug code has been included in programs for practically as long as they have been written. The elephant in the room however is that your testing cycles are not against the version that will eventually ship . While finding some issues early might outweigh the cost of finding hidden bugs later, it is certainly a risk to consider. You also need to be careful ...


1

I have always followed the maxim that a unit test tests one thing and only one thing. Programming such that each method does one thing and one thing only, should mean you can write a unit test for that method fairly easily. If your method calls out to any other services or classes, then you should mock those so their behavior during the test is completely ...


1

Testing terminology is very messy. Usually I've seen this type of tests to be called "integration tests", but to me it seems to be too overloaded term used also for quite different type of tests. So I call them component tests. Your microservice is one component of the larger system. Term is quite self-explanatory and fits well definition from Martin ...


1

At what point in that process should I really begin testing? Well, you asked specifically about tdd, so there's really only one correct answer: Before you do anything else. Test-Driven Development and Design is called Test-Driven, because the Tests drive the development and design. The tests tell you when to start writing code. They tell you what to ...



Only top voted, non community-wiki answers of a minimum length are eligible