Tag Info

New answers tagged

2

Consider the abs function, that returns the absolute value of a number. Here is a test (Python, imagine some test framework): def test_abs_of_neg_number_returns_positive(): assert abs(-3) == 3 This implementation is correct, but it only gets 60% code coverage: def abs(x): if x < 0: return -x else: return x This ...


5

Here's a simpler example to round things off. Consider the following sorting algorithm (in Java): int[] sort(int[] x) { return new int[] { x[0] }; } Now, let's test: sort(new int[] { 0xCAFEBABE }); Now, consider that (A) this particular call to sort returns the correct result, (B) all code paths have been covered by this test. But, obviously, the ...


24

In addition to Mason's answer, there is also another problem: coverage does not tell you what code was tested, it tells you what code was executed. Imagine you have a testsuite with 100% path coverage. Now remove all assertions and run the testsuite again. Voilà, the testsuite still has 100% path coverage, but it tests absolutely nothing.


42

If every path through a program is tested, does that guarantee finding all bugs? No If not, why not? How could you go through every possible combination of program flow and not find the problem if one exists? Because even if you test all possible paths, you still haven't tested them with all possible values or all possible combinations of values. ...


5

There are many cases where you write a test that is not simply the answer to a question. Compare 'does the controller send a notification mail' with 'controller should send notification mail'. Both are valid, but it is much easier to read a list of tests formulated as 'should' rather than formulated as questions. Another major point in TDD is that the test ...


3

BDD style tests (as might be the case with the framework you're using) follow the Given-When-Then convention, not Question-Answer. Depending on the approach prescribed by your testing library, shoehorning questions in there might feel awkward. However, questions could fit well in more classical test schemes where the test description is entirely contained ...


3

Ideally, you want both unit tests (which test individual functions) and integration tests (which test all the functions together), not just one or the other. You will also need a way to mock out the web service you're calling so that these tests don't depend on it. To directly address a few of the sub-questions: So is correct to create a big test that ...


0

Well, dependency injection comes to mind as a default sort of boxed approach to config testing. However, there are other ways, specifically designing your code to eliminate the dependency on config file all together. You can find more details in a canonical example of dependency elimination by Brian Geihsler that talks specifically about config files.


3

You are by far not the first person to have this need; many others have had it before you, so there has been quite a bit of research on the subject, and a solution in almost every decent language out there. For a theoretical discussion, you might want to look at Programming by Contract, preconditions, postconditions and invariants. It includes a list of ...


1

Some languages (Including Delphi where I used it for this very purpose) have the concept of setting variables that can be checked at compile time so you can use an "IFDEF debug" directive to surround your scaffolding code and define / undefine debug appropriately. IIRC you would set the variables in a dialog at the program level. If your language is a ...


0

Caching logic is something that should be unit tested by the developer as QA mainly does black-box testing. QA would only care about the performance aspects or whatever fix you implemented thus, you can provide QA with a mechanism to enable/disable caching or whatever mechanism you used to improve performance, and then they can verify the performance ...


0

Some things are better tested by a programmer, perhaps the one who wrote the code, using unit tests. Testing the correctness of your caching code is one of those things. (From the way you ask this question, I presume that your QA people treat the application as a "black box" and test it through its external interface.)


2

Remember to get the testers to reboot the servers and check that the data they have entered is still there. I saw a system that had many man months of testing, fail when that was done! The hardest part to test is that however the data is updated, there is never any out-of-date results returned from the cache. This can be partly helped by always ...


0

Depends on how you load data from your config file. Ideally, write your program to be easily testable and reduce unnecessary dependencies. So make sure that if you change the location or format of your config file, that this affects as little code as possible. Then make sure that code can also deal with the testing. If you have a single object or class or ...


3

You have to write a test that works independent of the config file, so you can test that depending on the "simulated configuration" the output of the function or behavior of that function is correct. You would need to inject the configuration file, or the value that you are trying to simulate on your function under test. This is the only way to guarantee ...


3

Burying important business logic or system state deep in a black box makes verifying correct system behavior difficult. It's easier to exhaustively test the behavior of a single component in the system than the entire system. I favor exposing such things explicitly through some mechanism so it can be unit / regression / integration / QA tested in some ...


3

Test performance, as indicated in Andy's answer. I have found that the biggest obstacle to implementing caching (and performance) at many organizations is actually having an environment where you can do good performance testing and run tests for various real world load and performance tests. To have this you should set up a performance testing environment ...


64

You implemented a cache (I assume) because the system wasn't performing well enough. That is something that is relevant for the user. Here are things that QA can check: That the system, after the cache was introduced, is still correct. This means they should attempt to trick the cache into providing stale data, which is something QA can verify, and make ...


0

I do not like system level tests written in java or C# for that reason. Look at SpecFlow for c# or one of the Cucumber based test framework for java (maybe JBehave). Then your tests can look more like this. And you can change your object design without having to change all your system tests. (“normal” unit tests are great when testing single ...


33

One question is whether the cache itself is really a requirement that should be tested by QA. Caching improves performance, so they could test the difference in performance to ensure it meets some requirement. But good idea to have some testing around caching, whoever is responsible for it. We used performance counters. If your cache system takes advantage ...


9

This felt odd as I was forcing the design of the implementation based on how I had decided at this early stage to write this first test. I think this is the key point in your question, Whether or not this is desirable depends on whether you lean towards codeninja's idea that you should design up-front then use TDD to fill in the implementation, or ...


0

Before you start writing your tests, you should think about how to design your system. You should spend some considerable amount of time during your designing phase. If you did it, you won't get this confusion over TDD. TDD is just a development approach link: TDD 1. Add a test 2. Run all tests and see if the new one fails 3. Write some code 4. Run tests 5. ...


16

In order to write the test in the first place, you have to design the API that you are then going to implement. You've already started on the wrong foot by writing your test to create the entire GameOfLife object and using that to implement your test. From Practical Unit Testing with JUnit and Mockito: At first you might feel awkward for writing ...


0

Even when I implement something in a "hack it together" way, I still think up the classes and steps that will be involved in the whole program. So you've thought this through and written these design thoughts down as a test first - that's great! Now keep iterating through both implementation to fulfill this initial test, and then add more tests to improve ...


0

There different school of thought about this. Some say: test not compiling is error - go fix write smallest available production code. Some Say: it is OK to write test first check if it sucks(or not) ant then create missing classes/methods With first approach you are really in a red-green-refactor cycle. With second you have a little bit wider overview ...


2

If you can think of a better name than call_method_a_and_b(), a name that is a single concept, then it isn't a violation of the 'one thing' principle. If you are calling a and b in that order for different reasons in foo than in bar, so one of the two call sites might need to change while the other stays the same, then you aren't violating DRY. (For ...


3

This is what setUp is for. It is executed automatically before running each test, and can be used to initialize class fields and properties or set up the environment (in a case of integration and system tests). As for the one thing principle, it is perfectly fine to have a method A which calls methods B and C if the method A is acting on a different level ...


1

Cyclomatic complexity when applied to testing places two bounds on the test cases needed. From Wikipedia Cyclomatic complexity: Implications for software testing M is an upper bound for the number of test cases that are necessary to achieve a complete branch coverage. M is a lower bound for the number of paths through the control flow graph (CFG). ...


0

If you do not own the REST API's, your responsibility ends with them You need tests that check if your application calls the API under your control correctly. Therefore, use the actual application with the actual service (the one you control) but calling mocked services (the ones you don't control). So... Use canned data for the services you don't ...


4

From the definition, "off points" can be inside or outside the domain, both cases match that definition. But the text says "how should you choose test points" (which means - to get a good test coverage with as few cases as possible). So when you have chosen an "on point" which is inside the domain in case of a closed boundary, it makes just more sense to ...


3

They are giving rules about how to chose test points. you have to have in test points and out one (or you'll have tested only one side of it). you have to have on test points (to be sure to test the limit) and off one. For closed borders, the on test points are in, thus there must be an off one which is out. For open borders, the on test points are ...


3

For closed domain, both OFF and ON points are outside domain, while for open domain, the OFF point is in inside the domain. I just do not get why is that. This is incorrect. For both open and closed domain, one point is inside and one point is outside the domain. The difference is which point is inside (and which outside). To test a domain boundary, ...


3

Your testers do not need to think like a developer - for black-box testing they need to think like a user of your software, and they should have additional knowledge about how to test software in general. So assumed your testers are smart enough to learn those things in a reasonable amount of time, train testers the same way you train your users, and make ...


5

If your test team does not have the necessary background knowledge, they are not going to be able to test your product adequately. (This is a special case of a more general problem: If a team member does not have the necessary background knowledge, he is not going to be able to do his job adequately.) You're going to have to figure out how to explain to ...


4

Your developers are going to have to write very detailed and explicit testing instructions until the testers learn the domain and the app to the point where they can write their own. Make testing training videos from screen recordings just like they would for user training. You may find some interesting bugs getting discovered because your testers don't ...


0

Generally, you don't need to mock value objects (and your container objects seems like value objects) and your approach is right. If you need to write assertions to the properties, I would consider it a code smell. Still, PHP Unit allows you to mock magic methods, so you can assert the __set calls for example, or you can just assert the end value of the ...


0

I could also use a template to generate derived classes via tagging rather than actually generating a new class each time. Something like class BaseException { ... }; template<typename T> struct DerivedException : BaseException { using BaseException::BaseException; }; struct CyclicReference {}; Then I could employ throw ...


0

In C#, the way this is done is by adding metadata to your object's members in the form of property attributes, each of which is a class. There aren't that many of them; they embody categories of errors, and some of them take parameters. For example, to validate a class property as a number between 0 and 50, you might decorate the property with an attribute ...


1

Aside the type, you can usually specify a textual description. For instance, in a method SetPercentage(value) a error OutOfRangeError thrown when the value is inferior to zero, but also superior to one hundred. Same type—two different errors. The text of the error can then specify what went wrong specifically. For instance: if (value < 0) { raise ...


1

I believe the standard approach is to use an error code (of enum type) and an error message (of string type). The "code" makes it easy to programmatically identify a specific type of error (if(e.code === DIVIDE_BY_ZERO)), and the "message" makes it easy to show something meaningful to the user (std::cout << e.message). A new derived exception class ...



Top 50 recent answers are included