Tag Info

New answers tagged

1

What strategy would one adopt to test such a form? You have to separate the GUI from the decision logic by adding a decision function that only depends on business attributes and not on GUI elements: bool isNativeLanguageInputVisible(RegistrationData currentRegistrationData) Your GUI would use this decision function to control visibility and your ...


1

I don't know about "in terms of reliability testing", but the meanings of those to terms is as follows: Calendar time - How long whatever you are testing took in actual time. In other words, if you start a stopwatch when you start a program and check it when the program is done, you are measuring calendar time. This is equivalent to the "real" time ...


0

IMHO TDD is primarily a whitebox technique, and, I think it is more an implementation technique than a testing technique. It is applied by the developer who writes the code in very small iterations "write test, write code, refactor", so by someone who knows exactly the already implemented parts, and the missing parts of the "subject under test". Whenever ...


0

Automated User Interface Testing solves the ability to: rapidly repeat testing of a large number of components remember to test a large number of functions each time compare runs and run times of test suites as the application grows set up runs with hundreds of different inputs and variable conditions enable people who did not write the test to run it and ...


2

Yes selenium can be used to automate the manual interaction steps. It's intended to be used against web browsers and it's widely used and implemented. However this is only half of the puzzle. Selenium is the core technology in browser automation tools, APIs and frameworks. To actually USE selenium you are going to need to pick an actual implementation. ...


1

Beyond mocking the network interface, you don't need two physical machines for a system test: Disable/reenable the network card using scripts that do ifconfig/ipconfig. use a virtual machine for one end of the system and script it up/down or enable/disable the networking. Hooking into WinSock is way more work than you need to do here.


1

You test these by mocking the client and server and using IP Address 127.0.0.1 Depending on the test you are performing, it could be a simple matter of having the mock-server simply closing its end of the connection after some predefined time and then verifying that the client reconnected within the specified time-frame. It may be difficult to test some of ...


2

First, I think this is a perfectly good question. It's an interesting area and i don't get why was this downvoted. You don't need winsock you can disable the network a adapter and change it's settings programmatically... but network issues could be very tricky to simulate this way... ...


4

These two types of testing are not directly related. A smoke test is a quick system test with the purpose of finding major flaws in a software artifact. For example, a test might deploy a web app to a test server, validate that it deploys and starts up, and the server can service simple requests. This would validate that the application file (e.g. WAR file ...


0

The term "smoke testing" can be thought of as a sanity check. These are quick tests to get a quick, overall view of whether or not the system seems to be working. It isn't careful, methodical testing. For example, smoke testing might catch things like obvious freezes or obvious integration problems. Black box testing is a completely different category. ...


1

One thing nobody else has mentioned yet: mutation testing. This is where an automated tool takes your source code and deliberately inserts bugs into it. (E.g., delete a randomly-chosen statement, change an AND to an OR, or whatever.) It then runs your full test suite, and checks whether the tests pass. If all tests pass, then there are two possibilities: ...


4

Basically, in smoke testing you turn on the system, especially for the first time ever, and see whether it starts to emit smoke... which would be a sign that something is very wrong with it. It doesn't tell you anything beyond that, but it can still be valuable to avoid further effort on something that is obviously not going to work at all. According to ...


0

Let's take it to the extreme, and assume that you found the bug much earlier: in your code, as you were writing it. Then you wouldn't have any qualms about fixing it right there -- you see a logic flaw in the code you just wrote, it doesn't do what you wanted it to do. You wouldn't feel a need to setup a whole environment to show that it's actually a bug. ...


-1

Sounds to me like you need more detailed logging. While adding more logging cannot guarantee that you won't need to debug (or, in this case, reproduce the situation), it will give you a far better insight into what actually went wrong. Especially in complicated/threading situations, or anything where you can't use a debugger, falling back on "debug by ...


0

Manual testing should be replaced with automated acceptance testing. Acceptance tests should be runnable just like other tests are, so no additional "ask that tester guy to make his click-click thing again" step is required. The problem you describe is not related to branching strategy at all, since you have to manually retest every feature after adding any ...


2

Multiple VMs is the way to go, but you should not really have to "login to each VM to run the test suite": you should be able to automate that part. You can set up all VMs to look at a common shared folder of the host machine, and run a process on each VM which checks the contents of that folder, say, once a second. When a new version of the app is ...


0

The idea behind test-driven development is that your tests are written in the form of scripts, running through the flow that a user or client package would take if they made use of the software you are about to write. Obviously at first your test classes will be full of errors which complain that the classes and methods don't exist (or they do exist but ...


1

This is the driving motivation behind Test-driven development, with the principles being thus: You do not write code without tests. You write just enough code for the test to pass. You refactor the code that you have written and ensure that tests pass. I think I understand what you're getting at - you don't know if you can/should write tests for objects ...


1

This is the kind of thing you definitely need to integration test, as real-world file systems have all kind of strange behavior (like the way Windows won't allow deleting a file if any process, including the deleter, has it open). So the TDD approach is to write the integration test first (TDD, strictly speaking, doesn't have distinct concepts of 'unit ...


2

In general, your tests should normally not reimplement the code under test. This can happen to you with traditional tests as well. In fact, from my experience, property-based testing tends to be better for avoiding re-implementations. When you think about the properties, you specifically do not think about implementation details (i.e. do not think about how ...


0

In the end, I thought of more properties and this became quite a useful test. The squared distance between any point and itself is 0.0 The distance between any two distinct points is > 0.0, measured from either point For any number n, the distance between Point(0,0) and Point(n,n) is n*n*2 (this one normalises one point to see the effect on another) Here ...


0

Short answer: test against hard-coded expected outputs. Long answer: You should be able to tell what is the expected output from a given input, by following the logic in your head or on paper, before implementing. Take the inputs and the outputs and turn them into tests, one by one, coding the algorithm little by little. However, like in your example ...


-1

White-box and black-box testing are kinds of automated tests, that can be run by a computer and produce a pass or fail result. White-box means tests that require you to "see inside the box" and know how it works (I believe regression tests and boundary tests often count), while black-box means you are testing the external interface without knowing any of the ...


1

Black-box testing means that the tester is not familiar with the inner workings of the system, while white-box testing is performed by someone who knows exactly how the program works. Both has advantages and disadvantages. Functional testing is testing whether or not the software product fulfills functional requirements, like "when loading an incorrectly ...


1

Black-box testing as opposed to white-box testing is a broad category of testing that includes tests where you do not know, and do not need to know, how the implementation works, you just want to test the outcome. For example: I click the buy button on a web site and the item I selected is now in my shopping cart. I don't care how the programmer made this ...


0

Well... yes actually, if every path “through” the program is tested. But that means, every possible path through the entire space of all possible states the program can have, including all variables. Even for a very simple statically compiled program – say, an old Fortran number cruncher – that's not feasible, though it can at least ...


2

What does it mean for every path to be tested? The other answers are great, but I just want to add that the condition "every path through a program is tested" is itself vague. Consider this method: def add(num1, num2) foo = "bar" # useless statement $global += 1 # side effect num1 + num2 # actual work end If you write a test that asserts add(1, ...


6

It's clear from the other answers that 100% code coverage in tests does not mean 100% code correctness, or even that all bugs that could be found by testing, will be found (never mind bugs that no test could catch). Another way of answering this question is one from practice: There are, in the real world, and indeed on your own computer, many pieces of ...


3

If every path through a program is tested, does that guarantee finding all bugs? As already said, the answer is NO. If not, why not? Besides what is being said, there are bugs appearing at different levels, which can't be tested with unit tests. Just to mention few : bugs caught with integration tests (unit tests shouldn't use real resources ...


4

Path coverage cannot tell you whether all the required features have been implemented. Leaving out a feature is a bug, but path coverage will not detect it.


4

Part of the issue is that 100% coverage only guarantees that the code will function correctly after a single execution. Some bugs like memory leaks may not be apparent or cause issue after a single execution, but over time will cause problems for the application. For example, say you have an application which connects to a database. Perhaps in one method ...


7

Yet another addition to Mason's answer, a program's behavior may depend on the runtime environment. The following code contains a Use-After-Free: int main(void) { int* a = malloc(sizeof(a)); int* b = a; *a = 0; free(a); *b = 12; /* UAF */ return 0; } This code is Undefined Behavior, depending on the configuration ...


11

Consider the abs function, that returns the absolute value of a number. Here is a test (Python, imagine some test framework): def test_abs_of_neg_number_returns_positive(): assert abs(-3) == 3 This implementation is correct, but it only gets 60% code coverage: def abs(x): if x < 0: return -x else: return x This ...


32

Here's a simpler example to round things off. Consider the following sorting algorithm (in Java): int[] sort(int[] x) { return new int[] { x[0] }; } Now, let's test: sort(new int[] { 0xCAFEBABE }); Now, consider that (A) this particular call to sort returns the correct result, (B) all code paths have been covered by this test. But, obviously, the ...


63

In addition to Mason's answer, there is also another problem: coverage does not tell you what code was tested, it tells you what code was executed. Imagine you have a testsuite with 100% path coverage. Now remove all assertions and run the testsuite again. Voilà, the testsuite still has 100% path coverage, but it tests absolutely nothing.


128

If every path through a program is tested, does that guarantee finding all bugs? No If not, why not? How could you go through every possible combination of program flow and not find the problem if one exists? Because even if you test all possible paths, you still haven't tested them with all possible values or all possible combinations of values. ...



Top 50 recent answers are included