Tag Info

Hot answers tagged

230

This sounds absolutely nutty. It is expending a great deal of effort for very questionable benefit, and the practice seems based on some faulty premises: That QA won't work hard unless they know they are being tested every day (which cannot be good for morale) That there are not enough unintentionally introduced bugs in the software for QA to find That ...


112

Well, based on what I've learned: It's not a school nor job interview; The testers are not children; It's not a game; It wastes company's money. The QA are not there only to find bugs but also to worry about how intuitive the system is, what is the learning curve for the user, usability, and accessibility in general. For example: "Is the system ugly?", ...


53

Bad idea. From the tester's point of view: "So they will test hard, because they know there are bugs present and not finding them might be considered as their incompetence." Basically the devs are booby-trapping the code. Few people like doing work which is ultimately pointless (because the bugs are known in advance) but which still affect how they are ...


36

Edit I want to be clear that this answer is only talking about the concept of testing your QA process, and I'm not defending the specific methodology portrayed in the question. End Edit There is a valid reason to check if your testing/checking is actually working. Let me give you an example from manufacturing, but the principle is the same. It's typical ...


28

I agree totally with the answers above as to why this is bad for motivation and just generally awful people management. However, there are probably sound technical reasons for not doing this as well: Just before the product goes to QA, the dev team adds some intentional bugs at random places in the code. They properly back up the original, working ...


17

Personally, I feel uncomfortable with this approach. The main thing that concerns me is the practicality of inserting intentional bugs. This seems to me to be difficult to do in any way that is predictable. Any code changes (intentional or otherwise) risk having side-effects. These side-effects may well be revealed during testing but it may not be obvious ...


16

It's a bad idea for all the reasons already given, but bug seeding is a useful tool for a different purpose. You can use it to get a rough metric of how effective the QA process is. In its simplest case, let's say you seed 100 bugs and they're representative of the full span of real bugs (I know, unlikely, but I'm simplifying). You do not tell QA you're ...


15

Here are four reasons not to do it: Anyone who reads your code will need to wade through lots of test code. This will make it harder to figure out what it is doing. All of the test code gets loaded into memory even when you are not testing. This is wasteful. (Perhaps not important in these days when 8 GB RAM is normal.) Test code should have exactly the ...


13

Bad idea. This is the sort of logical, binary approach that developers often bring, but it is demotivating for the QEs. It simply demonstrates a lack of trust. QEs often get placed in these situations without much input from them, and it assumed that they are OK with it, and it is not their place to suggest otherwise. This kind of thinking combines to ...


12

I'd say bad idea. One: Programmers are going to spend time putting deliberate bugs in the code, and some effort to save the good version. While the testers should presumably be testing everything, including the features with the planted bug, when they find one they will presumably have to go back and re-run that test to verify that this was indeed a bug ...


12

From what you describe I would say that the way you have been doing it - mocking the collaborators - is the best approach. It may be that you are over-specifying with your mock - for instance, requiring an order which is not really required by the business needs; if that is the case you could lighten up on the order requirements of your tests. But ...


10

I like the idea. Was it General Patton who said, "The more you sweat in peace, the less you bleed in war." Putting intentional bugs "wastes time" of the testers. But that also makes them work harder, meaning that they will also do a better job of finding UNintentional bugs. (And you have a copy of the "original" so you don't have to live with what you've ...


9

I don't really think this a bad idea. There's just a lot of things that I would speculate work better: Make QA accountable for the quality any way you can. For example by making support their responsibility also. This will increase their motivation to make sure the products shipped have higher quality. It always takes less effort to discover an inadequacy ...


8

Honestly, I'd call this behavior blatantly unethical and impractical. The PM is in need of some serious retraining, if not termination. It demonstrates a fundamental lack of understanding of the concept of quality assurance. Testers should not think like developers: they should think like end users. The entire reason for having QA teams is that developers ...


8

What you're talking about is called mutation testing, and there are a number of implementations available. I've not tried either, but there are at least two javascript versions: grunt-mutation-testing, and mutandis


5

Lets assume you are testing a class "interval", representing intervals of natural numbers: var interval= new Interval(1,100); Now your first two tests are "green": AssertIsTrue(interval.Contains(100)) AssertIsFalse(interval.Contains(101)) and you are confident the author made no "off-by-one-error". Unfortunately, the implementation looks like this ...


4

Coming from a world where developers are expected to write and run the tests themselves, this "testing" "QA" silo you are referring to frightens and confuses me, so I'll try to answer from this perspective. As an aside, qualified QA engineers, from my perspective, (as is described well in @SparK's answer), should focus on the larger issues of making sure ...


4

There is no basis for a reward or punishment on its own merit, but on the result of the behavior you're targeting. And sometimes there are unintended consequences. Is the goal to keep the QA team from slacking or to make some manager feel like he's actually contributing something without realizing he's just getting in the way. Positive Outcome - The QA team ...


4

Number of bugs caught or fixed has a very bad reputation as a software metric (it is no metric at all). Just Google it and you will find some horror stories. Summarizing these time values indeed won't tell much. What you want to tell your product owner alongside with the statistics is the following: You were working with a new tool, new methodology, you ...


4

Ideally in my world there would be a tool to benchmark the platforms and then a tool for running the program to get some figures so you could compare and understand which platform you'll be using later on. There is no such tool. There is not even a methodology to follow. The best you can do is extrapolate from the measured performance of the ...


4

A stub method is a method that just returns a simple but valid (though not necessarily correct) result. They are typically made when building the infrastructure and you don't want to spend time on every method needed right now. Instead you create stubs so everything compiles and your IDE's auto-complete knows about the methods you are planning to use. ...


3

If you mean how minimum/recommended system requirements are found, the application is simply tried on different machines. In most cases, there is no hard limit: if the application works with 512 MB of memory, it will probably work with 511 MB of RAM as well (unless it explicitly checks for the memory). This means that you may have a limited number of ...


3

I don't think there is anything else that you can do except testing that the functions inside that method are called. Since the "orchestration" method doesn't contain much logic unit-testing is not so important. It's a method that only wires up other methods. It integrates other components. Therefore there should be integration tests that cover those lines ...


3

It is absolutely not the responsibility of your coordinator to send mail, only to ask that it be sent. There are many reasons why a mail might not be sent, or delayed. It is the responsibility of your mail class to send valid mail reject invalid mail Log what it did (including unexpected failures to send valid mail) All your coordinator has to do, with ...


2

This isn't necessarily as crazy as it sounds. It rather depends on your motivation. If you are looking for a stick to beat your test team with, well that would be crazy. On the other hand, one of the most difficult things in software development is to know how effective your testing approach is. So if you structure it properly, you could use this technique ...


2

I would say this depends on what you call a "development feature". Unit tests or other automatic tests could be called "development features". Testing them separately does not make sense. In fact, you "test" them every time you run them. "Debugging tools" - maybe, if they are important to your team and you think someone could accidentally break them. ...


2

I think this is a great question. Unit testing, micro-testing, TDD, and BDD are difficult, sometimes confusing, and often not taken seriously. So kudos for trying and asking for help! I am by no means an expert, but I did just take a formal class and am also part of a small team that is leading an initiative to roll out unit testing and TDD within our ...


2

Since the Safari browser for Windows is horribly outdated (the latest download is from May 9, 2012), it is not a reliable browser to use as test and compare with the Mac version. On a project I am working on, we found there where to many differences between the Mac / iPad version of safari and the Windows version, so we decided it was worth to buy a Mac ...


2

Actually the same issue came up on a dojo last night. I did a quick research on it. This is what I came up with: Basically it is not forbidden explicitly by the TDD rules. Maybe some additional tests are needed to prove that a function works correctly for a generalized input. In this case the TDD practice is left aside just for a little while. Note that ...


2

You generally don't need to test controller classes, since they shouldn't hold complex logic. The actual work should be done in other classes if it is written correctly, so in the end you're essentially ensuring that the work gets delegated correctly. The complexity of this depends entirely on the language that you're using, since to do it correctly, you ...



Only top voted, non community-wiki answers of a minimum length are eligible