Test Interview Question

Question

If you only had time to test 5 features of a product that had 10 features total, how would you decide which features to test?

Another Question:

When is it okay to ship a product with a bug

Answer 1

My first question would be what the purpose of the product is. Until I know the purpose of the product, I wouldn't know what to prioritize. Second question would then be to find out what the milestone goals of the release, again to figure out what is important.

Then from there, I would test the features that are most important for accomplishing the purpose of the product. If the product is a word processing program, features involving input/output would be high on the list. Saving and usability features would also be high. Things that would be down on the list could be optimization code.

Answer 2

I would decrease the features to 5 and test them fully. There's no point in writing untested code. It presents more problems than it solves.

This is what i would reply at least :)

Answer 3

I think there is no point trying to give a general answer to such questions (in other words, a general solution would be less than ideal in most of the cases). As others have noted, one should know more about

• the individual features,
• their importance for the user and
• their implementation, especially its level of risk.

Based on this, one can decide which features to focus the tests on if testing resources are scarce.

However, one must also know

• the mission/business/life criticality of the product,
• its estimated user base and
• its method of deployment.

E.g. for an experimental, noncritical product used only by a couple of users in-house, including an auto-update facility, it can be acceptable to ship untested features. If the users understand this is experimental stuff, they tolerate bugs better, and patches will be installed automatically. However, for a life critical health care product built into custom hardware, without an easy way to deliver software updates, shipping a feature without adequate testing would be a fatal mistake: it could lead to loss of life, and probably the demise of the producer company itself via the ensuing lawsuits.

Answer 4

$ python
>>> import random
>>> random.sample(range(10), 5)
[9, 2, 7, 4, 1]

Answer 5

I would test the 5 features that are most likely to be used the most, as that would mean the least encountered bugs for users in general.

Answer 6

If the point of the task is to choose five out of ten features, then estimate either:

• what features are most important or
• what features cover the most lines of code.

Answer 7

Interesting no one is thinking about value to the business of the software / features. I think an interviewer would be looking for someone who understands that commercial software development (in general) has to provide some sense of revenue creation / cost avoidance to be worthwhile. What is the

<business speak> value proposition </ business speak>

behind the software and its function would be one sensible response to the question.

Answer 8

As a developer neither of those questions is up to me. My job is to develop a single use case at a time delivering it tested. It is up to a project manager to ensure my efforts are on the most important task at that time and when a bug is sever enough to keep software from shipping.

If a bug is that severe then I should be fixing it and not working on new code which may introduce a new bug.

Answer 9

For both questions, consider cost and benefit.

If there are ten features total, how much benefit is gained by testing each one? If only three of those ten are new features, I'd probably want to test those three and the two most important existing features, or the two existing features that are most likely to be impacted by the new code. As others have said, there's not enough information to give a specific answer, but that's the approach any sane person would choose. Also, consider the cost and benefit of slipping the schedule long enough to allow proper testing of all the features.

The same goes for shipping with a bug. All products ship with bugs, and most products ship with known bugs. The question is: what's the impact of leaving a given bug in the product. How much will it cost to fix it? How long will fixing it take? How much could not fixing the bug cost in terms of sales, reputation, liability, etc.? Also, what can be done to mitigate any of those costs?

Answer 10

I would ask the business team for the product to outline risk, impact and other factors and then for them to decide the priority for which to test.

These questions involve something that is not the responsibility of a programmer.

A Product Manager would decide testing priority and also when to release with known defects.