Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am experimenting with marking assignments for a programming course using unit tests. Is it likely to harm the learning process if I publish some of the tests I use as part of the problem statement? I figure that reading the code will be good practice and provide a more accurate problem statement than I can deliver in words... but I'd like to hear about any drawbacks.

I would change the exact numbers that I expect to put in and get out of the tests, and the assignments are more about testing introductory OO concepts than whether they correctly approximate Pi, i.e. I'm not concerned that showing some possible answers would give away too much.

share|improve this question

closed as primarily opinion-based by Ixrec, durron597, Ampt, Robert Harvey, gnat May 12 '15 at 10:06

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise.If this question can be reworded to fit the rules in the help center, please edit the question.

Your tests better be "randomized". Some 'geniuses' might learn to fool the test rather than do it right. I would maybe publish tests after the code has been submitted. The passionate coders will quickly learn to write their own tests. The rest of the herd will behave as always ... – Job Jul 30 '11 at 15:54
up vote 2 down vote accepted

A couple of thoughts...

Yes, I think it does have drawbacks, they may learn from the code, but they may also be steered too much. I guess it depends on what you give them and in how much detail.

Also, unit tests usually focus on a single class. Any unit test would therefore reveal way too much detail of what needs to be taken into consideration when implementing the problem. Which is something they should learn the hard way and not be handed on a silver platter.

For example, consider edge cases: passing an empty string, passing in a value just on the exact limit of the domain, did I need a A > B or an A >= B test, etc. I think everybody needs to fail to allow for an edge case and "suffer" the consequences at least once, preferably more. You simply need that experience to get in the habit of "considering the edges".

Another thing is that unit tests are very specific to the class/function they test. So if your students are to implement these on their own, how can you code your unit tests in advance? For classes you could consider "coding" your tests against an interface declaration (good habit for your students to pick up). For functions you could give them the function declaration (its signature) I guess. In any case you will have to specify some sort of contract for them to stick to with their implementations, or you can't do anything in advance that would allow you to test all your students' work with the same unit tests.

If it is the accuracy of the problem statement you are after. There are other ways. And one that I think would teach them something about how to do an intake of a problem statement (or when they are let loose in the real world: the specification of a feature). I would suggest not giving them the unit tests, but a write up of the "requirements" in simple statements that could be transformed into the function/method names of unit tests.

I have gotten in the habit of naming my unit tests in terms of "A Should B". Where A is a situation to test and B is the desired outcome.


procedure YearDigitsShouldRevertToDefaultForLessThanShortYear;
procedure YearDigitsShouldRevertToDefaultForBetweenShortAndLongYear;
procedure YearDigitsShouldRevertToDefaultForMoreThanLongYear;

procedure Custom_VariableSequence_ShouldZeroDigits;
procedure Custom_FixedSequence_ShouldSetDefaultDigitsWhenSpecifiedAsZero;
procedure Custom_FixedSequence_ShouldSetMinimumDigitsWhenSpecifiedBelowMinimum;
procedure Custom_FixedSequence_ShouldSetMaximumDigitsWhenSpecifiedOverMaximum;

Yes, it does make for long names, but it is self-documenting albeit in "short hand". And it allows for a specific test for each specific situation. What's more, most of these can be "generated" from a less accurate narrative description of a problem statement or feature specification.

When faced with a narrative description of an algorithm, nowadays I tend to break it down into single "A should B" statements and verify those with whomever wrote the specs. After which setting up my unit tests is a breeze: I just need to convert the statements into descriptive function names...

share|improve this answer
Thanks - your suggestion is more or less as I have done now, I see the value of letting the students learn to interpret not completely specific questions as practice for The Real World. – Fritz Meissner Jul 30 '11 at 19:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.