Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

For example I would like to validate that name is letters only and is between 4 and 14 letters length. I have the following code in model:

validates: name, :format => { :with => /^[a-zA-Z]+$/,
                              :message => 'Name should be letters only. },
                 :length => { :minimum => 4, :maximum => 14 }

So it clearly lets me do what I want.

But as for unit tests, I have a bit too much perfectionism so I set something like

invalid_names = ['1234',
                 'qwe',
                 '1%#$#$',
                 'Sam1',
                 '%',
                 random_string(15)] #I also have a test method to create random string with parametrized length

valid_names = %w['test',
                 'Travis',
                 'John',
                 random_string(5),
                 random_string(14),
                 random_string(4)]

and test each of them in a loop with asserts, like

invalid_names.each do |name|
    user = User.new(:name => name)
    user.save
    assert user.errors[:name].any?, "#{name} is valid."
end

So it definitely works great. But it is too verbose (because of valid/invalid names arrays, and added random_string method), also I can't be sure my test actually tests all symbols and their combinations possible and all lengths and stuff, even though I am definitely sure it works as expected.

So what is acceptable way to test my validation without being too much perfectionist, yet to leave most part of logic tested?

Am I just set in a mind trick of trying to write a perfect code just to write a perfect code and forgetting about main finish goal: working product?

share|improve this question
1  
Why do you think "it is too verbose"? –  Doc Brown Mar 12 '13 at 11:35
    
cause of valid/invalid names arrays, cause of added random_string method –  Konnigun Mar 12 '13 at 11:37

4 Answers 4

Some suggestions:

  • never use random strings for unit tests if there is no compelling reason for it. They have a high risk of producing non-reproducible cases. If you want to use a random string, choose the string randomly once and hardcode that choosen string into your test.

  • an old wisdom of testing is that you cannot test every possible combination of input data if the allowed range of input values is too huge. Choosing test cases is some kind of "art", the idea is to use as few test cases as needed to test the most possible failure cases.

For example, classify your input domain into categories:

  • string with length 0
  • strings with length between 0 and 4
  • strings with length 4
  • strings with length between 4 and 14
  • strings with length greater 14

And as a second classification

  • strings with no legal characters
  • strings with some legal characters
  • string with only legal characters

Now, make sure you have at least one test case for each category. Of course, this example shows only finding tests by using "equivalence classes". Other techniques are designing your tests for edge cases (like the ones in @MainMa's answer), or tests for getting full code coverage and branch coverage.

share|improve this answer
    
I see. But how do I test validation of string with maximum of 40 characters? Or should I just skip testing it? Or... something like 'a'*41 would work? And your second advice is brilliant. Thanks for your advices! –  Konnigun Mar 12 '13 at 11:48
    
@Konnigun youre missing the point, testing isnt supposed to cover every single input and calculate every single output, otherwise its the entire program right there, its supposed to take a small handful of verbose test cases that test the points it is most likely to fail, for example, you dont need to test a length of 1 2 3 4 5 ... 15 16, you need to test one below the range, one above the range and one in the range, as well as one On the range to check it falls in the right place. –  RhysW Mar 12 '13 at 11:51
    
..continued. So youve reduced 16 possible tests to 4. Increasing maintainability and still having a similar granularity of testing –  RhysW Mar 12 '13 at 11:52
1  
@Konnigun: a string length "40" is not different from "15" in your case - unit tests are typically "white box tests", you are allowed to make at least some assumptions about the code under test. –  Doc Brown Mar 12 '13 at 11:55
    
Thanks, it's clear for me now ^^ –  Konnigun Mar 12 '13 at 12:10

What you should test

  • General, ordinary case. Just to be sure it works. This is the simplest case:

    1. A name which contains only letters and has its length between 4 and 14.

    2. A name which contains only letters and has its length between 1 and 4.

    3. A name which contains only letters and has its length between 14 and the maximum authorized length.

    4. A name which contains only letters and has exactly 4 characters.

    5. A name which contains only letters and has exactly 14 characters.

    6. A name which contains letters and something else and has its length between 4 and 14.

  • Edge cases

    1. An empty string.

    2. A null (if your language makes a difference between an empty string and null).

    3. A bunch of unicode strings with different normalization and different characters. Those tests should match your documentation explaining what do you mean by "letters only". Is щ a letter? Is a letter? What about and its two possible unicode representations (combined and normalized)?

What you shouldn't test

You shouldn't test repetitive things. If you tested both "test" and "Travis", testing also "John" brings nothing useful, since it contains only lowercase and uppercase letters, just like "Travis", and is 4-characters long, just like "test".

In the same way, %, Sam1 are redundant.

As for random strings, you shouldn't use them, since if your code is not working as expected, they might in some cases give you random results: sometimes the test will pass, the next time it will fail for the exact same code.

share|improve this answer
    
Ye, it was kinda paranoidal to test both 'Travis' and 'test', they could fail only if logic would act in some rly strange way of choosing only some letters and stuff, it's just paranoidal and too much overheaded, cause I can clearly see regex in model actually. Thanks! –  Konnigun Mar 12 '13 at 12:11

Your code should meet a use case and it should do it in a way that is relative to the risk created by missing something. Is it illegal in the context of your application to allow numbers/characters in a name (Maybe this is a database for the Names with Letters Only Society)? Then test it as much as possible.

Not that you want to invite someone to micro-manage you, but take into consideration how much of your time the people paying for this software want you to spend. They may not share your level perfection and won't want to pay for it either.

share|improve this answer
    
It's my own project, probably that's why I am so obsessed with every aspect of it to be as good as possible xD –  Konnigun Mar 12 '13 at 13:29

So what is acceptable way to test my validation without being too much perfectionist, yet to leave most part of logic tested?

That depends. Different people consider different things to be "enough testing". In some cases, some types of tests are imposed by the nature or boundaries of the project itself.

Examples:

  • We had one project where we had to offer clients a performance guarantee (min. X messages processed per minute). This was tested throughout the development, in nightly tests. Performance bottlenecks were identified and we designed/implemented around them.

  • We had one project that did XML parsing and had to provide security guarantees (won't crash on invalid inputs). We ended up implementing a fuzzing library (that took each API, corrupted one of it's parameters in a predefined way and called the API in a loop, for each corruption).

Normally, I want to see that the default trivial cases are implemented, corner cases are implemented, generic cases are implemented and maybe a negative case or two (check that correct data is in the generated exception, when the inputs are wrong). In my current project though, such a level of testing is not realistic.

We also had a project where (by agreement) solving a defect also implied adding a new unit test. When running the test suite, full regression would run automatically (i.e. we were able to guarantee that we didn't re-create any of the old issues found in the application).

Am I just set in a mind trick of trying to write a perfect code just to write a perfect code and forgetting about main finish goal: working product?

Not from what I could tell.

Edit: a few other points:

  • If you need randomness in your tests, make sure it is repeatable (for example, initialize your random number generator with a constant seed)

  • At the very least, test for the most common case;

  • Adding unit tests for most corner cases is beneficial when the code changes (in your code, that would be lengths 4 and 14; not sure about the formats). When updating an algorithm, we tend to think (naturally) at the general case; These tests check things that you (the developer) tend not to think about at every edit.

share|improve this answer
    
Thank you, this explanation was very helpful! –  Konnigun Mar 12 '13 at 13:32

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.