Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The problem occurs while doing TDD. After a couple of test pass, the return types of some class/module change. In a statically typed programming language, if a previous mocked object was used in the tests of some other class and was not modified to reflect the type change, then compilation errors will occur.

For dynamic languages however, the change in return types might not be detected and the tests of the other class will still pass. Sure there might be integration tests that should fail later on, but unit tests would erroneously pass. Is there any way how to avoid this?

Updating with a trivial sample (on some made up language)...

Version 1:

Calc = {
    doMultiply(x, y) {return x * y}
//.... more code ....

// On some faraway remote code on a different file
Rect = {
    computeArea(l, w) {return Calc.doMultipy(x*y)}

// test for Rect
testComputeArea() { 
    Calc = new Mock()
    Calc.expect(doMultiply, 2, 30) // where 2 is the arity
    assertEqual(30, computeArea)

Now, on version 2:

// I change the return types. I also update the tests for Calc
Calc = {
    doMultiply(x, y) {return {result: (x * y), success:true}}

...Rect will then throw an exception on runtime, yet the test will still succeed.

share|improve this question

migrated from Jan 2 '13 at 15:37

This question came from our site for professional and enthusiast programmers.

What the answers so far seem to miss is that the question is not about the tests involving the changed class X, but the tests of class Y which depends on X and thus gets tested against a different contract than what it runs against in production. – Bart van Ingen Schenau Jan 2 '13 at 17:11
I just asked this question on SO, myself, in regard to Dependency Injection. See Reason 1: A dependent class can be changed at run time (think testing). We both have the same mindset but are lacking great explanations. – Scott Coates Jan 4 '13 at 23:53
I'm re-reading your question but am getting a little confused on the interpretation. Can you provide an example? – Scott Coates Jan 5 '13 at 0:16

To a certain extent, this is just part of the cost of doing business with dynamic languages. You get a lot of flexibility, otherwise known as "enough rope to hang yourself". Be careful with it.

To me, the problem suggest using different refactoring techniques than you would in a statically typed language. In a static language, you replace return types in part so that you can "lean on the compiler" to find what places may break. It is safe to do, and probably safer than modifying the return type in place.

In a dynamic language, you can't do that, so it is much safer to modify the return type in place, rather than replace it. Possibly, you modify it by adding your new class to it as a member, for the classes that need it.

share|improve this answer

If your code changes and your tests still pass, then there's either something wrong with your tests (i.e. you are missing an assertion), or the code didn't actually change.

What do I mean by that? Well, your tests describe the contracts between parts of your code. If the contract is "the return value must be iterable", then changing the return value from say an array to a list is not actually a change in contract, and therefore won't necessarily trigger a test failure.

In order to avoid missing assertions, you can use tools such as code coverage analysis (it won't tell you what parts of your code are tested, but it will tell you what parts definitely aren't tested), cyclomatic complexity and NPath complexity (which give you a lower bound on the number of assertions required to achieve full C1 and C2 code coverage) and mutation testers (which inject mutations in your code such as turning true to false, negative numbers into positive, objects into null etc. and check whether that makes tests fail).

share|improve this answer
+1 Either something wrong with tests or code didn't actually change. This took me a while to get used to after years of C++/Java. The contract between parts in dynamic languages of code should not be WHAT is returned, but what that thing that is returned contains and what it can do. – MrFox Jan 11 '13 at 15:44
@suslik: Actually, this is not about static vs. dynamic languages, but rather about Data Abstraction using Abstract Data Types vs. Object-Oriented Data Abstraction. The ability for one object to simulate another object as long as it behaves indistinguishable (even if they are of completely different types and instances of completely different classes) is fundamental to the whole idea of OO. Or put another way: if two objects behave the same, they are of the same type, regardless of what their classes say. Put yet another way: classes aren't types. Unfortunately, Java and C# get this wrong. – Jörg W Mittag Jan 12 '13 at 1:34

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.