Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm really struggling to write effective unit tests for a large Django project. I have reasonably good test coverage, but I've come to realize that the tests I've been writing are definitely integration/acceptance tests, not unit tests at all, and I have critical portions of my application that are not being tested effectively. I want to fix this ASAP.

Here's my problem. My schema is deeply relational, and heavily time-oriented, giving my model object high internal coupling and lots of state. Many of my model methods query based on time intervals, and I've got a lot of auto_now_add going on in timestamped fields. So take a method that looks like this for example:

def summary(self, startTime=None, endTime=None):
    # ... logic to assign a proper start and end time 
    # if none was provided, probably using datetime.now()

    objects = self.related_model_set.manager_method.filter(...)

    return sum(object.key_method(startTime, endTime) for object in objects)

How does one approach testing something like this?

Here's where I am so far. It occurs to me that the unit testing objective should be given some mocked behavior by key_method on its arguments, is summary correctly filtering/aggregating to produce a correct result?

Mocking datetime.now() is straightforward enough, but how can I mock out the rest of the behavior?

  • I could use fixtures, but I've heard pros and cons of using fixtures for building my data (poor maintainability being a con that hits home for me).
  • I could also setup my data through the ORM, but that can be limiting, because then I have to create related objects as well. And the ORM doesn't let you mess with auto_now_add fields manually.
  • Mocking the ORM is another option, but not only is it tricky to mock deeply nested ORM methods, but the logic in the ORM code gets mocked out of the test, and mocking seems to make the test really dependent on the internals and dependencies of the function-under-test.

The toughest nuts to crack seem to be the functions like this, that sit on a few layers of models and lower-level functions and are very dependent on the time, even though these functions may not be super complicated. My overall problem is that no matter how I seem to slice it, my tests are looking way more complex than the functions they are testing.

share|improve this question
    
You should write unit tests first from now one this helps you spot testability problems in your design before the actual production code is written. –  Chedy2149 Aug 21 '13 at 9:10
    
That's helpful, but doesn't really address the issue of how to best test inherently stateful, ORM-heavy applications. –  acjay Aug 22 '13 at 15:58
    
You have to abstract away the persistence layer –  Chedy2149 Aug 23 '13 at 9:07
    
Sounds great hypothetically, but when it comes to project maintenance, I think there's a non trivial cost to inserting a custom-made persistence layer between the business logic and the extremely well-documented Django ORM. Suddenly, the classes become packed with a bunch of tiny intermediary methods that themselves need to be refactored over time. But perhaps this is justified in places where testability is critical. –  acjay Aug 26 '13 at 14:07
    
Checkout this : vimeo.com/43612849 and this : vimeo.com/15007792 –  Chedy2149 Aug 26 '13 at 14:36

1 Answer 1

I'm going to go ahead and register an answer for what I've come up with so far.

My hypothesis is that for a function with deep coupling and state, the reality is that it's simply going to take a lot of lines to control for its outside context.

Here's what my test case looks like roughly, relying on the standard Mock library:

  1. I use the standard ORM to set up the sequence of events
  2. I create my own start datetime and subvert the auto_now_add times to fit a fixed timeline of my design. I thought that the ORM didn't allow this, but it works fine.
  3. I make sure the function-under-test uses from datetime import datetime so that I can patch datetime.now() in just that function (if I mock the entire datetime class, the ORM pitches a fit).
  4. I create my own replacement for object.key_method(), with simple but well defined functionality that depends on the arguments. I want it to depend on the arguments, because otherwise I might not know if the logic of the function-under-test is working. In my case, it simply returns the number of seconds between startTime and endTime. I patch it in by wrapping it in a lambda and patching directly on to object.key_method() using the new_callable kwarg of patch.
  5. Finally, I run a series of asserts on various calls of summary with different arguments to check equality with expected hand-calculated results accounting for the given behavior of the mock key_method

Needless to say, this is significantly longer and more complicated than the function itself. It depends on the DB, and doesn't really feel like a unit test. But it is also fairly decoupled from the internals of the function--just its signature and dependencies. So I think it might actually be a unit test, still.

In my app, the function is quite pivotal, and subject to refactoring to optimize its performance. So I think the trouble is worth it, complexity notwithstanding. But I'm still open to better ideas on how to approach this. All part of my long journey toward a more test-driven style of development...

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.