Does not testing internals entail diligent refactoring and/or rely on developer talent?

Question

I'm not asking here what the arguments are for/against testing internal methods (though I'll restate some, and don't mind hearing others). My questions relate to the implications of only testing public interface. Especially, whether I understand the extent of the "refactor" part of Red-Green-Refactor in by-the-book TDD, and if there are non TDD techniques TDD practitioners use that address the problems that make me want to test internals.

My question is: If you write the most straightforward code possible to get the light green, and if you don't test internal methods, does it follow that from time to time you allow yourself to start off by writing a class that does far more than it should with a bunch of member variables you know will not survive refactoring? And wind up after refactoring with a bunch of methods that aren't explicitly tested?

As I get familiar with TDD, it feels like anyone would at least be tempted to test internal methods. And many practitioners flatly say you shouldn't. E.g. (of about 23 million results):

Item 2 here and it's comments and this StackOverflow post

Given an implementation will involve more than one non-trivial problem, if you only test the public interface, there are two possibilities: 1) non-trivial logic winds up getting tested only indirectly, AND tested in the same calls as other non trivial logic. 2) you make methods public that you don't expect clients other than the SUT and the tests to call. I think what I hear from the advocates of by-the-book TDD is "1 is right. Yes, a lot of non trivial code is tested only indirectly, and that isn't a problem. After all, you don't want your tests to start failing when you improve the implementation. Even if they don't fail, it's not innocuous if tests keep passing when they're targetting a bunch of no-longer-used code, because the tests show how to use your class. You Should Only Test the Public Interface." And that sounds like a reasonable thing to say, but I haven't seen it said explicitly.

But even though it sounds reasonable -- if I don't test internals I feel like I lose part of what helps keep the tests DRIVING the coding (maybe part of my difficulty applying the precepts is that I don't distinguish between coding and developing). If there's some internal logic that has to turn a String to a valid int32, int64 or decimal value depending on what's in the string and the type of some other object, I want to test that little bit of logic, not just find out that the whole method failed. I want one test to pinpoint those few lines of code and exercise what should fail.

The thinking in the doubly downvoted answer here seems so obviously attractive that I have to wonder if the by-the-book fellows are neglecting to mention some instrumentation/tracing/logging code they use in addition to TDD, or if they watch it work in a symbolic debugger. Or do they just recognize that as a failure point and write a test with input that will cause a failure pinpointing those few lines of code? If they do that it's testing the implementation though. It seems so natural to want to test that code in isolation, and verify it fails where it should, even though it's an implementation detail.

I suppose that with talent and experience you can classify bigger chunks of code as "non-trivial" but I haven't seen anyone bluntly saying or condescendingly implying that that's why they don't have to test implementation details.

Lots of time it makes perfect sense to me to test the public interface, lots of methods really are like what you see in TDD examples. But for the feel of where I wonder if purists are doing something more than writing tests on a public member: Say you want to put data from an ISAM file into a database. There is one public method to the envisioned consumer of the code: public void LoadIt(SqlConnection c). "Loadit" has a bunch of other dependencies that will be resolved using configuration files and environment variables: some code has to determine that it can find the name of the folder where the ISAMs live; to find, load and parse the schema file for the isams using the ISAM vendor's DDL library; so forth+so on. But none of that is of any interest to any envisioned calling code.

My only reason for moving the dependency resolvers out of the Loadit class would be so I could test them OR to make the code easier to understand/maintain.

Moving to another class to make them testable is Doing It Wrong -- they're really just implementation details of LoadIt. And in my first implementation, I am just trying to write enough code to get the light to turn green. So I won't write anything with reasonable maintainability if it's easier to just manhandle a bunch of variables in a big method or two. I won't write helper classes. Instead, I'll get the light green and then do the Refactor part diligently. Even when I clean it up, any classes introduced keep "internal/friend" accessibility, and only get tested via the call to LoadIt.

Thanks for reading this far; as a reminder, my question is in the second paragraph.

Answer 1

You have a lot of intertwined questions.. the post was long and I'm confused. But let's work at it..

First off second paragraph.

• The common answer is not that you don't test internal/non-public methods, rather you do not test them directly via tests. The internal methods should be called by one or more types and their tests would end up verifying the internal methods as part of their execution. Reason: Refactoring becomes hard. e.g. It should be easy to move around private methods as long as you don't break the behavior. But if you have tests for these methods, then you end up breaking and fixing a lot of tests even though the functionality is intact. Remember refactoring is behavior-preserving and structure-improving.
• Ending up with code that isn't tested. As long as you write the simplest thing that can work to make something green and refactor, you never end up with any code that isn't pulled in by at least one test. You can make the test pass with a hack like return "canned value". But then the next test should expose this and pull in the real code.

You shouldn't have to increase visibility just for testing (make things public just for testing) but you need to design for testability. This may sound like a paradox, but an example would be.. don't new up dependencies within your method. Rather accept dependencies via ctor/method arguments. This allows your tests to easily substitute a fake dependency.

Your example: I would have

1. one test for Migrator.Transfer(DataSource, DataDestination) where the params are tech-agnostic roles/interfaces.
2. one integration test-suite to verify a real ISAM implementation of DataSource; that verifies the class can get data out of a real ISAM store
3. one integration test-suite to verify a real [insert DB tech] implementation of DataDestination; that verifies the class can insert data correctly into your DB>

Answer 2

The paradigm of object-oriented programming is to have a large collection of simple types that perform a independent unit of work accessed through public interfaces to create a network of communication that can convey tasks being performed in an abstract manner.

When you define a public interface your imposing a permanent constraint on the kind of work that can be performed. When your public interface starts concerning itself with implementing behavior that's not really related to it's focus you provide it in a separate class.

So yes, it's exactly as you described it. You'd have a bullet-proof protocol that wouldn't require any prior tests or consumers to be modified should the implementation of LoadIt() ever change.

What your test could tell you at the point of writing, though, is that by testing such an implementation of LoadIt() you'd be asserting several times over that it can load a ISAM file into the database because you'd have to set it up once that the directory path isn't configured; that it can discover all the ISAM files; that they're being represented in a suitable type. At the time of writing the tests you should observe a pattern emerging that hints that you should actually have a separate class that collects all the ISAM files and test that in isolation.

Well, in TDD, good code reflects good tests and good tests reflect good design principles. Definitely don't burden yourself with trying to get everything right immediately, but if at any point you become queasy and get this creepy feeling it may be evidence of a anti-pattern emerging somewhere nearby. Just write your tests freely and with practice you'll quickly be able to asses whether your taking on a good approach.

Update on comments

Well, what you definitely do apply extensively is building code coverage reports. There is no reason to introduce something that should be private as public for the sake of testing it, but the fact of the matter is that when you simply go for green lights to prove that you understand what is it that your doing, your code will probably be lacking design polish: it'll be tightly coupled into a complex structure where your public interfaces branch of doing varying amounts of work. Your tests will pound against them to assert they're exhibiting correct behavior under all circumstances until you simplify the code structure.

I've practiced testing on my own accord and made my share of mistakes. I'm not the best person to give you a concise educated answer. The problems your finding solutions for require approaches specific to your case. The single most hardest part of testing is when you reach the point that you don't know how and what to test, the circumstances that would force you to sway and abandon TDD and jump straight into the fray and just get it done. You have to think independently and develop a mode of hazard discovery test driving and damage control.

Having said that, I guess there is an aspect to applying TDD that you have to develop an approach for circumstances where your clueless as to how the implementation is supposed to turn out. We don't always operate within the boundaries of a well-defined architecture, working with specialized components that form groups of objects that have established patterns of communication that we fully understand and are comfortable to test against, sometimes you just need to start off with a mess.

Say that the implementation of LoadIt() was potentially volatile. What you do know is that all the tests up to that point are clear and simple. That means your can safely assume your public interfaces are properly designed, a bullet-proof protocol, which means you've isolated the development efforts away from the rest of the application.

I would say that on a per-situational basis anything goes. You then write tests to the best of your ability: write implementations and continue tracing your progress and refining the tests bit by bit. If you reach a point that your describing non-trivial code that seems to be closely related and resembling something that's well asserted, you refactor and extract it into an isolated component, come back to LoadIt(), simplify the tests and continue on, constantly reiterating.

For more involving examples, when your forced to venture deep into the unknown, you can start doing interactive programming or build small prototypes to get your bearings and work your way backwards and write tests as new insight arise, but even in such cases it should be that you have are containing the volatile code and surrounding every entry point: which is to say, never loose safe ground.

So in essence, you can cut to the chase and just go for green lights, but don't forget to apply at least some damage control when you leave code lying around that's not spacecraft-grade. At times you can be satisfied that the work being performed is at the very remote edge of your network with few consumers and defer from refactoring immediately; depending on your refactoring policy, you can wait to refine the design after you let everything settle in a bit or have more time, for the sake of maintainability, but definitely refactor if you ever revisit the code again and need to introduce new changes--it's just like putting flame to fire if you ignore that the existing tests require more attention.

Answer 3

Well there's scope to it.

I consider testing private methods directly foolish. Internal only in very specific circumstances, instantiating via reflection, using a public method that that calls a factory pattern for instance.

Protected might be wise, given you could be adding to the ancestor class somewhere completely different.

The main reason TDD dies in commercialville is because it gets introduced by some evalgelist type, and they go mad with it. Let them do that and TDD will be "temporarily" postponed while we get some code out of the door, and then postponed again, while we get version 2 done beacseu we didn't have any flipping tests....

Get the basics in there, beef it up as you go along. If you are really fortunate, fill in any when you fix the bugs from UAT, BETA stages.

The developer talent in question is very simple. Only people who know the rules should be able to get away with breaking them.

Answer 4

If you have no problem writing a good set of test cases, and no problem getting the tests to pass, so that each line of code is critical to at least one test, then you have no problem.

If you are having difficulty creating test cases to cover all the logic needed in some internal method, then you need to test that method separately. If it should not be a public method in it's current class, then make it a public method in some new class. At first you may think "It's a waste of time to create all those little classes". Soon you will find that all those little classes are quite handy, easy to understand, and are reused more than you might expect.