I'm not asking here what the arguments are for/against testing internal methods (though I'll restate some, and don't mind hearing others). My questions relate to the implications of only testing public interface. Especially, whether I understand the extent of the "refactor" part of Red-Green-Refactor in by-the-book TDD, and if there are non TDD techniques TDD practitioners use that address the problems that make me want to test internals.
My question is: If you write the most straightforward code possible to get the light green, and if you don't test internal methods, does it follow that from time to time you allow yourself to start off by writing a class that does far more than it should with a bunch of member variables you know will not survive refactoring? And wind up after refactoring with a bunch of methods that aren't explicitly tested?
As I get familiar with TDD, it feels like anyone would at least be tempted to test internal methods. And many practitioners flatly say you shouldn't. E.g. (of about 23 million results):
Given an implementation will involve more than one non-trivial problem, if you only test the public interface, there are two possibilities: 1) non-trivial logic winds up getting tested only indirectly, AND tested in the same calls as other non trivial logic. 2) you make methods public that you don't expect clients other than the SUT and the tests to call. I think what I hear from the advocates of by-the-book TDD is "1 is right. Yes, a lot of non trivial code is tested only indirectly, and that isn't a problem. After all, you don't want your tests to start failing when you improve the implementation. Even if they don't fail, it's not innocuous if tests keep passing when they're targetting a bunch of no-longer-used code, because the tests show how to use your class. You Should Only Test the Public Interface." And that sounds like a reasonable thing to say, but I haven't seen it said explicitly.
But even though it sounds reasonable -- if I don't test internals I feel like I lose part of what helps keep the tests DRIVING the coding (maybe part of my difficulty applying the precepts is that I don't distinguish between coding and developing). If there's some internal logic that has to turn a String to a valid int32, int64 or decimal value depending on what's in the string and the type of some other object, I want to test that little bit of logic, not just find out that the whole method failed. I want one test to pinpoint those few lines of code and exercise what should fail.
The thinking in the doubly downvoted answer here seems so obviously attractive that I have to wonder if the by-the-book fellows are neglecting to mention some instrumentation/tracing/logging code they use in addition to TDD, or if they watch it work in a symbolic debugger. Or do they just recognize that as a failure point and write a test with input that will cause a failure pinpointing those few lines of code? If they do that it's testing the implementation though. It seems so natural to want to test that code in isolation, and verify it fails where it should, even though it's an implementation detail.
I suppose that with talent and experience you can classify bigger chunks of code as "non-trivial" but I haven't seen anyone bluntly saying or condescendingly implying that that's why they don't have to test implementation details.
Lots of time it makes perfect sense to me to test the public interface, lots of methods really are like what you see in TDD examples. But for the feel of where I wonder if purists are doing something more than writing tests on a public member: Say you want to put data from an ISAM file into a database. There is one public method to the envisioned consumer of the code: public void LoadIt(SqlConnection c). "Loadit" has a bunch of other dependencies that will be resolved using configuration files and environment variables: some code has to determine that it can find the name of the folder where the ISAMs live; to find, load and parse the schema file for the isams using the ISAM vendor's DDL library; so forth+so on. But none of that is of any interest to any envisioned calling code.
My only reason for moving the dependency resolvers out of the Loadit class would be so I could test them OR to make the code easier to understand/maintain.
Moving to another class to make them testable is Doing It Wrong -- they're really just implementation details of LoadIt. And in my first implementation, I am just trying to write enough code to get the light to turn green. So I won't write anything with reasonable maintainability if it's easier to just manhandle a bunch of variables in a big method or two. I won't write helper classes. Instead, I'll get the light green and then do the Refactor part diligently. Even when I clean it up, any classes introduced keep "internal/friend" accessibility, and only get tested via the call to LoadIt.
Thanks for reading this far; as a reminder, my question is in the second paragraph.