You are correct, defects in the software do mean that the results are unreliable. However, there is a way to work around it, and build a reliable suite of tests.
The idea is to build a small testing harness that tests the very basic "core" functionality of each component of the testing system (usually, it consists of at least a framework and a runner). If testing framework is flexible enough, you should be able to plug that harness straight into your system, for example, by implementing an interface or by providing a required set of methods in the implementation of your harness.
The rest of the functionality should be tested relying solely on the "core" functionality, which is already tested using the harness. If you take care of using only the functionality from the "core" to test the non-core functionality, you would have a set of tests that you can trust.