Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

When doing the Red, Green & Refactor cycle we should always write the minimum code to pass the test. This is the way I have been taught about TDD and the way almost all books describe the process.

But what about the logging?

Honestly I have rarely used logging in an application unless there was something really complicated that was happening, however, I have seen numerous posts that talk about the importance of proper logging.
So other than logging an exception I couldn't justify the real importance of logging in a proper tested application (unit/integration/acceptance tests).

So my questions are:

  1. Do we need to log if we are doing TDD? won't a failing test reveal what wrong with the application?
  2. Should we add test for the logging process in each method in each class?
  3. If some log levels are disabled in the production environment for example, won't that introduce a dependency between the tests and enviroment?
  4. People talk about how logs ease debugging, but one of the main advantages about TDD is that I always know what's wrong due to a failing test.

Is there something I am missing out there?

share|improve this question
4  
I think logging is the special case to so many rules. A class/function should do one thing and one thing only... except for logging. Functions should preferably be pure, except for logging. So on and so forth. Logging breaks the rules. –  Phoshi Feb 24 at 16:34
1  
Isn't logging mostly orthogonal to any SW development methodology used? So perhaps you should just narrow it down and ask: Do we need test cases for logging when doing TDD? –  hyde Feb 24 at 18:54
add comment

9 Answers

up vote 47 down vote accepted

1) Do we need to log if we are doing TDD? won't a failing test reveal what wrong with the application?

That assumes you have every possible test your application needs, which is rarely true. Logs help you track down bugs you did not write tests for yet.

2) Should we add test for the logging process in each method in each class?

If the logger itself is tested, it would not need to be retested in each class, similar to other dependencies.

3) If some log levels are disabled in the production environment for example, won't that introduce a dependency between the tests and environment?

Humans (and log aggregators) depend on the logs, the tests should not depend on them. Typically there are several log levels, and some are used in production, and some additional levels are used in development, similar to:

"Rails log level is info in production mode and debug in development and test" - http://guides.rubyonrails.org/debugging_rails_applications.html

Other applications use a similar approach.

4) People talk about how logs ease debugging, but one of the main advantages about TDD is that I always know what's wrong due to a failing test.

Production bugs will have passed all the tests, so you may need some other reference to investigate those issues.

share|improve this answer
22  
Even perfect rigorously executed TDD will not prevent production problems with performance, system interaction, third-party interaction. Concurrency issues are also very hard to fully cover by test. A 100% test coverage "only" means that 100% of your code has been executed, not that is correct in all states or environments. –  Holstebroe Feb 24 at 16:29
add comment

Logging is useful to explain non-exceptional behavior of an application:

Event logs record events taking place in the execution of a system in order to provide an audit trail that can be used to understand the activity of the system and to diagnose problems. They are essential to understand the activities of complex systems, particularly in the case of applications with little user interaction (such as server applications)...

No matter how application was tested and no matter how well exceptions are logged, its users may ask,

output of your program is 0 while we expected it to be 1, why is that?

You need logging to verify what was application config, parameters and other runtime details to explain its (non-exceptional) behavior.

From above perspective, logging is more oriented on support than on development. After the application goes live, it is desirable to let someone else handle questions from users, to allow programmers focus on further development.

Logging what application does allows that someone else understand program behavior without digging into the code and without distracting developers by requests to explain what's going on.

share|improve this answer
5  
+1 for "logging is more oriented on support". Really hit the nail on the head. –  Tibos Feb 24 at 18:03
1  
+1 for "non-exceptional behavior" and also for "logging is more oriented on support". But could you please edit your answer to list the source of the paragraph you quote? –  logc Feb 24 at 21:48
    
@logc source of the quote is referred by the link in the very first word here, "Logging" - if you click it, it will you bring to Wikipedia article: en.wikipedia.org/wiki/Logfile –  gnat Feb 25 at 6:46
add comment

Most answers here focus on the aspect of correctness. But logging serves a different purpose, too: Logging can be a way to gather performanc relevant data. So even when the system works without error a log can tell why it is slow. Even with full test coverage of all aspects a test suite won't tell.

Of course a performance critical system can/should provide key performance metrics to some operational dashboard, but "classic" logging can provide a different level of detail.

share|improve this answer
1  
+1 for performance logging, good point! –  Andres F. Feb 24 at 13:48
2  
Also don't forget about logging for audit trail purposes. Or billing. Or various kinds of statistics. Or event logging for matching up with other kinds of statistics of other systems. –  PlasmaHH Feb 24 at 14:10
    
Isn't profiling something you would do without logging? Or do you just mean that you could continuously log the profiling results? –  Bergi Feb 24 at 18:17
    
@Bergi: entirely depends on how your application works. If its e.g. a webapplication then logging the service time of each request, and then later trying to cluster those events for "bad performers" may work too. –  PlasmaHH Feb 24 at 21:40
    
@Bergi profiling happens in development, but there are effects on live systems to e kept in mind, using disks can become slow, CPU can be more loaded, services might not respond in time, parallel threads might run into locking issues ... –  johannes Feb 24 at 22:06
show 1 more comment

The short answer to your main question is: as a general rule, bugs in your code will NOT be exposed by TDD. Some might, ideally many, but the absence of failing tests does not imply the absence of bugs. This is a very important maxim in software testing.

Since you cannot know whether you will have incorrect behavior in your system, maybe under rare conditions, logging is a useful tool that might help understand what's wrong when things inevitably do go wrong.

Logging and TDD address different concerns.

share|improve this answer
add comment
  1. Unless you have a 100% test cover, which is usually not the case, you can't know that your software will never crash (EDIT:and — as said in the comments — even if it does, something independent of your software might cause a crash); it's the same as thinking you can do a software that has absolutely no bugs (not even the NASA can do this). So at the very least, you need to log possible failures in case your program crashes so you can know why.

  2. The logging should be done by an external library or intern framework depending on the technology you're using. What I mean by that is that it should be something that has already been tested before and that you don't need to test yourself. It's overkill to test that every methods logs the things it's supposed to.

  3. The logs are not meant for tests, there should be no dependency whatsoever. That said, you don't need to disable logging for tests if it feels like a constraint to you, though the logs should be kept in a file corresponding to the environement (you should have a different file for the testing, development and production environment at the very least).

  4. An error might be very unclear and it's not always obvious what's gone wrong when one TDD test failed. Logs should be more precise. For instance, if you're doing a sorting algorithm and the entire test case fails, you should have logs for every single test of the algorithm which help you spot where the problem actually lies.

share|improve this answer
3  
Even if you have 100% coverage do you have it for every possible thing that can happen? What if the network connection to your database goes down? Will your tests tell you that? –  Zachary K Feb 24 at 13:35
    
You can never know your software will never crash EVEN if you have 100% coverage. 100% coverage, while desirable, gives a lot less information about correctness than it would seem. –  Andres F. Feb 24 at 13:35
    
Yes, you are right about that too. The point is, having no possibility of crash is not doable. I'll edit. –  Arlaud Pierre Feb 24 at 13:37
1  
The edit is still incorrect. Even if you have 100% test coverage, there can be a bug in your code (no need to blame external causes). Tests do NOT imply your code work; the only thing they imply with any certainty is that you failed to write a test that finds a bug :) Test coverage is an important metric, but is not directly related to the absence of bugs. –  Andres F. Feb 24 at 17:32
3  
It's trivial to prove that "100% test coverage that passes != bug free". Counterexample: add(x, y) = 2 (always returns 2). The following test passes and provides full coverage: assert(2 == add(1,1)). 100% test coverage for a buggy function :) –  Andres F. Feb 24 at 17:40
show 1 more comment

Yes, in the general case you need logging.

Logging is not about debugging. Well, OK, a part of logging is sometimes about debugging, and you can skip that part if you don't need it during development.

But the more important part of logging is about maintainability. Well-designed logging might answer the following questions:

  • Is the application still alive and kicking? (By logging a heartbeat every 1000 seconds.)
  • Did the performance of the application change over the last 10 months? (By logging statistics of the performance of use cases.)
  • Did the load of the appliation change over the last 10 months? (By logging the number of requests per request types.)
  • Did any of our interfaces change their performance characteristics?
  • Does the new release cause a different usage characteristic towards some of the subsystems?
  • Are we under a DoS attack?
  • What kinds of errors are happening?

All this can be achieved by logging. And yes, it should be planned and designed and tested, preferable automatic.

Logging is a feature that deservers treatment, just as other features.

share|improve this answer
add comment

TL;DR: Logging and TDD are orthogonal. Having one has no bearing on needing the other

Do we need to log if we are doing TDD? won't a failing test reveal what wrong with the application?

By and large most logging that I've implemented, and that I've seen implemented has been for operational troubleshooting, not for development debugging (although it may help). The primary audience for this logging are the admins and the ops who run your servers, support people who have logs sent to them for analysis, and customers who want to look at the logs and try and figure out what's going on.

These logs are there to help troubleshoot issues largely do to integration points. This may network services (database, soap, etc), local resources (disk, memory, etc), bad data (customer input, bad/corrupt data sources, etc), etc. Capturing exceptions, logging faults, and even doing informative logging (settings, configurations, etc) can all help with troubleshooting.

Should we add test for the logging process in each method in each class?

Add testing where ever you need to in order to test logging. If you have ad hoc calls to log out information then they should be tested. Although if you implement logging and log testing using Aspect Oriented Programming or meta programming, that may reduce the burden of testing.

If some log levels are disabled in the production environment for example, won't that introduce a dependency between the tests and enviroment?

If you are writing your code using IoC and you make use of mocks, you should be able to effectively test all your logging without relying on a particular environmental setup.

share|improve this answer
add comment

TDD generally helps reduce coding bugs. It helps a lot less with bugs with the specification or just misunderstandings about how things work.

"Oh? You can get a data message before you get confirmation the logon succeeded? I never knew that, well it won't handle that!"... That kind of thing. Logging is very useful for telling you what the software tried to do so you can spot the thing you did wrong.

share|improve this answer
add comment

In my experience great level of logging is added to the application when we don't do TDD. Then the level of uncertainty becomes high hence we add logging to see what is going on.

Whereas when doing TDD (or maybe test-whenever) I find myself adding a lot less log statements. This in turn means less LOC and can (not always) affect performance.

But we do add entering-exiting logs for functions in a semi-automatic way at my company in most cases regardless of development method. As I know this was considered mandatory for production issue analysis.

Example could be methods of an EJB service bean which are present on the public interface. Another might be a case when a function does complex calculations. It can help a lot to have figures entering the method (for example you can write a unit test to get back to the general topic at hand).

share|improve this answer
    
could you please expand on reasons why do you add entering-exiting logs for functions? why is this required at your company? –  gnat Feb 26 at 8:22
    
yes absolutely. i hope it is better now. –  dave00 Feb 26 at 12:34
add comment

protected by World Engineer Feb 24 at 15:28

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.