Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Hi I have fairly complex program that is doing computations in a quite large loop. I want to log some basic statistic about the run to be able to analyze its performance over time and vs loop number. For now I am outputting a set of numbers, about 5 or 6, each loop.

The problem is the number of loops is quite large so the log files are big. Also putting 10^6 of measurement points on a plot is kind of pointless.

I thought of "filtering" the results by for example printing every 1k'th loop but I would lose some points-of-interest. I also cannot log only POIs, because I would lose track of general progress.

I decided to switch to an event based logging. Primarily the events I have in mind are combination of the two aforementioned things: improvement of current solution and some progress over the loops.

The Implementation I have in mind is to write a simple StatLogger that would be responsible for outputting the data to the log stream. The main program would just issue a log() method whenever an event occurs. The events can occur simultaneously and it does not make sense to log the state twice, so the class would be responsible for not outputting the stats twice.

My questions are:

  1. Is this a good or common approach and pattern?
  2. Is it better to pass a handler to the number cruncher object, so the logger can get all the interesting stats like StatLogger::log(const RunEnviroment *this), or is better to make another structure to hold all the interesting stats and make a StatLogger::log(const Stats& stats) method.
  3. If my approach is bad, what are common solutions in similar cases?
share|improve this question
    
Actually, I don't understand what you mean by "Is it better to pass a handler to the number cruncher object, so the logger can get all the interesting stats like ..." - what kind of handler (or "handle" like in your question title)? Who references whom in your two alternatives? Perhaps a more detailed example would help. –  Doc Brown Jul 16 '13 at 10:31
    
Umm... sorry if I wrote it in an incomprehensible way. Example is: I want to log something from with-in a RunEnvironment its state or whatever. Is it ok to call something like logger.log(this)? It is a handle, it could also be a reference or anything equivalent. Maybe its better make-up a Stats object, fill and call logger.log(stats). I actually find drawbacks in both approaches, that is why I posted my question. I can try to reword it in question if it's still problematic. –  luk32 Jul 16 '13 at 10:43
add comment

2 Answers

I would avoid something general like a StatLogger to be dependent on something special like a RunEnviroment, whatever that looks like. Keeping StatLogger independent will most probably enhance the testability and reusability of that class.

If you need, however, a simple method like

  StatLogger::log( EnumEventType evenTtype, const std::string &logtext) 

or something like

  StatLogger::log(const Stats& stats) 

(where I guess Statsrepresents one log entry), will mainly depend on what information you are going to log and how complex that information is.

If you want to decouple your main program from a specific logger, you could also introduce an abstract base class as an interface (like IStatLogger), and let your main program only access the logger through that interface. That will allow you to replace the real logger easily by a mock logger for testing purposes.

share|improve this answer
    
The problem with 1st invocation is that it is not possible to control whether its worth or not to actually log something. Like I said events can occur at the same time, which would issue log() call twice, when there is no reason to log something two times. At lest I want to be able to handle it. Secondly, Stats object would be also tightly related to RunEnvrionment, it would only encapsulate few variables. Thus, making StatLogger in its current form specific to a RunEnvrionment. Maybe templates and specializations is a good option? –  luk32 Jul 16 '13 at 13:01
    
Sure you need your "Stats" object to be tightly coupled to "RunTimeEnvironment"? It is hard to discuss that without having the slightest notion of what you have in mind with that. Concerning your "1st invocation" note - I am pretty sure I did not understand what's your problem there - you seem to have a solution, so what is your question? –  Doc Brown Jul 16 '13 at 13:51
    
I have two ideas to a problem, and both have drawbacks I really do not like. I want to output something like log << iteration << ' ' << average_value << ' ' << best_so_far. I don't want to have millions of lines like that because most are not interesting, however I want to have general notion of progress, as well as POI in the run-time such as new best solution found. So lets say I want to output a line with stats every 0,1% of total iterations + POIs. They can occur in the same iteration. I don't want to log it twice. Ultimately I want to plot progress of stats vs iterations or time. –  luk32 Jul 16 '13 at 14:09
add comment

One solution would be to introduce the logging levels. Each level would correspond to importance (or maybe priority) of log information. Such strategy would enable a log information to be grouped and therefore more easily filtered out. These groups could be for example:

  • TRACE; DEBUG; INFO; WARN; ERROR; FATAL

One such implementation can be found in Linux kernel.

Also, I'm sure you will find more information by googling various logger implementations.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.