Hi I have fairly complex program that is doing computations in a quite large loop. I want to log some basic statistic about the run to be able to analyze its performance over time and vs loop number. For now I am outputting a set of numbers, about 5 or 6, each loop.
The problem is the number of loops is quite large so the log files are big. Also putting 10^6 of measurement points on a plot is kind of pointless.
I thought of "filtering" the results by for example printing every 1k'th loop but I would lose some points-of-interest. I also cannot log only POIs, because I would lose track of general progress.
I decided to switch to an event based logging. Primarily the events I have in mind are combination of the two aforementioned things: improvement of current solution and some progress over the loops.
The Implementation I have in mind is to write a simple
StatLogger that would be responsible for outputting the data to the log stream. The main program would just issue a
log() method whenever an event occurs. The events can occur simultaneously and it does not make sense to log the state twice, so the class would be responsible for not outputting the stats twice.
My questions are:
- Is this a good or common approach and pattern?
- Is it better to pass a handler to the number cruncher object, so the logger can get all the interesting stats like
StatLogger::log(const RunEnviroment *this), or is better to make another structure to hold all the interesting stats and make a
StatLogger::log(const Stats& stats)method.
- If my approach is bad, what are common solutions in similar cases?