I think that you simply need a better logging scheme.
If there is a performance issue:
- Buy a faster HDD, ideally dedicated to logging
- Send the traces, in UDP or similar, to a server whose sole role is logging, you may split the logs onto several servers if the volume is that high
- Use macros for logging, and strip the lower levels from the code in Release mode
If it's a storage issue:
- Archive! We compress logs on the fly. The files are rotated often (50 MB uncompressed) and bear a timestamp so that they can be organized and searched
- Move! Only keep the most recent files in the current directory, move the older files onto a cheaper disk
- Remove! Log files are not meant to live forever.
In both cases, you might want to make an audit of your logs, it may be that your logging practices are not adapted to the environment you're evolving in.
For example, where I work we have servers that process a few hundreds of transactions per second. The logging scheme is adapted:
- for a regular transaction (where everything went fine), you have a single line of log (at most two, one for the beginning and another for the end), summing up (minimally) what the transaction was about
- for an erroneous transaction, you get a line (tagged ERROR) for each error that occurred (if several)
- when we need to debug, this volume is multiplied by about 10 to 20
- in local (development), this volume is multiplied by about 100 to 200 (the supplementary traces are removed by the preprocessor in Release mode)
Some of our most verbose applications spew 50MB files twice per second (which is only ~15MB once compressed). Their logs are kept for 14 days still.
When a client calls you, asking why the transaction that he did 3 days ago at about 14h or maybe 15h (local time for him, of course) failed, you just cannot answer without logs. And no he cannot reproduce the issue, the data have changed since then.