Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Question should be clear from its title. For example Apache saves its access and error logs in files instead of RDBMS no matter on how large or small scale it is being utilized.

For RDMS we just have to write SQL queries and it will do the work while for files we must decide a particular format and then write regex or may be parsers to manipulate them. And those might even fail in particular circumstances if great care was not paid.

Yet everyone seems to prefer filesystem for maintaining the logs. I am not biased against any of these methods but I would like to know why it is practiced like this. Is it speed or maintainability or something else?

share|improve this question
6  
So how would you log DB errors (db unavailable for example) if your logging system logs to a DB? –  Marjan Venema Jul 12 '11 at 12:13
5  
@Marjan How would I log Filesystem errors if it fails?! –  Yasir Jul 12 '11 at 12:17
2  
Quite true, but if that fails, chances are your DB is inaccessible as well... After all, where / how would it write to its tables without the file system? –  Marjan Venema Jul 12 '11 at 12:30
2  
@Yasir: Send all log messages to a syslog server before logging to the filesystem :) –  staticx Jul 12 '11 at 12:35

5 Answers 5

up vote 18 down vote accepted

A) Too many things can fail with the database, and logging this failures is important too.

B) Unless you have a database system allowing autonomous transactions (or no transactions at all), logging would require a seperate connection so a rollback or commit in logging doesn't interfere with rollback or commit in the application.

C) Many things worth logging happen during startup, i.e. possibly before the database connection has been established.

share|improve this answer

I've seen logs written to the DB before (and sometimes you get configurable options for logging, where trace goes to file, errors to DB, fatals to Windows Event log).

The main reasons are speed and size, enabling some tracing can produce vast, vast qualtities of logging - I've trawled through log files gigabytes in size. The other main reason is that reading the logs needs to be sequential, there's no real need to query the log, except to find a certain error or entry - and find-in-file works perfectly well for that.

share|improve this answer
    
But I have a confusion for this. My notepad, wordpad, gedit or notepad++ or any web-browser won't be happy opening a file 4GB in size. The same browser, however, will be able to show me a list of thousand pages, each containing 500 records printed. Right? –  Yasir Jul 12 '11 at 12:14
4  
@Yasir because you are using editors that try to load the whole file in memory. Try to use a smarter editor that is able to 'stream' the big file. Vim is a good example. –  Chaker Nakhli Jul 12 '11 at 12:21
4  
@Yasir: This is true, but you are trying to optimize the wrong thing. The vast majority of the time, logs are written and never read. So you make the creation of logs very fast because it is the common case. –  unholysampler Jul 12 '11 at 12:45

Speed is one reason; others are:

  • Eliminating points of failure. A filesystem rarely fails under conditions where a DBMS wouldn't, but there are lots and lots of error conditions in databases that simple don't exist in filesystems.
  • Low-tech accessibility. If things go really really bad, you can boot into a rescue shell, or mount the disk on a different system, and still have adequate tools available to inspect log files. If it's a database, you're nowhere without a database server running.
share|improve this answer

First off.

And those might even fail in particular circumstances if great care was not paid.

Database transactions can't fail when you are not careful?

Writing to a text file has a number of benefits, the most important being

  • Text is human readable. Anyone can open up a log file with an basic text editor and see what the messages are. You don't need to understand how the database is organized.
  • Speed. Writing text to disc is much faster that a database service figuring out where the text goes in a database, writing it there, and ensuring the transaction completed.
share|improve this answer
    
Obviously any and everything may fail if we are not careful. But for this question I was refering to high level programmer. As a simple example, the programmer might want to separate values using a particular character. So his/her regex will work like a charm but will fail when the same character is contained inside a value block. This way he needs to take care of similar possible cases and he doesn't need to think about them if he were saving in DB. Also, can you please see my comment on gbjbaanb's answer? –  Yasir Jul 12 '11 at 12:21
1  
And if you are hand writing your SQL, you have the same problem. The difference being the write will fail (or corrupt your data) instead of annoying some developer slightly because his search string brought up some bad results. Yes, there are frameworks that mean you don't have to write SQL, but every extra layer slows down the process. And remember this is just logging. Every cycle you use to log is a cycle you are not using to do real work. –  unholysampler Jul 12 '11 at 12:51

Is it speed or maintainability or something else?

Speed.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.