Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm about to write the company guidelines about what must never appear in logs (trace of an application). In fact, some developers try to include as many information as possible in trace, making it risky to store those logs, and extremely dangerous to submit them, especially when the customer doesn't know this information is stored, because she never cared about this and never read documentation and/or warning messages.

For example, when dealing with files, some developers are tempted to trace the names of the files. For example before appending file name to a directory, if we trace everything on error, it will be easy to notice for example that the appended name is too long, and that the bug in the code was to forget to check for the length of the concatenated string. It is helpful, but this is sensitive data, and must never appear in logs.

In the same way:

  • Passwords,
  • IP addresses and network information (MAC address, host name, etc.)¹,
  • Database accesses,
  • Direct input from user and stored business data

must never appear in trace.

So what other types of information must be banished from the logs? Are there any guidelines already written which I can use?


¹ Obviously, I'm not talking about things as IIS or Apache logs. What I'm talking about is the sort of information which is collected with the only intent to debug the application itself, not to trace the activity of untrusted entities.


Edit: Thank you for your answers and your comments. Since my question is not too precise, I'll try to answer the questions asked in the comments:

  • What I'm doing with the logs?

The logs of the application may be stored in memory, which means either in plain on hard disk on localhost, in a database, again in plain, or in Windows Events. In every case, the concern is that those sources may not be safe enough. For example, when a customer runs an application and this application stores logs in plain text file in temp directory, anybody who has a physical access to the PC can read those logs.

The logs of the application may also be sent through internet. For example, if a customer has an issue with an application, we can ask her to run this application in full-trace mode and to send us the log file. Also, some application may sent automatically the crash report to us (and even if there are warnings about sensitive data, in most cases customers don't read them).

  • Am I talking about specific fields?

No. I'm working on general business applications only, so the only sensitive data is business data. There is nothing related to health or other fields covered by specific regulations. But thank you to talk about that, I probably should take a look about those fields for some clues about what I can include in guidelines.

  • Isn't it easier to encrypt the data?

No. It would make every application much more difficult, especially if we want to use C# diagnostics and TraceSource. It would also require to manage authorizations, which is not the easiest think to do. Finally, if we are talking about the logs submitted to us from a customer, we must be able to read the logs, but without having access to sensitive data. So technically, it's easier to never include sensitive information in logs at all and to never care about how and where those logs are stored.

share|improve this question
    
Are you taking the log level into consideration? I mean, it might be okay to debug a file name, but not to info a file name. –  Jeremy Heiler Mar 2 '11 at 20:35
    
@Jeremy Heiler: I'm talking only about the log data which is either stored to hard disk (often in an unsecured way) and/or sent through internet to the developers of the application for debugging purposes. –  MainMa Mar 2 '11 at 20:57
    
Many applications write the log directly to disc, no matter what the logging level is. Unless your logging storage is a database table... If you're sending the logfiles to other developers via network, you could encrypt the file, yes? –  FrustratedWithFormsDesigner Mar 2 '11 at 21:07
1  
Without knowing what you're doing, it's really hard to figure what the sensitive data is. Do you have regulatory concerns (PCI or HIPAA or whatever)? –  David Thornley Mar 2 '11 at 21:35
    
If you can talk about the specific field you're working in, then ask at security.stackexchange.com as the security/compliance experts there can tell you about regulatory, legal or other security issues. –  user4051 Mar 2 '11 at 21:37
add comment

7 Answers

up vote 3 down vote accepted

I believe the best way to handle this is to treat log files as just another user interface to the application. The fact that the information are stored in text files does not make the content any different from the information displayed on the regular screens in the user interface.

Think about how you would protect the same information if it was to be displayed in a regular user interface. You would have to identify who the user was and then expose only the information that this user was entitled to see.

Information in log files must be treated in the same way. You must first answer exactly who should be entiteled to see the log file and what information they should be allowed to see.

Passing badly designed log files around is a huge security risk. I don't believe you will get a good solution to that by blacklisting some kind of data. A better strategy is to whitelist what could go in each log file and design the log files bottom up from that.

share|improve this answer
add comment

Yeah but.

To debug some issues you need real data.

So you have to play a balance game: you must in fact discuss and agree with your main clients what they consider confidential or sensitive data and what not. If several clients disagree, take the worst case scenario for every aspect of it, unless you can justify it to those clients that might go overboard in labelling everything sensitive.

I have worked in air traffic control, finance and banking. In every situation there are sensitive data. There are tasks for which it is unavoidable to handle sensitive data, in those cases you need to make as sure as you can that you work with trustworthy people. This risk can be somewhat mitigated by legal clauses to be agreed upon before accessing such data (non disclosure agreements, use of data only for valid business reasons, limited access to data, prosecution penalties for failing to respect or enforce the agreements...) - and relevant processes that make it possible to track those things.

If data is critical then you have to pay the price in setting up systems that protect the integrity, coherence and security of this data.

That said you are right to ask the question 'which data'. Truly you answer it yourself: most of it is business related. So ask your clients if you can't answer yourself - bearing in mind all the above and retaining some way to identify and fix issues that may crop up.

share|improve this answer
    
I've worked with live mortgage data, which had a lot of things I could have misused. It was interesting that I wasn't vetted for security or asked to sign anything specific. The big difference between that and places where I've worked with less sensitive data was the lack of an office lottery pool. –  David Thornley Apr 27 '11 at 18:21
add comment

I'd say, log messages that seem funny while you're coding then. You won't find them funny down the line and they'll be there forever - just like that ill-advised blog/facebook/twiter post!

Keep log messages dull :)

share|improve this answer
add comment

Personally-identifiable health information covered by the Health Insurance Portability and Accountability Act of 1996 (HIPAA). This article lists the following examples:

  • Health care claims or health care encounter information, such as documentation of doctor's visits and notes made by physicians and other provider staff;
  • Health care payment and remittance advice;
  • Coordination of health care benefits;
  • Health care claim status;
  • Enrollment and disenrollment in a health plan;
  • Eligibility for a health plan;
  • Health plan premium payments;
  • Referral certifications and authorization;
  • First report of injury;
  • Health claims attachments.
share|improve this answer
add comment

if security is a concern, encrypt the log file

share|improve this answer
add comment

Off the top of my head....

Credit card information should not be in logs. SSN (or SIN) data should not be in logs.

...of course there are exceptions, if you should happen to work for some central data store for a credit card company or a government agency that manages SIN data, then you might have to log it, because it's the main meat of what you're processing/managing.

share|improve this answer
add comment

Credit card information should never be logged.

ID numbers (such as SSN in the US or Teudat Zehut # in Israel).

Network computer names, network share paths.

share|improve this answer
    
The PCI-DSS standard prohibits storing the card number in any way, shape or form. –  Tangurena Mar 2 '11 at 22:08
    
@Tangurena not true, they do allow storage but require that it be properly protected. (PCI-DSS Requirements and Security Assessment V2.0 October 2010, Requirement 3) –  Newtopian Mar 3 '11 at 5:29
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.