Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I am well aware of best practices for storing user passwords:

  • Never store passwords in plaintext
  • Never store encrypted passwords; always store password hashes
  • Always salt your password hash to deter brute force attacks
  • Always use unique salts to deter attacks using rainbow tables.

However, I have recently came upon on a potentially legitimate use case for keeping plaintext (or plaintext-accessible encrypted) user passwords: fraud detection. For example, if we have a system that needs to restrict purchases to 1 (or a small number) per person, we have observed that fraudsters often use the same password over and over, so if we have detected fraudulent purchases on one account, we can use the fact that password matches as a useful signal in a fraud detection system.

But implementing such a system requires breaking all the best practices for password storage.

Question: If it is determined that the fraud-detection ability of being to retrieve passwords is necessary, what are the best practices for storing passwords securely but in a way that is plaintext-accessible?

share|improve this question
8  
When first entering the password, how intensive is it for you to check that password against all the hashed and salted passwords you have stored already? –  Martijn Pieters Jul 31 '12 at 7:45
6  
Just wanted to point out that if you do this, you don't have to store plain text passwords. You can do this by simply throwing away the last requirement you list and hashing all passwords with the same salt. Then you can simply compare hashes. –  James Jul 31 '12 at 7:48
6  
Of course, allowing DB operators to know that two users have the same password is itself considered a security flaw. –  Brian Jul 31 '12 at 13:22
6  
This is a terrible way to detect 'user collision'. Especially given the number of 'uneducated' users who will all use 'password'. If I found a website that required 'unique' passwords, I'd probably use that to find out who used an existing password, and log in as that person. Note that even physical locations have problems with this - on the internet, without some independent way of identifying someone, you're just toast. –  Clockwork-Muse Jul 31 '12 at 17:05
4  
I know some ticketing services limit it by credit card number. Seems like getting multiple credit cards is a bigger problem than multiple passwords. –  JeffO Jul 31 '12 at 19:13
show 20 more comments

8 Answers 8

Making the passwords accesible in plaintext in one way or another is already a HUGE security flaw. This is one of the most important security components on any site anywhere. I believe you already know that.

My recommendation is to find another way to deal with those frauds. Exposing user passwords like that is just not worth it. You win a little and lose a lot. If you do it, you'll regret it later.

EDIT: I guess it depends a lot on what people can purchase on that site. You can check against the same passwords, the same IP, the same browser etc. but if someone wants to make 2 or more purchases, they can. If the goods we're talking about are actual objects, then in my opinion the best way to detect such "frauds" are by comparing shipping addresses. Someone may use a different account/password/browser (that is easy), but they will rarely change their address to accomplish something like this.

share|improve this answer
1  
+1 for just saying how it is "there are better ways for detecting fraud." –  Ramhound Jul 31 '12 at 16:14
    
Well, if you never have a security breach, and you prevent millions of dollars in fraud, I'd say you win a LOT and lose nothing. –  nohat Jul 31 '12 at 19:06
6  
"never have a security breach" is one damn big bet. –  DaveE Aug 1 '12 at 20:15
    
I'm just suggesting that the calculus of risk in this scenario is more complicated than this answer implies. –  nohat Aug 2 '12 at 18:24
2  
@nohat: These consequences: mashable.com/2012/06/21/linkedin-lawsuit –  StriplingWarrior Aug 2 '12 at 22:21
show 2 more comments

In this case you could, upon registration, generate a hash of the entered password for every salt used so far, then check these salted and hashed passwords against all the passwords already stored.

You could even cache all the salts used so far, this is potentially a smaller list than having to loop through all passwords every time to collect these.

If that is too expensive to compute on each registration, limit the number of salts you generate (pick randomly from a limited list) to reduce calculation time. That'll still be safer (by far) than storing plain text passwords.

Note that if this is just for the sake of fraud detection, a security breach and resulting (huge) negative publicity is going to cost you much more, wiping out any advantages you gained from catching a few fraudsters this way.

share|improve this answer
4  
Just for perspective, a security breach is only a potential loss of revenue, whereas fraud is actual loss of revenue. If your business is losing, say, hundreds of thousands of dollars a month due to fraud, management may be willing to take risks to stem that loss. –  nohat Jul 31 '12 at 18:47
    
I don't think people give a damn. Otherwise password storage criteria would be outlined in EULA. But yes, comparing hashes is the same thing as comparing real source data, it can even be faster. Minus few theoretically possible collisions. –  Coder Aug 1 '12 at 19:38
3  
@nohat The simplification of this issue down to concerns of revenue stream seems really crass and unprofessional to me. When you store your users' passwords in plaintext, you are not just putting your company's reputation (and revenue stream) at risk, you are putting your users' security at risk, too. If you care so little about your users that you are willing to risk their security on behalf of your own revenue stream, you should be ashamed of yourself. –  Eric King Aug 1 '12 at 22:16
3  
@nohat Well I do happen to believe that morals come into play when talking about professionalism. And I also happen to believe that your professional responsibility extends not just to management ("I was just following orders") but also to the customers. –  Eric King Aug 2 '12 at 16:52
1  
No business needs to store plaintext passwords. There are always other options. Sorry to be a Sith about it, but if you are presenting to management, why would you even present an option to them that is unnecessarily insecure? How do you expect to catch anyone anyway? Everyone uses the same 150 passwords anyway, so your possibility of false collision is very high; just go look through a passwords list. –  Jordan Aug 2 '12 at 19:18
show 7 more comments

Is the risk of the bad PR if your passwords leak worth it for the slight fraud-fighting gain you get? Personally, I wouldn't.

If you really have to "have a system that needs to restrict purchases to 1 (or a small number) per person" (which sounds an impossible task) is there not something else you can use?

  • Same IP address?
  • Same browser?
share|improve this answer
    
Sure, those are also useful signals for fraud detection, but if you are dealing with a lot of fraud you need to use all available to signals to detect it and maximize the loss. –  nohat Jul 31 '12 at 19:08
2  
Like I say, it's a trade-off. The cost of fraud may be high, but that's nothing compared to the cost to your business if you actually leak passwords. It's up to you. I wouldn't. –  James Aug 1 '12 at 11:45
    
Even in the case of worst-case scenario password security breaches (LinkedIn, eHarmony, Last.fm) it's not clear that there have been substantial revenue impacts. –  nohat Aug 2 '12 at 15:16
    
Would love to see some fiqures about that. Tho I suspect the situation would be very different for a small retailer than it has been for those big players, so you have to make sure your comparing like with like. –  James Aug 3 '12 at 7:10
    
@James - Don't bother...Its clear the user doesn't actually want to hear us. –  Ramhound Aug 3 '12 at 12:40
show 4 more comments

If I understand you correctly you intend to use this information only internally within your organization to detect possible fraud?

Now I don't know if you encrypt and salt the password on the client side using JavaScript or something similar before sending it to the server. But if not, then you could store unencrypted password at another db server in an isolated network not accessible from outside and run the checks on that?

Regarding checking for IP and browser I would guess that most people doing these types of fraud work through different proxies to change their ip around and use incognito features to "hide" their browser.

Update: Some people here seem to miss the fact that a question was raised. It has nothing to do with our own personal believes of password treatment and best security practices.

If one agrees to what the person asking the question wants to do or not is not the point. The question was plain and simple:

Question: If it is determined that the fraud-detection ability of being to retrieve passwords is necessary, what are the best practices for storing passwords securely but in a way that is plaintext-accessible?

He doesn't ask IF it's 100% secure. He asks for practices of making it as secure as it can be.

Yes, it will most likely suck and be impossible to protect with a 100% success rate. But IF IT HAS TO BE DONE, how could he do it as secure as possible. There will ALWAYS be a way around anything not encrypted with a, as of yet, unbroken hash algorithm. It might be more or less dramatic and might require some C4 to get into a server room 100m bellow the surface and kill 10 armed guards hardwired to a system that kills them if they even try to leave the room (so that they themselves can't steal the information and leave).

But it's still not the point of the question. He wants to know how to store PLAIN TEXT passwords as securely as possible. From whom isn't very specified and if he is the only one handling this system (and we assume he can trust himself) the only ones he has to protect the data from are outside parties. If not, then internal parties will also have to be taken into account.

Does it suck to store sensitive data as plaintext? HELL YES! Does the person asking the question deserve to know that? Yes!

Do we answer his question by only stating that? NO!

share|improve this answer
3  
And then the old, old backup of that internal server is leaked, and a load of plain-text passwords that haven't been changed by their users in eons is out in the open. Not a good idea still. –  Martijn Pieters Jul 31 '12 at 8:08
    
There should of course be a rigorous internal security plan in place for this. But the hardest thing to protect against is the own employees, this is true. But if they work there they could probably gain access to user accounts, create back doors etc just as easy. A isolated server, with access only to a single user in a locked room etc. could add enough security. If only one person has access and security is upheld, only one person has to be trusted. But someone must indeed be trusted. When we have had other sensitive material we have had security check points and surveillance all over :) –  inquam Jul 31 '12 at 8:12
1  
@inquam - No matter how much security there is, if the passwords are not hashed, they will be leaked. I have to down vote this answer for even trying to suggest storing passwords as plaintext. If a single person can access the user's passwords then its a security risk not worth having. –  Ramhound Jul 31 '12 at 16:17
    
I'm not saying plain text is great. But IF this is what the person asking the question wishes to do, then it has to be done as securely as possible. Then I don't know, he might be the only person working with the application in question and trusting other people isn't an issue. I myself has NEVER stored passwords as plain text but have instead used advanced algorithms to try to link accounts together with combinations of behaviour and account information. But that's a WAY harder approach :). Worked fine for large banks and online gaming anti-fraud use though. –  inquam Jul 31 '12 at 18:10
1  
@inquam "rigorous internal security plan"? Try "No ... plan survives first contact with the enemy". –  DaveE Aug 1 '12 at 20:21
show 2 more comments

It's probably not a good idea to store plaintext passwords, but if you wanted to use "same password" as an indicator of fraud, here's how I'd do it:

  • When a user signs up, generate a hash using a different salt to the one you otherwise use.
  • Truncate to the first n bits.

So for each user you'd have two distinct hashes -- one with, say, 64 bytes. The other with 2 bytes, say. Theoretically, this will make your hashing 65,536 times weaker (not entirely confident with this; maybe somebody could support/refute).

You'd then be able to use this value as a weak indicator of fraud.

share|improve this answer
add comment

The only way I'd consider doing something like this is if you put a stand-alone machine in a locked room with only an RS232 connection between it and the web server. No network card, no wi-fi, no USB thumbdrives allowed.

Perform an unsalted hash on the web server, send only the hash and a user ID over the serial port, and get back a count of the number of users with matching hashes from the locked computer. Don't cache the result on the webserver!

The locked computer can have a database of all the hashes stored by user ID and can do the comparison to say how many match.

Even then, this is all dependent on the discipline of the company involved. You don't want to be poking holes in the air gap here.

share|improve this answer
add comment

assuming the "same password" is a good way to check for fraud, and considering that giving a way to revert the stored password into the plaintext is just a very bad idea, here's how I would allow for duplicate detection : I would still go with the "per user unique salt" scheme in order to identify my users.

In addition to that, I would also have a unique salt that's used before performing another hash on the password. All the passwords would go through hashing and be stored in a DB (not in the same table as the user table), along with the number of times it's been used. This would still allow a rainbow attack on this specific table, but you wouldn't have the user<->password link.

In other word, your attacker would be reduced to use a bruteforce dictionary attack (which he would perform anyway, and which would probably give him access to a large number of your accounts). If your user base is small, nothing prevents you from putting dummy rows in that table, just to increase the size of the dictionary your attacker would have to use.

The key here is that you don't care about which user has the newly typed password. The only thing that you'll have access to is "how many users have this password", which will allow you to reject the last subscriptions (no matter what, you can't reject subscriptions before a certain number of users have used the same password).

share|improve this answer
1  
This seems like a stop gap measure to prevent a problem that can be prevented. You can detect fraud in other ways, detecting the same password, won't prevent fraud. –  Ramhound Aug 3 '12 at 12:41
add comment

If you are set on doing this, the ideal solution seems to me to use a Bloom filter of the known "bad passwords".

The advantage is that even in a compromise, you only leak information about the passwords of known/suspected bad actors, and you never directly leak any password hashes. In addition, a dense Bloom filter is very fast to search and can be sized to have a low false positive rate. Also, if the Bloom filter is sized for a 1 in a million false positive rate, then even an attacker who has been able to steal your Bloom filter will find all of the real "bad passwords" plus one out of a million of his/her random password guesses.

Let's say you have a 65,536 entry Bloom filter using 8 different hash functions. This will require just 8 kilobytes to store, and will hold more than exp(ln(1e-6)/8) * 65536 / 8 = 1,457 "bad" passwords before you hit a 1 in a million false positive rate.

The downside is that after you mark an account as bad, you'll need to wait until the next login into that account to add the password to the Bloom filter (this only time you see cleartext passwords), and after you add a "bad password" to the Bloom filter, you won't discover other accounts sharing the "bad password" until the next time they login.

The first problem can be easily overcome by expiring the cryptographic tokens/login cookies of a bad account, tricking the attacker into logging in immediately. Crypto tokens/login cookies should always contain a message-authentication-code-protected creation timestamp. This allows you to force periodic login and also keep a per-user earliestValidToken timestamp that you can bump up to the current time when they change their password (or if you need to expire their existing tokens for some other reason).

The second problem can be overcome using a combination of several methods. (1) It's good to make users login every 2 weeks to keep them from forgetting their passwords (2) you probably mostly care about new account creation using "bad passwords" (3) assuming most accounts become spammy soon after creation, when you discover a spammy account, you can expire the crypto tokens/login cookies of all accounts created in the past N days (or maybe all accounts created after the spammy account).

As far as how to get your 8 hash functions, you need 8 16-bit hashes, or 128 bits of hash output to cut into 16-bit chunks. You'll want your hash functions to use site-specific keys, to minimize the possibility of cross-site use of a stolen Bloom filter. Two invocations of Siphash-2-4 using two different keys is probably the fastest way to get your 128 bits of hash values. SHA-1 HMAC (truncate MAC to 128 bits) or AES CMAC are also perfectly acceptable.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.