Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm working on a website that will allow users to log in using OAuth credentials from the likes of Twitter, Google, etc. To do this, I have to register with these various providers and get a super-secret API key that I have to protect with pledges against various body parts. If my key gets ganked, the part gets yanked.

The API key has to travel with my source, as it is used at runtime to perform authentication requests. In my case, the key must exist within the application in a configuration file or within the code itself. That isn't a problem when I build and publish from a single machine. However, when we throw source control into the mix, things get more complicated.

As I'm a cheap bastard, I'd much prefer to use free source control services such as TFS in the cloud or GitHub. This leaves me with a slight conundrum:

How can I keep my body intact when my API keys are in my code, and my code is available in a public repository?

I can think of a number of ways to handle this, but none of them are that satisfying.

  • I could remove all private info from code, and edit it back in after deployment. This would be a severe pain to implement (I won't detail the many ways), and isn't an option.
  • I could encrypt it. But as I have to decrypt it, anyone with the source could figure out how to do so. Pointless.
  • I could pay for private source control. LOL j/k spend money? Please.
  • I could use language features to segregate sensitive info from the rest of my source and therefore keep it from source control. This is what I'm doing now, but it could easily be screwed up by mistakenly checking in the secret file.

I'm really looking for a guaranteed way to ensure I don't share my privates with the world (except on snapchat) that will work smoothly through development, debugging and deployment and be foolproof as well. This is completely unrealistic. So what realistically can I do?

Technical details: VS2012, C# 4.5, source control is either going to be TF service or GitHub. Currently using a partial class to split the sensitive keys off in a separate .cs file that won't be added to source control. I think GitHub may have the advantage as .gitignore could be used to ensure that partial class file isn't checked in, but I've screwed that up before. Am hoping for a "oh, common issue, this is how you do it" but I may have to settle for "that doesn't suck as much as it could have", :/

share|improve this question
4  
You can make sure, that configuration file that holds your API key is not in source controlled directory, which will make it impossible to check it in in first place. –  David Sergey Jul 21 '13 at 20:24
11  
BitBucket.org has unlimited private repositories. Free. And gitHub repository importer (keeps history) –  Rob van der Veer Jul 21 '13 at 21:34
2  
@Dainius I don't trust my developers because I know them. Intimately. In fact, I'm intimate with myself at least... no, I'll let that one lie. But I know how easy it is to screw up, and how hard it will be to scrub history of said screwup. –  Will Jul 22 '13 at 13:19
4  
@Dainius: Yes. I look at every single character my team codes. Seriously. I have no choice. I can't code blindfolded. Not reliably, at least. But I do, because I am my team. I'm the I in TEAM. There's one developer, and it's me. I'm him. Yes. I'm the guy who is going to screw this up if he doesn't do it right. Me. –  Will Jul 22 '13 at 14:00
2  
@DonalFellows: Configuration files tend to live in the same project as the source code, and get published to the web along side of the built assemblies. –  Will Jul 22 '13 at 16:59

10 Answers 10

up vote 62 down vote accepted

Don't put your secret information in your code. Put it into a configuration file which is read by your code at startup. Configuration files shouldn't be put on version control, unless they are the "factory defaults", and then they shouldn't have any private information.

See also the question Version control and personal configuration file for how to do this well.

share|improve this answer
7  
@RobertHarvey by just not putting it on version control, adding an ignore rule when necessary. Anyone using the software has to build their own configuration file with their own API key. –  Philipp Jul 21 '13 at 20:47
6  
So when you go to build and create a distribution of your software, how are you sure that it ships with a configuration file? Unless you have some file with reasonable defaults, it's usually not reasonable to expect your user to go through a process of making a configuration file. –  Thomas Owens Jul 21 '13 at 21:13
2  
Well, factory defaults are one part, "installers" or "first run wizards" another one –  johannes Jul 21 '13 at 21:56
1  
If many users have their own installation, shouldn't they create and use their own API key? Multiple sites/installs using the same key is probably a bad idea. If it's just one install, then using a configuration file is not a big hassle. –  Mike Weller Jul 22 '13 at 11:53
5  
@Will, if you can't do this because of the impracticality of implementation details, then I'd say you simply don't have the proper tooling for deployment. Deployment using a non-comitted secret config file should be completely painless. I can't offer specific advice to you since I live in the Ruby ecosystem, not C#. But Ruby people tend to use Capistrano for automated deploys. I'm sure C# has its tool for automated deployment as well, and this should make the process easy. –  Ben Lee Jul 22 '13 at 19:02

Pure Git way

  • .gitignore included file with private data
  • Use a local branch, in which you replace TEMPLATE with DATA
  • Use smudge/clean filters, in which (local) filter's script perform bidirectional replacement TEMPLATE <-> DATA

Mercurial way

  • MQ-patch(es) on top of dummy code, which replace TEMPLATE with DATA (changesets are public, patch is private)
  • Keyword extension with specially designed keywords (expanded only in your working directory)

SCM-agnostic way

  • Have replacement of keywords as part of build/deploy process
share|improve this answer
    
Hmmm... The git advice is good, and your agnostic advice gives me a good idea... I can use build events to introduce the file into the publishing process, then remove it after, thus helping to ensuring that it won't be accidentally added to source control.. –  Will Jul 22 '13 at 13:15
2  
No, no and once again - no! ignoring files is good for adding some very specific customization to build process or something, but it shoud never be used for storing any secure data. Don't store secure data in repo, even if you are ignoring it. –  shabunc Jul 23 '13 at 22:22
1  
@shabunc - RTFM! Ignored file not stored in repo –  Lazy Badger Jul 24 '13 at 4:03
4  
@LazyBadger - I know pretty well it is ignored. I also know that, being in repo, there's ALWAYS chance that somebody neverthless mistakingly will add it somehow to repo. Some external config path is way better. –  shabunc Jul 24 '13 at 5:14
1  
@shabunc - good point on keeping config out of the SCM path. This is why, for example, Postgres allows you to bypass password checks by putting the password in a file. But they require that the password file be put in ~/.pgpass - which presumably is not a location that's very convenient to check into source control. They know, for automation, they have to give you a gun, but they work hard to keep you from shooting yourself in the foot with it.. –  Steve Midgley Dec 9 '13 at 23:31

You could put all the private/protected keys as system environment variables. Your configuration file will look like this:

private.key=#{systemEnvironment['PRIVATE_KEY']}

This is how we handle those cases and nothing goes into code. It works very well combined with different property files and profiles. We use different property files for different environments. In our local development environment we put the development keys in the property files to simplify the local setup:

private.key=A_DEVELOPMENT_LONG_KEY
share|improve this answer
    
This would be a reasonable solution if I can get it to work with my hosting option. Won't be environment variables, but perhaps some key/value configuration pairs that don't get wiped after publishing... –  Will Jul 22 '13 at 13:23
    
How about putting those environment variables in your build server before shipping to the live environment? That way you will have ready for production resource/configuration files. –  Ioannis Tzikas Jul 22 '13 at 16:28
    
The build server is the development machine, which is why I'm concerned about this info possibly getting checked into source control accidentally. –  Will Jul 22 '13 at 17:01
    
The problem with this may be that the environment is readable by anyone on the server. –  JasonG Apr 20 at 19:00
    
A user's envvars are readable only by the user or root. (Ancient Linux and AIX didn't do this however) –  bye Nov 22 at 1:10

You're not suppose to distribute that key with your application or store it in the source code repository. This question is asking how to do that, and that isn't what is normally done.

Mobile Web Application

For Android/iPhone the device should request the KEY from your own web service when the app is first run. The key is then stored in a safe location. Should the key be changed or revoked by the publisher. Your web service can publish a new key.

Hosted Web Application

Customers using a license of your software will have to manually input the key when first configuring the software. You can give everyone the same key, different keys or they get their own.

Published Source Code

You store your source code in a public repository but not the KEY. In the configuration of the file you add the lines *place key here*. When a developer uses your source code they make a copy of the sample.cfg file and add their own key.

You do not keep yourconfig.cfg file used for development or production in the repository.

share|improve this answer
1  
This question is asking how to do that no, it absolutely does NOT. The fact is that these keys need to be used by code, therefore be accessed by code, and that usually means via code or configuration files, which if they aren't in source together they are at least close by and may accidentally end up in source. The hosted web app is nonsensical, unfortunately. You didn't have to apply for an api key to log into StackOverflow via your (hypothetical) facebook account. place key here is a massive oversimplification that won't work in dev->pub environments as is described in the Q. –  Will Jul 23 '13 at 19:33
    
I've answered the question correctly, as have many others. The fact that you haven't accepted one of them implies you don't understand how to work with these keys. –  Mathew Foscarini Jul 23 '13 at 19:39
    
@matthew Oh, shit, I guess I don't know how SE works. I have to pick an answer now? Thanks for edifying me. Also, your answer is not that good. Its not bad, I didn't downvote it, but you didn't exactly understand the question you answered. Better luck next time. –  Will Jul 23 '13 at 23:02

I put secrets into encrypted file(s) which I then commit. The pass phrase is provided when the system launches, or it is stored in small file that I don't commit. It's nice that Emacs will cheerfully manage these encrypted files. For example, emacs init file includes: (load "secrets.el.gpg"), which just works - prompting me for the password on those rare occations when I start the editor. I don't worry about somebody breaking the encryption.

share|improve this answer
    
This is a great solution - I'm surprised you don't have more up votes. I work with a company that deals with student data, which is federally regulated in the US, so they have to be extra careful with credentials and secrets. They also are a big company so they need to use SCM for credentials so IT can find/manage them after engr builds them. Your solution is exactly what they do. They have decrypt key files that hold decrypt keys for dev/staging/prod/etc (one file for each). Then all the secrets are encrypted and checked into files. The decrypt files are used to get them in each environment. –  Steve Midgley Dec 9 '13 at 23:36
    
Well, in some sense encrypting the secret (API key in this case) only shifts the problem from not committing the secret data to not committing the pass phrase (which now becomes the secret data). But of course, asking for it on system launch is a good option. –  siegi Apr 27 at 11:42

Use environment variables for secret things that change for each server.

http://en.wikipedia.org/wiki/Environment_variable

How to use them is language dependent.

share|improve this answer
2  
Security through obscurity isn't a recommended approach for many. Would you care to elaborate upon your answer to be more clear? –  GlenH7 Jul 23 '13 at 19:32
1  
That's not obscurity, environment variables are only available to the user you added them, so all your credentials have the same protection of the user context your app is running. I updated the answer to include the concept of environment variables. Is that more clear? –  Filipe Giusti Jul 25 '13 at 19:10

Use encryption, but provide a master key at startup, as a password at the console, in a file only the process's user can read, or from a system-provided key store like Mac OS keychain or Windows key store.

For continuous delivery, you'll want various keys recorded somewhere. Configuration should be demarcated from code, but it makes a lot of sense to keep it under revision control.

share|improve this answer

Keep private information out of your source control. Create a non-loaded default for distribution, and have your VCS ignore the real one. Your installation process (whether manual, configure/build or wizard) should handle creating and populating the new file. Optionally modify permissions on the file to ensure only the required user (webserver?) can read it.

Benefits:

  • Doesn't assume development entity == production entity
  • Doesn't assume all collaborators/code reviewers are trusted
  • Prevent easy mistakes by keeping it out of version control
  • Easy to automate installs with custom configuration for QA/builds

If you are already doing this and are accidentally checking it in, add it to your project's .gitignore. This'll make it impossible to do again.

There are plenty of free Git hosts around that provide private repositories. Although you should never version your credentials, you can be cheap and have private repos too. ^_^

share|improve this answer

I think this is an issue everyone has had some trouble with at some point.

Here's a workflow I have used, which might work for you. It uses .gitignore with a twist:

  1. All configuration files go in a special folder (w/ sample config files - optional)
  2. All configuration files are included in .gitignore, so that they don't go public
  3. Setup a gitolite server (or your favorite git server) on a private box
  4. Add a repo with all the config files in the private server
  5. Add a script to copy config files to the special folder in the main repo (optional)

Now, you can clone the config repo to any development and deployment system. Just run the script to copy the files to the correct folder and you're done.

You still get all the GitHub candy, share your code with the world and the sensitive data are never in the main repo, so they don't go public. They are still only a pull and a copy away from any deployment system.

I use a 15$/year box for the private git server, but you can also setup one at home, per the cheapskate requirement ;-)

PS: You could also use a git submodule (http://git-scm.com/docs/git-submodule), but I always forget the commands, so quick & dirty rules!

share|improve this answer

Instead of having the OAuth key stored as raw data anywhere, why not run the string through some encryption algorithm, and store it as a salted hash? Then use a configuration file to restore it at runtime. That way the key isn't stored anywhere, whether it is stored on a development box, or the server itself.

You could even create an API such that your server automatically generates a new salted and hashed API key on a per request basis, that way not even your team can see the OAuth source.

Edit: Perhaps try the Stanford Javascript Crypto Library, it allows for some pretty secure symmetric encryption/decryption.

share|improve this answer
1  
Hashes are generally one way scrambles. There are symmetric encryption algorithms that would do as you suggest though. –  GlenH7 Jul 23 '13 at 20:46
1  
Dude, you can't unencrypt (easily) a hash. That's the whole point of hashes. This is for ME consuming somebody else's API, where they assign ME a secret key. My hashing it ensures (unless I choose a poor algo and crack it every time) that I can't use their API. –  Will Jul 23 '13 at 23:04

protected by GlenH7 Dec 9 '13 at 12:07

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.