Environment Variables Considered Harmful for Your Secrets
1–10 of 120 posts
Re: Environment Variables Considered Harmful for Your Secrets
#2Fundamentally, any secrets you store will have some mode of access - there's a downside to each and every way of distributing them.
If you're shelling out to commands you think might snarf credentials, the environment is easy for them to pick it out of, but if they're running as the same user then they could probably read the secrets from the config file. If they aren't running as the same user, you need a way of passing in the secrets - and we tend to come back to environment variables..
The good practice here is just to reset the environment when calling shell commands, as he notes. It's not hard to do.
Re: Environment Variables Considered Harmful for Your Secrets
#3I've always been uncomfortable with the "store config in the environment" part of the 12 factor app thing, since it does imply storing things like database passwords and such, and the argument is that those shouldn't be in files. But, filesystem permissions are reasonably flexible and are easy to reason about (unlike the potential visibility of ENV).
I also don't really buy the arguments for ENV storage of even non-sensitive data. There's just not really any good reason to do so; your config has to be put in place by some tool, even if it is in ENV; why not make your tool produce a file, with reasonable permissions in a well-defined default location? The 12 Factor App article seems to believe that config files live in your source tree, and are thus easily accidentally checked into revision control. That's not where my config files live. My config files live in /etc; or, if I want it to isolate a service to a user, I make a /home/user/etc directory.
One could say, "Don't store passwords in the revision control system alongside your source." And that would be reasonable. But, there's no reason to throw the baby out with the bath water.
Re: Environment Variables Considered Harmful for Your Secrets
#4We had to migrate our software from single tenant (per machine) to multiple tenant for our cloud offering, on a 11 year old code base.
We used Michael's trick: environment variables pointing to config files works unbelievably well if you ever need to implement a multiple tenant cloud offering.
So apart from the security aspect, there's the aspect that it is a more versatile design.
Re: Environment Variables Considered Harmful for Your Secrets
#5Ultimately, secrets need to live somewhere and need to be accessed as plain text. Just make sure that the access as small window is as possible, and try to obliterate it after use, if possible.
If one absolutely needs to centralize secrets (TLS/SSL private keys, entropy sources, etc.) (at risk of SPOF or some HA setup), use some PSK style setup that delivers them directly, out-of-band (via separate NICs) or prioritized ahead of regular traffic. Keep it simple. Otherwise, prefer something like zookeeper with encrypted secrets (again PSK keying per box). Try to not deploy the same secret on every box, if possible. Also, try to avoid magic secrets if you can too (remove all local password hashes, use only auth keys).
If you're uncomfortable with plaintext secrets, encrypt them (as end-to-end as possible) and require an out-of-band decryption key at the last possible moment.
It's like having a secure document viewing system... ultimately, someone will need to browse just enough of the plaintext version, or it's not a document viewing system.
Re: Environment Variables Considered Harmful for Your Secrets
#6I got bit by env vars a few years back, but due to performance issues in getenv() and ended up writing a whole bunch of PHP magic to ship config files safely and fast.
With pecl/hidef, I can hook a bunch of text .ini files into the PHP engine and defines constants as each requests comes in.
Originally, it was written for a nearly static website, which was spending a ridiculous amount of time defining constants, which rolled over every 30 minutes.
Plus those .ini files were only readable to root, which reads it and then forks off the unprivileged processes.
But with the hidef.per_request_ini, I could hook that into the Apache2 vhost settings, so that the exact same code would operate with different constants across vhosts without changing any code between them.
Used two different RPMs to push those two bits so that I could push code & critical config changes as a two-step process.
And with a bit of yum trickery (the yum-multiverse plugin), I could install two versions of code and one version of config, and other madness like that with the help of update-alternatives.
That served as an awesome A/B testing setup, to innoculate a new release with a fraction of users hitting a different vhost for the micro-service.
I'm rambling now, but the whole point is that you need per-request/per-user config overlay layers, for which env vars are horrible, slow and possibly printed everytime someone throws some diagnostics up.
Re: Environment Variables Considered Harmful for Your Secrets
#7I've always been uncomfortable with the "store config in the environment" part of the 12 factor app thing, since it does imply storing things like database passwords and such, and the argument is that those shouldn't be in files. But, filesystem permissions are reasonably flexible and are easy to reason about (unlike the potential visibility of ENV). I also don't really buy the arguments for ENV storage of even non-s…
Config is ultra-sensitive information. It's like not having the right .gitignore if you're backing up /etc in git or not having enough password log filters.
The issue is that ultimately, one has to trust some infrastructure as being of an ultimately trusted network (whether it's Puppet/Chef/cfe2+ or offline authoritative CA's)
Re: Environment Variables Considered Harmful for Your Secrets
#8Fundamentally, any secrets you store will have some mode of access - there's a downside to each and every way of distributing them. If you're shelling out to commands you think might snarf credentials, the environment is easy for them to pick it out of, but if they're running as the same user then they could probably read the secrets from the config file. If they aren't running as the same user, you need a way of pas…
You're right that a tool which runs under the same user could read your config file and thus could access to your secrets.
But there is one main difference: that tool would need to do so explicitly, with the intent of reading (and possibly exposing) your secrets. For me, that's a huge difference from having the secrets being implicitly available to the process through the processes environment.
Re: Environment Variables Considered Harmful for Your Secrets
#9while i agree that storing api keys in the code repository is not the best idea, i am curious about the suggestion of moving it into chef configs.
wouldnt that, in turn, also be stored in a code repository, likely accessible in the same way as the main coe repo? then, this feels like a non-solution to me.
Re: Environment Variables Considered Harmful for Your Secrets
#10I've always been uncomfortable with the "store config in the environment" part of the 12 factor app thing, since it does imply storing things like database passwords and such, and the argument is that those shouldn't be in files. But, filesystem permissions are reasonably flexible and are easy to reason about (unlike the potential visibility of ENV). I also don't really buy the arguments for ENV storage of even non-s…
> filesystem permissions are reasonably flexible
Env makes things really flexible.For instance,you can call a program with env variables directly thus overriding the default ones,which simplify configuring applications. You dont have to have a test set-up,a production-setup or a staging set-up,just start a server or an app with different env variables in the command line.
You don't want your config or keys to depend on an OS,or a language. Finally Env variables can be restricted to a set of users,so third party process started with a different one cant access them.
I believe env variables are better than other solutions.