Don't use ENV variables for secret data (2017) • Hacker News Board

Re: Don't use ENV variables for secret data (2017)

#101

Post by katnegermis » Mon, Jul 13, 2020, 3:00 PM UTC

oceanfloor wrote: post #87

gchamonlive wrote: post #31

I am progressively adopting Hashicorp Vault as the secrets manager of choice. It can be used by a variety of different scenarios -- directly into the application using AppRole, with Terraform using it's secrets provider, by developers, during vault authentication when they get their secrets and access regenerated. This way I am not bound to docker swarm, or keywhiz, or god forbid AWS Secrets Manager. As of now, I am…

What's wrong with AWS Secrets Manager? If I'm already working 100% in Amazon, I'm tempted to use Secrets Manager rather than justify the cost in hours to deploy and maintain a Vault cluster. Interested in your opinion.

Not OP, but I've recently worked with AWS Parameter Store a lot, and I've been very happy with it so far. We store all configuration (including secrets) using it.

We started by using https://github.com/segmentio/chamber/, but because of the way we decided to structure our secrets we decided to write a clone of it ourselves (https://github.com/micvbang/confman-go).

In practice, we have a client written in Python that grabs configuration from Parameter Store and puts it into the process' ENV variables at startup. This allows us to avoid vendor lockin in the rest of our code, since we still just fetch all config from the environment. I like it so far :)

Re: Don't use ENV variables for secret data (2017)

#102

Post by [deleted] » Mon, Jul 13, 2020, 3:11 PM UTC

laumars wrote: post #67

ablanco wrote: post #7

I don't agree at all. The reasons in the article all seem like "envs are bad because if you make a mistake you can expose them". This is not exclusive to envs, it applies to all secrets, independent of the medium used to make it available to the process using it. In my experience, if you prevent using envs for secrets (as docker swarm does) all you get is a disgruntled programmer reading the contents of a secret file…

There are actually key/value stores that solve this securely, such as Hashicorp Vault. The issue isn’t that it can’t be done but more that most people either don’t already know it can be done or don’t want to invest in the infrastructure to do it. Regarding the latter point, for self hosted solutions I can sympathise a little and it’s really a question of risk analysis. But most cloud computing services do offer thei…

[deleted]

Re: Don't use ENV variables for secret data (2017)

#103

Post by tyingq » Mon, Jul 13, 2020, 3:12 PM UTC

ptx wrote: post #78

tyingq wrote: post #63

Is transferring them to memory in your startup routine, then doing unsetenv() a reasonable mitigation? It seems like it addresses several of the listed concerns. It's not perfect, of course, but perhaps better, and straightforward.

I don't understand why we have to put them into the environment in the first place (and then make sure we scrub it). Isn't it just as easy to read the secret from a file?

There are setups like batch schedulers and lambdas/serverless where using environment variables is pretty standard.

Re: Don't use ENV variables for secret data (2017)

#104

Post by tyingq » Mon, Jul 13, 2020, 3:15 PM UTC

michaelt wrote: post #71

ablanco wrote: post #7

I don't agree at all. The reasons in the article all seem like "envs are bad because if you make a mistake you can expose them". This is not exclusive to envs, it applies to all secrets, independent of the medium used to make it available to the process using it. In my experience, if you prevent using envs for secrets (as docker swarm does) all you get is a disgruntled programmer reading the contents of a secret file…

I think what the author means is environment variables are particularly vulnerable to being logged by accident, because: 1. They're stored right next to variables like PATH, JAVA_HOME, LC_ALL and PYTHONPATH which people might plausibly decide to log out every time 2. They'll get printed any time someone writes a shell script with set -x then uses the environment variable. 3. They'll probably end up in your developers…

Also mentioned in the article, they pass down to forked processes by default.

Re: Don't use ENV variables for secret data (2017)

#105

Post by tyingq » Mon, Jul 13, 2020, 3:22 PM UTC

gchamonlive wrote: post #91

oceanfloor wrote: post #87

Earlier quoted context omitted.

What's wrong with AWS Secrets Manager? If I'm already working 100% in Amazon, I'm tempted to use Secrets Manager rather than justify the cost in hours to deploy and maintain a Vault cluster. Interested in your opinion.

Cloud lock-in (not being able for example to manage cross-platform credentials) and cost. By itself AWS Secrets Manager is fine, but as with everything AWS, billing is pretty opaque. If you couple this with a poorly planed Lambda setup, costs can potentially uncontrollably scale. For what you are paying, AWS Secrets Manager seems too barebones/rigid.

Lock in shouldn't be too bad. It's create, get, set, list, delete, describe. I imagine the API footprint isn't much different if you switch over to something else. Most apps will just use get().

Re: Don't use ENV variables for secret data (2017)

#106

Post by gpmcadam » Mon, Jul 13, 2020, 3:38 PM UTC

linkdd wrote: post #58

gpmcadam wrote: post #50

Earlier quoted context omitted.

How do you share the token to access Vault to your code?

Using AppRoles and consul-template. And if you're on Kubernetes, you can use https://github.com/sethvargo/vault-kubernetes-authenticator in an init container.

And how does k8s pass that value to the app?

Re: Don't use ENV variables for secret data (2017)

#107

Post by hobbescotch » Mon, Jul 13, 2020, 3:51 PM UTC

ravenstine wrote: post #42

hobbescotch wrote: post #9

The way I got around this was to store secrets in Google KMS encrypted files in Google Cloud Storage. The KMS key and encrypted files share the same name and can be accessed by that name programmatically. This secret storage method works really well for me and lets you easily access & manage secrets across all environments. It's so convenient, I sometimes even use this system as a simple key/value store.

That seems nice, but won't you be pwned the day that GCS accidentally loses your files, or your entire Google account gets locked because its algorithm mistakes it as a bot? The danger of that sounds greater than the possibility that your environment variables get leaked.

They get backed up

Re: Don't use ENV variables for secret data (2017)

#108

Post by joshspankit » Mon, Jul 13, 2020, 4:03 PM UTC

Oddly not talked about much:

In a lot of cases, if an attacker gets even limited access to the application environment, dumping the ENV variables is trivial as they were never intended to be secure.

`process.env`, `printenv` (including php attacks), `ENV`, etc

Re: Don't use ENV variables for secret data (2017)

#109

Post by joshspankit » Mon, Jul 13, 2020, 4:05 PM UTC

photon12 wrote: post #75

Also this pattern makes path traversal vulnerabilities (a thing not uncommon in web frameworks) have the potential to allow for privilege escalation on Linux via the /proc/self/environ file. I've been on a pentest where a recently disclosed path traversal bug in Rails was not patched in the environment I was testing and I thought I would get at least some credentials from at least one service, but every host used a d…

Any chance they documented that setup publicly? Would be interested to dive in to how all that works and gain any new insights

Re: Don't use ENV variables for secret data (2017)

#110

Post by falcolas » Mon, Jul 13, 2020, 4:19 PM UTC

oceanfloor wrote: post #87

gchamonlive wrote: post #31

I am progressively adopting Hashicorp Vault as the secrets manager of choice. It can be used by a variety of different scenarios -- directly into the application using AppRole, with Terraform using it's secrets provider, by developers, during vault authentication when they get their secrets and access regenerated. This way I am not bound to docker swarm, or keywhiz, or god forbid AWS Secrets Manager. As of now, I am…

What's wrong with AWS Secrets Manager? If I'm already working 100% in Amazon, I'm tempted to use Secrets Manager rather than justify the cost in hours to deploy and maintain a Vault cluster. Interested in your opinion.

IMO, it gets expensive quickly, and it was designed around AWS' use case, which involves credentials that roll on a regular basis. Outside of RDS, its value for the price goes downhill quickly.

To a sister comment who mentions parameter store - that UI is the biggest leaking bag of horse manure I've ever had the displeasure of using. We made the mistake of using it, and moved over to Vault at the first available opportunity. Vault isn't a silver bullet either, but the UI is at least usable.