Live data from Hacker News

Don't use ENV variables for secret data (2017)

diogomonica.com

61–70 of 147 posts

Re: Don't use ENV variables for secret data (2017)

#61
post #10

Earlier quoted context omitted.

Why is that? Also, as an aside: The very premise of plaintext credentials for computer-computer database connections always seemed strange to me. Maybe I'm just not knowledgeable enough here, but I wish the standard for database credentials was key-based.

> Why is that? You don't want to accidentally commit your credentials to github and have the world see them. At least if they're in ENV they stay private as long as your environment does.

What about the global .gitconfig?

One of the basic key value pairs is the GPG signingkey, which is usually stored in cleartext in the aforementioned file. Although the credential.helper is in my Keychain (iCloud-backed 2FA)

In theory someone could copy this and try to sign commits as me, but I have to think this value is unique and they would get rejected if they tried to use it.

My login credentials are 2FA as well, at least on unknown machines, so they would be prompted there as well.

Personal Access Tokens for the CLI would be another way to prevent nefarious things from happening.

Re: Don't use ENV variables for secret data (2017)

#62
post #48

Earlier quoted context omitted.

Why is that? Also, as an aside: The very premise of plaintext credentials for computer-computer database connections always seemed strange to me. Maybe I'm just not knowledgeable enough here, but I wish the standard for database credentials was key-based.

Lots of reasons: - passwords would end up in version control repositories. A whole article could be written on this point alone but to summarise: those credentials will then be in your projects history forever more (or until nuking the history becomes more important than keeping the history) - you can’t then change the credentials easily without having to push a new version of the application - you expose the passwor…

>those credentials will then be in your projects history forever more (or until nuking the history becomes more important than keeping the history)

With Git, it's not "nuking" so much as "retroactively creating a different timeline in which the secret wasn't shared." Still a pain in the ass if there's more than one developer.

Re: Don't use ENV variables for secret data (2017)

#64
post #37

Great :-( Had to read to the end of the description of why environment variables are bad to discover that it is effectively an advertisement for Docker. I don't use Docker so the article told me pretty much nothing that wasn't fairly obvious already, although it is a valuable reminder.

This is not Docker related. If an application spawns a sub-process, that sub-process will inherit all environment variables. Which might be fine or might not be, e.g. if the spawned application is user controlled. Also tools such as Airbrake or Sentry often send all your ENV variables to the error collection server, effectively exposing your secret values. Most such tools offer to filter variables, but that's in my e…

> If an application spawns a sub-process, that sub-process will inherit all environment variables.

...“by default”, specifying the environment is something you do when creating a new process.

Re: Don't use ENV variables for secret data (2017)

#65

The way I got around this was to store secrets in Google KMS encrypted files in Google Cloud Storage. The KMS key and encrypted files share the same name and can be accessed by that name programmatically. This secret storage method works really well for me and lets you easily access & manage secrets across all environments. It's so convenient, I sometimes even use this system as a simple key/value store.

That seems nice, but won't you be pwned the day that GCS accidentally loses your files, or your entire Google account gets locked because its algorithm mistakes it as a bot? The danger of that sounds greater than the possibility that your environment variables get leaked.

GCS seems like overkill, not sure why the op went that route. You can just store the encrypted secrets in the code and decrypt them with kms at start up.

Re: Don't use ENV variables for secret data (2017)

#66

I am progressively adopting Hashicorp Vault as the secrets manager of choice. It can be used by a variety of different scenarios -- directly into the application using AppRole, with Terraform using it's secrets provider, by developers, during vault authentication when they get their secrets and access regenerated. This way I am not bound to docker swarm, or keywhiz, or god forbid AWS Secrets Manager. As of now, I am…

Vault looks cool, but looking at the reference architecture [1] my guts tell me it's much easier to fuck up setting up a Vault cluster than environment variables.

[1] https://learn.hashicorp.com/vault/operations/ops-reference-a...

Re: Don't use ENV variables for secret data (2017)

#67
post #7

I don't agree at all. The reasons in the article all seem like "envs are bad because if you make a mistake you can expose them". This is not exclusive to envs, it applies to all secrets, independent of the medium used to make it available to the process using it. In my experience, if you prevent using envs for secrets (as docker swarm does) all you get is a disgruntled programmer reading the contents of a secret file…

There are actually key/value stores that solve this securely, such as Hashicorp Vault.

The issue isn’t that it can’t be done but more that most people either don’t already know it can be done or don’t want to invest in the infrastructure to do it.

Regarding the latter point, for self hosted solutions I can sympathise a little and it’s really a question of risk analysis. But most cloud computing services do offer their own secrets management service.

(not affiliated with Hashicorp and other services exist).

Re: Don't use ENV variables for secret data (2017)

#68
Are secrets specific to docker-swarm? If you're using plain docker, I imagine this wouldn't work?

k8s has its own secrets system built into deployments. I haven't used it though. I've been at shops that use k8s+vault, and other places that uses marathon/DCOS+consoul.

If you're on AWS you can use pod2iam in a k8s cluster and then use the SSM parameter store to encrypt/decrypt secrets based on pod roles. I'm sure Google Cloud must have similar services.

The most agnostic way would be to mount in a file or volume at runtime. It's still accessible to the process, but just via the filesystem and not via environment variables. You still need to program with security in mind, but it's less likely for inadvertent leaks; basic layers of security. From there you could use something that encrypts that mount at rest and decrypt it when you start the container.

> Environment variables are passed down to child processes, which allows for unintended access.

Doesn't this depend on how you create the new process? fork() would keep a copy of the env in both parent/child processes and exec would keep the env because it replaces the current process. But if you start a process using something like the subprocessing module in Python, it would give you a fresh shell for that process, right?

Re: Don't use ENV variables for secret data (2017)

#69
post #20

Earlier quoted context omitted.

Use a config file not checked into git

This comes back to where does the config file live and how is it managed?

The plain config file must live in memory (tmpfs). The encrypted config file on a hard drive that is accessed only to decrypt the file to memory.

Re: Don't use ENV variables for secret data (2017)

#70
I came across a blogpost describing this workflow recently and I'm curious to hear HN opinions about it. Any pitfalls?

https://matthewdowney.github.io/encrypting-keys-in-clojure-a...

1. Generate a new set of API keys.

2. Read my encrypted map of keys from disk, decrypt it with a passphrase, assoc in the new key & secret, encrypt it again, and write it to disk.

3. At the entry point for my application, use (.readPassword (System/console)) to securely read in the passphrase, and then use it to decrypt the key file and read it into a Clojure map.

4. Instead of passing the key map around (allowing it to potentially escape into a debug log, or be printed at the REPL if I do something dumb), the top level code of my application passes the credentials into a signer-factory for each api that closes over the credentials.

    ;; The factory is shaped something like this
    (defn request-signer-factory 
      [{:keys [key secret]]
      (fn [request-to-sign]
        (sign-request request-to-sign key secret)))
       
    ;; Then an API endpoint looks like this
    (defn place-order! 
      [signer {:keys [price qty side market post-only?]}]
      (let [request (comment "Format the order data for the exchange")
            signed (singer request)]
        (do-http-request! signed)))
I like this workflow more than others which are centered around only encrypting credentials inside of your Git repository, and decrypting them when you clone / pull, because it means that not even on my development machine are keys just sitting around in plaintext.
Post reply on HN