Live data from Hacker News

Ask HN: How does your company manage its encryption keys?

news.ycombinator.com

141–150 of 246 posts

Re: Ask HN: How does your company manage its encryption keys?

#141

We use keybase

We also use Keybase. We wrote an extension to our existing lightweight management script, encpass.sh(https://github.com/plyint/encpass.sh), to be able to access/manage our secrets from the CLI and in our infrastructure scripts. The extension is encpass-keybase.sh (https://github.com/plyint/encpass.sh/blob/master/extensions/...) kept in the same repo.

The extension makes use of the per-user keys of Keybase. This allows you to manage access permissions using the Keybase GUI client. Once a user is added to the appropriate Keybase team, they will immediately have access to that teams secrets that are stored in the encrypted Git repos.

To use it you just need to download the 2 shell scripts to a directory in your path (e.g. /usr/local/bin/) or you can clone the git repo.

Re: Ask HN: How does your company manage its encryption keys?

#142

https://www.vaultproject.io/ We use Hashicorp's Vault product to manage SSH credentials, TLS certificates, as well as application secrets across thousands of users, tens of thousands of virtual machines, and hundreds of applications. We pay for the enterprise version, but the free version is more than capable for most needs. Avoid a password manager if you can, it leads to poor security practices and availability iss…

This just pushed the problem further down the stack. You should have keys to unlock vault when it is restarted. How do you secure those keys?

You can use GCPKMS (probably AWS KMS too) to unseal Vault automatically.

https://github.com/sethvargo/vault-on-gke/blob/master/terraf...

The KMS ring itself is only accessible by Vault. People with high enough privileges for our Vault GCP project could technically grant themselves access to it, but on day-to-day business, nobody can view the project.

At some other place, where we were using AWS, I wrote a script that would store encrypt unseal keys (need multiple due to shamir) via pgp using their keybase public key. IIRC you can store encrypted keys in Vault that can be accessed only for unsealing purposes (please correct me on this). When needing an unseal, the script on the user side would then decrypt the key and submit it to Vault for unsealing. It worked well enough, and it felt like being in the movie/game GoldenEye, but nowhere as slick as auto unseal.

Regardless of the setup, yes, at the end of the day any solution is really just pushing the problem further down the stack.

Re: Ask HN: How does your company manage its encryption keys?

#143
post #55

We use Bitwarden https://bitwarden.com/#organizations to store passwords as well as encryption secrets as a backup. Application repos have the encrypted secrets (meta) stored in their repos using Ansible-vault and the .vault_keys are stored in Bitwarden.

We do exactly the same.

Re: Ask HN: How does your company manage its encryption keys?

#147
post #73
post #66

The more complex your system grows, the more often it will fail and shoot you in the foot. I'd advise against systems like Hashicorp Vault - they just increase the complexity and while they have their merits in complex setups, you seem to be too small to be able to operate such a system. Have an offline backup printed along with the disaster recovery checklist and documentation and put them in a safe in your company…

Great answer, thank you. I agree with your point about testing the restore process, right now I'm trying to think of a way to automate it. As a side note: for example we had some backups that are probably useless, because they are way too small. Catching this would mean more manual regular checks, or some automated rules, at which point it becomes quickly more complex again.

The backup part is quite easy - generate a well-known file with 1kb of data and include it in your backups. After the backup completed, validate that you can restore the well-known file and compare the content of the original and the restored. Easy to automate if you run a well-customizable backup system.

Storytime: I did not check the restoration of my backups some time ago and had a faulty harddrive, so I needed to restore the backup. Backup was also as dumb as possible - essentially a tar and encrypting it with openssl. So I reinstalled the server, tried to decrypt it - got the error, that the key was wrong. Took me a good weekend to find out, that openssl changed the default hash algorithm between openssl 1.0 and 1.1. This would not have been catched with the proposed system, but now I really pin all default options in my scripts as well.

Re: Ask HN: How does your company manage its encryption keys?

#148

https://www.vaultproject.io/ We use Hashicorp's Vault product to manage SSH credentials, TLS certificates, as well as application secrets across thousands of users, tens of thousands of virtual machines, and hundreds of applications. We pay for the enterprise version, but the free version is more than capable for most needs. Avoid a password manager if you can, it leads to poor security practices and availability iss…

Same at our company we use vault via CLI

Re: Ask HN: How does your company manage its encryption keys?

#149

Can't speak to my current employer as it's above my pay-grade to know, but at Job-1 we did the following: - All "hot" keys were stored in an offline credential manager in specific vaults depending on who needed access to them. Only staff with actual clearance could request temporary access to a vault (fully background checked, 1 year employment, etc). - Copies of each vaulth and our master CA cert were written to 4 e…

How did you handle new secrets / rotations? Seems like a lot to keep in sync.

Seems like we hear more frequently about the actual secrets being stored encrypted (potentially with hardware protection) in a central place, and only the keys to unlock them being distributed like this.

Re: Ask HN: How does your company manage its encryption keys?

#150
post #66

The more complex your system grows, the more often it will fail and shoot you in the foot. I'd advise against systems like Hashicorp Vault - they just increase the complexity and while they have their merits in complex setups, you seem to be too small to be able to operate such a system. Have an offline backup printed along with the disaster recovery checklist and documentation and put them in a safe in your company…

For a PKI, you can also afford relatively more complexity with respect to the root CA key if you use intermediate CAs for most things and are careful with their expiration.
Post reply on HN