AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely • Hacker News Board

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#101

Post by epberry » Thu, Apr 05, 2018, 3:37 AM UTC

neuronexmachina wrote: post #48

scrollaway wrote: post #21

Earlier quoted context omitted.

ParameterStore lets you store String, StringList or SecureString. But there's no limit to SecureString. A SecureString can be `postgres://admin:hunter2@localhost:5432/db`. It can also be `{"username": "admin", "password": "hunter2", "host": "localhost", "port": "5432"}`.

Sure you could do that if you don't mind adding in your own string->dict conversion. I wouldn't be surprised if the internal implementation is near-identical. Secrets Manager seems like a slightly more easy-to-use version of Parameter Store that's also visible, instead of hidden away inside Systems Manager.

It did take me a minute to actually find Parameter Store. There's so much stuff hidden away in the EC2 menus.

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#102

Post by p0rkbelly » Thu, Apr 05, 2018, 3:39 AM UTC

0xCMP wrote: post #17

kondro wrote: post #12

At $0.40 per secret per month and $0.05 per 10,000 requests this is much more expensive than the practically free SSM Parameter Store product, even if you factor in the auto-rotating bits.

It'd be cheaper than running Vault with a backing Consul cluster which also provides rotation and other features. There is a point where Vault is more cost effective, but I believe it'd require a ton of requests and secrets to justify min 6 machines of at least t2.micro that also need to managed and secured.

...and that's without a FIPS 140-2 Level 3 backend for a HSM as well, right? (Secrets Manger has that I assume).

With Vault 1) you have to get Hashi Enterprise 2) Pay for a very pricey HSM.

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#103

Post by tyingq » Thu, Apr 05, 2018, 3:50 AM UTC

Au revoir Cyberark. I won't miss you.

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#104

Post by toomuchtodo » Thu, Apr 05, 2018, 3:54 AM UTC

scrollaway wrote: post #90

cassianoleal wrote: post #74

Earlier quoted context omitted.

Wait, you keep private SSH keys on bastion hosts? It's much better practice to use the bastion as a proxy to the other hosts. This is easily achieved using the ProxyCommand option of OpenSSH. No agent forwarding, no secrets kept in random hosts.

Do you know a good guide with standard practices for setting up and securing bastion hosts (preferably on aws)?

If you don’t require interactive sessions, consider using AWS SSM run command [1] instead. You install the agent on the instances, with commands sent from the client through the AWS control plane (with IAM and SSM documents for access control and CloudTrail logs of all commands issued).

I’m currently deploying it in an enterprise for ~5k users, and it works surprisingly well for providing the ability to run arbitrary commands on instances without ssh access.

[1] https://docs.aws.amazon.com/systems-manager/latest/userguide...

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#105

Post by JoachimSchipper » Thu, Apr 05, 2018, 5:15 AM UTC

cassianoleal wrote: post #74

JoachimSchipper wrote: post #44

Earlier quoted context omitted.

What are you assuming about exposure? If a SSH key lives on a well-secured workstation or bastion host (and you ideally don't agent-forward it to insecure hosts), rotating that key once per hour doesn't seem a top priority to me? E.g. a sudo password is (lower-impact, but) more likely to get exposed to compromised hosts? ("Well-secured workstation" is arguably an oxymoron, of course...)

Wait, you keep private SSH keys on bastion hosts? It's much better practice to use the bastion as a proxy to the other hosts. This is easily achieved using the ProxyCommand option of OpenSSH. No agent forwarding, no secrets kept in random hosts.

No, keeping SSH keys on bastion hosts was the silliest reasonable thing I could think of.

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#106

Post by Dunedan » Thu, Apr 05, 2018, 5:26 AM UTC

drodgers wrote: post #84

Dunedan wrote: post #37

Earlier quoted context omitted.

My biggest criticism of Parameter Store is actually that it's free. Let me explain: Because it's free they limit the requests per seconds you can make to the Parameter Store. That's especially noticeable when doing requests for all parameters of a given path, as the limit is way lower there than for requesting (a bunch of) individual parameters. In the past that caused serious problems for us when using Parameter Sto…

Oh, yeah, we just ran into this recently when rolling-out parameter store. For anyone else having this problem, the trick was to use the GetParametersByPath API method ( https://docs.aws.amazon.com/systems-manager/latest/APIRefere... ) instead of the more flexible DescribeParameters (which has stupidly low, undocumented rate limits).

At the time we experienced the problems GetParametersByPath had way lower limits than GetParameters (~an order of magnitude)! That's the problem we ran into. That was already back at the end of last year and the SSM team promised to improve the situation, but so far at least no publicly available news states something about it.

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#107

Post by mrep » Thu, Apr 05, 2018, 5:58 AM UTC

toomuchtodo wrote: post #25

scrollaway wrote: post #18

First reaction: Holy crap! They finally turned ParameterStore into a proper product! Second reaction: Holy shit that's expensive [for what it does]. ParameterStore is free (minus the KMS component). The only value-add is secret rotation and that's not something that most of the time makes sense to use. [Edit: I'm not advocating for no rotation; see replies] Edit: Had more time to think about it. Someone enlighten me:…

> The only value-add is secret rotation and that's not something that most of the time makes sense to use. From a security perspective, you should be rotating secrets somewhere between annually and every 90 days, depending on your business/security/compliance requirements and the nature of the data secured by the secret.

Got a link for rotation time frame recommendations because I have seen them all over the board. I understand the basic one that you want old employees to have no knowledge of current credentials but I have worked with security rotation requirements for minutely, hourly, daily, monthly, and yearly and I've never really seen a reasoning for the difference between them.

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#108

Post by tanilama » Thu, Apr 05, 2018, 6:04 AM UTC

ronnier wrote: post #45

whateveracct wrote: post #7

This looks like an AWS equivalent of the Amazon-internal secret management tool called Odin. Which is very nice because Odin was pretty much universally loved from what I saw.

After leaving Amazon you really go on to understand how good the internal tools are at Amazon and how they probably have the best tools in the industry.

Brazil/Apollo needs huge renovation though, especially when Containers are getting so much more popular.

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#109

Post by mrep » Thu, Apr 05, 2018, 6:07 AM UTC

mabbo wrote: post #100

ronnier wrote: post #45

Earlier quoted context omitted.

After leaving Amazon you really go on to understand how good the internal tools are at Amazon and how they probably have the best tools in the industry.

Brazil + Apollo + Pipelines. I left (and later boomeranged) and was surprised that the outside world hadn't solved these problems nearly as well. Don't get me wrong, Brazil and Apollo probably are ready for a rethink and rewrite, but they do a great job.

What would you change about them?

Re: AWS Secrets Manager – Store, Distribute, and Rotate Credentials Securely

#110

Post by jknoepfler » Thu, Apr 05, 2018, 6:28 AM UTC

ronnier wrote: post #45

whateveracct wrote: post #7

This looks like an AWS equivalent of the Amazon-internal secret management tool called Odin. Which is very nice because Odin was pretty much universally loved from what I saw.

After leaving Amazon you really go on to understand how good the internal tools are at Amazon and how they probably have the best tools in the industry.

I had the exact opposite reaction after leaving Amazon. Brazil is a slow, bloated pile of crap that regularly blocked fixes, made it impossible to develop in a non-standard language, and regularly resulted in 100+ dev build blockages due to versioning issues.

Our team was abandoning Apollo for MAWS, so our deployments were bridged between the two, resulting in another twisted pile of incomprehensible script-glue.

Pipelines was ok (!)

Everything on public EC2 for managing deployments (CodeDeploy and CloudFormation in particular) were so broken and bug-ridden and non-performant that we couldn't reliably deploy production code with them.

I prefer open source solutions in almost every way. I actually can't think of a single problem that the Amazon internal stack solves that isn't better solved in the open-source world. Which isn't surprising, given the constraints involved.