Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Confidant: an open-source secret management service

eng.lyft.com

Post reply on HN
Print view View original

41–50 of 71 posts

Re: Confidant: an open-source secret management service

#41

Post by pamad05 »

mitchellh wrote: post #28
_ondq wrote: post #15

Earlier quoted context omitted.

Vault is a nice piece of engineering (we use it), but it has what I call serious "backend-itis". Everything is pluggable, which makes it a bit of a nightmare to understand and use. For example, "secret backends" and "storage backends" are entirely separate things, but the docs aren't super clear about it (not to mention auth backends, audit backends, listeners ...). Unsurprisingly it buys completely into the Hashicor…

I just want to correct some of your points, I don't want to detract or compare Vault to Confidant here, as this is their time to shine! "Buys into the HashiCorp ecosystem": Consul is completely optional. We also support ZooKeeper, SQL, etcd, and more. There is zero forced tie to Consul. HCL is correct though! "you can't just stick N Vault servers behind an LB": Actually, that is exactly what we recommend! https://vau…

Hi, quick question since you seem to be on top of Vault.

My team and I were investigating possible configuration options and secret management, and we couldn't figure out a good reason as to why the tool doesn't use S3 as one of the backends, which would essentially eliminate the need to maintain our own etcd/consul/sql whatever - plus, since the entire thing is path based, it seems incredibly well suited for that backend.

I couldn't think of one, but I might be missing something.

Re: Confidant: an open-source secret management service

#42

Post by adzuci »

_ondq wrote: post #15
austinjp wrote: post #10

How does this compare to Vault? [1] Looks like Confidant is tied to AWS whereas Vault can use various backends..? [1] https://www.vaultproject.io/

Vault is a nice piece of engineering (we use it), but it has what I call serious "backend-itis". Everything is pluggable, which makes it a bit of a nightmare to understand and use. For example, "secret backends" and "storage backends" are entirely separate things, but the docs aren't super clear about it (not to mention auth backends, audit backends, listeners ...). Unsurprisingly it buys completely into the Hashicor…

We also wanted things to be simple & wrote a custom cli and ansible module to wrap around https://github.com/oleiade/trousseau, but have slowly found the need for more of the features that tools like these provide.

We're planning on switching to Vault or something similar in the near future & will be testing out both of these. I definitely agree with the "backend" confusion surrounding Vault but I'm interested in where Confidant may be lacking. We host services on AWS and Heroku, so in some places AWS' KMS may not be the best option.

Regardless, I do have to say it's so refreshing to see security tools with decent documentation & a clean user experience. I'm almost stunned to see a swiftype search box, one page threat model description & preconfigured turn-key container all in one place. Props to Lyft for opening up a great tool to the masses!

Re: Confidant: an open-source secret management service

#44

Post by adzuci »

bpicolo wrote: post #17

Another one of those problems every company seems to try to solve on their own. :P

I totally agree (we did), though Confidant is in some ways trying to be an extension of existing AWS tools. I'm curious what you're using though since I've felt for a while now that every one of these secrets stores have faults or are a giant pain to work with. Confidant may be the least offender in that regard & anyone that is trying to make being secure easier or trying to make better tools gets an A in my book.

Re: Confidant: an open-source secret management service

#45

Post by [deleted] »

pamad05 wrote: post #41
mitchellh wrote: post #28

Earlier quoted context omitted.

I just want to correct some of your points, I don't want to detract or compare Vault to Confidant here, as this is their time to shine! "Buys into the HashiCorp ecosystem": Consul is completely optional. We also support ZooKeeper, SQL, etcd, and more. There is zero forced tie to Consul. HCL is correct though! "you can't just stick N Vault servers behind an LB": Actually, that is exactly what we recommend! https://vau…

Hi, quick question since you seem to be on top of Vault. My team and I were investigating possible configuration options and secret management, and we couldn't figure out a good reason as to why the tool doesn't use S3 as one of the backends, which would essentially eliminate the need to maintain our own etcd/consul/sql whatever - plus, since the entire thing is path based, it seems incredibly well suited for that ba…

[deleted]

Re: Confidant: an open-source secret management service

#46

Post by Merkur »

akerl_ wrote: post #37
Merkur wrote: post #31

Please correct me if I am wrong, but I think there is no secure way to store stuff in an virtual environment. I wish I am wrong - cause my heart always bleeds if I see db passwords in configuration files! But As long as there is a hypervisor you do not control access to - you must trust the owner of the bare metal to (1) honor your privacy (2) be competent to secure his system. Trust is nice, but it is not security.…

Security is a spectrum. There's definitely a path for someone with full local access to the hypervisor and system memory to do some careful reconstruction or other malicious injection, but that attack vector is amazingly rare compared to attacks based on bad protocols for network traffic, bad app-level auth, and insecure storage practices (mixing code and creds, for example). There's immense value in defending agains…

Indeed! After all we live in a world where many ssl private certificates are only protected by OS file system rights on a internet facing server. And clear text database passwords in config files are common, in all content management systems - and even many customer relationship systems!

mysqldump got a CLI argument to provide the password! As far as I know - visible to anyone with some access on the system. the documentation warns about that and suggest to create a config file to store the password.

security is a spectrum - but if it's about password storage in modern web applications live on the lower end.

I am increasingly frustrated by that - and if I raise concerns many admins stick to binary security "storing stuff unencrypted on disk is okay, cause the attacker is inside already" followed by "you must store the key somewhere" it's wrong and it's true at the same time. it's also sad - because not only the users, but also the ppl paying us trust us to keep them save. and we don't. :( we can say "it's a spectrum" and we are truthful - but we just can't keep em safe. I think it's important to recognize that simple fact.

...or is it me who is just meticulous?

Re: Confidant: an open-source secret management service

#47

Post by jefferai »

pamad05 wrote: post #41
mitchellh wrote: post #28

Earlier quoted context omitted.

I just want to correct some of your points, I don't want to detract or compare Vault to Confidant here, as this is their time to shine! "Buys into the HashiCorp ecosystem": Consul is completely optional. We also support ZooKeeper, SQL, etcd, and more. There is zero forced tie to Consul. HCL is correct though! "you can't just stick N Vault servers behind an LB": Actually, that is exactly what we recommend! https://vau…

Hi, quick question since you seem to be on top of Vault. My team and I were investigating possible configuration options and secret management, and we couldn't figure out a good reason as to why the tool doesn't use S3 as one of the backends, which would essentially eliminate the need to maintain our own etcd/consul/sql whatever - plus, since the entire thing is path based, it seems incredibly well suited for that ba…

S3 is one of the available storage backends. You only need consul/etcd/zookeeper if you want to have a high availability setup.

Re: Confidant: an open-source secret management service

#48

Post by yeukhon »

normanjoyner wrote: post #33
mpdehaan2 wrote: post #27

Earlier quoted context omitted.

I'd say the opposite - the industry is in general being a lot more comfortable with investing in AWS to the point of lockin being acceptable now. More people are treating cloud providers less like a big pool of IP addresses. Not saying it's a good or bad thing, but attend AWS reInvent or something and it's pretty easy to percieve huge uptake. If you adopt extra management stacks running on or against your cloud you l…

That's an interesting perspective and observation. I've always found lock-in to be a bit scary, and I've heard this echoed by a number of colleagues & companies. No one can deny the usefulness of tools provided by hosting providers, namely AWS; however, to me, architecting a system in such a way where you are literally unable to jump ship when the need arises (compliance, cost, etc) is unacceptable. With hybrid / mul…

We tried running on three cloud service providers while connecting back to our own data center (corporate stuff) at the same time, for a couple years, because of the whole "cloud agnostic". The code requires us to be compatible with OpenStack and AWS. Trust me, no matter what people say about the compatibility of some of the components between OpenStack and AWS, you cannot do the AWS-way 100% in OpenStack. In the end I got a pile of shit code with a lot of workaround and condition.

Sure Heat is like a comparable version of CloudFormation, but you can't do what you did in AWS and translate to Heat. OpenStack doesn't have all the services that AWS has. Not every product (and not every API in a product of AWS) works 100% with CloudFormation. If you are running an old release of OpenStack, you are more fucked in that case.

We are now moving our cloud infrastructure to AWS entirely so no more "it works in this environment", or "they sort of work and sort of different because of X and Y tool not comparable."

The right attitude should be "do one thing well, and improve." Architect "right" and than unlock yourself whenever possible. The same goes to people migrating to Docker or any container technology. Dockerfile is a lock-in. You still have to move back to regular shell script later if you drop Docker.

Re: Confidant: an open-source secret management service

#49

Post by yeukhon »

viraptor wrote: post #36
vangale wrote: post #21

This is the first time I've seen a nicely documented requirements.txt and I like! https://github.com/lyft/confidant/blob/master/requirements.t...

+ for readability, - for precise versions. Good luck receiving critical updates this way...

> + for readability, - for precise versions. Good luck receiving critical updates this way...

Real lesson. Authentication failed and no clue why app couldn't start. Developer spent almost two days and realized a new gem version had been released and the new version was not compatible with the current release. Quote developer "fool me once shame on you, fool me twice shame on me."

Imagine auto-scale (no image) or installing packages during docker container initialization, in production, the word "fuck" will fill up your mailbox/IRC/Slack/HipChat/ChatBot from your dear SRE / DevOps because of lack of regression testing.

Same argument goes for auto system update in Ubuntu. To receive critical update you better have a process to review, and warn critical updates. One change in the system API can screw up your entire system.

If you want the fail hard model, which is really useful, the best way is to have two sets of requirements.txt. One for random Saturday testing (just launch a docker instance running tests with latest version of packages installed), and one for development all the way to production.

Re: Confidant: an open-source secret management service

#50

Post by rdxm »

pamad05 wrote: post #41
mitchellh wrote: post #28

Earlier quoted context omitted.

I just want to correct some of your points, I don't want to detract or compare Vault to Confidant here, as this is their time to shine! "Buys into the HashiCorp ecosystem": Consul is completely optional. We also support ZooKeeper, SQL, etcd, and more. There is zero forced tie to Consul. HCL is correct though! "you can't just stick N Vault servers behind an LB": Actually, that is exactly what we recommend! https://vau…

Hi, quick question since you seem to be on top of Vault. My team and I were investigating possible configuration options and secret management, and we couldn't figure out a good reason as to why the tool doesn't use S3 as one of the backends, which would essentially eliminate the need to maintain our own etcd/consul/sql whatever - plus, since the entire thing is path based, it seems incredibly well suited for that ba…

not trying to be snarky, but you do understand who posted the response above(i.e. mitchellh) right??
Post reply on HN