Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 79 posts

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#11

Post by verdverm »

thehucklecat wrote: post #5
verdverm wrote: post #4

Earlier quoted context omitted.

I'm not your target user, I don't feel the priority on this problem even though our permissions are more permissive than we'd like. Thing is, to rein them in typically requires application changes. You cannot just sprinkle magic LLM dust on IAM and make things better. My concern is for those who blindly trust LLMs. Security posturing is not the place to be an early adopter of AI tools. You have to understand both IAM…

what kind of application changes are you thinking it would equire? my policies are definitely too broad, but feels like I should be able to tighten them up without changing code. (just potentially breaking things if I get it wrong and go too tight).

Some scenarios

1. The application has to start using credentials for the first time, or consume them a different way. For example, stop consuming an environment variable and rely on a service account.

2. You have to change ops to support new workflows. Often you have to put approval workflows in place because fewer people can do things and you want only the machines touching production

3. You have to change human behaviors and habits (this is the real hard one). I've had to revert changes because the increased security blocked developers and they don't have time to adapt for the next deadline.

4. Getting parity in local development workflows is also challenging. How and where do you match vs except from IAM parity?

5. Should I give the current server access to a particular cloud service/resource or break out that particular function into a lambda and minimize the permissions there? You have to think through the implications of a breach and how/where you want to limit the blast radius.

6. This is probably obvious, but implementing application level controls, like API endpoint permissioning. IAM is not limited to cloud infra

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#12

Post by slalmeidabbm »

jgalt212 wrote: post #7
verdverm wrote: post #2

Repost: https://news.ycombinator.com/item?id=34038663 (11 months ago) > We employ Large Language Models (currently OpenAI GPT-4) For IAM, this seems like a disaster waiting to happen. Combining hallucination problems with security settings is not a path I would consider

I dunno. LLM generated config + formal verification could work.

This would be the way to go with the initial offering. Adding static code analysis + LLMs will help with reducing LLM usage and hallucinations and then adding a way to test out the policies to make sure that they are enough to run the code without being too broad will increase trust in the results.

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#14

Post by verdverm »

DanielSlauth wrote: post #9
verdverm wrote: post #4

Earlier quoted context omitted.

I'm not your target user, I don't feel the priority on this problem even though our permissions are more permissive than we'd like. Thing is, to rein them in typically requires application changes. You cannot just sprinkle magic LLM dust on IAM and make things better. My concern is for those who blindly trust LLMs. Security posturing is not the place to be an early adopter of AI tools. You have to understand both IAM…

First of all its pretty awesome your permissions are very tight. You are definitely on the other side of the spectrum compared to the rest. I get it that there is a lot of skepticism because of people hyping LLM's so indeed for now we use it as Copilot and not the driver. Hopefully you can agree though its pretty random that we are still manually creating IAM policies and need to get accustomed with the thousands of…

We are actively working on reining in permissions, I would not call them "tight". It's just not a top 3 priority, though that is likely changing with the upcoming SOC2 efforts. I still don't see us reaching for LLMs to help us here.

I'm not saying don't use them, just use them as an extra pair of eyes, mostly to catch errors rather than to drive and architect

> get it that there is a lot of skepticism because of people hyping LLM's

The skepticism is not from the hype, it's from experiencing LLM output personally. They are fine if the output can be fuzzy, like a blog post or a function signature, not so much if there is a specific and fragile target.

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#15

Post by DanielSlauth »

justchad wrote: post #8

I imagine there will be times where the LLM hallucinates or gives too many permissions. I'm sure it is still better than the try and see approach that most humans take when it comes to IAM setup. I'm just wondering if this product becomes irrelevant as Amazon Q rolls out and probably has some of this functionality baked in. Regardless, anything to make IAM provisioning easier is worth a go as long as you verify the r…

Thanks and let's see what Q will look like. I'm hoping the project will further evolve in integrating it in your CI/CD with PR's when commit requires IAM changes.

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#16

Post by verdverm »

zebomon wrote: post #13

I know you can't take responsibility for the 5% failure rate GPT-4 produces, but maybe things change when you have your simulator running. At that point, what kind of SLA do you plan on offering with the service?

More important than SLAs is who takes on the liability for these mistakes? With the changing laws and regulations for breaches, do I want to rely on an LLM that isn't going to own that liability?

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#17

Post by DanielSlauth »

verdverm wrote: post #11
thehucklecat wrote: post #5

Earlier quoted context omitted.

what kind of application changes are you thinking it would equire? my policies are definitely too broad, but feels like I should be able to tighten them up without changing code. (just potentially breaking things if I get it wrong and go too tight).

Some scenarios 1. The application has to start using credentials for the first time, or consume them a different way. For example, stop consuming an environment variable and rely on a service account. 2. You have to change ops to support new workflows. Often you have to put approval workflows in place because fewer people can do things and you want only the machines touching production 3. You have to change human beh…

The open-source project is a CLI you can put into your CI/CD so i think a pretty neat workflow where there should be less friction considering DevOps/security don't need to ping-pong on permissions.

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#18

Post by zebomon »

verdverm wrote: post #16
zebomon wrote: post #13

I know you can't take responsibility for the 5% failure rate GPT-4 produces, but maybe things change when you have your simulator running. At that point, what kind of SLA do you plan on offering with the service?

More important than SLAs is who takes on the liability for these mistakes? With the changing laws and regulations for breaches, do I want to rely on an LLM that isn't going to own that liability?

Yeah, I think it's a curious decision to have launched this thing with the LLM alone, as using an LLM for something this potentially disastrous is a moot point for me. If they can get a formal simulator running with it, on the other hand, then I'd imagine they may feel more comfortable putting out a guarantee and taking on some kind of liability themselves.

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#19

Post by verdverm »

DanielSlauth wrote: post #17
verdverm wrote: post #11

Earlier quoted context omitted.

Some scenarios 1. The application has to start using credentials for the first time, or consume them a different way. For example, stop consuming an environment variable and rely on a service account. 2. You have to change ops to support new workflows. Often you have to put approval workflows in place because fewer people can do things and you want only the machines touching production 3. You have to change human beh…

The open-source project is a CLI you can put into your CI/CD so i think a pretty neat workflow where there should be less friction considering DevOps/security don't need to ping-pong on permissions.

when you keep telling, you ain't selling

ask questions to deepen your understanding

> ...ping-pong...

It was a scheduling problem rather than a decision problem. The impact radius is always more than you anticipate

Re: Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

#20

Post by DanielSlauth »

zebomon wrote: post #13

I know you can't take responsibility for the 5% failure rate GPT-4 produces, but maybe things change when you have your simulator running. At that point, what kind of SLA do you plan on offering with the service?

I believe the minute you connect a Dev or staging environment to Slauth.io and we can run simulations and show divs we can offer pretty strong SLA's..
Post reply on HN