Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IAM is hard – Thoughts on $80M fine from the Capital One Breach

twitter.com

Post reply on HN
Print view View original

91–100 of 125 posts

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#91

Post by tgsovlerkhgsel »

freedomben wrote: post #12
mesofile wrote: post #8

My general experience with crafting IAM policies is very reminiscent of SELinux, in that it's very difficult to work agnostically while adhering to a principle of least privilege. Especially given that this kind of task is often done by admin/ops people, one typically can't know in advance everything that the app might need to be able to access in order to work correctly. The process of discovering this is: try runni…

I've seen this a ton. I have been an on/off security professional so academically I am committed to the principle of least privilege, but holy hell it can be painful or impossible in real life. Where possible I've started adopting the "run it and see" or audit2allow approach (there are awesome tools that can do this for AWS IAM perms too), but then before applying the policy, somebody needs to put a quick line beside…

> Not perfect but sometimes perfect is the enemy of good.

This. Add enough hurdles, and people will

a) spend all the energy they're willing to spend on your process, hate it, and as a result never do anything except what you force them to, and in particular, never voluntarily reduce permissions unless forced (because they might need them later and then it'll be a pain). They'll also see you as an enemy, not a partner, which is not the place a security team should be in.

b) optimize for not what is best, but what is least affected by your wall of process, even if it's less secure (e.g. because it's a legacy system that you didn't get around to locking down yet).

c) outsource to less secure vendors, and get it approved because management knows that getting it deployed internally would take forever due to the process

d) in the most extreme case, set up uncontrolled shadow IT with zero security controls and hide it from you - and because they already spent all their energy dealing with "perfect", they don't want to hear the word "security" ever again, and the security posture of their shadow IT shows this.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#92

Post by sushshshsh »

brendoelfrendo wrote: post #88
gray_-_wolf wrote: post #78

Earlier quoted context omitted.

In our company we went the other road. We have the developers write the policies (since I mean, they know what their app needs) and test it in dev environments. After that, ops guys step in during code review to check for too broad allow in the policies. So far it seems to work in acceptable manner.

This sounds like a good process, but it depends a lot on the relationship between dev and ops... I've seen too many dev shops push against changes requested by ops or security because their main pressure is to ship features fast. And then it turns into a management fight and whoever has the more influential management gets the final say while the other side is forced to grumble.

Currently experiencing this at my current employer. I have a suspicion that slowing the dev and deployment process is in everyone's perceived best interest, until of course the day where we are out competed.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#93

Post by coreyoconnor »

jariel wrote: post #72

Aside from the inherent complexity of complex systems, there's another layer we don't talk about and that is arbitrary complexity, particularly in communications, standards and documentation. A lot of IAM does not need need to be that hard but concepts need to be poignantly clear. It's harder than it needs to be.

I wonder about the complexity and AWS motivations.

What does AWS gain by improving IAM? There are barely any competitors, so they won't be losing people for that. They offer their own AWS professional services happy to charge you for making it "understandable". Their service agreements largely absolve them of client mistakes. Which usually result in larger bills from AWS.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#94

Post by tedk-42 »

meritt wrote: post #86
tedk-42 wrote: post #85

Earlier quoted context omitted.

If you're deploying a microservice on an individual ER2 inctance with an instance role, you're doing it wrong. Not sure where you got the idea it was a microservice. Names like a dodgy app server of some kind.

The point I am trying to make is "microservice" does nothing whatsoever to solve the problem of people incorrectly provisioning IAM permissions.

Nor does it in make developers better at their jobs.

The point i'm trying to make is if the scope of what a system can do is limited, it's permission boundary/model is easy to define.

Many things led to the incorrect provisioning of the IAM role. Lake of understanding of IAM for starters as well as consequences around it.

By no means am I saying microservices would solve the problem. But it sure does make it easier to define what permissions your app needs to have as well as limit the blast radius of what is exposed when done correctly.

This is impossible with monoliths on EC2 instances.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#95

Post by mden »

tedk-42 wrote: post #20

Another reason why microservices are a good thing. They result in micro level permissions for individual resources. In saying that though, k8s in AWS was really shit at limiting what IAM roles containers could assume without it being the instance role. Crap like kube2iam and kiam came out to butcher the AWS metadata/instance networking. Thankfully AWS solved it with their new OIDC IDP. True DevOps culture workplaces…

Capital One uses microservices with multiple layers for privileging and is "true devops" with devs typically deciding IAM policies.

One of the big issues with microservices is that you need to invest a whole lot more into infrastructure and tooling to be able to run them sanely. Most companies are really hesitant to invest sufficiently which leads to burn out, low productivity, and various ultimately avoidable bugs. This compounded with microservices being inherently more complex you often end up with tangled and messy products that are hard to understand software wise which also impacts security. Having worked at Capital One I think that's certainly one of the big issues at play.

For devops, I personally don't get it, maybe I just haven't seen it done well or maybe when it's done well it just disappears into the background. But expecting all developers to be good at application coding and also have an in depth understanding of AWS security best practices is unreasonable. Something is wrong with the levels of abstractions being worked at.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#96

Post by jariel »

coreyoconnor wrote: post #93
jariel wrote: post #72

Aside from the inherent complexity of complex systems, there's another layer we don't talk about and that is arbitrary complexity, particularly in communications, standards and documentation. A lot of IAM does not need need to be that hard but concepts need to be poignantly clear. It's harder than it needs to be.

I wonder about the complexity and AWS motivations. What does AWS gain by improving IAM? There are barely any competitors, so they won't be losing people for that. They offer their own AWS professional services happy to charge you for making it "understandable". Their service agreements largely absolve them of client mistakes. Which usually result in larger bills from AWS.

That's pretty cynical.

AWS is a ball of complexity because it grew organically that way, and they don't have a culture of explaining, or, keeping things simple.

Both of those things would require strong strategic guidance, and a real effort to do.

Unless Bezos edicts: "Our APIs must remain simple even as they scale, and we must document in a manner that keeps the 80% common path easy to use, while the remaining 20% arcane functionality available ..." then it would happen.

But it won't.

It's reasonably well curated arbitrary complexity, it is what it is.

This is not an issue anyone handles well.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#97

Post by scarface74 »

brendoelfrendo wrote: post #88
gray_-_wolf wrote: post #78

Earlier quoted context omitted.

In our company we went the other road. We have the developers write the policies (since I mean, they know what their app needs) and test it in dev environments. After that, ops guys step in during code review to check for too broad allow in the policies. So far it seems to work in acceptable manner.

This sounds like a good process, but it depends a lot on the relationship between dev and ops... I've seen too many dev shops push against changes requested by ops or security because their main pressure is to ship features fast. And then it turns into a management fight and whoever has the more influential management gets the final say while the other side is forced to grumble.

Well, developers can fight all they want to. I was the dev lead at a company that had to be HIPAA compliant. My neck was on the line if we were found to be out of compliance along with the security director and the operations people.

Now as a consultant working with many large customers, “shipping fast” is nowhere near as important as security.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#98

Post by scarface74 »

shadowgovt wrote: post #66

In my experience, the hardest thing about this whole space is the number of developers who don't understand what the problem is until it is costing them $80 million dollars. Limiting blast radius, for example. Why does your threat model include the possibility of one of your applications being compromised and using its credentials to do desired things to other applications? That implies your programs are buggy. And c…

It’s not always about even being malicious. At my last company I was an admin. But I locked myself down so I wouldn’t make a stupid mistake.

In the past, both Apple and Google made mistakes in their installers that if anyone else did it, people would assume it was malicious.

There was a bug in the iTunes installer that erased files on people’s hard disk if there was a space in the name.

There was also a bug in the Chrome installer that made your hard drive unbootable if you had system integrity protection turned off.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#99

Post by zymhan »

staticassertion wrote: post #82
orf wrote: post #75

Earlier quoted context omitted.

Cloudformation. Ewww.

Sure, CF is garbage, I won't try to argue otherwise. There's this though: https://www.hashicorp.com/blog/cdk-for-terraform-enabling-py...

Wonder what Pulumi will do to stay relevant now.

Re: IAM is hard – Thoughts on $80M fine from the Capital One Breach

#100

Post by lmm »

shadowgovt wrote: post #66

In my experience, the hardest thing about this whole space is the number of developers who don't understand what the problem is until it is costing them $80 million dollars. Limiting blast radius, for example. Why does your threat model include the possibility of one of your applications being compromised and using its credentials to do desired things to other applications? That implies your programs are buggy. And c…

> Limiting blast radius, for example. Why does your threat model include the possibility of one of your applications being compromised and using its credentials to do desired things to other applications? That implies your programs are buggy. And clearly, your programs aren't buggy; you're using best practices. How could they be?

I find that a lot these "defence in depth" style policies end up muddling where the actually important boundaries are. You get people slapping on a "security" layer everywhere because "it can't hurt", but then people don't worry about bypassing those layers either because "that part is just in case, it doesn't have to be 100% secure" and then the holes in your Swiss cheese line up and you get hacked.

Realistically you have a budget for how much effort people are willing to put into understanding your security systems and you need to spend it where it will do the most good. For most organisations that means a clear, centralised access control model that the rest of your systems trust to do its job, and no ad-hoc mitigation measures.

Post reply on HN