Live data from Hacker News

Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

theregister.com

21–30 of 162 posts

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#21

S3 security is a pain. S3 options are tricky to master, docs are beyond confusing. A common S3 use case is: Use a Bucket for Read-only static content, (js, html, imgs). From a dev perspective ideally it will work like a protected folder + Web Server, whereas the webserver will have read-only privileges, but reality is far more complicated. I can understand why was easy for Twillio to have this infosec issue, since we…

Is it really a pain though? Maybe because I’ve been working with it since the beginning. Bucket ACL with NO permissions, then manage all permissions on an IAM role on the account. If it’s cross account then allow assuming to other accounts, but no reason to bother with the bucket ACL. Leave the bucket policy blank and you’ll never have to worry about an open bucket. Better yet make a deny rule to everything but a sin…

This is the correct design pattern as of 2020. Roles + STS, and always be stingy when granting a role any permissions, both at the resource or API level.

But, as others have said, the documentation at times is poor at best, and it can be easy to make mistakes.

Some preventives every ops team can take are:

1. Never make manual permission changes in the console. Everything is done via SCM’d code.

2. All permission changes are peer reviewed.

3. Create logical separations of buckets so it’s more obvious what the permissions should be of a bucket. You can go so far as to have separate accounts that are for public uploads, separate accounts for public assets, etc. Avoid granting elevated permissions at a folder level, which is harder to understand as a developer without access to view the ACLs.

Good luck everyone!

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#22

Aws should start taking blame for this shit, then maybe they would start auditing their customers setups...

AWS does audit this. You get a huge warning when you make a bucket public in this way, and they send you emails about it.

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#23

Aws should start taking blame for this shit, then maybe they would start auditing their customers setups...

There are big signs on world-accessible buckets when you go into the S3 console. It's a fairly new thing but they are (slowly) trying to help auditing and finding these things I guess.

However I'm not sure if there are any built-in tools to find weird nested policies attached that could cause a similar permission to be granted to all..

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#24

Aws should start taking blame for this shit, then maybe they would start auditing their customers setups...

What does AWS have to do with this? Sure, the technical debt of S3 being one of the first services is clear to see. But it was a world writable bucket. What is AWS going to do about that? Eventually, you have to know and care about what the hell you are doing.

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#25
One of the most shocking things I realized when I graduated college and started working as a software engineer is how insecure, or really just improperly-configured, large enterprise systems are.

This is anecdotal, of course, but it seems to me that any company that didn't start cloud-native is going to have flaws like this, and it's often not at the fault of developers but at the fault of top-down initiatives to "get to the cloud" ASAP instead of correctly.

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#26
post #15

would using the `integrity` attribute in the ` ` tag mitigate this attack?

Yes! Subresource Integrity is the exact counter-measure to this kind of attack: https://developer.mozilla.org/en-US/docs/Web/Security/Subres... However, on Twilio's documentation site, they do not include the integrity attribute in their examples: https://www.twilio.com/docs/taskrouter/js-sdk/workspace/task...

It's difficult. I like the idea of subresource integrity a lot, but a counter example: Twilio discovers a bug in their JS SDK. Without subresource integrity they can push up a fix and propagate it to all clients immediately. With subresource integrity they would need every client to update their web site to change the integrity attribute. If it was a serious bug then that could be a huge issue.

Even putting the security arguments aside you can bet a lot of customers would be really irritated having to constantly update this stuff on their site. If a competitor didn't have that restriction it would become a selling point. IMO the ideal is that Twilio provides both options: a "sdk-latest.js" that's a moving target, as well as "sdk-v1.3.js" that is frozen and can be locked with subresource integrity.

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#27

S3 security is a pain. S3 options are tricky to master, docs are beyond confusing. A common S3 use case is: Use a Bucket for Read-only static content, (js, html, imgs). From a dev perspective ideally it will work like a protected folder + Web Server, whereas the webserver will have read-only privileges, but reality is far more complicated. I can understand why was easy for Twillio to have this infosec issue, since we…

Is it really a pain though? Maybe because I’ve been working with it since the beginning. Bucket ACL with NO permissions, then manage all permissions on an IAM role on the account. If it’s cross account then allow assuming to other accounts, but no reason to bother with the bucket ACL. Leave the bucket policy blank and you’ll never have to worry about an open bucket. Better yet make a deny rule to everything but a sin…

[deleted]

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#28
Having an S3 Bucket with writing permissions to anyone is a major fuck-up. I know many people open buckets to the world at the read-level for whatever reason. I do that myself for static websites. But also writing permissions?

This requires a Bucket Policy that is written explicitly to allow this. I don't want to make assumptions because of course we don't know all the information and I personally know very little about infosec, but that sounds very strange.

Why would you do that? I assume the SDK gets uploaded from a secure environment that has the credentials to upload objects, so there's no legitimate reason to have writing permissions open to the world.

Re: Twilio: Someone broke into our AWS S3 silo, added 'non-malicious' code to JS SDK

#29

English is not my main language, but "broke into" doesn't implicate "breaking" something to get "into" something?

Yeah, in reality its like complaining someone "broke" into your care when you left all of your doors open.

> Yeah, in reality its like complaining someone "broke" into your care when you left all of your doors open.

And you wouldn't get much sympathy, but this is a legally viable complaint. When my car was stolen, the thief was charged for theft, but his girlfriend who was joyriding with him was charged for criminal trespass. Just being in the car when you don't have permission is illegal.

Post reply on HN