Live data from Hacker News

AWS Support able to access any S3 object due to permission change

twitter.com

61–70 of 134 posts

Re: AWS Support able to access any S3 object due to permission change

#61

When I worked in aws, this is primarily used to check for permissions of an object. I know how dumb customers can be, for the most part this is used to see why a customer cannot delete a bucket or object those sort of things. I don't remember having ability to see actual customers data only metadata is accessible. Edit: Based on what I know, I'm pretty sure support will not be able see any of the customers data.

> I know how dumb customers can be I find this insulting as a customer. Is AWS usually contemptuous of its customers? I don't think I've ever called my customer "dumb", and working as a consultant I've seen all kinds of interesting things. People make mistakes. They're always in a hurry. They may have a hard time understanding ambiguous, complex or incomplete documentation. The interface may be confusing and lead the…

> working as a consultant I've seen all kinds of interesting things.

You shouldn't compare consulting vs tech support. Especially when your billing rate is noticeably higher.

Re: AWS Support able to access any S3 object due to permission change

#62
post #9

Okay, opinions up front: I don't think this is worthy of "declaring a security incident. Having some experience working behind the scenes, just because this policy was changes this way doesn't mean "All AWS Support personnel had unrequited access to your S3 objects." To me, this reads as Twitter inflammatory nonsense. Here's why: * KMS Encrypted objects would not be accessible because the support personnel would need…

AWS uses envelope encryption. Data encryption (for example bucket-level) keys are generated and managed outside KMS for some time/services to minimize calls to KMS that can be expensive.

Thus, I imagine with AWS-managed KMS keys, in some cases permission to KMS service is not needed to access data.

KMS-CMKs are better in this respect, can be controlled and audited.

Re: AWS Support able to access any S3 object due to permission change

#63
post #22

I have been looking into this lately for a company that wanted to host important data on AWS S3. I couldn’t find conclusive information in public domain. It’s hard to decipher the AWS Data Privacy policy: https://aws.amazon.com/compliance/data-privacy-faq/ In section Who Owns Customer Content, it’s implied that AWS doesn’t access customers’ data: “As a customer, you maintain ownership of your content, and you select…

> AWS doesn’t access customers’ data Amazon's product people also don't look at third party seller statistics to decide which products to sell themselves. Until they got caught doing just that. To assume that they don't look at data feels naive. A German super market chain with online ambitions has a rule that nothing touching their pipeline can be hosted on AWS. Want to sell them SaaS? You can't run your nodes on AW…

That's great. It is heart-warming to hear some businesses are taking this threat seriously.

We built are service in similar vain. Nothing can reside under authority on non-EU entity, and nothing can be hosted on servers owned by non-EU entity. This effectively removed AWS, Azure, GCP and Alibaba.

And still, we had plenty of choice.

We specifically picked “boring” cloud provider. No thrills cloud vendor which has core business in building infrastructure rather than snooping on customers.

Re: AWS Support able to access any S3 object due to permission change

#64
post #22

I have been looking into this lately for a company that wanted to host important data on AWS S3. I couldn’t find conclusive information in public domain. It’s hard to decipher the AWS Data Privacy policy: https://aws.amazon.com/compliance/data-privacy-faq/ In section Who Owns Customer Content, it’s implied that AWS doesn’t access customers’ data: “As a customer, you maintain ownership of your content, and you select…

When you think about risks it’s productive to understand what could happen vs what may.

I was a stakeholder in an early cloud negotiation back in 2010-11, and one of the key issues was the support personnel. Ultimately, they have access to your data.

Re: AWS Support able to access any S3 object due to permission change

#65
post #55
post #45

Such concerns specifically led to my decision of only uploading sensitive data to S3 with client side encryption. Since the aws cli tool only supports server side encryption with keys stored on amazon servers (where the non-default managed keys cost like 1 USD per month), I decided to simply symmetrically encrypt the backup of my syncthing data volume with AES256 using gnupg and only then pushing it to the S3 bucket.

this should be the default user behaviour for any cloud storage. Don't put unencrypted (company) data on a cloud infrastructure you don't have full control over. Also reminds me of the (hyped?) "outrage" when a former facebook developer stated that they used to have a "default password" that allowed fb devs to log into every account and the media were like "omg they could have logged in and seen your photos". I mean.…

> yeah they're the developers they could always do that even without the password

Not really. Obviously facebook the company can always access your data. Weather or not an individual developer can do the same, which developers can do it, how they can do it, and under what level of supervision this would be is a design choice.

It is possible to design a system with very high level of security and ones with none too. As with any design considerations it has trade-offs. A super secure system might introduce dev and operational frictions which the company might deem unnacceptable. But even with that consideration the question is a lot more complicated than a simple “yeah they’re the developers”.

Re: AWS Support able to access any S3 object due to permission change

#66
post #37

Earlier quoted context omitted.

> AWS doesn’t access customers’ data Amazon's product people also don't look at third party seller statistics to decide which products to sell themselves. Until they got caught doing just that. To assume that they don't look at data feels naive. A German super market chain with online ambitions has a rule that nothing touching their pipeline can be hosted on AWS. Want to sell them SaaS? You can't run your nodes on AW…

AFAIK Walmart has that rule too.

Yes, they do - we pitched to them, and they said for our talks to continue we’d need to use another cloud (at least for their tenancy).

Re: AWS Support able to access any S3 object due to permission change

#67
I guess taking some risks as I am not really authorized to use social media, but at AWS there is a sauron like focus on not letting internal engineers view customer data. On my service we don't even have tools to do it and the security controls are very tight and getting tighter all the time. It's a big fucking deal, if not the biggest fucking deal, besides KTLO.

Posting to get ahead of some of the comment here on security posture.

Re: AWS Support able to access any S3 object due to permission change

#68
post #60
post #45

Such concerns specifically led to my decision of only uploading sensitive data to S3 with client side encryption. Since the aws cli tool only supports server side encryption with keys stored on amazon servers (where the non-default managed keys cost like 1 USD per month), I decided to simply symmetrically encrypt the backup of my syncthing data volume with AES256 using gnupg and only then pushing it to the S3 bucket.

Cloudstore[1] can do something similar automatically. [1] https://github.com/infor-cloud/cloud-store

Just skimmed it, looks useful, with mentioning keypair I'm assuming they use RSA keys or something similar. Why the choice for asymmetric encryption?

Re: AWS Support able to access any S3 object due to permission change

#69
post #44

When I worked in aws, this is primarily used to check for permissions of an object. I know how dumb customers can be, for the most part this is used to see why a customer cannot delete a bucket or object those sort of things. I don't remember having ability to see actual customers data only metadata is accessible. Edit: Based on what I know, I'm pretty sure support will not be able see any of the customers data.

> I know how dumb customers can be Maybe a more constructive way to look at this would be that people simply do "dumb" things. In customer support where you only see those moments, it might not always seem that way, but dealing with people's simple mistakes is also educating them to do better next time.

People can be ignorant, lazy, not give a shit about the work they are doing, have poor learning ability and or skills, and cross their fingers, mashing buttons, hoping everything just works, and then expect everyone else around them to help them out of their screw ups.

If you've ever worked in CS, or known anyone that works in CS, you know that there are an absolute fucking shitload of these people. Often in roles they are unqualified for and with privileges and power no sane person would ever give them.

Re: AWS Support able to access any S3 object due to permission change

#70
post #37

Earlier quoted context omitted.

> AWS doesn’t access customers’ data Amazon's product people also don't look at third party seller statistics to decide which products to sell themselves. Until they got caught doing just that. To assume that they don't look at data feels naive. A German super market chain with online ambitions has a rule that nothing touching their pipeline can be hosted on AWS. Want to sell them SaaS? You can't run your nodes on AW…

AFAIK Walmart has that rule too.

They do have that rule. And my previous company boasted that they signed a contract with us even though we used AWS. It was a pretty big deal, IMHO.
Post reply on HN