Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS S3: Sometimes you should press the $100k button

cyclic.sh

Post reply on HN
Print view View original

81–90 of 240 posts

Re: AWS S3: Sometimes you should press the $100k button

#81

Post by theshrike79 »

CydeWeys wrote: post #55
lenkite wrote: post #45

sigh . My team is facing all these issues. Drowning in data. Crazy S3 bill spikes. And not just S3 - Azure, GCP, Alibaba, etc since we are a multi-cloud product. Earlier, we couldn't even figure out lifecycle policies to expire objects since naturally every PM had a different opinion on the data lifecycle. So it was old-fashioned cleanup jobs that were scheduled and triggered when a byzantine set of conditions were m…

The data protection regulations really are so freeing, huh. It's amazing to be able to delete all this stuff without worrying about having to keep it forever.

Yep, having everything disappear at 2 months max is a life-saver.

That "absolutely essential thing" isn't essential any more when there is a possible GDPR/CCPA violation with a significant fine just around the corner.

Re: AWS S3: Sometimes you should press the $100k button

#82

Post by electroly »

vdm wrote: post #21

DeleteObjects takes 1000 keys per call. Lifecycle rules can filter by min/max object size. (since Nov 2021)

Thank you for mentioning that lifecycle rule change. I must have missed the announcement; that is exactly the functionality I needed.

Re: AWS S3: Sometimes you should press the $100k button

#83

Post by rmbyrro »

asim wrote: post #60

The AWS horror stories never cease to amaze me. It's like we're banging our heads against the wall expecting a different outcome each time. What's more frustrating, the AWS zealots are quite happy to tell you how you're doing it wrong. It's the users fault for misusing the service. The reality is, AWS was built for a specific purpose and demographic of user. It's now complexity and scale makes it unusable for newer d…

What do you see missing or not well explained in AWS documentation that newer devs wouldn't understand?

I started using S3 early in my career and didn't see this problem. I always thought in data retention during design phase.

My opinion is that lazy, careless or under time pressure developers will not, and then will get bitten. But it would happen to any tool. Maybe a different problem, but they'll always get bitten hard ...

Re: AWS S3: Sometimes you should press the $100k button

#84

Post by hinkley »

jeff_vader wrote: post #76
CydeWeys wrote: post #55

Earlier quoted context omitted.

The data protection regulations really are so freeing, huh. It's amazing to be able to delete all this stuff without worrying about having to keep it forever.

In case of my previous employer it led to incredibly complicated encryption system. It took couple years to maybe implement in 10% of the system. Deleting any old data was rejected.

I wonder sometimes if it would help if we collectively watched more anti-hoarding shows, in order to see how the consultants convince their customers they can get rid of stuff.

Re: AWS S3: Sometimes you should press the $100k button

#85

Post by samlambert »

StratusBen wrote: post #69
lenkite wrote: post #45

sigh . My team is facing all these issues. Drowning in data. Crazy S3 bill spikes. And not just S3 - Azure, GCP, Alibaba, etc since we are a multi-cloud product. Earlier, we couldn't even figure out lifecycle policies to expire objects since naturally every PM had a different opinion on the data lifecycle. So it was old-fashioned cleanup jobs that were scheduled and triggered when a byzantine set of conditions were m…

Disclosure: I'm Co-Founder and CEO of a cloud cost company named https://www.vantage.sh/ - I also used to be on the product management team at AWS and DigitalOcean. I'm not intentionally trying to shill but this is exactly why people choose to use Vantage. We give them a set of features for automating and understanding what they can do to manage and save on costs. We're also adding multi-cloud support (GCP is in earl…

Vantage is a seriously awesome product. We love it at PlanetScale. Obviously being a cloud product things can get pricy and so Vantage is essential.

Re: AWS S3: Sometimes you should press the $100k button

#86

Post by dasil003 »

asim wrote: post #60

The AWS horror stories never cease to amaze me. It's like we're banging our heads against the wall expecting a different outcome each time. What's more frustrating, the AWS zealots are quite happy to tell you how you're doing it wrong. It's the users fault for misusing the service. The reality is, AWS was built for a specific purpose and demographic of user. It's now complexity and scale makes it unusable for newer d…

I'm not sure any sizable group is banging their head against a wall. Yes, AWS is complex. Yes, AWS has cost foot guns. These are natural outcomes of removing friction from scaling.

Sure we could start with something simpler, but as you may have noticed, even the more basic hosting providers like DigitalOcean and Linode have been adding S3-compatible object storage because of its proven utility.

In terms of making something meaningfully simpler, I think Heroku was the high water mark. But even though it was a great developer experience, the price/performance barriers were a lot more intractable than dealing with AWS.

Re: AWS S3: Sometimes you should press the $100k button

#87

Post by _jal »

asim wrote: post #60

The AWS horror stories never cease to amaze me. It's like we're banging our heads against the wall expecting a different outcome each time. What's more frustrating, the AWS zealots are quite happy to tell you how you're doing it wrong. It's the users fault for misusing the service. The reality is, AWS was built for a specific purpose and demographic of user. It's now complexity and scale makes it unusable for newer d…

> we need a completely new experience for the next generation

I mean, at some point, if you're (say) using some insane amount of storage, you're going to pay for that.

I would agree that getting alerting right for billing-relevant events at whatever you're currently operating at should be a lot easier than it is. And I agree that there is a lot of room to baby-proof some of the less obvious mistakes that people frequently make, to better expose the consequences of some changes, etc.

But the flip side is that infra has always been expensive, and vendors have always been more than happy to sell you far more than you need along with the next new shiny whatever.

To the extent that these are becoming implicit decisions made by developers rather than periodic budgeted refresh events built by infra architects, developers need to take responsibility for understanding the implications of what they're doing.

Re: AWS S3: Sometimes you should press the $100k button

#88

Post by res0nat0r »

amelius wrote: post #7
Tehchops wrote: post #3

We’ve got data in S3 buckets not nearly at that scale and managing them, god forbid trying a mass delete, is absolute tedium.

Mass delete also takes an eternity on my Linux desktop machine. The filesystem is hierarchical, but the delete operation still needs to visit all the leaves.

Use the delete-objects instead and it will be much faster, as you can supply up to 1000 keys to remove per a single API call.

https://awscli.amazonaws.com/v2/documentation/api/latest/ref...

Re: AWS S3: Sometimes you should press the $100k button

#89

Post by candiddevmike »

lenkite wrote: post #45

sigh . My team is facing all these issues. Drowning in data. Crazy S3 bill spikes. And not just S3 - Azure, GCP, Alibaba, etc since we are a multi-cloud product. Earlier, we couldn't even figure out lifecycle policies to expire objects since naturally every PM had a different opinion on the data lifecycle. So it was old-fashioned cleanup jobs that were scheduled and triggered when a byzantine set of conditions were m…

Are you multi-cloud because your customers need you to be multi-cloud?

Re: AWS S3: Sometimes you should press the $100k button

#90

Post by jstarfish »

ebingdom wrote: post #31
inopinatus wrote: post #25

Earlier quoted context omitted.

There’s no delimiter. There is only the appearance of a delimiter, to appease folks who think S3 is a filesystem, and fool them into thinking they’re looking at folders. The object name is the entire label, and every character is equally significant for storage. When listing objects, a prefix filters the list. That’s all. However, S3 also uses substrings to partition the bucket for scale. Since they’re anchored at th…

So if I have a bunch of objects whose names are hashes like 2df6ad6ca44d06566cffde51155e82ad0947c736 that I expect to access randomly, is there any performance benefit to introducing artificial delimiters like 2d/f6/ad6ca44d06566cffde51155e82ad0947c736? I've seen this used in some places.

I don't know what impact that partitioning pattern has on s3, but it has some obvious benefits if your app needs to revert to write to a normal filesystem instead (like for testing).
Post reply on HN