Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS in 2025: Stuff you think you know that's now wrong

lastweekinaws.com

Post reply on HN
Print view View original

41–50 of 284 posts

Re: AWS in 2025: Stuff you think you know that's now wrong

#41

Post by bilalq »

solatic wrote: post #37
SOLAR_FIELDS wrote: post #4

You know what's still stupid? That if you have an S3 bucket in the same region as your VPC that you will get billed on your NAT Gateway to send data out to the public internet and right back in to the same datacenter. There is simply no reason to not default that behavior to opt out vs opt in (via a VPC endpoint) beyond AWS profiting off of people's lack of knowledge in this realm. The amount of people who would want…

It's a design that is secure by default. If you have no NAT gateway and no VPC Gateway Endpoint for S3 (and no other means of Internet egress) then workloads cannot access S3. Networking should be closed by default, and it is. If the user sets up things they don't understand (like NAT gateways), that's on them. Managed NAT gateways are not the only option for Internet egress and users are responsible for the networks…

Fine for when you have no NAT gateway and have a subnet with truly no egress allowed. But if you're adding a NAT gateway, it's crazy that you need to setup the gateway endpoint for S3/DDB separately. And even crazier that you have to pay for private links per AWS service endpoint.

Re: AWS in 2025: Stuff you think you know that's now wrong

#42

Post by time0ut »

williamdclt wrote: post #18
hnlmorg wrote: post #14

Earlier quoted context omitted.

Not the OP but I’ve had AWS-staff recommend different prefixes even as recently as last year. If key prefixes don’t matter much any more, then it’s a very recent change that I’ve missed.

Might just be that the AWS staff wasn't up to date on this

I have had the same experience within the last 18 months. The storage team came back to me and asked me to spread my ultra high throughput write workload across 52 (A-Za-z) prefixes and then they pre-partitioned the bucket for me.

S3 will automatically do this over time now, but I think there are/were edge cases still. I definitely hit one and experienced throttling at peak load until we made the change.

Re: AWS in 2025: Stuff you think you know that's now wrong

#43

Post by kbolino »

otterley wrote: post #33
kbolino wrote: post #30

Earlier quoted context omitted.

The problem is that VPC endpoints aren't free. They should be, of course, at least when the destination is an AWS service in the same region. [edit: I'm speaking about interface endpoints, but S3 and DynamoDB can use gateway endpoints, which are free to the same region]

Gateway endpoints are free. Network endpoints (which are basically AWS-managed ENIs that can tunnel through VPC boundaries) are not free. S3 can use either, and we recommend establishing VPC Gateway endpoints by default whenever you need S3 access. (Disclaimer: I work for AWS, opinions are my own.)

Fair point, and valid for S3 (the topic at hand) and DynamoDB.

Other AWS services, though, don't support gateway endpoints.

Re: AWS in 2025: Stuff you think you know that's now wrong

#44

Post by SOLAR_FIELDS »

otterley wrote: post #32
SOLAR_FIELDS wrote: post #4

You know what's still stupid? That if you have an S3 bucket in the same region as your VPC that you will get billed on your NAT Gateway to send data out to the public internet and right back in to the same datacenter. There is simply no reason to not default that behavior to opt out vs opt in (via a VPC endpoint) beyond AWS profiting off of people's lack of knowledge in this realm. The amount of people who would want…

This is the intended use case for S3 VPC Gateway Endpoints, which are free of charge. https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpo... (Disclaimer: I work for AWS, opinions are my own.)

The problem is that the default behavior for this is opt-in, rather than opt-out. No one prefers opt-in. So why is it opt-in?

Re: AWS in 2025: Stuff you think you know that's now wrong

#45

Post by darkwater »

otterley wrote: post #32
SOLAR_FIELDS wrote: post #4

You know what's still stupid? That if you have an S3 bucket in the same region as your VPC that you will get billed on your NAT Gateway to send data out to the public internet and right back in to the same datacenter. There is simply no reason to not default that behavior to opt out vs opt in (via a VPC endpoint) beyond AWS profiting off of people's lack of knowledge in this realm. The amount of people who would want…

This is the intended use case for S3 VPC Gateway Endpoints, which are free of charge. https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpo... (Disclaimer: I work for AWS, opinions are my own.)

I think they know it. They are complaining it's not enabled by default (and so do I).

Re: AWS in 2025: Stuff you think you know that's now wrong

#46

Post by andrewmcwatters »

simonw wrote: post #7

S3: "Block Public Access is now enabled by default on new buckets." On the one hand, this is obviously the right decision. The number of giant data breeches caused by incorrectly configured S3 buckets is enormous. But... every year or so I find myself wanting to create an S3 bucket with public read access to I can serve files out of it. And every time I need to do that I find something has changed and my old recipe d…

This sort of thing drives me nuts in interviews, when people are like, are you familiar with such-and-such technology?

Yeah, what month?

Re: AWS in 2025: Stuff you think you know that's now wrong

#47

Post by immibis »

SOLAR_FIELDS wrote: post #4

You know what's still stupid? That if you have an S3 bucket in the same region as your VPC that you will get billed on your NAT Gateway to send data out to the public internet and right back in to the same datacenter. There is simply no reason to not default that behavior to opt out vs opt in (via a VPC endpoint) beyond AWS profiting off of people's lack of knowledge in this realm. The amount of people who would want…

A company making revenue is not stupid.

Re: AWS in 2025: Stuff you think you know that's now wrong

#48

Post by Hikikomori »

otterley wrote: post #34
dmart wrote: post #13

Earlier quoted context omitted.

VPC endpoints in general should be free and enabled by default. That you need to pay extra to reach AWS' own API endpoints from your VPC feels egregious.

Gateway endpoints are free. Network endpoints (which are basically AWS-managed ENIs that can tunnel through VPC boundaries) are not free. S3 can use either, and we recommend establishing VPC Gateway endpoints by default whenever you need S3 access. (Disclaimer: I work for AWS, opinions are my own.)

Why don't you have gateway endpoints for all your APIs?

Re: AWS in 2025: Stuff you think you know that's now wrong

#50

Post by hnlmorg »

time0ut wrote: post #42
williamdclt wrote: post #18

Earlier quoted context omitted.

Might just be that the AWS staff wasn't up to date on this

I have had the same experience within the last 18 months. The storage team came back to me and asked me to spread my ultra high throughput write workload across 52 (A-Za-z) prefixes and then they pre-partitioned the bucket for me. S3 will automatically do this over time now, but I think there are/were edge cases still. I definitely hit one and experienced throttling at peak load until we made the change.

That’s sounds like the problem we were having. Lots of writes to a prefix over a short period of time and then low activity to it after about 2 weeks.
Post reply on HN