Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS in 2025: Stuff you think you know that's now wrong

lastweekinaws.com

Post reply on HN
Print view View original

241–250 of 284 posts

Re: AWS in 2025: Stuff you think you know that's now wrong

#241

Post by awongh »

simianwords wrote: post #223
dcminter wrote: post #219

Earlier quoted context omitted.

Well, the "accidentally making the S3 bucket public" scenario would be a good one. If you review carefully with full understanding of what e.g. all your policies are doing then great, no problem. If you don't do that will you necessarily notice that you accidentally leaked customer data to the world? The problem isn't the LLM it's assuming its output is correct just the same as assuming Stack Overflow answers are cor…

I agree but its about the extent. I'm willing to accept the risk of ocassionally making S3 public but getting things done much faster, much like I don't meticulously read documentation when I can get the answer from stackoverflow. If you are comparing with stackoverflow then I guess we are on the same page - most people are fine with taking stuff from stackoverflow and it doesn't count as "brave".

The thing is that you can now ask the LLM for links and you can ask it to break down why it thinks a piece of code, for example, protects the bucket from being public. Things that are easy to verify against the actual docs.

I feel like this workflow is still less time, easier and less error prone than digging out the exact right syntax from the AWS docs.

Re: AWS in 2025: Stuff you think you know that's now wrong

#242

Post by jiggawatts »

awongh wrote: post #153
simonw wrote: post #7

S3: "Block Public Access is now enabled by default on new buckets." On the one hand, this is obviously the right decision. The number of giant data breeches caused by incorrectly configured S3 buckets is enormous. But... every year or so I find myself wanting to create an S3 bucket with public read access to I can serve files out of it. And every time I need to do that I find something has changed and my old recipe d…

This is exactly what I use LLMs for. To just read the docs for me and pull out the base level demo code that's buried in all the AWS documentation. Once I have that I can also ask it for the custom tweaks I need.

Back when GPT4 was the new hotness, I dumped the markdown text from the Azure documentation GitHub repo into a vector index and wrapped a chatbot around it. That way, I got answers based on the latest documentation instead of a year-old LLM model's fuzzy memory.

I now have the daunting challenge of deploying an Azure Kubernetes cluster with... shudder... Windows Server containers on top. There's a mile-long list of deprecations and missing features that were fixed just "last week" (or whatever). That is just too much work to keep up with for mere humans.

I'm thinking of doing the same kind of customised chatbot but with a scheduled daily script that pulls the latest doco commits, and the Azure blogs, and the open GitHub issue tickets in the relevant projects and dumps all of that directly into the chat context.

I'm going to roll up my sleeves next week and actually do that.

Then, then, I'm going to ask the wizard in the machine how to make this madness work.

Pray for me.

Re: AWS in 2025: Stuff you think you know that's now wrong

#243

Post by donavanm »

mdaniel wrote: post #197
kassner wrote: post #116

I haven’t used AWS in the last 5 years. Is IPv6 still somewhat of an issue? I remember some services not supporting it at all and making it impossible to manage as a IPv6-only network.

You'll be glad(sic) to know they have specialized hostnames for IPv6 control plane endpoints, because AAAA records scare the olds https://github.com/aws/aws-cli/blob/2.28.14/awscli/botocore/...

> because AAAA records scare the olds

No. They break existing customer expectations.

There are heaps of dualstack API endpoints https://docs.aws.amazon.com/general/latest/gr/rande.html#dua... if that's what the client wants.

The amazonaws.com domain endpoints did not introduce ipv6/AAAA directly is (mostly) due to access control. For better or worse there are a lot of "v4 centric" IAM statements, like aws:SourceIp, in identity/resource/bucket policies. Introducing a new v6 value is going to break all of those existing policies with either unexpected DENYs or, worse, ALLOWs. Thats a pretty poor customer experience to unexpectedly break your existing infrastructure or compromise your access control intentions.

AWS _could_ have audited every potential IAM policy and run a MASSIVE outreach campaign, but something as simple as increasing (opaque!) instance ID length was a multi year effort. And introducing backwards compatibility on a _per policy_ basis is its own infinite security & UX yak shaving exercise as well.

So thats why you have opt-in usage of v6/dualstack in the client/SDK/endpoint name.

Re: AWS in 2025: Stuff you think you know that's now wrong

#244

Post by donavanm »

mappu wrote: post #122
bob1029 wrote: post #69

Earlier quoted context omitted.

I am also assuming that Amazon intends for the Deep Archive tier to be a profitable offering. At $0.00099/gb-month, I don't see how it could be anything other than tape.

My understanding is some AWS products (e.g. RDS) need very fast disks with lots of IOPS. To get the IOPS, though, you have to buy +++X TB sized SSDs, far more storage space than RDS actually needs. This doesn't fully utilize the underlying hardware, you are left with lots of remaining storage space but no IOPS. It's perfect for Glacier. The disks for Glacier cost $0 because you already have them.

Since ~2014 or so the constraint on all HDD based storage has been IOPs/throughput/queue time. Shortly after that we started seeing "minimum" device sizes that were so large as to be challenging to productively use their total capacity. Glacier type retrieval is also nice in that you have much more room for "best effort" scheduling and queuing compared to "real time" request like S3:PutObject.

Last I was aware flash/nvme storage didnt have quite the same problem, due to orers of magnitude improved access times and parallelism. But you can combine the two in a kind of distributed reimplementation of access tiering (behind a single consistent API or block interface).

Re: AWS in 2025: Stuff you think you know that's now wrong

#245

Post by donavanm »

simonw wrote: post #76
bob1029 wrote: post #69

Earlier quoted context omitted.

I am also assuming that Amazon intends for the Deep Archive tier to be a profitable offering. At $0.00099/gb-month, I don't see how it could be anything other than tape.

I wonder if it's where old S3 hard drives go to die? Presumably AWS have the world's single largest collection of used storage devices - if you RAID them up you can probably get reliable performance out of them for Glacier?

Not quite. Hardware with customer data or corp IP (eg any sort of storage or nvram) doesnt leave the "red zone"[1] without being destroyed. And reusing EOL hardware is a nightmare of failure rates and consistency issues. Its usually more cost effective to scrap the entire rack once its depreciated, or potentially at the 4-5 year mark at most. More revenue is generated by replacing the entire rack with new hardware that will make better use of the monthly recurring cost (MRC) of that rack position/power whips/etc.

[1] https://www.aboutamazon.com/news/aws/aws-data-center-inside

Re: AWS in 2025: Stuff you think you know that's now wrong

#246

Post by dcminter »

awongh wrote: post #240
dcminter wrote: post #210

Earlier quoted context omitted.

This could not possibly go wrong... You're braver than me if you're willing to trust the LLM here - fine if you're ready to properly review all the relevant docs once you have code in hand, but there are some very expensive risks otherwise.

This is LLM as semantic search- so it's way way easier to start from the basic example code and google to confirm that it's correct than it is to read the docs from scratch and piece together the basic example code. Especially for things like configurations and permissions.

Sure, if you do that second part of verifying it. If you just get the LLM to spit it out then yolo it into production it is going to make you sad at some point.

Re: AWS in 2025: Stuff you think you know that's now wrong

#247

Post by jagged-chisel »

saghm wrote: post #154
simonw wrote: post #74

Earlier quoted context omitted.

I think this is the key of why I find it confusing: I need a very clear diagram showing which rules override which other rules.

My understanding is that there isn't actually any "overriding" in the sense of two rules conflicting and one of them having to "win" and take effect. I think it's more that an enabled rule always is in effect, but it might overlap with another rule, in which case removing one of them still won't remove the restrictions on the area of overlap. It's possible I'm reading too much into your choice of words, but it does s…

If one rule explicitly restricts access and another explicitly grants access, which one is in effect? Do restrictions override grants? Does a grant to GroupOne override a restriction to GroupAlpha when the authenticated use in is both groups? Do rules set by GodAdmin override rules set by AngelAdmin?

Re: AWS in 2025: Stuff you think you know that's now wrong

#248

Post by dcminter »

crinkly wrote: post #38
simonw wrote: post #7

S3: "Block Public Access is now enabled by default on new buckets." On the one hand, this is obviously the right decision. The number of giant data breeches caused by incorrectly configured S3 buckets is enormous. But... every year or so I find myself wanting to create an S3 bucket with public read access to I can serve files out of it. And every time I need to do that I find something has changed and my old recipe d…

I just stick CloudFront in front of those buckets. You don't need to expose the bucket at all then and can point it at a canonical hostname in your DNS.

Not always that simple - for example if you want to automatically load /foo/index.html when the browser requests /foo/ you'll need to either use the web serving feature of S3 (bucket can't be private) or set up some lambda at edge or similar fiddly shenanigans.

Re: AWS in 2025: Stuff you think you know that's now wrong

#249

Post by kbolino »

watermelon0 wrote: post #203
mdaniel wrote: post #124

Earlier quoted context omitted.

https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-e... ~~I get the impression there are several others, too, but that one is of especial interest to me~~ Wowzers, they really are much better now: aws --region us-east-1 ec2 describe-vpc-endpoint-services | jq '.ServiceNames|length' 459 If you're saying "other services should offer VPC Endpoints," I am 100% on-board. One should never have to traverse the Inter…

Those are VPC endpoints, not gateway endpoints.

Both interface endpoints and gateway endpoints are also called VPC endpoints. The former get distinct IP addresses in your VPC subnets while the latter get distinct entries in your VPC routing tables. They are even created with the same API call: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_C...

Re: AWS in 2025: Stuff you think you know that's now wrong

#250

Post by no_wizard »

wiether wrote: post #208
no_wizard wrote: post #141

Earlier quoted context omitted.

At what point is AWS worth using over other compute competitors when you’re using them as a storage bucket + VPS. They’re wholly more expensive at that point. Why not go with a more traditional but rock solid VPS provider? I have the opposite philosophy for what it’s worth: if we are going to pay for AWS I want to use it correctly, but maximally. So for instance if I can offload N thing to Amazon and it’s appropriate…

I agree that using them as a VPS provider is a mistake. If you don't use the E(lasticity) of EC2, you're burning cash. For prod workloads, if you can go from 1 to 10 instances during an average day, that's interesting. If you have 3 instances running 24/7/365, go somewhere else. For dev workloads, being able to spin instances in a matter of seconds is a bliss. I installed the wrong version of a package on my instance…

>business logic should be cloud agnostic

In practice I found this to be more burden than it’s worth. I have yet to work somewhere that is on Azure, GCP or AWS and actually switch between clouds. I am sure it happens, but is it really that common?

I instead think of these platforms as a marriage, you’re going to settle in one and do your best to never divorce

Post reply on HN