Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A $1k AWS mistake

geocod.io

Post reply on HN
Print view View original

211–220 of 281 posts

Re: A $1k AWS mistake

#211

Post by jairuhme »

maciekkmrk wrote: post #184

An entire blog article post to say "read the docs and enable VPC S3 endpoint". It's all in the docs: https://docs.aws.amazon.com/vpc/latest/privatelink/concepts.... > There is another type of VPC endpoint, Gateway, which creates a gateway endpoint to send traffic to Amazon S3 or DynamoDB. Gateway endpoints do not use AWS PrivateLink, unlike the other types of VPC endpoints. For more information, see Gateway endpoints…

> An entire blog article post to say "read the docs and enable VPC S3 endpoint".

I think it's okay if someone missed something in the docs and wanted to share from their experience. In fact, if you look at the the s3 pricing page [0], under Data Transfer, VPC endpoints are mentioned at all. It simply says data transfer is free between AWS services in the same region. I think that much detail would be enough to reasonably assume you didn't have to set up additional items to accomplish.

[0]https://aws.amazon.com/s3/pricing/

Re: A $1k AWS mistake

#212

Post by antonvs »

seniorThrowaway wrote: post #209
antonvs wrote: post #111

Earlier quoted context omitted.

> But not as easy a maintenance story That's my whole point. Zero maintenance. For a tinkerer who's focused on the infra, then sure, hosting your own can make sense. But for anyone who's focused on literally anything else, it doesn't make any sense.

Cloud is not great for GPU workloads. I run a nightly workload that takes 6-8 hours to run and requires a Nvidia GPU, along with high RAM and CPU requirements. It can't be interrupted. It has a 100GB output and stores 6 nightly versions of that. That's easily $600+ a month in AWS just for that one task. By self-hosting it I have access to the GPU all the time for a fixed up front relatively low cost and can also use…

Literally anything else except GPU. :)

I kind of assume that goes without saying, but you're right.

The company I'm with does model training on cloud GPUs, but it has funding for that.

> RDS + EKS for a couple hundred a month is an amazing deal for what is essentially zero maintenance application hosting.

Right. That's my point, and aside from GPU, pretty much any normal service or app you need to run can be deployed on that.

Re: A $1k AWS mistake

#213

Post by perching_aix »

luhn wrote: post #195
perching_aix wrote: post #125

Earlier quoted context omitted.

I personally prefer to just memorize the data and recite it really quickly on-demand. Only half-joking. When something grossly underperforms, I do often legitimately just pull up calc.exe and compare the throughput to the number of employees we have × 8 kbit/sec [0], see who would win. It is uniquely depressing yet entertaining to see this outperform some applications. [0] spherical cow type back of the envelope esti…

8kbit/min, you mean.

Oh yeah lol, whoops. Still applies sadly.

Re: A $1k AWS mistake

#214

Post by throwaway-aws9 »

ipsento606 wrote: post #196
cristiangraz wrote: post #162

Earlier quoted context omitted.

AWS just released flat-rate pricing plans with no overages yesterday. You opt into a $0, $15, or $200/mo plan and at the end of the month your bill is still $0, $15, or $200. It solves the problem of unexpected requests or data transfer increasing your bill across several services. https://aws.amazon.com/blogs/networking-and-content-delivery...

https://aws.amazon.com/cloudfront/pricing/ says that the $15-per-month plan comes with 50TB of "data transfer" Does "data transfer" not mean CDN bandwidth here? Otherwise, that price seems two orders of magnitude less than I would expect

With AWS, there's always a catch. In this case, it's for 10M requests. In other words, you pay $15 for 10M requests of up to 5MB each.

[edit: looks like there's no overages but they may force you to flip to the next tier and seems like they will throttle you https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope....]

Re: A $1k AWS mistake

#215

Post by Hikikomori »

embedding-shape wrote: post #28
pavlov wrote: post #23

Earlier quoted context omitted.

Yes…? Egress bandwidth costs money. Consumer cloud services bake it into a monthly price, and if you’re downloading too much, they throttle you. You can’t download unlimited terabytes from Google Drive. You’ll get a message that reads something like: “Quota exceeded, try again later.” — which also sucks if you happen to need your data from Drive. AWS is not a consumer service so they make you think about the cost dir…

"Premium bandwidth" which AWS/Amazon markets to less understanding developers is almost a scam. By now, software developers think data centers, ISPs and others part of the peering on the internet pay per GB transferred, because all the clouds charge them like that.

I mean transit is usually billed like that, or rather a commit.

Re: A $1k AWS mistake

#216

Post by rikafurude21 »

antonvs wrote: post #204
rikafurude21 wrote: post #188

Earlier quoted context omitted.

It doesnt make any sense to you that I would like to avoid a potential 60K bill because of a configuration error? If youre not working at faang your employer likely cares too. Especially if its your own business you would care. You really can't think of _one_ case where self hosting makes any sense?

> It doesnt make any sense to you that I would like to avoid a potential 60K bill because of a configuration error? This is such an imaginary problem. The examples like this you hear about are inevitably the outliers who didn't pay any attention to this issue until they were forced to. For most services, it's incredibly easy to constrain your costs anyway. You do have to pay attention to the pricing model of the serv…

Thinking about "constraining cost" is the last thing I want to do. I pay a fixed 200 dollars a month for a dedicated server and spend my time solving problems using code. The hardware I rent is probably overkill for my business and would be more than enough for a ton of businesses' cloud needs. If youre paying per GB of traffic, or disk space, or RAM, you're getting scammed. Hyperscalers are not the right solution for most people. Developers are scared of handling servers, which is why you're paying that premium for a hyperscaler solution. I SSH into my server and start/stop services at will, configure it any way i want, copy around anything I want, I serve TBs a week, and my bill doesnt change. You would appreciate that freedom if you had the will to learn something you didnt know before. Trust me its easier than ever with Ai!

Re: A $1k AWS mistake

#217

Post by Hikikomori »

plantain wrote: post #34
embedding-shape wrote: post #28

Earlier quoted context omitted.

"Premium bandwidth" which AWS/Amazon markets to less understanding developers is almost a scam. By now, software developers think data centers, ISPs and others part of the peering on the internet pay per GB transferred, because all the clouds charge them like that.

Try a single threaded download from Hetzner Finland versus eu-north-1 to a remote (i.e. Australia) destination and you'll see premium bandwidth is very real. Google Cloud Storage significantly more so than AWS. Sure you can just ram more connections through the lossy links from budget providers or use obscure protocols, but there's a real difference. Whether it's fairly priced, I suspect not.

AWS like most do hot potato routing, not so premium when it exits instantly. This is usually a tcp tuning problem rather than bandwidth being premium.

Re: A $1k AWS mistake

#218

Post by patabyte »

wulfstan wrote: post #80

This happens so often that the S3 VPC endpoint should be setup by default when your VPC is created. AWS engineers on here - make this happen. Also, consider using fck-nat ( https://fck-nat.dev/v1.3.0/ ) instead of NAT gateways unless you have a compelling reason to do otherwise, because you will save on per-Gb traffic charges. (Or, just run your own Debian nano instance that does the masquerading for you, which every…

> which every old-school Linuxer should be able to do in their sleep.

Oof, this hit home, hah.

Re: A $1k AWS mistake

#220

Post by mystifyingpoi »

SOLAR_FIELDS wrote: post #129
Spivak wrote: post #126

Earlier quoted context omitted.

The reason to not include the endpoint by default is because VPCs should be secure by default. Everything is denied and unless you explicitly configure access to the Internet, it's unreachable. An attacker who manages to compromise a system in that VPC now has a means of data exfiltration in an otherwise air gapped set up. It's annoying because this is by far the more uncommon case for a VPC, but I think it's the rig…

There’s zero reason why AWS can’t pop up a warning if it detects this behavior though. It should clearly explain the implications to the end user. I mean EKS has all sorts of these warning flags it pops up on cluster health there’s really no reason why they can’t do the same here.

To be fair, while EKS warnings are useful, I've grown a habit to ignore them completely, since I've seen every single RDS cluster littered with "create a read replica please" and "enable performance insights" bs warnings.
Post reply on HN