Live data from Hacker News

Slashing data transfer costs in AWS

bitsand.cloud

41–50 of 268 posts

Re: Slashing data transfer costs in AWS

#41
post #24

I don't understand how AWS can keep ripping people off with these absurd data transfer fees, when there is Cloudflare R2 just right over there offering a 100 times better deal.

we just built a new feature for our pretty bandwidth heavy SaaS on R2. Works pretty damn good with indeed massive savings. We just use the AWS-SDK (Node.js) and use the R2 endpoint.

Re: Slashing data transfer costs in AWS

#42
post #21

Earlier quoted context omitted.

But it's the same mindset as "cloud good", which was also a growing line of thought once. Mantras aren't useful; tradeoff analysis is useful.

Mantras are good for orgs that are not mature enough to do actual analysis. A lead developer left recently where I work and while it was likely higher pay that was the biggest decision to move, I suspect the real reason he left is higher ups simply don't listen when he says things like you can't just take full virtual machines on azure, refuse any rewrite/redesign while complaining about high azure spend.

Well, then they can flip a coin for which mantra to follow. If you pick "cloud bad" you'll get stories also about companies that refused to go to cloud when it makes sense to.

Re: Slashing data transfer costs in AWS

#43

GCP patched a similar loophole [1] in 2023 presumably because some of their customers were abusing it. I'd expect AWS to do the same if this becomes widespread enough. [1]: https://cloud.google.com/storage/pricing-announce#network

Unlikely. The "loophole" GCP patched was that you can use GCS to transfer data between regions on the same continent for free. This is already non-free on AWS. What OP mentioned is that transferring data between availability zones *in the same region* also costs $0.02 per GB and can be worked around.

Re: Slashing data transfer costs in AWS

#44

An alternative to sophisticated cloud cost minimization systems is…….. don’t use the cloud. Host it yourself. Or use Cloudflare which has 0 cents per gigabyte egress fees. Or just rent cloud servers from one of the many much cheaper VPS hosting services and don’t use all the expensive and complex cloud services all designed to lock you in and drain cash from your at 9 or 12 or 17 cents per gigabyte. Seriously, if you…

I don't think "host yourself" in this instance would've helped. I think AWS in this instance is operating at a loss. Author found a loophole in AWS pricing and that's why it's so cheap. Doing it on their own would've been more expensive.

Now as to why the AWS pricing the way it is... we may only guess, but likely to promote one service over the other.

Re: Slashing data transfer costs in AWS

#45

I'll share my trick :) Lightsail instances can be used to "proxy" data from other AWS resources (eg EC2 instances or S3 buckets). Each Lightsail instance has a certain amount of data transfer included in it's price ($3.5 instance has 1TB, $5 instance has 2TB, $10 instance has 3TB, $20 instance has 4TB, $40 instance has 5TB). The best value (dollar per transferred data) is the $10 instance, which gives you 3TB of traf…

https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspe…

At least AWS is fully aware how premium their normal data transfer is and that one might want to optimise those costs.

Re: Slashing data transfer costs in AWS

#46
post #2

S3 is a nice trick. More tricks: 1. Ask for discounts if you are a big AWS customer (e.g., spend $1mln+/year). At some point, they were huge for inter-AZ transfers. 2. Put things in one AZ. Running DB in "b" zone and your only server in "a" is even worse than just standardizing on one zone. 3. When using multiple AZ do load aware AZ balancing.

4. Activate S3 Inteligent-Tier storage class?

You've got to be careful of the automation charge with Intelligent Tiering.

https://discourse.nixos.org/t/the-nixos-foundations-call-to-...

Re: Slashing data transfer costs in AWS

#47

Earlier quoted context omitted.

https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspe…

A had a suspicion that this was against AWS's terms, but I never bothered to look if that was actually the case. Thank you for the heads up!

It’s mostly in there to scare people into not doing it. AFAIK they’ve never taken action on that.

Of course if you abuse it, you’re asking for trouble.

Re: Slashing data transfer costs in AWS

#48
post #21

Earlier quoted context omitted.

Mantras are good for orgs that are not mature enough to do actual analysis. A lead developer left recently where I work and while it was likely higher pay that was the biggest decision to move, I suspect the real reason he left is higher ups simply don't listen when he says things like you can't just take full virtual machines on azure, refuse any rewrite/redesign while complaining about high azure spend.

Well, then they can flip a coin for which mantra to follow. If you pick "cloud bad" you'll get stories also about companies that refused to go to cloud when it makes sense to.

[deleted]

Re: Slashing data transfer costs in AWS

#49
post #36

Earlier quoted context omitted.

> A lot of org move to cloud based hosting because it enable them to go way further in FinOps / cost control I think a lot of orgs move to cloud simply because it's popular and gartner told them so. But taking a step away from that, it's really about self-service. When the alternative is logging a ticket for someone to manually misconfigure a VM and then fail to send you the login credentials, then your delivery is s…

> I think a lot of orgs move to cloud simply because it's popular and gartner told them so. Nah, I think it's mostly about the second part of your comment. Everyone hates waiting for months to get a VM or a database or a firewall rule because the infrastructure/DBA teams are stuck ten years in the past and take pride in their artisanal infrastructure building. So moving to the cloud eliminates a useless layer of time…

The layer will still be there, because those teams are now managing some cloud infrastructure central to the organization.

Re: Slashing data transfer costs in AWS

#50
post #36

Earlier quoted context omitted.

> A lot of org move to cloud based hosting because it enable them to go way further in FinOps / cost control I think a lot of orgs move to cloud simply because it's popular and gartner told them so. But taking a step away from that, it's really about self-service. When the alternative is logging a ticket for someone to manually misconfigure a VM and then fail to send you the login credentials, then your delivery is s…

> I think a lot of orgs move to cloud simply because it's popular and gartner told them so. Nah, I think it's mostly about the second part of your comment. Everyone hates waiting for months to get a VM or a database or a firewall rule because the infrastructure/DBA teams are stuck ten years in the past and take pride in their artisanal infrastructure building. So moving to the cloud eliminates a useless layer of time…

It also allows management to hide bad decisions and poor planning.

Project is a dud? just nuke the cloud project and no more charges for it.

Project is poorly architected and running like a dog? throw more resources at it.

Both of the above are harder to hide when you have to order equipment for on prem.

Post reply on HN