Or... Build your own cloud and transfer data to your hearts content for free (minus power).
Slashing data transfer costs in AWS
201–210 of 268 posts
Re: Slashing data transfer costs in AWS
#202Earlier quoted context omitted.
Apparently we're doing the impossible for over 12 years now. Who knew? Some people act like it's some kind of black magic. It's not. We've some customers in our DC and some on AWS for various reasons. AWS isn't less problematic. AWS is about 10x more expensive. Both on prem and cloud require people familiar with them and cloud-engineers are in no way cheaper. Only meaningful problem is that on-prem requires some up f…
Cloud clearly makes sense for: - small business with at least some reliability expectation, and little to none IT expertise - huge workload requirement volatility - having someone else to blame - solution is already working in cloud, with teams being very comfortable there, and perceiving on-prem as “enemy” (analogy: forcing devs to rewrite stuff from haskell to java) - that extra cost is small budget line for you On…
Cloud doesn't make sense for small business. A vps would. If you are spending less than 100,000 you probably don't need it for your 10,000 million or less daily visitors
Re: Slashing data transfer costs in AWS
#203An alternative to sophisticated cloud cost minimization systems is…….. don’t use the cloud. Host it yourself. Or use Cloudflare which has 0 cents per gigabyte egress fees. Or just rent cloud servers from one of the many much cheaper VPS hosting services and don’t use all the expensive and complex cloud services all designed to lock you in and drain cash from your at 9 or 12 or 17 cents per gigabyte. Seriously, if you…
The tipping point for self-hosting would be the point at which paying for full-time, 24/7 on-call sysadmins (salary + benefits) is less than your cloud hosting bill.
That's just not gonna happen for a lot of services.
Re: Slashing data transfer costs in AWS
#204Earlier quoted context omitted.
Cloud clearly makes sense for: - small business with at least some reliability expectation, and little to none IT expertise - huge workload requirement volatility - having someone else to blame - solution is already working in cloud, with teams being very comfortable there, and perceiving on-prem as “enemy” (analogy: forcing devs to rewrite stuff from haskell to java) - that extra cost is small budget line for you On…
Cloud doesn't make sense for small business. A vps would. If you are spending less than 100,000 you probably don't need it for your 10,000 million or less daily visitors
If you're spending less than 100,000, you almost certainly aren't spending enough to pay salary + benefits for a sysadmin.
Re: Slashing data transfer costs in AWS
#205I'll share my trick :) Lightsail instances can be used to "proxy" data from other AWS resources (eg EC2 instances or S3 buckets). Each Lightsail instance has a certain amount of data transfer included in it's price ($3.5 instance has 1TB, $5 instance has 2TB, $10 instance has 3TB, $20 instance has 4TB, $40 instance has 5TB). The best value (dollar per transferred data) is the $10 instance, which gives you 3TB of traf…
https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspe…
> You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services
This requires proving the users intent, which is not obvious except in the most blatant of cases (i.e. using Lightsail as a bent-pipe by writing the exact bytes you're reading). If it is a "CSV to Parquet translation layer", how would AWS possibly prove it's anything other than what it claims to be? You'd be paying a few more cents for compute, but that's the price of plausible deniability
Re: Slashing data transfer costs in AWS
#206GCP patched a similar loophole [1] in 2023 presumably because some of their customers were abusing it. I'd expect AWS to do the same if this becomes widespread enough. [1]: https://cloud.google.com/storage/pricing-announce#network
I know of a way to get data out of GCP for free, although I haven't tried it in practice. Wonder if I could find a buyer for this info ;)
A guess: tunnel through 169.254.169.254 DNS server?
Re: Slashing data transfer costs in AWS
#207Earlier quoted context omitted.
https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspe…
> You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services This requires proving the users intent, which is not obvious except in the most blatant of cases (i.e. using Lightsail as a bent-pipe by writing the exact bytes you're reading). If it is a "CSV to Parquet translation layer", how would AWS possibly prove it's anything other than what it claims to be? You'd be payin…
> This requires proving the users intent
Companies are permitted to deny service to anyone at any time for any (non-protected) reason. They typically don't have to justify service terminations to a court of law. Who would they be required to prove user intent to, and why?
Re: Slashing data transfer costs in AWS
#208Earlier quoted context omitted.
Not using VPCs (no need without instances/containers/RDS) mean most of the “absurd” costs go away. It’s cheap by any standard.
VPCs don’t cost anything.
Sorry, that was indeed nonspecific, you’re right. The add-on features for VPCs are commingled with the concept for me since they almost always go hand in hand. Internet gateways, transit gateways, EIPs, service endpoints, etc., and their fixed costs. Yuck.
Re: Slashing data transfer costs in AWS
#209[deleted]
Re: Slashing data transfer costs in AWS
#210Earlier quoted context omitted.
https://aws.amazon.com/service-terms/ > 51.3. You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services (e.g., proxying network traffic from Services to the public internet or other destinations or excessive data processing through load balancing or content delivery network (CDN) Services as described in the technical documentation), and if you do, we may throttle or suspe…
> You may not use Amazon Lightsail in a manner intended to avoid incurring data fees from other Services This requires proving the users intent, which is not obvious except in the most blatant of cases (i.e. using Lightsail as a bent-pipe by writing the exact bytes you're reading). If it is a "CSV to Parquet translation layer", how would AWS possibly prove it's anything other than what it claims to be? You'd be payin…
This is not a court. Amazon does not have prove anything to anyone.
There is going to be a program that will have rules to detect patterns in customer traffic and automatically block when those patterns are tripped.
At best you could complain in the forums and maybe if you are lucky a sympathetic community manager may look into your use case.