AWS Customers Rack Up Hefty Bills for Moving Data • Hacker News Board

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#221

Post by satanspastaroll » Tue, Oct 22, 2019, 7:25 AM UTC

nine_k wrote: post #24

gruez wrote: post #6

Is there a plausible explanation why egress fees from cloud providers costs around $0.1/GB? "Traditional" server providers such as Hetzner are able to offer bandwidth at orders of magnitude lower price (eg. $1.1/TB). I understand that cloud providers may have better interconnects or better uptimes, but that doesn't justify the magnitudes higher pricing.

Because this is what the market agrees to bear. They want you to keep all of your data in their cloud, do all your processing there using their services (because doing it elsewhere incurs expensive egress fees), and get paid handsomely for your need to actually serve the data to your customers. This also makes a migration additionally expensive, because you would need to egress all your data (old logs, some fresh bac…

There are some bandaid-solutions people have used to migrate away from this without taking a major hit. One is utilizing the free outbound bandwidth from lightsail, and bigger transfers can use snowball which is 'only' $0,03/GB outbound + shipping

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#222

Post by satanspastaroll » Tue, Oct 22, 2019, 7:28 AM UTC

geggam wrote: post #169

EB66 wrote: post #126

Earlier quoted context omitted.

+1 for this. People forget just how affordable it can be to maintain your own infrastructure. You can have the hardware and network capable of supporting 10X your average traffic loads and still have it operate far more cost effectively than the equivalent traffic on AWS. At my work, we slashed our overall hosting costs by moving a data warehouse off of AWS and on to our own self-maintained infrastructure. But with a…

I have had this debate with so many people, AWS is not cheaper than your own hosting. Period. What you get from AWS is the logistics pipeline is already built as is the infrastructure should you suddenly require to serve factors of traffic more. The ROI of AWS comes from the backend and capex vs opex debates. Example : The CFO can go to the board and explain we are getting ready to reduce opex by laying off 10 develo…

I think the best cases for AWS is counter to marketing, very small companies. Their native offerings do make things great for a while when you're starting out.

The other is stupendous burst activity, like you just need a thousand(s) cores for a couple hours. Of course this doesen't mean the baseload has to be in AWS, just easy for small teams.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#223

Post by fauigerzigerk » Tue, Oct 22, 2019, 7:34 AM UTC

Pfhreak wrote: post #193

fauigerzigerk wrote: post #69

Earlier quoted context omitted.

>But I generally don't find this too hard to follow? This is just the static picture though. What's harder to predict are the consequences of some innocuous looking code change.

Sure, but surely teams have monitoring on their usage, right? With automated rollbacks or at least one click manual rollbacks?

Sure, but rolling back work that was already done is a waste of development resources.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#224

Post by satanspastaroll » Tue, Oct 22, 2019, 7:47 AM UTC

crazysim wrote: post #160

js4ever wrote: post #105

I have recently talked with a friend about a very cheap way to egress from AWS, at $2.5 per TB with the AWS lightsail $5 instance which include 2TB of egress per instance. This friend have extracted 200TB of backups for $500 instead of $17,000!

That's against Lightsail TOS, if you care.

You can do it somewhat cheaper than direct upload with snowballs, at ~$33/TB (S3 + 80TB/$250 shipping), but it's still very pricey

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#225

Post by Havoc » Tue, Oct 22, 2019, 7:53 AM UTC

chickenpotpie wrote: post #185

Havoc wrote: post #151

Earlier quoted context omitted.

Pretty sure paying 3 cloud providers is more expensive than 1 cloud provider

It's actually cheaper when you want global redundant storage. Cloud providers often charge twice as much for global redundant data. RAID 2,3,4 offer global redundancy, but only take up 1.5 times us much space. Instead of paying twice as much you only end up paying 1.5 times as much because you can get away with locally redundant pricing. If you're large enough you'll actually save money by having more negotiating pow…

Sounds interesting. Link?

I'd probably use commodity VMs for this rather than big clouds if it is indeed resilient.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#226

Post by satanspastaroll » Tue, Oct 22, 2019, 7:53 AM UTC

Elect2 wrote: post #123

Data transfer fee is the biggest reason stopped me from using AWS/GCP/Azure. Anyone using Oracle Cloud? I found their bandwidth is really cheap.

The thing about that is that it's Oracle. You might survive stuffing your hand in a hornet's nest, but it's unlikely.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#227

Post by _16k4 » Tue, Oct 22, 2019, 8:16 AM UTC

partiallypro wrote: post #67

tyingq wrote: post #4

"The charges don’t appear to be a case of cloud providers gouging their customers" I disagree on this one. The margins on egress are, well...egregious.

If I'm not mistaken AWS has the highest egress rates of the major cloud providers.

I just put 1 TB in AWS/Azure/GCP's cost calculators.

AWS (US East 1, no free tier) - $92.07

Azure (East US) - $88.65

GCP (Americals) - $122.88

I'm quite surprised by GCP being the highest cost here, and by such a wide margin.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#228

Post by privateSFacct » Tue, Oct 22, 2019, 9:13 AM UTC

semiotagonal wrote: post #168

berns wrote: post #85

Earlier quoted context omitted.

> Is there a plausible explanation why egress fees from cloud providers costs around $0.1/GB? Isn't DigitalOcean a cloud provider ($0.01/GB)?. Isn't Oracle a cloud provider (first 10 TB free, $ 0.0085 after)? Isn't OVH a cloud provider (free bandwidth)?

Unless I'm making a mistake someplace, Digital Ocean includes 1TB egress to the internet from a $5-per-month host. On Google Cloud Platform that egress would cost $120/month by itself. https://www.digitalocean.com/pricing/#Compute https://cloud.google.com/compute/network-pricing

What you find is if you spin up 1000 of these $5 hosts (ie, $5000 per month which is pretty) and try to egress 1PB per month you can't. Either they disconnect you, the systems can't actually handle it or some other gimmick.

One thing folks like about AWS - you can actually know what you will be paying and there is no fake / hidden limits.

That's my somewhat outdated experience wasting a TON of time on this idea ages ago.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#229

Post by chriselles » Tue, Oct 22, 2019, 9:32 AM UTC

It sounds very similar to physical documents/records storage service “roach motel”-like fee structures.

Make it nearly free to move documents in.

Charge monthly rent per box of documents.

But charge like a wounded bull if customers try to permanently remove documents.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#230

Post by tallanvor » Tue, Oct 22, 2019, 10:01 AM UTC

gruez wrote: post #6

Is there a plausible explanation why egress fees from cloud providers costs around $0.1/GB? "Traditional" server providers such as Hetzner are able to offer bandwidth at orders of magnitude lower price (eg. $1.1/TB). I understand that cloud providers may have better interconnects or better uptimes, but that doesn't justify the magnitudes higher pricing.

I doubt that you'll find you can consistently push or pull 1Gbps from a Hetzner server. I certainly can't. --That's not to say that they're a bad deal, but their total bandwidth is nowhere near what the major cloud providers can give you.