Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Customers Rack Up Hefty Bills for Moving Data

theinformation.com

Post reply on HN
Print view View original

51–60 of 252 posts

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#51

Post by YawningAngel »

notaround1111 wrote: post #17
snagglegaggle wrote: post #9

Earlier quoted context omitted.

At some point everyone realized they could charge a massive premium per bit and as long as everyone did it, customers would have to pay. So here we are.

Customers are also to blame, when comparing the costs of two services they tend to look at the cost of an instance hour, or lambda execution and often don't look at transfer costs. Even if a cloud provider had competitive transfer costs they likely wouldn't attract any new customers and would have less margin left over to subsidize the main cost customers look at, $ per instance hour. The less attention is paid to tr…

I don't think this is true. At $JOB the extent of our cloud cost management is me reading a breakdown by SKU and looking for obvious inefficiencies, and we are very aware of transit fees. I would imagine that anyone in the 5MM+ range has actual models that account for this stuff.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#52

Post by bsder »

danpalmer wrote: post #36
jrockway wrote: post #13

Earlier quoted context omitted.

It has no basis in reality. You can buy a 10Gbps transit circuit for $1500/month. That's $0.00046296296 per gigabyte.

That depends on how much saturation you can get. Most end users/companies can’t saturate this reliably, or have much spoiler traffic. Only the biggest players can minimise their seasonality enough to get close to this, and doing that is a service that it’s probably worth paying for.

Well, reverse it: Amazon is about $1K per 10TB.

That's an expensive disk drive.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#53

Post by kick »

nickjj wrote: post #49
nine_k wrote: post #24

Earlier quoted context omitted.

Because this is what the market agrees to bear. They want you to keep all of your data in their cloud, do all your processing there using their services (because doing it elsewhere incurs expensive egress fees), and get paid handsomely for your need to actually serve the data to your customers. This also makes a migration additionally expensive, because you would need to egress all your data (old logs, some fresh bac…

> Because this is what the market agrees to bear. I think it's more than that because a lot of folks aren't aware of the costs until they need to or want to move and then they get hit with a massive bill. So maybe they chalk it up to experience and pay the bill because they have no other choice or calculated it's still better in the long term. To me that's a much difference scenario than walking into a store and happ…

That's still the market.

The market isn't exclusively good, and has many failure modes. This is one of them.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#54

Post by asdfman123 »

qvrjuec wrote: post #44
yeldarb wrote: post #32

Wow, does it really cost 4-5x more to download a 1 GB file from S3 or Google Cloud Storage than it does to store it for a month? That is mind-blowing.

Bandwidth is expensive, storage is cheap.

Amazon Snowball, if I understand it correctly, is a box full of hard drives that they mail you to transfer data. So no bandwidth there.

It still costs $0.03/GB, which is on top of the $200 they charge you to use the service.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#55

Post by brightball »

whoisjuan wrote: post #11
toomuchtodo wrote: post #2

Anyone (no longer under NDA) know if Apple intends to migrate user iCloud data from AWS and GCP object stores to their own data centers at some point?

> no longer under NDA Do NDAs expire? AFAIK you sign an NDA and you’re bound to respect that forever. Unless the knowledge becomes publicly available.

IANAL

I have spoken to a lot of lawyers about contract terms and what I've been told for both NDA and Non-compete style agreements is that they have to have very clearly defined scope to be enforceable.

For a non-compete for example, it requires distance and specific field with a clearly defined time period (that's reasonable).

For an NDA you have to ensure that the covered information is explicitly labeled (which is why people have confidentiality labels in email footers) and that there is a defined time to expiration. An NDA without those two criteria place an undue burden on a person who is not being compensated for their compliance.

Reasonable time period is generally 2 years.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#56

Post by throwaway_bad »

ben509 wrote: post #38
gruez wrote: post #6

Is there a plausible explanation why egress fees from cloud providers costs around $0.1/GB? "Traditional" server providers such as Hetzner are able to offer bandwidth at orders of magnitude lower price (eg. $1.1/TB). I understand that cloud providers may have better interconnects or better uptimes, but that doesn't justify the magnitudes higher pricing.

Disclaimer: I worked on an AWS service team. This is, oddly enough, similar to a debate people have about consumers TV or Internet: should pricing be "unlimited" or "a la carte"? AWS is combining all your networking charges into one lump "outgoing data transfer" fee. So it's heavily marked up in comparison to what they're paying for the outgoing data transfer, and you're not sure how much is profit vs. whether it's g…

> If there's a compelling case for doing it differently, someone should do it and see if it works.

Cloudflare doesn't charge for bandwidth. I always throw cloudflare on top of anything I do, not because I really need a CDN or anything, but because the bandwidth cost would bankrupt me otherwise. The ceo of cloudflare gave the rationale on why they don't charge:

> There’s a fixed cost of setting up those peering arrangements, but, once in place, there’s no incremental cost. That’s why we have similar agreements to Backblaze in place with Google, Microsoft, IBM, Digital Ocean, etc. It’s pretty shameful, actually, that AWS has so far refused. When using Cloudflare, they don’t pay for the bandwidth, and we don’t pay for the Bandwidth, so why are customers paying for the bandwidth. Amazon pretends to be customer-focused. This is a clear example where they’re not.

https://news.ycombinator.com/item?id=20791563

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#58

Post by silon42 »

cosmie wrote: post #26
skunkworker wrote: post #3

For a service I built on AWS lambda, the far majority of the cost was just data egress. Unfortunately I haven’t found a competitor that offers the same kind of features that I’m using lambda for (running golang with additional custom binaries)

Azure is part of Cloudflare's Bandwidth Alliance[1]. If you use Azure's serverless functions and put your service behind Cloudflare, you'll get free egress. [1] https://www.cloudflare.com/bandwidth-alliance/

so, bandwidth cartel

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#60

Post by chickenpotpie »

I've actually been working on a library to help mitigate cloud storage lock-in. The idea is to treat cloud storage providers like disks are treated in RAID. For example, you have 3 separate cloud providers. Cloud providers 1 and 2 have every other byte of data striped across them. Cloud provider 3 has parity data. To pull a file you can only need 2 of the 3 cloud providers. If you don't like how a cloud storage provider is treating you or charging you just pull from the other 2 providers and use them as a backup in case one goes down. You can also just remove them entirely from the equation, but then you have no redundancy if one of the others goes down. It gives you a lot of negotiating power to lower egress costs because you can just pull them out of the equation at any time and reinstate them once you get better pricing.
Post reply on HN