Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Customers Rack Up Hefty Bills for Moving Data

theinformation.com

Post reply on HN
Print view View original

111–120 of 252 posts

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#111

Post by pmoriarty »

I recently saw a talk by a couple of former Google employees who have a business helping cloud customers save money, and they were saying that they see a lot of money being wasted by companies in overprovisioning and neglecting to shut down or delete unused resources like VMs or virtual disks.

Some of their advice for saving money was to keep track of who created each resource and why, so there's less reason to doubt whether an apparently unused resource can be deleted, and to make some limits regarding how many resources can be automatically created (especially in dev environments). Some other ideas were to look for signs of inactivity like low CPU or bandwidth use, and consider deleting such little used resources.

There was much more to the talk, but those were some of the highlights that I can remember without digging out my notes. It was a good talk.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#112

Post by nostrebored »

rsync wrote: post #107
ben509 wrote: post #38

Earlier quoted context omitted.

Disclaimer: I worked on an AWS service team. This is, oddly enough, similar to a debate people have about consumers TV or Internet: should pricing be "unlimited" or "a la carte"? AWS is combining all your networking charges into one lump "outgoing data transfer" fee. So it's heavily marked up in comparison to what they're paying for the outgoing data transfer, and you're not sure how much is profit vs. whether it's g…

"I think AWS's billing is probably already on the falling side of diminishing marginal returns. That is, it's complex enough that more information would tend to hinder customers from getting the best price. Right now, if I plan to reduce my data charges, I have one variable to tinker with." No, it's two variables - the egress charges you refer to and the actual cost to store the data . We[1] have found that it is, as…

And how many PoPs regional interconnects, highly availabile, high throughout connections, cross continent highly available connections do you have? Do you detect failure across these connections? Do you detect grey failures? Do you have a team of infrastructure engineers to look after this network?

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#113

Post by nostrebored »

GordonS wrote: post #100
ben509 wrote: post #38

Earlier quoted context omitted.

Disclaimer: I worked on an AWS service team. This is, oddly enough, similar to a debate people have about consumers TV or Internet: should pricing be "unlimited" or "a la carte"? AWS is combining all your networking charges into one lump "outgoing data transfer" fee. So it's heavily marked up in comparison to what they're paying for the outgoing data transfer, and you're not sure how much is profit vs. whether it's g…

> AWS is combining all your networking charges into one lump "outgoing data transfer" fee > So it might be fairer if AWS broke out separate line items for internal, incoming and outgoing data transfer This explanation doesn't cut it for me - most (all?) "traditional" VPS providers don't charge for ingress traffic, and I doubt anyone, ever, has charged for internal traffic. So what exactly is 'all the networking charg…

How many of these VPS providers are actually managing global highly availabile network infrastructure?

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#114

Post by bobbonew »

tyho wrote: post #101
tyingq wrote: post #94

Earlier quoted context omitted.

Snowball still charges ~3 cents/GB to get it out of S3 and into Snowball.

Well, there is always https://aws.amazon.com/snowmobile/

Keeping track of all the different offerings AWS has is a full time job.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#115

Post by FlyingAvatar »

js4ever wrote: post #105

I have recently talked with a friend about a very cheap way to egress from AWS, at $2.5 per TB with the AWS lightsail $5 instance which include 2TB of egress per instance. This friend have extracted 200TB of backups for $500 instead of $17,000!

Reading this, I immediately thought: the cost has to go way down with 200TB of volume.

It goes down, but only to $14,131.11!

Nice find. I will file this away if I ever need to do a full remote recovery.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#116

Post by 3wolf »

The $0.01/GB inter-availability-zone data transfer cost can be a killer for misconfigured workloads. It's a shame AWS doesn't make this more clear on their pricing page. I've seen people run Spark clusters across multiple AZs, incurring a huge costs whenever a shuffle happens.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#117

Post by nostrebored »

bsder wrote: post #52
danpalmer wrote: post #36

Earlier quoted context omitted.

That depends on how much saturation you can get. Most end users/companies can’t saturate this reliably, or have much spoiler traffic. Only the biggest players can minimise their seasonality enough to get close to this, and doing that is a service that it’s probably worth paying for.

Well, reverse it: Amazon is about $1K per 10TB. That's an expensive disk drive.

Thinking of it as a disk drive is fallacious

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#118

Post by spike021 »

pmoriarty wrote: post #111

I recently saw a talk by a couple of former Google employees who have a business helping cloud customers save money, and they were saying that they see a lot of money being wasted by companies in overprovisioning and neglecting to shut down or delete unused resources like VMs or virtual disks. Some of their advice for saving money was to keep track of who created each resource and why, so there's less reason to doubt…

Makes sense. If you're not properly tagging your resources then it can be very hard to track down if/where it's been used, how often, or when it was used last. You can automate/template as much orchestration as you want with stuff like Terraform to bring up well-defined resources, but there will always be outliers without tagging.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#119

Post by cabaalis »

Merrill wrote: post #61

My GP's practice went to a SaaS provider for electronic health records. Never mind transmission costs -- there is no standard way to export their patient records in order to move to a different provider. It requires a custom consulting services contract with their existing provider to retrieve their patient records, at an exorbitant price tag.

This is quite typical. Even with MIPS PI measures and the push for interoperability. Getting patient data in bulk out of a hosted EHR can be a frustrating exercise. What they are doing is holding the providers' data hostage, in the name of HIPAA and security.

Re: AWS Customers Rack Up Hefty Bills for Moving Data

#120

Post by Elect2 »

js4ever wrote: post #105

I have recently talked with a friend about a very cheap way to egress from AWS, at $2.5 per TB with the AWS lightsail $5 instance which include 2TB of egress per instance. This friend have extracted 200TB of backups for $500 instead of $17,000!

So where is your friend's data in? s3? ebs? I'm curious what is the cost to transfer data from s3/ec2 to Lightsail.
Post reply on HN