Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS: Inaccurate Estimated Billing Data – $1.7 billion

news.ycombinator.com

Post reply on HN
Print view View original

471–480 of 793 posts

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#471

Post by pudgywalsh »

donavanm wrote: post #449

Ive dealt with this error at AWS. It’s a unit error. In my case we _meant_ to charge like 5¢/GB, but missed the unit (GB), and then the billing system defaults to bytes. 5¢ per Byte of data transferred meant some customers were seeing MM bills within hours. Got paged by support around 2am, had it fixed and amendments issues by 3-4am, apology emails shortly after. Services emit metering values that arent directly tied…

"I must've put a decimal point in the wrong place or something. I always mess up some mundane detail."

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#472

Post by yunnpp »

idiotsecant wrote: post #390
VulgarExigency wrote: post #289

Earlier quoted context omitted.

The user is asking me to calculate how much money they should charge their customer. The values they've given me are 0.45, 1.67, and 2.50. This is 2.50 + 1.67 + 0.45 = 4.62, but it could be any other number. Perhaps we should be concatenating the numbers instead. Wait! The . could also mean multiplication. 0 . 45 . 1. 67 . 2 . 50 = 3015000. But wouldn't multiplying by 0 zero it out? That can't be right, we wouldn't b…

Would be funny if it wasn't so close to true

'My absurd statement doesn't sound right, so the "opposite" (assuming it's well-defined and unique) must be true' is peak LLM logic. You can tell it was trained on Reddit commentary.

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#473

Post by AlienRobot »

bobbiechen wrote: post #254

AWS saw Anthropic billing a guy for $16 million on zero usage and thought, why stop at the millions? https://www.techtimes.com/articles/320266/20260712/anthropic...

>AI billing audit startup Vaudit reviewed $34 million in AI invoices submitted by 60 enterprise customers and found approximately $1.7 million in mistaken overcharges — a billing error rate of roughly five percent.

That sounds bad.

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#474

Post by akdev1l »

01284a7e wrote: post #466
donavanm wrote: post #449

Ive dealt with this error at AWS. It’s a unit error. In my case we _meant_ to charge like 5¢/GB, but missed the unit (GB), and then the billing system defaults to bytes. 5¢ per Byte of data transferred meant some customers were seeing MM bills within hours. Got paged by support around 2am, had it fixed and amendments issues by 3-4am, apology emails shortly after. Services emit metering values that arent directly tied…

No tests? Just mess up some mundane detail [1] and voila! Wake-up calls and heart attacks for 100,000s of administrators? 1: "Oh, well, this is not a mundane detail, Michael!" https://www.youtube.com/watch?v=3fGHaVn5rGo

Not even tests but just some basic anomaly detection lol.

Like maybe if the bill amounts increase by like 10M% there should be someone that looks into it

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#477

Post by wat10000 »

positr0n wrote: post #455
drew870mitchell wrote: post #285

Earlier quoted context omitted.

AWS is basically a utility. I think it's inevitable that their carelessness around billing will end up with them being regulated like one.

I can't think of another regulated utility that doesn't provide service to (essentially) all humans directly in their homes. Everyone knows what water and electricity are, the vast majority couldn't explain what service AWS provides.

And utilities are typically natural monopolies. They're good candidates for regulation because they're essentials and they don't have competitive forces to keep them behaving reasonably.

AWS has plentiful competitors. If you don't like their behavior, don't patronize them!

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#478

Post by AlienRobot »

astonex wrote: post #142

From their status page >The second path involves rolling back a recent change to the billing computation subsystem. Want to bet AI code was involved?

Do people not write tests anymore?

How is 1 + 1 = 2 hard to test?

Vibe coded implementation paired with vibe coded tests made to fit the implementation?

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#479

Post by qurren »

akdev1l wrote: post #474
01284a7e wrote: post #466

Earlier quoted context omitted.

No tests? Just mess up some mundane detail [1] and voila! Wake-up calls and heart attacks for 100,000s of administrators? 1: "Oh, well, this is not a mundane detail, Michael!" https://www.youtube.com/watch?v=3fGHaVn5rGo

Not even tests but just some basic anomaly detection lol. Like maybe if the bill amounts increase by like 10M% there should be someone that looks into it

You overestimate how much people give shits at big techs like Amazon. When literally everything is driven with sticks instead of carrots, the work culture does not invite employees to proactively care about product quality.

You'd be better off letting the heart attacks happen and take the 3am on-call and be the hero instead. It would be good promo doc material, and being a hero is extremely good insurance against getting kicked out of the country (via the PIP->H1B grace period expiry mechanism).

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#480

Post by 01284a7e »

akdev1l wrote: post #474
01284a7e wrote: post #466

Earlier quoted context omitted.

No tests? Just mess up some mundane detail [1] and voila! Wake-up calls and heart attacks for 100,000s of administrators? 1: "Oh, well, this is not a mundane detail, Michael!" https://www.youtube.com/watch?v=3fGHaVn5rGo

Not even tests but just some basic anomaly detection lol. Like maybe if the bill amounts increase by like 10M% there should be someone that looks into it

If only there was some way to get anomaly detection services [1] inside of AWS...

1: https://aws.amazon.com/what-is/anomaly-detection/

Post reply on HN