Live data from Hacker News

AWS: Inaccurate Estimated Billing Data – $1.7 billion

news.ycombinator.com

321–330 of 793 posts

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#322

One can almost smell the vibes. This is peanuts compared to a major cybersecurity catastrophe that’s surely in the making. To give credit to the technology and the people using it - and I’m not being facetious - it’s actually incredible that at the current levels of usage the unprecedented catastrophic event has not yet happened.

Oh, that's the really fun part. The unprecedented catastrophic event is already happening. Several of them, in fact.

By the time we notice, it'll be too late.

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#323
post #98

Woke up to a billing alarm email. Thought I had leaked my AWS keys accidentally and somehow run up 437 billion dollars of charges. Joke's on them though, I don't have 437 billion dollars Anyways I didn't need coffee. That produced an adrenaline release unlike any I've experienced before. Thanks AWS

I went through a two month long fight with AWS over a compromised account. I lost 20K even when my previous usage for the last 3 years was like $20/month, and I had more invoices coming worth tens of thousands of dollars. AWS refused to help until I stopped the breach myself, which required me to spend several all nighters after work learning how to script for AWS infra. They also refused to close my account after the breach was under control. They asked me to sign a shared responsibility agreement before they could look at my case, which I refused to do. I finally contacted the AG office in my state and they email AWS directly. In less than 24h I had a AWS manager calling me to fix my account and issue a refund for the 20K. Still, they refused to close my account after all this ordeal, apparently there’s no way for you to completely get out of AWS once you are in. It’s the shittiest business ever.

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#324
post #239

I got a 20K bill once and it was actually drafted from my bank account. It took me a couple of months and involving the office of the AG of my state to get the issue resolved and get my money back. Since then I never touched any AWS product, moved my small stuff to Azure. It’s been years since AWS have these issues with billing, you can find the stories online, students billed 60K for a compromised account launching…

I wouldn't expect their detection of hacked accounts to be 100% correct. Sure, it might be obvious when a human takes a look, but humans can't proactively look at every account's usage.

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#325

Earlier quoted context omitted.

Wow: In this role you will: - Design and build agentic AI systems that analyze, generate, and validate... - Build agentic architectures that compose specialized AI agents dynamically... - Build AI-driven continuous validation frameworks powered by agentic workflows and large language models that autonomously manage... This is invoicing? If ever there was a domain that was purely deterministic, you'd hope it was invoi…

The irony is, the only purely deterministic thing, will be token consumption...

I severely doubt the world ever gets to such a point that the entire world melts into AI hallucination. And token consumption depends on so many other things, it's not all that deterministic either.

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#326
post #43

If you owe the bank $100, that's your problem. If you owe the bank $1.7 billion, that's the bank's problem.

I saw this in action on a smaller scale. In a past job, my wife organized events for a decent sized company. After an event, she'd typically have a $300k+ balance on her corporate Amex. When she went on maternity leave, the person filling in for her job neglected to actually pay the bills, so when she returned there were quite a few emails and voicemails from Amex regarding the over $500k balance. The messages starte…

I think that this might reflect more on Amex to be honest.

Amex realises that threatening would hurt their business trust more than anything. During the great depression, Amex accepted checks from other banks which were falling and paying through their own wallet as a matter of integrity. Amex has always been built around this idea of trust and prestige.

They make most of money from what I have heard on the transaction fees which are more than others (3% compared to 1%). They might get desperate but I am sure that they are one of the last guys who would wanna threaten you if you are paying some large bills for them (as compared to normal credit card companies which might even hire people to extract your loans in some messy situations)

So perhaps be so rich that the credit card company understands it as well and treats ya differently :-D

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#327

Earlier quoted context omitted.

That job description feels so far beyond parody that I could scarcely believe it until opening the link! What a world.

It gets worst: "Senior Software Development Manager, AWS Global Bill Generation" https://www.amazon.jobs/de/jobs/10471948/senior-software-dev... "We're transforming from monthly batch processing and manual war rooms to continuous billing, autonomous agents, and self-healing infrastructure. We believe operational burden is a technical problem, not a staffing problem" This looks clearly...a staffing problem...

The best bit of that is:

> In this role, you will own end-to-end bill run execution across all AWS partitions, drive the technical vision for autonomous billing operations, and build the team that ensures every customer receives an accurate cost estimated in minutes ...

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#328

One can almost smell the vibes. This is peanuts compared to a major cybersecurity catastrophe that’s surely in the making. To give credit to the technology and the people using it - and I’m not being facetious - it’s actually incredible that at the current levels of usage the unprecedented catastrophic event has not yet happened.

Always messing up some mundane detail!

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#329

One can almost smell the vibes. This is peanuts compared to a major cybersecurity catastrophe that’s surely in the making. To give credit to the technology and the people using it - and I’m not being facetious - it’s actually incredible that at the current levels of usage the unprecedented catastrophic event has not yet happened.

some things never change. Pre AI I was always shocked that such large and complex systems actually run as well as they do. Especially after getting to see how the sausage is made/works.

Re: AWS: Inaccurate Estimated Billing Data – $1.7 billion

#330
post #322

One can almost smell the vibes. This is peanuts compared to a major cybersecurity catastrophe that’s surely in the making. To give credit to the technology and the people using it - and I’m not being facetious - it’s actually incredible that at the current levels of usage the unprecedented catastrophic event has not yet happened.

Oh, that's the really fun part. The unprecedented catastrophic event is already happening. Several of them, in fact. By the time we notice, it'll be too late.

its like slowly boiling the frog
Post reply on HN