Same spiel as all the big clouds...not allowed to protect yourself. Unlimited risk all on you and afterwards you may beg for forgiveness from support and when that fails go to the other support page aka hn.
Bloody scam.
Netlify just sent me a $104k bill for a simple static site
741–750 of 836 posts
Re: Netlify just sent me a $104k bill for a simple static site
#742Earlier quoted context omitted.
That sounds awful and it's not even cheap!
I guess that shows how bad the alternative is. How much is unlimited talk, text, data, hotspot with a watch data add-on where you are from? edit: Actually I was wrong–it is $30, not $35.
In New York, I pay $26 for T-Mobile, one of the most mainstream carriers.
Re: Netlify just sent me a $104k bill for a simple static site
#743Just switched my site to Cloudflare Pages. It took all of 5 minutes including futzing around with DNS stuff through the (ever-annoying) Google Domains settings page. I'm usually pretty chill, and can't help but feel like I'm overreacting - but the risk of being sent an unexpected bill of even a few hundred dollars for something outside my control is unacceptable. Trusting that Netlify will do right by me and forgive…
Your comment with it taking only 5 minutes made me move my sites. And indeed it was super fast. Thanks for the needed motivation!
I don’t think you are overreacting. I had Netlify in pro mode at work and their pricing tactics are predatory. It was really hard to figure out why I got charged a seat, even though nobody had rights etc. And I couldn’t then remove these seats, I had to wait a month and hope I figured out where those contributors came from.
Re: Netlify just sent me a $104k bill for a simple static site
#744Earlier quoted context omitted.
You're welcome to disagree, of course. My main concern is to explain what the principles are. I'm not saying we apply them perfectly—sometimes we make bad calls. I can tell you pretty much for certain though, that we'd hear many more complaints if a Reddit thread about a customer support shitstorm stayed on HN's front page for very long. Btw, the Customer Support Fuckup category is one of several $X where HN has beco…
Thanks for explaining. Something that makes me feel uneasy about the fact that the post gets hidden is that this strongly benefits Netlify. It seemed like lots of people moved off Netlify after reading the post. I'm not suggesting that HN actively took an action in Netlify's favor, but the potential is there. Is the algorithm for flame war detection open source? Or do we essentially need to trust you that there was n…
We didn't have any private contact with anyone about this post other than a couple users emailing to ask why it wasn't on the front page anymore. Nor were we thinking about who would or wouldn't benefit—that never crossed my mind. I was just thinking about standard HN moderation practice.
I don't know how to get every user to trust us. All we can do is answer questions when asked. That seems to be enough to satisfy most of the community most of the time, and it's probably not possible to do a lot better than that, much as I would like to.
Re: Netlify just sent me a $104k bill for a simple static site
#745Earlier quoted context omitted.
1. Even if you assume perfectly good intentions, they're not incentivized to fix the problem because they're not on the hook for the bill, but stand to profit from it. Their margins are eye-watering, so sending out a $100k bill for a service which cost them no more than $5 to provide doesn't have a downside (other than bad PR, which they seem to be blind to). If providing this bandwidth cost them $50k rather than $5,…
> If providing this bandwidth cost them $50k rather than $5, No, I believe you are assuming that proving a "hard limit" feature is cheaper than just refunding people from time to time. If you consider the all the products AWS has to offer, all the different ways they are billed, then figuring out what do to for each product once that limit is reached, and potentially doing a destructive action, and then all the code…
> There are 100s of AWS services and many where you can't just apply a sensible rejection policy once you hit that hard limit without doing something the user cannot recover from. It's only not hard because you aren't actually thinking about the matrix of AWS products that exist and how they are billed.
I don't think you understand what a hard limit is. When I set a hard limit, that means that under no circumstance should I be billed more than this amount. From this you should be able to deduce that the platform needs to stop you from ever entering into a situation where they could end up having to decide between nuking your data, or overcharging you.
This means that for all data storage scenarios, they should apply limits to projected monthly usage rather than actual monthly usage.
If 1 GB of storage costs $1 per month, and my monthly limit is set to $10, then they should not allow me store more than 10GB of data, rejecting writes as needed. This design guarantees that my hard limit won't be exceeded and the data is never at risk.
Any service which doesn't persist data can apply limits based on actual usage, and safely shut them down when the limit is exceeded.
> Does Amazon drop your entire database once the credit limit is reached?
No, they should refuse further writes when your database reaches a size that would exceed $10 a month. That's a safe behavior.
> Consider secret manager. You are charged 40 cents/month per secret. You have 100 secrets, so $40/mo, but you set your hard limit to $20.
If you set your hard limit to $20, then Amazon should refuse to create more than 50 secrets. That's a safe behavior which limits your spend to at most $20.
> Building all this out just so that some college kid isn't charged a bill they could obviously never pay likely isn't a good use of resources. It's not likely AWS has a reputation of being greedy either, if you explain the situation they refund you. If they are truly doing this to make money they have remarkably poor execution.
This is just a bad faith argument, everyone from broke college students to medium sized businesses has complained about this at one point or another. Relying on the good will of a megacorporation to forgive a surprise $100k bill is nuts and if you scroll through these comments you'll find some that claim that AWS is no longer forgiving these bills like they used to.
I'm not claiming that this solution is perfect, or that it would cover every failure mode, but it sure beats the status quo and would solve the vast majority of surprise bills that result from unexpected traffic.
Re: Netlify just sent me a $104k bill for a simple static site
#746Earlier quoted context omitted.
That sounds awful and it's not even cheap!
I guess that shows how bad the alternative is. How much is unlimited talk, text, data, hotspot with a watch data add-on where you are from? edit: Actually I was wrong–it is $30, not $35.
My phone bill is 10€. I get unlimited calls and text, 100Gb 4G/5G then unlimited 3g, in France.
I could also get 50 MB data, 2h calls, unlimited texts for 2€.
Re: Netlify just sent me a $104k bill for a simple static site
#747Render may have the same problem: https://community.render.com/t/usage-100gb-for-a-static-site...
(Render CEO) Cloudflare DDoS protection is built into Render, and we don't charge for bandwidth classified as DDoS.
Re: Netlify just sent me a $104k bill for a simple static site
#748Earlier quoted context omitted.
We did this at DigitalOcean for similar reasons, wasn't a feature that was commonly used. Additionally, when you set that limit people then get upset because usually when they go over it for a good reason, like going viral, they aren't anticipating it, and just when their traffic is most valuable the site is down. What Netlify is doing here is really the best approach for both parties. And typically speaking a $104k…
That sort of adversarial weaponization of credit cards is really a US thing. In most of the world, payment is not relevant to the question of liability: you create a debt, whether or not you can debit a card. So, looks like DigitalOcean is another name to avoid. Also: what's common? At the scale these companies so desire, small percentages are still thousands of users. Supporting them with what's ultimately a fairly…
Partially agree, yes the liability is independent of the payment. But in practice the one who has the money has the benefit of the status quo. The other party without money has to actively take legal action which is costly enough that it's often not pursued.
Re: Netlify just sent me a $104k bill for a simple static site
#749Earlier quoted context omitted.
Frankly the only reason I can even come up with that Netlify wouldn't have such controls in place is exactly if they do _not_ simply forgive these sorts of jumps in costs (as the CEO here seems to be claiming). I'm pretty sure if they'd be left holding the bag, they'd manage to find some way to cut off these kinds of jumps in usage.
Maybe it’s a tax dodge! “Forgive” 100k of “overages” which cost Netlify next to nothing, then report it as a write off on taxes.
Not a dodge, (tax) accounting doesn't work like that. You can create accounts receivable of 100k and then write them off, that leaves you in exactly the same state as if you had just not entered the whole thing into the books at all. No benefit to "writing off".
Re: Netlify just sent me a $104k bill for a simple static site
#750After doing some math it doesn't feel like a ddos: - $104k at ($55 / 100 GB) = 189 TB of traffic - It means the popular ~3.5 MB media file was downloaded ~54M times - Which sound like a lot, but if you get popular in a country with 1.4B people, it's not (~3% accessed). What if it happens at AWS Cloudfront? At $.1/GB it sums up to ~$18k. In the light of these, Netlify's offer of ~$5k seems generous.
The author claimed Netlify's own support agent said it was an L7 ddos and offered a 95% concession because it was a ddos. There is no reason to question whether it was a ddos or not because, allegedly, both parties in this dispute already agree it was a ddos.
Also, DDoS or not DDoS it's very reasonable to believe that an individual (or even a company) isn't ok for a 100K USD bill.
Billing amounts should go through quotas requests, so you can explicitly ask to be migrated to the upper level, but by default have an active safety net.