Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How not to get a $30k bill from Firebase

medium.com

Post reply on HN
Print view View original

101–110 of 114 posts

Re: How not to get a $30k bill from Firebase

#101

Post by TeMPOraL »

rocqua wrote: post #69
TeMPOraL wrote: post #59

Earlier quoted context omitted.

You're missing a step between "Try to monetize" and "Get outcompeted" - the "Service turns to user-abusing, steaming pile of hot garbage". This cycle does happen repeatedly (image sharing being a prime example), and services that die do so because their monetization strategies destroy almost all the good aspects. I believe it's hard, but not impossible, to refrain from abusive monetization and stay afloat longer.

My point is that there is very little difference between "user abusing" and monetization. This is precisely because of this cycle. Users expect these services to be free, because they are used to services being sold under-cost. The alternatives for 'platforms' that I know of are: 1) accept 10% of paying customers and 90% leaving, bad plan if you rely on the mass of present users 2) go for a tiered system, where certa…

Thanks for elaborating. I agree, but I wanted to see this point mentioned explicitly. It's not just the free competitors that are causing established business' downfall; the business itself usually engages in plenty of self-destructive action in a desperate attempt to monetize users who are used to free service.

Re: How not to get a $30k bill from Firebase

#102

Post by Data_Junkie »

rocqua wrote: post #37
klaaz0r wrote: post #12

Earlier quoted context omitted.

Yeah I can get passed it but damn it's become a mess, I understand they have to make money.

It seems we have a pretty fucked up cycle here: * Create quality service, prioritizing user experience over money * Grow site to significant user base * Start running out of money, because hosting and running a site does cost money * Try to monetize service * Get out-competed by another service still at step 1 The presence of VC money that's all about capturing market share now, and monetizing later does not help. In…

Well the solution is find a market gap, not give people things they don't really need and pull a bait and switch. You can find users of a lot of things if you give them away, don't be surprised if they don't now want to pay for your thing they only used because it was free.

Re: How not to get a $30k bill from Firebase

#103

Post by donmcronald »

influx wrote: post #92
donmcronald wrote: post #61

Earlier quoted context omitted.

Alarms are like a garage leaving you a voicemail message before doing $25k of repairs on your car. It’s ridiculous that I can’t set a hard cap on my spending.

Sounds like the oomkiller. Sounds good in theory until it starts shooting processes in the head that you didn’t intend. What would you like Amazon to do for stateful services? Should they stop and delete EBS volumes? What about databases? Simply shut them down? What happens when you lose data or it doesn’t come back up?

EBS volumes have a size, so there's an upper bound to the cost. Most of the storage is predictable, so if I try to allocate storage where the monthly cost of the raw storage is (ex) 10x my budget, I wouldn't have a problem with the request being denied.

For non-storage resources like EC2, network bandwidth, etc. I'd be fine with having a hard limit where everything just breaks, especially for stuff that's not production.

There could also be better, self managed quotas on resources. SES is a good example. AFAIK the quota is all or none across the entire account. IMO, it's not a good idea to give a user that needs to send (made up numbers) 1k emails per day credentials that can send 250k emails a day.

I have 3 AWS accounts. I don't keep anything in my main account. It's for billing only. I have a sub account for production that I try to keep pristine. I have a sub account for development and testing. It's the development account that scares me. I spend less than $50 per month. I'd rather have my whole development account de-allocated than get a bill for $1k.

Re: How not to get a $30k bill from Firebase

#104

Post by donmcronald »

brianwawok wrote: post #77
donmcronald wrote: post #61

Earlier quoted context omitted.

Alarms are like a garage leaving you a voicemail message before doing $25k of repairs on your car. It’s ridiculous that I can’t set a hard cap on my spending.

On the flip side I would never want my real business to shut off. Hobby project yes! Real business no,

AFAIK it's already considered a best practice to set up separate accounts for development and production (https://blog.codeship.com/separate-aws-production-and-develo...). Having a hard cap on an account dedicated to development would help a lot IMO.

Re: How not to get a $30k bill from Firebase

#105

Post by paulgb »

7ewis wrote: post #76
paulgb wrote: post #23

I really wish cloud providers would allow users to set a hard budget that just stops the service if you exceed a threshold. I got a surprise bill from AWS this month (fortunately orders of magnitude less than this, but still ~15x my usual) and am thinking of moving from Lambda to a VPS just so the possibility of this doesn't keep me up at night.

How did you manage to increase your bill 15x in Lambda?

It seems to have been a combination of a traffic surge and a third-party API being temporarily slow. The slow API meant that I ended up paying for more idle CPU time during the request round-trip.

Re: How not to get a $30k bill from Firebase

#106

Post by cavisne »

paulgb wrote: post #23

I really wish cloud providers would allow users to set a hard budget that just stops the service if you exceed a threshold. I got a surprise bill from AWS this month (fortunately orders of magnitude less than this, but still ~15x my usual) and am thinking of moving from Lambda to a VPS just so the possibility of this doesn't keep me up at night.

You can limit the max concurrency of lambda functions which can help a bit here.

Imo the reason is this is for business/enterprise use cases configuring this wrong could be catastrophic, for side project/personal use cases its pretty well known you get 1 chance for a refund from the public clouds, after some manual checking to make sure you aren't trying to abuse the service.

Re: How not to get a $30k bill from Firebase

#107

Post by rkangel »

klaaz0r wrote: post #6

Offtopic I need to register with medium now to actually read the article?

The chrome extension 'Make medium readable again' solves this for me:

https://chrome.google.com/webstore/detail/make-medium-readab...

It is ridiculous that I have to install an extension to make a website usable though. I'm genuinely curious why people publish on Medium.

Re: How not to get a $30k bill from Firebase

#108

Post by brianwawok »

faster wrote: post #99
brianwawok wrote: post #77

Earlier quoted context omitted.

On the flip side I would never want my real business to shut off. Hobby project yes! Real business no,

Assuming your revenue and your AWS bill are strongly correlated, I agree. However, when a free service or a bug can increase costs by orders of magnitude without a corresponding increase in revenue your choices are to eat it (not sustainable, for most of us), throttle, or stop. Throttling is hard, so stopping is often the best choice.

Disagree there, for a paid business.

If my business had 100k in revenue per month. Normal cloud spend is 2k per month.. some glitch made spend go to 2k a day. That sucks, and I hope I have good alerts to catch it within a day or two.. but telling my customers "screw you, bill messed up" for however long it took me to sort out the bug? I would not do that.

Re: How not to get a $30k bill from Firebase

#109

Post by hoaxcracker »

vertis wrote: post #78
paulgb wrote: post #23

I really wish cloud providers would allow users to set a hard budget that just stops the service if you exceed a threshold. I got a surprise bill from AWS this month (fortunately orders of magnitude less than this, but still ~15x my usual) and am thinking of moving from Lambda to a VPS just so the possibility of this doesn't keep me up at night.

A few years ago I got a ~$3500 Amazon bill when someone broke into my account (through keys I'd inadvertently added to GitHub)[1]. I had no expectation of that money being waived, but Amazon did anyway. Since then I've had billing alerts turned on at about the $150 mark (which is about double what my personal monthly Amazon bill is). [1]: https://news.ycombinator.com/item?id=6911908

Shameless plug: I know this plug should not be here but I think I can help some people with cost problem via (the easiest automation platform for AWS)[1].

I have a startup plan where you can run 60 executions in a month free of cost. Just reach out to me via https://totalcloud.io/solutions.html and refer YComb.

[1]: https://totalcloud.io

Re: How not to get a $30k bill from Firebase

#110

Post by kkarakk »

dvfjsdhgfv wrote: post #49
kkarakk wrote: post #46

Earlier quoted context omitted.

They were just told to make the portal as a learning teambuilding exercise, choice of tools was entirely up to them - i even told the person company would pony up for anything they ended up spending. HR was being a pain about getting card details(it's not easy getting that stuff in india) The dude bounced off google maps as soon as he saw the form to enter payment info and went straight to open street maps.

Was the choice of OSM a bad one?

Seeing as how i had to learn all the GIS jargon OSM uses by default, it was bad for me. It's not a bad choice but since we as a company can afford the easy of Google maps it's a waste of time
Post reply on HN