Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How not to get a $30k bill from Firebase

medium.com

Post reply on HN
Print view View original

61–70 of 114 posts

Re: How not to get a $30k bill from Firebase

#61

Post by donmcronald »

tcgv wrote: post #51
paulgb wrote: post #23

I really wish cloud providers would allow users to set a hard budget that just stops the service if you exceed a threshold. I got a surprise bill from AWS this month (fortunately orders of magnitude less than this, but still ~15x my usual) and am thinking of moving from Lambda to a VPS just so the possibility of this doesn't keep me up at night.

Amazon provides Billing Alarms for monitoring charges, which is very useful for anticipating unplanned cost surges: - https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...

Alarms are like a garage leaving you a voicemail message before doing $25k of repairs on your car. It’s ridiculous that I can’t set a hard cap on my spending.

Re: How not to get a $30k bill from Firebase

#62

Post by EugeneOZ »

stickfigure wrote: post #32
paulgb wrote: post #23

I really wish cloud providers would allow users to set a hard budget that just stops the service if you exceed a threshold. I got a surprise bill from AWS this month (fortunately orders of magnitude less than this, but still ~15x my usual) and am thinking of moving from Lambda to a VPS just so the possibility of this doesn't keep me up at night.

You can? I have daily limits for my Google App Engine apps.

Don't forget they are only warnings, they will not shut down your app and their update rate is not as frequent as you might expect.

Re: How not to get a $30k bill from Firebase

#63

Post by hvidgaard »

kkarakk wrote: post #46
p0llard wrote: post #44

Earlier quoted context omitted.

Why would a junior dev be using _their_ card details? Surely your company has a corporate billing account with Google for this purpose? Perhaps I've misunderstood the situation, but if I were working as a dev at a company and was expected to provide my own card details to pay for APIs, I would walk out the door.

They were just told to make the portal as a learning teambuilding exercise, choice of tools was entirely up to them - i even told the person company would pony up for anything they ended up spending. HR was being a pain about getting card details(it's not easy getting that stuff in india) The dude bounced off google maps as soon as he saw the form to enter payment info and went straight to open street maps.

> The dude bounced off google maps as soon as he saw the form to enter payment info and went straight to open street maps.

As he should. Unless you have in written form that the company will cover the exact expense, then no one should use a personal CC for work use.

Re: How not to get a $30k bill from Firebase

#64

Post by OJFord »

paulgb wrote: post #23

I really wish cloud providers would allow users to set a hard budget that just stops the service if you exceed a threshold. I got a surprise bill from AWS this month (fortunately orders of magnitude less than this, but still ~15x my usual) and am thinking of moving from Lambda to a VPS just so the possibility of this doesn't keep me up at night.

I use VPSs for personal use for exactly this reason - the bill's the same (small amount) every month, and if load spikes I'm happier not being able to handle it than handling it for a surprise Xx bill.

Re: How not to get a $30k bill from Firebase

#65

Post by arthurfm »

comprev wrote: post #7
klaaz0r wrote: post #6

Offtopic I need to register with medium now to actually read the article?

It looks like you get 1 month "free". Open the article in incognito mode and it's available without registration.

Another alternative is the 'Medium.com Unlimited Reading: Original' extension [1][2].

[1] https://chrome.google.com/webstore/detail/mediumcom-unlimite...

[2] https://github.com/Gyvastis/chrome-medium-unlimited-reading

Re: How not to get a $30k bill from Firebase

#66

Post by katzgrau »

intricatedetail wrote: post #50

That's why I don't use cloud offerings. Their pricing models are designed to trap someone with a exorbitant bill and you don't have control over it. For example let's say you put an image on a CDN and someone who doesn't like you runs AB for a couple of days making billions of requests to bankrupt you or someone finds a page that runs costly queries and sets up curl against it. No thanks.

I actually switched from AWS/Cloudfront to Cloudflare because pricing isn't based on bandwidth and I have greater control over security and access (firewall, rules, etc)

I had a string of sketchy accounts using my SaaS service as a file host which ran up a bill of about 1.5k. Obviously I put some safeguards in place (firewall, rules, alerts), but the scenario you're mentioning does happen and is largely mitigated by using a service with a pricing model like Cloudflare's.

Re: How not to get a $30k bill from Firebase

#67

Post by StreamBright »

intricatedetail wrote: post #55
StreamBright wrote: post #52

Earlier quoted context omitted.

Interesting, I saved roughly 2M USD / year for customers moving them to the cloud. Your example is silly, all of the CDN I set up has very strict limit on how much a single IP can use and have multiple alerts if you are passing 100, 200, 500, etc. USD limits. On the top of that if that is not enough you can add more limitations to avoid that exact scenario that a public resource can be abused to cause you financial t…

You see this is too complicated and "If you do it right" imply you can get it wrong and go bankrupt. I prefer to use dedicated server with dedicated bandwidth and Kubernetes on top of that. This makes me sleep comfortably at night. I tried the cloud and this was just too much anxiety for me to handle.

Yes, just like driving, if you doing it wrong you got to the hospital or die, if you doing it the right way you get from A to B. Personal responsibility does not go away because you are using the cloud.

Re: How not to get a $30k bill from Firebase

#68

Post by StreamBright »

BartBoch wrote: post #54
StreamBright wrote: post #52

Earlier quoted context omitted.

Interesting, I saved roughly 2M USD / year for customers moving them to the cloud. Your example is silly, all of the CDN I set up has very strict limit on how much a single IP can use and have multiple alerts if you are passing 100, 200, 500, etc. USD limits. On the top of that if that is not enough you can add more limitations to avoid that exact scenario that a public resource can be abused to cause you financial t…

Not taking sides here, but I think that the cloud is complicated enough (especially for a person that does not specialize in it), to miss one of the edge cases that can lead to the huge bill. In a few minutes, I can set simple PHP script with curl, that will launch 100 requests each second, using a pool of hundreds of thousands of IP's thank to the rotating VPN. This is an edge case of course, but it can happen. I us…

Absolutely, just like with software engineering missing the edge cases can drive you out of business.

https://dougseven.com/2014/04/17/knightmare-a-devops-caution...

I am not sure why people think that cloud is magically will shield them from every bad decision.

Re: How not to get a $30k bill from Firebase

#69

Post by rocqua »

TeMPOraL wrote: post #59
rocqua wrote: post #37

Earlier quoted context omitted.

It seems we have a pretty fucked up cycle here: * Create quality service, prioritizing user experience over money * Grow site to significant user base * Start running out of money, because hosting and running a site does cost money * Try to monetize service * Get out-competed by another service still at step 1 The presence of VC money that's all about capturing market share now, and monetizing later does not help. In…

You're missing a step between "Try to monetize" and "Get outcompeted" - the "Service turns to user-abusing, steaming pile of hot garbage". This cycle does happen repeatedly (image sharing being a prime example), and services that die do so because their monetization strategies destroy almost all the good aspects. I believe it's hard, but not impossible, to refrain from abusive monetization and stay afloat longer.

My point is that there is very little difference between "user abusing" and monetization. This is precisely because of this cycle.

Users expect these services to be free, because they are used to services being sold under-cost. The alternatives for 'platforms' that I know of are:

1) accept 10% of paying customers and 90% leaving, bad plan if you rely on the mass of present users

2) go for a tiered system, where certain features are gated by a pay-wall. This is very tricky, and asking for essential features will see free competitors come up. This can work if your sheer user-base is enough of a moat. (Reddit is still trying to make this one work)

3) sell user-data

3 is definitely seen as 'user-abusing' 2 tends to be as-well and 1 is not an option for any platform.

I can't think of a single platform that started of as 'free' and got profitable without being widely condemned for 'selling out users' except for YouTube (and I guess PornHub). The moat there is massive infrastructure needed for video and a huge, hard-to-move catalogue of content. That is, it is too expensive for some start-up to grow and capture the market.

I'd love to hear of any other success stories. Especially one that could work for a medium / reddit / flicker kind of site.

Post reply on HN