Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we spent $30k in Firebase in less than 72 hours

hackernoon.com

Post reply on HN
Print view View original

11–20 of 249 posts

Re: How we spent $30k in Firebase in less than 72 hours

#11

Post by itcmcgrath »

t0mbstone wrote: post #3

This is why infinitely scaling pay-as-you-go cloud services terrify me. I refuse to use a service like this unless it gives me the ability to automatically cap costs and alert me when thresholds are met. All it takes is a rogue line of code in an endless loop or something, and you are bankrupt. Their site seems pretty basic. I'm struggling to understand why they couldn't just run it with something like Postgres for l…

Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it.

Re: How we spent $30k in Firebase in less than 72 hours

#12

Post by itcmcgrath »

ghaff wrote: post #7
t0mbstone wrote: post #3

This is why infinitely scaling pay-as-you-go cloud services terrify me. I refuse to use a service like this unless it gives me the ability to automatically cap costs and alert me when thresholds are met. All it takes is a rogue line of code in an endless loop or something, and you are bankrupt. Their site seems pretty basic. I'm struggling to understand why they couldn't just run it with something like Postgres for l…

From what I understand, AWS at least is pretty good about giving you at least one "get out of jail free" card if things go awry. Caps are tough though. I can certainly understand a use case that would want a hard circuit breaker that just kills everything it can once it hits a certain threshold. Sort of; you presumably don't want everything on S3, for example, to be deleted. On the other hand, moving up the scale of…

Caps are definitely hard, getting them right as both a feature as well as setting them as a customer is still an art. This holds true for 'flat rate' too, where your trading correct sizing vs downtime when either get code wrong or get unexcepted success. I've been involved in all these situations so I definitely emphasize.

We had daily hard caps and budgets alerts, but it's still an area we can do better.

(Disclaimer: Product Manager for Cloud Firestore)

Re: How we spent $30k in Firebase in less than 72 hours

#14

Post by alasdair_ »

itcmcgrath wrote: post #11
t0mbstone wrote: post #3

This is why infinitely scaling pay-as-you-go cloud services terrify me. I refuse to use a service like this unless it gives me the ability to automatically cap costs and alert me when thresholds are met. All it takes is a rogue line of code in an endless loop or something, and you are bankrupt. Their site seems pretty basic. I'm struggling to understand why they couldn't just run it with something like Postgres for l…

Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it.

>Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it.

Google Cloud user here. A warning: If you ever happen to get, say, frontpage on reddit or techcrunch or other big boost to publicity, your site could be down until the next billing cycle (i.e. 24 hours) and you will have no way to fix it.

This bit me hard one day with appengine and lost us a ton of converting traffic, even though we tried to get the limit increased within ten minutes of the spike (and well before our limit was hit).

Re: How we spent $30k in Firebase in less than 72 hours

#15

Post by vuln »

t0mbstone wrote: post #3

This is why infinitely scaling pay-as-you-go cloud services terrify me. I refuse to use a service like this unless it gives me the ability to automatically cap costs and alert me when thresholds are met. All it takes is a rogue line of code in an endless loop or something, and you are bankrupt. Their site seems pretty basic. I'm struggling to understand why they couldn't just run it with something like Postgres for l…

You only have to be 'terrified' if you don't write good code. Maybe people should actually test their code before pushing it to production?

It's not firebase's fault you ran bad code and have a huge bill. They have bills to pay also. Think they can tell their vendors 'sorry we can pay you this week. A customer ran up a 30k bill and they can't pay it. So we can't pay you right now. but lol bad code right? '

Re: How we spent $30k in Firebase in less than 72 hours

#16

Post by iamgopal »

alasdair_ wrote: post #14
itcmcgrath wrote: post #11

Earlier quoted context omitted.

Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it.

>Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it. Google Cloud user here. A warning: If you ever happen to get, say, frontpage on reddit or techcrunch or other big boost to publicity, your site could be down until the next billing cycle (i.e. 24 hours) a…

So basically we want to have cake and eat it too.

Re: How we spent $30k in Firebase in less than 72 hours

#17

Post by marktangotango »

everdev wrote: post #4
t0mbstone wrote: post #3

This is why infinitely scaling pay-as-you-go cloud services terrify me. I refuse to use a service like this unless it gives me the ability to automatically cap costs and alert me when thresholds are met. All it takes is a rogue line of code in an endless loop or something, and you are bankrupt. Their site seems pretty basic. I'm struggling to understand why they couldn't just run it with something like Postgres for l…

Even with a cap, a rogue line or a legitimate surge in traffic could shut down your app. It's endless bill monitoring and budget approval. I'll stick to a flat rate DO droplet.

What’s the difference between a cap limiting your traffic and server cpu maxed out, or db connection pool or ... limiting your traffic?

Re: How we spent $30k in Firebase in less than 72 hours

#18

Post by ajmurmann »

iamgopal wrote: post #16
alasdair_ wrote: post #14

Earlier quoted context omitted.

>Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it. Google Cloud user here. A warning: If you ever happen to get, say, frontpage on reddit or techcrunch or other big boost to publicity, your site could be down until the next billing cycle (i.e. 24 hours) a…

So basically we want to have cake and eat it too.

Or have it be adjustable within a reasonable time frame?

Re: How we spent $30k in Firebase in less than 72 hours

#20

Post by itcmcgrath »

alasdair_ wrote: post #14
itcmcgrath wrote: post #11

Earlier quoted context omitted.

Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it.

>Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it. Google Cloud user here. A warning: If you ever happen to get, say, frontpage on reddit or techcrunch or other big boost to publicity, your site could be down until the next billing cycle (i.e. 24 hours) a…

Urgh, that's terrible! That definitely shouldn't be the case. I'd love for you to send me any support case you had so I can review.

Even if the front door of the system didn't help you, we definitely shouldn't have been able to get you in a good state much quicker. My profile has Twitter and my DMs are open (I can give you my email there too).

Doing my kids dinner, so responses might be slightly delayed.

Post reply on HN