We had this exact same problem (the key initially wasn’t a secret but became a secret once we enabled Gemini API with no warnings). We managed to catch it somewhat early through alerting, so the damage was only $26k. We asked our Google cloud support rep for a refund - they initially came back with a no but now the case is under further consideration. I’d escalate this up the chain as much as possible.
€54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
171–180 of 325 posts
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#172Earlier quoted context omitted.
Not if your contractor had you first sign a 15 page contract that commits you to whatever costs they dream up and requires forced arbitration by a corporate friendly firm when any dispute arises. Because that's somehow normal in today's tech world.
So if their TOS say they can also rape my cat, then I cannot do anything about it, right? Ridiculous
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#173> We had a budget alert (€80) and a cost anomaly alert, both of which triggered with a delay of a few hours > By the time we reacted, costs were already around €28,000 > The final amount settled at €54,000+ due to delayed cost reporting So much for the folks defending these three companies that refused to provide hard spending cap ("but you can set the budget", "you are doing it wrong if you worry about billing", "ha…
I'd buy the technically impossible angle. Even if you manage to get your microservices to synch every penny spent to your payment account at realtime (impossible) you still have to waiver the excess, losing some money every time someone goes past their quota.
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#174> We had a budget alert (€80) and a cost anomaly alert, both of which triggered with a delay of a few hours > By the time we reacted, costs were already around €28,000 > The final amount settled at €54,000+ due to delayed cost reporting So much for the folks defending these three companies that refused to provide hard spending cap ("but you can set the budget", "you are doing it wrong if you worry about billing", "ha…
That's actually crazy. So I can build a project I love, that does good, but somehow get in a situation where I'm accidentally paying 30.000€ (or 50.000€) to a big tech company? How is that fair? I mean yes, as a software engineer, you ought to reflect on all possible weaknesses, but there was a time when overlooking something meant something completely different than being down 30/50k. That is actually life-altering.
When the FTC went investigating a decade-ish ago they found Facebook saying the quiet parts out loud: it was all extremely deliberate.
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#175Earlier quoted context omitted.
Not if your contractor had you first sign a 15 page contract that commits you to whatever costs they dream up and requires forced arbitration by a corporate friendly firm when any dispute arises. Because that's somehow normal in today's tech world.
So if their TOS say they can also rape my cat, then I cannot do anything about it, right? Ridiculous
We're pretty far from the Lochner era in the US, where even minimum wage laws were held to be unconstitutional violations of a very broad view of freedom to contract. But it is still a principle in most legal system.
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#176Earlier quoted context omitted.
I'd buy the technically impossible angle. Even if you manage to get your microservices to synch every penny spent to your payment account at realtime (impossible) you still have to waiver the excess, losing some money every time someone goes past their quota.
Sure, but 80 -> 28,000 -> 54,000 is a hell of a lot of slippage. Trading platforms can guarantee a maximum slippage on stops, and often even offer guaranteed stops (with an attached premium), so I don’t see why Google and Firebase can’t do similar. The way it works at present is ridiculous.
Yeah no, physically impossible. If nobody is selling at that price, there is no guarantee your sell stop will execute near that price. They can sweep the market, find the best seller price and execute.
There might be a costly way to do it with microservices as I indicated, but your example easily falls apart.
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#177> We had a budget alert (€80) and a cost anomaly alert, both of which triggered with a delay of a few hours > By the time we reacted, costs were already around €28,000 > The final amount settled at €54,000+ due to delayed cost reporting So much for the folks defending these three companies that refused to provide hard spending cap ("but you can set the budget", "you are doing it wrong if you worry about billing", "ha…
This should be illegal. If a contractor your hired to swap out a tile on your bathroom floor billed you for remodelling your back garden, you would obviously have the legal right to refuse that.
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#178Earlier quoted context omitted.
That's actually crazy. So I can build a project I love, that does good, but somehow get in a situation where I'm accidentally paying 30.000€ (or 50.000€) to a big tech company? How is that fair? I mean yes, as a software engineer, you ought to reflect on all possible weaknesses, but there was a time when overlooking something meant something completely different than being down 30/50k. That is actually life-altering.
Your kid can do this in a smartphone game designated suitable for children, heavily optimized to exacerbate the possibility, and depending on where you live they can just choose not to refund you. When the FTC went investigating a decade-ish ago they found Facebook saying the quiet parts out loud: it was all extremely deliberate.
Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#179Re: €54k spike in 13h from unrestricted Firebase browser key accessing Gemini APIs
#180I think the logistics of calculating cost in real time is something that is extremely hard. I don't think there is one big cloud service provider that has hard limits instead of alerts. As long as they revert the charge when notified of scenarios like this , and they have historically done so for many cases, it's fine. It's an acceptable workaround for a hard problem and the cost of doing business ( just like Credit…
Overcharge protection doesn't have to be free. It could be +5% on prices or a fee of 25% when you reach the threshold.
They would have financial interest in calculating cost in real time and it'd magically become more and more precise over releases.