311–320 of 325 posts
Earlier quoted context omitted.
We have spend caps at the billing account level and the project level (developer set) in the Gemini API now. There is up to a 10 minute delay in processing everything but this should significantly mitigate the risk here: https://ai.google.dev/gemini-api/docs/billing#tier-spend-cap... By default, new Tier 1 paid accounts can only spend $250 in a given month.
How much of a bill can you run up in 10 minutes?
> We had a budget alert (€80) and a cost anomaly alert, both of which triggered with a delay of a few hours > By the time we reacted, costs were already around €28,000 > The final amount settled at €54,000+ due to delayed cost reporting So much for the folks defending these three companies that refused to provide hard spending cap ("but you can set the budget", "you are doing it wrong if you worry about billing", "ha…
Earlier quoted context omitted.
My favorite Google LLM benchmark is asking Gemini models to create a script that fetches API usage (just request counts) for a project from GCP. 100% failure rate.
I've yet to receive an accurate response from Gemini about GCP services, beyond completely trivial topics. The most recent, I think, was Gemini advising me that I could attach an existing pd SSD PVC to a n4 or c4 VM. For whatever unknowable reason, Google doesn't allow this and doesn't offer a migration path, and Gemini doesn't "know" anything about it either. It's wild.
What bothers me is even having it do extensive research in documentation, it still can't figure it out.
GCP must simply be so unintuitive that the LLM mind cannot comprehend it.
These are all poorly designed systems from a CX perspective (the billing systems). Billing is usually event driven. Each spending instance (e.g. API call) generates an event. Events go to queues/logs, aggregation is delayed. You get alerts when aggregation happens, which if the aggregation service has a hiccup, can be many hours later (the service SLA and the billing aggregator SLA are different). Even if you have ha…
It's not typically a problem that usage is event driven. At least not for prepaid phone plans. Or debit cards. Or mailboxes. Or any myriad of prepaid or quota'd services. It's not rocket science, just a bad business practice on the part of Google.
Forgive my ignorance - but what's the payoff for fraudsters in getting access to a generative AI service for a short-ish period of time, before they get cut off? With EC2 / GCC credentials, I could understand going all out on bitcoin mining - but what are they asking the AI to do here that's worth setting up some kind of botnet or automation to sift the internet for compromised keys?
Earlier quoted context omitted.
It's not typically a problem that usage is event driven. At least not for prepaid phone plans. Or debit cards. Or mailboxes. Or any myriad of prepaid or quota'd services. It's not rocket science, just a bad business practice on the part of Google.
Can you not see any relevant difference between an entire cloud platforms worth of APIs and… a prepaid phone plan?
Earlier quoted context omitted.
I just find it extraordinary that the biggest tech company in the world can do cutting edge real time AI for millions of people, run Youtube and of course all the other google services with having literally the smartest people in the world and unlimited resources on board, but still can't keep real time track of the user's current billing and their spending limits, it's all best effort still. Somehow it doesn't add u…
Not just Google, also Microsoft and Amazon. Real-time cost tracking is technically impossible to solve according to the major cloud providers. I have huge respect for those sales & finops engineers.
Poooooor AT&T, goodness it's hard to know how much data they just sent to us. Hard twenty years ago, just as hard today.
Two things that should be default on any GCP project touching generative-AI APIs: 1 API-key restrictions by HTTP referrer AND by API (`generativelanguage.googleapis.com` only), 2 a billing budget with a Pub/Sub "cap" action, not just an email alert. Neither is on by default, and almost nobody sets them before shipping. 13 hours is actually fast for detection. most teams find out at end-of-month reconciliation.
I want API keys with monthly and hourly quotas and RATE LIMITING. like 50k requests per hour, above that 1/s/client up to 20 req/sec. I don't want to shotgun my service for every user if one user is misbehaving. I want to set rate of bleeding
Earlier quoted context omitted.
Google API keys have been used for ages on the frontend. For example on Google Maps embeds. Those are not possible without exposing a key to the frontend. They weren't secret, until Gemini arrived. https://trufflesecurity.com/blog/google-api-keys-werent-secr... https://medium.com/@ahhyesic/your-google-maps-api-key-now-ha... https://www.malwarebytes.com/blog/news/2026/02/public-google...
If one ignores 70% of the documentation, it makes for a demonizing blog post about it, sure. " API keys for Firebase services are not secret API keys for Firebase services only identify your Firebase project and app to those services. Authorization is handled through Google Cloud IAM permissions, Firebase Security Rules, and Firebase App Check. All Firebase-provisioned API keys are automatically restricted to Firebas…
Back then, they did not automatically restrict those keys to only Firebase-related APIs.
So yes, if you read the documentation as it exists today it's much more clear what they're trying to prevent, but this is only after this issue has become more apparent.
Earlier quoted context omitted.
If one ignores 70% of the documentation, it makes for a demonizing blog post about it, sure. " API keys for Firebase services are not secret API keys for Firebase services only identify your Firebase project and app to those services. Authorization is handled through Google Cloud IAM permissions, Firebase Security Rules, and Firebase App Check. All Firebase-provisioned API keys are automatically restricted to Firebas…
They've updated that documentation significantly since thousands of projects were issued automatically generated Firebase API keys, pre-Gemini. See this version from Oct 2022: https://web.archive.org/web/20221001052713/https://firebase.... Back then, they did not automatically restrict those keys to only Firebase-related APIs. So yes, if you read the documentation as it exists today it's much more clear what they're…
If that is the case, why is this also in the 2022 link?
The part about scoping links to the restrictions documentation.
"Understand API keys
API keys for Firebase services are not secret
Firebase uses API keys only to identify your app's Firebase project to Firebase services, and not to control access to database or Cloud Storage data, which is done using Firebase Security Rules. For this reason, you do not need to treat API keys for Firebase services as secrets, and you can safely embed them in client code. Learn more about API keys for Firebase.
Set up API key scoping
As an additional deterrent against an attacker attempting to use your API key to spoof requests, you can create API keys scoped to your app clients.
Keep FCM server keys secret
Unlike API keys for Firebase services, FCM server keys (used by the legacy FCM HTTP API) are sensitive and must be kept secret.
Keep service account keys secret
Also unlike API keys for Firebase services, service account private keys (used by the Admin SDK) are sensitive and must be kept secret. "