Demand for tokens raises exponentially, we are in the middle of a compute crisis, and people still think AI is a bubble...
Google limits Meta's use of its Gemini AI models
41–50 of 78 posts
Re: Google limits Meta's use of its Gemini AI models
#42Google makes claims here about high demand for Gemini - does anyone here have insight into how much of the load on Google is paid use vs the load from putting AI summaries into every web search?
I don't know numbers, but their APIs have a bad uptime in my experience for some models. Too often failure because of "traffic too high".
Re: Google limits Meta's use of its Gemini AI models
#43Google makes claims here about high demand for Gemini - does anyone here have insight into how much of the load on Google is paid use vs the load from putting AI summaries into every web search?
I don't know numbers, but their APIs have a bad uptime in my experience for some models. Too often failure because of "traffic too high".
Yeah I had a trial for AI Pro or whatever it's called and could never use Gemini CLI (when it still existed) because it was constantly "overloaded". Using the API directly (wihtout a subscription) sometimes works but the models are so buggy and the endpoints constantly spew errors that it's not usable. See this forum thread for example: https://discuss.ai.google.dev/t/frequent-503-errors-service-... it started with 503 errors since JANUARY and it's still not fixed. These are "stable" GA models!
I HIGHLY doubt that Gemini is overloaded, Google has been bullshitting with their crap models since release. Waste of everyone's time.
Re: Google limits Meta's use of its Gemini AI models
#44Earlier quoted context omitted.
These kind of limits happen all the time for big clients. Cloud services like to present the illusion of an infinite amount of compute available at a fixed price per unit, but the reality is if you try to use too much of any service you'll find you have a quota and requests to increase it will fall on deaf ears if the provider doesn't have more of that resource. Too much of my working life has been spent shoehorning…
If you allow me a bit of pedantry, it's infinite "for all intents and purposes". It doesn't mean you can request civilizational levels of compute, but for a blog, a crud, an ETL and such, that is regular use cases with sensible scale you can absorb any elastic demand. Having said that, I agree with you. You have to request limit increases often and can't scale even in those instances if you don't plan ahead.
Even as a small customer it's easy to hit quotas or hit availablity constraints of more unusual instance types.
Re: Google limits Meta's use of its Gemini AI models
#45Earlier quoted context omitted.
if im going to have to ask for capacity, why dont I just get my own bare metal servers then?
Because you don't have to wait for weeks just for delivery. And you pay for elastic usage.
You can order bare metal servers delivery time in minutes from any number of hosting providers and the cost difference is so huge you can afford to keep excess capacity and still come out ahead.
Re: Google limits Meta's use of its Gemini AI models
#46This seems to be a bit of a misleading headline. In the current climate limiting someone's use of AI might be expected to be about restricting access or restricting what someone can do with it, but the story here ostensibly seems to be about capacity constraints, not any limitation on what models or capabilities Google is giving Meta access to.
Given Meta’s current AI situation though, I wouldn’t be surprised if they were trying to do distillation and the capacity story is a cover
Re: Google limits Meta's use of its Gemini AI models
#47Google makes claims here about high demand for Gemini - does anyone here have insight into how much of the load on Google is paid use vs the load from putting AI summaries into every web search?
We use Gemini for some specific tasks. It is often unavailable due to capacity limits or other downtime.
It's probably the best multimodal model I've worked with (if somebody knows a better one for audio analysis, please let me know!)
Re: Google limits Meta's use of its Gemini AI models
#48Earlier quoted context omitted.
If you allow me a bit of pedantry, it's infinite "for all intents and purposes". It doesn't mean you can request civilizational levels of compute, but for a blog, a crud, an ETL and such, that is regular use cases with sensible scale you can absorb any elastic demand. Having said that, I agree with you. You have to request limit increases often and can't scale even in those instances if you don't plan ahead.
definitionally that's "for some intents and purposes" my man
For all intents and purposes is a figure of speech, meaning in every practical sense.
Re: Google limits Meta's use of its Gemini AI models
#49Facebook does seem to be falling behind. Does anyone here use Llama over more recent options for any technical reasons?
Meta's latest model is Spark Muse and not available outside of its products. https://ai.meta.com/blog/introducing-muse-spark-msl/
still waiting on that API launch which was supposed to happen very very soon