Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google limits Meta's use of its Gemini AI models

cnbc.com

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Google limits Meta's use of its Gemini AI models

#61

Post by vineyardmike »

kouunji wrote: post #26

Google makes claims here about high demand for Gemini - does anyone here have insight into how much of the load on Google is paid use vs the load from putting AI summaries into every web search?

My curiosity is not the free AI summaries (which they can opaquely tune as necessary), but instead the renting of TPUs to Anthropic and OpenAI. Many of these contracts were announced last minute and seemed to involve a very desperate Anthropic. Based on the Anthropic/xAI data center contract, they’re willing to pay crazy markup to get immediate access to compute.

I want to know how impacted Gemini has been by that, because that will reveal a lot about their margins and revenue generating first party demand. Each MSFT earnings report they discuss the balance they’re dealing with between supplying GPUs to Azure customers and first party demand.

My pet theory is that Gemini is “losing” the LLM race because they’re preferentially selling the TPUs to competitors, while keeping just enough for themselves to stay competitive and build their own products.

Re: Google limits Meta's use of its Gemini AI models

#62

Post by tough »

microgpt wrote: post #60
tough wrote: post #53

Earlier quoted context omitted.

I run the CI infra for our company, and our bare metal costs (sans my salary baked in), are one order of magnitude less than if using any other CI saas provider like github or others. Like literally 10x times more expensive to do so, to run CI jobs... I dont want to imagine the margin AWS has like generally, cause it can easily be a 90% too

Right? It's actually crazy how much they don't cost. Are you using it more than 10%? If so, you're saving money. I assume you're using your owned server and not a provider like Hetzner? So you did have a substantial delivery time. Although in my city is a recycled that resells used servers, and I could show up there with a truck and get a server within hours if I'm not too picky. Or use some random desktop or laptop…

No we're on hetzner/ovh boxes, so delivery time really isnt an issue.

Right now the biggest issue is the vibe coded CI program is not really meant to be a distributed multi-node thing yet, so we're on the biggest machines (there's some newer bigger stuff we could migrate too) and the only issue is on peak hours queue can get a bit slow.. but that was also some other bugs etc making not ideal.

Tbh it works pretty well, we just need now to scale it to more than one node etc (which is not to say that is easy, but still, x10 headroom to work with)

Re: Google limits Meta's use of its Gemini AI models

#63

Post by JV00 »

dofm wrote: post #15
HarHarVeryFunny wrote: post #5

It's interesting that Meta is heavily using Google's models (as opposed to Anthropic or OpenAI) given that they are not SOTA for coding. I wonder if this for some strategic/competitive reason, or maybe for cost saving?

I would imagine there are many situations within Meta's applications where relatively small models can do a good job — sentiment analysis, abusive language detection, characterising users based on their posts, summarising a user's complaint so it can be ignored more efficiently, assessing whether ads are likely to be fraudulent so they can be run more often, etc.

But Meta has their own models for anything non-SOTA, why buy from outside?

Re: Google limits Meta's use of its Gemini AI models

#64

Post by HarHarVeryFunny »

dwroberts wrote: post #46
HarHarVeryFunny wrote: post #3

This seems to be a bit of a misleading headline. In the current climate limiting someone's use of AI might be expected to be about restricting access or restricting what someone can do with it, but the story here ostensibly seems to be about capacity constraints, not any limitation on what models or capabilities Google is giving Meta access to.

Given Meta’s current AI situation though, I wouldn’t be surprised if they were trying to do distillation and the capacity story is a cover

You can't actually "distill" reasoning from a model that doesn't expose it's genuine thinking tokens, and none of these do.

When Anthropic accuse Alibaba of distilling their models, you have run that by a reality check of what is actually possible.

1) You can use another model as "LLM as judge" to rate alternative outputs that your own model has generated. Useful data perhaps, but certainly not distillation.

2) If what you are interested in are the reasoning steps (that are hidden from you) that arrived at an answer, not the answer itself, then you can try to train a model to guess what those steps were (this is a published technique). This may be better than nothing, but hardly distillation if it's your model that is suggesting the reasoning!

3) Depending on the model, you may be able prompt engineer it to reveal it's reasoning, not just show a summary, but this should be very obvious. Anthropic cite this as something they have seen. This would be useful data if you can get it (presumably they've now done a better job of preventing it), but at the end of the day all you'd be getting is some training data cheaper than if you'd had to create it by hand.

Re: Google limits Meta's use of its Gemini AI models

#65

Post by khurs »

therealdrag0 wrote: post #56
khurs wrote: post #7

Earlier quoted context omitted.

if you use this as a rough gauge: https://openrouter.ai/models?order=top-weekly Llama Meta 70b is 50th or so down the list of popular models. It has 24.1b tokens used in 7 days vs the top models that have trillions or hundreds of billions of tokens. So practically dead!

Is that biased towards code generation? As opposed to application features using LLMs, which I think is more what we’re talking about.

It's API only, so yes.

Re: Google limits Meta's use of its Gemini AI models

#67

Post by vidarh »

tough wrote: post #53
vidarh wrote: post #45

Earlier quoted context omitted.

You can order bare metal servers delivery time in minutes from any number of hosting providers and the cost difference is so huge you can afford to keep excess capacity and still come out ahead.

I run the CI infra for our company, and our bare metal costs (sans my salary baked in), are one order of magnitude less than if using any other CI saas provider like github or others. Like literally 10x times more expensive to do so, to run CI jobs... I dont want to imagine the margin AWS has like generally, cause it can easily be a 90% too

Their aggregate margin is typically around 30%-40% based on Amazon financials, but that reflects a lot of cost they have as a provider that you as a user wouldn't, e.g. cost of sales and the complexities of managing a multi-tenanted setup, as well as a number of huge customers that clearly are not paying list prices -anyone who is paying six figures or more at AWS and paying the publicly listed prices are overpaying (they can and do negotiate).

The margin on the list prices are clearly far higher.

Re: Google limits Meta's use of its Gemini AI models

#68

Post by sowbug »

HarHarVeryFunny wrote: post #3

This seems to be a bit of a misleading headline. In the current climate limiting someone's use of AI might be expected to be about restricting access or restricting what someone can do with it, but the story here ostensibly seems to be about capacity constraints, not any limitation on what models or capabilities Google is giving Meta access to.

Or even more banal, perhaps Meta wasn't willing to pay a higher price for more capacity.

Re: Google limits Meta's use of its Gemini AI models

#69

Post by dofm »

JV00 wrote: post #63
dofm wrote: post #15

Earlier quoted context omitted.

I would imagine there are many situations within Meta's applications where relatively small models can do a good job — sentiment analysis, abusive language detection, characterising users based on their posts, summarising a user's complaint so it can be ignored more efficiently, assessing whether ads are likely to be fraudulent so they can be run more often, etc.

But Meta has their own models for anything non-SOTA, why buy from outside?

They would still have to find the resources to host it. Perhaps Gemini works out cheaper.

Re: Google limits Meta's use of its Gemini AI models

#70

Post by microgpt »

vidarh wrote: post #67
tough wrote: post #53

Earlier quoted context omitted.

I run the CI infra for our company, and our bare metal costs (sans my salary baked in), are one order of magnitude less than if using any other CI saas provider like github or others. Like literally 10x times more expensive to do so, to run CI jobs... I dont want to imagine the margin AWS has like generally, cause it can easily be a 90% too

Their aggregate margin is typically around 30%-40% based on Amazon financials, but that reflects a lot of cost they have as a provider that you as a user wouldn't, e.g. cost of sales and the complexities of managing a multi-tenanted setup, as well as a number of huge customers that clearly are not paying list prices -anyone who is paying six figures or more at AWS and paying the publicly listed prices are overpaying…

Also all the stuff they have to do that you otherwise don't. Like carrying all your disk traffic between EC2 and EBS instead of having the disk inside the server.
Post reply on HN