Is there a good benchmark comparing multilingual and/or translation abilities of most recent LLMs? GPT-4o struggles for some tasks in my language learning app.
Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
71–80 of 151 posts
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#72Its just not as smart as ChatGPT or LLAMA, its mind boggling Google fell so far behind.
Has it changed much since March when this was written? Gemini won, 6-4. And it matches my experiences of just using Gemini instead of ChatGPT because it gives me more useful responses, when I feel like I want an AI answer.
https://www.tomsguide.com/ai/google-gemini-vs-openai-chatgpt
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#73This sounds interesting: "We will continue to offer a suite of safety filters that developers may apply to Google’s models. For the models released today, the filters will not be applied by default so that developers can determine the configuration best suited for their use case."
This is the most important update.
Pricing and speed doesn't matter when your call fails because of "safety".
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#74I only use regular Gemini and the main feature I care about is absolutely terrible: summarizing YouTube videos. I'll ask for a breakdown or analysis of the video, and it'll give me a very high level overview. If I ask for timestamps or key points, it begins to hallucinate and make stuff up. It's incredibly disappointing that such a huge company with effectively unlimited access to both money and intellectual resource…
I imagine you'd be paying more in ML costs than YouTube makes off your views.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#75I only use regular Gemini and the main feature I care about is absolutely terrible: summarizing YouTube videos. I'll ask for a breakdown or analysis of the video, and it'll give me a very high level overview. If I ask for timestamps or key points, it begins to hallucinate and make stuff up. It's incredibly disappointing that such a huge company with effectively unlimited access to both money and intellectual resource…
I've had it fail when a video does not have subtitles - I'm guessing that's what it uses. I have had good success having it answer the clickbait video titles like "is this the new best thing?"
It's not watching the video as far as I can tell.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#76This price drop is significant. For For comparison, GPT-4o is currently $5/million input and $15/million output and Claude 3.5 Sonnet is $3/million input and $15/million output. Gemini 1.5 Pro was already the cheapest of the frontier models and now it's even cheaper.
I wonder if they're pulling the wall-mart model. Ruthlessly cut costs and sell at-or-below costs until your competitors go out of business, then ratchet up the prices once you have market dominance.
Probably not. Do they really believe they are going to knock OpenAI out of business, when the OpenAI models are better?
Instead I think they are going after the "Android model". Recognize they might not be able to dethrone the leader who invented the space. Define yourself in the marketplace as the cheaper alternative. "Less good but almost as good." In the end, they hope to be one of a small number of surviving members of an valuable oligopoly.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#77This price drop is significant. For For comparison, GPT-4o is currently $5/million input and $15/million output and Claude 3.5 Sonnet is $3/million input and $15/million output. Gemini 1.5 Pro was already the cheapest of the frontier models and now it's even cheaper.
I wonder if they're pulling the wall-mart model. Ruthlessly cut costs and sell at-or-below costs until your competitors go out of business, then ratchet up the prices once you have market dominance.
Yes, and it's the exact same thing OpenAI/Microsoft and Facebook are doing. In Facebook's case, they are giving it away for free.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#78Anyone want to take bets on how long it takes for this to hit the Google Graveyard?
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#79Earlier quoted context omitted.
I wonder if they're pulling the wall-mart model. Ruthlessly cut costs and sell at-or-below costs until your competitors go out of business, then ratchet up the prices once you have market dominance.
You think Google would engage in monopolistic practices like that? Because I do
I have no idea if this is dumping or not. At Microsoft/Google scale, what does it cost to serve a million LLM tokens?
Tough to disentangle the capex vs opex costs for them. If they did not have so many other revenue streams, potentially dicey as there are probably still many untapped performance optimizations.
Re: Two new Gemini models, reduced 1.5 Pro pricing, increased rate limits, and more
#80Earlier quoted context omitted.
They are going for large corporate customers. They are a brand name with deep pockets and a pretty risk-adverse model. So even if Gemini sucks, they'll still win over execs being pushed to make a decision.
That doesn't seem like much of a plan given their trailing position in the cloud space and the fact that Microsoft and AWS both have their own offerings.
Maybe Google is holding back it's far superior, truly sentient AI until other companies have broken the ice. Not long ago there was a Google AI engineer who rage quit over Google's treatment of sentienet AI.