Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.0 spotted in the wild through A/B testing

ricklamers.io

Post reply on HN
Print view View original

141–150 of 280 posts

Re: Gemini 3.0 spotted in the wild through A/B testing

#141

Post by coffeeaddict1 »

gnulinux wrote: post #9
jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I agree with you, I consistently find Gemini 2.5 Pro better than Claude and GPT-5 for the following cases: * Creative writing: Gemini is the unmatched winner here by a huge margin. I would personally go so far as to say Gemini 2.5 Pro is the only borderline kinda-sorta usable model for creative writing if you squint your eyes. I use it to criticize my creative writing (poetry, short stories) and no other model unders…

My experience with complex reasoning is that Gemini 2.5 Pro hallucinates way too much and it's far below gpt 5 thinking. And for some reason it seems that it's gotten worse over time.

Re: Gemini 3.0 spotted in the wild through A/B testing

#142

Post by cageface »

ofek wrote: post #107

The sentiment in this thread surprises me a great deal. For me, Gemini 2.5 Pro is markedly worse than GPT-5 Thinking along every axis of hallucinations, rigidity in its self-assured correctness and sycophancy. Claude Opus used to be marginally better but now Claude Sonnet 4.5 is far better, although not quite on par with GPT-5 Thinking. I frequently ask the same question side-by-side to all 3 and the only situation i…

For writing code at least this has been exactly my experience. GPT5 is the best but slow. Sonnet 4.5 is a few notches below but significantly faster and good enough for a lot of things. I have yet to get a single useful result from Gemini.

Re: Gemini 3.0 spotted in the wild through A/B testing

#143

Post by coffeeaddict1 »

ofek wrote: post #107

The sentiment in this thread surprises me a great deal. For me, Gemini 2.5 Pro is markedly worse than GPT-5 Thinking along every axis of hallucinations, rigidity in its self-assured correctness and sycophancy. Claude Opus used to be marginally better but now Claude Sonnet 4.5 is far better, although not quite on par with GPT-5 Thinking. I frequently ask the same question side-by-side to all 3 and the only situation i…

Yep, I agree. Gpt 5 thinking is by far the best reasoning model ime. Gemini 2.5 pro is worse in pretty much everything.

Re: Gemini 3.0 spotted in the wild through A/B testing

#144

Post by cindyllm »

altopex wrote: post #127
brokencode wrote: post #103

Earlier quoted context omitted.

Just imagine you’re trying to build a custom D&D campaign for your friends. You might have a fun idea don’t have the time or skills to write yourself that you can have an LLM help out with. Or at least make a first draft you can run with. What do your friends care if you wrote it yourself or used an LLM? The quality bar is going to be fairly low either way, and if it provides some variation from the typical story boo…

LLMs have issues with creative tasks that might not be obvious for light users. Using them for an RPG campaign could work if the bar is low and it's the first couple of times you use it. But after a while, you start to identify repeated patterns and guard rails. The weights of the models are static. It's always predicting what the best association is between the input prompt and whatever tokens its spitting out with…

[dead]

Re: Gemini 3.0 spotted in the wild through A/B testing

#145

Post by jasonjmcghee »

sega_sai wrote: post #10
jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I like Gemini 2.5 as a chatbot, but it has been mostly useless as an agent comparing to Claude Code (at least for my complex tasks)

Exactly my experience.

You have to convince it of basic things it refuses to do - no actually you CAN read files outside of the project- try it.

And it'll frequently write \n instead of actually doing a newline when writing files.

It'll straight up ignore/forget a pattern it was JUST properly doing.

Etc.

Re: Gemini 3.0 spotted in the wild through A/B testing

#146

Post by pdimitar »

kridsdale3 wrote: post #62
dmd wrote: post #15

Earlier quoted context omitted.

I find Claude and Gemini to be wildly inferior to ChatGPT when it comes to doing searches to establish grounding. Gemini seems to do a handful of searches and then make shit up, where ChatGPT will do dozens or even hundreds of searches - and do searches based on what it finds in earlier ones.

Try "AI Mode" on Google.com (Disclaimer, I recently joined the team that makes this product). It isn't Gemini (the product, those are different orgs) though there may (deliberately left ambiguous) be overlap in LLM level bytes. My recommendation for you in this use-case comes from the fact that AI Mode is a product that is built to be a good search engine first, presented to you in the interface of an AI Chatbot. Rat…

Well if you have even a smidgen of decision power, please tell somebody that Google's AI products are all over the place. They are confusing, we are bombarded with information from all sides (I would not use the word "revolution" to describe what's been happening with AI + coding during 2025 but it's IMO not far from that) and everyone screaming for attention by spinning off newer and newer brands and sub-brands of tooling are _not_ helping.

I take no sides; not a fanboy. Only used free Claude and free Gemini Pro 2.5. But some months ago I scoffed at the expression "try it in Google AI Studio" -- that by itself is a branding / marketing failure.

Something like the existing https://ai.google website and with links to the different offerings indeed goes a LONG way. I like that website though it can be done better.

But anyway. Please tell somebody higher up that they are acting like 50 mini companies forced into a single big entity. Google should be better than that.

FWIW, I like Gemini Pro 2.5 best even though I had the free Claude run circles around it sometimes. It one-shot puzzling problems with minimal context multiple times while Gemini was still offering me ideas about how my computer might be malfunctioning if the thing it just hallucinated was not working. Still, most of the time it performs really great.

Re: Gemini 3.0 spotted in the wild through A/B testing

#148

Post by dieortin »

gs17 wrote: post #78
dmd wrote: post #15

Earlier quoted context omitted.

I find Claude and Gemini to be wildly inferior to ChatGPT when it comes to doing searches to establish grounding. Gemini seems to do a handful of searches and then make shit up, where ChatGPT will do dozens or even hundreds of searches - and do searches based on what it finds in earlier ones.

That's my experience as well. Gemini doesn't seem interested in doing searches outside of Deep Research mode, which is kind of funny given it should have the easiest access to a top search engine.

That does not match my experience at all. Basically any Gemini query will run a search.

Re: Gemini 3.0 spotted in the wild through A/B testing

#149

Post by i_have_an_idea »

jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

> I've consistently found Gemini to be better than ChatGPT [ because ] Google has crawled the internet so they have more data to work with.

This commonly expressed non-sequitur needs to die.

First of all, all of the big AI labs have crawled the internet. That's not a special advantage to Google.

Second, that's not even how modern LLMs are trained. That stopped with GPT-4. Now a lot more attention is paid to the quality of the training data. Intuitively, this makes sense. If you train the model on a lot of garbage examples, it will generate output of similar quality.

So, no, Google's crawling prowess has little to do with how good Gemini can be.

Re: Gemini 3.0 spotted in the wild through A/B testing

#150

Post by bl4ckneon »

jjcm wrote: post #89

There are a lot more of these Gemini 3 examples out on twitter right now. After seeing them, I bought Google stock. What shocks me about its output is it actually feels like it's producing net new creative designs, not just regurgitated template output. Its extremely hard to design in code in a way that produces consistent, beautiful output, but it seems to be achieving it. That combined with Google being the only on…

I'm no financial advisor but I can tell you that it's not a financially sound decision to buy stock based off of speculative hype Twitter posts.

But you do you if you have "fun money" to throw around!

Post reply on HN