Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.0 spotted in the wild through A/B testing

ricklamers.io

Post reply on HN
Print view View original

121–130 of 280 posts

Re: Gemini 3.0 spotted in the wild through A/B testing

#121

Post by simonw »

dmd wrote: post #15
jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I find Claude and Gemini to be wildly inferior to ChatGPT when it comes to doing searches to establish grounding. Gemini seems to do a handful of searches and then make shit up, where ChatGPT will do dozens or even hundreds of searches - and do searches based on what it finds in earlier ones.

https://www.google.com/ai is the best version I've seen from Google of LLM-driven search. It feels like ChatGPT GPT-5 Thinking, but a lot faster.

Re: Gemini 3.0 spotted in the wild through A/B testing

#122

Post by dmd »

simonw wrote: post #121
dmd wrote: post #15

Earlier quoted context omitted.

I find Claude and Gemini to be wildly inferior to ChatGPT when it comes to doing searches to establish grounding. Gemini seems to do a handful of searches and then make shit up, where ChatGPT will do dozens or even hundreds of searches - and do searches based on what it finds in earlier ones.

https://www.google.com/ai is the best version I've seen from Google of LLM-driven search. It feels like ChatGPT GPT-5 Thinking, but a lot faster.

Love your blog. What do you think of what was said in the sibling comments about it?

Re: Gemini 3.0 spotted in the wild through A/B testing

#123

Post by delaminator »

gnulinux wrote: post #9
jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I agree with you, I consistently find Gemini 2.5 Pro better than Claude and GPT-5 for the following cases: * Creative writing: Gemini is the unmatched winner here by a huge margin. I would personally go so far as to say Gemini 2.5 Pro is the only borderline kinda-sorta usable model for creative writing if you squint your eyes. I use it to criticize my creative writing (poetry, short stories) and no other model unders…

When I was using Cursor and they got screwed by Anthropic and throttled Sonnet access I used Gemini-2.5-mini and it was a solid coding assistant in the Cursor style - writing functions one at a time, not one-shotting the whole app.

Re: Gemini 3.0 spotted in the wild through A/B testing

#124

Post by irthomasthomas »

whatever1 wrote: post #83
jmkni wrote: post #76

Earlier quoted context omitted.

Different prompts/approaches? I "grew up", as it were, on StackOverflow, when I was in my early dev days and didn't have a clue what I was doing I asked question after question on SO and learned very quickly the difference between asking a good question vs asking a bad one There is a great Jon Skeet blog post from back in the day called "Writing the perfect question" - https://codeblog.jonskeet.uk/2010/08/29/writing-…

Sure but if one is bad at asking questions they would be consistently bad across chatbots

More likely just different tasks. The frontier is jagged.

Re: Gemini 3.0 spotted in the wild through A/B testing

#125

Post by HDThoreaun »

montebicyclelo wrote: post #30
jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

Agreed, and its larger context window is fantastic. My workflow: - Convert the whole codebase into a string - Paste it into Gemini - Ask a question People seem to be very taken with "agentic" approaches were the model selects a few files to look at, but I've found it very effective and convenient just to give the model the whole codebase, and then have a conversation with it, get it to output code, modify a file, etc…

the cli tools really are way faster. You can use them the same way if you want you just dont have to copy paste stuff around all the time

Re: Gemini 3.0 spotted in the wild through A/B testing

#126

Post by markdown »

jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

Why would you use Gemini instead of something purpose-built for you, like Replit?

Re: Gemini 3.0 spotted in the wild through A/B testing

#127

Post by altopex »

brokencode wrote: post #103
dingnuts wrote: post #92

Earlier quoted context omitted.

using an LLM for "creative writing" is like getting on a motorcycle and then claiming you went for a ride on a bicycle no, wait, that analogy isn't even right. it's like going to watch a marathon and then claiming you ran in it.

Just imagine you’re trying to build a custom D&D campaign for your friends. You might have a fun idea don’t have the time or skills to write yourself that you can have an LLM help out with. Or at least make a first draft you can run with. What do your friends care if you wrote it yourself or used an LLM? The quality bar is going to be fairly low either way, and if it provides some variation from the typical story boo…

LLMs have issues with creative tasks that might not be obvious for light users.

Using them for an RPG campaign could work if the bar is low and it's the first couple of times you use it. But after a while, you start to identify repeated patterns and guard rails.

The weights of the models are static. It's always predicting what the best association is between the input prompt and whatever tokens its spitting out with some minor variance due to the probabilistic nature. Humans can reflect on what they've done previously and then deliberately de-emphasize an old concept because its stale, but LLMs aren't able to. The LLM is going to give you a bog standard Gemini/ChatGPT output, which, for a creative task, is a serious defect.

Personally, I've spent a lot of time testing the capabilities of LLMs for RP and storytelling, and have concluded I'd rather have a mediocre human than the best LLMs available today.

Re: Gemini 3.0 spotted in the wild through A/B testing

#128

Post by scrollaway »

OtherShrezzing wrote: post #113
jacquesm wrote: post #106

Earlier quoted context omitted.

That's good? Looks like complete crap to me.

I like the pelican riding a bike test, but my standards for what’s “good” seem higher than generally expected by others. The models can generate hyper realistic renders of pelicans riding bikes in png format. They also have perfect knowledge of the SVG spec, and comprehensive knowledge of most human creative artistic endeavours. They should be able to produce astonishing results for the request. I don’t want to see a…

It’s not true agi until it can recreate the emotional state of Van Gogh when he cut his ear and express the pain through the brush, in svg format.

Re: Gemini 3.0 spotted in the wild through A/B testing

#129

Post by irl_zebra »

jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I use it a lot for ideation on things like strategy and creative tasks. I've found Gemini to be much better than Claude, but I almost want to switch back to Claude because of the "Projects" primitive where I can add specific context to the project and ask questions within that project, and switch around to different projects with different context. Gemini just wants to take all context from everything ever asked and use it in the answers, or I can add the context in the individual prompt, which is tedious.

Re: Gemini 3.0 spotted in the wild through A/B testing

#130

Post by ziml77 »

jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I use the models via Cursor and I prefer the output and speed of Claude Sonnet reasoning mode over Gemini 2.5 Pro. But my work is heavily in ETL/ELT processes and backend business processes. So maybe if I was doing a lot of web stuff it would be different.
Post reply on HN