Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.0 spotted in the wild through A/B testing

ricklamers.io

Post reply on HN
Print view View original

101–110 of 280 posts

Re: Gemini 3.0 spotted in the wild through A/B testing

#101

Post by faebi »

jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I do feel like LLM's start to match certain personalities and characteristics of users which makes them unattractive to others. I assume we will need a better kind of personalization layer in the future or the ecosystems will start to drift. For example I very much feel like grok fits my thought patters by far the best.

Re: Gemini 3.0 spotted in the wild through A/B testing

#102

Post by moffkalast »

jmkni wrote: post #6
OsrsNeedsf2P wrote: post #4

Earlier quoted context omitted.

What's your use case? We've found Gemini to work well with large context windows, but it sucks at calling MCPs and is worse at writing code

Building out user interfaces in html and scss (mainly in Angular) You need to give it detailed instructions and be willing to do the plumbing yourself, but we've found it to be very good at it

Angular is probably what sets your use case apart. It has a very rigidly defined style which Gemini can't break, so you avoid the main downside of it, i.e. completely refactoring everything for no reason.

Re: Gemini 3.0 spotted in the wild through A/B testing

#103

Post by brokencode »

dingnuts wrote: post #92
bogtog wrote: post #59

Earlier quoted context omitted.

I agree with the bit about creative writing, and I would add writing more generally. Gemini also allows dumping in >500k tokens of your own writing to give it a sense of your style. The other big use-case I like Gemini for is summarizing papers or teaching me scholarly subjects. Gemini's more verbose than GPT-5, which feels nice for these cases. GPT-5 strikes me as terrible at this, and I'd also put Claude ahead of G…

using an LLM for "creative writing" is like getting on a motorcycle and then claiming you went for a ride on a bicycle no, wait, that analogy isn't even right. it's like going to watch a marathon and then claiming you ran in it.

Just imagine you’re trying to build a custom D&D campaign for your friends.

You might have a fun idea don’t have the time or skills to write yourself that you can have an LLM help out with. Or at least make a first draft you can run with.

What do your friends care if you wrote it yourself or used an LLM? The quality bar is going to be fairly low either way, and if it provides some variation from the typical story books then great.

Re: Gemini 3.0 spotted in the wild through A/B testing

#104

Post by mountainriver »

jmkni wrote: post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

Definitely subjective, I find it significantly worse than GPT or Claude. Particularly for software systems design and coding problems.

Re: Gemini 3.0 spotted in the wild through A/B testing

#105

Post by oneredoak »

SweetSoftPillow wrote: post #41
CaptainOfCoit wrote: post #22

Earlier quoted context omitted.

> consistently found Gemini to be better than ChatGPT, Claude and Deepseek I used Pro Mode in ChatGPT since it was available, and tried Claude, Gemini, Deepseek and more from time to time, but none of them ever get close to Pro Mode, it's just insanely better than everything. So when I hear people comparing "X to ChatGPT", are you testing against the best ChatGPT has to offer, or are you comparing it to "Auto" and ca…

It seems you also did not compare ChatGPT to the best offers of the competitors, as you did not mention Gemini Deepthink mode which is Google's alternative to GPT's Pro mode.

I find Gemini Deep Think to be unbelievably underrated. In my testing, it consistently comes out far ahead of any other model or harness (for system architecture debugging, coming up with excellent YouTube title and hook ideas, etc). You can through a ton of context at it, and Deep Think's attention to detail is excellent.

My only exceptions being Sonnet 4.5 / Codex for code implementation, and Deep Research for anything requiring a ton of web searches.

Re: Gemini 3.0 spotted in the wild through A/B testing

#107

Post by ofek »

The sentiment in this thread surprises me a great deal. For me, Gemini 2.5 Pro is markedly worse than GPT-5 Thinking along every axis of hallucinations, rigidity in its self-assured correctness and sycophancy. Claude Opus used to be marginally better but now Claude Sonnet 4.5 is far better, although not quite on par with GPT-5 Thinking.

I frequently ask the same question side-by-side to all 3 and the only situation in which I sometimes prefer Gemini 2.5 Pro is when making lifestyle choices, like explaining item descriptions on Doordash that aren't in English.

edit: It's more of a system prompt issue but I despise the verbosity of Gemini 2.5 Pro's responses.

Re: Gemini 3.0 spotted in the wild through A/B testing

#108

Post by tuesdaynight »

Topfi wrote: post #2

Has been ongoing for roughly a month now, with a variety of checkpoints along the usual speculation. As it stands, I'd just wait for the official announcement, prior to making any judgement. What their release plans are, whether a checkpoint is a possible replacement for Pro, Flash, Flash Lite, a new category of model, won't be released at all, etc. we cannot know. More importantly, because of the way AIStudio does A…

I understand that hyping is the career of a lot of people, but it's a little annoying how every Twitter link posted here is full of "IT'S A GAME CHANGER!!! NOTHING IS THE SAME ANYMORE!!! BRACE FOR IMPACT!!!" energy. The examples look great, but it's hard to ignore the unprofessional evaluation that you described.

Re: Gemini 3.0 spotted in the wild through A/B testing

#110

Post by Diggsey »

ofek wrote: post #107

The sentiment in this thread surprises me a great deal. For me, Gemini 2.5 Pro is markedly worse than GPT-5 Thinking along every axis of hallucinations, rigidity in its self-assured correctness and sycophancy. Claude Opus used to be marginally better but now Claude Sonnet 4.5 is far better, although not quite on par with GPT-5 Thinking. I frequently ask the same question side-by-side to all 3 and the only situation i…

I've found Gemini to be much better at completing tasks and following instructions. For example, let's say I want to extract all the questions from a word document and output them as a CSV.

If I ask ChatGPT to do this, it will do one of two things:

1) Extract the first ~10-20 questions perfectly, and then either just give up, or else hallucinate a bunch of stuff.

2) Write code that tries to use regex to extract the questions, which then fails because the questions are too free-form to be reliably matched by a regex.

If I ask Gemini to do the same thing, it will just do it and output a perfectly formed and most importantly complete CSV.

Post reply on HN