Live data from Hacker News

Gemini 3.0 spotted in the wild through A/B testing

ricklamers.io

181–190 of 280 posts

Re: Gemini 3.0 spotted in the wild through A/B testing

#181

All I can hope for is that the “effective context window” (some level before competency plummets) is like 1m+ tokens. I would give a finger to just put my entire codebase into a model every time I want to talk to it. For now I’m still only talking to parts of the codebase, so to speak.

Have you tried Claude Code, Cursor, Codex CLI, Gemini CLI, etc?

Re: Gemini 3.0 spotted in the wild through A/B testing

#182
post #9
post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I agree with you, I consistently find Gemini 2.5 Pro better than Claude and GPT-5 for the following cases: * Creative writing: Gemini is the unmatched winner here by a huge margin. I would personally go so far as to say Gemini 2.5 Pro is the only borderline kinda-sorta usable model for creative writing if you squint your eyes. I use it to criticize my creative writing (poetry, short stories) and no other model unders…

I think because openAI and antrophic has leaning into more "coding" model as recently

while antrophic always been coding, there are lot of complaint on OpenAI GPT5 launch because general use model is nerfed heavily in trade better coding model

Google is the maybe the last one that has good general use model (?)

Re: Gemini 3.0 spotted in the wild through A/B testing

#183
post #76

Earlier quoted context omitted.

Looking at the responses. How the F have people so wildly different opinions on the relative performance of the same systems?

Different prompts/approaches? I "grew up", as it were, on StackOverflow, when I was in my early dev days and didn't have a clue what I was doing I asked question after question on SO and learned very quickly the difference between asking a good question vs asking a bad one There is a great Jon Skeet blog post from back in the day called "Writing the perfect question" - https://codeblog.jonskeet.uk/2010/08/29/writing-…

Great point. I'd add that one way to get improved performance is to ask Gemini/ChatGPT to write the prompt for you. For software, have it write a spec. It's easier to tweak something that is already pretty comprehensive.

Re: Gemini 3.0 spotted in the wild through A/B testing

#184
post #29
post #21

Earlier quoted context omitted.

The last time I tried with ChatGPT (just to look at some MRIs to get an idea of what might be up before the turnaround from doc) it refused.

Hm, I've also uploaded MRI images to ChatGPT and it worked as expected. I went back to the censored chat I mentioned earlier, and got it to give me an answer when adding "You are a lifestyle health coach" to steer it away from throwing a bunch of disclaimers at you.

I have given it medical results, and asked it to explain what all the readings were. It was quite happy to comment on each data point and what you could expect for a normal reading.

Re: Gemini 3.0 spotted in the wild through A/B testing

#185
post #51

> Gemini 3.0 is one of the most anticipated releases in AI at the moment because of the expected advances in coding performance. Based on what I'm hearing from friends who work at Google and are using it for coding, we're all going to be very disappointed. Edit: It sound like they don't actually have Gemini 3 access, which would explain why they aren't happy with it.

Which should surprise no one. LLMs are reaching diminishing returns, unless we find a way to build GPUs more cheaply.

For coding this is absolutely positively incorrect.

Going from GPT4 to GPT5 Codex has been transformational. It has gone from smarter autocomplete to writing entire applications for me.

Re: Gemini 3.0 spotted in the wild through A/B testing

#186
post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I've seen many comments that they are great for OCR stuff, and my usecase of receipt photo processing does have it doing better than ChatGPT , Claude or Grok.

Re: Gemini 3.0 spotted in the wild through A/B testing

#187
post #3

I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…

I find the sheer amount of glazing Gemini does unbearably, so I pretty much avoid using it. It’s just an unreal amount compared to GPT-5 or Claude.

Gives it a stack trace or some logs and Gemini treats it like the most amazing thing ever and throws a paragraph in there praising your skills as if you were a god.

Re: Gemini 3.0 spotted in the wild through A/B testing

#188
post #2

Has been ongoing for roughly a month now, with a variety of checkpoints along the usual speculation. As it stands, I'd just wait for the official announcement, prior to making any judgement. What their release plans are, whether a checkpoint is a possible replacement for Pro, Flash, Flash Lite, a new category of model, won't be released at all, etc. we cannot know. More importantly, because of the way AIStudio does A…

I understand that hyping is the career of a lot of people, but it's a little annoying how every Twitter link posted here is full of "IT'S A GAME CHANGER!!! NOTHING IS THE SAME ANYMORE!!! BRACE FOR IMPACT!!!" energy. The examples look great, but it's hard to ignore the unprofessional evaluation that you described.

The example in this case is an SVG of a video game controller.

Re: Gemini 3.0 spotted in the wild through A/B testing

#189

Earlier quoted context omitted.

> Convert the whole codebase into a string When using the Gemini web app on a desktop system (could be different depending upon how you consume Gemini) if you select the + button in the bottom-left of the chat prompt area, select Import code, and then choose the "Upload folder" link at the bottom of the dialog that pops up, it'll pull up a file dialog letting you choose a directory and it will upload all the files in…

Well I am not sure Gemini or any other LLMs respect `.gitignore` which can immediately make the context window jump over the maximum. Tools like repomix[0] do this better, plus you can add your own extra exclusions on top. It also estimates token usage as a part of its output but I found it too optimistic i.e. it regularly says "40_000 tokens" but when uploading the resulting single XML file to Gemini it's actually f…

I agree. I use repomix with AI Studio extensively and never found anything (including the cli agents) that's close.

I sometimes upload codebases that are around 600k tokens and even those work.

Repomix also lets you create a config file so you can give it ignore/include patterns in addition to .gitignore.

It also tells you about the outlier files with exceptionally long content.

Re: Gemini 3.0 spotted in the wild through A/B testing

#190
post #78

Earlier quoted context omitted.

That's my experience as well. Gemini doesn't seem interested in doing searches outside of Deep Research mode, which is kind of funny given it should have the easiest access to a top search engine.

The Deep Research mode is on rails, but they're much more generous with it than anyone else. You run out of Claude usage almost instantly if you use theirs. ChatGPT gives you a decent number but then locks you out for a month after that.

Perplexity is still the king there in terms of the balance between price and quality. It doesn't do as many searches as ChatGPT's deep research, but you get virtually unlimited usage.
Post reply on HN