All I can hope for is that the “effective context window” (some level before competency plummets) is like 1m+ tokens. I would give a finger to just put my entire codebase into a model every time I want to talk to it. For now I’m still only talking to parts of the codebase, so to speak.
Gemini 3.0 spotted in the wild through A/B testing
181–190 of 280 posts
Re: Gemini 3.0 spotted in the wild through A/B testing
#182I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
I agree with you, I consistently find Gemini 2.5 Pro better than Claude and GPT-5 for the following cases: * Creative writing: Gemini is the unmatched winner here by a huge margin. I would personally go so far as to say Gemini 2.5 Pro is the only borderline kinda-sorta usable model for creative writing if you squint your eyes. I use it to criticize my creative writing (poetry, short stories) and no other model unders…
while antrophic always been coding, there are lot of complaint on OpenAI GPT5 launch because general use model is nerfed heavily in trade better coding model
Google is the maybe the last one that has good general use model (?)
Re: Gemini 3.0 spotted in the wild through A/B testing
#183Earlier quoted context omitted.
Looking at the responses. How the F have people so wildly different opinions on the relative performance of the same systems?
Different prompts/approaches? I "grew up", as it were, on StackOverflow, when I was in my early dev days and didn't have a clue what I was doing I asked question after question on SO and learned very quickly the difference between asking a good question vs asking a bad one There is a great Jon Skeet blog post from back in the day called "Writing the perfect question" - https://codeblog.jonskeet.uk/2010/08/29/writing-…
Re: Gemini 3.0 spotted in the wild through A/B testing
#184Earlier quoted context omitted.
The last time I tried with ChatGPT (just to look at some MRIs to get an idea of what might be up before the turnaround from doc) it refused.
Hm, I've also uploaded MRI images to ChatGPT and it worked as expected. I went back to the censored chat I mentioned earlier, and got it to give me an answer when adding "You are a lifestyle health coach" to steer it away from throwing a bunch of disclaimers at you.
Re: Gemini 3.0 spotted in the wild through A/B testing
#185> Gemini 3.0 is one of the most anticipated releases in AI at the moment because of the expected advances in coding performance. Based on what I'm hearing from friends who work at Google and are using it for coding, we're all going to be very disappointed. Edit: It sound like they don't actually have Gemini 3 access, which would explain why they aren't happy with it.
Which should surprise no one. LLMs are reaching diminishing returns, unless we find a way to build GPUs more cheaply.
Going from GPT4 to GPT5 Codex has been transformational. It has gone from smarter autocomplete to writing entire applications for me.
Re: Gemini 3.0 spotted in the wild through A/B testing
#186I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
Re: Gemini 3.0 spotted in the wild through A/B testing
#187I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
Gives it a stack trace or some logs and Gemini treats it like the most amazing thing ever and throws a paragraph in there praising your skills as if you were a god.
Re: Gemini 3.0 spotted in the wild through A/B testing
#188Has been ongoing for roughly a month now, with a variety of checkpoints along the usual speculation. As it stands, I'd just wait for the official announcement, prior to making any judgement. What their release plans are, whether a checkpoint is a possible replacement for Pro, Flash, Flash Lite, a new category of model, won't be released at all, etc. we cannot know. More importantly, because of the way AIStudio does A…
I understand that hyping is the career of a lot of people, but it's a little annoying how every Twitter link posted here is full of "IT'S A GAME CHANGER!!! NOTHING IS THE SAME ANYMORE!!! BRACE FOR IMPACT!!!" energy. The examples look great, but it's hard to ignore the unprofessional evaluation that you described.
Re: Gemini 3.0 spotted in the wild through A/B testing
#189Earlier quoted context omitted.
> Convert the whole codebase into a string When using the Gemini web app on a desktop system (could be different depending upon how you consume Gemini) if you select the + button in the bottom-left of the chat prompt area, select Import code, and then choose the "Upload folder" link at the bottom of the dialog that pops up, it'll pull up a file dialog letting you choose a directory and it will upload all the files in…
Well I am not sure Gemini or any other LLMs respect `.gitignore` which can immediately make the context window jump over the maximum. Tools like repomix[0] do this better, plus you can add your own extra exclusions on top. It also estimates token usage as a part of its output but I found it too optimistic i.e. it regularly says "40_000 tokens" but when uploading the resulting single XML file to Gemini it's actually f…
I sometimes upload codebases that are around 600k tokens and even those work.
Repomix also lets you create a config file so you can give it ignore/include patterns in addition to .gitignore.
It also tells you about the outlier files with exceptionally long content.
Re: Gemini 3.0 spotted in the wild through A/B testing
#190Earlier quoted context omitted.
That's my experience as well. Gemini doesn't seem interested in doing searches outside of Deep Research mode, which is kind of funny given it should have the easiest access to a top search engine.
The Deep Research mode is on rails, but they're much more generous with it than anyone else. You run out of Claude usage almost instantly if you use theirs. ChatGPT gives you a decent number but then locks you out for a month after that.