Adjhe
Gemini 3.0 spotted in the wild through A/B testing
61–70 of 280 posts
Re: Gemini 3.0 spotted in the wild through A/B testing
#62I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
I find Claude and Gemini to be wildly inferior to ChatGPT when it comes to doing searches to establish grounding. Gemini seems to do a handful of searches and then make shit up, where ChatGPT will do dozens or even hundreds of searches - and do searches based on what it finds in earlier ones.
Try "AI Mode" on Google.com (Disclaimer, I recently joined the team that makes this product).
It isn't Gemini (the product, those are different orgs) though there may (deliberately left ambiguous) be overlap in LLM level bytes.
My recommendation for you in this use-case comes from the fact that AI Mode is a product that is built to be a good search engine first, presented to you in the interface of an AI Chatbot. Rather than Gemini (the app/site) which is an AI Chatbot that had search tooling added to it later (like its competitors).
AI Mode does many more searches (in my experience) for grounding and synthesis than Gemini or ChatGPT.
Re: Gemini 3.0 spotted in the wild through A/B testing
#63> Gemini 3.0 is one of the most anticipated releases in AI at the moment because of the expected advances in coding performance. Based on what I'm hearing from friends who work at Google and are using it for coding, we're all going to be very disappointed. Edit: It sound like they don't actually have Gemini 3 access, which would explain why they aren't happy with it.
Which should surprise no one. LLMs are reaching diminishing returns, unless we find a way to build GPUs more cheaply.
Re: Gemini 3.0 spotted in the wild through A/B testing
#64I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
It has been consistently better at least with C++ ever since like o3, in my experience. The last ChatGPT model I loved was o1-pro.
Re: Gemini 3.0 spotted in the wild through A/B testing
#65I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
Yeah it’s really good. A few weeks ago, some third party script was messing with click events of my react buttons so I figured I should just add a mousedown even to capture the click before the other script. It was late at night and I was exhausted so I wanted to do a quick and dirty approach of simulating a click after a few ms after the mousedown even. So I told Gemini my plan and asked it to tell me the average time in ms for a click event in order to simulate it… and I was shocked when it straight up refused and told me instead to trigger the event on mouseup in combination with mousedown (on mouse down set state and on mouse up check the state and trigger the event). This was of course a much better solution. I was shocked at how it understood the problem perfectly and instead of giving me exactly what I asked for it gave me the right way to go about it.
Re: Gemini 3.0 spotted in the wild through A/B testing
#66https://x.com/chetaslua is experimenting a lot with Gemini 3 and posting its results (various web desktops, a vampire survivor clone which is actually very playable, voxel 3d models, other game clones, SVG etc). They look really good, specially when they are one-shot.
This was cool: https://codepen.io/ChetasLua/pen/yyezLjN
Somewhat amusing 4th wall breaking if you open Python from the terminal in the fake Windows. Examples: 1. If you try to print something using the "Python" print keyword, it opens a print dialog in your browser. 2. If you try to open a file using the "Python" open keyword, it opens a new browser tab trying to access that file.
That is, it's forwarding the print and open calls to your browser.
Re: Gemini 3.0 spotted in the wild through A/B testing
#67I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
[dead]
Re: Gemini 3.0 spotted in the wild through A/B testing
#68Earlier quoted context omitted.
Agreed, and its larger context window is fantastic. My workflow: - Convert the whole codebase into a string - Paste it into Gemini - Ask a question People seem to be very taken with "agentic" approaches were the model selects a few files to look at, but I've found it very effective and convenient just to give the model the whole codebase, and then have a conversation with it, get it to output code, modify a file, etc…
I usually do that in a 2 step process. Instead of giving the full source code to the model, I will ask it to write a comprehensive, detailed, description of the architecture, intent, and details (including filenames) of the codebase to a Markdown file. Then for each subsequent conversation I would ask the model to use this file as reference. The overall idea is the same, but going through an intermediate file allows…
My special hack on top of what you suggested: Ask it to draw the whole codebase in graphviz compatible graphing markup language. There are various tools out there to render this as an SVG or whatever, to get an actual map of the system. Very helpful when diving in to a big new area.
Re: Gemini 3.0 spotted in the wild through A/B testing
#69I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
For pure text responses, agree 100%. Gemini falls way short on tool/function calling, and it's not very token-efficient for those of us using the API. But if they can fix those two things or even just get them in the same ballpark like they did with flash and flash-lite, it would easily become my primary model.
Re: Gemini 3.0 spotted in the wild through A/B testing
#70I might be in the minority here but I've consistently found Gemini to be better than ChatGPT, Claude and Deepseek (I get access to all of the pro models through work) Maybe it's just the kind of work I'm doing, a lot of web development with html/scss, and Google has crawled the internet so they have more data to work with. I reckon different models are better at different kinds of work, but Gemini is pretty excellent…
Gemini was good when the thinking tokens were shown to the user. As soon as Google replaced those with some thought summary, I stopped finding it as useful. Previously, the thoughts were so organized that I would often read those instead of the final answer.
In the API, the thinking tokens are just a different stream. You can still read them.