Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.1 Pro

blog.google

Post reply on HN
Print view View original

811–820 of 951 posts

Re: Gemini 3.1 Pro

#811

Post by dzhiurgis »

briHass wrote: post #720
Decabytes wrote: post #555

Earlier quoted context omitted.

Any tips for working with Gemini through its chat interface? I’ve worked with ChatGPT and Claude and I’ve generally found them pleasant to work with, but everytime I use Gemini the output is straight dookie

Even though I don't like the privacy implications, make sure you use the option to save and use past chats for context. After a few months of back and forth (hundreds of 'chat' sessions), the responses are much higher quality. It sometimes does 'callbacks' to things discussed in past chats, which are typically awkward non-sequiturs, but it does improve it overall. When I play with it in 'temporary chat' mode that ign…

You must be joking. I’ve turned that off after first month of use. It’s unbearable. “Oh since you are in {place i mentioned a week ago while planning trip but ultimately didnt go} the home assistant integration question changes completely”. Or ending every answer with “since you are salesforce consultant, would you like to learn more about iron smelting?”

Re: Gemini 3.1 Pro

#812

Post by MrGreenTea »

embedding-shape wrote: post #229
cdelsolar wrote: post #182

Earlier quoted context omitted.

share

The harness? Trivial to build yourself, ask your LLM for help, it's ~1000 LOC you could hack together in 10-15 minutes. As for the test cases themselves, that would obviously defeat the purpose, so no :)

Would you be willing to give a rough outline of one or a few test cases? I am having a bit of a hard time imagining what and how you are testing. Is it like "change the signature of function X in file @Y to take parameter Z" and then comparing the result with what you expect?

Re: Gemini 3.1 Pro

#813

Post by meeq »

minimaxir wrote: post #483
oblio wrote: post #479

Earlier quoted context omitted.

> Knowledge cutoff is unchanged at Jan 2025. Isn't that a bit old?

Old relative to its competitors, but the Search tool can compensate for it.

It could in practice. Just get ready for some very interesting thinking tokens, akin to a psychotic break, once it interacts with the "simulated reality" and "the user‘s fabrication of a nonexistent timeline within the hypothetical future".

Gemini 3.0 was convinced that my dependency versions pinned in package.json were hallucinated by an AI, because they "shouldn't yet exist". I just hope this kind of behavior is gone.

Re: Gemini 3.1 Pro

#814

Post by energy123 »

tasuki wrote: post #397
energy123 wrote: post #321

Earlier quoted context omitted.

Francois Chollet accuses the big labs of targeting the benchmark, yes. It is benchmaxxed.

Didn't the same Francois Chollet claim that this was the Real Test of Intelligence? If they target it, perhaps they target... real intelligence?

He said in an interview that it doesn't count if it's explicitly targeted, only if a model generalizes to it.

He also said that the "real test of intelligence" is being unable to come up with new tests that a human can easily do that the AI can't, not in being able to pass any specific benchmark.

Re: Gemini 3.1 Pro

#815

Post by suddenlybananas »

AlphaAndOmega0 wrote: post #538
suddenlybananas wrote: post #307

Earlier quoted context omitted.

I'm not denying any progress, I'm saying that reasoning failures that are simple which have gone viral are exactly the kind of thing that they will toss in the training data. Why wouldn't they? There's real reputational risks in not fixing it and no costs in fixing it.

Given that Gemini 3 Pro already did solid on that test, what exactly did they improve? Why would they bother? I double checked and tested on AI Studio, since you can still access the previous model there: >You should drive. >If you walk there, your car will stay behind, and you won't be able to wash it. Thinking models consistently get it correct and did when the test was brand new (like a week or two ago). It is the…

Why would they bother? Because it costs essentially nothing to add it to the training data. My point is that once a reasoning example becomes sufficiently viral, it ceases to be a good test because companies have a massive incentive to correct it. The fact some models got it right before (unreliably) doesn't mean they wouldn't want to ensure that the model gets it right.

Re: Gemini 3.1 Pro

#816

Post by magicalhippo »

cheema33 wrote: post #551
agentifysh wrote: post #450

My enthusiasm is a bit muted this cycle because I've been burned by Gemini CLI. These models are very capable but Gemini CLI just doesn't seem to be able to work for one it never follows instructions strictly like its competitors do, and it hallucinates even which is a rarity. More importantly feels like Google is stretched thin across different Gemini products and pricing reflects this, I still have no idea how to p…

> I still have no idea how to pay for Gemini CLI, in codex/claude its very simple $20/month for entry and $200/month for ton of weekly usage. This! I would like to sign up for a paid plan for Gemini CLI. But I have not been able to figure out how. I already have Codex and Claude plans. Those were super easy to sign up for.

Supposedly the Google One AI plans[1] allow for this. I've been testing an AI Pro plan, but haven't gotten CLI to work yet (keeps asking me to sign in). So yeah...

[1]: https://one.google.com/about/#compare-plans

Re: Gemini 3.1 Pro

#817

Post by ckdot »

spankalee wrote: post #199

I hope this works better than 3.0 Pro I'm a former Googler and know some people near the team, so I mildly root for them to at least do well, but Gemini is consistently the most frustrating model I've used for development. It's stunningly good at reasoning, design, and generating the raw code, but it just falls over a lot when actually trying to get things done, especially compared to Claude Opus. Within VS Code Copi…

Gemini 3.1 is surprisingly bad at coding, especially if you consider that they built an IDE (Antigravity) around it: I let it carefully develop a plan according to very specific instructions. The outcome was terrible: AGENTS.md ignored, syntax error in XML (closing tag missed), inconsistent namings, misinterpreting console outputs, which where quite clear ("You forgot to add some attribute foobar"). I‘m quite disappointed.

Re: Gemini 3.1 Pro

#818

Post by EugeneOZ »

xrd wrote: post #306

These models are so powerful. It's totally possible to build entire software products in the fraction of the time it took before. But, reading the comments here, the behaviors from one version to another point version (not major version mind you) seem very divergent. It feels like we are now able to manage incredibly smart engineers for a month at the price of a good sushi dinner. But it also feels like you have to b…

You still need a human (working at human speed) to review every generated line, if it’s not a throwaway app or some demo to impress investors.

Re: Gemini 3.1 Pro

#819

Post by cherryteastain »

quacky_batak wrote: post #54

I’m keen to know how and where are you using Gemini. Anthropic is clearly targeted to developers and OpenAI is general go to AI model. Who are the target demographic for Gemini models? ik that they are good and Flash is super impressive. but i’m curious

I switched to it for my personal subscription because on discount it was less than half the price of ChatGPT Plus/Claude Pro

Re: Gemini 3.1 Pro

#820

Post by EugeneOZ »

nickandbro wrote: post #51

Does well on SVGs outside of "pelican riding on a bicycle" test. Like this prompt: "create a svg of a unicorn playing xbox" https://www.svgviewer.dev/s/NeKACuHj Still some tweaks to the final result, but I am guessing with the ARC-AGI benchmark jumping so much, the model's visual abilities are allowing it to do this well.

Still not usable in production, not even near. But I'm happy to see any progress in this area.
Post reply on HN