Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

blog.jcz.dev

Post reply on HN
Print view View original

51–60 of 97 posts

Re: Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

#51

Post by murukesh_s »

blauditore wrote: post #48
murukesh_s wrote: post #45

Earlier quoted context omitted.

I used to think the same until latest agents started adding perfectly fine features to a large existing react app with just basic input (in English) . Most of the jobs require levels of intelligence below that. It's just a matter of time before agents get to that.

It's about the complexity of the task. Front end apps tend do be much less complex and boilerplate-y than backends, hence AI tends to work better.

I disagree - having worked on backends most of the time, I find modern frontend much more complex (and difficult to test) than pure backend. When I say modern frontend - its mostly React, state management like Redux, Zustand, Router framework like React Router, a CSS framework like Tailwind and component framework like Shadcn. Not to mention different versions of React, different ways of managing state, animation/transitions etc. And on top of that the ever increasing complex quirks in the codebase still needed to be compatible with all the modern browsers and device sizes/orientation out there.

Re: Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

#52

Post by murukesh_s »

etse wrote: post #50
blauditore wrote: post #48

Earlier quoted context omitted.

It's about the complexity of the task. Front end apps tend do be much less complex and boilerplate-y than backends, hence AI tends to work better.

Isn’t frontend more complex? If my task starts with a Figma UI design, how well does a code agent do at generating working code that looks right, and iterate on it (presuming some browser MCP)? Some automated tests seem enough for an genetic loop on backend.

>Isn’t frontend more complex? If my task starts with a Figma UI design, how well does a code agent do at generating working code that looks right, and iterate on it (presuming some browser MCP)? Some automated tests seem enough for an genetic loop on backend.

Haven't tried a Figma design, but i built an internal tool entirely via instructions to agent. The kind of work I could easily quote 3 weeks previously.

Re: Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

#55

Post by [deleted] »

ctoth wrote: post #41
andrepd wrote: post #8

Earlier quoted context omitted.

There was a well-publicised "Claude plays Pokémon" stream where Claude failed to complete Pokemon Blue in spectacular fashion, despite weeks of trying. I think only a very gullible person would assume that future LLMs didn't specifically bake this into their training, as they do for popular benchmarks or for penguins riding a bike.

> as they do for popular benchmarks or for penguins riding a bike. Citation?

[deleted]

Re: Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

#56

Post by orbital-decay »

The baked-in assumptions observation is basically the opposite of the impression I get after watching Gemini 3's CoT. With the maximum reasoning effort it's able to break out of the wrong route by rethinking the strategy. For example I gave it an onion address without the .onion part, and told it to figure out what this string means. All reasoning models including Gemini 2.5 and 3 assume it's a puzzle or a cipher (because they're trained on those) and start endlessly applying different algorithms to no avail. Gemini 3 Pro is the only model that can break the initial assumption after running out of ideas ("Wait, the user said it's just a string, what if it's NOT obfuscated"), and correctly identify the string as an onion address. My guess is they trained it on simulations to enforce the anti-jailbreaking commands injected by the Model Armor, as its CoT is incredibly paranoid at times. I could be wrong, of course.

Re: Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

#57

Post by rybosome »

sussmannbaka wrote: post #43

So after years of being gleefully told that AI will replace all jobs an omniscient state of the art model, with heavy assistance, takes more than two weeks and thousands of dollars in tokens to do what child me did in a few days? Huh.

“And, because AI never got any better or any cheaper after that point, sussmanbaka’s wry observation remained true in perpetuity, forever.”

- History, most likely

Re: Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

#58

Post by ribosometronome »

blauditore wrote: post #48
murukesh_s wrote: post #45

Earlier quoted context omitted.

I used to think the same until latest agents started adding perfectly fine features to a large existing react app with just basic input (in English) . Most of the jobs require levels of intelligence below that. It's just a matter of time before agents get to that.

It's about the complexity of the task. Front end apps tend do be much less complex and boilerplate-y than backends, hence AI tends to work better.

Or perhaps the sort of things it's been trained on? There's not really a huge corpus of material re: beating Pokemon in the manner it has to play Pokemon, especially compared to the mountains of code these models have access to.

Re: Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal

#60

Post by prmoustache »

squimmy26 wrote: post #15

How certain can we be that these improvements aren't just a result of Gemini 3 Pro pre-training on endless internet writeups of where 2.5 has struggled (and almost certainly what a human would have done instead)? In other words, how much of this improvement is true generalization vs memorization?

Isn't that the point of a new model anyway?
Post reply on HN