Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

Post reply on HN
Print view View original

431–440 of 728 posts

Re: Gemini 2.5 Pro Preview

#431

Post by tastysandwich »

segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

Re hallucinating APIs that don't exist - I find this with Golang sometimes. I wonder if it's because the training data doesn't just consist of all the docs and source code, but potentially feature proposals that never made it into the language.

Regexes are another area where I can't get much help from LLMs. If it's something common like a phone number, that's fine. But anything novel it seems to have trouble. It will spit out junk very confidently.

Re: Gemini 2.5 Pro Preview

#432

Post by bdangubic »

jstummbillig wrote: post #314
segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

> no amount of prompting will get current models to approach abstraction and architecture the way a person does I find this sentiment increasingly worrisome. It's entirely clear that every last human will be beaten on code design in the upcoming years (I am not going to argue if it's 1 or 5 years away, who cares?) I wished people would just stop holding on to what amounts to nothing, and think and talk more about wha…

It's entirely clear that every last human will be beaten on code design in the upcoming years (I am not going to argue if it's 1 or 5 years away, who cares?)

Our entire industry (after all these years) does not have even remotely sane measure or definition as what is good code design. Hence, this statement is dead on arrival as you are claiming something that cannot be either proven or disproven by anyone.

Re: Gemini 2.5 Pro Preview

#433

Post by sweezyjeezy »

sigmaisaletter wrote: post #427
sweezyjeezy wrote: post #425

Earlier quoted context omitted.

Again - I'd argue that the extraordinary success of LLMs, in a relatively short amount of time, using a fairly unsophisticated training approach, is strong evidence that coding models are going to get a lot better than they are right now. Will it definitely surpass every human? I don't know, but I wouldn't say we're lacking extraordinary evidence for that claim either. The way you've framed it seems like the only evi…

Well, predicting the future is always hard. But if someone claims some extraordinary future event is going to happen, you at least ask for their reasons for claiming so, don't you. In my mind, at this point we either need (a) some previously "hidden" super-massive source of training data, or (b) another architectural breakthrough. Without either, this is a game of optimization, and the scaling curves are going to pla…

A couple of comments

a) it hasn't even been a year since the last big breakthrough, the reasoning models like o3 only came out in September, and we don't know how far those will go yet. I'd wait a second before assuming the low-hanging fruit is done.

b) I think coding is a really good environment for agents / reinforcement learning. Rather than requiring a continual supply of new training data, we give the model coding tasks to execute (writing / maintaining / modifying) and then test its code for correctness. We could for example take the entire history of a code-base and just give the model its changing unit + integration tests to implement. My hunch (with no extraordinary evidence) is that this is how coding agents start to nail some of the higher-level abilities.

Re: Gemini 2.5 Pro Preview

#434

Post by WaltPurvis »

mvdtnz wrote: post #310

I truly do not understand how people are getting worthwhile results from Gemini 2.5 Pro. I have used all of the major models for lots of different programming tasks and I have never once had Gemini produce something useful. It's not just wrong, it's laughably bad. And people are making claims that it's the best. I just... don't... get it.

That's weird. What languages/frameworks/tasks are you using it for? I've been using Gemini 2.5 with Dart recently and it frequently produces indisputably useful code, and indisputably helpful advice. Along with some code that's pretty dumb or misguided, and some advice that would be counterproductive if I actually followed it. But "never once had Gemini produce something useful" is wildly different from my recent experience.

Re: Gemini 2.5 Pro Preview

#435

Post by nicce »

paulirish wrote: post #250

> Gemini 2.5 Pro now ranks #1 on the WebDev Arena leaderboard It'd make sense to rename WebDev Arena to React/Tailwind Arena. Its system prompt requires [1] those technologies and the entire tool breaks when requesting vanilla JS or other frameworks. The second-order implications of models competing on this narrow definition of webdev are rather troublesome. [1] https://blog.lmarena.ai/blog/2025/webdev-arena/#:~:text…

> It'd make sense to rename WebDev Arena to React/Tailwind Arena.

Funnily, training of these models feels getting cut mid of v3/v4 Tailwind release, and Gemini always try to correct my mistakes (… use v3 instead of v4)

Re: Gemini 2.5 Pro Preview

#436

Post by astrange »

cft wrote: post #392
giovannibonetti wrote: post #376

Earlier quoted context omitted.

A board game has a much narrower scope than programming in general.

Thus this was in 2016. 9 years have passed.

LLMs and AlphaGo don't work at all similarly, since LLMs don't use search.

I think everyone expected AlphaGo to be the research direction to pursue, which is why it was so surprising that LLMs turned out to work.

Re: Gemini 2.5 Pro Preview

#437

Post by astrange »

mattgreenrocks wrote: post #342
jstummbillig wrote: post #314

Earlier quoted context omitted.

> no amount of prompting will get current models to approach abstraction and architecture the way a person does I find this sentiment increasingly worrisome. It's entirely clear that every last human will be beaten on code design in the upcoming years (I am not going to argue if it's 1 or 5 years away, who cares?) I wished people would just stop holding on to what amounts to nothing, and think and talk more about wha…

I'm always impressed by the ability of the comment section to come up with more reasons why decent design and architecture of source code just can't happen: * "it's too hard!" * "my coworkers will just ruin it" * "startups need to pursue PMF, not architecture" * "good design doesn't get you promoted" And now we have "AI will do it better soon." None of those are entirely wrong. They're not entirely correct, either.

> * "my coworkers will just ruin it"

This turns out to be a big issue. I read everything about software design I could get my hands on in years, but then at an actual large company it turned out to not help, because I'd never read anything about how to get others to follow the advice in my head from all that reading.

Re: Gemini 2.5 Pro Preview

#438

Post by astrange »

mdp2021 wrote: post #350
disgruntledphd2 wrote: post #305

Earlier quoted context omitted.

They are definitely pattern matching. Like, that's how we train them, and no matter how many layers of post training you add, you won't get too far from next token prediction. And that's fine and useful.

> fine and useful And crippled, incomplete, and deceiving, dangerous.

That's normal for any professional tool, but it's not normal to be so upset about it. A saw will take your finger off, but you still want to use it for woodworking.

Re: Gemini 2.5 Pro Preview

#439

Post by nicce »

aero142 wrote: post #354
paulirish wrote: post #250

> Gemini 2.5 Pro now ranks #1 on the WebDev Arena leaderboard It'd make sense to rename WebDev Arena to React/Tailwind Arena. Its system prompt requires [1] those technologies and the entire tool breaks when requesting vanilla JS or other frameworks. The second-order implications of models competing on this narrow definition of webdev are rather troublesome. [1] https://blog.lmarena.ai/blog/2025/webdev-arena/#:~:text…

If llms are able to write better code with more declarative and local programming components and tailwind, then I could imagine a future where a new programming language is created to maximize llm success.

> I could imagine a future where a new programming language is created to maximize llm success.

Who will write the useful training data without LLMs? I feel we are getting less and less new things. Changes will be smaller and incremental.

Re: Gemini 2.5 Pro Preview

#440

Post by sirstoke »

jstummbillig wrote: post #314
segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

> no amount of prompting will get current models to approach abstraction and architecture the way a person does I find this sentiment increasingly worrisome. It's entirely clear that every last human will be beaten on code design in the upcoming years (I am not going to argue if it's 1 or 5 years away, who cares?) I wished people would just stop holding on to what amounts to nothing, and think and talk more about wha…

I’ve been thinking about the SWE employment conundrum in a post-LLM world for a while now, and since my livelihood (and that of my loved ones’) depends on it, I’m obviously biased. Still, I would like to understand where my logic is flawed, if it is. (I.e I’m trying to argue in good faith here)

Isn’t software engineering a lot more than just writing code? And I mean like, A LOT more?

Informing product roadmaps, balancing tradeoffs, understanding relationships between teams, prioritizing between separate tasks, pushing back on tech debt, responding to incidents, it’s a feature and not a bug, …

I’m not saying LLMs will never be able to do this (who knows?), but I’m pretty sure SWEs won’t be the only role affected (or even the most affected) if it comes to this point.

Where am I wrong?

Post reply on HN