Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

Post reply on HN
Print view View original

311–320 of 728 posts

Re: Gemini 2.5 Pro Preview

#311

Post by cbg0 »

mohsen1 wrote: post #87

I use Gemini for almost everything. But their model card[1] only compares to o3-mini! In known benchmarks o3 is still ahead: +------------------------------+---------+--------------+ | Benchmark | o3 | Gemini 2.5 | | | | Pro | +------------------------------+---------+--------------+ | ARC-AGI (High Compute) | 87.5% | — | | GPQA Diamond (Science) | 87.7% | 84.0% | | AIME 2024 (Math) | 96.7% | 92.0% | | SWE-bench Veri…

o3 is $40/M output tokens and 2.5 Pro is $10-15/M output tokens so o3 being slightly ahead is not really worth 4 times more than gemini.

Re: Gemini 2.5 Pro Preview

#312

Post by mbesto »

segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

To date, LLMs can't replace the human element of:

- Determining what features to make for users

- Forecasting out a roadmap that are aligned to business goals

- Translating and prioritizing all of these to a developer (regardless of whether these developers are agentic or human)

Coincidentally these are the areas that frequently are the largest contributors to software businesses successes....not wether you use NextJs with a Go and Elixir backend against a multi-geo redundant multi sharded CockroachDB database, or that your code is clean/elegant.

Re: Gemini 2.5 Pro Preview

#313

Post by mattlondon »

obsolete_wagie wrote: post #188

o3 is so far ahead of antrhopic and google, these models arent even worth using

The benchmarks (1) seem to suggest that o3 is in 3rd place after Gemini 2.5 pro preview and Gemini 2.5 pro exp (for text reasoning, o3 4th for webdev). o3 doesn't even appear on the openrouter leaderboards (2) suggesting is hardly used (if at all) by anyone using LLMs do actually do anything (such as coding) which makes one question if it is actually any good at all (otherwise if it was so great I'd expect to see heavy usage)

Not sure where your data is coming from but everything else is pointing to Google supremacy in AI right now. I look forward to some new models from Anthropic, xAi, Meta et al (remains to be seen if OpenAI has anything left apart from bluster). Exciting times.

1 - https://beta.lmarena.ai/leaderboard

2 - https://openrouter.ai/rankings

Re: Gemini 2.5 Pro Preview

#314

Post by jstummbillig »

segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

> no amount of prompting will get current models to approach abstraction and architecture the way a person does

I find this sentiment increasingly worrisome. It's entirely clear that every last human will be beaten on code design in the upcoming years (I am not going to argue if it's 1 or 5 years away, who cares?)

I wished people would just stop holding on to what amounts to nothing, and think and talk more about what can be done in a new world. We need good ideas and I think this could be a place to advance them.

Re: Gemini 2.5 Pro Preview

#315

Post by otabdeveloper4 »

codebolt wrote: post #80
crat3r wrote: post #27

So, are people using these tools without the org they work for knowing? The amount of hoops I would have to jump through to get either of the smaller companies I have worked for since the AI boom to let me use a tool like this would make it absolutely not worth the effort. I'm assuming large companies are mandating it, but ultimately the work that these LLMs seem poised for would benefit smaller companies most and I…

If you can get them to approve GitHub Copilot Business then Gemini Pro 2.5 and many others are available there. They have guarantees that they don't share/store prompts or code and the parent company is Microsoft. If you can argue that they will save money (on saved developer time), what would be their argument against?

> They have guarantees that they don't share/store prompts or code

"They trust me. Dumb ..."

Re: Gemini 2.5 Pro Preview

#316

Post by celeritascelery »

disgruntledphd2 wrote: post #306
toomuchtodo wrote: post #300

Earlier quoted context omitted.

It seems like the fix is straightforward (check the output against a machine readable spec before providing it to the user), but perhaps I am a rube. This is no different than me clicking through a search result to the underlying page to verify the veracity of the search result surfaced.

Why coding agents et al don't make use of the AST through LSP is a question I've been asking myself since the first release of GitHub copilot. I assume that it's trickier than it seems as it hasn't happened yet.

What good do you think that would do?

Re: Gemini 2.5 Pro Preview

#317

Post by mynameisvlad »

sanderjd wrote: post #256
mynameisvlad wrote: post #236

Earlier quoted context omitted.

Sure, but that's not the problem people have with them nor the general criticism. It's that people without the knowledge to do abstraction and architecture don't realize the importance of these things and pretend that "vibe coding" is a reasonable alternative to a well-thought-out project.

The way I see this is that it's just another skill differentiator that you can take advantage of if you can get it right. That is, if it's true that abstraction and architecture are useful for a given product, then people who know how to do those things will succeed in creating that product, and those who don't will fail. I think this is true for essentially all production software, but a lot of software never reache…

> "vibecoded" proofs of concept

The fact that you called it out as a PoC is already many bars above what most vibe coders are doing. Which is considering a barely functioning web app as proof that vibe coding is a viable solution for coding in general.

> I do worry about what the careers of entry level people will look like. It isn't obvious to me how they'll naturally develop any of these skills.

Exactly. There isn't really a path forward from vibe coding to anything productizable without actual, deep CS knowledge. And LLMs are not providing that.

Re: Gemini 2.5 Pro Preview

#318

Post by nearbuy »

mbesto wrote: post #312
segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

To date, LLMs can't replace the human element of: - Determining what features to make for users - Forecasting out a roadmap that are aligned to business goals - Translating and prioritizing all of these to a developer (regardless of whether these developers are agentic or human) Coincidentally these are the areas that frequently are the largest contributors to software businesses successes....not wether you use NextJ…

What does it say when you ask it to?

Re: Gemini 2.5 Pro Preview

#319

Post by froh »

segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

searching and ranking existing fragments and recombining them within well known paths is one thing, exploratively combining existing fragments to completely novel solutions quickly runs into combinatorial explosion.

so it's a great tool in the hands of a creative architect, but it is not one in and by itself and I don't see yet how it can be.

my pet theory is that the human brain can't understand and formalize its creativity because you need a higher order logic to fully capture some other logic. I've been contested that the second Gödel incompleteness theorem "can't be applied like this to the brain" but I stubbornly insist yes, the brain implements _some_ formal system and it can't understand how that system works. tongue in cheek, somewhat, maybe.

but back to earth I agree llms are a great tool for a creative human mind.

Re: Gemini 2.5 Pro Preview

#320

Post by jorl17 »

cbg0 wrote: post #311
mohsen1 wrote: post #87

I use Gemini for almost everything. But their model card[1] only compares to o3-mini! In known benchmarks o3 is still ahead: +------------------------------+---------+--------------+ | Benchmark | o3 | Gemini 2.5 | | | | Pro | +------------------------------+---------+--------------+ | ARC-AGI (High Compute) | 87.5% | — | | GPQA Diamond (Science) | 87.7% | 84.0% | | AIME 2024 (Math) | 96.7% | 92.0% | | SWE-bench Veri…

o3 is $40/M output tokens and 2.5 Pro is $10-15/M output tokens so o3 being slightly ahead is not really worth 4 times more than gemini.

Also, o3 is insanely slow compared to Gemini 2.5 Pro
Post reply on HN