Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

Post reply on HN
Print view View original

521–530 of 728 posts

Re: Gemini 2.5 Pro Preview

#521

Post by AnIrishDuck »

energy123 wrote: post #517
xyzzy123 wrote: post #493

Earlier quoted context omitted.

I can't point to any evidence. Also I can't think of what direct evidence I could present that would be convincing, short of an actual demonstration? I would like to try to justify my intuition though: Seems like the key question is: should we expect AI programming performance to scale well as more compute and specialised training is thrown at it? I don't see why not, it seems an almost ideal problem domain? * Short…

This is my view. We've seen this before in other problems where there's an on-hand automatic verifier. The nature of the problem mirrors previously solved problems. The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective. I don't believe they can. Until they can, I'm going to assume that LLMs will soon be better than any living human at writing good code.

> The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective.

An obviously correct automatable objective function? Programming can be generally described as converting a human-defined specification (often very, very rough and loose) into a bunch of precise text files.

Sure, you can use proxies like compilation success / failure and unit tests for RL. But key gaps remain. I'm unaware of any objective function that can grade "do these tests match the intent behind this user request".

Contrast with the automatically verifiable "is a player in checkmate on this board?"

Re: Gemini 2.5 Pro Preview

#522

Post by nnnnnande »

ArthurStacks wrote: post #512
ssalazar wrote: post #454

Earlier quoted context omitted.

I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…

I run a software development company with dozens of staff across multiple countries. Gemini has us to the point where we can actually stop hiring for certain roles and staff have been informed they must make use of these tools or they are surplus to requirements. At the current rate of improvement I believe we will be operating on far less staff in 2 years time.

[flagged]

Re: Gemini 2.5 Pro Preview

#523

Post by ArthurStacks »

mrheosuper wrote: post #520
ArthurStacks wrote: post #514

Earlier quoted context omitted.

Beating humans isnt really what matters. Its enabling developers to design who cant. Last month I had a staff member design and build a distributed system that would be far beyond their capabilities without AI assistance. As a business owner this allows me to reduce the dependency and power of the senior devs.

"With great power comes great responsibility" Does that junior dev take responsibility when that system breaks ?

Its his and his managers product, so yes. We don't care if they code it, don't code it, whether an AI builds it or a cheap Indian. Theyre still responsible.

Re: Gemini 2.5 Pro Preview

#524

Post by microtonal »

xyzzy123 wrote: post #493
ssalazar wrote: post #454

Earlier quoted context omitted.

I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…

I can't point to any evidence. Also I can't think of what direct evidence I could present that would be convincing, short of an actual demonstration? I would like to try to justify my intuition though: Seems like the key question is: should we expect AI programming performance to scale well as more compute and specialised training is thrown at it? I don't see why not, it seems an almost ideal problem domain? * Short…

LLMs will still hit a ceiling without human-like reasoning. Even two weeks ago, Claude 3.7 made basic mistakes like trying to convince me the and >= operators on Python sets have the same semantics [1]. Any human would quickly reject something like that (why would be two different operators evaluate to the same value), unless there is overwhelming evidence. Mistakes like this show up all the time, which makes me believe LLMs are still very good at matching/reproducing code it has seen. Besides that I've found that LLMs are really bad at novel problems that were not seen in the training data.

Also, the reward functions that you mention don't necessarily lead to great code, only running code. The should be possible in the third bullet point does very heavy lifting.

At any rate, I can be convinced that LLMs will lead to substantially-reduced teams. There is a lot of junior-level code that I can let an LLM write and for non-junior level code, you can write/refactor things much faster than by hand, but you need a domain/API/design expert to supervise the LLM. I think in the end it makes programming much more interesting, because you can focus on the interesting problems, and less on the boilerplate, searching API docs, etc.

[1] https://ibb.co/pvm5DqPh

Re: Gemini 2.5 Pro Preview

#527

Post by ArthurStacks »

nnnnnande wrote: post #522
ArthurStacks wrote: post #512

Earlier quoted context omitted.

I run a software development company with dozens of staff across multiple countries. Gemini has us to the point where we can actually stop hiring for certain roles and staff have been informed they must make use of these tools or they are surplus to requirements. At the current rate of improvement I believe we will be operating on far less staff in 2 years time.

[flagged]

[flagged]

Re: Gemini 2.5 Pro Preview

#528

Post by goatlover »

ArthurStacks wrote: post #518
auggierose wrote: post #516

Earlier quoted context omitted.

Hehe, have fun with that distributed system down the line.

Why? We fully checked the design, what he built, and it was fully tested over weeks for security and stability. Don't parrot what you read online that these systems are unable do this stuff. It's from the clueless or devs coping. Not only are they capable but theyre improving by the month.

I can't tell on this site who has genuinely experienced radical changes in software development from dedicated LLM usage, and who is trying to sell something. But given previous hype cycles with all exciting new tech at the time, including past iterations of AI, I tend to believe it's more in the trying to sell something camp.

Re: Gemini 2.5 Pro Preview

#530

Post by tmoravec »

MR4D wrote: post #444
sirstoke wrote: post #440

Earlier quoted context omitted.

I’ve been thinking about the SWE employment conundrum in a post-LLM world for a while now, and since my livelihood (and that of my loved ones’) depends on it, I’m obviously biased. Still, I would like to understand where my logic is flawed, if it is. (I.e I’m trying to argue in good faith here) Isn’t software engineering a lot more than just writing code? And I mean like, A LOT more? Informing product roadmaps, balan…

I think an analogy that is helpful is that of a woodworker. Automation just allowed them to do more things at in less time. Power saws really reduced time, lathes even more so. Power drills changed drilling immensely, and even nail guns are used on roofing project s because manual is way too slow. All the jobs still exist, but their tools are way more capable.

How many wood workers were there as a proportion of the population in the 1800s and now?
Post reply on HN