Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

Post reply on HN
Print view View original

621–630 of 728 posts

Re: Gemini 2.5 Pro Preview

#621

Post by klabb3 »

energy123 wrote: post #517
xyzzy123 wrote: post #493

Earlier quoted context omitted.

I can't point to any evidence. Also I can't think of what direct evidence I could present that would be convincing, short of an actual demonstration? I would like to try to justify my intuition though: Seems like the key question is: should we expect AI programming performance to scale well as more compute and specialised training is thrown at it? I don't see why not, it seems an almost ideal problem domain? * Short…

This is my view. We've seen this before in other problems where there's an on-hand automatic verifier. The nature of the problem mirrors previously solved problems. The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective. I don't believe they can. Until they can, I'm going to assume that LLMs will soon be better than any living human at writing good code.

> The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective.

I see the burden of proof has been reversed. That’s stage 2 already of the hubris cycle.

On a serious note, these are nothing alike. Games have a clear reward function. Software architecture is extremely difficult to even agree on basic principles. We regularly invalidate previous ”best advice”, and we have many conflicting goals. Tradeoffs are a thing.

Secondly programming has negative requirements that aren’t verifiable. Security is the perfect example. You don’t make a crypto library with unit tests.

Third, you have the spec problem. What is the correct logic in edge cases? That can be verified but needs to be decided. Also a massive space of subtle decisions.

Re: Gemini 2.5 Pro Preview

#622

Post by seidleroni »

I'm not sure if this is just me, but with the "Starter Apps" I don't see how you can extend them using AI in aistudio. For example, there doesn't seem to be a way to add more code to the app with AI, even if you copy the Starter App. Am I missing something, or is this just a big miss from Google?

Re: Gemini 2.5 Pro Preview

#623

Post by taf2 »

throwup238 wrote: post #278
taf2 wrote: post #38

Earlier quoted context omitted.

I really liked the Gemini 2.5 pro model when it was first released - the upload code folder was very nice (but they removed it). The annoying things I find with the model is it does a really bad job of formatting the code it generates... I know I can use a code formatting tool and I do when i use gemini output but otherwise I find grok much easier to work with and yields better results.

> I really liked the Gemini 2.5 pro model when it was first released - the upload code folder was very nice (but they removed it). Removed from where? I use the attach code folder feature every day from the Gemini web app (with a script that clones a local repo that deletes .git and anything matching a gitignore pattern).

Maybe I got stuck with a bad experiment that removed it but it has been gone for me for a few weeks so I just stopped using it

Re: Gemini 2.5 Pro Preview

#624

Post by concats »

sirstoke wrote: post #440
jstummbillig wrote: post #314

Earlier quoted context omitted.

> no amount of prompting will get current models to approach abstraction and architecture the way a person does I find this sentiment increasingly worrisome. It's entirely clear that every last human will be beaten on code design in the upcoming years (I am not going to argue if it's 1 or 5 years away, who cares?) I wished people would just stop holding on to what amounts to nothing, and think and talk more about wha…

I’ve been thinking about the SWE employment conundrum in a post-LLM world for a while now, and since my livelihood (and that of my loved ones’) depends on it, I’m obviously biased. Still, I would like to understand where my logic is flawed, if it is. (I.e I’m trying to argue in good faith here) Isn’t software engineering a lot more than just writing code? And I mean like, A LOT more? Informing product roadmaps, balan…

The way I see it:

* The world is increasingly ran on computers.

* Software/Computer Engineers are the only people who actually truly know how computers work.

Thus it seems to me highly unlikely that we won't have a job.

What that job entails I do not know. Programming like we do today might not be something that we spend a considerable amount of time doing in the future. Just like most people today don't spend much time handing punched-cards or replacing vacuum tubes. But there will still be other work to do, I don't doubt that.

Re: Gemini 2.5 Pro Preview

#625

Post by piokoch »

thecupisblue wrote: post #581
ssalazar wrote: post #454

Earlier quoted context omitted.

I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…

You're using them in reverse. They are perfect for generating code according to your architectural and code design templete. Relying on them for architectural design is like picking your nose with a pair of scissors - yeah technically doable, but one slip and it all goes to hell.

Well, I have asked LLM to fix some piece of Python Django code so it uses pagination for the list of entities. And LLM came up with the working solution, impressively complicated piece of Django ORM code, which was totally needles, as Django ORM has Paginator class that does all the job without manual fetching pages, etc.

LLM sees pagination, it does pagination. After all LLM is an algorithm that calculates probability of the next word in a sequence of words, nothing less and nothing more. LLM does not think or feel, even though people believe in this saying thank you and using polite words like "please". LLM generates text on the base of what it was presented. That's why it will happily invent research that does not exist, create a review of a product that does not exist, invent a method that does not exist in a given programming language. And so on.

Re: Gemini 2.5 Pro Preview

#626

Post by mbil »

concats wrote: post #620
jstummbillig wrote: post #314

Earlier quoted context omitted.

> no amount of prompting will get current models to approach abstraction and architecture the way a person does I find this sentiment increasingly worrisome. It's entirely clear that every last human will be beaten on code design in the upcoming years (I am not going to argue if it's 1 or 5 years away, who cares?) I wished people would just stop holding on to what amounts to nothing, and think and talk more about wha…

I won't deny that in a context with perfect information, a future LLM will most likely produce flawless code. I too believe that is inevitable. However, in real life work situations, that 'perfect information' prerequisite will be a big hurdle I think. Design can depend on any number of vague agreements and lots of domain specific knowledge, things a senior software architect has only learnt because they've been at t…

I agree that there’s a lot of ambiguity and tacit information that goes into building code. I wonder if that won’t change directly as a result of wanting to get more value out of agentic AI coders.

> Sure, if you write down a summary of each and every meeting you've attended for the past 12 months, as well as attach your entire company confluence, into the prompt, perhaps then the LLM can design the right architecture. But is that realistic?

I think it is definitely realistic. Zoom and Confluence already have AI integrations. To me it doesn’t seem long before these tools and more become more deeply MCPified, with their data and interfaces made available to the next generation of AI coders. “I’m going to implement function X with this specific approach based on your conversation with Bob last week.”

It strikes me that remote first companies may be at an advantage here as they’re already likely to have written artifacts of decisions and conversations, which can then provide more context to AI assistants.

Re: Gemini 2.5 Pro Preview

#627

Post by Hauthorn »

AnIrishDuck wrote: post #521
energy123 wrote: post #517

Earlier quoted context omitted.

This is my view. We've seen this before in other problems where there's an on-hand automatic verifier. The nature of the problem mirrors previously solved problems. The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective. I don't believe they can. Until they can, I'm going to assume that LLMs will soon be better than any living human at writing good code.

> The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective. An obviously correct automatable objective function? Programming can be generally described as converting a human-defined specification (often very, very rough and loose) into a bunch of precise text files. Sure, you can use proxies like compilation success / failure and unit tests for RL. But key gaps rema…

This is in fact not how a chess engine works. It has an evaluation function that assigns a numerical value (score) based on a number of factors (material advantage, king "safety", pawn structure etc).

These heuristics are certainly "good enough" that Stockfish is able to beat the strongest humans, but it's rarely possible for a chess engine to determine if a position results in mate.

I guess the question is whether we can write a good enough objective function that would encapsulate all the relevant attributes of "good code".

Re: Gemini 2.5 Pro Preview

#628

Post by pjmlp »

lallysingh wrote: post #617
ssalazar wrote: post #454

Earlier quoted context omitted.

I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…

The software tool takes a higher-level input to produce the executable. I'm waiting for LLMs to integrate directly into programming languages. The discussions sound a bit like the early days of when compilers started coming out, and people had been using direct assembler before. And then decades after, when people complained about compiler bugs and poor optimizers.

Exactly, I also see code generation to current languages as output only an intermediary step, like we had to have those -S switches, or equivalent, to convince developers during the first decades of compiler existence, until optmizing compilers took over.

"Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning"

https://arxiv.org/html/2311.13721v3

Re: Gemini 2.5 Pro Preview

#629

Post by polyaniline »

mgw wrote: post #72
puika wrote: post #57

Earlier quoted context omitted.

I have the same issue plus unnecessary refactorings (that break functionality). it doesn't matter if I write a whole paragraph in the chat or the prompt explaining I don't want it to change anything else apart from what is required to fulfill my very specific request. It will just go rogue and massacre the entirety of the file.

This has also been my biggest gripe with Gemini 2.5 Pro. While it is fantastic at one-shotting major new features, when wanting to make smaller iterative changes, it always does big refactors at the same time. I haven't found a way to change that behavior through changes in my prompts. Claude 3.7 Sonnet is much more restrained and does smaller changes.

Asking it explicitly once (not necessarily every new prompt in context) to keep output minimal and strive to do nothing more than it is told works for me.
Post reply on HN