Meanwhile Gemini 2.5 Pro support for VSCode Copilot is still broken :/
https://github.com/microsoft/vscode-copilot-release/issues/8...
Gemini 2.5 Pro Preview
651–660 of 728 posts
Re: Gemini 2.5 Pro Preview
#652Earlier quoted context omitted.
> The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective. An obviously correct automatable objective function? Programming can be generally described as converting a human-defined specification (often very, very rough and loose) into a bunch of precise text files. Sure, you can use proxies like compilation success / failure and unit tests for RL. But key gaps rema…
This is in fact not how a chess engine works. It has an evaluation function that assigns a numerical value (score) based on a number of factors (material advantage, king "safety", pawn structure etc). These heuristics are certainly "good enough" that Stockfish is able to beat the strongest humans, but it's rarely possible for a chess engine to determine if a position results in mate. I guess the question is whether w…
Maybe I am misunderstanding what you are saying, but eg stockfish, given time and threads, seems very good at finding forced checkmates within 20 or more moves.
Re: Gemini 2.5 Pro Preview
#653Earlier quoted context omitted.
I can't point to any evidence. Also I can't think of what direct evidence I could present that would be convincing, short of an actual demonstration? I would like to try to justify my intuition though: Seems like the key question is: should we expect AI programming performance to scale well as more compute and specialised training is thrown at it? I don't see why not, it seems an almost ideal problem domain? * Short…
LLMs will still hit a ceiling without human-like reasoning. Even two weeks ago, Claude 3.7 made basic mistakes like trying to convince me the and >= operators on Python sets have the same semantics [1]. Any human would quickly reject something like that (why would be two different operators evaluate to the same value), unless there is overwhelming evidence. Mistakes like this show up all the time, which makes me beli…
I asked ChatGPT, Claude, Gemini and DeepSeek what the AE and OE mean in "Harman AE OE 2018 curve". All of them made up complete bullshit, even for the OE (Over Ear) term. AE is Around Ear. The OE term is absurdly easy to find even with the most basic of search skills, and is in fact the fourth result on Google.
The problem with LLMs isn't that they can't do great stuff: it's that you can't trust them to do it consistently. Which means you have to verify what they do, which means you need domain knowledge.
Until the next big evolution in LLMs or a revolution from something else, we'll be alright.
Re: Gemini 2.5 Pro Preview
#654Earlier quoted context omitted.
I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…
I can't point to any evidence. Also I can't think of what direct evidence I could present that would be convincing, short of an actual demonstration? I would like to try to justify my intuition though: Seems like the key question is: should we expect AI programming performance to scale well as more compute and specialised training is thrown at it? I don't see why not, it seems an almost ideal problem domain? * Short…
Thanks -- this is much more thoughtful than the persistent chorus of "just trust me, bro".
Re: Gemini 2.5 Pro Preview
#655Earlier quoted context omitted.
I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…
You're using them in reverse. They are perfect for generating code according to your architectural and code design templete. Relying on them for architectural design is like picking your nose with a pair of scissors - yeah technically doable, but one slip and it all goes to hell.
Im using them fine.
Im refuting the grandparent's point that they will replace basically all programming activities (including architecture) in 5 years.
Re: Gemini 2.5 Pro Preview
#656Earlier quoted context omitted.
I tried this as well. I'm interfacing with Gemini 2.5 using Cursor and I have rules to to limit the comments. It still ends up over-commenting.
I have a feeling this may be a cursor issue, perhaps cursors system prompt asks for comments? Asking in the aistudio UI for code and ending the prompt with "no code comments" has always worked for me
Re: Gemini 2.5 Pro Preview
#657Can someone tell me if windsurf is better than cursor? ( pref someone who has used both for a few days? )
Yes, I can. I am using Cursor extensively and have tried Visual Studio code with Roo, Cline, Copilot, Trae, Windsurf and Cursor. I believe Cursors UX to be vastly superior and I am getting the best results with Cursor + Gemini 2.5 Pro right now.
Thank you so much for your reply even tho i had a downvote for some unknown reason. All the best! I am now using cursor and gemini too.
Re: Gemini 2.5 Pro Preview
#658Earlier quoted context omitted.
I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…
I run a software development company with dozens of staff across multiple countries. Gemini has us to the point where we can actually stop hiring for certain roles and staff have been informed they must make use of these tools or they are surplus to requirements. At the current rate of improvement I believe we will be operating on far less staff in 2 years time.
Thanks -- this is what I mean by evidence, someone with actual experience and skin in the game weighing in rather than blustering proclamations based on vibes.
I agree they improve productivity to where you need fewer developers for a similar quantity of output than before. But I dont think LLMs specifically will reduce the need for some engineer to do the higher level technical design and architecture work, just given what Ive seen and my understanding of the underlying tech.
Re: Gemini 2.5 Pro Preview
#659Earlier quoted context omitted.
I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…
We're talking about predicting the future, so we can only extrapolate. Seeing the evidence you're thinking of would mean that LLMs will have solved software development by next month.
Im saying, lets see some actual reasoning behind the extrapolation rather than "just trust me bro" or "sama said this in a TED talk".
Many of the comments here and elsewhere have been in the latter categories.
Re: Gemini 2.5 Pro Preview
#660Earlier quoted context omitted.
LLMs will still hit a ceiling without human-like reasoning. Even two weeks ago, Claude 3.7 made basic mistakes like trying to convince me the and >= operators on Python sets have the same semantics [1]. Any human would quickly reject something like that (why would be two different operators evaluate to the same value), unless there is overwhelming evidence. Mistakes like this show up all the time, which makes me beli…
I asked ChatGPT, Claude, Gemini and DeepSeek what the AE and OE mean in "Harman AE OE 2018 curve". All of them made up complete bullshit, even for the OE (Over Ear) term. AE is Around Ear. The OE term is absurdly easy to find even with the most basic of search skills, and is in fact the fourth result on Google. The problem with LLMs isn't that they can't do great stuff: it's that you can't trust them to do it consist…
Both Gemini 2.5 Flash and Kagi's small built in model in their search got this right first try.