I’m always in awe of LLM code generation capabilities but it does make me sad, it’s not joyful and I feel terrible that I’m not learning anything.
Gemini 2.5 Pro Preview
661–670 of 728 posts
Re: Gemini 2.5 Pro Preview
#662Earlier quoted context omitted.
[flagged]
IMO this is completely "based". Delivering customer values and making money off of it is own thing, and software companies collectively being a social club and an place for R&D is another - technically a complete tangent to it. It doesn't always matter how sausages came to be on the served plate. It might be the Costco special that CEO got last week and dumped into the pot. It's none of your business to make sure tha…
Turning this into a moral discussion is besides the point, a point that both of you missed in your efforts to be based, although the moral discussion is also interesting—but I'll leave that be for now. It appears as if I stepped on ArthurStack's toes, but I'll give you the benefit of the doubt and reply.
My point actually has everything to do with making money. Making money is not a viable differentiator in and of itself. You need to put in work on your desired outcomes (or get lucky, or both) and the money might follow. My problem is that directives such as "software developers need to use tool x" is an _input_ with, at best, a questionable causal relationship to outcome y.
It's not about "social clubs for software developers", but about clueless execs. Now, it's quite possible that he's put in that work and that the outcomes are attributable to that specific input, but judging by his replies here I wouldn't wager on it. Also, as others have said, if that's the case, replicating their business model just got a whole lot easier.
> This is technically what everyone should aspire to be
No, there are other values besides maximizing utility.
Re: Gemini 2.5 Pro Preview
#663> We’ve seen developers doing amazing things with Gemini 2.5 Pro, so we decided to release an updated version a couple of weeks early to get into developers hands sooner. Today we’re excited to release Gemini 2.5 Pro Preview (I/O edition). What's up with AI companies and their model naming? So is this an updated 2.5 Pro and they indicate it by appending "Preview" to the name? Or was it always called 2.5 Preview and t…
Honestly they're iterating so fast that it looks like they simply gave up giving their models proper version names.
They should just use a date ¯\_(ツ)_/¯
Re: Gemini 2.5 Pro Preview
#664Earlier quoted context omitted.
I code with multiple LLMs every day and build products that use LLM tech under the hood. I dont think we're anywhere near LLMs being good at code design. Existing models make _tons_ of basic mistakes and require supervision even for relatively simple coding tasks in popular languages, and its worse for languages and frameworks that are less represented in public sources of training data. I am _frequently_ having to t…
The software tool takes a higher-level input to produce the executable. I'm waiting for LLMs to integrate directly into programming languages. The discussions sound a bit like the early days of when compilers started coming out, and people had been using direct assembler before. And then decades after, when people complained about compiler bugs and poor optimizers.
> I'm waiting for LLMs to integrate directly into programming languages.
What do you mean? How would this look like in your view?
Re: Gemini 2.5 Pro Preview
#665Earlier quoted context omitted.
This is my view. We've seen this before in other problems where there's an on-hand automatic verifier. The nature of the problem mirrors previously solved problems. The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective. I don't believe they can. Until they can, I'm going to assume that LLMs will soon be better than any living human at writing good code.
> The LLM skeptics need to point out what differs with code compared to Chess, DoTA, etc from a RL perspective. I see the burden of proof has been reversed. That’s stage 2 already of the hubris cycle. On a serious note, these are nothing alike. Games have a clear reward function. Software architecture is extremely difficult to even agree on basic principles. We regularly invalidate previous ”best advice”, and we have…
> I see the burden of proof has been reversed.
Isn't this just a pot calling the kettle black? I'm not sure why either side has the rightful position of "my opinion is right until you prove otherwise".
We're talking about predictions for the future, anyone claiming to be "right" is lacking humility. The only think going on is people justifying their opinions, no one can offer "proof".
Re: Gemini 2.5 Pro Preview
#666Earlier quoted context omitted.
Nope, just my opinion, derived from watching monthly and weekly exponential improvement over a few year period. I worked through at least two,AI winters since 1982, so current progress is good to see.
Exponential over which metric exactly? Training dataset size, compute required yeah these have grown exponentially. But has any measure capability? Because exponentially growing costs with linear or not measurable improvements is not a great trajectory.
Exponential in how useful LLM APIs and LLM based products like Google AI Lab, ChatGPT, etc. are to me personally. I am the data point I care about. I have a pet programming problem that every few months I try to solve with the current tools of the day. I admit this is anecdotal, just my personal experiences.
Metrics like training data set size are less interesting now given the utility of smaller synthetic data sets.
Once AI tech is more diffused to factory automation, robotics, educational systems, scientific discovery tools, etc., then we could measure efficiency gains.
My personal metric for the next 5 to 10 years: the US national debt and interest payments are perhaps increasing exponentially and since nothing will change politically to change this, exponential AI capability growth will either juice-up productivity enough to save us economically, or it won’t.
Re: Gemini 2.5 Pro Preview
#667Earlier quoted context omitted.
I asked ChatGPT, Claude, Gemini and DeepSeek what the AE and OE mean in "Harman AE OE 2018 curve". All of them made up complete bullshit, even for the OE (Over Ear) term. AE is Around Ear. The OE term is absurdly easy to find even with the most basic of search skills, and is in fact the fourth result on Google. The problem with LLMs isn't that they can't do great stuff: it's that you can't trust them to do it consist…
Both Gemini 2.5 Flash and Kagi's small built in model in their search got this right first try.
That is my point though. Gemini got it wrong for me. Which means it is inconsistent.
Say you and I ask Gemini what the perfect internal temperature for a medium-rare steak is. It tells me 72c, and it tells you 55c.
Even if it tells 990 people 55c and 10 people 55c, with a tens to hundreds of million users that is still a gargantuan amount of ruined steaks.
Re: Gemini 2.5 Pro Preview
#668I agree it's very good but the UI is still usually an unusable, scroll-jacking disaster. I've found it's best to let a chat sit for around a few minutes after it has finished printing the AI's output. Finding the `ms-code-block` element in dev tools and logging `$0.textContext` is reliable too.
I've noticed this has gotten a bit better lately, they have obviously been making a lot of UI changes to studio. But yeah, the scroll-jacking as response chunks are streamed in is incredibly frustrating since the model is pretty wordy.
I should add as well, on long complex threads the UI can become completely unusable. I'll hover over the tab and see it using over 2Gb of memory in chrome. Every so often I have to open a completely new tab, cut-n-paste the url and continue the conversation in that new tab (where the memory tends to drop back down to 600MB).
Re: Gemini 2.5 Pro Preview
#669Earlier quoted context omitted.
> It's entirely clear that every last human will be beaten on code design in the upcoming years LOLLLLL. You see a good one-shot demo and imagine an upward line, I work with LLM assistance every day and see... an asymptote (which is only budged by exponential power expenditure). As they say in sailing, you'll never win the race by following the guy in front of you... which is exactly what every single LLM does: Do a…
Sometimes LLMs are much better at obsequiously apologizing, making up post hoc rationalization blaming the user and tools, and writing up descriptions of how repeatedly terrible they are at following instructions, than actually following instructions after trying so many times. (This is the expensive Claude 3.7 Sonnet Max with thinking, mind you.) Just goes to show that management and executives like ArthurStacks are…
LOL, wow. Both to the Dilbert PHB-IRL "ArthurStacks" and to the LLM being so obsequious. At least it had candor, I guess? You want to say "Just stop filling your context window with apologetic analysis and do it correctly."
But yes. Sometimes it is so brilliant I smile (even if it's just copying a transliterated version of someone else's brilliance). Sometimes it is SO DUMB that I can't help but get frustrated.
In short, job security assured for the time being. If only because bosses and clients need someone to point at when the shit hits the fan.
Re: Gemini 2.5 Pro Preview
#670Earlier quoted context omitted.
[flagged]
[flagged]
What if I told you that a dev group with a sensibly-limited social-club flavor is where I arguably did my best and also had my happiest memories from? In the midst of SOME of the "socializing" (which, by the way, almost always STILL sticks to technical topics, even if they are merely adjacent to the task at hand) are brilliant ideas often born which sometimes end up contributing directly to bottom lines. Would you like evidence of social work cohesion leading to more productivity and happier employees? Because I can produce that. (I'd argue that remote work has negatively impacted this.)
But yes, I also once worked at a company (Factset) where the CTO had to put a stop to something that got out of hand- A very popular game at the time basically took over the mindshare of most of the devs for a time, and he caught them whiteboarding game strategies during work hours. (It was Starcraft 1 or 2, I forget. But both date me at this point.) So he put out a stern memo. Which did halt it. And yeah, he was right to do that.
Just do me this favor- If a dev comes to you with a wild idea that you think is too risky to spend a normal workday on, tell them they can use their weekend time to try it out. And if it ends up working, give them the equivalent days off (and maybe an extra, because it sucks to burn a weekend on work stuff, even if you care about the product or service). That way, the bet is hedged on both sides. And then maybe clap them on the back. And consider a little raise next review round. (If it doesn't work out, no extra days off, no harm no foul.)
I think your attitude is in line with your position (and likely your success). I get it. Slightly more warmth wouldn't hurt, though.