Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything around LLMs is still magical and wishful thinking

dmitriid.com

Post reply on HN
Print view View original

121–130 of 377 posts

Re: Everything around LLMs is still magical and wishful thinking

#121

Post by mlinsey »

timr wrote: post #110
mlinsey wrote: post #97

Earlier quoted context omitted.

I don't disagree with your assessment of the world today, but just 12 months ago (before the current crop of base models and coding agents like Claude Code), even that 10X improvement of writing some-of-the-code wouldn't have been true.

> I don't disagree with your assessment of the world today, but just 12 months ago (before the current crop of base models and coding agents like Claude Code), even that 10X improvement of writing some-of-the-code wouldn't have been true. So? It sounds like you're prodding us to make an extrapolation fallacy (I don't even grant the "10x in 12 months" point, but let's just accept the premise for the sake of argument).…

12 months ago, we had no reasoning models and even very basic arithmetic was outside of the models' grasp. Coding assistants mostly worked on the level of tab-completing individual functions, but now I can one-shot demo-able prototypes (albeit nothing production-ready) of webapps. I assume you consider the latter "integration", but I think coding is so key to how the base models are being trained that this is due to base model improvements too. This is testable - it would be interesting to get something like Claude Code running on top of a year-old open source model and see how it does.

If you're going to call all of that not substantial improvement, we'll have to agree to disagree. Certainly it's the most rapid rate of improvement of any tech I've personally seen since I started programming in the early '00s.

Re: Everything around LLMs is still magical and wishful thinking

#122

Post by coolKid721 »

tasty_freeze wrote: post #18

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

On my personal projects it's easily 10x faster if not more in some circumstances. At work where things are planned out months in advanced and I'm working with 5 different teams to figure out the right way to do things for requirements that change 8 times during development? Even just stuff with PR review and making sure other people understand it and can access it. idk sometimes it's probably break even or that 10-15%. It just doesn't work well in some environments and what really makes it flourish (having super high quality architectural planning/designs/standardized patterns etc.) is basically just not viable at anything but the smallest startups and solo projects.

Frankly even just getting engineers to agree upon those super specificized standardized patterns is asking a ton, especially since lots of the things that help AI out are not what they are used to. As soon as you have stuff that starts deviating it can confuse the AI and makes that 10x no longer accessible. Also no one would want to review the PRs I'd make for the changes I do on my "10x" local project... Especially maintaining those standards is already hard enough on my side projects AI will naturally deviate and create noise and the challenge is constructing systems to guide that to make sure nothing deviates (since noise would lead to more noise).

I think it's mostly a rebalancing thing, if you have 1 or a couple like minded engineers who intend to do it they can get that 10x. I do not see that EVER existing in any actual corporate environment or even once you get more then like 4 people tbh.

Ai for middle management and project planning on the other hand...

Re: Everything around LLMs is still magical and wishful thinking

#123

Post by timr »

mlinsey wrote: post #121
timr wrote: post #110

Earlier quoted context omitted.

> I don't disagree with your assessment of the world today, but just 12 months ago (before the current crop of base models and coding agents like Claude Code), even that 10X improvement of writing some-of-the-code wouldn't have been true. So? It sounds like you're prodding us to make an extrapolation fallacy (I don't even grant the "10x in 12 months" point, but let's just accept the premise for the sake of argument).…

12 months ago, we had no reasoning models and even very basic arithmetic was outside of the models' grasp. Coding assistants mostly worked on the level of tab-completing individual functions, but now I can one-shot demo-able prototypes (albeit nothing production-ready) of webapps. I assume you consider the latter "integration", but I think coding is so key to how the base models are being trained that this is due to…

I consider the reasoning models to be primarily a development of efficiency/cost, and I thought the first one was about a year ago, but sure, ok. I don’t think it changes the argument I’m making. The LLM ourobouros / robot centipede has been done, and is not itself a path towards exponential improvement.

To be quite honest, I’ve found very little marginal value in using reasoning models for coding. Tool usage, sure, but I almost never use “reasoning” beyond that.

Also, LLMs still cannot do basic math. They can solve math exams, sure, but you can’t trust them to do a calculation in the middle of a task.

Re: Everything around LLMs is still magical and wishful thinking

#124

Post by timr »

ssk42 wrote: post #119
timr wrote: post #110

Earlier quoted context omitted.

> I don't disagree with your assessment of the world today, but just 12 months ago (before the current crop of base models and coding agents like Claude Code), even that 10X improvement of writing some-of-the-code wouldn't have been true. So? It sounds like you're prodding us to make an extrapolation fallacy (I don't even grant the "10x in 12 months" point, but let's just accept the premise for the sake of argument).…

What exactly are you basing any of your assertions off of?

The same sort of rigorous analysis that the parent comment used (that’s a joke, btw).

But seriously: If you find yourself agreeing with one and not the other because of sourcing, check your biases.

Re: Everything around LLMs is still magical and wishful thinking

#125

Post by abletonlive »

tasty_freeze wrote: post #18

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

I’m a tech lead and I have maybe 5X output now compared to everybody else under me. Quantified by scoring tickets at a team level. I also have more responsibilities outside of IC work compared to the people under me. At this point I’m asking my manager to fire people that still think llms are just toys because I’m tired of working with people with this poor mindset. A pragmatic engineer continually reevaluates what they think they know. We are at a tipping point now. I’m done arguing with people that have a poor model of reality. The rest of us are trying to compete and get shit done. This isn’t an opinion or a game. It’s business with real life consequences if you fall behind. I’ve offered to share my workflows, prompts, setup. Guess how many of these engineers have taken me up on my offer. 1-2 and the juniors or ones that are very far behind have not.

Re: Everything around LLMs is still magical and wishful thinking

#126

Post by g-b-r »

CharlesW wrote: post #100
98eb1d0ff7fb96 wrote: post #90

Earlier quoted context omitted.

See, your comment is a good example of what's going wrong. The OP specifically mentioned "mission critical things" - My interpretation of that would be things that are not allowed to break, because otherwise people might die, in the worst case - and you were talking about just SOMETHING that got "done" faster. No mention about anything critical. Of course, I was playing around with claude code, too, and I was fascina…

> So it kinda worked, but I would not use that for anything "mission critical" (whatever this means). It means projects like Cloudflare's new OAuth provider library. https://github.com/cloudflare/workers-oauth-provider > "This library (including the schema documentation) was largely written with the help of Claude, the AI model by Anthropic. Claude's output was thoroughly reviewed by Cloudflare engineers with careful…

https://neilmadden.blog/2025/06/06/a-look-at-cloudflares-ai-...

Re: Everything around LLMs is still magical and wishful thinking

#127

Post by yyammka »

rdgthree wrote: post #28
Arainach wrote: post #16

Earlier quoted context omitted.

Amen. At my job right now there is an imminent threat from a team empowered to say "what if we asked an AI to just build X instead of having a team build and maintain it?" X is something where it's straightforward when N is below 50 but deeply complex when N is in the thousands, which for our team it is, and there is a huge risk that this team will get a demo with N=15 that attracts leadership attention and trying to…

FWIW the comment you are responding to was authored by AI.

two em-dashes!

Re: Everything around LLMs is still magical and wishful thinking

#128

Post by xoralkindi »

hotpotat wrote: post #30

I have to say I’m in the exact camp the author is complaining about. I’ve shipped non trivial greenfield products which I started back when it was only ChatGPT and it was shitty. I started using Claude with copying and pasting back and forth between the web chat and XCode. Then I discovered Cursor. It left me with a lot of annoying build errors, but my productivity was still at least 3x. Now that agents are better an…

> I audit everything myself before making PRs and test rigorously

How do you audit code from an untrusted source that quickly, LLMs do not have the whole project in their heads and are proned to hallucinate.

On average how long are your prompts and does the LLM also write the unit tests?

Re: Everything around LLMs is still magical and wishful thinking

#129

Post by dgfitz »

abletonlive wrote: post #125
tasty_freeze wrote: post #18

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

I’m a tech lead and I have maybe 5X output now compared to everybody else under me. Quantified by scoring tickets at a team level. I also have more responsibilities outside of IC work compared to the people under me. At this point I’m asking my manager to fire people that still think llms are just toys because I’m tired of working with people with this poor mindset. A pragmatic engineer continually reevaluates what t…

I will thank God every day I don’t work with you or for you. How toxic.

Re: Everything around LLMs is still magical and wishful thinking

#130

Post by TeMPOraL »

tasty_freeze wrote: post #18

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

Maybe it's due to a more R&D-ish nature of my current work, but for me, LLMs are delivering just as much gains in the "thinking" part as in "coding" part (I handle the "communicating" thing myself just fine for now). Using LLMs for "thinking" tasks feels similar to how mastering web search 2+ decades ago felt. Search engines enabled access to information provided you know what you're looking for; now LLMs boost that by helping you figure out what you're looking for in the first place (and then conveniently searching it for you, too). This makes trivial some tasks I previously classified as hard due to effort and uncertainty involved.

At this point I'd say about 1/3 of my web searches are done through ChatGPT o3, and I can't imagine giving it up now.

(There's also a whole psychological angle in how having LLM help sort and rubber-duck your half-baked thought makes many task seem much less daunting, and that alone makes a big difference.)

Post reply on HN