Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything around LLMs is still magical and wishful thinking

dmitriid.com

Post reply on HN
Print view View original

151–160 of 377 posts

Re: Everything around LLMs is still magical and wishful thinking

#151

Post by leptons »

standardUser wrote: post #146

> Like most skeptics and critics, I use these tools daily. And 50% of the time they work 50% of the time. I use LLMs nearly every day for my job as of about a year ago and they solve my issues about 90% of the time. I have a very hard time deciphering if these types of complaints about AI/LLMs should be taken seriously, or written off as irrational use patterns by some users. For example, I have never fed an LLM a co…

Your comment is no better than the comment in the article that the author is calling out.

"90%" also seems a bit suspect.

Re: Everything around LLMs is still magical and wishful thinking

#152

Post by jorl17 »

TeMPOraL wrote: post #130
tasty_freeze wrote: post #18

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

Maybe it's due to a more R&D-ish nature of my current work, but for me, LLMs are delivering just as much gains in the "thinking" part as in "coding" part (I handle the "communicating" thing myself just fine for now). Using LLMs for "thinking" tasks feels similar to how mastering web search 2+ decades ago felt. Search engines enabled access to information provided you know what you're looking for; now LLMs boost that…

This, and if you add in a voice mode (e.g. ChatGPT's Advanced Mode), it is perfect for brainstorming.

Once I decide I want to "think a problem through with an LLM", I often start with just the voice mode. This forces me to say things out loud — which is remarkably effective (hear hear rubber duck debugging) — and it also gives me a fundamentally different way of consuming the information the LLM provides me. Instead of being delivered a massive amount of text, where some information could be wrong, I instead get a sequential system where I can stop/pause the LLM/redirect it as soon as something gets me curious or as I find problems with it said.

You would think that having this way of interacting would be limiting, as having a fast LLM output large chunks of information would let you skim through it and commit it to memory faster. Yet, for me, the combination of hearing things and, most of all, not having to consume so much potentially wrong info (what good is it to skim pointless stuff), ensures that ChatGPT's Advanced Voice mode is a great way to initially approach a problem.

After the first round with the voice mode is done, I often move to written-form brainstorming.

Re: Everything around LLMs is still magical and wishful thinking

#153

Post by kgeist »

DebtDeflation wrote: post #148
hx8 wrote: post #145

Earlier quoted context omitted.

I think you are underestimating the negative impacts that overhype cause. It's distorting the market, causing over investment, preemptively slashing departments, and creating an expectation that will never be meet. These articles are important for cooling expectations. When people sell LLMs, they usually aren't talking about summarizing customer support calls, they are trying to sell the idea of firing customer suppo…

If anything, the LLM overhype is starting to die down........to make way for the AI Agent hype which is on trajectory to be 1000X worse. People are writing articles and making videos about how AI Agents will replace SaaS. What?

The OP compares the current LLM hype to crypto, but I think it's more fair to compare it to the dotcom bubble. When a new, interesting technology appears, there's always a lot of hype around it - I think it's natural. People are still figuring out what works and what doesn't. Naturally, some overoptimistic people overhype it. The dotcom bubble burst; nonetheless, the internet is now an integral part of our lives. Despite the hype, crypto was always a very niche area, while currently even my grandmother uses ChatGPT - just like the internet.

Re: Everything around LLMs is still magical and wishful thinking

#154

Post by gabrieledarrigo »

abletonlive wrote: post #125
tasty_freeze wrote: post #18

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

I’m a tech lead and I have maybe 5X output now compared to everybody else under me. Quantified by scoring tickets at a team level. I also have more responsibilities outside of IC work compared to the people under me. At this point I’m asking my manager to fire people that still think llms are just toys because I’m tired of working with people with this poor mindset. A pragmatic engineer continually reevaluates what t…

Dude, if you are a tech lead, and you measure productivity by scoring tickets, you are doing it pretty badly. I would fire you instead.

Re: Everything around LLMs is still magical and wishful thinking

#155

Post by rester324 »

hotpotat wrote: post #30

I have to say I’m in the exact camp the author is complaining about. I’ve shipped non trivial greenfield products which I started back when it was only ChatGPT and it was shitty. I started using Claude with copying and pasting back and forth between the web chat and XCode. Then I discovered Cursor. It left me with a lot of annoying build errors, but my productivity was still at least 3x. Now that agents are better an…

But you just confirmed everything the blogpost claimed.

You didn't share any evidence with us even though you claim unbelievable things.

You even went as far as registering a throwavay account to hide your identity and to make verifying any of your claims impossible.

Your comment feels more like a joke to me

Re: Everything around LLMs is still magical and wishful thinking

#156

Post by gabrieledarrigo »

hotpotat wrote: post #30

I have to say I’m in the exact camp the author is complaining about. I’ve shipped non trivial greenfield products which I started back when it was only ChatGPT and it was shitty. I started using Claude with copying and pasting back and forth between the web chat and XCode. Then I discovered Cursor. It left me with a lot of annoying build errors, but my productivity was still at least 3x. Now that agents are better an…

> I started a job at a demanding startup and it’s been several months and I have still not written a single line of code by hand

Damn, this sounds pretty boring.

Re: Everything around LLMs is still magical and wishful thinking

#157

Post by thunky »

tasty_freeze wrote: post #18

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

> One thing I find frustrating is that management where I work has heard of 10x productivity gains.

That may also be in part because llms are not as big of an accelerant for junior devs as they are for seniors (juniors don't know what is good and bad as well).

So if you give 1 senior dev a souped up llm workflow I wouldn't be too surprised if they are as productive as 10 pre-llm juniors. Maybe even more, because a bad dev can actually produce negative productivity (stealing from the senior), in which case it's infinityx.

Even a decent junior is mostly limited to doing the low level grunt work, which llms can already do better.

Point is, I can see how jobs could be lost, legitimately.

Re: Everything around LLMs is still magical and wishful thinking

#158

Post by richardw »

leptons wrote: post #151
standardUser wrote: post #146

> Like most skeptics and critics, I use these tools daily. And 50% of the time they work 50% of the time. I use LLMs nearly every day for my job as of about a year ago and they solve my issues about 90% of the time. I have a very hard time deciphering if these types of complaints about AI/LLMs should be taken seriously, or written off as irrational use patterns by some users. For example, I have never fed an LLM a co…

Your comment is no better than the comment in the article that the author is calling out. "90%" also seems a bit suspect.

I just went through the last 10 chat titles and all of them were spot on for me. Maybe the person you’re responding to has a different experience than you do and calling their perspective “suspect” is somewhat uncharitable.

(There are times I do other kinds of work and it fails terribly. My main point stands.)

Re: Everything around LLMs is still magical and wishful thinking

#159

Post by swader999 »

estomagordo wrote: post #116
smcleod wrote: post #86

Earlier quoted context omitted.

That's about right for copilots. It's much higher for agentic coding.

[citation needed]

Agentic coding had really only taken off in the last few weeks due to better pricing.

Re: Everything around LLMs is still magical and wishful thinking

#160

Post by jorl17 »

timr wrote: post #110
mlinsey wrote: post #97

Earlier quoted context omitted.

I don't disagree with your assessment of the world today, but just 12 months ago (before the current crop of base models and coding agents like Claude Code), even that 10X improvement of writing some-of-the-code wouldn't have been true.

> I don't disagree with your assessment of the world today, but just 12 months ago (before the current crop of base models and coding agents like Claude Code), even that 10X improvement of writing some-of-the-code wouldn't have been true. So? It sounds like you're prodding us to make an extrapolation fallacy (I don't even grant the "10x in 12 months" point, but let's just accept the premise for the sake of argument).…

12 months ago, if I fed a list of ~800 poems with about ~250k tokens to an LLM and asked it to summarize this huge collection, they would be completely blind to some poems and were prone to hallucinating not simply verses but full-blown poems. I was testing this with every available model out there that could accept 250k tokens. It just wouldn't work. I also experimented with a subset that was at around ~100k tokens to try other models and results were also pretty terrible. Completely unreliable and nothing it said could be trusted.

Then Gemini 2.5 pro (the first one) came along and suddenly this was no longer the case. Nothing hallucinated, incredible pattern finding within the poems, identification of different "poetic stages", and many other rather unbelievable things — at least to me.

After that, I realized I could start sending in more of those "hard to track down" bugs to Gemini 2.5 pro than other models. It was actually starting to solve them reliably, whereas before it was mostly me doing the solving and models mostly helped if the bug didn't occur as a consequence of very complex interactions spread over multiple methods. It's not like I say "this is broken, fix it" very often! Usually I include my ideas for where the problem might be. But Gemini 2.5 pro just knows how to use these ideas better.

I have also experimented with LLMs consuming conversations, screenshots, and all kinds of ad-hoc documentation (e-mails, summaries, chat logs, etc) to produce accurate PRDs and even full-on development estimates. The first one that actually started to give good results (as in: it is now a part of my process) was, you guessed it, Gemini 2.5 pro. I'll admit I haven't tried o3 or o4-mini-high too much on this, but that's because they're SLOOOOOOOOW. And, when I did try, o4-mini-high was inferior and o3 felt somewhat closer to 2.5 pro, though, like I said, much much slower and...how do I put this....rude ("colder")?

All this to say: while I agree that perhaps the models don't feel like they're particularly better at some tasks which involve coding, I think 2.5 pro has represented a monumental step forward, not just in coding, but definitely overall (the poetry example, to this day, still completely blows my mind. It is still so good it's unbelievable).

Post reply on HN