Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Talk Is Cheap: The Operational Impact of LLM Use

unessays.substack.com

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Talk Is Cheap: The Operational Impact of LLM Use

#11

Post by northstar702 »

sublinear wrote: post #5

More tasks get "done" while rework is sky high and overall throughput to production drops. First, I'd like to thank all the people working on testing and doing the lord's work. Anyway, this isn't even a unique pattern to LLM use. We've all seen this exact same thing when more devs are added a project running late, teams are siloed, outsourcing to contractors, etc.

Definitely not unique. Thats said, given how "easy" it is to scale LLM output, compared to human output, this pattern could be messier in the LLM era? The whole tokenmaxxing motion assumed more output equals more outcomes. What do you think?

Re: Talk Is Cheap: The Operational Impact of LLM Use

#13

Post by sublinear »

northstar702 wrote: post #11
sublinear wrote: post #5

More tasks get "done" while rework is sky high and overall throughput to production drops. First, I'd like to thank all the people working on testing and doing the lord's work. Anyway, this isn't even a unique pattern to LLM use. We've all seen this exact same thing when more devs are added a project running late, teams are siloed, outsourcing to contractors, etc.

Definitely not unique. Thats said, given how "easy" it is to scale LLM output, compared to human output, this pattern could be messier in the LLM era? The whole tokenmaxxing motion assumed more output equals more outcomes. What do you think?

Already covered in the post.

> So this is inherent to the technology. No amount of tokenmaxxing is going to change it. LLM development even breaks common and well accepted quality norms in software development - like backwards compatibility. You literally can’t (and wouldn’t want!) an LLM to do the same thing in the same way twice. But this means, LLMs - on their own - are not a solid foundation to build a revolution on. They never can be.

Re: Talk Is Cheap: The Operational Impact of LLM Use

#14

Post by northstar702 »

sublinear wrote: post #13
northstar702 wrote: post #11

Earlier quoted context omitted.

Definitely not unique. Thats said, given how "easy" it is to scale LLM output, compared to human output, this pattern could be messier in the LLM era? The whole tokenmaxxing motion assumed more output equals more outcomes. What do you think?

Already covered in the post. > So this is inherent to the technology. No amount of tokenmaxxing is going to change it. LLM development even breaks common and well accepted quality norms in software development - like backwards compatibility. You literally can’t (and wouldn’t want!) an LLM to do the same thing in the same way twice. But this means, LLMs - on their own - are not a solid foundation to build a revolution…

Fair. I suppose, the question behind my question is -- what else do we need. Thanks.

Re: Talk Is Cheap: The Operational Impact of LLM Use

#17

Post by bigstrat2003 »

deadbabe wrote: post #7

I'm curious, LLMs have been around for a while now... How many of you would say you need LLMs now for work? Not that you want it because it's nice to have, but rather you would literally not be able to do your job at all because you don't have an LLM to use? If your company said "We're not paying for LLMs anymore.", would you begrudgingly pay for or host your own LLM that complies with company policies, or just go ba…

> If your company said "We're not paying for LLMs anymore.", would you begrudgingly pay for or host your own LLM that complies with company policies, or just go back to writing everything by hand?

I already write code myself, so perhaps I'm not your target audience. But I strongly believe that anyone who pays for his own LLM is being a fool. You should never pay for tools for work. If you do so, you're letting them take advantage of you rather than paying the needs of their own business. It's a bad deal, don't go down that road.

Re: Talk Is Cheap: The Operational Impact of LLM Use

#18

Post by gtirloni »

ozlikethewizard wrote: post #6
gtirloni wrote: post #3

People are still figuring things out, there's a lot of wasted tokens, etc. This is like complaining a student isn't as productive as a senior engineering. I think we as an industry haven't even graduated to junior level when it comes to figuring our how to use AI to improve things.

This is discussed in the article, and I think the author makes pretty reasonable arguments for why by nature we will not see the reliability of LLM usage improve. They also discuss what I agree as the more effective method of using an LLM is, as a feedback and refinement tool, not a decision maker.

> This is not a limitation that can be overcome by LLMs. Their generative value is in their unreliability. If you turn temperature down to zero, you get a deterministic machine - but you also break every meaningful application I know of in production.

This is not a reasonable argument. Setting the temperature down to zero does NOT give you a deterministic machine. And I have never seen that break any application in production, quite the contrary.

Re: Talk Is Cheap: The Operational Impact of LLM Use

#19

Post by sublinear »

northstar702 wrote: post #14
sublinear wrote: post #13

Earlier quoted context omitted.

Already covered in the post. > So this is inherent to the technology. No amount of tokenmaxxing is going to change it. LLM development even breaks common and well accepted quality norms in software development - like backwards compatibility. You literally can’t (and wouldn’t want!) an LLM to do the same thing in the same way twice. But this means, LLMs - on their own - are not a solid foundation to build a revolution…

Fair. I suppose, the question behind my question is -- what else do we need. Thanks.

Yeah I have a different take. We will end up scaling human output by hiring more devs as wages for entry to mid level continue to stagnate.

I think LLMs make finding information and learning way more accessible. Even with realistic expectations it already is a revolution for literacy, education, and search. LLMs are a massive achievement that would be celebrated appropriately if only the public wasn't introduced to them during global political/economic crises encouraging grifters and authoritarians muddying the waters.

To be clear, hiring at scale is a sorely needed step. Software is just like any other form of writing. It carries weight and needs cultural and community context to work. We have needed way more technical literacy for decades to make this digital always-on world work. I think most would at least agree that LLMs bridge knowledge gaps. They're a net good thing despite the current abuse by bad actors.

Re: Talk Is Cheap: The Operational Impact of LLM Use

#20

Post by deadbabe »

bigstrat2003 wrote: post #17
deadbabe wrote: post #7

I'm curious, LLMs have been around for a while now... How many of you would say you need LLMs now for work? Not that you want it because it's nice to have, but rather you would literally not be able to do your job at all because you don't have an LLM to use? If your company said "We're not paying for LLMs anymore.", would you begrudgingly pay for or host your own LLM that complies with company policies, or just go ba…

> If your company said "We're not paying for LLMs anymore.", would you begrudgingly pay for or host your own LLM that complies with company policies, or just go back to writing everything by hand? I already write code myself, so perhaps I'm not your target audience. But I strongly believe that anyone who pays for his own LLM is being a fool. You should never pay for tools for work. If you do so, you're letting them t…

But you don’t need an LLM.
Post reply on HN