Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs aren't remotely like compilers or power tools

blainehansen.me

Post reply on HN
Print view View original

31–40 of 55 posts

Re: LLMs aren't remotely like compilers or power tools

#31

Post by bluefirebrand »

throw310822 wrote: post #19
suprjami wrote: post #12

You had me up until you equated LLM usage to unreliable human delegation. No human being is going to genuinely suggest to glue cheese onto a pizza to stop it sliding off. These things aren't people. Stop anthropomorphizing a computer program. They'll make all the mistakes humans can make, as well as a new class of mistakes because they're just fancy next word prediction with no lived experience.

> No human being is going to genuinely suggest to glue cheese onto a pizza to stop it sliding off. That mistake was made more than two years ago by whatever experimental version of Google's AI overview (which has strict performance requirement- i.e. needs to answer within a split-second) was up at the time. In other words, it was a very small and primitive system specialised in spitting answers as quickly as possible…

It's a clear illustration that these systems are not oracles. No matter how much they've improved since then, we should assume they are still capable of making hilariously big mistakes. Only now the obvious mistakes are fixed and the big mistakes will be a lot more subtle and a lot harder to catch.

Re: LLMs aren't remotely like compilers or power tools

#32

Post by yifanl »

david-gpu wrote: post #24
pennomi wrote: post #17

Earlier quoted context omitted.

The very existence of a “do anything” tool causes humans to forgo learning the multitude of tools we already have. I suspect there will be no next generation of software engineers.

The existence of a "do anything" IDE causes humans to forgo learning the multitude of tools we already have. I suspect there will be no next generation of software engineers. People said stuff like that in the 90s. I was there.

There's actually zero difference between a bottle of water and a river reservoir if you think about it.

Re: LLMs aren't remotely like compilers or power tools

#33

Post by vivzkestrel »

camkego wrote: post #5

I've got to give the author credit, it seems like this is written without an LLM. But, there seems to be a lot that is arguable. For example, the article says "Natural language is the interface of last resort." But, is that actually true? Isn't the AI boom happening because N/L is truly available as an interface? Anyway, please keep writing and asking tough questions.

natural language will never be 100% deterministic

Re: LLMs aren't remotely like compilers or power tools

#34

Post by inigyou »

smcg wrote: post #27
inigyou wrote: post #22

Earlier quoted context omitted.

Evidence from listening to slopcoders speak says they feel it's night and day. Evidence from observing reality shows a 20% speed increase at best and 50% speed decrease at worst and almost always a significant reduction in quality.

Twitter is a rather crude place, but it can be succinct. An exchange I saw recently: A: "I spent more tokens than the entire OpenCode team combined." B: "Why aren't you more successful?"

Telling it like it is. I bet Sam got Musk to restrict user B.

Re: LLMs aren't remotely like compilers or power tools

#35

Post by inigyou »

throw310822 wrote: post #28
inigyou wrote: post #20

Earlier quoted context omitted.

Apparently if you show one the recently found counterexample to the Jacobian conjecture, it spins in circles double-checking it because it knows the conjecture is true.

And finally gives up admitting it is a counterexample. Even saying "holy shit". That's what any human expert would do: presented with an extremely improbable fact, it would verify it, then in disbelief verify it again, then change method, then try yet another one, checking its understanding of the problem itself... and finally concede. Because this is what intelligence looks like- understanding the problem, its relev…

I think Haiku still does this with the seahorse emoji. Doesn't feel like it's finding contradictory evidence, though, just feels like it's trying both options in case one works.

Re: LLMs aren't remotely like compilers or power tools

#36

Post by throw310822 »

bluefirebrand wrote: post #31
throw310822 wrote: post #19

Earlier quoted context omitted.

> No human being is going to genuinely suggest to glue cheese onto a pizza to stop it sliding off. That mistake was made more than two years ago by whatever experimental version of Google's AI overview (which has strict performance requirement- i.e. needs to answer within a split-second) was up at the time. In other words, it was a very small and primitive system specialised in spitting answers as quickly as possible…

It's a clear illustration that these systems are not oracles. No matter how much they've improved since then, we should assume they are still capable of making hilariously big mistakes. Only now the obvious mistakes are fixed and the big mistakes will be a lot more subtle and a lot harder to catch.

> It's a clear illustration that these systems are not oracles.

Of course they are not oracles. Oracles belong to magic, these things are real.

I really can't understand how people seem to expect "intelligence" to be deterministic and infallible and uniform across domains when all examples we have of it (that is, us) are anything but.

Re: LLMs aren't remotely like compilers or power tools

#37

Post by david-gpu »

yifanl wrote: post #32
david-gpu wrote: post #24

Earlier quoted context omitted.

The existence of a "do anything" IDE causes humans to forgo learning the multitude of tools we already have. I suspect there will be no next generation of software engineers. People said stuff like that in the 90s. I was there.

There's actually zero difference between a bottle of water and a river reservoir if you think about it.

I am just saying that people have been making these sorts of shallow dismissive comments about people who use different tooling since forever. I remember a college professor snided at students who didn't know how to use vi.

In the same way that most people are no longer proficient in assembly, but it has not spelled doom for software engineering, some folks are now using AI tools, and it will probably be fine. The day to day work flow of a typical programmer will probably be different ten or twenty years from now, and we will all move on, as usual.

Re: LLMs aren't remotely like compilers or power tools

#38

Post by jaredcwhite »

throw310822 wrote: post #19
suprjami wrote: post #12

You had me up until you equated LLM usage to unreliable human delegation. No human being is going to genuinely suggest to glue cheese onto a pizza to stop it sliding off. These things aren't people. Stop anthropomorphizing a computer program. They'll make all the mistakes humans can make, as well as a new class of mistakes because they're just fancy next word prediction with no lived experience.

> No human being is going to genuinely suggest to glue cheese onto a pizza to stop it sliding off. That mistake was made more than two years ago by whatever experimental version of Google's AI overview (which has strict performance requirement- i.e. needs to answer within a split-second) was up at the time. In other words, it was a very small and primitive system specialised in spitting answers as quickly as possible…

Nice try with the gaslighting. Google's AI "summaries" get things wrong all the time even now.

Re: LLMs aren't remotely like compilers or power tools

#39

Post by Jemaclus »

inigyou wrote: post #22
the_real_cher wrote: post #15

This person is taking things too literally. By power tool we simply mean speeds up work. And by that definition LLMs are absolutely a power tool of epic proprtions! If you ever coded before LLMs existed its like night and day.

Evidence from listening to slopcoders speak says they feel it's night and day. Evidence from observing reality shows a 20% speed increase at best and 50% speed decrease at worst and almost always a significant reduction in quality.

I was also skeptical, but I started measuring quality (I'm using rework as a rough metric) and quantity (PR throughput), and I've found that, on average, LLMs are 60% worse at writing code (meaning they rework at a rate 60% higher than humans) but my team produce PRs about 2.4x faster. So objectively my team's numbers don't match up with yours: our speed is about 2.5x increase _on average_ and a 60% reduction _on average_ in quality. Some basic math suggests that our ROI is something around 50-60% improvement over just humans.

I'm not convinced that throwing more money (i.e. tokens) at the problem would make us any more efficient, but 1.5x improvement ain't nothing.

And just to address the "night and day" bit.. if you gave a construction foreman a machine that made his team 20% more productive, he'd take that all day long and tell everyone it's "night and day." I'm not sure what your threshold is, but +20% isn't nothing either. That foreman would hit more deadlines than not, and get more bonuses than not.

Re: LLMs aren't remotely like compilers or power tools

#40

Post by yifanl »

david-gpu wrote: post #37
yifanl wrote: post #32

Earlier quoted context omitted.

There's actually zero difference between a bottle of water and a river reservoir if you think about it.

I am just saying that people have been making these sorts of shallow dismissive comments about people who use different tooling since forever. I remember a college professor snided at students who didn't know how to use vi. In the same way that most people are no longer proficient in assembly, but it has not spelled doom for software engineering, some folks are now using AI tools, and it will probably be fine. The da…

Its true that things change, but it's also true that there will eventually be a day when the last programmer is hired.
Post reply on HN