AI is a floor raiser, not a ceiling raiser • Hacker News Board

Re: AI is a floor raiser, not a ceiling raiser

#31

Post by smiley1437 » Thu, Jul 31, 2025, 7:33 PM UTC

LeftHandPath wrote: post #27

There are some things that you still can't do with LLMs. For example, if you tried to learn chess by having the LLM play against you, you'd quickly find that it isn't able to track a series of moves for very long (usually 5-10 turns; the longest I've seen it last was 18) before it starts making illegal choices. It also generally accepts invalid moves from your side, so you'll never be corrected if you're wrong about…

> people aren't aware of how wrong they can be, and the errors take effort and knowledge to notice.

I have friends who are highly educated professionals (PhDs, MDs) who just assume that AI\LLMs make no mistakes.

They were shocked that it's possible for hallucinations to occur. I wonder if there's a halo effect where the perfect grammar, structure, and confidence of LLM output causes some users to assume expertise?

Re: AI is a floor raiser, not a ceiling raiser

#32

Post by bluefirebrand » Thu, Jul 31, 2025, 7:34 PM UTC

AI is not a floor raiser

It is a false confidence generator

Re: AI is a floor raiser, not a ceiling raiser

#33

Post by falcor84 » Thu, Jul 31, 2025, 7:37 PM UTC

I agree with most of TFA but not this:

> This means cheaters will plateau at whatever level the AI can provide

From my experience, the skill of using AI effectively is of treating the AI with a "growth mindset" rather than a "fixed" one. What I do is that I roleplay as the AI's manager, giving it a task, and as long as I know enough to tell whether its output is "good enough", I can lend it some of my metagcognition via prompting to get it to continue working through obstacles until I'm happy with the result.

There are diminishing returns of course, but I found that I can get significantly better quality output than what it gave me initially without having to learn the "how" of the skill myself (i.e. I'm still "cheating"), and only focusing my learning on the boundary of what is hard about the task. By doing this, I feel that over time I become a better manager in that domain, without having to spend the amount of effort to become a practitioner myself.

Re: AI is a floor raiser, not a ceiling raiser

#34

Post by anthk » Thu, Jul 31, 2025, 7:38 PM UTC

recroad wrote: post #11

AI isn't a pit. AI is a ladder.

Yeah, like in Nethack, while being blind and stepping on a cockatrice.

Re: AI is a floor raiser, not a ceiling raiser

#35

Post by LeftHandPath » Thu, Jul 31, 2025, 7:38 PM UTC

TimPC wrote: post #20

People should be worried because right now AI is on an exponential growth trajectory and no-one knows when it will level off into an s-curve. AI is starting to get close to good enough. If it becomes twice as good in seven months then what?

I was worried about that a couple of years ago, when there was a lot of hope that deeper reasoning skills and hallucination avoidance would simply arrive as emergent properties of a large enough model.

More recently, it seems like that's not the case. Larger models sometimes even hallucinate more [0]. I think the entire sector is suffering from a Dunning Kruger effect -- making an LLM is difficult, and they managed to get something incredible working in a much shorter timeframe than anyone really expected back in the early 2010s. But that led to overconfidence and hype, and I think there will be a much longer tail in terms of future improvements than the industry would like to admit.

Even the more advanced reasoning models will struggle to play a valid game of chess, much less win one, despite having plenty of chess games in their training data [1]. I think that, combined with the trouble of hallucinations, hints at where the limitations of the technology really are.

Hopefully LLMs will scare society into planning how to handle mass automation of thinking and logic, before a more powerful technology that can really do it arrives.

[0]: https://techcrunch.com/2025/04/18/openais-new-reasoning-ai-m...

[1]: https://dev.to/maximsaplin/can-llms-play-chess-ive-tested-13...

Re: AI is a floor raiser, not a ceiling raiser

#36

Post by falcor84 » Thu, Jul 31, 2025, 7:40 PM UTC

stillpointlab wrote: post #29

This mirrors insights from Andrew Ng's recent AI startup talk [1]. I recall he mentions in this video that the new advice they are giving to founders is to throw away prototypes when they pivot instead of building onto a core foundation. This is because of the effects described in the article. He also gives some provisional numbers (see the section "Rapid Prototyping and Engineering" and slides ~10:30) where he sugge…

Great point, but just mentioning (nitpicking?) that I never heard about machines/containers referred to as "livestock", but rather in my milieu it's always "pets" vs "cattle". I now wonder if it's a geographical thing.

Re: AI is a floor raiser, not a ceiling raiser

#37

Post by fellowniusmonk » Thu, Jul 31, 2025, 7:41 PM UTC

The greatest use of LLMs is the ability to get accurate answers to queries in a normalized format without having to wade through UI distraction like ads and social media.

It's the opposite of finding an answer on reddit, insta, tvtropes.

I can't wait for the first distraction free OS that is a thinking and imagination helper and not a consumption device where I have to block urls on my router so my kids don't get sucked into a skinners box.

I love being able to get answers from documentation and work questions without having to wade through some arbitrary UI bs a designer has implemented in adhoc fashion.

Re: AI is a floor raiser, not a ceiling raiser

#38

Post by falcor84 » Thu, Jul 31, 2025, 7:41 PM UTC

itsoktocry wrote: post #12

andrenotgiant wrote: post #8

This tracks for other areas of AI I am more familiar with. Below average people can use AI to get average results.

That explains why people here are against it, because everyone is above average I guess.

I'm not against it. I wonder where in the distribution it puts me.

Re: AI is a floor raiser, not a ceiling raiser

#39

Post by spion » Thu, Jul 31, 2025, 7:43 PM UTC

manmal wrote: post #14

Since agents are good only at greenfield projects, the logical conclusion is that existing codebases have to be prepared such that new features are (opinionated) greenfield projects - let all the wiring dangle out of the wall so the intern just has to plug in the appliance. All the rest has to be done by humans, or the intern will rip open the wall to hang a picture.

I think agents have a curve where they're kinda bad at bootstrapping a project, very good if used in a small-to-medium-sized existing project and then it goes downhill from there as size increases, slowly.

Something about a brand-new project often makes LLMs drop to "example grade" code, the kind you'd never put in production. (An example: claude implemented per-task file logging in my prototype project by pushing to an array of log lines, serializing the entire thing to JSON and rewriting the entire file, for every logged event)

Re: AI is a floor raiser, not a ceiling raiser

#40

Post by HPsquared » Thu, Jul 31, 2025, 7:43 PM UTC

falcor84 wrote: post #36

stillpointlab wrote: post #29

This mirrors insights from Andrew Ng's recent AI startup talk [1]. I recall he mentions in this video that the new advice they are giving to founders is to throw away prototypes when they pivot instead of building onto a core foundation. This is because of the effects described in the article. He also gives some provisional numbers (see the section "Rapid Prototyping and Engineering" and slides ~10:30) where he sugge…

Great point, but just mentioning (nitpicking?) that I never heard about machines/containers referred to as "livestock", but rather in my milieu it's always "pets" vs "cattle". I now wonder if it's a geographical thing.

Boxen? (Oxen)