Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happens when people don't understand how AI works

theatlantic.com

Post reply on HN
Print view View original

101–110 of 359 posts

Re: What happens when people don't understand how AI works

#101

Post by lordnacho »

greg_V wrote: post #94
lordnacho wrote: post #76

The article skirts around a central question: what defines humans? Specifically, intelligence and emotions? The entire article is saying "it looks kinds like a human in some ways, but people are being fooled!" You can't really say that without at least attempting the admittedly very deep question of what an authentic human is. To me, it's intelligent because I can't distinguish its output from a person's output, for…

> Another thing that I hear from time to time is an argument along the line of "it just predicts the next word, it doesn't actually understand it". Rather than an argument against AI being intelligent, isn't this also telling us what "understanding" is? Before we all had computers, how did people judge whether another person understood something? Well, they would ask the person something and the person would respond.…

Hmm, I was actually thinking of a viva situation. You sit with a panel of experts, they talk to you, they decide whether you passed your PhD in philosophy/history/physics/etc.

That process is done purely by language, but we supposed that inside you there is something deeper than a token prediction machine.

Re: What happens when people don't understand how AI works

#102

Post by zzbzq »

kelseyfrog wrote: post #54
krisoft wrote: post #50

Earlier quoted context omitted.

This sounds very wise but doesn’t seem to describe any of my use cases. Maybe some use cases are divination but it is a stretch to call all of them that. Just looking at my recent AI prompts: I was looking for the name of the small fibers which form a bird’s feather. ChatGPT told me they are called “barbs”. Then using straight forward google search i could verify that indeed that is the name of the thing i was lookin…

Sure, some people stepped up to the Oracle and asked how to conquer Persia. Others probably asked where they left their sandals. The quality of the question doesn't change the structure of the act. You presented clear, factual queries. Great. But even there, all the components are still in play: you asked a question into a black box, received a symbolic-seeming response, evaluated its truth post hoc, and interpreted…

The LLMs do have "latent knowledge," indisputably, the latent knowledge is beyond reproach. Because what we do know about the "black box" is that inside it, is a database of not just facts, but understanding, and we know the model "understands" nearly every topic better than any human. Where the doubt-worthy part happens is the generative step, since it is tasked with producing a new "understanding" that didn't already exist, the mathematical domain of the generative function exceeds the domain of reality. And, second of all, because the reasoning faculties are far less proven than the understanding faculties, and many queries require reasoning about existing understandings to derive a good, new one.

Re: What happens when people don't understand how AI works

#103

Post by suddenlybananas »

tim333 wrote: post #97
kelseyfrog wrote: post #7

LLMs are divinatory instruments, our era's oracle, minus the incense and theatrics. If we were honest, we'd admit that "artificial intelligence" is just a modern gloss on a very old instinct: to consult a higher-order text generator and search for wisdom in the obscure. They tick all the boxes: oblique meaning, a semiotic field, the illusion of hidden knowledge, and a ritual interface. The only reason we don't call i…

Maybe LLMs can be divinatory instruments but that sounds a bit highbrow going by my use. I use it more as a better Google search. Like the most recent thing I said to ChatGPT is "will clothianidin kill carpet beetles?" (turns out it does by the way.)

Until it makes stuff up.

Re: What happens when people don't understand how AI works

#104

Post by JKCalhoun »

andy99 wrote: post #18

I agree with the substance, but would argue the author fails to "understand how AI works" in an important way: LLMs are impressive probability gadgets that have been fed nearly the entire internet, and produce writing not by thinking but by making statistically informed guesses about which lexical item is likely to follow another Modern chat-tuned LLMs are not simply statistical models trained on web scale datasets.…

Many like the author fail to convince me because they never also explain how human minds work. They just wave their hand, look off to a corner of the ceiling with, "But of course that's not how humans think at all," as if we all just know that.

Re: What happens when people don't understand how AI works

#105

Post by datadrivenangel »

zzbzq wrote: post #102
kelseyfrog wrote: post #54

Earlier quoted context omitted.

Sure, some people stepped up to the Oracle and asked how to conquer Persia. Others probably asked where they left their sandals. The quality of the question doesn't change the structure of the act. You presented clear, factual queries. Great. But even there, all the components are still in play: you asked a question into a black box, received a symbolic-seeming response, evaluated its truth post hoc, and interpreted…

The LLMs do have "latent knowledge," indisputably, the latent knowledge is beyond reproach. Because what we do know about the "black box" is that inside it, is a database of not just facts, but understanding, and we know the model "understands" nearly every topic better than any human. Where the doubt-worthy part happens is the generative step, since it is tasked with producing a new "understanding" that didn't alrea…

LLMs have latent knowledge insofar as it can be distilled out of the internet...

Re: What happens when people don't understand how AI works

#106

Post by clejack »

Are people still experiencing llms getting stuck in knowledge and comprehension loops? I used them but not excessively, and I'm not heavily tracking their performance either.

For example, if you ask an llm a question, and it produces a hallucination then you try to correct it or explain to it that it is incorrect; and it produces a near identical hallucination while implying that it has produced a new, correct result, this suggests that it does not understand its own understanding (or pseudo-understanding if you like).

Without this level of introspection, directing any notion of true understanding, intelligence, or anything similar seems premature.

Llms need to be able to consistently and accurately say, some variation on the phrase "I don't know," or "I'm uncertain." This indicates knowledge of self. It's like a mirror test for minds.

Re: What happens when people don't understand how AI works

#107

Post by JKCalhoun »

MrZander wrote: post #53
daveguy wrote: post #32

Earlier quoted context omitted.

The transformers are accurately described in the article. The confusion comes in the Reinforcement Learning Human Feedback (RLHF) process after a transformer based system is trained. These are algorithms on top of the basic model that make additional discriminations of the next word (or phrase) to follow based on human feedback. It's really just a layer that makes these models sound "better" to humans. And it's a gre…

Oh, interesting, TIL. Didn't realize there was a second step to training these models.

Got 3½ hours? https://youtu.be/7xTGNNLPyMI

(I watched it all, piecemeal, over the course of a week, ha, ha.)

Re: What happens when people don't understand how AI works

#108

Post by martindbp »

Many people who claim that people don't understand how AI works often have a very simplified view of the short comings of LLMs themselves, e.g. "it's just predicting the next token", "it's just statistics", "stochastic parrot" and seems to be grounded in what AI was 2-3 years ago. Rarely have they actually read the recent research on interpretability. It's clear LLMs are doing more than just pattern matching. They may not think like humans or as well, but it's not k-NN with interpolation.

Re: What happens when people don't understand how AI works

#109

Post by tim333 »

suddenlybananas wrote: post #103
tim333 wrote: post #97

Earlier quoted context omitted.

Maybe LLMs can be divinatory instruments but that sounds a bit highbrow going by my use. I use it more as a better Google search. Like the most recent thing I said to ChatGPT is "will clothianidin kill carpet beetles?" (turns out it does by the way.)

Until it makes stuff up.

Well nothing's perfect. It actually got that one wrong - it said "not typically effective" which isn't true.

Re: What happens when people don't understand how AI works

#110

Post by rhubarbtree »

jdkee wrote: post #2

Someone said, "The AI you use today is the worst AI that you will ever use."

If there is one cliche that I’d like to outlaw, it is this one. It’s a vacuous tautology and yet I hear so many people parrot it all the time.

There are many reasons to believe LLMs in particular are not going anywhere fast.

We need major breakthroughs now, and “chain of thought” is not one.

Post reply on HN