Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

newyorker.com

Post reply on HN
Print view View original

141–150 of 164 posts

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#141

Post by jacobgold »

SpicyLemonZest wrote: post #139
jacobgold wrote: post #132

Earlier quoted context omitted.

What I'm pushing back on is, apparently, that some people genuinely believe these LLM-based computer programs are human-like intelligences. In reality, they're more like very good search engines that output relevant snippets of text. If you run them in a loop (feeding them their output as input) you can make them return even better search results. The software developers who created these systems used sexy words like…

It seems to me that they used words like these because the LLM-based computer programs they sell can solve problems which humans apply reasoning and thinking to solve. Why do you think it's more than that?

In the past, when people built programs to extract text from PDFs, they didn't wrap them in a chat interface which claimed it had the human cognitive ability to "read" human languages.

They could have done this. They could have claimed they'd recreated human vision and hyped it as the beginning of a full human brain, but they didn't.

Instead, they used real technical terms like "OCR" (optical character recognition), which gave people a much more accurate understanding of the technology and didn't encourage silly analogies to humans.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#143

Post by jacobgold »

vidarh wrote: post #136
jacobgold wrote: post #130

Earlier quoted context omitted.

Yes, people speak multiple languages and switch between them. But it's a superficial analogy to the behavior of LLMs which do something different and for different reasons.

You're misrepresenting what I wrote. I specifically pointed out that I switch languages without intent to do so. When you suggest that is a "superficial analogy" after you were the one pointing out LLMs switching language as something that sets them apart, you're seriously reaching. I can often pinpoint afterward what was likely the trigger: E.g. I used a word that is the same in two languages, and continue in the se…

That's a fair point. I agree that the behavior can look similar even when the mechanism is different.

But in practice, all the analogies I've seen are in fact superficial, including this one.

The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#144

Post by orangecat »

jacobgold wrote: post #132
Kim_Bruning wrote: post #116

Earlier quoted context omitted.

> That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful. Oh, ok then. That does change things a bit. The impression I'm getting is that you were suggesting they're not. What's succinctly the thing you're objecting to? Is it Anthropomorphization? I mean, sure, but watch out : when defending on that axis, it's easy to slip into…

What I'm pushing back on is, apparently, that some people genuinely believe these LLM-based computer programs are human-like intelligences. In reality, they're more like very good search engines that output relevant snippets of text. If you run them in a loop (feeding them their output as input) you can make them return even better search results. The software developers who created these systems used sexy words like…

some people genuinely believe these LLM-based computer programs are human-like intelligences

"If LLMs were human-like intelligences they would do X, but they don't". What is X?

In reality, they're more like very good search engines that output relevant snippets of text.

What are the "relevant snippets" that contained the solutions for the unit distance and Jacobian conjectures?

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#145

Post by jacobgold »

vidarh wrote: post #137
jacobgold wrote: post #131

Earlier quoted context omitted.

> So just like me, including the prompt injections if you count "nerd sniping" as such? Who would count that as prompt injection? It's a superficial analogy. If you were vulnerable to prompt injection, I could order you to do absolutely anything you're capable of doing and you would be helpless to do otherwise.

Not nearly all prompt injections are by any means that absolute unless starting from the exact same state. Many of them will also work only probabilistically unless you turn temperature to 0 for exactly that reason. And at the same time, whole books have been written about how reliably we can induce certain behaviours from humans. E.g. the Blue-seven phenomenon [1] - I've personally experienced that second hand and i…

Anyone is free to draw whatever analogies they want, but they either make sense or they don't. Outside of philosophical discussions or science fiction, comparing influencing humans to prompt injection is silly.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#146

Post by TeMPOraL »

jacobgold wrote: post #143
vidarh wrote: post #136

Earlier quoted context omitted.

You're misrepresenting what I wrote. I specifically pointed out that I switch languages without intent to do so. When you suggest that is a "superficial analogy" after you were the one pointing out LLMs switching language as something that sets them apart, you're seriously reaching. I can often pinpoint afterward what was likely the trigger: E.g. I used a word that is the same in two languages, and continue in the se…

That's a fair point. I agree that the behavior can look similar even when the mechanism is different. But in practice, all the analogies I've seen are in fact superficial, including this one. The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.

What makes you think the similarity is the superficial part, and not the difference in mechanism? I'd argue it's the latter.

> The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.

Haven't met many kids, I see. Or even normie adults talking. I know plenty that tend to jump from topic to topic once they get into a stride talking, and they're not the ones diagnosed with ADHD.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#147

Post by TeMPOraL »

jacobgold wrote: post #131
TeMPOraL wrote: post #115

Earlier quoted context omitted.

So just like me, including the prompt injections if you count "nerd sniping" as such? (And in particular, switching languages on the fly is normal for people who speak more than one well, it's something you learn not to do for the sake of people less comfortable with the languages involved.)

> So just like me, including the prompt injections if you count "nerd sniping" as such? Who would count that as prompt injection? It's a superficial analogy. If you were vulnerable to prompt injection, I could order you to do absolutely anything you're capable of doing and you would be helpless to do otherwise.

> If you were vulnerable to prompt injection, I could order you to do absolutely anything you're capable of doing and you would be helpless to do otherwise.

Yes. Depending on specificity and timescales involved, we call that "reading comprehension" or "social engineering" or "peer pressure" or "motivating literature" or "advertising" or "propaganda" or "religion".

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#148

Post by TeMPOraL »

jacobgold wrote: post #132
Kim_Bruning wrote: post #116

Earlier quoted context omitted.

> That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful. Oh, ok then. That does change things a bit. The impression I'm getting is that you were suggesting they're not. What's succinctly the thing you're objecting to? Is it Anthropomorphization? I mean, sure, but watch out : when defending on that axis, it's easy to slip into…

What I'm pushing back on is, apparently, that some people genuinely believe these LLM-based computer programs are human-like intelligences. In reality, they're more like very good search engines that output relevant snippets of text. If you run them in a loop (feeding them their output as input) you can make them return even better search results. The software developers who created these systems used sexy words like…

That's the thing - they are more human-like intelligence than search engines. I'm gonna push back on your push-back here strongly. I'm not saying they are intelligent - but they're much more like human-like intelligences than like any kind of classical software systems, which is why it makes more sense to talk and think about them in these terms than as software system.

To do the opposite invites confused thinking like considering "lethal trifecta" a solvable programming problem.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#149

Post by Kim_Bruning »

jacobgold wrote: post #143
vidarh wrote: post #136

Earlier quoted context omitted.

You're misrepresenting what I wrote. I specifically pointed out that I switch languages without intent to do so. When you suggest that is a "superficial analogy" after you were the one pointing out LLMs switching language as something that sets them apart, you're seriously reaching. I can often pinpoint afterward what was likely the trigger: E.g. I used a word that is the same in two languages, and continue in the se…

That's a fair point. I agree that the behavior can look similar even when the mechanism is different. But in practice, all the analogies I've seen are in fact superficial, including this one. The LLM that abruptly switches languages will also likely switch to a wildly unrelated topic. If a human behaved that way, you'd call a doctor.

It's called "code-switching" or "code-mixing", and bilinguals do it all the time. When an immigrant kid does it, you don't call a doctor, you call it adorable.

By the way, it's not switching topic. You just pick the concept closest to what you mean from your combined vocabulary. If you're not paying close attention, you might switch language though (until the next concept you need is from the other language again, at which point you switch back)

And you're aware the paper "Attention is all you need" came out of machine translation research at Google, right? You hold an internal semantic representation and map in and out from arbitrary natural languages. I think the (bi-, tri-, multi-)lingual approach is the only proper way to translate, and this is a hill I will fight on!

Google may have gotten more than they bargained for on that particular translation experiment; though they failed to capitalize on it initially, with OpenAI running with the ball.

Re: Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

#150

Post by vidarh »

jacobgold wrote: post #145
vidarh wrote: post #137

Earlier quoted context omitted.

Not nearly all prompt injections are by any means that absolute unless starting from the exact same state. Many of them will also work only probabilistically unless you turn temperature to 0 for exactly that reason. And at the same time, whole books have been written about how reliably we can induce certain behaviours from humans. E.g. the Blue-seven phenomenon [1] - I've personally experienced that second hand and i…

Anyone is free to draw whatever analogies they want, but they either make sense or they don't. Outside of philosophical discussions or science fiction, comparing influencing humans to prompt injection is silly.

This is very much a philosophical question, and the only reason you're calling it silly instead of giving an actual argument is that it doesn't support your views.
Post reply on HN