Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happens when people don't understand how AI works

theatlantic.com

Post reply on HN
Print view View original

301–310 of 359 posts

Re: What happens when people don't understand how AI works

#301

Post by jibal »

more_corn wrote: post #260
spacemadness wrote: post #211

Earlier quoted context omitted.

Or saying they’re close to AGI because LLM behavior is indistinguishable from thinking to them. Especially here on HN I see “what’s the difference?” arguments all the time. It looks like it to me so it must be it. QED.

Having studied those things I can say that from their perspective “what’s the difference?” is an entirely legitimate question. Boldly asserting that what LLMS do is not cognition is even worse than asserting that it is. (If you dig deep into how they do what they do we find functional differences, but the outcome are equivalent) The butlerian view is actually a great place to start. He asserts that when we solve a pr…

"the outcome are equivalent"

Talk about a "bold assertion".

Re: What happens when people don't understand how AI works

#302

Post by QuantumGood »

UncleOxidant wrote: post #231
kelseyfrog wrote: post #7

LLMs are divinatory instruments, our era's oracle, minus the incense and theatrics. If we were honest, we'd admit that "artificial intelligence" is just a modern gloss on a very old instinct: to consult a higher-order text generator and search for wisdom in the obscure. They tick all the boxes: oblique meaning, a semiotic field, the illusion of hidden knowledge, and a ritual interface. The only reason we don't call i…

You're getting some pushback about the analogy to divination, but I think most people here are reasonably technically literate and they assume that everyone else in society has the same understanding of how LLMs work that they do. When I chat about LLM usage with non-technical friends and family it does indeed seem as though they're using these AI chatbots as oracles. When I suggest that they should be wary because t…

I only recommend Perplexity to non-technical users looking for a news or general information interpreter. Others can search the web, but seem not do use search as their primary source.

Re: What happens when people don't understand how AI works

#303

Post by jemiluv8 »

Most people without any idea about the foundations on which LLMs are built call them AI. But I insist on calling them LLMs, further creating confusion. How do you explain what a large language model is to someone that can't comprehend how a machine can learn a "word model" on a large corpus of text/data to make it generate "seemingly sound/humane" responses without making them feel like they are interacting with the AI that they've been hearing about in the movies/sci-fi?

Re: What happens when people don't understand how AI works

#304

Post by dotancohen »

UncleOxidant wrote: post #231
kelseyfrog wrote: post #7

LLMs are divinatory instruments, our era's oracle, minus the incense and theatrics. If we were honest, we'd admit that "artificial intelligence" is just a modern gloss on a very old instinct: to consult a higher-order text generator and search for wisdom in the obscure. They tick all the boxes: oblique meaning, a semiotic field, the illusion of hidden knowledge, and a ritual interface. The only reason we don't call i…

You're getting some pushback about the analogy to divination, but I think most people here are reasonably technically literate and they assume that everyone else in society has the same understanding of how LLMs work that they do. When I chat about LLM usage with non-technical friends and family it does indeed seem as though they're using these AI chatbots as oracles. When I suggest that they should be wary because t…

I often phrase it something along these lines: "They are designed to return grammatically valid sentences, not factually correct sentences. If they return something that looks like a fact, either that was in their training data or they made it up to return a grammatically valid sentence. Either way, double check."

Of course, nobody listens anyway.

Re: What happens when people don't understand how AI works

#305

Post by jemiluv8 »

Even I have limited understanding of how LLMs learn the semantic meaning of words. My knowledge is shallow at best. I know however that LLMs understand text now. Are able to understand concepts they "glean" from text and are able to give responses to queries that is not entirely made up. All these makes it a lot harder to explain to non-technical people what this is. I tell them these LLMs are not AI but when they go to these websites - they see it labelled as an AI chatbot. It also mostly does as advertised. And they are often in awe of whatever responses they tend to receive because they are not subject matter experts nor do they care to become one. They just want to get their "homeworks" done, complete their work assignments and this gets them there faster. How can I tell them it is not AI when it spews humane looking text. Heck, even I don't quite understand the "real" difference between LLMs and AI. The difference is nuanced but the line is clearer with a bit of technical understanding. The machine understands text. And can make conversation - however sycophantic. But without understanding why that is - I don't see why we won't exult its powers. I see religions sprouting from these soon. LLMs can deliver awesome sermons. And once you train them well enough, can take on the role of Messiah's.

Re: What happens when people don't understand how AI works

#306

Post by UncleOxidant »

dotancohen wrote: post #304
UncleOxidant wrote: post #231

Earlier quoted context omitted.

You're getting some pushback about the analogy to divination, but I think most people here are reasonably technically literate and they assume that everyone else in society has the same understanding of how LLMs work that they do. When I chat about LLM usage with non-technical friends and family it does indeed seem as though they're using these AI chatbots as oracles. When I suggest that they should be wary because t…

I often phrase it something along these lines: "They are designed to return grammatically valid sentences, not factually correct sentences. If they return something that looks like a fact, either that was in their training data or they made it up to return a grammatically valid sentence. Either way, double check." Of course, nobody listens anyway.

I know someone who is the accountant for a smallish company. He mentioned to me that he was using chatGPT like a spreadsheet. I was like, no you definitely don't want to do that.

Re: What happens when people don't understand how AI works

#307

Post by jemiluv8 »

more_corn wrote: post #260
spacemadness wrote: post #211

Earlier quoted context omitted.

Or saying they’re close to AGI because LLM behavior is indistinguishable from thinking to them. Especially here on HN I see “what’s the difference?” arguments all the time. It looks like it to me so it must be it. QED.

Having studied those things I can say that from their perspective “what’s the difference?” is an entirely legitimate question. Boldly asserting that what LLMS do is not cognition is even worse than asserting that it is. (If you dig deep into how they do what they do we find functional differences, but the outcome are equivalent) The butlerian view is actually a great place to start. He asserts that when we solve a pr…

> Boldly asserting that what LLMS do is not cognition is even worse than asserting that it is.

That's the issue I was driving at. The machine is so convincing. How can we say what it does is not "thinking" when it seems to be breaking down a query like a human does. The distinction between what an AI is and what an LLM is - is so thin that most of us will be ignorant and combine the two because you really need to see what is under the hood before you understand that the responses you're getting are from a "model" - not some sentient thinking machine.

But what does it matter if it is from a "model" that understands text? It still produces more or less what other humans produce. Most of us won't care about the difference.

Re: What happens when people don't understand how AI works

#308

Post by ufmace »

SnowflakeOnIce wrote: post #237
ufmace wrote: post #232

Earlier quoted context omitted.

I've tended to agree with this line of argument, but on the other hand... I expect that anybody you asked 10 years ago who was at least decently knowledgeable about tech and AI would have agreed that the Turing Test is a pretty decent way to determine if we have a "real" AI, that's actually "thinking" and is on the road to AGI etc. Well, the current generation of LLMs blow away that Turing Test. So, what now? Were we…

> Well, the current generation of LLMs blow away that Turing Test Maybe a weak version of Turing's test? Passing the stronger one (from Turing's paper "Computing Machinery and Intelligence") involves an "average interrogator" being unable to distinguish between human and computer after 5 minutes of questioning more than 70% of the time. I've not seen this result published with today's LLMs.

I'll admit that I was not familiar with the strong version of it. But I am still surprised that nobody has done that. Has nobody even seriously attempted to see how LLMS do at that? Now I might just have to check for myself.

I would have presumed it would be a cake walk. Depending of course on exactly how we define "average interrogator". I would think if we gave a LLM enough pre-prepping to pretend it was a human, and the interrogator was not particularly familiar with ways of "jailbreaking" LLMs, they could pass the test.

Re: What happens when people don't understand how AI works

#309

Post by acdha »

codethief wrote: post #241
acdha wrote: post #234

Earlier quoted context omitted.

Her problem is that BMBL is down 92% and they need to tell investors that they’ll all be rich again: https://finance.yahoo.com/quote/BMBL/ Most of the dumb AI pitches share that basic goal: someone is starting from what investors want to be true and using “AI” like it’s a magic spell which can make that possible, just as we’ve seen going back to the dawn of the web. Sober voices don't get attention because it’s borin…

> Her problem is that BMBL is down 92% and they need to tell investors that they’ll all be rich again Is this also why Bumble has undergone so many drastic changes in recent times? I always thought they must hired some new & overzealous product managers that didn't actually understand the secret sauce that had made their product so successful in the first place. Either way, it seems the usual enshittification has beg…

I don’t have any inside knowledge but that’d be my working theory for anything like that: some PM has been told that their job depends on making a number go from X to 2X by the end of the year.

Re: What happens when people don't understand how AI works

#310

Post by soraminazuki »

selfhoster11 wrote: post #130
thoroughburro wrote: post #118

Earlier quoted context omitted.

Trusting LLM advice about poisons seems… sort of like being a test pilot for a brand new aerospace company with no reputation for safety.

Only if you don't check it against a classical search query later. Not to mention that all you might get is search results from slop farms that aren't concerned with safety or factuality - ones that were a problem before LLMs.

So you agree that you need to do thorough research and be careful about which sources you trust. At that point, why not jump straight to actual research and bypass the information laundering machine entirely? Even you acknowledge that you can't trust what it says.
Post reply on HN