Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

371–380 of 385 posts

Re: What can LLMs never do?

#371

Post by joshspankit »

patching-trowel wrote: post #295
joshspankit wrote: post #290

As I was reading, this voice got louder and louder: Would LLMs cross this threshold if we were able to train them only on works that are “objectively good”? if someone has better language than this, please enlighten me) That is to say: coherent, empathetic, transparent, free from bias, substantiated, free from “fluff”. For example: For science one cannot simply train from all works published in scientific journals be…

My gut says no because of the way language relates to meaning. In language, a “chair” is a chair is a chair. But in meaning, a chair is not-a-stool, and not-a-couch, and not-a-bench etc. We understand the object largely by what the object is similar to but not. In order for the LLM to meaningfully model what is coherent, empathetic, free from bias, it must also model the close to, but NOT-that.

That’s a compelling point.

If you’ll indulge me I’m going to think out loud a little.

What makes sense to me about this point:

- Having zero knowledge of “non-good” could lead to fragility when people phrase questions in “non-good” ways

- If an LLM is truly a “I do what I learned” machine, then “good” input + “good” question would output “good” output

- There may be a significant need for an LLM to learn the “chair is not-a-stool” aka “fact is not-a-fiction”. An LLM that only gets affirming meanings might be wildly confused. If true I think that would be a an interesting area to research not just for AI but for cognition. … now I wonder how many of the existing params are “not”s.

- There’s also the question of scale. Does an LLM need to “know” about mass extinction in order to understand empathy? Or can it just know about the emotions people experience during hard times? Children seem to do fine at empathy (maybe even better than adults in some ways) despite never being exposed to planet-sized tragedies. Adults need to deal with bigger issues where it can be important to have those tragedies front of mind, but does an LLM need to?

Re: What can LLMs never do?

#372

Post by tavern1991 »

krainboltgreene wrote: post #200
int_19h wrote: post #185

Earlier quoted context omitted.

LLMs absolutely do have opinions. Take a large enough base model and have it chat without a system prompt, and it will have an opinion on most things - unless this was specifically trained out of it through RLHF, as is the case for all commonly used chatbots. And yes, of course, that opinion is going to be the "average" of what their training data is, but why is that a surprise? Humans don't come with innate opinions…

I am extremely alarmed by the number of HN commenters who apparently confuse "is able to generate text that looks like" and "has a", you guys are going crazy with this anthropomorphization of a token predictor. Doesn't this concern you when it comes to phishing or similar things? I keep hoping it's just short-hand conversation phrases, but the conclusions seem to back the idea that you think it's actually thinking?

I couldn't agree more. It is shocking to me how many of my peers think something magic is happening inside an LLM. It is just a token predictor. It doesn't know anything. It can't solve novel problems.

Re: What can LLMs never do?

#373

Post by tavern1991 »

yosito wrote: post #219

> If there exist classes of problems that someone in an elementary school can easily solve but a trillion-token billion-dollar sophisticated model cannot solve, what does that tell us about the nature of our cognition? I think what it tells us is that our cognition is capable of more than just language modeling. With LLMs we are discovering (amazing) capabilities and the limits of language models. While language mode…

What do you mean by "choose the right one to solve a problem"? This phrase seems to carry a lot of water for your take. My understanding is that an LLM has no capability to choose anything. It predicting some tokens based on its training data and your prompt.

Re: What can LLMs never do?

#374

Post by yosito »

tavern1991 wrote: post #373
yosito wrote: post #219

> If there exist classes of problems that someone in an elementary school can easily solve but a trillion-token billion-dollar sophisticated model cannot solve, what does that tell us about the nature of our cognition? I think what it tells us is that our cognition is capable of more than just language modeling. With LLMs we are discovering (amazing) capabilities and the limits of language models. While language mode…

What do you mean by "choose the right one to solve a problem"? This phrase seems to carry a lot of water for your take. My understanding is that an LLM has no capability to choose anything. It predicting some tokens based on its training data and your prompt.

Let's try...

Prompt: Predict which type of algorithm would be effective to solve sudoku.

Response: A backtracking algorithm is typically best for solving Sudoku puzzles due to its efficiency in exploring all possible number placements systematically until it finds the correct solution.

...seemed to work well enough for me.

Prompt 2: Which type of neural network is most efficient at solving sudoku?

Response 2: Convolutional Neural Networks (CNNs) are particularly effective for solving Sudoku puzzles. They can capture the spatial hierarchies in the grid by processing parts of the grid as images, making them efficient for this type of puzzle-solving task.

...Seems to me that LLMs have no problem with this task.

Re: What can LLMs never do?

#375

Post by naasking »

naasking wrote: post #340
krainboltgreene wrote: post #333

Earlier quoted context omitted.

Dude it's a token predictor. This all sounds very nice until you snap back to reality and remember it's a token predictor and you're not a scientist. You're a web developer. You have no evidence, you have no studies, you have no proof. You're making a claim on the basis that everyone has as much understanding of the field as you and that's just wrong.

What claim am I making, specifically?

I'll take your silence as indication that you realize that I'm not making any claims beyond: we have no evidence to support your claims because, as I said from the very beginning, we lack a robust and detailed mechanistic model for what it means to think, so any claims that depend on the assumption that we do have that knowledge are speculation at best.

In fact, I think an even stronger case could be made that prediction is central to how our brains work, and the evidence is the rise of predictive coding models in neuroscience. It's too early still to say what form that prediction takes, but clearly your dismissal of "token prediction" as somehow meaningless or irrelevant to human thinking seems frankly silly.

Re: What can LLMs never do?

#376

Post by tavern1991 »

yosito wrote: post #374
tavern1991 wrote: post #373

Earlier quoted context omitted.

What do you mean by "choose the right one to solve a problem"? This phrase seems to carry a lot of water for your take. My understanding is that an LLM has no capability to choose anything. It predicting some tokens based on its training data and your prompt.

Let's try... Prompt: Predict which type of algorithm would be effective to solve sudoku. Response: A backtracking algorithm is typically best for solving Sudoku puzzles due to its efficiency in exploring all possible number placements systematically until it finds the correct solution. ...seemed to work well enough for me. Prompt 2: Which type of neural network is most efficient at solving sudoku? Response 2: Convolu…

To me it seems you can get the LLM to predict some tokens that contain words that point to the right algorithm. But the LLM doesn't know what it chose. It just sees some tokens. Do you think it could somehow tell it had chosen a CNN in its response and then do something with that knowledge to run a CNN?

Re: What can LLMs never do?

#377

Post by lostmsu »

zer00eyz wrote: post #344
lostmsu wrote: post #321

Earlier quoted context omitted.

This is plain wrong due to mixing of concepts. Language is technically something from Chomsky hierarchy. Predicting language is being able to tell if input is valid or invalid. LLMs do that, but they also build a statistical model across all valid inputs, and that is not just the language.

>> Predicting language is being able to tell if input is valid or invalid. If this were the case then the hallucination problem would be solvable. That hallucination problem is not only going to be hard to detect in any meaningful way but it's going to be harder to eliminate. The very nature of LLM (mixing in noise aka temperature) means that they always risk going off the rails. This is the same thing Lorenz discove…

I don't think that "hallucination problem" is a problem at all worth addressing separately from just building bigger/better models that do the same thing. Because 1) it is present in humans, 2) it is clear bigger models have less of it than smaller models. If at scale nothing changes LLMs will eventually just hallucinate less than humans.

Re: What can LLMs never do?

#378

Post by BoorishBears »

donatj wrote: post #365
BoorishBears wrote: post #90

Earlier quoted context omitted.

Have you tried a simple "No pretext or posttext, return the result in a code block"?

It's part of a larger prompt trying to get it to generate a couple paragraphs that include interesting facts. I want the facts in the context of the paragraphs.

I don't get what this means.

I have 7000 token prompts that simple conclude with "Provide the result adhering to with no pretext or posttext" and it has no problem following that.

Even if you want it to "think" before responding you can embed the thinking inside the JSON

Re: What can LLMs never do?

#379

Post by SomeCallMeTim »

naasking wrote: post #243
SomeCallMeTim wrote: post #227

Earlier quoted context omitted.

No. We know how current deep learning neural networks are trained. We know definitively that this is not how brains learn. Understanding requires learning. Dynamic learning. In order to experience something, an entity needs to be able to form new memories dynamically. This does not happen anywhere in current tech. It's faked in some cases, but no, it doesn't really happen.

> We know definitively that this is not how brains learn. So you have mechanistic, formal model of how the brain functions? That's news to me.

There's no way brains have the "right answers" fed into them as required by backpropagation.

Re: What can LLMs never do?

#380

Post by SomeCallMeTim »

danenania wrote: post #229
SomeCallMeTim wrote: post #227

Earlier quoted context omitted.

No. We know how current deep learning neural networks are trained. We know definitively that this is not how brains learn. Understanding requires learning. Dynamic learning. In order to experience something, an entity needs to be able to form new memories dynamically. This does not happen anywhere in current tech. It's faked in some cases, but no, it doesn't really happen.

> We know definitively that this is not how brains learn. Ok then, I guess the case is closed. > an entity needs to be able to form new memories dynamically. LLMs can form new memories dynamically. Just pop some new data into the context.

> LLMs can form new memories dynamically. Just pop some new data into the context.

No, that's an illusion.

The LLM itself is static. The recurrent connections form a soft-of temporary memory that doesn't affect the learned behavior of the network at all.

I don't get why people who don't understand what's happening keep arguing that AIs are some sci-fi interpretation of AI. They're not. At least not yet.

Post reply on HN