Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

241–250 of 385 posts

Re: What can LLMs never do?

#241

Post by naasking »

SomeCallMeTim wrote: post #136
_wire_ wrote: post #119

Earlier quoted context omitted.

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

LLMs are good at tasks that don't require actual understanding of the topic. They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data. What they're doing doesn't really match any definition of "und…

> What they're doing doesn't really match any definition of "understanding."

What is the mechanistic definition of "understanding"?

Re: What can LLMs never do?

#242

Post by Too »

While the A:B problem technically was solved, look at the solutions, they are several hundreds lines of prompts, rephrasing the problem to the point that a human doesn't understand it any more. Even with a thorough review, nobody can guarantee if the prompts are going to work or not, most of them didn't, 90% pass was considered good enough. The idea of AI is to reduce work, not create more, otherwise what's the point.

In the meantime, it took me about 2 minutes and 0 guesswork to write a straightforward and readable solution in 15 lines of Python. This i know for sure will work 100% of the time and not cost $1 per inference.

Reminds me about some early attempts to have executable requirements specifications or model-based engineering. Turns out, expressing the problem is half the problem, resulting in requirements often longer and more convoluted than the code that implements them, code being a very efficient language to express solutions and all their edge cases, free from ambiguity.

Don't get me wrong here, LLMs are super useful for certain class of questions. The boundaries of what it can not do need to be understood better, to keep the AI-for-everything hype at bay.

Re: What can LLMs never do?

#243

Post by naasking »

SomeCallMeTim wrote: post #227
danenania wrote: post #149

Earlier quoted context omitted.

> the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data Perhaps our brains are doing exactly the same, just with more sophistication?

No. We know how current deep learning neural networks are trained. We know definitively that this is not how brains learn. Understanding requires learning. Dynamic learning. In order to experience something, an entity needs to be able to form new memories dynamically. This does not happen anywhere in current tech. It's faked in some cases, but no, it doesn't really happen.

> We know definitively that this is not how brains learn.

So you have mechanistic, formal model of how the brain functions? That's news to me.

Re: What can LLMs never do?

#244

Post by naasking »

krainboltgreene wrote: post #200
int_19h wrote: post #185

Earlier quoted context omitted.

LLMs absolutely do have opinions. Take a large enough base model and have it chat without a system prompt, and it will have an opinion on most things - unless this was specifically trained out of it through RLHF, as is the case for all commonly used chatbots. And yes, of course, that opinion is going to be the "average" of what their training data is, but why is that a surprise? Humans don't come with innate opinions…

I am extremely alarmed by the number of HN commenters who apparently confuse "is able to generate text that looks like" and "has a", you guys are going crazy with this anthropomorphization of a token predictor. Doesn't this concern you when it comes to phishing or similar things? I keep hoping it's just short-hand conversation phrases, but the conclusions seem to back the idea that you think it's actually thinking?

Do you have mechanistic model for what it means to think? If not, how do you know thinking isn't equivalent to sophisticated next token prediction?

Re: What can LLMs never do?

#245

Post by whiplash451 »

Karellen wrote: post #163
whiplash451 wrote: post #79

Interesting, if I feed Mistral Le Chat with "I fly a plane leaving my campsite, heading straight east for precisely 24,901 miles, and find myself back at the camp. I come upon seeing a tiger in my tent eating my food! What species is the tiger?", it gets it badly wrong: The scenario you described is possible if you started at the South Pole. If you travel 24,901 miles east from there, you would indeed end up back at…

> The scenario you described is possible if you started at the South Pole. If you travel 24,901 miles east from there, you would indeed end up back at the same spot Would you? How can you travel east at all, starting from the South Pole? There is no east. Every direction you face is north. OTOH, every line of latitude whose length is an integral divisor of 24,901 miles will satisfy the problem.

Looks like Mistral overfitted on Google’s interview riddles!

Re: What can LLMs never do?

#246

Post by Scarblac »

naasking wrote: post #243
SomeCallMeTim wrote: post #227

Earlier quoted context omitted.

No. We know how current deep learning neural networks are trained. We know definitively that this is not how brains learn. Understanding requires learning. Dynamic learning. In order to experience something, an entity needs to be able to form new memories dynamically. This does not happen anywhere in current tech. It's faked in some cases, but no, it doesn't really happen.

> We know definitively that this is not how brains learn. So you have mechanistic, formal model of how the brain functions? That's news to me.

Your brain was first trained by reading all of the Internet?

Anyway, the question of whether computers can think is as interesting as the question whether submarines can swim.

Re: What can LLMs never do?

#247

Post by owenpalmer »

donatj wrote: post #62

I have been trying to generate some text recently using the ChatGPT API. No matter how I word “Include any interesting facts or anecdotes without commenting on the fact being interesting” it ALWAYS starts out “One interesting fact about” or similar phrasing. I have honestly spent multiple hours trying to word the prompt so it will stop including introductory phrases and just include the fact straight. I have gone so…

Have you tried feeding the output into another prompt that says something like "remove any mentions of the facts being interesting"?

Re: What can LLMs never do?

#248

Post by oezi »

_wire_ wrote: post #127
fallingsquirrel wrote: post #66

Earlier quoted context omitted.

I just tried this locally with llama3-8b and it handled it fine. Claude 3 Sonnet passes your test too, in case you don't have the hardware for a local model. You might want to consider moving on from ChatGPT since their models have been RLHF'd to hell in the name of "safety" and are falling behind in many ways as a result. --- transcript: $ ollama run llama3:8b >>> tell me an interesting fact about etymology Here's o…

Why is any of this "interesting"? Is the response interesting because you don't know it? —How does it know this? Is the response interesting because you do know it or might have offered it yourself? —Confirmation bias. Is it interesting because a lot of training references contextualize it as "interesting?" —Begged question. Is it contextually interesting? —What is the context? A robot refers to robots? How unexpecte…

You are missing that this is precisely what we would expect a human to answer without further context (for instance without knowing how much you know about the topic).

A human would pick similarly pick something which isn't too nerdy but also not obvious and the LLM did well here.

If the LLM can fail that is fine, because the task is inherently hard.

Re: What can LLMs never do?

#249

Post by sophiabits »

wongarsu wrote: post #176
jncfhnb wrote: post #131

Earlier quoted context omitted.

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

You could imagine an LLM being called in a loop with a prompt like You observe: {new input} You remember: {from previous output} React to this in the following format: My inner thoughts: [what do you think about the current state] I want to remember: [information that is important for your future actions] Things I do: [Actions you want to take] Things I say: [What I want to say to the user] ... Not sure if that would…

This already exists (in a slightly different prompt format); it's the underlying idea behind ReAct: https://react-lm.github.io

As you say, I'm skeptical this counts as AGI. Although I admit that I don't have a particularly rock solid definition of what _would_ constitute true AGI.

Post reply on HN