Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

271–280 of 385 posts

Re: What can LLMs never do?

#271

Post by ccppurcell »

CamperBob2 wrote: post #132
ccppurcell wrote: post #128

Earlier quoted context omitted.

I would argue that the G in AGI means it can't require better prompting.

We should probably draw a distinction between a human-equivalent G, which certainly can require better prompting (why else did you go to school?!) and god-equivalent G, which never requires better prompting. Just using the term 'General' doesn't seem to communicate anything useful about the nature of intelligence.

School is not better prompting, it's actually the opposite! It's learning how to deal with poorly formed prompts!

Re: What can LLMs never do?

#272

Post by ccppurcell »

dragonwriter wrote: post #135
ccppurcell wrote: post #128

Earlier quoted context omitted.

I would argue that the G in AGI means it can't require better prompting.

That would like saying that because humans’ output can be better or worse based on better or worse past experience (~prompting, in that it is the source of the equivalent of “in-context learning”), humans lack general intelligence.

No, it's saying that I have general intelligence in part because I am able to reason about vague prompts

Re: What can LLMs never do?

#273

Post by jncfhnb »

adrianN wrote: post #233
jncfhnb wrote: post #131

Earlier quoted context omitted.

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

I try my best not to experience continuous time for at least eight hours a day.

Then for at least eight hours a day you don’t qualify as a generally intelligent system.

Re: What can LLMs never do?

#274

Post by PopePompus »

Hunpeter wrote: post #268
kaibee wrote: post #252

Earlier quoted context omitted.

I think its note worthy that humans actually fail this test... We have to go dormant for 8 hours every day.

Yes, but our brain is still working and processing information at those times as well, isn't it? Even if not in the same way as it does when we're conscious.

What about general anesthesia? I had a major operation during which most of my brain was definitely offline for at least 8 hours.

Re: What can LLMs never do?

#275

Post by naasking »

Scarblac wrote: post #246
naasking wrote: post #243

Earlier quoted context omitted.

> We know definitively that this is not how brains learn. So you have mechanistic, formal model of how the brain functions? That's news to me.

Your brain was first trained by reading all of the Internet? Anyway, the question of whether computers can think is as interesting as the question whether submarines can swim.

> Anyway, the question of whether computers can think is as interesting as the question whether submarines can swim.

Given the amount of ink spilled on the question, gotta disagree with you there.

Re: What can LLMs never do?

#276

Post by PopePompus »

jncfhnb wrote: post #269
PopePompus wrote: post #196

Earlier quoted context omitted.

I don't understand why you believe it must experience continuous time. If you had a system which clearly could reason, which could learn new tasks on its own, which didn't hallucinate any more than humans do, but it was only active for the period required for it to complete an assigned task, and was completely dormant otherwise, why would that dormant period disqualify it as AGI? I agree that such a system should pro…

Active for a period is still continuous during that period. As opposed to “active when called”. A function, being called repeatedly over a length of time is reasonably “continuous” imo

I don't see what the difference between "continuous during that period" and "active when called" is. When an AI runs inference, that calculation takes time. It is active during the entire interval during which it is responding to the prompt. It is then inactive until the next prompt. I don't see why a system can't be considered intelligent merely because its activity is intermittent.

Re: What can LLMs never do?

#277

Post by grobgambit »

SomeCallMeTim wrote: post #136
_wire_ wrote: post #119

Earlier quoted context omitted.

>We don't fully understand why current LLMs are bad at these tasks. In complete seriousness, can anyone can explain why LLMs are good at some tasks?

LLMs are good at tasks that don't require actual understanding of the topic. They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data. What they're doing doesn't really match any definition of "und…

I would just add that I think I have encountered situations that knowing the weighted average answer from the training data for topics I didn't previously understand created better initial conditions for MY learning of the topic than not knowing the weighted average answer.

The problem to me is we are holding LLMs to a standard of usefulness from science fiction and not reality.

A new, giant set of encyclopedias has enormous utility but we wouldn't hold it against the encyclopedias that they aren't doing the thinking for us or 100% omniscient.

Re: What can LLMs never do?

#278

Post by naasking »

krainboltgreene wrote: post #256
naasking wrote: post #244

Earlier quoted context omitted.

Do you have mechanistic model for what it means to think? If not, how do you know thinking isn't equivalent to sophisticated next token prediction?

How do you know my cat isn't constantly solving calculus problems? I also can't come up with a "mechanistic model" for what it means to do that either. Further, if your rubric for "can reason with intelligence and have an opinion" is "looks like it" (and I certainly hope this isn't the case because woo-boy), then how did you not feel this way about Mark V. Shaney? Like I understand that people live learning about the…

> but we actually know it's a program and how it works. There is no mystery.

You're right, we do know how it works. Your mistake is concluding that because we know how LLMs work and they're not that complicated, but we don't know how the brain works and it seems pretty complicated, therefore the brain can't be doing what LLMs do. That just doesn't follow.

You made exactly the same argument in the opposite direction, asking if my rubric for "can reason with intelligence and have an opinion" is "seems like it", and your rubric for "thinking is not a token predictor driven by matrix multiplications" is "seems like it".

You can make a case for the plausibility of each conclusion, but that's doesn't make it a fact, which is how you're presenting it.

Re: What can LLMs never do?

#279

Post by naasking »

jncfhnb wrote: post #270
naasking wrote: post #240

Earlier quoted context omitted.

Some good prompt-reply interactions are probably fed back in to subsequent training runs, so they're still stateful/have memory in a way, there's just a long delay.

That’s not the AGI’s state. That’s just some past information.

State is a function of accumulated past information.

Re: What can LLMs never do?

#280

Post by iraqmtpizza »

naasking wrote: post #275
Scarblac wrote: post #246

Earlier quoted context omitted.

Your brain was first trained by reading all of the Internet? Anyway, the question of whether computers can think is as interesting as the question whether submarines can swim.

> Anyway, the question of whether computers can think is as interesting as the question whether submarines can swim. Given the amount of ink spilled on the question, gotta disagree with you there.

Endless ink has been spilled on the most banal and useless things. Deconstructing ice cream and physical beauty from a Marxist-feminist race-conscious postmodern perspective.
Post reply on HN