Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large models of what? Mistaking engineering achievements for linguistic agency

arxiv.org

Post reply on HN
Print view View original

51–60 of 162 posts

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#51

Post by obirunda »

nativeit wrote: post #26

I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that make…

I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, even while they wax poetic about scaling laws, they know there is a bottleneck ahead.

Notice how LeCun is the only researcher being honest about this in a public fashion. Meta is committed to AI already and will at least match the spend of competitors anyway, so he doesn't have as much pressure to try and convince investors that this rabbit whole is deeper.

Don't get me wrong, LLMs are a tremendous improvement on knowledge compression and distillation, but it's still unreliable enough that old school search is likely a superior method nonetheless.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#52

Post by jdietrich »

nsagent wrote: post #30
jdietrich wrote: post #24

Earlier quoted context omitted.

The question I gave is a literal textbook example of abstract reasoning. LLMs are just very advanced text prediction, but they are also provably capable of abstract reasoning. If you think that those statements are contradictory, I would encourage you to read up on the Bayesian hypotheses in cognitive science - it is highly plausible that our brains are also just very advanced prediction models.

You're quite right that LLMs can seemingly do some abstract reasoning problems, but I would not say they aren't in the training data. Sure, the exact form using the made up word gronk might not be in the training data, but the general form of that reasoning problem definitely exists, quite frequently in fact.

Yes, but the general form of the problem tells you nothing about the answer to any specific case. To perform any better than chance, the model has to actually reason through the problem.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#53

Post by lgas »

kazinator wrote: post #48
JohnKemeny wrote: post #45

Earlier quoted context omitted.

They are two researchers/assistant professors working with cognitive science, psychology, and trustworthy AI. The paper is peer reviewed and has been accepted for publication in the Journal of Language Sciences. You should publish your critique of their research in that same journal. P.s. if you find any grave mistakes, you can contact the editor in chief, who happens to be a linguist.

> You should publish your critique of their research in that same journal. No thanks; that would be at least twice removed from Making Stuff. (Once removed is writing about Making Stuff.)

One might argue that a critique itself is stuff.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#54

Post by lucianbr »

Lerc wrote: post #28
et1337 wrote: post #21

Earlier quoted context omitted.

> Future models will not be able to do those things if they are the same as the current ones I think a lot of people disagree with this. People think if we just keep adding parameters and data, magic will happen. That’s kind of what happened with ChatGPT after all.

I'm not so sure that view is very widespread amongst people familiar with how LLMs work. Certainly they become more capable with parameters and data, but there are fundamental things that can't be overcome with a basic model and I don't think anyone is seriously arguing otherwise. For instance LLMs are pretty much stateless without their context window. If you treat the raw generated output as the first and final res…

Has anyone done the thing, and achieved some interesting results? Seems a pretty obvious thing to try, but I never heard of anything like it.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#55

Post by lucianbr »

JohnKemeny wrote: post #45
kazinator wrote: post #35

The authors of this paper are just another instance of the AI hype being used by people who have no connection to it, to attract some kind of attention. "Here is what we think about this current hot topic; please read our stuff and cite generously ..." > Language completeness assumes that a distinct and complete thing such as `a natural language' exists, the essential characteristics of which can be effectively and c…

They are two researchers/assistant professors working with cognitive science, psychology, and trustworthy AI. The paper is peer reviewed and has been accepted for publication in the Journal of Language Sciences. You should publish your critique of their research in that same journal. P.s. if you find any grave mistakes, you can contact the editor in chief, who happens to be a linguist.

An appeal to authority if ever there was one.

Their critique is written here, in plain english. Any fault with it you can just mention. The "I won't read your comment unless you get X journal to publish it" seems really counterproductive. Presumably even the great Journal of Language Sciences is not above making mistakes or publishing things that are not perfect.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#56

Post by flimflamm »

How would the authors consider a paralyzed individual who can only move their eyes since birth? That person can learn the same concepts as other humans and communicate as richly (using only their eyes) as other humans. Clearly, the paper is viewing the problem very narrowly.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#57

Post by 8n4vidtmkvmk »

sriku wrote: post #44
GeneralMayhem wrote: post #8

I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already belie…

The crux of the video game analogy seems to be that when you go close to an object, the resolution starts blurring and the illusion gets broken, and there is a similar thing that happens with LLMs (as of today) as well. This is, so far, reasonable based on daily experience with these models. The extension of that argument being made in the paper is that a model trained on language tokens spewed by humans is incapable…

Why are LLMs incapable of reaching that limit? It's very easy to imagine video games getting to that point. We have all the data to see objects right down to the atomic level, which is plenty more than you'd need for a game. It's mostly a matter of compute. Why then should LLMs breakdown if they can at least mimic the smartest humans? We don't need "resolution" beyond that.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#58

Post by card_zero »

stirfish wrote: post #29
jdietrich wrote: post #17

Earlier quoted context omitted.

LLMs do contain conceptual representations and LLMs are capable of abstract reasoning. This is trivially provable by asking them to reason about something that is a) purely abstract and b) not in the training data, e.g. "All floots are gronks. Some gronks are klorps. Are any floots klorps?" Any of the leading LLMs will correctly answer questions of this type much more often than chance.

I just asked chatgpt "All floots are gronks. Some gronks are klorps. Are any floots klorps?" ------ To determine if any floots are klorps, let's analyze the given statements: 1. All floots are gronks. This means every floot falls into the category of gronks. 2. Some gronks are klorps. This means there is an overlap between the set of gronks and the set of klorps. Since all floots are included in the set of gronks and…

Huh, almost right. ("possible, but not guaranteed?" it's necessarily true. That whole sentence was a waste of space, and wrong.)

Edit: I mean "if there is any overlap", it's necessarily true. I should have quoted the whole thing.

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#59

Post by jdietrich »

wonnage wrote: post #36
jdietrich wrote: post #17

Earlier quoted context omitted.

LLMs do contain conceptual representations and LLMs are capable of abstract reasoning. This is trivially provable by asking them to reason about something that is a) purely abstract and b) not in the training data, e.g. "All floots are gronks. Some gronks are klorps. Are any floots klorps?" Any of the leading LLMs will correctly answer questions of this type much more often than chance.

Or maybe they're just pattern matching on the very particular sentence structure you've chosen. This isn't a convincing example at all

This isn't something I should convince you of. Just open up ChatGPT or Claude and try it for yourself. Think up a batch of your own questions and see how a modern LLM fares. I assure you that it'll do much better than chance. If you're so inclined, you can run enough tests to achieve statistical significance in the course of your lunch break.

It depresses me that we seem to be spending more time arguing and hypothesising about LLMs than empirically testing them. The question of whether LLMs can think is completely settled, as their performance at zero-shot problems is simply impossible through pure memorisation or pattern-matching. The question that remains is far more interesting - how do they think?

https://arxiv.org/pdf/2205.11916

Re: Large models of what? Mistaking engineering achievements for linguistic agency

#60

Post by jdietrich »

card_zero wrote: post #58
stirfish wrote: post #29

Earlier quoted context omitted.

I just asked chatgpt "All floots are gronks. Some gronks are klorps. Are any floots klorps?" ------ To determine if any floots are klorps, let's analyze the given statements: 1. All floots are gronks. This means every floot falls into the category of gronks. 2. Some gronks are klorps. This means there is an overlap between the set of gronks and the set of klorps. Since all floots are included in the set of gronks and…

Huh, almost right. ("possible, but not guaranteed?" it's necessarily true. That whole sentence was a waste of space, and wrong.) Edit: I mean "if there is any overlap", it's necessarily true. I should have quoted the whole thing.

Nope, ChatGPT was right, the answer is indeterminable. The klorps that are gronks could be a wholly distinct subset to the klorps that are floots. It also correctly evaluates "All gronks are floots. Some gronks are klorps. Are any floots klorps?", to which the answer is definitively yes.
Post reply on HN