Earlier quoted context omitted.
Transformer models have been shown to spontaneously form internal, predictive models of their input spaces. This is one of the most pervasive misunderstandings about LLMs (and other transformers) around. It is of course also true that the quality of these internal models depends a lot on the kind of task it is trained on. A GPT must be able to reproduce a huge swathe of human output, so the internal models it picks o…
> can provide links if you're interested Please do :)
Large models of what? Mistaking engineering achievements for linguistic agency
81–90 of 162 posts
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#82Earlier quoted context omitted.
They are two researchers/assistant professors working with cognitive science, psychology, and trustworthy AI. The paper is peer reviewed and has been accepted for publication in the Journal of Language Sciences. You should publish your critique of their research in that same journal. P.s. if you find any grave mistakes, you can contact the editor in chief, who happens to be a linguist.
An appeal to authority if ever there was one. Their critique is written here, in plain english. Any fault with it you can just mention. The "I won't read your comment unless you get X journal to publish it" seems really counterproductive. Presumably even the great Journal of Language Sciences is not above making mistakes or publishing things that are not perfect.
>> An appeal to authority if ever there was one.
I read it as a clear refutation of the assertion that the authors "have no connection" to AI (or to AI hype; unclear from the OP).
Btw, the OP is a typical ad-hominem, drawing attention to who is speaking rather than what they're saying.
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#83The authors of this paper are just another instance of the AI hype being used by people who have no connection to it, to attract some kind of attention. "Here is what we think about this current hot topic; please read our stuff and cite generously ..." > Language completeness assumes that a distinct and complete thing such as `a natural language' exists, the essential characteristics of which can be effectively and c…
Babies have feedback and interaction with someone speaking to them. Would they learn to speak if you just dumped them in front of a TV and never spoke to them? I'm not sure. But anyway I agree with you. This is just a confused HN comment in paper form.
I personally don’t get much value out of the paper, but it is orders of magnitude more substantive and thoughtful than a median “confused Hacker News comment”.
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#84The authors of this paper are just another instance of the AI hype being used by people who have no connection to it, to attract some kind of attention. "Here is what we think about this current hot topic; please read our stuff and cite generously ..." > Language completeness assumes that a distinct and complete thing such as `a natural language' exists, the essential characteristics of which can be effectively and c…
Please argue on the merits and substance. I’m less interested in speculation on the authors’ motivations.
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#85I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that make…
I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, e…
I don’t hold LeCun’s opinions in high regard because of his often hyperbolic statements.
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#86I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that make…
I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, e…
Put aside consciousness or hype or investment. Look at the results; LLMs are well beyond old-school search in many ways. Sure, they are flawed in someways. Previous paradigms for search, were also flawed in their own ways.
Look at the arc of NLP. Large language models fit the pattern. One could even say that their development (next token prediction with a powerful function approximator) is obvious in hindsight.
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#87I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that make…
I find discussions of consciousness even more taxing than religion, free will, or politics.
With very careful discussion, there are some really interesting concepts in play. This paper however does not strike me as worth most people’s time. Especially not regarding consciousness.
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#88I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already belie…
The only thing this paper prove is that folks at Trinity College in Dublin are poor, envious anthropocentric drunkards, ready to throw every argument to defend their crown of creating, without actually understanding the linguistics concepts they use to make their argument.
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#89Earlier quoted context omitted.
If you really want to phrase it that way, organisms like us are "just" distributions of genes that have been pushed this way and that by natural selection until they converged to something we consider intelligent (humans). It's pretty clear that these optimisation processes lead to emergent behaviour, both in ML and in the natural sciences. Computability theory isn't really relevant here.
I don't even know where to begin to address your confusion. Without computability theory there are no computers, no operating systems, no networks, no compilers, and no high level frameworks for "AI".
[deleted]
Re: Large models of what? Mistaking engineering achievements for linguistic agency
#90“Enactivism” really? I wonder if these complaints will continue as LLMs see wider adoption, the old first they ignore you, then they ridicule you, then they fight you… trope that is halfways accurate. Any field that focuses on building theories on top of theories is in for a bad time. https://en.m.wikipedia.org/wiki/Enactivism
What is the thing the LLMs are fighting for?