Earlier quoted context omitted.
Is creativity not just the application of a pattern in an adjacent space?
No, lol
thank you for your considered response
Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
141–147 of 147 posts
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#142Earlier quoted context omitted.
Language is a symbolic system. From an absolute or spiritual standpoint, meaning transcends pure linguistic probabilities. Language itself emerges as a limited medium for the expression of consciousness and abstract thought. Indeed, to say meaning arises purely from language (as probability alone) or, to deny language influences meaning entirely are both overly simplistic extremes.
"When he to whom one speaks does not understand, and he who speaks himself does not understand, that is metaphysics." - Voltaire Like I said in another comment, I can think of a dozen statistical and computational methods where if you give me a text and its synthesis I can find a strong probabilistic link between the two.
"Not everything that counts can be counted, and not everything that can be counted counts." - Someone.
Statistical correlation between text and synthesis undoubtedly exists, but capturing correlation does not imply you've encapsulated meaning itself. My point is precisely that: meaning isn't confined entirely within what we can statistically measure, though it may still be illuminated by it.
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#143Earlier quoted context omitted.
It depends what you mean by "engineering these solutions into the model". Using better data leads to better models given the same architecture and training. Nothing wrong with it, it's hard work, it might be with as specific goal in mind. LLM "breakthroughs" aren't really a thing at this point. It's just one little thing after another.
Sure, I specifically pre-agreed to it not being ill will. What I mean is keeping tabs on the latest demand (newer benchmarks) and making sure their model delivers in some fashion. But it is mundane and they don't say that. And when a major number increases, people don't assume they just added more specific training data.
Yup, it's a fair point. We very quickly got down to the nitty gritty with these things. Hopefully, like semiconductors nitty gritty results in a lot of big performance gains for decades.
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#144Earlier quoted context omitted.
Thanks for the links, the "tradeoff" aspect of paraconsistent logic is interesting. I think one way to achieve consensus with your debate partner might be to consider that the language rep is "just" a nondeterministic decompression of "the facts". I'm primed to agree with you but https://news.ycombinator.com/item?id=41892090 (It's very common, esp. with educationally traumatized Americans, e.g., to identify Math with…
if conceptual thinking is manipulating abstract concepts after having been given concrete particulars, I'd say it relies heavily upon projection, which, as generalised "K" (from SKI), sounds awfully like calculation.
And this is why I think gibson1 is wrong: we can argue about which projections or systems of logic should be used, concepts are still "calculations".
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#145Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#146Earlier quoted context omitted.
if conceptual thinking is manipulating abstract concepts after having been given concrete particulars, I'd say it relies heavily upon projection, which, as generalised "K" (from SKI), sounds awfully like calculation.
And this is why I think gibson1 is wrong: we can argue about which projections or systems of logic should be used, concepts are still "calculations".
Here is why I think Gibson could in principle still be right (without necessarily summoning religious feelings)
[if we disregard that he said "concepts are key" -- though we can be yet more charitable and assume that he doesn't accept (median) human-level intelligence as the final boss]
Para-doxxing ">" Under-standing
(More on that "sophistiKated" aspect of "projeKtion": turns out not to be as idiosynKratic as I'd presumed, but I traded bandwidth for immediacy here, so I'll let GP explain why that's interesting, if he indeed finds it is :)
Wolfram (selfstyled heir to Leibniz/Galois) seems to be serving himself a fronthanded compliment:
https://writings.stephenwolfram.com/2020/12/combinators-a-ce...
>What I called a “projection” then is what we’d call a function now; a “filter” is what we’d now call an argument )
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#147As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…
> It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between the words of a text and the gist of the content,
I am not an expert by any means but have some knowledge of the technicalities of the LLMs and my limited knowledge allows me to disagree with your statement. The models are trained on an ungodly amount of text, so they become very advanced statistical token prediction machines with magic randomness sprinkled in to make the outputs more interesting. After that, they are fine tuned on very believable dialogues, so their statistical weights are skewed in a way that when subject A (the user) tells something, subject B (the LLM-turned-chatbot) has to say something back which statistically should make sense (which it almost always does since they are trained on it in the first place). Try to paste random text - you will get a random reply. Now try to paste the same random text and ask the chatbot to summarize it - your randomness space will be reduced and it will be turned into a summary because the finetuning gave the LLM the "knowledge" what the summarization _looks like_ (not what it _means_).
Just to prove that you are wrong: ask your favorite LLM if your statement is correct and you will probably see it output that it is not.