131–140 of 253 posts
Personally I see little evidence that this "just scale a transformer until sentience" hype-train is going to take us anywhere interesting or particularly useful.
And for the people who claim it is super useful already, can you actually trust its outputs without any manual inspection in a production setting? If not it's probably not as useful as you think it might be.
I'm sure his group has done some rigorous research that I can't even understand. But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers. Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.
“AI” replacing the jobs of AI engineers. But we were told it was only going to do that to blue collar work!
Earlier quoted context omitted.
> Artificial Intelligence: A Modern Approach, 2nd ed Published in 2002. At that point, ANN research had reached a pretty hard plateau with very few tangible results. Faulting Russel and Norvig for not going into depth about ANNs is kind of like faulting Richard Feynman for not going into depth about quantum computers in the Feynman Lectures. Also, a lot of the subsequent work and breakthroughs on ANNs has been done a…
As opposed to the AI techniques taught in the AIMA book (KR and logic reasoning), which had plateau'ed in the 70s...? Norvig had to be pretty clueless to decide ANNs are such a dead-end, that they don't deserve even a chapter in his book, where all around him there are biological living proofs that neural networks are probably a pretty good bet for AI... (Note: I held the same opinion in the mid 90s when I reviewed h…
100 years ago you would have been arguing that all around you are living proofs that ornithopters are probably a pretty good bet for artificial flight. You would have been wrong about that too.
Earlier quoted context omitted.
You are just a fancy and efficient autocomplete too. When you speak or write, some words have a higher probability than others. You pick alternatives, but they are limited. Of course there are more layers in the human mind, but GPT-3 is a really impressive milestone towards AGI. It's so easy to downplay every advanced tech, it's actually fun. Planes? Just a flying metal tube. Self landing rockets? Just applied physic…
I'm not reducing GPT-3 to the extent that you're suggesting. I'm pointing out (and so does LeCun in his post) that it's a language model designed to continue a sequence of words. It has no understanding of the world and is no particularly suited for knowledge extraction or conversation. > GPT-3 is a really impressive milestone towards AGI We really don't know this. It's a big step for the field of language models, th…
What does this mean? The world with countries, borders, celebrities, hypes, newsletters and ideologies?
Because that world is as virtual and not grounded in reality as the world of data points fed into this model. Culture is made up. Language is made up. It may express itself in reality but so do the algorithms of social network sites.
If something can only live in a reality of data points on the internet this thing would live mostly in the same reality as we do.
Earlier quoted context omitted.
What evidence do I have that I'm more than a fancy autocomplete, myself? The use of squishy protestations, in lieu of objective metrics, make LeCun's argument rather unconvincing.
I was about to write a reply claiming that you're different from autocomplete because you take input from more sources than just the words you've said before (e.g. your vision), but actually I can't see how that's much different from a language model. The approach seems the same, and all that's really different is the shape of the input data. But this uncovers difficult questions about free will. If we're all just au…
That's pretty much it. I do belive it's possible to actually develop "will", but almost nobody thinks that they need to work on such things. They confuse being a programmed robot with being a programmer.
Earlier quoted context omitted.
Can you go into more details where it's useful? As your comment here goes directly against what's argued in the linked Facebook post. Also, if you've found a use case where GPT-3 replaces real humans, what did those humans actually spend their time on? Seems like either you're over-hyping GPT-3, or under-hyping humanity
The humans spent their time building a hideously difficult classification model. Out of the box GPT-3 worked better than the result of a year of their work.
Reading this is really interesting: > GPT-3 doesn't have any knowledge of how the world actually works. I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only r…
Can someone please comment his post here? (I don't have fb account and don't want to sign up)