91–100 of 253 posts
One thing I've been wondering, could you train a GPT-3 model to generate "better" text prompts for another GPT-3 model By better I mean grading based on whether there is any nonsense in the output or any internal contradictions, or similar criteria
Earlier quoted context omitted.
Extremely complex classification task is all I can say
GPT-3 is a generative model, isn't it? Can you explain how you converted GPT-3 to a classification model?
> Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do. Just want to point out that he's saying the people on the upper end of the expectation distribution are wrong, not the people in the middle of it. So if you're takeaway from this is that GPT3 is nothing special, that's probably the wrong message.
His next paragraph claims that Nabla "debunks" the idea that "large language models" can be used in healthcare. That's not just "some people have unrealistic expectations" it's "this tool, when when more advanced and find tuned, will never be appropriate to use in a very broad class of use cases". He also says "GPT-3 has no knowledge of how the world works", which is clearly an overstatement meant to clear up hype, b…
Earlier quoted context omitted.
The iPhone didn't replace your flip phone, apple did. The iPhone didn't build itself.
Yes except they were saying the iPhone replaced Nokia's engineers. GPT-3 is not doing what the ML engineers were doing (building models), GPT-3 is the end goal. The company just decided to outsource the work to OpenAI and pay a monthly fee to them instead of salaries to their ML engineers. "We have already found several use cases for it, one of which replaces 2 ML engineers." -> Clearly makes it sounds like GPT-3 can…
I think everyone understood that.
I'm sure his group has done some rigorous research that I can't even understand. But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers. Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.
Can you go into more details where it's useful? As your comment here goes directly against what's argued in the linked Facebook post. Also, if you've found a use case where GPT-3 replaces real humans, what did those humans actually spend their time on? Seems like either you're over-hyping GPT-3, or under-hyping humanity
Earlier quoted context omitted.
I happen to use slightly less fancy and expensive GPT-2 based autocomplete, and it's amazing. https://tabnine.com
Interesting. As a reading researcher, I imagine that this could potentially introduce subtle and difficult to spot bugs when you get a proposed completion that looks about right i.e. close enough to what you imagined. Has this been an issue in your experience?
> Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do. Just want to point out that he's saying the people on the upper end of the expectation distribution are wrong, not the people in the middle of it. So if you're takeaway from this is that GPT3 is nothing special, that's probably the wrong message.
His next paragraph claims that Nabla "debunks" the idea that "large language models" can be used in healthcare. That's not just "some people have unrealistic expectations" it's "this tool, when when more advanced and find tuned, will never be appropriate to use in a very broad class of use cases". He also says "GPT-3 has no knowledge of how the world works", which is clearly an overstatement meant to clear up hype, b…
It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing.
If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this?
The same argument can be made about those kind of models, the knowledge they have is about the structure of the language and what word is most likely to come next, but they have no way to ground those words in actual relation with the world.
IMO, the real innovation in GPT-3 is that the API plus "playground" setup is far easier to use than sharing a big chunk of Python code and data files in a Jupyter notebook. If the researchers behind other cutting-edge language models made their work just as easy to use, I believe we would see similar excitement there as well.
Earlier quoted context omitted.
You are just a fancy and efficient autocomplete too. When you speak or write, some words have a higher probability than others. You pick alternatives, but they are limited. Of course there are more layers in the human mind, but GPT-3 is a really impressive milestone towards AGI. It's so easy to downplay every advanced tech, it's actually fun. Planes? Just a flying metal tube. Self landing rockets? Just applied physic…
I'm not reducing GPT-3 to the extent that you're suggesting. I'm pointing out (and so does LeCun in his post) that it's a language model designed to continue a sequence of words. It has no understanding of the world and is no particularly suited for knowledge extraction or conversation. > GPT-3 is a really impressive milestone towards AGI We really don't know this. It's a big step for the field of language models, th…
If a language model were able to do this task perfectly, it would be indistinguishable from intelligence, because continuing a sequence of words requires reasoning. You cannot conclude that has no understanding based solely on what it is trained to do when the task it is trained on would be sufficient to demonstrate understanding were it to fully succeed. There are lots of reasons to be skeptical of its potential, but this isn't one of them.