Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

271–280 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#271

Post by [deleted] »

visarga wrote: post #236
bra-ket wrote: post #137

The article is of course right but also a bit silly. Language models like GPT-X are producing grammatically correct sentences, along the lines of "Colorless green ideas sleep furiously". The NLP research more or less solved the old syntax problem using 'distributional semantics' but 'semantics' is a misnomer, it's all about syntax. In fact the most useful part of the article for me is that they mentioned Douglas Summ…

Yes, how dare these machine learning scientists use backprop? I mean, there are better methods our there, right?

[deleted]

Re: GPT-3 has no idea what it’s talking about

#273

Post by mannykannot »

IanAWP wrote: post #258
mannykannot wrote: post #227

Earlier quoted context omitted.

I think the point here is that it hs no understanding of what you are calling a 'bad' prompt. That does not mean that it lacks any understanding whatsoever (personally, I think it has made a small step in that direction), but it is a valid case of it not understanding something . Furthermore, I do not think we can assume that if it were trained on examples of satire or intentionally absurd text, it would perform bett…

GPT is both general purpose and stateless. Lacking another mechanism, it's the requirement of the prompt to request a particular genre of response if that's what you want. There can be no such thing as a bad prompt from gpt3's perspective. A bad prompt is one where the user has a specific purpose which is not expressed. It's bad because you know beforehand that gpt3 can not align with it. Someone pours grape juice in…

> GPT3 correctly grasps that there is a hidden context to this weird prompt.

That is interesting. How does it indicate its understanding that there is a hidden context?

Re: GPT-3 has no idea what it’s talking about

#274

Post by kmeisthax »

webmaven wrote: post #263
throwawaygh wrote: post #148

Earlier quoted context omitted.

Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy".

> Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy". You seem to have chang…

Maybe the person you were replying to is actually a GPT-3 comment bot and we're already screwed.

Re: GPT-3 has no idea what it’s talking about

#275

Post by dwild »

webmaven wrote: post #263
throwawaygh wrote: post #148

Earlier quoted context omitted.

Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy".

> Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy". You seem to have chang…

He didn't, he is saying that the previous comment anecdotal evidence could be easily explained by the amount of interest into the project by many people, which can't all get access to it, but then show how this same explanation shouldn't apply to Gary Marcus and Ernest Davis.

Re: GPT-3 has no idea what it’s talking about

#276

Post by ClumsyPilot »

voces wrote: post #239
ClumsyPilot wrote: post #223

Earlier quoted context omitted.

you are sidestepping the question above - what can it be used for? What kind of fine-tuning is required? Because to me it looks like 'fixing catastrophic errors' not fine-tuning - as anywhere where the text can deliver value, it has to be correct. AWS/Azure are a platform, and they work reliably. What kind of platform 'sometimes works'?

Language models can help the disabled communicate better and faster (fine-tune on their online output, then the BCI can offer better continuations, so the patient does not have to type as much). Fine-tune on educational materials, and a language model could be a 24/7 assistant to students. Fine-tune on psychology or medicine data, and a language model could tell you of all medicine interactions, or act as a better EL…

I did not downvote, but I think it's because you kinda missed the crux of my question is - what is fine-tuning, like what do you have to actually do?

Has anyone done it and demonstrated that it can be done on GPT to get useful and reliable output for some domain? If we exclude fake news/ twitter-bots or something equally silly.

Re: GPT-3 has no idea what it’s talking about

#277

Post by ClumsyPilot »

dodobirdlord wrote: post #155
ClumsyPilot wrote: post #139

Earlier quoted context omitted.

> if continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. Well then you can justify it outputting anything at all.

This is why it’s fair to call some prompt designs bad. With a bad prompt you can’t really judge anything from the response. But better prompt design can uncover interesting behaviors. GPT-3 will generally not reject a prompt as nonsense or unreasonable, and will instead do a straight-faced continuation, probably because it has been trained on plenty of jokes, satires, dream sequences, etc. But if you specifically giv…

I have read your link, but think this is an unfair argument.

We are not talking about nonsense, we are talking about unexpected input, which will happen in any real-world situation. It might be text about something a kid, or depressed person has done. The text about stirring with a cigar is firmly withing the realm of plausibility.

And what happens is that GPT seemingly goes off the rails.

Re: GPT-3 has no idea what it’s talking about

#278

Post by throwawaygh »

webmaven wrote: post #263
throwawaygh wrote: post #148

Earlier quoted context omitted.

Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy".

> Look, they're probably getting millions of requests. At least some of those people who are respected scientists in NLP/AI but cannot get access were almost certainly just over-looked. It was really seeing that even Gary Marcus and Ernest Davis cannot get access that pushed me over the line on this one from "be gracious, moderating access to a finite resource is hard" to "wow this is sketchy". You seem to have chang…

I had a belief (assume best intentions unless there's evidence otherwise). And then, later, I modified that belief given some new compelling evidence.

Re: GPT-3 has no idea what it’s talking about

#279

Post by Rioghasarig »

BrokrnAlgorithm wrote: post #250
echelon wrote: post #246

Earlier quoted context omitted.

I wonder if you can get GPT-3 bots to spam Reddit, Twitter, and Facebook into oblivion. I also wonder what percentage of users would notice. Give them a political bent - that's probably what the state actors are trying to productionize right now. Target posts with a sentiment that disagrees with yours, then make the bots follow those users and inundate them with replies wherever they go. Hell, even brands might step…

I've been wondering about the exact same thing. Basically, a sufficiently calibrated and targeted GPT3 bot swarm could be employed to render at least some parts of these communities into useless echo chambers. I think that while GPT3 posts are usually identifiable after reading some sentences, I often also find that it's harder to realize when consuming precisely this kind of social media. I often just consume it abs…

I don't now about that. I believe if you had it calibrated correctly no one could tell the difference between a single GPT-3 comment and a human comment.

Maybe after a bit of dialogue you'd have a higher chance, but even then I suspect (from playing around in AI Dungeon) that GPT-3 could do very well.

Re: GPT-3 has no idea what it’s talking about

#280

Post by Rioghasarig »

amelius wrote: post #238
lumost wrote: post #153

Earlier quoted context omitted.

For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. This could realistically only occur if they cost-effectively fine-tune away the more egregious problems in beta - or convince enough investors that their next model with a 100 million dollar price tag will be able to handle something approximating AGI for realistic applications. This is…

> For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.

I actually just sent out an API request for a particular application that I believe GPT-3 should be capable.

I'm a university mathematics instructor. I find that a lot of students struggle with doing proofs and I think GPT-3 can help with that. A proof is essentially a sequence of logical inferences. I believe that, when given a proof written in natural language GPT-3 should be capable of detecting the logical implications made within. From there, it would be possible to correct student's mistakes or help them improve their answer.

Post reply on HN