Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

251–260 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#251

Post by gwd »

ClumsyPilot wrote: post #144
gweinberg wrote: post #132

Earlier quoted context omitted.

To be fair to the AI, stirring lemonade with a cigarette is so batshit insane that there really can't be a sensible continuation.

So basically GPT is useless if I feed it any text about life in Russia?

No; to a certain extent, GPT-3's job is: "Make a text that looks like one you might actually find floating around the internet."

What's not clear to me, even from this thread, is what exactly they were expecting it to do instead.

It sounds like they were expecting it to say something like, "...but that turned out to be a bad idea, because the cigarette dissolved and then it tasted like cheap tobacco and paper."

But honestly, is that the most likely thing you'd find on the internet? Nobody goes a random direction like "I decided to stir my lemonade with a cigarette" and then plays it totally mundane. If you writing goes in an absurd direction, it continues to go that way. I don't think this particular example shows anything one way or another.

The bit about buying the shirt was obviously nonsense though.

Re: GPT-3 has no idea what it’s talking about

#252

Post by echelon »

BrokrnAlgorithm wrote: post #250
echelon wrote: post #246

Earlier quoted context omitted.

I wonder if you can get GPT-3 bots to spam Reddit, Twitter, and Facebook into oblivion. I also wonder what percentage of users would notice. Give them a political bent - that's probably what the state actors are trying to productionize right now. Target posts with a sentiment that disagrees with yours, then make the bots follow those users and inundate them with replies wherever they go. Hell, even brands might step…

I've been wondering about the exact same thing. Basically, a sufficiently calibrated and targeted GPT3 bot swarm could be employed to render at least some parts of these communities into useless echo chambers. I think that while GPT3 posts are usually identifiable after reading some sentences, I often also find that it's harder to realize when consuming precisely this kind of social media. I often just consume it abs…

With most comments being a paragraph or less, I think social media is done for unless it finds a way to put up a wall.

One thought is the subscription model of older social media sites, such as Something Awful. Pay $5 to get an account. Break the rules, get banned, pay another $5.

That's a lot of friction for your most valuable consumers, though. Marketers want users that are less discriminating.

Another is a shift to video and multimedia-based social networking. TikTok, Instagram, Twitch. It doesn't save the comments, though. I'm not sure how long we've got before the videos themselves can be generated with a high degree of novelty.

AI is going to change the game significantly. We're perfectly timed for a technological change of winds that enables new upstarts to challenge the incumbents. I'm kind of excited.

Re: GPT-3 has no idea what it’s talking about

#253

Post by notriddle »

pintxo wrote: post #209
rytill wrote: post #120

Earlier quoted context omitted.

I’m not implying it can’t! It might be able to in many cases, if you do prompt design right and fine-tune.

Sure, as in: if the spec is flawless, we can source out the coding to a bunch of minimum wage dudes in {location of your choice}. Anyone happy with this approach?

Apparently, lots of businesses are happy with this approach. They keep using it.

Re: GPT-3 has no idea what it’s talking about

#254

Post by MayeulC »

pintxo wrote: post #209
rytill wrote: post #120

Earlier quoted context omitted.

I’m not implying it can’t! It might be able to in many cases, if you do prompt design right and fine-tune.

Sure, as in: if the spec is flawless, we can source out the coding to a bunch of minimum wage dudes in {location of your choice}. Anyone happy with this approach?

That kind of argument has been made many times: a flawless spec is already code.

Re: GPT-3 has no idea what it’s talking about

#255

Post by ma2rten »

echelon wrote: post #246
throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

I wonder if you can get GPT-3 bots to spam Reddit, Twitter, and Facebook into oblivion. I also wonder what percentage of users would notice. Give them a political bent - that's probably what the state actors are trying to productionize right now. Target posts with a sentiment that disagrees with yours, then make the bots follow those users and inundate them with replies wherever they go. Hell, even brands might step…

The GPT-3 paper goes into this. They consulted experts:

The assessment was that language models may not be worth investing significant resources in [by state actors] because there has been no convincing demonstration that current language models are significantly better than current methods for generating text, and because methods for “targeting” or “controlling” the content of language models are still at a very early stage.

The paper has more details.

Re: GPT-3 has no idea what it’s talking about

#256

Post by Miraste »

wombatmobile wrote: post #245
amelius wrote: post #238

Earlier quoted context omitted.

> For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.

Entertainment. e.g. $3 per month for My Virtual Friend. Before you scoff, consider that Pet Rocks were once a (profitable) thing. Eliza, despite its limitations, sparked considerable engagement with those who were willing to chat with it at length without derailing it. https://qz.com/1439200/loneliness-costs-the-us-almost-7-bill...

Eliza comes up a lot in these discussions. I’d like to know if would have the same results today, now that chatbots, video games, phone robots, etc. have given millions of people the mental model of getting responses from a box with no thoughts. I expect it would very different from running the experiment at a time when the only things that could talk back when talked to were other people.

Re: GPT-3 has no idea what it’s talking about

#257

Post by zatel »

turing_complete wrote: post #226

"At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, which has become a major cremation provider with 145 locations." I mean that's just brilliant…

+1 this could easily be the story line of a Rick and Morty episode or any of the other similar off the cuff shows that are popular right now. I think that will be one of the main profit streams for things like this, you can get the weird wild stories that don't really make sense but are interesting enough that who cares and you don't have to associate your network with eccentric individuals that attract malcontents.

Re: GPT-3 has no idea what it’s talking about

#258

Post by IanAWP »

mannykannot wrote: post #227
hackinthebochs wrote: post #64

Earlier quoted context omitted.

>If it did have a real understanding, prompt construction wouldn't matter as much This is only true if we assume GPT was never trained on satire or intentionally absurd text. But there's no reason to think this. Because it continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. If you treat GPT as a conversation bot and expect it to call you out when you give it a…

I think the point here is that it hs no understanding of what you are calling a 'bad' prompt. That does not mean that it lacks any understanding whatsoever (personally, I think it has made a small step in that direction), but it is a valid case of it not understanding something . Furthermore, I do not think we can assume that if it were trained on examples of satire or intentionally absurd text, it would perform bett…

GPT is both general purpose and stateless. Lacking another mechanism, it's the requirement of the prompt to request a particular genre of response if that's what you want.

There can be no such thing as a bad prompt from gpt3's perspective. A bad prompt is one where the user has a specific purpose which is not expressed. It's bad because you know beforehand that gpt3 can not align with it.

Someone pours grape juice into a bottle and becomes worried that is not safe to drink. GPT3 correctly grasps that there is a hidden context to this weird prompt, however when given no other information it guesses that this hidden context is something known only to the hypothetical character in the prompt. I would probably do this too.

When you give it the correct context (this weird prompt is a logic test) then it gives you the answer you expected.

Re: GPT-3 has no idea what it’s talking about

#259

Post by CamperBob2 »

echelon wrote: post #246
throwawaygh wrote: post #113

Some of the criticism in this comment section is completely fair — the authors are providing exactly the type of prompts that GPT-3 breaks down on and some of these examples might be cherry-picked continuations. And the authors do have personal interests at stake. (NB, the exact same criticism is true about a lot of articles lauding GPT-3, which is why public discussion of GPT-3 in general is such a dumpster fire.) S…

I wonder if you can get GPT-3 bots to spam Reddit, Twitter, and Facebook into oblivion. I also wonder what percentage of users would notice. Give them a political bent - that's probably what the state actors are trying to productionize right now. Target posts with a sentiment that disagrees with yours, then make the bots follow those users and inundate them with replies wherever they go. Hell, even brands might step…

Very true. An example I just saw (and reported) of something that's about to get much worse:

https://rennlist.com/forums/987-981-forum/1177333-best-oil-f...

("I have Tucson 2.0 and I can say that the mechanisms are as similar as possible and it was also very difficult to pick up oil, because it is also old. That is, hard oil will not fit there, maybe even spoil the engine and transmission. Therefore, it is better not to take risks and take exactly those oils that are best suited to such types of cars and engines. You can also read about the best oil for tucson 2.0 (spam link redacted), there are different variants of such machine oil described in details. I personally took Mobil 1 Extended Performance 5W-20 Full Synthetic Motor Oil and I do not regret, because of its full synthetics it is quite loyal to the engine. Therefore, it is ideal if you have an old car, but in any case, study the top and then already select from it.")

This is pre-GPT quality material, and it's already good enough to hang around unreported for days in some cases. A lot will depend on how much effort forums are willing to put into vetting new users.

Re: GPT-3 has no idea what it’s talking about

#260

Post by sitkack »

rirarobo wrote: post #231
monkpit wrote: post #197

Earlier quoted context omitted.

Is your argument that a human can write better prompted fiction than gpt3?

No, I was just addressing the claim regarding the prompt itself, that "stirring lemonade with a cigarette is so batshit insane that there really can't be a sensible continuation". That seemed like a fun challenge to me. Sorry I got a little carried away trying to come up with an interesting continuation!

I’d like to secure the rights to an adaptation of your continuation
Post reply on HN