Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

401–410 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#401

Post by [deleted] »

pjc50 wrote: post #370
thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

> Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread. This kills the forum. Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless instead of just mostly pointless. You could potentially convince a human; you can't convince a neural net that isn't in learning mode.

[deleted]

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#402

Post by [deleted] »

richk449 wrote: post #265
nullc wrote: post #157

Earlier quoted context omitted.

I write a prompt, often copying text from the articles or other comments, and have it generate a lot of completions. I skim over the completions and grab interesting parts. For example, complaining about "quoted text" in my above comment was GPT2's suggestion (and also an actual issue with GPT2 which it was exhibiting by producing that text). Actually, all the text above from "There are a few" and beyond were written…

Pretty much everyone agrees that these models, no matter how good they are at writing, don’t understand what they are writing, and can’t develop new ideas, only combine existing ideas together. Given that, it’s seems that what you are doing adds only noise, and no value, to HN. Of course, that is probably also true of a large fraction of pure-human posters ...

[deleted]

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#403

Post by rapnie »

luckylion wrote: post #342
rapnie wrote: post #289

Earlier quoted context omitted.

It will be very annoying for e.g. forum moderators to determine whether first user posts are just a bit incoherent, or generated spam garbage.

That used to be a pretty annoying thing back in the days of IRC as a kind of DOS: run a bunch of bots that just replay a conversation from another channel. Engaging them fails, but is that because they are bots, or because they're just ignoring you?

The new kind can be more targeted for specific purposes. They could be excellent tools for trolling a forum, inciting flame wars and such.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#404

Post by sytelus »

ypcx wrote: post #124
jcahill wrote: post #53

Earlier quoted context omitted.

GPT-3 is a neat party trick. But the things that'll be done with web archives* in the next 20y will make it look like the PDP-8. ~love, a web archivist * GPT-3 is trained on one

The transformer model as presented in GPT-3 may be a few tweaks away from a human-acceptable reasoning, at which point we may realize that human brain is just a neat party trick as well. This may come difficult for some people to internalize, especially those who understand the technology in depth. Because it means that the medium of our reality is the consciousness.

You are confusing pattern matching with reasoning. If your brain was replaced by GPT-3 model and you were cast away on a distant island, I highly doubt you will be able to perceive, plan and prosper during your survival against all the calamity nature would through at you.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#405

Post by heavenlyblue »

amelius wrote: post #393
pjc50 wrote: post #370

Earlier quoted context omitted.

> Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread. This kills the forum. Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless instead of just mostly pointless. You could potentially convince a human; you can't convince a neural net that isn't in learning mode.

Perhaps it will destroy anonymity. Because the only way to be sure a human wrote something is if you somehow know who the comment came from. We might end up with reputation based conversations.

You can still be anonymous with this. You just need to be pseudonymous.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#406

Post by stared »

"...since Bitcoin" - I hope not.

Bitcoin is still a speculative technology, with value based on speculation. It might become useful (or not).

NLP is being used right now for practical, commercial tasks. Advancement in NLP is going to serve practical purposes now, with potential for further expansion.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#407

Post by dannyw »

vladTheInhaler wrote: post #398
Alex3917 wrote: post #114

Earlier quoted context omitted.

> I'm sure it can be influenced easily. Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are…

I think the issue is that text doesn't exist in a vacuum, but the corpus that the model is learning from does. A piece of human writing exists for a particular reason - to persuade, to inform, to ask a question, etc - and its value is judged on its ability to perform that task. But that's not a quality that is evident from the text itself, only from looking at the world outside the text. This suggests to me some limi…

It could also be a result of training data. If every page is weighted equally, you'd expect SEO spam and even autogenerated content to far surpass high quality content in volume.

I would like to see a GPT model where training data is weighted by credibility / authority (e.g. using Pagerank).

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#408

Post by amelius »

heavenlyblue wrote: post #405
amelius wrote: post #393

Earlier quoted context omitted.

Perhaps it will destroy anonymity. Because the only way to be sure a human wrote something is if you somehow know who the comment came from. We might end up with reputation based conversations.

You can still be anonymous with this. You just need to be pseudonymous.

Robots can also have a pseudonym.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#409

Post by spion »

setzer22 wrote: post #275
thewarrior wrote: post #151

Earlier quoted context omitted.

The thing is that this thing has now crossed into the uncanny valley. Earlier it would have great trouble making a single sentence that makes sense. You only ever remember whether the last two sentences made sense and go together. And with GPT-3 any pair of sentences always makes almost perfect sense. By the time you're four sentences down you go wait a minute ...

This was very apparent when reading the generated stories [1]. Especially the shoggoth cat dialogue, I found that one really creepy. The fragment below comes straight out from the uncanny valley: Human: Those memes sound funny. But you didn’t include any puns. So tell me, what is your favorite cat pun? AI: Well, the best pun for me was the one he searched for the third time: “You didn’t eat all my fish, did you?” You…

An instance of a Voight-Kampff test.
Post reply on HN