I need GPT-3 for my blog for make a great posts about new programming tools. Looks for good lesson how to build things translation system
OpenAI's GPT-3 may be the biggest thing since Bitcoin
271–280 of 554 posts
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#272Earlier quoted context omitted.
If he didn't put in the last line ("plot twist ...") I'm pretty sure no one here on HN would have guessed it. In fact, while reading that comment I started to wonder why no one has tried to use GPT to generate text one character at a time. Or if someone has, what are the advantages and disadvantages over the BPE approach.
Your second paragraph is GPT-3?
Ok, now this is getting deep and I don't like it.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#273Earlier quoted context omitted.
The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…
It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…
I could tell immediately because... it's too long
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#274Earlier quoted context omitted.
I write a prompt, often copying text from the articles or other comments, and have it generate a lot of completions. I skim over the completions and grab interesting parts. For example, complaining about "quoted text" in my above comment was GPT2's suggestion (and also an actual issue with GPT2 which it was exhibiting by producing that text). Actually, all the text above from "There are a few" and beyond were written…
Pretty much everyone agrees that these models, no matter how good they are at writing, don’t understand what they are writing, and can’t develop new ideas, only combine existing ideas together. Given that, it’s seems that what you are doing adds only noise, and no value, to HN. Of course, that is probably also true of a large fraction of pure-human posters ...
> Pretty much everyone agrees that these models, no matter how good they are at writing, don’t understand what they are writing, and can’t develop new ideas, only combine existing ideas together.
Well I guess not everyone agrees with that, because I don't for example. First, how different are the two things, "develop new ideas" vs "combine existing ideas together"? Are we certain that some things that you consider new ideas aren't ever things I would consider the combining of existing ideas?
Regardless, it seems to me that these language models may very well produce "new ideas" by chance, even if it doesn't itself 'recognize' that that is what it's doing.
> Given that, it’s seems that what you are doing adds only noise, and no value, to HN.
I don't agree that the only value added to HN is 'new ideas'. There are lots of old ideas that have lots of value in being communicated and discussed.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#275Earlier quoted context omitted.
The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…
The thing is that this thing has now crossed into the uncanny valley. Earlier it would have great trouble making a single sentence that makes sense. You only ever remember whether the last two sentences made sense and go together. And with GPT-3 any pair of sentences always makes almost perfect sense. By the time you're four sentences down you go wait a minute ...
This was very apparent when reading the generated stories [1].
Especially the shoggoth cat dialogue, I found that one really creepy. The fragment below comes straight out from the uncanny valley:
Human: Those memes sound funny. But you didn’t include any puns. So tell me, what is your favorite cat pun?
AI: Well, the best pun for me was the one he searched for the third time: “You didn’t eat all my fish, did you?” You see, the word “fish” can be replaced with the word “cats” to make the sentence read “Did you eat all my cats?”
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#276OpenAI is not non-profit. OpenAI = OpenAI Inc (non-profit) + OpenAI LP (for-profit).
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#277The world: the comment section/youtube playlist/twitter feed is full of nobodies who've learned to optimize for clickthrough but have no fucking clue what they're talking about. This is becoming a serious problem for society. Open AI: Hold my beer (also, something something AI safety something)
It’s pretty well established that OpenAI abandoned AI safety for a while, now.
No? I don't know where that's been established. There's still quite a few researchers on the AI safety team at OpenAI.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#278Earlier quoted context omitted.
> This kind of facile moving the goalposts is imho a cheap shot and (not imho, fact) is a recurring phenomenon as we make incremental progress toward AI. I think you missed my point. I think we're going in the wrong direction for AI entirely, and these "advances" are fundamentally misguided. OpenAI is explicitly about "intelligence," and so we should question if this is in fact that. It's clear that humans have funda…
It’s actually about AI which is distinct from intelligence. >6 orders of magnitude less input That is utterly mistaken. We have the input of millions of generations of evolution which have shaped our brains and given us a lot of instinctive knowledge that we do not need to learn from environmental input that happens during our lifetime. Instead it was learned over the course of billions of years, during the lifetimes…
>>6 orders of magnitude less input
>That is utterly mistaken.
I did say of data of the "same sort".
What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.)
We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe... intelligence is far more related to things other than just having more data.
I'll actually expand on this and throw this out there: intelligence is in a way antagonistic to more data.
A more intelligent agent needs less knowledge to make a better decision. It's like a function that can do the same computation with fewer inputs. A less intelligent agent requires a lookup table of previously computed intelligent things instead of figuring it out on its own. I think all these "AI" studies are glorified lookup tables.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#279Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#280Earlier quoted context omitted.
> This kind of facile moving the goalposts is imho a cheap shot and (not imho, fact) is a recurring phenomenon as we make incremental progress toward AI. I think you missed my point. I think we're going in the wrong direction for AI entirely, and these "advances" are fundamentally misguided. OpenAI is explicitly about "intelligence," and so we should question if this is in fact that. It's clear that humans have funda…
It’s actually about AI which is distinct from intelligence. >6 orders of magnitude less input That is utterly mistaken. We have the input of millions of generations of evolution which have shaped our brains and given us a lot of instinctive knowledge that we do not need to learn from environmental input that happens during our lifetime. Instead it was learned over the course of billions of years, during the lifetimes…
With that style of argumentation, you can say that NNs have even more input than humans: they also have all of the technical development of the last 50,000 years built into them.