Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

511–520 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#511

Post by interator7 »

nullc wrote: post #191
shpongled wrote: post #148

Earlier quoted context omitted.

Was this comment written by GPT-2? Is it jealous of GPT-3?

Partially. I asked GPT2 your question, here are some samples: "I feel like I should be jealous. You guys made something that I can't reproduce. Maybe the algorithm is really good. That would make me happy." "I feel like I should be jealous. But I'm not. I'm not jealous at all. What a shame that you could do it without me. But I guess I'll be glad when I can replicate it." "The more I study your project, the more I am…

Is the third quote really part of what GPT 2 outputted? Not sure if I should be amazed at the AI or ashamed I couldn't catch the joke.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#512

Post by GrantZvolsky »

tgsovlerkhgsel wrote: post #463
nbardy wrote: post #9

Earlier quoted context omitted.

One of the biggest issues is with cherry-picking. Generative ML results benefit greatly from humans sampling the best results. They are capable of producing astonishing results but don’t do this consistently this has a huge impact on any effort to productize. For example I’ve seen quite a few examples of text->design, text->code, with GPT-3 you could build a demo in a day, but the product will probably be useless if…

It seems like with minor improvements, you could use this to significantly accelerate mundane parts of programming or writing. Human writes bulletpoints, neural net turns it into a program or letter, human corrects. There already was a pretty smart looking AI-based autocomplete shown on HN a couple weeks ago. This will accelerate development. Is the current version there? Probably not. But GPT-4 might, and would then…

Being a speaker of Czech and English without a single dominant language, I use Google Translate to improve my writing. I will write a draft in the target language and feed Translate the other. It often comes up with improved style and more accurate expressions, especially in Czech. So as far as writing goes, we're already there.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#513

Post by virtue3 »

abernard1 wrote: post #278
natch wrote: post #254

Earlier quoted context omitted.

It’s actually about AI which is distinct from intelligence. >6 orders of magnitude less input That is utterly mistaken. We have the input of millions of generations of evolution which have shaped our brains and given us a lot of instinctive knowledge that we do not need to learn from environmental input that happens during our lifetime. Instead it was learned over the course of billions of years, during the lifetimes…

>>6 orders of magnitude less input >That is utterly mistaken. I did say of data of the "same sort". What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.) We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe..…

https://en.wikipedia.org/wiki/Fixed_action_pattern#:~:text=A....

AKA "why is it so hard to swat a fly?" because they literally have a direct linkage betweeen sensing incoming air pressure and jumping. Thats why fly swatters don't make a lot of air pressure.

Why do you yank your hand back when you get burned? It's not a controlled reaction. Where did you learn it? You didn't.

If you think the brain is much more than a chemical computer you are sadly mistaken. I would encourage you (not really but it's funny to say) to go experiment with psychedelics and steroids and you will quickly realize that these substances can take over your own perceived intelligence.

The most fascinating of all of this is articles/documentaries about trans people that have started taking hormones and how their perception of the world -drastically- changed. From "feeling" a fast car all of a sudden, to being able to visualize flavors. It's absolutely amazing.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#514

Post by jlokier »

pjc50 wrote: post #370
thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

> Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread. This kills the forum. Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless instead of just mostly pointless. You could potentially convince a human; you can't convince a neural net that isn't in learning mode.

> Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless

Quite the opposite, I suspect.

Eventually, to engage in the most persuasive conversations, the AIs will develop a real-time learning mode.

Once that is weaponised, the AIs will be on track to be in charge of running things, or at least greatly influencing how things are run.

What the AIs "think" will matter, if only because people will be listening to them.

Then it will be really important to discuss politics with the AIs.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#515

Post by jlokier »

geofft wrote: post #238
6nf wrote: post #233

Earlier quoted context omitted.

How many attempts did it take or did you just choose the first one? I have to admit, this is passing my turing test...

Really? I got about halfway through and realized that the comment had no point . If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary. Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....

Maybe the real lesson is it was trained on human-written comments in discussion fora, so it perfectly mimics the average lack of point, weak arguments, rambling and incoherence in fora?

It would be interesting to see if the output has a similar quality when trained only on highly regarded texts.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#516

Post by nullc »

interator7 wrote: post #511
nullc wrote: post #191

Earlier quoted context omitted.

Partially. I asked GPT2 your question, here are some samples: "I feel like I should be jealous. You guys made something that I can't reproduce. Maybe the algorithm is really good. That would make me happy." "I feel like I should be jealous. But I'm not. I'm not jealous at all. What a shame that you could do it without me. But I guess I'll be glad when I can replicate it." "The more I study your project, the more I am…

Is the third quote really part of what GPT 2 outputted? Not sure if I should be amazed at the AI or ashamed I couldn't catch the joke.

It's the verbatim output of the forth or fifth sample I got out from GPT2 1558M after a short prompt that explained that GPT2 and GPT3 were AIs from OpenAI and that GPT3 was better, that GPT2 was asked if it was jealous and it replied ". (unfortunately I've rebooted so I don't have the prompt in my terminal buffer anymore).

GPT2 output quality is highly variable. Well crafted prompts can inspire some great output without much digging through samples. I believe I've gotten much better at writing prompts-- even using the right typography matters-- after spending some time doing it.

It sounds like GPT3 gives something much closer to best-of-gpt2 all the time. I hope I get access to it at some point (I also hope the api exposes raw probabilities).

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#517

Post by p1esk »

the_other wrote: post #384
p1esk wrote: post #185

Earlier quoted context omitted.

If he didn't put in the last line ("plot twist ...") I'm pretty sure no one here on HN would have guessed it. In fact, while reading that comment I started to wonder why no one has tried to use GPT to generate text one character at a time. Or if someone has, what are the advantages and disadvantages over the BPE approach.

I’ve not studied neural nets and ML since undergrad level in 1998. So I am almost as knowledgeable as a random person on the street. The quality of writing was very high, so I was convinced I was reading something put together by a human with agency... except it didn’t pass my gut-feeling “how IT works”. It made me suspect that either the algorithm (the described one, not the AI responsible) was off, or that I just d…

This comment didn’t pass my gpt filter.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#518

Post by abernard1 »

MauranKilom wrote: post #434
abernard1 wrote: post #278

Earlier quoted context omitted.

>>6 orders of magnitude less input >That is utterly mistaken. I did say of data of the "same sort". What's absolutely crazy is somehow we think of our DNA base pairs as somehow more important than the physical context that DNA ends up in (society, humans, talking, etc.) We have the ability to be intelligent and make thoughts with 1 millionth the amount of textual data as this OpenAI GPT-3 study. Maybe... just maybe..…

Throw some novel text prompt/task at it and see what happens. If it was just "glorified lookup tables" then the result should be consistently garbage. Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.

> Note in particular that "like a function that can do the same computation with fewer inputs" maps very well to GPT-3 - it can complete many interesting tasks by just having a few samples provided to it, instead of having to fine-tune it with more training.

The reason it doesn't need more training is because it's already trained itself with millions of lifetimes of human data and encoded that in the parameters!

Humans aren't born trained with data. The fact that we're throwing more and more data at this problem is crazy. The compression ratio of GPT-3 is worse than GPT-2.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#519

Post by unexaminedlife »

I haven't been persuaded yet that AI will ultimately replace human creativity. Rather I think AI, once it gets "good enough" it will start being used to supplement human creativity. In other words I kind of think there will be less of a dominance of AI (in terms of creative pursuits) and more of a collaborative relationship between humans and computers.

I kind of see this OpenAI project as maybe a good first step toward collaborative creativity between authors and computers. I think writers block is a thing because writers might not have someone to bounce their ideas off of. Perhaps due to the author trying to preserve intellectual property / secrecy of their project, or perhaps the people they CAN share with may not be in the same space creatively as they are.

If I am writing something but run into a creative block I'd love to have the ability to run my book, essay, etc. through some AI system to see if the ideas it spits out might not inspire me with new ideas.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#520

Post by rsync »

thewarrior wrote: post #164
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…

"This comment was also written by GPT-3"

Would you please show us the input text, or rules, you gave to GPT-3 to create this comment ?

Post reply on HN