Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-Neo – Building a GPT-3-sized model, open source and free

eleuther.ai

Post reply on HN
Print view View original

11–20 of 260 posts

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#11

Post by technocratius »

Erlich_Bachman wrote: post #6
anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

Photoshop has existed for decades. Is it really that big of a problem for photo news?

The difference between Photoshop and generative models is not in what it can technically achieve, but the cost of achieving the desired result. Fake news photo or text generation is possible by humans, but scales poorly compared (more humans) to a algorithmically automated process (some more compute).

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#12

Post by draugadrotten »

Erlich_Bachman wrote: post #6
anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

Photoshop has existed for decades. Is it really that big of a problem for photo news?

Yes!

https://en.wikipedia.org/wiki/Adnan_Hajj_photographs_controv... https://www.bbc.com/news/world-asia-china-55140848

and many more

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#13

Post by yread »

turing_complete wrote: post #9
anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

Don't read news. Go to original sources and scientific papers. If you really want to understand something, a news website should only be your starting point to look for keywords. That is true today as it will be "post-GPT-3".

Primary sources need to be approached with caution https://clas.uiowa.edu/history/teaching-and-writing-center/g...

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#14

Post by taneq »

anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

The fake news thing is a real problem (and may become worse under GPT3 but certainly exists already). As for the others - to quote Westworld, "if you can't tell the difference, does it really matter?"

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#15

Post by normanmatrix »

anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

You will not. Welcome to the scary generative future.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#16

Post by Agentlien »

anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

Deep fakes still feel quite uncanny valley to me. Even if they move beyond that convincing fake images have existed for a long while.

As for support, I don't really see why it matters if I'm talking to a clever script or an unmotivated human.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#17

Post by danielscrubs »

Erlich_Bachman wrote: post #6
anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

Photoshop has existed for decades. Is it really that big of a problem for photo news?

Touch ups where done before photoshop but now it’s ALWAYS done. The issues this has created in society might have a bigger emotional impact than we give it credit for.

Regarding photo news there has been quite a lot of scandals to the point that I’d guess the touchups is more or less accepted.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#18

Post by savolai »

turing_complete wrote: post #9
anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

Don't read news. Go to original sources and scientific papers. If you really want to understand something, a news website should only be your starting point to look for keywords. That is true today as it will be "post-GPT-3".

This scales badly today and will scale even worse in the future. Those without education or time resources will at best manage to read the news. Humanity will need a low effort way to relay reliable information to more of it's members.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#19

Post by wraptile »

Erlich_Bachman wrote: post #4

It's a shame that it has turned out to be necessary to externally re-make and re-train a model that has come out of company called `OPEN`AI. Wasn't one of the founding principles of it that all of the research would be available to the public? Isn't that the premise on which the initial funding was secured? Best of luck to Eleuther.

OpenAI turning out to be a total bait and switch. Especially true when your co-founder is actively calling you out on it[1]

Remember kids: if it's not a non-profit organization it is a _for_ profit one! It was silly to expect anything else:

> In 2019, OpenAI transitioned from non-profit to for-profit. The company distributed equity to its employees and partnered with Microsoft Corporation, who announced an investment package of US$1 billion into the company. OpenAI then announced its intention to commercially license its technologies, with Microsoft as its preferred partner [2]

1 - https://edition.cnn.com/2020/09/27/tech/elon-musk-tesla-bill...

2 - https://en.wikipedia.org/wiki/OpenAI

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#20

Post by JacobiX »

Erlich_Bachman wrote: post #4

It's a shame that it has turned out to be necessary to externally re-make and re-train a model that has come out of company called `OPEN`AI. Wasn't one of the founding principles of it that all of the research would be available to the public? Isn't that the premise on which the initial funding was secured? Best of luck to Eleuther.

It's a recurring theme in OpenAI research, they become more and more closed. For instance their latest model called DALL·E hit the headlines before the release of the paper. Needless to say, the model is not available and no online demo has been published so far.
Post reply on HN