Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-Neo – Building a GPT-3-sized model, open source and free

eleuther.ai

Post reply on HN
Print view View original

101–110 of 260 posts

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#101

Post by visarga »

qayxc wrote: post #92
taneq wrote: post #14

Earlier quoted context omitted.

The fake news thing is a real problem (and may become worse under GPT3 but certainly exists already). As for the others - to quote Westworld, "if you can't tell the difference, does it really matter?"

> "if you can't tell the difference, does it really matter?" It indeed does. The problem is that societies and cultures are heavily influenced and changed by communication, media, and art. By replacing big portions of these components with artificial content, generated from previously created content, you run the risk of creating feedback cycles (e.g. train future systems from output of their predecessors) and formin…

This problem affects all types of agents - natural or artificial. Agent acts in the environment, this causes experience and learning, and thus conditioning the future. The agent has no idea what other opportunities are lost behind past choices.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#102

Post by throwaway_6142 »

Lucasoato wrote: post #88
hooande wrote: post #66

In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…

> I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling them?". And honestly, not a lot of productive applications come to mind. Damn, this could replace so many programmers, we're doomed!

*realizing GPT-3 was probably created by programmers who's job really is mostly googling for stackoverflow answers*

#singularity

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#103

Post by ccozan »

mekkkkkk wrote: post #80
hooande wrote: post #58

Earlier quoted context omitted.

why does this matter? the internet is filled with millions of very low quality human generated discussions right now. There might not be much of a difference between thousands of humans generating comment spam and thousands of gpt-3 instances doing the same

It does matter. The nice feeling of being one of many is a feature of echo chambers. If you can create that artificially for anything with a push of a button, it's a powerful tool to edit discourse or radicalize people. Have a look at Russias interference in the previous US election. This is what they did, but manually. To be able to scale and automate it is huge.

But careful, the human psyche has some kind of tipping point. Too much fake news, and it will flip. Too less, no real influence is made.

The exact balance must be orchestrated by a human.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#104

Post by Kuinox »

dkjaudyeqooe wrote: post #44
anticristi wrote: post #3

I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…

I know! One day it's going to get so bad people are going to have to deploy critical thinking instead of accepting what they read at face value and suffer the indignity of having to think for themselves.

Sadly they don't. They start to believe to random things in and reject what they don't like.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#105

Post by b3kart »

littlestymaar wrote: post #91
b3kart wrote: post #71

Earlier quoted context omitted.

I buy the former, researchers might be happier knowing their work potentially benefits all of humanity, not just a bunch of investors. But wouldn’t it be more difficult to get funding as a non-profit?

It's just never going to be difficult to get funding when you have Elon Musk and Sam Altman as founders (and even more so when founders put one billion of their own money into it).

Sure, but that's OpenAI's particular set of circumstances. Generally speaking I struggle to see investors preferring a nebulous non-profit over a for-profit with a clear path to market.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#106

Post by pydry »

visarga wrote: post #95
hooande wrote: post #66

In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…

> honestly, not a lot of productive applications come to mind Not so convincing when you enumerate so many applications yourself. > but won't be able to say anything about current events There are variants that use transformer + retrieval, so they got unlimited memory that can be easily extended.

Content farms are hardly a productive application.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#107

Post by astrange »

2Gkashmiri wrote: post #10

So how much money would it take to rebuild this foss alternative ? And distributive power like seti@home? If it can be done and I hope it does, what benefit would the original proprietary one have over this? Licensing?

OpenAI will execute the original one for you. If you can get an account, anyway.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#109

Post by littlestymaar »

b3kart wrote: post #105
littlestymaar wrote: post #91

Earlier quoted context omitted.

It's just never going to be difficult to get funding when you have Elon Musk and Sam Altman as founders (and even more so when founders put one billion of their own money into it).

Sure, but that's OpenAI's particular set of circumstances. Generally speaking I struggle to see investors preferring a nebulous non-profit over a for-profit with a clear path to market.

Sure, but we're explicitly talking about OpenAI here.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#110

Post by pessimizer »

Closi wrote: post #86
dvfjsdhgfv wrote: post #75

Earlier quoted context omitted.

It will be interesting to see the attitude of Microsoft towards this project in the light of their "Microsoft loves open source" propaganda.

I don't know where people got this idea that Microsoft can't participate positively in Open Source, and do that sincerely, without open sourcing absolutely everything. Of course they can - just because you contribute to open source, and do that because you also benefit from open source projects, doesn't mean you have to do absolutely everything under open source. Especially considering OpenAI isn't even Microsoft's I…

I don't know where people got the idea that companies can be "sincere." Sincerity is faithfully communicating your mental state to others. A company's mental state can change on a dime based on the decisionmaking of people who rely on the company for the degree of profit it generates. Any analog to sincerity that you think you see can probably be eliminated by firing one person after an exceptionally bad quarter (or an exceptionally good one.)
Post reply on HN