If they succeed, Eleuther should change their name to ReallyOpenAI.
Or for extra irony, ClosedAI
GPT-Neo – Building a GPT-3-sized model, open source and free
181–190 of 260 posts
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#182In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
Have you tried conversing with it, after a few lines of setting a proper context? Like two scientist talking or something like that. It can provide very interesting outputs that are not googlable.
Yes, every time you see something that for human obviously doesn't make sense it makes you dismiss it. You would look at that output differently though if you were talking with a child. Just like a child can miss some information making it say something ridiculous it may miss some patterns connections.
But have you ever observed carefully how we connect patterns and make sentences? Our highly sophisticated discussions and reasoning is just pattern matching. Then most prominent patterns ordered in time also known as consciousness.
Watch hackernews comments and look how after somebody used a rare adjective or cluster of words more commenters tend to use it without even paying conscious attention to that.
Long story short, give it a try and see what examples of what people already did with it even in it's limited form.
To me you are looking at an early computer and saying that it's not doing anything that a bunch of people with calculators couldn't do.
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#183Earlier quoted context omitted.
But why on this thread then, about GPT-3? It's not even their own company, IP or source to give away. But even when Microsoft can't open source it because it's not theirs , we still have people posting in this thread that this is further evidence that Microsoft is hypocritical. It sounds a lot like a form of Confirmation Bias to me where any evidence is used as proof that Microsoft is 'anti-open-source'.
I think it is because each model from OpenAi was public until Microsoft became an investor.
[deleted]
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#184I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…
The fake news thing is a real problem (and may become worse under GPT3 but certainly exists already). As for the others - to quote Westworld, "if you can't tell the difference, does it really matter?"
If you ask GPT-3 for three Lord of the Rings quotes it might give you two real ones and one fake one, because it doesn’t know what truth is and just wants to give you something plausible.
There are creative applications for bullshit, but something that cites its sources (so you can check) and doesn’t hallucinate things would be much more useful. Like a search engine.
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#185Once Microsoft became the sole and exclusive licensee of GPT-3, a credible open source effort was bound to emerge, and I hope it really is an effective alternative.
Eleuther works with Google. I'd rather prefer the Microsoft demon rather than the Google Demon.
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#186I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…
Don't read news. Go to original sources and scientific papers. If you really want to understand something, a news website should only be your starting point to look for keywords. That is true today as it will be "post-GPT-3".
Most people are not smart enough to do this, and even if they are, they don't have enough time in their day.
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#187Earlier quoted context omitted.
The fake news thing is a real problem (and may become worse under GPT3 but certainly exists already). As for the others - to quote Westworld, "if you can't tell the difference, does it really matter?"
What scares me personally is the idea that I might be floating in a sea of uncanny valley content. Content that's 98% human-like, but then that 2% sticks out like a nail and snaps me out of it. Sure, I might not be able to tell the difference the majority of the time, but when I can tell the difference it's gonna bother me a lot.
To me, a lot of content seems to be digital marketing horseshit tbh.
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#188Earlier quoted context omitted.
I feel your stance [1] is demonstrably false in two challenges. 1) Please play a winning game of Go against Alpha Zero, just by googling the topic. 2) Next please explain how Alpha Zero’s game’s could forever change Go opening theory[2], without any genuine creativity. [1] that “the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top result…
GPT-3 can’t play Go.
It almost definitely can to some extent given that gpt2 could play chess [0].
0. https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-...
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#189In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
Did anyone try to ask GPT-3 something about the insurrection? I can't find any free/open text prompts for GPT-3, this is what GPT-2 has to say: After Trump lost the 2020 election his supporters stormed the Capitol > They smashed windows, tore apart the Republican Party's headquarters, destroyed a statue of Ronald Reagan and set fire to the Texas statehouse. > A month later, after the Democrats took control of the Sen…
You can play with GPT3 in a custom world at AIdungeon.io The responses are biased towards giving you RPG second person narrative, but the corpus of data, mastery of syntax and more uncertain grasp of events and relationships is all there
Example with the prompt You are Donald Trump. The recent election results have been a disappointment to you.
Props for turns of phrases like "Your opponent is a typical liberal. He hails from the right wing of the Democratic Party, but has been trying to appeal to the left to gain more support.", but poor marks for apparently not having grasped how elections work. (There's a joke in there somewhere)
If you don't pick a custom world and your own prompt, you get something more like this:
> You are Donald Trump, a noble living in the kingdom of Larion. You are awakened by a loud noise outside the gate. You look out the window and see a large number of orcish troops on the road outside the castle.
I'd like 'orcish troops' better if I thought it was inspired by media reports of Capitol events rather than a corpus of RPGs.
Re: GPT-Neo – Building a GPT-3-sized model, open source and free
#190It's a shame that it has turned out to be necessary to externally re-make and re-train a model that has come out of company called `OPEN`AI. Wasn't one of the founding principles of it that all of the research would be available to the public? Isn't that the premise on which the initial funding was secured? Best of luck to Eleuther.
It's a recurring theme in OpenAI research, they become more and more closed. For instance their latest model called DALL·E hit the headlines before the release of the paper. Needless to say, the model is not available and no online demo has been published so far.
Because its winner-take-all in this research, not "winner-take-some".
Andrew Yang talked about this and why breaking up Big Tech won't work. No one wants to use the second best search engine. The second best search engine is Bing and I almost never go there.
Tech isn't like automobiles, where you might prefer a Honda over a Toyota, but ultimately they're interchangeable. A Camry isn't dramatically different and doesn't perform dramatically better than an Accord. Whoever builds the best AI "wins" and wins totally.