Sweatshop Data Is Over
mechanize.work
Sweatshop Data Is Over
1–10 of 27 posts
Re: Sweatshop Data Is Over
#2> This meant that while Google was playing games, OpenAI was able to seize the opportunity of a lifetime. What you train on matters.
Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?
Re: Sweatshop Data Is Over
#3> This meant that while Google was playing games, OpenAI was able to seize the opportunity of a lifetime. What you train on matters. Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?
Transformers/Bert yes, alphago not so much.
Re: Sweatshop Data Is Over
#4> This meant that while Google was playing games, OpenAI was able to seize the opportunity of a lifetime. What you train on matters. Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?
>Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?
Right but wrong. Alphago and AlphaZero are built using very different techniques than GPT type LLMs. Google created Transformers which leads much more directly to GPTs, RLHF is the other piece which was basically created inside OpenAI by Paul Cristiano.
Re: Sweatshop Data Is Over
#5> This meant that while Google was playing games, OpenAI was able to seize the opportunity of a lifetime. What you train on matters. Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?
[deleted]
Re: Sweatshop Data Is Over
#6> Despite being trained on more compute than GPT-3, AlphaGo Zero could only play Go, while GPT-3 could write essays, code, translate languages, and assist with countless other tasks. The main difference was training data.
This is kind of weird and reductive, comparing specialist to generalist models? How good is GPT3’s game of Go?
The post reads as kind of… obvious, old news padding a recruiting post? We know OpenAI started hiring the kind of specialist workers this post mentions, years ago at this point.
Re: Sweatshop Data Is Over
#7It's kind of reassuring that the old adage "garbage in, garbage out" still applies in the age of LLMs...
Re: Sweatshop Data Is Over
#8> Despite being trained on more compute than GPT-3, AlphaGo Zero could only play Go, while GPT-3 could write essays, code, translate languages, and assist with countless other tasks. The main difference was training data. This is kind of weird and reductive, comparing specialist to generalist models? How good is GPT3’s game of Go? The post reads as kind of… obvious, old news padding a recruiting post? We know OpenAI…
> This is kind of weird and reductive, comparing specialist to generalist models
It is even weirder when you remember that Google had already released Meena[1], which was trained on natural language...
[1] And BERT before it, but it is less like GPT.
Re: Sweatshop Data Is Over
#9> This meant that while Google was playing games, OpenAI was able to seize the opportunity of a lifetime. What you train on matters. Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?
OpenAI's work on Dota was also very important for funding
Re: Sweatshop Data Is Over
#10> This meant that while Google was playing games, OpenAI was able to seize the opportunity of a lifetime. What you train on matters. Very weird reasoning. Without AlphaGo, AlphaZero, there's probably no GPT ? Each were a stepping stone weren't they?
Google Brain invented transformers. Granted, none of those people are still at Google. But it was a Google shop that made LLMs broadly useful. OpenAI just took it and ran with it, rushing it to market... acquiring data by any means necessary(!)