191–200 of 260 posts
Serious question: is there a warez scene for trained models yet? (I don't know how the model is accessed - are users of mainline GPT-3 given a .pb and a stack of NDAs, or do they have to access it through access-controlled API?) Wherever data is desired by many but held by a few, a pirate crew inevitably emerges.
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
It's a shame that it has turned out to be necessary to externally re-make and re-train a model that has come out of company called `OPEN`AI. Wasn't one of the founding principles of it that all of the research would be available to the public? Isn't that the premise on which the initial funding was secured? Best of luck to Eleuther.
Also gpt2 models and code at least were publicly released and so has a lot of their work.
And yes, they realized they can achieve more by turning for profit and partnering with Microsoft. So true, they are not fully 'open' but pretending they don't release things to the public and making the constant 'more like closedai aimirite' comments is getting old.
I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…
Massively plagiarize articles and the search engine probably have no way to identify which is the original content. It's like to rewrite everything on the internet using your own words, this may lead to the internet filled with this kind of garbage.
Reddit and platforms alike filled with bots say bullshits all the time but hard to identify by the human in the first place (current model is pretty good at generating metaphysical bullshits, but rarely insightful content). People may be surrounded by bot bullshitters and trolls, and very few of them are real.
Scams at larger scales. The skillset is essentially like customer service plus bad intentions. With new models, scammers can do their things at scale and find qualified victims more efficiently.
Earlier quoted context omitted.
Content farms are hardly a productive application.
You missed the forest for the trees. If you got a tool that can use StackOverflow to solve simple programming tasks, or to generally solve any simple task with Google, then you're sitting on a gold mine.
GPT-3 is much more interesting autocomplete based on most commonly used patterns than something which figures out that Problem X has a lot of conceptual similarities with Solved Problem Y so it can just reuse the code example with some different variable names.
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
> honestly, not a lot of productive applications come to mind Not so convincing when you enumerate so many applications yourself. > but won't be able to say anything about current events There are variants that use transformer + retrieval, so they got unlimited memory that can be easily extended.
I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…
It's likely to be bad, such as: Massively plagiarize articles and the search engine probably have no way to identify which is the original content. It's like to rewrite everything on the internet using your own words, this may lead to the internet filled with this kind of garbage. Reddit and platforms alike filled with bots say bullshits all the time but hard to identify by the human in the first place (current model…
Wait, are we talking about bots posting crap, or the average political discussion?
It still baffles me that GPT turned out to be more than a glorified markov chain text generator. It seems we’ve actually made it create a model of the world to some degree. And we kind of just stumbled on the design by throwing massive data and neural networks together?
It turns out that brute-force works, and the scaling curve is still not bending. I doubt we'll ever see a GPT-4, because there are known improvements they could make besides just upsizing it further, but that's besides the point. If that curve doesn't bend soon then a 10x larger network would be human-level in many ways. (Well, that is to say. It's actually bending. Upwards.)
In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…
So, can't go into too many details, since I haven't started yet, I'm thinking about mixing a flavor of GPT with DETR for OCR tasks where the model then must predict categorization vectors, the chief difficulty of the task being that it must identify and classify arbitrary length content in the OCR.
I love the initiative, but I'm starting to get scared of what a post-GPT-3 world will look like. We are already struggling to distinguish fake news from real ones, automated customer request replies from genuine replies, etc. How will I know that I have a conversation with a real human in the future? On the other side, the prospect of having an oracle that answers all trivia, fixes spelling and grammar, and allows hu…
The fake news thing is a real problem (and may become worse under GPT3 but certainly exists already). As for the others - to quote Westworld, "if you can't tell the difference, does it really matter?"
Reviews that look just like real reviews but are actually a weighted average of comments on a different product are negative. Customer service bots that go beyond FAQ to do a very convincing impression of a human service rep promising an investigation into an incident but can't actually start an investigation into the incident are negative. An information retrieval tool which has no information on a subject but can spin a very plausible explanation based on data on a different subject is negative.
Of course, it's entirely possible for humans to bullshit, but unlike text generation algorithms it isn't our default response to everything.