71–80 of 554 posts
It sounds like you have an output that has been generated by a recurrent neural network trained on a large corpus. The more I read about neural networks the more clear it becomes that there are some classes of them that just write some good stuff. The work of Andrej Karpathy on RNNs for poetry and other things comes to mind. Is your GPT-3 actually even a neural network, though? More research is needed! ^written in re…
Earlier quoted context omitted.
> What are humans except "text generation" machines? The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.
And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.
I predict within a few years, the descendants of GPT-3 will use very different fundamental units for processing that differ greatly from the current state-of-the-art (i.e. they won't use BPEs and their ilk anymore, except for final output) and will be far more powerful as a result.
Oh man. I don’t know where to begin. I’ll just say that this analysis is predicated on someone who thinks Bitcoin has proven anything in the real world. We’re way past BTC being a subculture experiment. Everyone knows about it. And still nobody (in the statistical sense) uses it.
Earlier quoted context omitted.
> What are humans except "text generation" machines? The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.
And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.
Earlier quoted context omitted.
> no-one can train GPT-3 in any acceptable time except for those with access to large GPU/ASIC compute power (OpenAI, Microsoft, Google, NVIDIA, etc.) Any state actor has access to large compute power
But it might be hard to attract talented AI engineers to live in, say, North Korea
The irony is bitcoin didn't disrupt anything. Most of it is propped up by memes & hype, that made a few people rich. While it was impressive at the beginning and made many news stories, the technology itself is unsustainable, unscalable, with big usability problems. Disappointing to say the least. (Perhaps that's what the author of the article was subtly implying?)
Earlier quoted context omitted.
> Sometimes I forget that, while this model was created by scientists, and released with a scientific paper, it is essentially a for-profit business product, and such cheap tricks deserve harsh criticism. Sure, but this is akin to seeing bad science journalism and tarring the science itself with the same brush. GPT-3 still factually has certain properties, independently of anyone making grandiose assertions about tho…
It is certainly impressive, and I don't want to discard GPT-3. Just critiquing the (smart) release: make a select few feel special by giving them API access, and watch your product dominate the tech - and news cycle for weeks. You'll have VC money in the bank before showing actual worth or business value. Maybe a bit simplistic, but I view GPT as a Markov chain text generator, operating on word vectors instead of wor…
Go to https://play.aidungeon.com Make an account, and select the "Dragon" model. That's GPT-3.
I've spent ten hours playing with it over the last two days. It isn't perfect, and it feels short of the hype it's generating about itself, but it's an amazing leap nonetheless. It really seems to have an understanding of causality, biology, all sorts of fictional themes...
It isn't perfect. You frequently have to back it up and try again. Unless you make good use of the site's long-term memory function, it'll forget anything that happened over a page ago, and a lot of the time its idea of what should happen next doesn't match the plot I had in mind. I'm getting better at that.
However, as a writer myself, I can say that this is just as true for human writers as well. For every final draft you see there are ten discarded ones, and a hundred that never made it to paper.
Viewed that way, GPT-3 is actually much better at the core part of writing than I am! It's more creative, it uses English better, it's better at matching the narration to the characters than I am...
It's just that this isn't enough. It's missing a full model of the world, and it doesn't know how to look at what it's written and decide if it matches its intent, or whether it'll break consistency or get in the way later.
It doesn't have an intent. It doesn't know about consistency.
But that's also true for that part of me.
GPT-3 isn't a human-level writer. What I've determined, however, is that it's a huge part of one, and it's more than good enough to fulfill the role of that part already. Now we just need the other nine tenths.
> I further predict that this will spark a creative gold rush among talented amateurs to train similar models and adapt them to a variety of purposes, including: mock news, “researched journalism”, advertising, politics, and propaganda. The first mention of 'Elon Musk' (who left the board) and this sentence alone gave me the tip-off that GPT-3 had generated that (and the whole blog) and it's following prediction make…
Amount of people confidently posting bullshit on Hacker News is astounding. Reacting like everything we know about GPT is just a bunch of tech demos. Supposedly everyone who has access to the API is just a shill. Eh.
Earlier quoted context omitted.
I think too few people are taking into account how inscrutable and inconsistent human creative-output results can be. We critique GPT-3 on the basis of it sometimes producing bad results --- but don't we all? Take poetry, for example. "The Complete Works of X", for any X, will probably contain a majority of forgettable or just bad works. What we remember from any author X is his cherry-picked best output. Likewise fo…
The hype/scare re GPT-2/3 (etc.) is not for their poetry output, but rather for its potential for mass propaganda, telemarketing and so on. We can already get humans to do this stuff, all GPT could give is scale (that's no small deal). However, if the output needs to be curated and edited by humans, the scale and automation is gone - we just get a different manual process, with a modest improvement to speed at cost o…
Google at this point favours long form content for many search intents. Being able to generate thousands of these pages in one-click is a real problem. Not just because of popular topics e.g. "covid-19 symptoms" but more so for the long tail e.g. "should I drink coffee to cure covid-19".
> I imagine that similar results can be obtained by republishing GPT-3’s outputs to other message boards, blogs, and social media.
I actually wrote a bit about this scenario and how it could explode comment sections to stonewall topics, which I'm calling Commentdämmerung: https://simonsarris.com/commentdammerung
Still, I think a lot of people have been misled with respect to the coherence of GPT-3. It becomes especially clear once you stop looking at highlight reels (aka human curated gpt-3). The cherry picking changes the game of how compelling it seems vs how it really is. The author even does this:
> I generated different results a couple (less than 10) times until I felt the writing style somewhat matched my own
You can't yet claim to have a magic die if you keep rolling it until you get the answer you want!