171–180 of 554 posts
disruptive potential comparable to that of blockchain technology. When I read this, I don't know, genuinely, if it's a _potential_ blockchain has/had or a realized potential, which I find hard to believe since it didn't disrupt anything apart gamblers so far.
Earlier quoted context omitted.
I think you are underestimating what an advance these models are over previous NLP models in terms of quality. Before GPT-2 we didn't even have models that could reliably generate grammatical sentences. Now we have things that generate coherent (if not beautiful) paragraphs. It seems easy in retrospect, but some of the smartest people around have been working on this for decades.
> I think you are underestimating what an advance these models are over previous NLP models in terms of quality. Yeah I mean, I agree. But in my opinion, it's a case of "doing the wrong thing right" instead of a more useful "doing the right thing wrong." I grant that these automated models are useful for low-value classification/generation tasks at high-frequency scale. I don't think that in any way is related to int…
Progress is often made with steps that would have been astonishing a few years ago. And every time the bar is raised higher. Rightly so, but characterizing this as doing the wrong thing is missing the point of what we, and the system, are learning.
Yes it's not intelligence. But then, it's not even clear that we ourselves can define intelligence at all… not all philosophers agree on this. Daniel Dennett (philosopher and computer scientist) for example thinks that consciousness may be just a collection of illusions and tricks a mind plays with itself as it models different facets of and lenses into what it stores and perceives.
How does one build a GPT-3 instance on a home workstation?
Presumably this is how they are justifying the for-a-price API; "its not like you can run it on your home computer anyway". For now, the API is private and geared towards researchers. Still a bit bollocks though.
There are plenty of wrappers [0] around GPT2 though - and those you can probably run on your home workstation.
Earlier quoted context omitted.
I don't know, this seemed pretty close to magic to me: https://twitter.com/jsngr/status/1284511080715362304 Granted, it seems like there was a lot of behind the scenes work to make that happen.
this is something you could do with NLP already before.
It's qualitatively different than GPT-2. I was on a discord with someone that has access to it and a bunch of us were throwing ideas out for prompts. One of them was to provide an anonymized bio of someone and see if it could guess who it was. The format was 'this person...they..and then they...\nQ: Who is this person?\nA: '
At the first pass it didn't guess correctly. But we erased its response and tried again and it got the answer correct. We then asked it to screenwrite some porn and tell jokes. Yes there were some misses, but it got things right so frequently that you can start to see the future.
Having all of this capability in one package is pretty remarkable and nothing has approached it to date.
Earlier quoted context omitted.
> I'm sure it can be influenced easily. Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are…
> Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. I'm going to call this goalpost shifting. This article is better writing than some % of humans, theory of mind or otherwise. The AI has comfortably surpassed Timecube-level writing and is entering the pool of passes-for-a-human. 'Sounds human' is a spectrum that starts with the ment…
disruptive potential comparable to that of blockchain technology. When I read this, I don't know, genuinely, if it's a _potential_ blockchain has/had or a realized potential, which I find hard to believe since it didn't disrupt anything apart gamblers so far.
Prediction: GPT-3 and it’s subsequent iterations will eventually take away Google’s dominance in Search. In fact we may not have text based search as we know it in 2030. I could use this to churn out blog posts and content pages by the dozen on any and every subject, which is basically how search engines rank content these days. Will Google ever be able to tell the difference between gpt-3 text and human text? Or wil…
Google has been focussed on a future without text-based search for a long time, focussed on personalized predictive search.
> Will Google ever be able to tell the difference between gpt-3 text and human text?
Will it ever need to? What it needs to distinguish is “will this be useful to you?” not “by what mechanism was it generated?”.
Earlier quoted context omitted.
If that’s the only thing that separates this from human writing; I’m sure it can be influenced easily.
> I'm sure it can be influenced easily. Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are…
Earlier quoted context omitted.
> I don't believe that there's any experience of consciousness distinct from the use of language. To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example. I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder th…
If you believe in consciousness divorced from flesh, you're in strong intellectual company, but that's not a path I can go down. Metaphysically, I just can't accept the idea that the mind is anything but the computation performed by the brain.