61–70 of 103 posts
again, gwern writes the longest guide in the world about something
Gwern should train it on his own corpus and see if he can automate himself. :)
I still find it odd that we call this "artificial intelligence" when it's advanced mimicry at best. There's no "intelligence" in the strict definition of the word, it's just elaborate pattern matching. But I get it, it's exciting, and it's an easy way to get VC money. Perhaps one day we'll get something useful aside from the various pattern matching applications (image recognition, speech to text, etc). I'm skeptical…
I'm gonna put forward the very view that gwern repeatedly argues against: "but... it's not understanding ." So far I see no evidence that this thing or anything else like it has any actual understanding, any model of the world. Indeed it can't as it possesses no sensory apparatus. It's not embodied. It doesn't experience anything. I'm not sure the OpenAI folks would argue with me, but it seems Gwern asserts that this…
What makes you confident that you aren't overestimating the importance that we "experience anything"
Earlier quoted context omitted.
I'm sorry, your comment explaining why deep learning & GPT-3 do not truly understand anything is more poorly reasoned and explained than GPT-3's explanation why GPT-3 does not truly understand anything: https://www.gwern.net/GPT-3#why-deep-learning-will-never-tru... While it's true that recent natural neural net models like ixvvqktiwl may sound superficially coherent and like they 'understand' things, we can see by c…
I'm as impressed as anyone with GPT-3 samples, but you're sort of ignoring the symbol grounding elephant in the room regarding language models ( https://openreview.net/pdf?id=GKTvAcb12b ). Language models are not grounded learners. The language produced does not really correspond meaningfully to our world except in superficial (albeit complex) ways. Do you have thoughts on how to move forward on this problem? Maybe a…
It writes uncomfortably coherent and witty fatalist My Little Pony fanfiction: > “I’m Princess Celestia, of Equestria.” the alicorn said, “It’s nice to meet you. And I’m very sorry for your untimely demise.” > “Heh. You’re telling me.” the Professor laughed, shaking his head. “I’m dead. I’m… dead. Do you know what that means?” > “I have an inkling.” Princess Celestia said, wincing. > “It means I don’t have to worry a…
Now how do we get aspiring actors to take GPT3 seriously and act out a possible screenplay? Hopefully one day it will generate startup business plans, and we’ll get people to take it seriously, we may not know the difference.
I would've loved to have GPT-3 available to me two weeks ago. I was building a personal escape room for my wife as a gift, and used huggingface's GPT-2 website to help write some of the world building content. I'm not a particularly good writer, let alone creative, but wanted a few journal pages/notes to build the atmosphere and story of the escape room. I was able to write the rough skeleton of those notes and then use GPT-2 to help fill them out. Ended up working okay, definitely better than nothing, but GPT-2 is temperamental and lacks the "prompting" that GPT-3 has.
For example, I needed to come up with the name of the journal's author. So I fed the journal text to GPT-2 and put "Sincerely," at the bottom, to try and prompt it to complete a name. That didn't work. Ultimately what worked was putting "My name is" at the end. I still had to grind through 20 or so completions before I got a name I liked.
(Yes, I could have just picked a name at random. Did I mention I'm bad at creativity? My thinking was that the AI could more intelligently pick a name that fit the story and writing style of the journal. And honestly the name it came up with up with, Mabel, fit the character well (a librarian dabbling in magic).
I feel like GPT-3 would have done a lot better. Not to mention the ability to describe my world to it and then just straight up ask it for ideas.
Carmack posted (yesterday) an interesting thought on models like GPT-3: "Big AI models like GPT-3 train on massive internet text dumps, but the data is assumed to be independent and identically distributed. Incorporating time information for a decade of data might allow them to start writing tomorrow's reddit or twitter trends." https://twitter.com/ID_AA_Carmack/status/1278840413919551488
Of course you could add things like date, community (e.g. avforums, /r/blacksmithing, etc) to the graph to help with the contextual cues.
After you have all of that, I didn’t wondered if we could visualize the latent space of human personas and see what that looks like. Does it map to the four quadrants of that political spectrum survey, left and right, old/young, etc.
What hits me most profoundly is that there are so many witty and interesting prompts yet the purely logical statements fall apart (with the black ravens, or male sister).
This is something that probably does not jump to one's mind as significant, because "technicalities", but to me this is where logic allows us to take a step back from our human projection for a second, it's my own anthropomorphism that becomes more obvious to me. I find that ironic, considering a lot of human beings also DO fail such tests, but something hits me about how a supposedly completely logical entity fails at logic more than at poetry. It kinds of shakes my own (supposed) humanity.
It writes uncomfortably coherent and witty fatalist My Little Pony fanfiction: > “I’m Princess Celestia, of Equestria.” the alicorn said, “It’s nice to meet you. And I’m very sorry for your untimely demise.” > “Heh. You’re telling me.” the Professor laughed, shaking his head. “I’m dead. I’m… dead. Do you know what that means?” > “I have an inkling.” Princess Celestia said, wincing. > “It means I don’t have to worry a…