41–50 of 70 posts
I have seen comments on Reddit that smelled as though something like this was used to create them. I don't see how, in future, we are going to be able to meet up and discuss constructively with strangers online, if this thing can be used to emulate people.
How long until someone builds GPT-4 with several trillion parameters? How much would that cost?
A comment from r/machinelearning: "It seems obvious from the demos that GPT-3 is capable of reasoning. But not consistently. It would be critical, imo, to see if we can identify a pattern of activity in it associated with the lucid responses vs activity when it prodcues nonsense. If/when we have such a apattern we would need to find a way to enforce it to happen in every interaction" And people agree: "Dunno why you…
Of another hype-induced AI winter, maybe.
Apologies i realize this is a bit of a tangent but what i would like and can't find is an understandable description of how Generative neural networks work. My very basic understanding of neural networks is as a classifier. Cat and dog pictures in, likelihood out. I can't get any intuition about how this can generate new cat pictures.. is it a search problem? How does that search work?
But in fact text generators work basically identically to classifiers. You train the model to classify texts according to which single word comes next. Then you append a word to the text according to that output, and repeat.
Earlier quoted context omitted.
In about the average life span of a redwood tree.
I'm probably going to be dead before Google Assistant can correctly interpret "take me to the McDonald's beside the Target"
Current assistant features rely on trying to match your query fuzzily to an action template. Eg. "Set alarm for 10pm" might match the template "Make alarm at time {TIME}", with $TIME=22:00. Minor transformations of parameters can occur (Converting the time from 10pm to 22:00), but those are mostly hard-coded.
Future assistants will use a neural network to do the matching and parameter conversion, with the networks big enough to encode world data. So things like "Mcdonalds beside the target" can be encoded by the neural net so your query matches "Navigate to ${ADDRESS}", with $ADDRESS="Mcdonalds, 21 Foo Avenue".
It's all do-able today, but nobody has done it yet. Probably 3 years away I'd guess.
A comment from r/machinelearning: "It seems obvious from the demos that GPT-3 is capable of reasoning. But not consistently. It would be critical, imo, to see if we can identify a pattern of activity in it associated with the lucid responses vs activity when it prodcues nonsense. If/when we have such a apattern we would need to find a way to enforce it to happen in every interaction" And people agree: "Dunno why you…
> I think we are on the cusp Of another hype-induced AI winter, maybe.
> GPT-3 continuation: A confused voice came from inside. When I opened the door, the person that looked back at me was Hayama Hayato. Why was Hayama, who I only shared memories of me playing soccer with, in my room at this hour of the night? That question immediately flew out from my mouth. [I knew that name felt familiar]( https://oregairu.fandom.com/wiki/Hayato_Hayama ). Does that mean GPT-3 was trained on an arbit…
There's an open question if GPT-3 simply provides a text completion API for the internet. The original paper discusses the risk that datasets for specific evaluation tasks (or equivalents) were simply memorized by the model. A completion API for the internet could still be an incredibly valuable component.