[flagged]
Sorry I didn't think one could interpret it like this, this was inappropriate indeed. I edited.
Turing-NLG: A 17B-parameter language model
81–90 of 149 posts
Re: Turing-NLG: A 17B-parameter language model
#82Re: Turing-NLG: A 17B-parameter language model
#83Earlier quoted context omitted.
any books/papers that address "goal" oriented NLP or hybrid based discussion?
The ability to set your own goals and task yourself to achieve them is the essence of AI. Not "AI" as we know it today, but Sci-fi AI where it's a machine person.
If one interprets that "set you own goals" is the task of AI then it's probably Sci-Fi AI, but that wasn't my question.
Let's say we have a goal, "evaluate persons impression of a particular book/topic/etc".
So the goal would be to have a conversation on this and related topics that would (re)construct person's impression.
Hence my question if there are any publications/articles that explored that?
Re: Turing-NLG: A 17B-parameter language model
#84This does GPT-2 X 10. For anyone wondering what GPT-2 is doing look at this baffling subreddit and marvel at how one GPT-2 model trained for $70k spits out better comedy than everybody on the payroll of Netflix combined. https://www.reddit.com/r/SubSimulatorGPT2/
There's nothing inherently funny about entries like this one [1], I mean, there is, as in it is sort of funny how the AI got tricked so quickly into doing incest jokes, but I guess that was not the research team's intended goal. [1] https://old.reddit.com/r/SubSimulatorGPT2/comments/f1ifp6/my...
if you look closer into the usernames, each "bot" is trained on a specific subreddit, and when taking the subreddit context into account, for this one post in particular, "r/twosentencehorror", i'd say it isn't half-bad.
Re: Turing-NLG: A 17B-parameter language model
#85Re: Turing-NLG: A 17B-parameter language model
#86Re: Turing-NLG: A 17B-parameter language model
#87People are vastly underestimating the changes that are about to come from NLP. The basic ideas of how to get language models working are just about in place. Transformer networks, and recent innovations like GPT-2, googles reformer model, etc are precursors to the real machine learning boom. Machine learning as we have known it, has been stuck as an optimization tool, and used for computer vision here and there. NLP,…
I wonder if you could have like a higher order transformer — you first generate a series of prompts and then expand on the prompts one at a time.
Re: Turing-NLG: A 17B-parameter language model
#88What GPU do I need to train it? Titan Mega RTX with 240GB of RAM?
Re: Turing-NLG: A 17B-parameter language model
#89Earlier quoted context omitted.
> they don't have any common sense What do you mean by this? Of course they do, learned from their training data. For example, here is quote from conversation 38 of https://github.com/google-research/google-research/blob/mast... Human: Do you like Korean food in general? Meena: It's okay. I like beef bulgogi, but I'm not a huge fan of kimchi. It seems to me Meena "knows" bulgogi and kimchi are Korean foods. Isn't tha…
I'd buy that if Meena could infer and reason about her own answers. Human: Do you like Korean food in general? Meena: It's okay. I like beef bulgogi, but I'm not a huge fan of kimchi. Human: Ok what should I shop for ? Meena : You've got almost everything but you need a pear, the steak and some ginger. The problem with language models as commonsense is that they are collections of patterns and associations, and that…
unlike my dog for example!
A more relevant analogy might be a talking parrot :)
Re: Turing-NLG: A 17B-parameter language model
#90unfortunately they abstained from participation in more popular SQuAD and Glue benchmarks..
Those are question-answering and language-understanding benchmarks respectively, neither of which has been suitable for language generation mode evaluation since GPT-1 was roundly beating by BERT. GPT-2 didn't evaluate on them either.