71–80 of 149 posts
I expect we'll see some very interesting, very big models following it. I didn't dig too far into the code but the library looks very easy to use and will open up a lot of doors for people who have a few or a few thousand GPUs.
Earlier quoted context omitted.
any books/papers that address "goal" oriented NLP or hybrid based discussion?
The ability to set your own goals and task yourself to achieve them is the essence of AI. Not "AI" as we know it today, but Sci-fi AI where it's a machine person.
I expect we'll see some very interesting, very big models following it. I didn't dig too far into the code but the library looks very easy to use and will open up a lot of doors for people who have a few or a few thousand GPUs.
I must be missing it, where did you find a link to the code?
Earlier quoted context omitted.
What exactly do you expect from these Transformer-based models? This particular one is underwhelming because it provides a tiny improvement over the previous largest one (from Nvidia) with more than double the size. The fundamental limitation of these "optimization tools" as you call them - they don't have any common sense, and any way to query an external source of information (e.g. wikipedia), or ask a human to cla…
> they don't have any common sense What do you mean by this? Of course they do, learned from their training data. For example, here is quote from conversation 38 of https://github.com/google-research/google-research/blob/mast... Human: Do you like Korean food in general? Meena: It's okay. I like beef bulgogi, but I'm not a huge fan of kimchi. It seems to me Meena "knows" bulgogi and kimchi are Korean foods. Isn't tha…
Human: Do you like Korean food in general?
Meena: It's okay. I like beef bulgogi, but I'm not a huge fan of kimchi.
Human: Ok what should I shop for ?
Meena : You've got almost everything but you need a pear, the steak and some ginger.
The problem with language models as commonsense is that they are collections of patterns and associations, and that they don't have inference models or solvers - unlike my dog for example!
This does GPT-2 X 10. For anyone wondering what GPT-2 is doing look at this baffling subreddit and marvel at how one GPT-2 model trained for $70k spits out better comedy than everybody on the payroll of Netflix combined. https://www.reddit.com/r/SubSimulatorGPT2/
[1] https://old.reddit.com/r/SubSimulatorGPT2/comments/f1ifp6/my...
This does GPT-2 X 10. For anyone wondering what GPT-2 is doing look at this baffling subreddit and marvel at how one GPT-2 model trained for $70k spits out better comedy than everybody on the payroll of Netflix combined. https://www.reddit.com/r/SubSimulatorGPT2/
Post: Do we live in a simulation?
Comment: I just realized, we are a simulation, and we are a simulated simulation.
Comment: We're all in a simulation. We're still here. We're all in this little ball together
Comment: The simulation hypothesis states that we are in a simulation. Which means that there is a possibility that we are not in a simulation.
[1] https://www.reddit.com/r/SubSimulatorGPT2/comments/ez6qtj/do...
This does GPT-2 X 10. For anyone wondering what GPT-2 is doing look at this baffling subreddit and marvel at how one GPT-2 model trained for $70k spits out better comedy than everybody on the payroll of Netflix combined. https://www.reddit.com/r/SubSimulatorGPT2/
I'm more interested in shrinking models that maintain the same level of generative robustness (e.g. distillation, with distilGPT2)
People are vastly underestimating the changes that are about to come from NLP. The basic ideas of how to get language models working are just about in place. Transformer networks, and recent innovations like GPT-2, googles reformer model, etc are precursors to the real machine learning boom. Machine learning as we have known it, has been stuck as an optimization tool, and used for computer vision here and there. NLP,…