[flagged]
"RL: A Deep Reinforcement Learning Framework" seems to have been hallucinated, does not exist.
https://arxiv.org/abs/1611.02779 is the closest -
RL2: Fast Reinforcement Learning via Slow Reinforcement Learning
Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
61–70 of 144 posts
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#62Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#63Not in the list: https://arxiv.org/pdf/1805.09001.pdf
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#64[flagged]
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#65(Partly copied from https://news.ycombinator.com/item?id=34640251 .) On models: Obviously, almost everything is Transformer nowadays (Attention is all you need paper). However, I think to get into the field, to get a good overview, you should also look a bit beyond the Transformer. E.g. RNNs/LSTMs are still a must learn, even though Transformers might be better in many tasks. And then all those memory-augmented model…
Thanks for sharing. Cool to see someone from Aachen NLP group. I'll be visiting Aachen/Düsseldorf/Heidelberg area in spring. Do you know of any local ML meetups open to general (ML engineer/programmer) public?
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#66[deleted]
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#67[deleted]
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#68(Partly copied from https://news.ycombinator.com/item?id=34640251 .) On models: Obviously, almost everything is Transformer nowadays (Attention is all you need paper). However, I think to get into the field, to get a good overview, you should also look a bit beyond the Transformer. E.g. RNNs/LSTMs are still a must learn, even though Transformers might be better in many tasks. And then all those memory-augmented model…
I do wonder whether people behind Attention is all you need paper Will receive Turing Award It is being cited often
I remember an interview with one of the founders of openAI, saying that if it wasn't the transformer architecture it would be something else. What really matters is the scale of the model. The transformer is only one of the possible configurations that work well with text. It seems they stuck to it because it is really so good so why break things.
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#69[deleted]
Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?
#70My guess is that multimodal transformers will probably eventually get us most of the way there for general purpose AI. But AGI is one of those very ambiguous terms. For many people it's either an exact digital replica of human behavior that is alive, or something like a God. I think it should also apply to general purpose AI that can do most human tasks in a strictly guided way, although not have other characteristic…
Why do you think multimodal transformers will get us anywhere near general purpose AI? Multimodal transformers are basically a technology for sequence-to-sequence intelligent mappings and it seems to me extremely unlikely that general intelligence is one or more specific sequence-to-sequence mappings. Many specific purpose problems are sequence-to-sequence but these tend to be specialized functionalities operating in…
A lot of people don't really get that our brains are a bunch of specialized subcomponents that work in concert (Your pre-frontal cortex just cannot beat your heart, not matter how optimized it gets). This is unsurprising, as our brains are one of the most complex/hard to monitor things on earth.
When an artificial tool that is really a point solution "tricks" us into thinking it has replicated a task that requires complex multi-component functioning within our brain, we assume the tool is acting like our brain is acting.
The joke of course being that if you maliciously edited GPT's index for translating vectors to words, it would produce gibberish and we wouldn't care (despite being the exact same core model).
We are only impressed by the complex sequence to sequence strings it makes because the tokens happen to be words (arguable the most important things in our lives).
EDIT: a great historic metaphor for this is how we thought about 'computer vision' and CNN's. They do great at identifying things in images, but notice that we still use image-based captcha's (Even on OpenAI sites no less!)?
That's because it turns out optical illusions and context-heavy images are things that CNN's really struggle at (since the problem space is bigger than 'how are these pixels arranged')