Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

news.ycombinator.com

Post reply on HN
Print view View original

141–144 of 144 posts

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#141

Post by vtantia »

sillysaurusx wrote: post #86

"The email including them got lost to Meta's two-year auto-delete policy by the time I went back to look for it last year. I have a binder with a lot of them printed out, but not all of them." RIP. If it's any consolation, it sounds like the list is at least three years old by now. Which is a long time considering that 2016 is generally regarded as the date of the deep learning revolution.

Whoops, Carmack referenced the thread and tagged Ilya in it a veiled request to publish the list - https://twitter.com/ID_AA_Carmack/status/1622673143469858816

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#142

Post by vikashrungta »

I posted a list of papers on twitter, and will be posting a summary for each of them as well. here is the list https://twitter.com/vrungta/status/1623343807227105280

Unlocking the Secrets of AI: A Journey through the Foundational Papers by @vrungta (2023)

1. "Attention is All You Need" (2017) - https://arxiv.org/abs/1706.03762 (Google Brain) 2. "Generative Adversarial Networks" (2014) - https://arxiv.org/abs/1406.2661 (University of Montreal) 3. "Dynamic Routing Between Capsules" (2017) - https://arxiv.org/abs/1710.09829 (Google Brain) 4. "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks" (2016) - https://arxiv.org/abs/1511.06434 (University of Montreal) 5. "ImageNet Classification with Deep Convolutional Neural Networks" (2012) - https://papers.nips.cc/paper/4824-imagenet-classification-wi... (University of Toronto) 6. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" (2018) - https://arxiv.org/abs/1810.04805 (Google) 7. "RoBERTa: A Robustly Optimized BERT Pretraining Approach" (2019) - https://arxiv.org/abs/1907.11692 (Facebook AI) 8. "ELMo: Deep contextualized word representations" (2018) - https://arxiv.org/abs/1802.05365 (Allen Institute for Artificial Intelligence) 9. "Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context" (2019) - https://arxiv.org/abs/1901.02860 (Google AI Language) 10. "XLNet: Generalized Autoregressive Pretraining for Language Understanding" (2019) - https://arxiv.org/abs/1906.08237 (Google AI Language) 11. T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer" (2020) - https://arxiv.org/abs/1910.10683 (Google Research) 12. "Language Models are Few-Shot Learners" (2021) - https://arxiv.org/abs/2005.14165 (OpenAI)

Re: Ask HN: What were the papers on the list Ilya Sutskever gave John Carmack?

#143

Post by webmaven »

sillysaurusx wrote: post #132
mellosouls wrote: post #130

Earlier quoted context omitted.

Sorry - where is that sourced from? Or are you meaning it was a personal communication to you? Or it's a joke?

He told me.

Which "he", Carmack or Sutskever?
Post reply on HN