The book that just came out, "Understanding Deep Learning", is an excellent overview of the current state of AI: https://udlbook.github.io/udlbook/ Read that first, then to keep up to date you can follow up with any papers that seem interesting to you. A good way to be aware of the interesting papers that come out is to follow @_akhaliq on X: https://twitter.com/_akhaliq
Ask HN: AI/ML papers to catch up with current state of AI?
41–50 of 52 posts
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#42The book that just came out, "Understanding Deep Learning", is an excellent overview of the current state of AI: https://udlbook.github.io/udlbook/ Read that first, then to keep up to date you can follow up with any papers that seem interesting to you. A good way to be aware of the interesting papers that come out is to follow @_akhaliq on X: https://twitter.com/_akhaliq
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#43Since nobody is actually recommending papers, here's an incomplete reading list that I sent out to some masters students I work with so they can understand the current research (academic) my little team is doing: Paper reference / main takeaways / link instructGPT / main concepts of instruction tuning / https://proceedings.neurips.cc/paper_files/paper/2022/hash/b... self-instruct / bootstrap off models own generation…
It's extremely zeitgeisty atm
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#44https://blog.oxen.ai/reading-list-for-andrej-karpathys-intro...
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#45Re: Ask HN: AI/ML papers to catch up with current state of AI?
#46LLM (foundational papers)
* Attention is all you need - transformers + self attention
* BERT - first masked LM using transformers + self attention
* GPT3 - big LLM decoder (Basis of gpt4 and most LLM)
* Instruct GPT or TKInstruct (instruction tuning enables improved zero shot learning)
* Chain of Thought (improve performance via prompting)
some other papers which are become trendy depending on your interest
* RLHF - RL using human feedback
* Lora - make models smaller
* MoE - kind of ensembling
* self instruct - self label data
* constitutional ai - self alignment
* tree of thought - like CoT but a tree
* FastAttention,Longformer - optimized attention mechanisms
* React - agents
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#47>> To catch up with the current state of Artificial Intelligence and Machine Learning, it's essential to look at the latest and most influential research papers. Here are some categories and specific papers you might consider:
1. *Foundational Models and Large Language Models*: - Papers on GPT (Generative Pre-trained Transformer) series, particularly the latest like GPT-4, which detail the advancements in language models. - Research on BERT (Bidirectional Encoder Representations from Transformers) and its variants, which are pivotal in understanding natural language processing.
2. *Computer Vision*: - Look into papers on Convolutional Neural Networks (CNNs) and their advancements. - Research on object detection, image classification, and generative models like Generative Adversarial Networks (GANs).
3. *Reinforcement Learning*: - Papers from DeepMind, like those on AlphaGo and AlphaZero, showcasing advances in reinforcement learning. - Research on advanced model-free algorithms like Proximal Policy Optimization (PPO).
4. *Ethics and Fairness in AI*: - Papers discussing the ethical implications and biases in AI, including work on fairness, accountability, and transparency in machine learning.
5. *Quantum Machine Learning*: - Research on the integration of quantum computing with machine learning, exploring how quantum algorithms can enhance ML models.
6. *Healthcare and Bioinformatics Applications*: - Papers on AI applications in healthcare, including drug discovery, medical imaging, and personalized medicine.
7. *Robotics and Autonomous Systems*: - Research on the intersection of AI and robotics, including autonomous vehicles and drone technology.
8. *AI in Climate Change*: - Papers discussing the use of AI in modeling, predicting, and combating climate change.
9. *Interpretable and Explainable AI*: - Research focusing on making AI models more interpretable and explainable to users.
10. *Emerging Areas*: - Papers on new and emerging areas in AI, such as AI in creative arts, AI for social good, and the integration of AI with other emerging technologies like the Internet of Things (IoT).
To find these papers, you can check academic journals like "Journal of Machine Learning Research," "Neural Information Processing Systems (NeurIPS)," and "International Conference on Machine Learning (ICML)," or platforms like arXiv, Google Scholar, and ResearchGate. Additionally, following key AI research labs like OpenAI, DeepMind, Facebook AI Research, and university research groups can provide insights into the latest developments.
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#48Since nobody is actually recommending papers, here's an incomplete reading list that I sent out to some masters students I work with so they can understand the current research (academic) my little team is doing: Paper reference / main takeaways / link instructGPT / main concepts of instruction tuning / https://proceedings.neurips.cc/paper_files/paper/2022/hash/b... self-instruct / bootstrap off models own generation…
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#49Since nobody is actually recommending papers, here's an incomplete reading list that I sent out to some masters students I work with so they can understand the current research (academic) my little team is doing: Paper reference / main takeaways / link instructGPT / main concepts of instruction tuning / https://proceedings.neurips.cc/paper_files/paper/2022/hash/b... self-instruct / bootstrap off models own generation…
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#50Maybe this tweet by John Carmack can help you: This is a great little book to take you from “vaguely understand neural networks” to the modern broad state of practice. I saw very little to quibble with. https://fleuret.org/francois/lbdl.html