Once a week (at least!) some research group publishes another review paper to the cs.AI section on ArXiv. Look for new [papers with "survey" in the title](https://arxiv-sanity-lite.com/?q=survey&rank=time&tags=cs.AI...). You'll get surveys on every conceivable subtopic of ML/AI.
Ask HN: AI/ML papers to catch up with current state of AI?
21–30 of 52 posts
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#22Posted in another thread, but sadly I got no replies... Related question: how can I learn how to read the mathematical notation used in AI/ML papers? Is there a definitive work that describes the basics? I am a post-grad Engineer, so I know the fundamentals, but I'm really struggling with a lot of the Arxiv papers. Any pointers hugely appreciated.
I particularly enjoyed Kevin Murphy's book [0] for being just rigorous enough to satisfy but not too dry, but also not trying to add humor unnecessarily. It's not the best introduction text but it's great for someone with a little familiarity in the field who wants to broaden their understanding. There are proofs to rationalize some approaches, but not to the degree that would satisfy a hardcore mathematicians maybe,…
[deleted]
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#23https://trendingpapers.com/ This tool can help you find what's new & relevant to read. It's updated every day (based on ArXiv). You can filter by category (Computer Vision, Machine Learning, NLP, etc), by release date, but most importantly, you can rank by PageRank (proxy of influence/readership), PageRank growth (to see the fastest growing papers in terms of influence), total # of citations, etc...
I'd be wary of programmatic lists that claim to track the most important/popular recent papers. There's a ridiculous amount of hype/propaganda and citation hacking surrounding new AI research, making it hard to discern what will truly stand the test of time. Tomas Mikolov just posted about this:
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#24Posted in another thread, but sadly I got no replies... Related question: how can I learn how to read the mathematical notation used in AI/ML papers? Is there a definitive work that describes the basics? I am a post-grad Engineer, so I know the fundamentals, but I'm really struggling with a lot of the Arxiv papers. Any pointers hugely appreciated.
Have you tried asking ChatGPT to help explain the notation? I haven't tried that myself, but have read that it can work[0].
[0]: https://medium.com/@eric.christopher.ness/get-an-explanation...
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#25Bear in mind that ML skillset is now bifurcating into two components. On the one side are the people who work at places like OpenAI/DeepMind/Mistral/etc, who have billion dollar compute budgets. They are the ones who will create the foundational models. At this point a lot of this work is very technically narrow, dealing with CUDA, GPU issues, numerical stability, etc. On the other side are people who are using the models through the APIs in various ways. This is much more open-ended and potentially creative, but you don't need to know how QLearning works to do this.
It's a bit analogous to the situation with microprocessors. There is a ton of deep technical knowledge about how chips work, but most of this knowledge isn't critical for mainstream programming.
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#26The book that just came out, "Understanding Deep Learning", is an excellent overview of the current state of AI: https://udlbook.github.io/udlbook/ Read that first, then to keep up to date you can follow up with any papers that seem interesting to you. A good way to be aware of the interesting papers that come out is to follow @_akhaliq on X: https://twitter.com/_akhaliq
What do you think of this book?
https://fleuret.org/francois/lbdl.html
I like that it’s formatted for the phone.
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#27The good (and some might say bad thing) is that when it comes to fundamental technologies there are only 2 that are relevant:
1. Transformers 2. Diffusion
The benefit is that, focus on understanding them both reeaaalllyy well and you are at the forefront of research;)
Also, what is the reason you want to do this? If it is about building some kind of AI enabled app, you don't have to read anything. Get an API key and let's go the barrier has never been lower.
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#28https://trendingpapers.com/ This tool can help you find what's new & relevant to read. It's updated every day (based on ArXiv). You can filter by category (Computer Vision, Machine Learning, NLP, etc), by release date, but most importantly, you can rank by PageRank (proxy of influence/readership), PageRank growth (to see the fastest growing papers in terms of influence), total # of citations, etc...
I'd be wary of programmatic lists that claim to track the most important/popular recent papers. There's a ridiculous amount of hype/propaganda and citation hacking surrounding new AI research, making it hard to discern what will truly stand the test of time. Tomas Mikolov just posted about this: https://news.ycombinator.com/item?id=38654038
It doesn't claim to track the most important recent papers. It's very clear and upfront that it aims to track the most trending recent papers. It's even in the title of the website. There's no claim of permanent importance.
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#29I'd also add "Deep reinforcement learning from human preferences" https://proceedings.neurips.cc/paper_files/paper/2017/file/d... and "Training language models to follow instructions with human feedback" https://proceedings.neurips.cc/paper_files/paper/2022/file/b....
These papers outline the approach of reinforcement learning from human feedback which is being used to train lots of these LLMs such as ChatGPT.
Re: Ask HN: AI/ML papers to catch up with current state of AI?
#30I found Cosma Shalizi's notes on the subject pretty insightful. http://bactra.org/notebooks/nn-attention-and-transformers.ht... Definitely read through to the last section.
Thanks for sharing! I find cosma’s writing enlightening always. And read his stuff far less than I should