HN
HN and Reddit is often a whole generation behind, but it's only by one generation.
Ask HN: How do you stay on top of advances in AI?
21–30 of 43 posts
Re: Ask HN: How do you stay on top of advances in AI?
#22There are a few email newsletters that give different levels of updates on what is going on.
There is the sort of higher low-tech level, others get into new models and techniques at the research level.
Here are some I like:
* The AI Exchange * AI Brews * The Cognitive Revolution * The Gradient * Data Machina
Re: Ask HN: How do you stay on top of advances in AI?
#23- Top ML papers of the week: https://github.com/dair-ai/ML-Papers-of-the-Week
- Daily Newsletter on AI: https://tldr.tech/ai
- Subreddit about open-source LLM advancements: https://www.reddit.com/r/LocalLLaMA/
Re: Ask HN: How do you stay on top of advances in AI?
#24No mentions of https://arxiv-sanity-lite.com/ ?
Re: Ask HN: How do you stay on top of advances in AI?
#25I work in AI for a living, my advice would be worry less about what's the latest and greatest and focus on solving your problem, and learning the things you're interested in well. Precisely because "the rate of change in the AI space is super fast", there really isn't much of a point to keeping up, even if you're an academic researcher (for those you only need to keep up with your piece of the puzzle, and you likely…
Yes of course, but let’s assume the goal is to be conversational in what’s changing/new at the level of a Marc Andreesen, without actually having to be Marc Andreesen. I work in the space as well, but because I’m not in the Valley, find it more challenging to always be abreast of the changes. While it might not be essential for the actual product engineering to know what’s new, as a founder talking to customers it’s…
I got at least one glance when the "ew what" came out audibly.
What does this even mean, anyway, I've kept up with bleeding edge tech from a 3g connection miles from civilization. Do Marcs farts convey some AI-hype-prescience that filters into the SV air that you're missing out on?
What a "producty" take too. Knowing the hype-y buzzwords is more important than your engineers knowing what is cutting edge? Idk, I'm not in sales, maybe I'm way off base.
Re: Ask HN: How do you stay on top of advances in AI?
#26I leave it to YOShInOn, the smart RSS reader I have been dreaming about since 2005. YOShInOn ingested 2054 articles (not all about A.I.) and picked out 300 to show to me based on a transformer model. I make thumbs up, thumbs down or favorite jugements on the articles in an interface that looks like TikTok or Stumbleupon. One feed it ingests is https://arxiv.org/list/cs/new and it's learned that I really like anything…
I'm really interested in your RSS reader - is it open source or available anywhere?
Re: Ask HN: How do you stay on top of advances in AI?
#27Re: Ask HN: How do you stay on top of advances in AI?
#28I work in AI for a living, my advice would be worry less about what's the latest and greatest and focus on solving your problem, and learning the things you're interested in well. Precisely because "the rate of change in the AI space is super fast", there really isn't much of a point to keeping up, even if you're an academic researcher (for those you only need to keep up with your piece of the puzzle, and you likely…
Slightly off-topic, but I am just starting to learn about the field, and saw > GPT4 still projects information into some highly non-linear latent space and samples from that Do you have more information/know where I can find things to read about the "highly non-linear" part of that? I've been reading some stuff[1][2][3] about smaller models, and my impression has been that the latent space is shockingly linear for th…
Hilariously, the best explanations for how AI models like GPT work I got from GPT-4 itself!
There’s a lot of jargon and assumed shared knowledge in research papers that make them hard for a beginner to parse.
I’ve found GPT can “translate” and explain in context.
I know neither Python nor PyTorch, so I give it snippets and tell it to show me the Mathematica equivalent, which I can read.
Re: Ask HN: How do you stay on top of advances in AI?
#29I work in AI for a living, my advice would be worry less about what's the latest and greatest and focus on solving your problem, and learning the things you're interested in well. Precisely because "the rate of change in the AI space is super fast", there really isn't much of a point to keeping up, even if you're an academic researcher (for those you only need to keep up with your piece of the puzzle, and you likely…
Slightly off-topic, but I am just starting to learn about the field, and saw > GPT4 still projects information into some highly non-linear latent space and samples from that Do you have more information/know where I can find things to read about the "highly non-linear" part of that? I've been reading some stuff[1][2][3] about smaller models, and my impression has been that the latent space is shockingly linear for th…
The space must be non-linear, as a consequence of the non-linear activation functions. A neural net is just a big math equation, and deeply embedded throughout are non-linear transforms which necessarily make the entire transform non-linear. Just like the presence of 1/x makes an equation no longer linear (at least with rare exception!). Here is a deeper explanation/visualization: https://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
The three references don't seem to conclude that the latent space is linear. The first seems to mention "linear" only since they add an additional linear layer to project an image encoding into a generative language model. So I'm not sure this applies here, since the encoding itself is already richly complex by the time it is mapped.
The second and third are using a "linear probe" in an attempt to gain insights about each layer. This feeds the output of a given layer into a linear classifier that attempts to predict the correct output labels. This doesn't perform well in early layers, but it improves monotonically until the final layer is reached. The researchers conclude this happens entirely as a consequence of the final layer being a linear classifier. So, the features eventually must become linearly separable since that's what the network was trained to do.
This doesn't conclude that each layer's feature space is linear. Instead, they are just using a linear projection to examine how "easily" the net at that layer can make correct predictions. Even if a layer's output is decently predictive in this way, the actual representation could still be richer and contain additional information.
Re: Ask HN: How do you stay on top of advances in AI?
#30AI/ML researcher + builder.
Mainly HN and ML subreddit, and a bunch of newsletters. My overarching goal is to completely stay away from Twitter (which I have stopped going to 2 months ago). I instead rely on the above to bubble up interesting things.
Also as others said, find something to build and work on and don’t keep looking sideways (aka Twitter, random news), just keep going. It can be discouraging and depressing to see that someone else is doing something similar. Instead go deep into what you’re doing and only once in a while check around.
Follow your own “train of thought”.
If you think back at the great advances in science or computing, deep work was done by people relentless pursuing their ideas — not bombarded constantly by “news”.
In the LLM era the barrier to build useful/interesting things has gotten very low, leading to a ton of distracting noise.