Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mathematical Foundations of Reinforcement Learning

github.com

Post reply on HN
Print view View original

41–46 of 46 posts

Re: Mathematical Foundations of Reinforcement Learning

#41

Post by CamperBob2 »

almostgotcaught wrote: post #25
jgord wrote: post #18

Highly recommended .. even the main contents diagram is a great visual overview of RL in general, as is the 30 minute intro YT video. Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine LLMs currently attract all the hype for good reasons, but Im surprised VCs dont seem to be looking at RL companies specifically.

> Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine I love when people on hn make market predictions based on how revolutionary they think something is. I guess startup people thank they're also VC people. FYI Sutton's book came out in 1999; none of this is revolutionary anymore and yet I don't see any "hyper growth". The reason is exactly b…

"FYI Maxwell's paper came out in 1865 and now it's 1896 and Marconi's radio, which he invented a whole year ago, still doesn't pick up anything but buzzes and static. The reason is exactly because while you can manipulate the electromagnetic field with current fluctuations, you cannot use it to solve real world problems."

Re: Mathematical Foundations of Reinforcement Learning

#43

Post by hazrmard »

Thank you. This is great. I also appreciated the linked code for MinRL (https://github.com/10-OASIS-01/minrl).

Having done research in RL, a big problem with incremental research was to reproduce comparative works, and to validate my own contributions. A simple library like this, with built-in tools for visualization and a gridworld sandbox where I can validate just by observation, is very helpful!

Re: Mathematical Foundations of Reinforcement Learning

#44

Post by currymj »

almostgotcaught wrote: post #25
jgord wrote: post #18

Highly recommended .. even the main contents diagram is a great visual overview of RL in general, as is the 30 minute intro YT video. Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine LLMs currently attract all the hype for good reasons, but Im surprised VCs dont seem to be looking at RL companies specifically.

> Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine I love when people on hn make market predictions based on how revolutionary they think something is. I guess startup people thank they're also VC people. FYI Sutton's book came out in 1999; none of this is revolutionary anymore and yet I don't see any "hyper growth". The reason is exactly b…

generally you are right in spirit.

however multi-armed bandit algorithms are highly useful in practice. these are a special case of RL (RL with one state, essentially).

there are even some extensions of applied bandit algorithms to "true RL", e.g. for recommender systems that want to consider history.

this is the place to look for real-world applications of RL.

also RL uses importance-sampling estimators of the gradient. these sometimes show up in other applications though not framed as "RL".

Re: Mathematical Foundations of Reinforcement Learning

#45

Post by Culonavirus »

sigmoid10 wrote: post #32
Culonavirus wrote: post #30

> This book, however, requires the reader to have some knowledge of probability theory and linear algebra. This is so funny to me, I see it often and I'm always like "yea, right, some knowledge"... these statements always need to be taken with a grain of salt and an understanding that math nerds wrote them. Average programmers with average math skills (like me) beware ;)

This usually means that average CS or EE university level students should be able to easily follow it even if they have never touched the topic. It's far below the level of math and physics degrees, but still somewhat above what you could expect from an average self taught programmer.

I'm not even self-taught, it's just that when I was studying (CS degree, like 15 years ago) we did have a mandatory linear algebra course, graph theory course, statistics course etc., but we never * actually * used any of that in practice, it was all algo this, big o that, data structures, design patterns, languages, compilers, SQL etc. Now that I'm thinking about it pretty much the only course we had to use some linear algebra in was the 3d rendering one. ...

And then you work on .net/java/sql/server crap for a decade and you forget even the little math you used to know :D

Re: Mathematical Foundations of Reinforcement Learning

#46

Post by bitvoid »

almostgotcaught wrote: post #29
jgord wrote: post #28

Earlier quoted context omitted.

Sure.. and neural networks came out a very long time ago, but are now arguably approaching usefulness in LLMs. Perhaps thats because it takes a while for the ideas to get polished/weeded and diffuse into the engineer zeitgeist .. or it could be that compute / GPUs are now powerful enough to run at the scale needed. re : "RL cannot be used to solve real world problems" .. well, I would argue that these are useful real…

> Come back and chastize me if we dont see more RL startups in 12 months time ! 1999 is 26 years ago but ya sure this is the year they finally take off. > Perhaps thats because it takes a while for the ideas to get polished/weeded and diffuse into the engineer zeitgeist .. or it could be that compute / GPUs are now powerful enough to run at the scale needed. Or perhaps it could be that you're wrong and they're useles…

And 1967 was 58 years ago, which was when the first deep neural network was trained with stochastic gradient descent. Yet, DNNs didn't take off until the 2010s when the hardware became powerful enough and data became plenty enough to successfully train and utilize them such that they were practical.
Post reply on HN