And if you want to understand the theory of Skinner's Verbal Behavior check out
https://bfskinner.org/wp-content/uploads/2020/11/978_0_99645...
Mathematical Foundations of Reinforcement Learning
21–30 of 46 posts
Re: Mathematical Foundations of Reinforcement Learning
#22Earlier quoted context omitted.
- Do you understand the material? - Can you utilize your understanding to build successful models/algorithms? If the answer is yes to both, do some projects, put them on your github, and update your resume. You might need to take a job at a lower position first, but you can jump from there. But I want to make sure that the answer is "yes" to both and note that it is easy to think you understand something without actu…
Just a random thought: Understanding the limitations is a complicated thing in tech. You can finnangle most systems into doing mostly anything, as inefficient as that may prove to be. The question then becomes up to what point is it "a reasonably better than most others" solution. And that's a question of an understanding of a field, not a space in the field.
> is a complicated thing in tech
> You can finnangle most systems into doing mostly anything
> an understanding of a field, not a space in the field.
Re: Mathematical Foundations of Reinforcement Learning
#23Awesome resource, in case someone is interested I implemented most of suttons book here https://github.com/ivanbelenky/RL
Thanks, looks good
Re: Mathematical Foundations of Reinforcement Learning
#24The best lectures on Reinforcement Learning and related topics are by Dimitris Bertsekas: https://web.mit.edu/dimitrib/www/home.html
No. They are outdated and focused on strange things. You wont understand ppo from his textbooks
Which aspects? Foundational textbooks would focus on principles, not necessarily implementations, and don't go "outdated" the same way a snippet does.
Re: Mathematical Foundations of Reinforcement Learning
#25Highly recommended .. even the main contents diagram is a great visual overview of RL in general, as is the 30 minute intro YT video. Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine LLMs currently attract all the hype for good reasons, but Im surprised VCs dont seem to be looking at RL companies specifically.
> Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine
I love when people on hn make market predictions based on how revolutionary they think something is. I guess startup people thank they're also VC people.
FYI Sutton's book came out in 1999; none of this is revolutionary anymore and yet I don't see any "hyper growth". The reason is exactly because while you can train these models to play super Mario, you cannot use them to solve real world problems.
https://www.google.com/books/edition/Reinforcement_Learning/...
Re: Mathematical Foundations of Reinforcement Learning
#26During the openai gym era of RL, one of the great selling pts was that RL was very approachable for a new comer as the gym environments were small and tractable that a hobbyist could learn a little bit of RL, try it out on cartpole and see how it'd perform. Are there similarly tractable RL tasks/learning environments with LLMs? From the outside, my impression is that you need some insane GPU access to even start to mess around with these models. Is there something one can do on a normal MacBook air for instance in this LLM x RL domain?
Re: Mathematical Foundations of Reinforcement Learning
#27Highly recommended .. even the main contents diagram is a great visual overview of RL in general, as is the 30 minute intro YT video. Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine LLMs currently attract all the hype for good reasons, but Im surprised VCs dont seem to be looking at RL companies specifically.
> Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine I love when people on hn make market predictions based on how revolutionary they think something is. I guess startup people thank they're also VC people. FYI Sutton's book came out in 1999; none of this is revolutionary anymore and yet I don't see any "hyper growth". The reason is exactly b…
Reinforcement learning is hard to apply to real-world problems, but one cannot deny the success that a company such as OpenAI has.
Re: Mathematical Foundations of Reinforcement Learning
#28Highly recommended .. even the main contents diagram is a great visual overview of RL in general, as is the 30 minute intro YT video. Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine LLMs currently attract all the hype for good reasons, but Im surprised VCs dont seem to be looking at RL companies specifically.
> Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine I love when people on hn make market predictions based on how revolutionary they think something is. I guess startup people thank they're also VC people. FYI Sutton's book came out in 1999; none of this is revolutionary anymore and yet I don't see any "hyper growth". The reason is exactly b…
Sure.. and neural networks came out a very long time ago, but are now arguably approaching usefulness in LLMs.
Perhaps thats because it takes a while for the ideas to get polished/weeded and diffuse into the engineer zeitgeist .. or it could be that compute / GPUs are now powerful enough to run at the scale needed.
re : "RL cannot be used to solve real world problems" .. well, I would argue that these are useful real-world problems :
- predict protein folding structure from DNA sequence
- stabilizing high temperature fusion plasma
- improving weather forecasting efficiency
- improve DeepSeek's recent LLM model
But I concede I cannot point to a plethora of small startups using RL for these real-world problems .. yet.
This is a prediction, and I could be wrong in many ways - not least that LLMs digest RLs in full and learn to express their logical reasoning, approaching AGI, and use RLs internally, and so subsume and automate the use of RL.
Are VCs better at predicting the future.. I guess that is their job, and they have money on the line... but I think even they would admit they need a large portfolio to capture the unicorns.
VCs probably get a less detailed tech view than founders, but the large number of pitches they review should give them a noisy but wider overview of the whole bleeding edge of innovation.
I think startup founders are in the same future prediction business .. and arguably have more skin in the game.
Predictions would be pretty useless if they weren't somewhat controversial - a prediction we all agree on doesn't say much. Come back and chastize me if we dont see more RL startups in 12 months time !
Re: Mathematical Foundations of Reinforcement Learning
#29Earlier quoted context omitted.
> Im expecting to see a lot of hyper growth startups using RL to solve a realworld problem in engineering / logistics / medicine I love when people on hn make market predictions based on how revolutionary they think something is. I guess startup people thank they're also VC people. FYI Sutton's book came out in 1999; none of this is revolutionary anymore and yet I don't see any "hyper growth". The reason is exactly b…
Sure.. and neural networks came out a very long time ago, but are now arguably approaching usefulness in LLMs. Perhaps thats because it takes a while for the ideas to get polished/weeded and diffuse into the engineer zeitgeist .. or it could be that compute / GPUs are now powerful enough to run at the scale needed. re : "RL cannot be used to solve real world problems" .. well, I would argue that these are useful real…
> Come back and chastize me if we dont see more RL startups in 12 months time !
1999 is 26 years ago but ya sure this is the year they finally take off.
> Perhaps thats because it takes a while for the ideas to get polished/weeded and diffuse into the engineer zeitgeist .. or it could be that compute / GPUs are now powerful enough to run at the scale needed.
Or perhaps it could be that you're wrong and they're useless? Nah that couldn't be it.
Re: Mathematical Foundations of Reinforcement Learning
#30> This book, however, requires the reader to have some knowledge of probability theory and linear algebra.
This is so funny to me, I see it often and I'm always like "yea, right, some knowledge"... these statements always need to be taken with a grain of salt and an understanding that math nerds wrote them. Average programmers with average math skills (like me) beware ;)