How is it different from the first edition?
New Draft of “Reinforcement Learning: An Introduction, Second Edition”
11–20 of 36 posts
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#12I've uploaded a mirror of the PDF: https://instant.io/#678d0be07a0f2260ec6f9b134ec0a1d7c4325e99 If you can, leave the tab open for a while to keep seeding the file! (It's a torrent.)
Thanks!
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#13Will RL take over Deep Convolutional Networks as having the best results?!
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#14Will RL take over Deep Convolutional Networks as having the best results?!
I think they are parallel and can be used together, like Deep Reinforcement Learning.
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#15If anyone is interested in exploring this, I highly recommend skipping to the blackjack exercise. This isn't to say you should read the book fully but I feel blackjack is simple and enough to get you addicted. Starting with Monte Carlo sampling is quite approachable and, once you've done that, extending it is relatively easy.
I also highly recommend reading about Edward O. Thorp[1]. He was a friend of Claude Shannon, whom he frequented Las Vegas with, and used the IBM 704 (first mass produced computer with floating point ops) to explore blackjack game theory in ~1956. To apply his research he borrowed $10,000 from someone with mob connections and won $11,000 in a single weekend. He also developed the first wearable computer (for a specific definition of computer).
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#16Will RL take over Deep Convolutional Networks as having the best results?!
I think they are parallel and can be used together, like Deep Reinforcement Learning.
I think they are parallel and can be used together
Yep:
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#17I've uploaded a mirror of the PDF: https://instant.io/#678d0be07a0f2260ec6f9b134ec0a1d7c4325e99 If you can, leave the tab open for a while to keep seeding the file! (It's a torrent.)
Thanks a lot !
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#18How is it different from the first edition?
There's a "Preface to the Second Edition" near the front, which has a summary of changes. Main points are: 1) notation was overhauled, 2) Chapters 2-8 were reworked to only use tabular methods, with function approximation introduced later; 3) the function approximation coverage is then greatly expanded in the second section of the book (Chs. 9-13); and 4) new chapters 14-15 on connections between RL and psychology and neuroscience.
The scope is generally about the same though, perhaps because it's intended to be used as a single-semester textbook, so there isn't a big expansion into areas of RL other than those covered in the first edition (e.g. POMDPs are only briefly mentioned).
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#19just reading page 15, arg max = maximal ..., I think that global maximum or local maximum is better than maximal.
I would like to read all the interesting fruit of RL in just one hour, can someone suggest a short book for someone with advanced maths skills?
Thanks a lot to the authors the book seems to be really interesting.
Edit: In page 25, an extended example: tic-tac-toe the rule to update the value of each state v(s)=v(s)+a(v(s')-v(s)) doesn't take into account that if in s' there is a winning strategy by the policy then previous values is also part of a winning strategy. So if v(s')=1 (win) then v(s)=1 (I can win). In my very humble opinion, the author should digress a title to talk about this very important point.
Re: New Draft of “Reinforcement Learning: An Introduction, Second Edition”
#20I did my thesis about reinforced learning. Unfortunately didn't work with that afterwards, but I really think it's one of the most interesting approaches to machine learning for real world application.