Neuro-Dynamic Programming (Optimization and Neural Computation Series, 3)

Do the authors have a deep aversion to writing out algorithms with pseudo code?Because otherwise I'm unable to explain why they would use such terse notations instead of simple pseudo code. In fact, there is not a single algorithm outline through pseudo code in the book.For example, first visit and every visit Monte Carlo methods are not hard concepts. Yet this book would choose the most obscure notation to explain the concept. Im not saying there are errors in the book, but the way the authors choose to explain is lost to a first time student This book should not be first choice, especially given that there are such great sources for this material nowadays.Even theory wise, it leaves out important proofs to reader (several proofs refer to the authors' other text on dynamic programming).On the neural networks side, most things have completely changed since the book was written. Those chapters may not be that useful anymore.---Why you should buy the book:If you want to learn RL through the dynamic programming approach with a mathematically sound, proof based method, then the book may be for you.---The Good:This book is old, yet not outdated because it still covers important material, but it just goes to show how one might not teach the subject.Note to instructors:Please use Sutton and Barto to teach a first course in RL and make lives of your students easier.It may make for a good supplementary book for a first course, but definitely not a primary book for an RL course.

Great book. Bear in mind this is an advanced book in reinforcement learning.

The updated version is a definitive treatise of the math that go along with the ideas of reinforcement learning, approximate dynamic programming. Thoroughly enjoying the book.

Excellent text book for those who are interested in mathematical foundations of reinforcement learning.

Great book!

New one. I like it.