Preface xix
Acknowledgments xxi
1 Introduction 1
Part I Probabilistic Reasoning
2 Representation 19
3 Inference 43
4 Parameter Learning 71
5 Structure Learning 97
6 Simple Decisions 111
Part II Sequential Problems
7 Exact Solution Methods 133
8 Approximate Value Functions 161
9 Online Planning 181
10 Policy Search 213
11 Policy Gradient Estimation 231
12 Policy Gradient Optimization 249
13 Actor-Critic Methods 267
14 Policy Validation 281
Part III Model Uncertainty
15 Exploration and Exploitation 299
16 Model-Based Methods 317
17 Model-Free Methods 335
18 Imitation Learning 335
Part IV State Uncertainty
19 Beliefs 379
20 Exact Belief State Planning 407
21 Offline Belief State Planning 427
22 Online Belief State Planning 453
23 Controller Abstractions 471
Part V Multiagent Systems
24 Multiagent Reasoning 493
25 Sequential Problems 517
26 State Uncertainty 533
27 Collaborative Agents 545
Appendices
A Mathematical Concepts 561
B Probability Distributions 573
C Computational Complexity 575
D Neural Representations 581
E Search Algorithms 599
F Problems 609
G Julia 627
References 651
Index 671