Jiongmin Yong Knihy

Pontryagin's maximum principle and Bellman's dynamic programming are the two main approaches for solving stochastic optimal control problems. Interestingly, these methods have developed independently, prompting the question of their relationship in this context. Prior research on this relationship, before the 1980s, often presented results in heuristic terms and relied on restrictive assumptions that were not widely applicable. The Pontryagin-type maximum principle involves an adjoint equation, which is an ordinary differential equation (ODE) in deterministic scenarios and a stochastic differential equation (SDE) in stochastic cases. This combination of the adjoint equation, the original state equation, and the maximum condition forms what is known as an (extended) Hamiltonian system. Conversely, Bellman's dynamic programming employs a partial differential equation (PDE)—first-order in deterministic cases and second-order in stochastic ones—known as the Hamilton-Jacobi-Bellman (HJB) equation. The exploration of these two frameworks reveals their foundational roles in understanding and solving stochastic optimal control problems, despite their independent evolution in the literature.

This book uses a small volume to present the most basic results for deterministic two-person differential games. The presentation begins with optimization of a single function, followed by a basic theory for two-person games. For dynamic situations, the author first recalls control theory which is treated as single-person differential games. Then a systematic theory of two-person differential games is concisely presented, including evasion and pursuit problems, zero-sum problems and LQ differential games. The book is intended to be self-contained, assuming that the readers have basic knowledge of calculus, linear algebra, and elementary ordinary differential equations. The readership of the book could be junior/senior undergraduate and graduate students with majors related to applied mathematics, who are interested in differential games. Researchers in some other related areas, such as engineering, social science, etc. will also find the book useful.

The book presents essential theories of mathematical analysis, covering topics such as metric spaces, limits of sequences, continuous functions, differentiation, the Riemann integral, uniform convergence, and series. It emphasizes the foundational role of mathematical analysis in various fields, including physics, chemistry, biology, engineering, finance, and economics, making it a vital tool for both pure and applied mathematics research.