Numerical methods for control and games

Dr. Qingshuo Song

Department of Mathematics
Wayne State University

March 14, 2006
202 Madison Hall
1:00 PM (Coffee and Cookies at 12:30)

Abstract

This work is concerned with numerical methods for controlled regime-switching diffusions. Numerical methods using Markov chain approximation techniques are developed for stochastic control problems, and two-player differential game problems. In the stochastic control problem, the approximating Markov chain has two components: One component is an approximation to the diffusion, whereas the other keeps track of the regimes. A new definition for local consistency of the two-component Markov chain approximation is provided. Using the consistent approximation scheme, convergence of the algorithms is derived by means of weak convergence methods. Then the cost and value functions are shown to be convergent. In addition, controlled regime-switching jump diffusion models and discounted cost control models are treated. For the stochastic differential game problem, the constructed Marko chain has two components similar to the control problem. In addition to consistency and convergence of the numerical methods, a systematic study on the existence of the saddle points of the game is carried out. A new proof of the existence of saddle point of the game is provided using dynamic programming equations. This new proof enables us to treat more general systems with certain nonseparable structure so as to extending the existing literature in this direction. For both the stochastic control problem and the game problem, which use regime-switching models, simulations and numerical experiments are conducted. Several examples, including manufacturing systems and pursuit-evasion games, are used to demonstrate the performance of the methods and their convergence.