Stochastic Approximation: A Dynamical Systems Viewpoint

This book serves as an advanced text for a graduate course on stochastic algorithms for the students of probability and statistics, engineering, economics and machine learning. This second edition gives a comprehensive treatment of stochastic approximation algorithms based on the ordinary differential equation (ODE) approach which analyses the algorithm in terms of a limiting ODE. It has a streamlined treatment of the classical convergence analysis and includes several recent developments such as concentration bounds, avoidance of traps, stability tests, distributed and asynchronous schemes, multiple time scales, general noise models, etc., and a category-wise exposition of many important applications. It is also a useful reference for researchers and practitioners in the field.

Presents a comprehensive view of the ODE-based approach for the analysis of stochastic approximation algorithms Discusses important themes on stability tests, concentration bounds, and avoidance of traps Covers very recent developments with copious pointers to related literature

Auteur

Vivek Shripad Borkar is Professor at the Department of Electrical Engineering, Indian Institute of Technology (IIT) Bombay, Powai, Mumbai, India. Earlier, he held positions at the TIFR Centre for Applicable Mathematics and Indian Institute of Science in Bengaluru; Indian Institute of Science, Bengaluru; Tata Institute of Fundamental Research and Indian Institute of Technology Bombay in Mumbai. He also held visiting positions at the Massachusetts Institute of Technology (MIT), the University of Maryland at College Park, the University of California at Berkeley, and the University of Illinois at Urbana-Champaign, USA.
Professor Borkar obtained his B.Tech. (Electrical Engineering) from the IIT Bombay in 1976, MS (Systems and Control Engineering) from Case Western Reserve University in 1977, and Ph.D. (Electrical Engineering and Computer Sciences) from the University of California, Berkeley, USA, in 1980. He is Fellow of American Mathematical Society, IEEE, and the World Academy of Sciences, and of various science and engineering academies in India. He has won several awards and honours in India and was an invited speaker at the ICM 2006 in Madrid. He has authored/co-authored six books and several archival publications. His primary research interests are in stochastic optimization and control, covering theory and algorithms.

Contenu

1. Introduction.- 2. Convergence Analysis.- 3. Finite Time Bounds and Traps.- 4. Stability Criteria.- 5. Stochastic Recursive Inclusions.- 6. Asynchronous Schemes.- 7. A Limit Theorem for Fluctuations.- 8. Multiple Timescales.- 9. Constant Stepsize Algorithms.- 10. General Noise Models.- 11. Stochastic Gradient Schemes.- 12. Liapunov and Related Systems.- 13. Appendix A: Topics in Analysis.- 14. Appendix B: Ordinary Differential Equations.- 15. Appendix C: Topics in Probability.- Bibliography.- Index.