Introduction To Stochastic Processes

The objective here is to introduce the elements of stochastic processes in a rather concise manner where we present the two most important parts in stochastic processes — Markov chains and stochastic analysis. The readers are lead directly to the core of the topics, and further details are collated in a section containing abundant exercises and more materials for further reading and studying.

In the part on Markov chains, the core is the ergodicity. By using the minimal non-negative solution method, we deal with the recurrence and various ergodicity. This is done step by step, from finite state spaces to denumerable state spaces, and from discrete time to continuous time. The proof methods adopt the modern techniques, such as coupling and duality methods. Some very new results are included, such as the estimate of the spectral gap. The structure and proofs in the first part are rather different from other existing textbooks on Markov chains.

In the part on stochastic analysis, we cover the martingale theory and Brownian motions, the stochastic integral and stochastic differential equations with emphasis on one dimension, and the multidimensional stochastic integral and stochastic equation based on semimartingales. We introduce three important topics here: the Feynman–Kac formula, random time transform and Girsanov transform. As an essential application of the probability theory in classical mathematics, we also deal with the famous Brunn–Minkowski inequality in convex geometry.