Markov chains Markov processes. Classification of states and Markov chains. Stationary distributions and asymptotic distributions. Asorbing states and time to absorption. Intensities, the Poisson process and spatial Poisson processes, Hidden Markov models