Intereting Posts

Why is the math for negative exponents so?
Integral formula for $\int_{0}^{\infty}e^{-3\pi x^{2}}((\sinh \pi x)/(\sinh 3\pi x))\,dx$ by Ramanujan
Why does $A^TAx = A^Tb$ have infinitely many solution algebraically when $A$ has dependent columns?
Ideas about Proofs
Is there a General Formula for the Transition Matrix from Products of Elementary Symmetric Polynomials to Monomial Symmetric Functions?
How to find the center of mass of half a torus?
Integral $\int_0^\infty\frac{1}{\sqrt{x}}\left(1+\log\frac{1+e^{x-1}}{1+e^x}\right)dx$
Proving $\sqrt2$ is irrational
Homomorphisms of graded modules
Can an odd perfect number be divisible by $825$?
Problem about limit of Lebesgue integral over a measurable set
Prove by induction that every connected undirected graph with n vertices has at least n-1 edges
Implementing Ornsteinâ€“Uhlenbeck in Matlab
Express $x^8-x$ as a product of irreducibles in $\Bbb Z_2$
If $a,b,c$ are positive integers, with $a^2+b^2-ab=c^2$ prove that $(a-b)(b-c)\le0$.

Which is a good introductory book for Markov chains and Markov processes?

Thank you.

- Markov chain: join states in Transition Matrix
- Example of a Markov chain transition matrix that is not diagonalizable?
- Nice references on Markov chains/processes?
- Modified gambler's ruin problem: quit when going bankruptcy or losing $k$ dollars in all
- Computational methods for the limiting distribution of a finite ergodic Markov chain
- Finite State Markov Chain Stationary Distribution

- Understanding the Musical Isomorphisms in Vector Spaces
- Book about technical and academic writing
- Analytic Capacity
- History of the matrix representation of complex numbers
- Reference for combinatorial game theory.
- Need Suggestions for beginner who is in transition period from computational calculus to rigorous proofy Analysis
- Status of the classification of non-finitely generated abelian groups.
- mathematical analysis books with many examples
- An introductory textbook on functional analysis and operator theory
- Are hyperoperators primitive recursive?

*Theory of Markov Processes* by Eugene Dynkin is a paperback published by Dover, so it has the advantage of being inexpensive. The author has made many contributions to the subject. Dynkin’s lemma, the Dynkin diagram and the Dynkin system are named after him.

“An Introduction to Stochastic Modeling” by Karlin and Taylor is a very good introduction to Stochastic processes in general. Bulk of the book is dedicated to Markov Chain. This book is more of applied Markov Chains than Theoretical development of Markov Chains. This book is one of my favorites especially when it comes to applied Stochastics.

An Introduction to Markov Chain Analysis and An Introduction to Markov Processes would be a good start.

A foundational paper is :

An Introduction to Hidden Markov Models – Rabiner 1986

which explains the processes from first principles which is sadly rare on academic papers.

Allen, Arnold O.: “Probability, Statistics, and Queueing Theory with Computer Science Applications”, Academic Press, Inc., San Diego, 1990 (second Edition)

This is a very good book including some chapters about Markov chains, Markov processes and queueing theory.

- Find two elements that don't have a gcd in a subring of Gaussian integers
- Explain why $x^+=A^+b$ is the shortest possible solution to $A^TA\hat{x}=A^Tb$
- Help with Dedekind cuts
- What is wrong with this proof that there are no odd perfect numbers?
- Difference of random variables
- asymptotic expansion from 3 leading terms
- Relationship between eigendecomposition and singular value decomposition
- How to compute Riemann-Stieltjes / Lebesgue(-Stieltjes) integral?
- How to integrate $\int\frac{\ln x\,dx}{x^2+2x+4}$
- separation theorem for probability measures
- How is the extended real number line modeled?
- Prove that $f$ continuous and $\int_a^\infty |f(x)|\;dx$ finite imply $\lim\limits_{ x \to \infty } f(x)=0$
- What is the max of $n$ such that $\sum_{i=1}^n\frac{1}{a_i}=1$ where $2\le a_1\lt a_2\lt\cdots\lt a_n\le 99$?
- Existence of d-regular graphs
- If $(e_1,…,e_n)$ is an orthonormal basis, why does $\operatorname{trace}(T) =\langle Te_1,e_1\rangle +\cdots+\langle Te_n,e_n\rangle $?