Intereting Posts

Path density between two points
Properties of automorphism group of $G={Z_5}\times Z_{25}$
Comparing the deviation of a function from its mean on concentric balls
How to find the maximum value for the given $xcos{\lambda}+ysin{\lambda}$?
Evaluating $\prod\limits_{n=2}^{\infty}\left(1-\frac{1}{n^3}\right)$
Computing matrices of linear transformation under different basis
Summing $ \sum _{k=1}^{n} k\cos(k\theta) $ and $ \sum _{k=1}^{n} k\sin(k\theta) $
Proofs of consistency for two formal systems
Proof that $\ln(n^2)(\ln(n) – 1) < n$ for all $n\in\mathbb{N}$
Cryptography and Coding Theory
If $f$ is an entire function where every power series expansion has at least one 0 term, show it is a polynomial
A Variation on the Coin Problem
Shouldn't this function be discontinuous everywhere?
Partial Derivatives on Manifolds – Is this conclusion right?
Recursive Sum of Previous Term and its Inverse

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes me miss the big picture, and I might better look at some other references.

So I would like to know if there are some references with nice treatment on Markov chains/processes? They can be a whole book, some chapters of a book, some lecture notes, some webpages, …. Their approaches can be different, either with or without referring to measure theory (I think a measure-theory approach will be helpful in clarifying things, but I hope it also has some intuitive interpretation), as long as they have clear organization, and provide both a big picture and cover most of the important topics of Markov chains/processes.

I really appreciate your contribution!

- Building a hidden markov model with an absorbing state.
- Good introductory book for Markov processes
- Probability distribution for the position of a biased random walker on the positive integers
- Markov Chain transitional probability query.
- Expected number of runs
- Modified gambler's ruin problem: quit when going bankruptcy or losing $k$ dollars in all

- Equivalence between norms in $H_0^1(\Omega)\cap H^2(\Omega)$.
- What is combinatorial homotopy theory?
- Five squares in a box.
- Chased By a Lion and other Pursuit Problems
- Asymptotic (divergent) series
- Reference to a basic result implying existence and uniqueness of the base-$b$ representation
- Definition of definition
- Books for Hyperbolic Geometry.
- Why define norm in $L_p$ in that way?
- Recommending books for introductory differential geometry

Two excellent introductions are James Norris’s “Markov Chains” and Pierre Bremaud’s “Markov Chains: Gibbs fields, Monte Carlo simulation, and queues”. Both books assume a motivated student who is somewhat mathematically mature, though Bremaud reviews basic probability before he gets going.

Also the wonderful book “Markov Chains and Mixing Times” by Levin, Peres, and Wilmer is available online here. It starts right with the definition of Markov Chains, but eventually touches on topics in current research. So it is pretty advanced, but also well worth a look.

While not as advanced as the books mentioned above, if you are looking for examples related to applications of Markov Chains and a nice “brief” treatment you might look at Chapter 5, of Fred Robert’s book: Discrete Mathematical Models, Prentice-Hall, 1976.

- Prove that $\lim\limits_{n\to\infty}1 + \frac{1}{1!} + \frac {1}{2!} + \cdots + \frac{1}{n!}\ge\lim\limits_{n\to\infty}(1+\frac{1}{n})^n$
- Calculating an improper integral as a limit of a sum.
- Does differentiation only work on open sets?
- Finite Group and normal Subgroup
- Complex numbers and Roots of unity
- Every metrizable space with a countable dense subset has a countable basis
- $\sup$, $\limsup$ or else, what is the error here?
- Probability of a point taken from a certain normal distribution will be greater than a point taken from another?
- Show differential of $f:S^{m}\times S^{n}\to S^{m+n+mn}$ is injective
- Evaluate the double sum $\sum_{m=1}^{\infty}\sum_{n=1}^{m-1}\frac{ 1}{m n\left(m^2-n^2\right)^2}$
- Finite vs infinite dimensional vector spaces
- Find the number of irreducible polynomials in any given degree
- Help with calculating infinite sum $\sum_{n=0}^{\infty}\frac1{1+n^2}$
- Limit distribution of infinite sum of Bernoulli random variables
- Separated schemes and unicity of extension