Intereting Posts

Find the shortest distance from the triangle with with vertices in $(1,1,0),(3,3,1),(6,1,0)$ to the point $(9,5,0)$.
$S=\{(n,{1\over n}):n\in\mathbb{N}\}$ is closed in $X$?
Why can't three unit regular triangles cover a unit square?
Solving a system of PDEs with method of characteristics
Show that the $k$th forward difference of $x^n$ is divisible by $k!$
$p \leqslant q \leqslant r$. If $f \in L^p$ and $f \in L^r$ then $ f \in L^q$?
How to determine which amounts of postage can be formed by using just 4 cent and 11 cent stamps?
Greatest number of planes we can get when dividing with lines and circles
Calculate the lengths of the heights descending from triangle vertices
the sum of a series
Why is Lebesgue so often spelled “Lebesque”?
What is the strategy-proofness of maximal lottery?
Circles Of Descartes
Why is $-\gamma = \int_0^1 \frac{e^{-z}-1}{z}dz+\int_1^\infty \frac{e^{-z}}{z}dz$
Math games for car journeys

I am currently learning about Markov chains and Markov processes, as part of my study on stochastic processes. I feel there are so many properties about Markov chain, but the book that I have makes me miss the big picture, and I might better look at some other references.

So I would like to know if there are some references with nice treatment on Markov chains/processes? They can be a whole book, some chapters of a book, some lecture notes, some webpages, …. Their approaches can be different, either with or without referring to measure theory (I think a measure-theory approach will be helpful in clarifying things, but I hope it also has some intuitive interpretation), as long as they have clear organization, and provide both a big picture and cover most of the important topics of Markov chains/processes.

I really appreciate your contribution!

- Generalization of the Jordan form for infinite matrices
- Markov Chain and Forward and Backward Probabilities with Alice and Bob
- Differences of consecutive hitting times
- Getting at least $k$ heads in a row $l$ times over after at most $m$ flips.
- Time to reach a final state in a random dynamical system (answer known, proof unknown)
- What is the difference between all types of Markov Chains?

- Complex analysis textbook advise
- Ramanujan's False Claims
- Logic and number theory books
- efficient and accurate approximation of error function
- Unitary Farey Sequence Matrices
- Reference for matrix calculus
- Centre of symmetric group algebra
- Representation theory over $\mathbb{Q}$
- A good book for metric spaces?
- Soft Question: Weblinks to pages with explanation on quadratics.

Two excellent introductions are James Norris’s “Markov Chains” and Pierre Bremaud’s “Markov Chains: Gibbs fields, Monte Carlo simulation, and queues”. Both books assume a motivated student who is somewhat mathematically mature, though Bremaud reviews basic probability before he gets going.

Also the wonderful book “Markov Chains and Mixing Times” by Levin, Peres, and Wilmer is available online here. It starts right with the definition of Markov Chains, but eventually touches on topics in current research. So it is pretty advanced, but also well worth a look.

While not as advanced as the books mentioned above, if you are looking for examples related to applications of Markov Chains and a nice “brief” treatment you might look at Chapter 5, of Fred Robert’s book: Discrete Mathematical Models, Prentice-Hall, 1976.

- Radical extension
- l'Hopital's questionable premise?
- Frog on a 3 by 3 matrix
- Ideal Coin Value Choices
- Galerkin method for Poisson's equation
- True or False: Every finite dimensional vector space can made into an inner product space with the same dimension.
- If multiplication is not repeated addition
- Is this interpretation of the Dirac-measure property correct?
- Every compact metric space is image of space $2^{\mathbb{N}}$
- Must a certain continued fraction have “small” partial quotients?
- What are good books to learn graph theory?
- For 3 numbers represented with $n$ bits in binary, how many bits is required for their product.
- Likelihood Functon.
- Is a left invertible element of a ring necessarily right invertible?
- Conjugacy classes of the nonabelian group of order 21