Articles of markov chains

Chuck Norris' Coupling of Markov Chains: An Invariant Distribution

I’m having some difficulty understanding a proof in James(‘Chuck’) Norris book on markov chains. Let $P$ be irreducible and aperiodic, with an invariant distribution $\pi$. Let $\left(X_n\right)_{n\geq 0}$ be a discrete time Markov$\left(\lambda,P\right)$, where $\lambda$ is the initial distribution. In this setting, Chuck wants to prove that the $X_n$ converges to an equilibrium distribution. The […]

Finding Markov chain transition matrix using mathematical induction

Let the transition matrix of a two-state Markov chain be $$P = \begin{bmatrix}p& 1-p\\ 1-p& p\end{bmatrix}$$ Questions: a. Use mathematical induction to find $P^n$. b. When n goes to infinity, what happens to $P^n$? Attempt: i’m able to find $$P^n = \begin{bmatrix}1/2 + 1/2(2p-1)^n& 1/2 – 1/2(2p-1)^n\\ 1/2 – 1/2(2p-1)^n & 1/2 + 1/2(2p-1)^n\end{bmatrix}$$ I […]

Given an invariant distribution is the (finite state) Markov transition matrix unique?

Doeblin’s theorem states that for a given transition probability matrix there exists a unique invariant distribution for that chain. Is the converse true as well? Can two (finite state, discrete) Markov Chains have the same invariant distribution, but have different transition matrices? I don’t think so, but I have only tried a proof by contradiction: […]

Unique Stationary Distribution for Reducible Markov Chain

Is it possible for a reducible markov chain to have a unique stationary distribution. Consider e.g. the markov chain with transition matrix below $$A= \begin{pmatrix} 1 & 0 & 0 \\\ 0.2 & 0.7 & 0.1 \\\ 0.3 & 0.3 & 0.4 \end{pmatrix}$$ I believe A is a reducible markov chain ({1} and {2,3} are […]

Error in Billingsley?

Problem 8.25 in the third edition of Probability and Measure by Billingsley (1995, p. 142) is as follows: Suppose that an irreducible [Markov] chain of period $t>1$ has a stationary distribution $\{\pi_j\}$. Show that, if $i\in S_{\nu}$ and $j\in S_{\nu+\alpha}$ ($\nu+\alpha$ reduced modulo $t$), then $$\lim_np_{ij}^{(nt+\alpha)}=\pi_j.\tag{$\diamondsuit$}$$ Show that $$\lim_n n^{-1}\sum_{m=1}^np_{ij}^{(m)}=\pi_j/t\quad\text{for all $i$ and $j$.}\tag{$\clubsuit$}$$ Here, the sets […]

expected life absorbing Markov Chain

No idea on how to start this question. Any help would be much appreciated. A flea lives on a polyhedron with N vertices, labelled $1, . . . , N$. It hops from vertex to vertex in the following manner: if one day it is on vertex $i > 1$, the next day it hops […]

Finding the steady state Markov chain?

I have drawn a certain Markov chain with a weird transition matrix. Here’s the drawing: And here’s the transition matrix: My problem is that I don’t quite know how to calculate the steady state probabilities of this chain, if it exists. I need a bit of help, because just getting the $\pi$’s, they don’t sum […]

Example of a Markov chain transition matrix that is not diagonalizable?

It is well-known that every detailed-balance Markov chain has a diagonalizable transition matrix. I am looking for an example of a Markov chain whose transition matrix is not diagonalizable. That is: Give a transition matrix $M$ such that there exists no invertible matrix $U$ with $U^{-1} M U$ a diagonal matrix. Is there a combinatorial […]

Markov chain with finite positive recurrent states

If I have a Markov chain with finite positive recurrent states $\in S$, then that means starting from a given state $y$, the expected number of steps to return to state $y$ is finite. Now, if I start at state $j\ne y$ where $y,j\in S$, then I am under the impression that the expected number […]

Characterization of conditional independence

Definition: Let $\mathcal{G},\mathcal{K},\mathcal{H}$ be $\sigma $-subalgebras of $\mathcal{F}$, where $\left( {\Omega ,\mathcal{F},\mathbb{P}} \right)$ is a given probability space. We say that $\mathcal{G}$ and $\mathcal{H}$ are conditionally independent given $\mathcal{K}$ if, $\mathbb{P}$-almost surely, $\mathbb{P}\left( {G \cap H|\mathcal{K}} \right) = \mathbb{P}\left( {G|\mathcal{K}} \right)\mathbb{P}\left( {H|\mathcal{K}} \right),\forall G \in \mathcal{G},\forall H \in \mathcal{H}$. Let $\mathcal{K} \subseteq \mathcal{H}$. Prove that […]