Intereting Posts

Minimum of a quadratic form
Largest eigenvalue of a positive semi-definite matrix is less than or equal to sum of eigenvalues of its diagonal blocks
Proof of properties of dual cone
Does there exist an unbounded function that is uniformly continuous?
Intuitive explanation of variance and moment in Probability
Prove that CX and CY are perpendicular
Homomorphism from $\mathbb{Q}/ \mathbb{Z}\rightarrow \mathbb{Z}/n\mathbb{Z}$
Positive definiteness of Fubini-Study metric
General rule for the integral of $\frac1{x^n}$
Equivalence of definitions of prime ideal in commutative ring
Property of $\{0^n, 1^n, \ldots\}$
Calculation of the $s$-energy of the Middle Third Cantor Set
Fourier transform as a Gelfand transform
If $f\!: X\simeq Y$, then $X\!\cup_\varphi\!\mathbb{B}^k \simeq Y\!\cup_{f\circ\varphi}\!\mathbb{B}^k$.
Limit-Fundamental Concept?

I looked for a formal definition of Markov chain and was confused that all definitions I found restrict chain’s state space to be countable. I don’t understand purpose of such a restriction and I have feeling that it does not make any sense.

So my question is: can state space of a Markov chain be continuum? And if not then why?

Thanks in advance.

- Nice references on Markov chains/processes?
- Null-recurrence of a random walk
- Expected number of runs
- When the product of dice rolls yields a square
- Symmetric random walk and the distribution of the visits of some state
- Unique Stationary Distribution for Reducible Markov Chain

- Expectation of hitting time of a markov chain
- Markov process vs. markov chain vs. random process vs. stochastic process vs. collection of random variables
- Modified gambler's ruin problem: quit when going bankruptcy or losing $k$ dollars in all
- 6-digit password - a special decoding method
- Why do Markov matrices converge to the same vector after infinite steps irrespective of the starting vector?
- Unique Stationary Distribution for Reducible Markov Chain
- Example of a Markov chain transition matrix that is not diagonalizable?
- Knight returning to corner on chessboard — average number of steps
- Symmetric random walk and the distribution of the visits of some state
- How can a Markov chain be written as a measure-preserving dynamic system

It seems you are thinking of a *Markov process*, rather than a *Markov Chain*.

The Markov chain is usually defined as a Markov process that has a discrete (finite or countable) state space.

“Often, the term Markov chain is used to mean a Markov process which

has a discrete (finite or countable) state-space” (ref)

A Markov process is (usually) a slightly more general thing, it’s only required to exhibit the Markov property, which makes sense for uncountable states (and “time”), or more general supports. So, for example, the continuous random walk is normally considered a Markov process, but not a Markov Chain.

Yes, it can. In some quarters the “chain” in Markov chain refers to the discreteness of the time parameter. (A notable exception is the work of K.L. Chung.) The evolution in time of a Markov chain $(X_0,X_1,X_2,\ldots)$ taking values in a measurable state space $(E, {\mathcal E})$ is governed by a one-step transition kernel $P(x,A)$, $x\in E$, $A\in{\mathcal E}$:

$$

{\bf P}[ X_{n+1}\in A|X_0,X_1,\ldots,X_n] = P(X_n,A).

$$

Two fine references for the subject are *Markov Chains* by D. Revuz and *Markov Chains and Stochastic Stability* by S. Meyn and R. Tweedie.

- Unusual pattern in the distribution of odd primes
- Integral is area under the graph
- Av = Bv for all v implies A = B?
- Perspective problem – trapezium turned square
- Showing $(P\to Q)\land (Q\to R)\equiv (P\to R)\land$ {without truth table}
- How to prove Lagrange trigonometric identity
- On integrals related to $\int^{+\infty}_{-\infty} e^{-x^2} dx = \sqrt{\pi}$
- Can someone explain this proof of the product property of square roots?
- let $H\subset G$ with $|G:H|=n$ then $\exists~K\leq H$ with $K\unlhd G$ such that $|G:K|\leq n!$ (Dummit Fooote 4.2.8)
- Extreme points of unit ball in $C(X)$
- Proof that $\inf A = \sup B$
- The kernel and range of the powers of a self-adjoint operator
- Convergence of test-functions is not induced by any metric.
- Intuition for Smooth Manifolds
- Incredible frequency of careless mistakes