Articles of markov process

Markovian Gaussian stationary process with continuous paths

Could you, please, help me figure out the following problem. We call a stationary Gaussian process $\xi_t$ (with continuous paths) an Ornstein-Uhlenbeck process if its correlation function $\mathbb{E}\xi_{s+t}\bar \xi_s$ is $$ \mathbb{E}\xi_{s+t}\bar \xi_s=R(t)=R(0)e^{-\alpha|t|}, \alpha\ge 0 $$ The question is that Ornstein-Uhlenbeck process is the unique Markovian stationary Gaussian process. By Markovian I mean that $$ […]

Ornstein-Uhlenbeck process: Markov, but not martingale?

I’m puzzled about properties of the Ornstein-Uhlenbeck process, given by the Itō integral $$ X_t = x e^{-\lambda t} + \sigma \int_0^t e^{-\lambda(t-s)} d W_s \,. $$ I compute that $\{X_t\}$ is not a martingale process: $$ E[X_{t’} | \mathcal{F}_t] = X_t e^{-\lambda {t’}} \neq X_t , \qquad 0 \leq t < t’. $$ by […]

Connections between SDE and PDE

I have encountered a number of situations where the solution of a PDE and a certain expectation associated to a Markov process are equal. Two examples include: The heat equation $u_t = \frac{1}{2} \Delta u$ with initial data $u(0,x)=f(x)$, considered on the whole space. Here the solution is given by $u(t,x)=\mathbb{E}(f(x+W_t))$ where $W_t$ is a […]

Building a hidden markov model with an absorbing state.

I’m working on trying to implement a hidden markov model to model the affect of a specific protein that can cut an RNA when the ribosome is translating the RNA slowly. Some brief background: The ribosome translates mRNA into protein The ribosome can occasionally pause on the mRNA due to things such as secondary structure […]

Interpretation for the determinant of a stochastic matrix?

Is there a probabilistic interpretation for the determinant of a stochastic matrix (i.e. an $n \times n$ matrix whose columns sum to unity)?

Estimating the transition matrix given the stationary distribution

Let’s say we are given a Markov chain for variable $X = [x_1, …, x_n]$; also we are given a desired stationary distribution for this graph $P_\infty = [p_1, …, p_n]^\top$. How can we design an initial distribution and a transition matrix such that in the limit gives us to the stationary distribution ? Note […]

The strong Markov property with an uncountable index set

The following definition of the strong Markov property, from Klenke’s book, supposes an index set $I$ that is not necessarily countable. However, it is explicitly mentioned previously (following Lemma 9.23) that for uncountable $I$, $X_\tau$ is not always measurable. So how can i make sense of this definition in case $I$ isn’t countable? Definition 17.12 […]

Strong Markov property of Bessel processes

I am thinking about the following: If $(B_t)_{t \geq 0}$ is a Brownian motion in $\mathbb{R}^3$, how can we show that the Bessel process (of order $3$) $(|B_t|)_{t \geq 0}$ has the strong Markov property? Any hints?

A variant of Kac's theorem for conditional expectations?

This is part of the proof of the Strong Markov property of Brownian motion given in Schilling’s Brownian motion. Here $B_t$ is a $d$-dimensional Brownian motion with admissible filtration $\mathscr{F}_t$ and some a.s. finite stopping time $\sigma$. Let $t_0=0<t_1<\cdots t_n$ and $\xi_1,\dots,\xi_n\in \mathbb{R}^d$ and $F\in \mathscr{F}_{\sigma +}$. Then $$E\left[e^{i\sum_{j=1}^n \langle \xi_j, B_{\sigma+t_j} – B_{\sigma+t_{j-1}} \rangle} […]

Is the infinitesimal generator for Lie groups the same as the infinitesimal generator of a Markov semigroup?

Is the infinitesimal generator for Lie groups related to the infinitesimal generator of a Markov semigroup? Or are they totally different concepts? Both seem to be related to the exponential map. The connection would also explain why so many people, when discussing infinitesimal generators of a Markov process, seem to have such a […]