Articles of markov process

Markovian Gaussian stationary process with continuous paths

Could you, please, help me figure out the following problem. We call a stationary Gaussian process $\xi_t$ (with continuous paths) an Ornstein-Uhlenbeck process if its correlation function $\mathbb{E}\xi_{s+t}\bar \xi_s$ is $$ \mathbb{E}\xi_{s+t}\bar \xi_s=R(t)=R(0)e^{-\alpha|t|}, \alpha\ge 0 $$ The question is that Ornstein-Uhlenbeck process is the unique Markovian stationary Gaussian process. By Markovian I mean that $$ […]

Ornstein-Uhlenbeck process: Markov, but not martingale?

I’m puzzled about properties of the Ornstein-Uhlenbeck process, given by the Itō integral $$ X_t = x e^{-\lambda t} + \sigma \int_0^t e^{-\lambda(t-s)} d W_s \,. $$ I compute that $\{X_t\}$ is not a martingale process: $$ E[X_{t’} | \mathcal{F}_t] = X_t e^{-\lambda {t’}} \neq X_t , \qquad 0 \leq t < t’. $$ by […]

Connections between SDE and PDE

I have encountered a number of situations where the solution of a PDE and a certain expectation associated to a Markov process are equal. Two examples include: The heat equation $u_t = \frac{1}{2} \Delta u$ with initial data $u(0,x)=f(x)$, considered on the whole space. Here the solution is given by $u(t,x)=\mathbb{E}(f(x+W_t))$ where $W_t$ is a […]

Building a hidden markov model with an absorbing state.

I’m working on trying to implement a hidden markov model to model the affect of a specific protein that can cut an RNA when the ribosome is translating the RNA slowly. Some brief background: The ribosome translates mRNA into protein The ribosome can occasionally pause on the mRNA due to things such as secondary structure […]

Interpretation for the determinant of a stochastic matrix?

Is there a probabilistic interpretation for the determinant of a stochastic matrix (i.e. an $n \times n$ matrix whose columns sum to unity)?

Estimating the transition matrix given the stationary distribution

Let’s say we are given a Markov chain for variable $X = [x_1, …, x_n]$; also we are given a desired stationary distribution for this graph $P_\infty = [p_1, …, p_n]^\top$. How can we design an initial distribution and a transition matrix such that in the limit gives us to the stationary distribution ? Note […]

The strong Markov property with an uncountable index set

The following definition of the strong Markov property, from Klenke’s book, supposes an index set $I$ that is not necessarily countable. However, it is explicitly mentioned previously (following Lemma 9.23) that for uncountable $I$, $X_\tau$ is not always measurable. So how can i make sense of this definition in case $I$ isn’t countable? Definition 17.12 […]

Strong Markov property of Bessel processes

I am thinking about the following: If $(B_t)_{t \geq 0}$ is a Brownian motion in $\mathbb{R}^3$, how can we show that the Bessel process (of order $3$) $(|B_t|)_{t \geq 0}$ has the strong Markov property? Any hints?

A variant of Kac's theorem for conditional expectations?

This is part of the proof of the Strong Markov property of Brownian motion given in Schilling’s Brownian motion. Here $B_t$ is a $d$-dimensional Brownian motion with admissible filtration $\mathscr{F}_t$ and some a.s. finite stopping time $\sigma$. Let $t_0=0<t_1<\cdots t_n$ and $\xi_1,\dots,\xi_n\in \mathbb{R}^d$ and $F\in \mathscr{F}_{\sigma +}$. Then $$E\left[e^{i\sum_{j=1}^n \langle \xi_j, B_{\sigma+t_j} – B_{\sigma+t_{j-1}} \rangle} […]

Is the infinitesimal generator for Lie groups the same as the infinitesimal generator of a Markov semigroup?

Is the infinitesimal generator for Lie groups related to the infinitesimal generator of a Markov semigroup? Or are they totally different concepts? https://en.wikipedia.org/wiki/Lie_group#The_exponential_map https://en.wikipedia.org/wiki/Infinitesimal_generator_(stochastic_processes) Both seem to be related to the exponential map. The connection would also explain why so many people, when discussing infinitesimal generators of a Markov process, seem to have such a […]