So the problem is: Let $X$ be a Markov chain with state space $E = ${a,b} and transition matrix $$p=\begin{bmatrix}0.4 &0.6 \\1 & 0 \end{bmatrix}$$ and suppose that a reward of $g(i,j)$ units is received for every jump from $i$ to $j$ where $$g=\begin{bmatrix}3 &2 \\-1 & 1 \end{bmatrix}$$ Find $$\lim_{n\to \infty}\frac 1{n+1}\sum_{m=0}^{n}g(X_m, X_{m+1})$$ The […]

System Alice and Bob are moving independently from one city to another. There are $d$ cities, the probability of moving to another city (for each individual) is $m$ and each move is equiprobable (there is no preferred city). The choice of moving and choice of where to move to of Alice are independent of the […]

I need an help with the following exercise. I’ve already seen this question Prove that Brownian Motion absorbed at the origin is Markov but I don’t understand the answer. Also I would like to prove the thing differently (at least the two ways seem different to me). So let $\tau$ be the first time the […]

“To every Q-matrix corresponds a unique Markov process.” I’m trying to understand Klenke’s proof of the uniqueness part of this proposition. Klenke’s proof Following is an adapted version of Klenke’s proof of the uniqueness part. The original can be found here (“Now assume that…”). Let $(p_t)_{t\geq0}, (r_t)_{t\geq0}$ be Markov semigroups. Let $q$ be a bounded […]

1. Given a discrete-time Markov chain without independent increments, is the embedding of it into a continuous time Markov chain (i.e. via the use of exponential waiting times) an example of a continuous time Markov process without independent increments? 2. Does there exist a continuous-time Markov process with continuous sample paths which does not have […]

A Markov Chain on states 0,1,….. has transition probabilities $P_{ij}=1/(i+2)$ for j=0,1,….,i,i+1. I’m supposed to find the stationary distribution. So do I take the limit as n goes to infinity of 1/n multiplied by $P_{ij}$?

In his treatise “An Introduction to Lévy and Feller Processes” (arXiv link), Prof. Dr. René Schilling gives a short and seemingly straightforward proof for the claim that a continuous Lévy process is integrable (Lemma 8.2 on p. 50). More precisely, the claim goes as follows. Lemma Let $(X_t)_{t\geq0}$ be a Lévy process with càdlàg paths […]

Let $X=(X_t)_{t\geq0}$ be a Markov process on a state space $\Gamma$ (a Hausdorff topological vector space), let $A$ be the infinitesimal generator of $X$ and let $\mathcal C(\Gamma)$ the space of continuous functions on $\Gamma$. Then for each continuous function $f \in \mathcal C(\Gamma)$ the operator $A+f$ is the infinitesimal generator of the semigroup $(P_t^f)_{t\geq0}$ […]

I’m trying to find a solution to a system of equations, but its quite different from anything I’ve come across before. I believe there is a solution, but I could be wrong. $\mathbf{A} = \mathrm{diag}(\mathbf{a})$ is an $n \times n$ diagonal matrix whose diagonal entries $\mathbf{a}_i \in [0,1]$ are the unknown quantities in my system. […]

Could you, please, help me figure out the following problem. We call a stationary Gaussian process $\xi_t$ (with continuous paths) an Ornstein-Uhlenbeck process if its correlation function $\mathbb{E}\xi_{s+t}\bar \xi_s$ is $$ \mathbb{E}\xi_{s+t}\bar \xi_s=R(t)=R(0)e^{-\alpha|t|}, \alpha\ge 0 $$ The question is that Ornstein-Uhlenbeck process is the unique Markovian stationary Gaussian process. By Markovian I mean that $$ […]

Intereting Posts

How to prove that if $a\equiv b \pmod{2n}$ then $a^2\equiv b^2 \pmod{2^2n}$
Tensor double dot product
$k$ litres of Milk delivery with minimum cost
Intuitive explantions for the concepts of divisor and genus
Definition of semi-ring homomorphism
Show $au_x+bu_y=f(x,y)$ gives $u(x,y)=(a^2+b^2)^{-\frac{1}{2}}\int_{L}fds +g(bx-ay)$ if $a\neq 0$.
Is there any partial sums of harmonic series that is integer?
What contour should be used to evaluate $\int_0^\infty \frac{\sqrt{t}}{1+t^2} dt$
Is the function $ f(x,y)=xy/(x^{2}+y^{2})$ where f(0,0) is defined to be 0 continuous?
Evaluating the factorial-related limit $\lim_{x \to \infty} (x + 1)!^{1 / (x + 1)} – x!^{1/x}$
Proving that a Function is Analytic Given that it is Equal to the Complex Conjugate of an analytic Function
Applications for Homology
Extended Euclidean algorithm with negative numbers
Finding specific ideals of a ring
Itō Integral has expectation zero