# Show a logarithmic sequence converging almost surely and compute its limit

Given i.i.d $X_1, X_2,\cdots, X_n$ with their values belong to $\left\{1,\cdots, l\right\}$ such that $p_k = P(X_i=k) > 0$ for each $k$. Let $N_n(k)$ be the number of $X_1, X_2,\cdots$ whose value equals to $k$, and $\prod_n = \prod_{k=1}^{l} (p_k)^{N_n(k)}$. Show that the sequence $\left\{\frac{\log{\prod_n}}{n}\right\}$ converges almost surely and compute its limit.

My attempt: First, for proving a sequence converging almost surely, we need to show: for every $\epsilon > 0$, $\lim_{n\rightarrow \infty} \ P(\sup_{k\geq 1} |\frac{\log{\prod_{n+k}}}{n+k} -\frac{\log{\prod_n}}{n}| > \epsilon) = 0$.

Using property of logarithmic function, we have: $|\frac{\log{\prod_{n+k}}}{n+k} – \frac{\log{\prod_{n}}}{n}| = |\sum_{i=1}^{l} (\frac{N_{n+k}(i)\log(p_i)}{n+k} – \frac{N_n(i)\log(p_i)}{n})|\leq \frac{2\max_{i=1,2,…,l}|\log(p_i)|nk}{n(n+k)}$

(this is due to $4$ facts: $n(N_{n+k}(i) – N_n(i))\leq kn$, the triangle inequality, $\sum_{i=1}^{l} N_{n}(i) = n$, and $max_{i=1,2,…,l} |\log(p_i)|$ exists, as $p_is$ are all fixed numbers).

Now, since $p_{i}$ is given, dividing both the numerator and denominator by $k\geq 1$, we easily see that the $RHS\leq \frac{2n\max_{i=1,2,…,l}|\log(p_i)|}{n(n+1)}$. Let $n\rightarrow \infty$, $\frac{2\max_{i=1,2,…,l}|\log(p_i)|}{n+1}\rightarrow 0$, which implies $\lim_{n\rightarrow \infty} (\sup_{k\geq 1}|\frac{\log{\prod_{n+k}}}{n+k} – \frac{\log{\prod_{n}}}{n}|)\rightarrow 0$. Done!

For the limit, we could use this inequality: $\frac{l[\max_{i=1,2,…,l}\log(p_i)]}{n}\geq \frac{\log{\prod_n}}{n}\geq \frac{l[\min_{i=1,2,…,l} \log(p_i)]}{n}$. Let $n\rightarrow \infty$, both the upper and lower bound go to $0$, so by Squeeze’s theorem, we conclude that our limit is $\fbox{0}$.

My question: Is my solution above correct? It’s quite a technical proof for the a.s convergent, so I’m not sure if I accidentally made a mistake (as far as my review goes, nothing was found yet). Any thoughts would really be appreciated.

#### Solutions Collecting From Web of "Show a logarithmic sequence converging almost surely and compute its limit"

Note that $N_n(k)=\sum_{i=1}^n 1_{X_i=k}$. Therefore, by SLLN, $\dfrac{N_n(k)}{n}=\dfrac{\sum_{i=1}^n 1_{X_i=k}}{n}\to E(1_{X_1=k})=P(X_1=k)=p(k)$ almost surely.

Therefore, $\dfrac{\log \prod_n}{n}=\dfrac{\sum_{k=1}^l N_n(k)\log(p_k)}{n}=\sum_{k=1}^l (\log p_k) \dfrac{N_n(k)}{n}\to\sum_{k=1}^l (\log p_k)p_k =\sum_{k=1}^l p_k\log p_k$ almost surely.

NB: See the importance of finite support. It enables you to interchange limit and sum.