Articles of matrix calculus

Tangent Space of SL(n,R) at arbitrary point, e.g. not at $\mathbb{1}$

I am looking for the tangent space of $SL(n,\mathbb{R}) = \{A\in\mathbb{R}^{n\times n} | \det{A}=1 \}$ (actually $n=3$) at an arbitrary $M\in SL(n,\mathbb{R})$. From now on I will omit the $\mathbb{R}$ for convenience. It is well known and easy to prove that the tangent space at the identity matrix $\mathbb{1}$, $T_1SL(n)$, is the set of all […]

Jacobian of $A (A^\top X A)^{-1} A^\top$

Let $A\in\mathbb{R}^{n\times m}$, $n\geq m$, be a full column rank matrix, and consider the function \begin{align} f&\colon \mathbb{R}^{n\times n} \to \mathbb{R}^{n\times n}\\ & X\mapsto A (A^\top X A)^{-1} A^\top, \end{align} where $\bullet^\top$ denotes transposition. Assuming that $(A^\top X A)^{-1}$ exists, I’m interested in the computation of the Jacobian matrix of $f$, i.e. $$\tag{1}\label{a} \mathbf{J}[f] = […]

Is there anything like upper tridiagonal matrix? How to find the determinant of such a matrix?

I want to find the determinant of the following matrix. $$\left[\begin{matrix} -\alpha_1 & \beta_2 & -\gamma_3 & 0 & 0 & 0 & \cdots & 0&0 \\ 0 & -\alpha_2 & \beta_3 & -\gamma_4 & 0 & 0 & \cdots & 0 & 0 \\0 & 0 & -\alpha_3 & \beta_4 & -\gamma_5&0&\cdots&0 & 0 […]

Gradient and Hessian of this function

What is the gradient $\nabla_x$ / hessian $\nabla_x^2$ of $\frac{(Ax)^{\top}y}{||Ax||}$ where $A \in \mathbb{R}^{m \, \times \, n}$, $x \in \mathbb{R}^n$ and $y \in \mathbb{R}^m$? I don’t really know how to approach this problem because I don’t know how to deal with quotients in matrix calculus.

Solution to system of linear ODE's

Let $\Delta_n$ be the closed unit simplex in $\mathbb R^n$. For any $a,b \in \Delta_n$, define the differential equation: $$ a'(u) = b-a(u) \quad\quad\quad a(0) = a $$ How does one go about solving this system of ODEs? Solving the case where $n=1$ is simple, does this carry over to the $n$ dimensional case? I’ve […]

How to calculate the derivative of $F=\frac{e^{W}-1}{W}$ where $W = _\times$ in $\omega$?

How to calculate the derivative of $F=\frac{e^{W}-1}{W}$ in $\omega$ where $W = [\omega]_\times$ and $\omega\in\mathbb{R}^3$? I am aware of a solution (solution 1) and briefly show below, but I am not happy with it for its complexity and am looking for a possibly better solution (solution 2). Inspired by hans’ brilliant answer to a related […]

Lower bound of the smallest eigenvalue of block hermitian matrix

Let $J$ and $L$ two squared matrix and let $A$ the matrix as follows A=\begin{pmatrix} J^TJ+L^TL&J^T\\ J &I\\ \end{pmatrix} where I denote the identity matrix I ask if there is a lower bounds of the smallest eignevalue of A in function of the eigenvalue of $J$ and $L$.

Find $\exp(D)$ where $D = \begin{bmatrix}2& -1 \\ 1 & 2\end{bmatrix}. $

$$C = \begin{bmatrix}2& -1 \\ 0 & 2\end{bmatrix}\quad $$ I break it down into two matrices $$A = \begin{bmatrix}2& 0 \\ 0 & 2\end{bmatrix}\quad \text{and}\quad B =\begin{bmatrix}0 & -1 \\ 0 & 0\end{bmatrix}.$$ For matrix $A$, $$\operatorname{exp}(A) = \begin{bmatrix}e^2& 0 \\ 0 & e^2\end{bmatrix}\quad.$$ For matrix $B$, we have that $B^k=0$, for all $k\ge 2$$$ […]

Derivative of the determinant of the right stretch tensor

I have to evaluate the derivative $$ \frac{\partial\det\mathcal{U}}{\partial F} $$ where $\mathcal{U}=\sqrt{F^TF}$ and $F$ is a $m\times n$ real matrix. Any suggestion would be appreciated. Thank you all, guys!! You helped me a lot.

How to compute the gradient of the softmax function w.r.t. matrix?

Let us consider the following functions \begin{equation} y = \operatorname{softmax}(z) \end{equation} \begin{equation} z = h\cdot W + b \end{equation} where $y, h, W$ and $b$ are $1 \times n$, $1 \times m$, $m \times n$ and $1 \times n$ matrices. Compute $\frac{\partial{y_i}}{\partial{W}}$. My efforts: \begin{equation} \frac{\partial{y_i}}{\partial{W}} = \frac{\partial{y_i}}{\partial{z}} \times \frac{\partial{z}}{\partial{W}} \end{equation} Here $z$ is a […]