Derivation of the Riccati Differential Equation

I am attempting to derive the Riccati Equation for linear-quadratic control. The original equation is:

$-\partial V/\partial t = \min_{u(t)} \{x^TQx + u^TRu + \partial V^T/\partial x(Ax + Bu) \}$

$x \in \Re^n$, $u \in \Re^m$, $Q \in \Re^{n\times n}$, $R \in \Re^{m\times m}$, $A \in \Re^{n\times n}$, $B \in \Re^{n\times m}$.

It can be shown that the minimal $u$ is $u^*=-\frac{1}{2}R^{-1}B^T\partial V/\partial x$; also, $V(x,t)$ can be shown to be quadratic in $x$, so it is of the form $V(x(t),t) = x(t)^{T}P(t)x(t)$, so $\partial V/\partial x = 2P(t)x(t)$. Thus $u^*(t) = -R^{-1}B^TP(t)x(t)$. We’d like to solve for $P$, which is symmetrical.

Plugging into the original equation, I obtain

$-\partial V/\partial t = -x^T\dot{P}x \equiv x^TQx + (-R^{-1}B^TPx)^TR(-R^{-1}B^TPx)+2x^TP(Ax+B[-R^{-1}B^TPx])$

Somehow this gets reduced to

$-x^T\dot{P}x = x^T\{A^TP + PA + Q – PBR^{-1}B^TP\}x$

I cannot figure out the manipulation to get to the final equation. In particular, how is there both an $A^TP$ and $PA$ term in the final expression, when I distribute $2x^TP$ into $Ax+Bu^*$? Any insight would be helpful. Thanks.

Solutions Collecting From Web of "Derivation of the Riccati Differential Equation"

$P$ is real symmetric and so is diagonalizable: $P = QLQ^T$ with some orthogonal $Q$.

Then,
$$\begin{align*}
A^TP+PA &= A^TQLQ^T + QLQ^TA \\
Q^TA^TP+Q^TPA &= Q^TA^TQLQ^T+LQ^TA \\
Q^TA^TPQ+Q^TPAQ &= Q^TA^TQL+LQ^TAQ \\
Q^T(A^TP+PA)Q &=Q^TA^TQL+(Q^TAQ)^TL \\
&= Q^TA^TQL+Q^TA^TQL \\
&= 2(Q^TA^TQ)L
\end{align*}
$$

Moving back the $Q$’s from the left-hand side,

$$\begin{align*}
A^TP+PA &= 2Q(Q^TA^TQ)LQ^T \\
&= 2A^T(QLQ^T)\\
&= 2A^TP \\
&= 2PA
\end{align*}
$$

Associativity of matrix operations and the symmetry of $P$, namely $P^T=P$, are exploited a few times here.

I know it’s a little late but it might help someone in the future.

If you notice that $2x^TPAx$ is a scalar, then it is equivalent to it’s transpose, so:

\begin{align}
2x^TPAx &= x^TPAx + x^TPAx \\
&= x^TPAx + x^TA^TP^Tx,\ \text{but}\ P = P^T \\
&= x^TPAx + x^TA^TPx \\
&= x^T(PA + A^TP)x
\end{align}

which is what you were looking for.

I hope this helps.