next up previous
Next: Linear observations Up: The derivation of the Previous: The derivation of the

The Derivation

We want to derive an optimal estimate, $\hat{x}$, of a field, x, as a linear combination of observation data, $\theta$,


\begin{displaymath}\hat{x} = A \theta
\end{displaymath} (1)

Our goal is to derive the form of the matrix, A, so that the expected mean square difference between the estimated field and the actual field ( x ) is minimized,


\begin{displaymath}E[\varepsilon \varepsilon^T]= E[(\hat{x} - x)(\hat{x} - x)^T] = \mbox{minimum}
\end{displaymath} (2)

If we put (1) into (2) and expand, we get


\begin{displaymath}E[\varepsilon \varepsilon^T]= E[ A \theta \theta^T A^T - x \theta^T A^T - A \theta x^T + x x^T]
\end{displaymath} (3)

If we let Cx be the autocorrelation of the field ( E[x xT] ), $C_\theta$ be the autocorrelation of the observations ( $E[\theta \theta^T]$ ), and $C_{x\theta}$ be the cross correlation between the field and the observations ( $E[x \theta^T]$ ), then we can write the above as


\begin{displaymath}C_\varepsilon = A C_\theta A^T - C_{x \theta} A^T - A C_{x \theta}^T + C_x
\end{displaymath} (4)

The next step requires the application of the following matrix identity (proved in the appendix),


(A - B C-1) C (A - B C-1)T - B C-1 BT = A C AT - B AT - (B AT)T (5)

using A in (4) for A in (5), and $C_{x\theta}$ for B as well as $C_\theta$ for C, we can reduce (x) to


\begin{displaymath}C_\varepsilon = (A - C_{x \theta} C_\theta^{-1}) C_\theta
(A...
...\theta^{-1})^T
- C_{x\theta}C_\theta^{-1}C_{x \theta}^T + C_x
\end{displaymath} (6)

(note we have also used the fact that $C_\theta = C_\theta^T$).

The matrices $C_\theta$, is an autocorrelation matrix therefore both it and $C_\theta^{-1}$ are nonnegative definite (see appendix), therefore

\begin{displaymath}(A - C_{x \theta} C_\theta^{-1}) C_\theta
(A - C_{x \theta} C_\theta^{-1})^T
\end{displaymath} (7)

and

\begin{displaymath}C_{x\theta}C_\theta^{-1}C_{x \theta}^T
\end{displaymath} (8)

are both matrices with positive diagonal elements. This means that the diagonal elements of $C_\varepsilon$ are therefore minimized when it is true that,


\begin{displaymath}A - C_{x\theta} C_\theta^{-1} = 0
\end{displaymath} (9)

Therefore we have,

\begin{displaymath}A = C_{x \theta} C_{\theta}^{-1}
\end{displaymath} (10)

This is the estimator that we are seeking.


\begin{displaymath}\hat{x} = C_{x \theta} C_{\theta}^{-1} \theta
\end{displaymath} (11)

Further, we can write down what the expected error for the estimator as,

\begin{displaymath}C_\varepsilon = C_x - C_{x\theta} C_\theta^{-1} C_{x \theta}^T
\end{displaymath} (12)

Equations (11) and (12) constitute the Gauss-Markov estimator for the linear minimum means square estimate of a random variable.


next up previous
Next: Linear observations Up: The derivation of the Previous: The derivation of the
Skip Carter
1999-12-08