One can look at the operation of a matrix times a vector as changing the basis set for the vector or as changing the vector with the same basis description. Many signal and systems problem can be posed in this form.
A matrix times a vector
In this chapter we consider the first problem posed in the introduction
$$\mathbf{A}\mathbf{x}=\mathbf{b}$$
where the matrix
$\mathbf{A}$ and vector
$\mathbf{x}$ are given and we want to interpret and give structure to
the calculation of the vector
$\mathbf{b}$ . Equation
[link] has a variety of special
cases. The matrix
$\mathbf{A}$ may be square or may be rectangular. It may have
full column or row rank or it may not. It may be symmetric or orthogonalor non-singular or many other characteristics which would be interesting
properties as an operator. If we view the vectors as signals and thematrix as an operator or processor, there are two interesting interpretations.
- The operation
[link] is a change of basis or coordinates for a
fixed signal. The signal stays the same, the basis (or frame) changes.
- The operation
[link] alters the characteristics of the signal
(processes it) but within a fixed basis system. The basis stays the same, the signalchanges.
An example of the first would be the discrete Fourier transform (DFT)
where one calculates frequency components of a signal which arecoordinates in a frequency space for a given signal.
The definition of the DFT from
[link] can be written as a matrix-vector
operation by
$\mathbf{c}=\mathbf{Wx}$ which, for
$w={e}^{-j2\pi /N}$ and
$N=4$ , is
$$\left[\begin{array}{c}{c}_{0}\\ {c}_{1}\\ {c}_{2}\\ {c}_{3}\end{array}\right]=\left[\begin{array}{cccc}{w}^{0}& {w}^{0}& {w}^{0}& {w}^{0}\\ {w}^{0}& {w}^{1}& {w}^{2}& {w}^{3}\\ {w}^{0}& {w}^{2}& {w}^{4}& {w}^{6}\\ {w}^{0}& {w}^{3}& {w}^{6}& {w}^{9}\end{array}\right]\left[\begin{array}{c}{x}_{0}\\ {x}_{1}\\ {x}_{2}\\ {x}_{3}\end{array}\right]$$
An example of the
second might be convolution where you are processing or filtering asignal and staying in the same space or coordinate system.
$$\left[\begin{array}{c}{y}_{0}\\ {y}_{1}\\ {y}_{2}\\ \vdots \end{array}\right]=\left[\begin{array}{ccccc}{h}_{0}& 0& 0& \cdots & 0\\ {h}_{1}& {h}_{0}& 0& & \\ {h}_{2}& {h}_{1}& {h}_{0}& & \\ \vdots & & & & \vdots \end{array}\right]\left[\begin{array}{c}{x}_{0}\\ {x}_{1}\\ {x}_{2}\\ \vdots \end{array}\right].$$
A particularly powerful sequence of operations is to first change the basis
for a signal, then process the signal in this new basis, and finally returnto the original basis. For example, the discrete Fourier transform (DFT)
of a signal is taken followed by setting some of the Fourier coefficients tozero followed by taking the inverse DFT.
Another application of
[link] is made in linear regression where the
input signals are rows of
$\mathbf{A}$ and the unknown weights of the hypothesisare in
$\mathbf{x}$ and the outputs are the elements of
$\mathbf{b}$ .
Change of basis
Consider the two views:
- The operation given in
[link] can be viewed as
$\mathbf{x}$ being a set of weights so that
$\mathbf{b}$ is a
weighted sum of the columns of
$\mathbf{A}$ . In other words,
$\mathbf{b}$ will lie in the
space spanned by the columns of
$\mathbf{A}$ at a location determined by
$\mathbf{x}$ . This
view is a composition of a signal from a set of weights as in
[link] and
[link] below. If the vector
${\mathbf{a}}_{\mathbf{i}}$ is the
${i}^{th}$ column of
$\mathbf{A}$ , it is illustrated by
$$\mathbf{A}\mathbf{x}={x}_{1}\left[\begin{array}{c}\vdots \\ {\mathbf{a}}_{\mathbf{1}}\\ \vdots \end{array}\right]+{x}_{2}\left[\begin{array}{c}\vdots \\ {\mathbf{a}}_{\mathbf{2}}\\ \vdots \end{array}\right]+{x}_{3}\left[\begin{array}{c}\vdots \\ {\mathbf{a}}_{\mathbf{3}}\\ \vdots \end{array}\right]=\mathbf{b}.$$
- An alternative view has
$\mathbf{x}$ being a signal vector
and with
$\mathbf{b}$ being a vector whose entries are inner products of
$\mathbf{x}$ and the
rows of
A . In other words, the elements of
$\mathbf{b}$ are the projection
coefficients of
$\mathbf{x}$ onto the coordinates given by the rows of
A . The
multiplication of a signal by this operator decomposes the signal and gives thecoefficients of the decomposition. If
${\overline{\mathbf{a}}}_{\mathbf{j}}$ is the
${j}^{th}$ row of
$\mathbf{A}$ we have:
$${b}_{1}=\left[\begin{array}{c}\cdots {\overline{\mathbf{a}}}_{\mathbf{1}}\cdots \end{array}\right]\left[\begin{array}{c}\vdots \\ \mathbf{x}\\ \vdots \end{array}\right]\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}{b}_{2}=\left[\begin{array}{c}\cdots {\overline{\mathbf{a}}}_{\mathbf{2}}\cdots \end{array}\right]\left[\begin{array}{c}\vdots \\ \mathbf{x}\\ \vdots \end{array}\right]\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}\phantom{\rule{4pt}{0ex}}etc.$$
Regression can be posed from this view with the input signal being the rows of
A .