This section gives a review of some basic concepts and operations that will be used throughout the chapter to discuss matrix operations.

The norm of a mathematical object is a measurement of the length, size, or extent of the object. In Mathematica norms are available for scalars, vectors, and matrices.

For numbers, the norm is the absolute value.

In[1]:=

Out[1]=

In[2]:=

Out[2]=

For vector spaces, norms allow a measure of distance. This allows the definition of many familiar concepts such as neighborhood, closeness of approximation, and goodness of fit. A vector norm is a function that satisfies the following relations.

Typically this function uses the notation . The subscript is used to distinguish different norms, of which the p-norms are particularly important. For , the p-norm is defined as follows.

Some common p-norms are the 1-, 2-, and -norms.

In Mathematica, vector p-norms can be computed with the function Norm. The 1-, 2-, and -norms are demonstrated in the following examples.

In[1]:=

In[2]:=

Out[2]=

In[3]:=

Out[3]=

In[4]:=

Out[4]=

The 2-norm is particularly useful and this is the default.

In[5]:=

Out[5]=

Norms are also implemented for vectors with symbolic entries.

In[6]:=

Out[6]=

Matrix norms are used to give a measure of distance in matrix spaces. This is necessary if you want to quantify what it means for one matrix to be near to another, for example, to say that a matrix is nearly singular.

Matrix norms also use the double bar notation of vector norms. One of the most common matrix norms is the Frobenius norm (also called the Euclidean norm).

Other common norms are the p-norms. These are defined in terms of vector p-norms as follows.

Thus the matrix p-norms show the maximum expansion that a matrix can apply to any vector.

In Mathematica the Frobenius norm can be computed as follows.

In[1]:=

In[2]:=

Out[2]=

The matrix 2-norm can be computed as follows.

In[3]:=

Out[3]=

It is also possible to give an argument to specify the 1, 2, or matrix p-norms.

In[4]:=

Out[4]=

In[5]:=

Out[5]=

In[6]:=

Out[6]=

This computes the expansion that a collection of different vectors gets from an input matrix. You can see that the maximum value is still less than the 2-norm.

In[7]:=

Out[7]=

One of the fundamental subspaces associated with each matrix is the nullspace. Vectors in the nullspace of a matrix are mapped to zero by the action of the matrix.

For some matrices (for instance, nonsingular square matrices), the nullspace is empty.

In[1]:=

Out[2]=

This matrix has a nullspace with one vector.

In[3]:=

Out[4]=

In[5]:=

Out[5]=

This matrix has a nullspace with two basis vectors.

In[6]:=

Out[7]=

The rank of a matrix corresponds to the number of linearly independent rows or columns in the matrix.

If an matrix has no linear dependencies between its rows, its rank is equal to and it is said to be full rank.

In[6]:=

Out[7]=

If a matrix has any dependencies in its rows then its rank is less than . In this case the matrix is said to be rank deficient.

In[8]:=

Out[9]=

Note that the rank of a matrix is equal to the rank of its transpose. This means that the number of linearly independent rows is equal to the number of linearly independent columns.

In[10]:=

Out[10]=

For an matrix the following relations hold: Length[NullSpace[]]+MatrixRank[] n, and Length[NullSpace[]]+MatrixRank[] m. From this it follows that the nullspace is empty if and only if the rank is equal to and that the nullspace of the transpose of is empty if and only if the rank of is equal to .

If the rank is equal to the number of columns, it is said to have full column rank. If the rank is equal to the number of rows, it is said to have full row rank. One way to understand the rank of a matrix is to consider the row echelon form.

A matrix can be reduced to a row echelon form by a combination of row operations that start with a pivot position at the top-left element and subtract multiples of the pivot row from following rows so that all entries in the column below the pivot are zero. The next pivot is chosen by going to the next row and column. If this pivot is zero and any nonzero entries are in the column beneath, the rows are exchanged and the process is repeated. The process finishes when the last row or column is reached.

A matrix is in row echelon form if any row that consists entirely of zeros is followed only by other zero rows and if the first nonzero entry in row is in the column , then elements in columns from 1 to in all rows below are zero. The row echelon form is not unique but its form (in the sense of the positions of the pivots) is unique. It also gives a way to determine the rank of a matrix as the number of nonzero rows.

A matrix is in reduced row echelon form if it is in row echelon form, each pivot is one, and all the entries above (and below) each pivot are zero. It can be formed by a procedure similar to the procedure for the row echelon form, also taking steps to reduce the pivot to one (by division) and reduce entries in the column above each pivot to zero (by subtracting multiples of the current pivot row). The reduced row echelon form of a matrix is unique.

The reduced row echelon form (and row echelon form) give a way to determine the rank of a matrix as the number of nonzero rows. In Mathematica the reduced row echelon form of a matrix can be computed by the function RowReduce.

The reduced row echelon form of this matrix only has one nonzero row. This means that the rank is 1.

In[11]:=

Out[12]//MatrixForm=

This is a 32 random matrix whose columns are linearly independent.

In[13]:=

Out[14]//MatrixForm=

The reduced row echelon form has two nonzero rows, thus its rank should be 2.

In[15]:=

Out[15]//MatrixForm=

MatrixRank also computes the rank as 2.

In[16]:=

Out[16]=

The reduced row echelon form of the transpose of the matrix also has two nonzero rows. This is consistent with the fact that the rank of a matrix is equal to the rank of its transpose, even when the matrix is rectangular.

In[17]:=

Out[17]//MatrixForm=

The inverse of a square matrix is defined as follows.

Where is the identity matrix. The inverse can be computed in Mathematica with the function Inverse.

Here is a sample 22 matrix.

In[1]:=

Out[3]//MatrixForm=

This demonstrates that the matrix satisfies the definition.

In[4]:=

Out[4]//MatrixForm=

Not all matrices have an inverse; if a matrix does not have an inverse it is said to be singular. If a matrix is rank-deficient then it is singular.

In[5]:=

Out[6]=

In[7]:=

Out[7]=

The matrix inverse can in principle be used to solve the matrix equation

by multiplying it by the inverse of .

When the matrix is singular or is not square it is still possible to find an approximate inverse that minimizes . When the 2-norm is used this will find a least squares solution known as the pseudoinverse.

This finds the pseudoinverse for the singular matrix defined above.

In[8]:=

Out[10]//MatrixForm=

The result is not the identity matrix, but it is 'close' to the identity matrix.

In[11]:=

Out[11]//MatrixForm=

This computes the pseudoinverse of a rectangular matrix.

In[12]:=

Out[14]//MatrixForm=

The result is quite close to the identity matrix.

In[15]:=

Out[15]//MatrixForm=

The determinant of an matrix is defined as follows.

It can be computed in Mathematica with the function Det.

Here is a sample 22 matrix.

In[1]:=

Out[2]=

One useful property of the determinant is that if and only if is singular.

In[3]:=

Out[4]=

In[1]:=

Out[2]=

The size of the submatrices can be controlled by a second option. Here the 11 minors are computed; these are just the entries of the matrix.

In[3]:=

Out[3]=

Note that Minors[m] is equivalent to Minors[m, n-1]. In general, Minors[m, k] computes determinants of all the possible submatrices that can be generated from the matrix (this is done by picking different rows and columns). For the 2×2 minors of a 4×4 matrix we obtain a 6×6 matrix because there are only 6 different ways to pick 2 rows from 4 rows, and to pick 2 columns from 4 columns.

In[4]:=

Out[4]=

One of the properties of the rank of a matrix is that it is equal to the size of the largest nonsingular submatrix. This can be demonstrated with Minors. In this example, the following matrix has a rank of 2.

In[5]:=

Out[6]=

The determinant of the matrix is zero,

In[7]:=

Out[7]=

When the 22 minors are computed you can see that not all are zero, confirming that the rank of the matrix is 2.

In[8]:=

Out[8]=