Documentation MathematicaBuilt-in Functions Advanced DocumentationLinear AlgebraExamples

Full Rank Least Squares Solutions

Solving a matrix equation is one of the key problems of matrix computation; techniques for solving it were discussed in the section Solving Linear Systems. If is an matrix, this is a set of linear equations in unknowns. If , there are more equations than unknowns and the system is overdetermined. A general technique for finding least squares solutions was given in the section Least Squares Solutions. This example will show some different techniques for finding approximate solutions to overdetermined systems for full rank matrices.

Note that all the example solutions in this section will work just as well if they were given sparse matrices as input.

Least Squares Cholesky

This technique uses a Cholesky decomposition to find a least squares solution. If a matrix has full rank, the solution can be found in the following way.

These steps can be placed into a Mathematica function as follows.

In[1]:=

The function can be applied as follows.

In[2]:=

Out[4]=

The solution is close to satisfying the equation.

In[5]:=

Out[5]=

This compares the solution to that from PseudoInverse.

In[6]:=

Out[6]=

This technique is fast, but is less accurate because it computes .

This computes 1-, 2-, and -norms for the solution that was found.

In[7]:=

Out[7]=

Least Squares QR

When is full rank, a more accurate way to solve the problem is to use the QR decomposition as follows.

A Mathematica program to find the solution is as follows.

In[1]:=

The solution for this problem is very similar to the Cholesky solution.

In[2]:=

Out[4]=

The QR decomposition solution can be more expensive than the Cholesky approach, but it is more accurate. An example that demonstrates the difference in accuracy is shown below.

In[5]:=

This shows how close the Cholesky solution is using the 2-norm.

In[9]:=

Out[9]=

This shows how close the QR solution is using the 2-norm. It can be seen that the QR solution is a better approximation.

In[10]:=

Out[10]=