4.2.3 Higher-Dimensional Classification

Classification in higher-dimensional problems, that is, when the dimension of the input pattern x is higher than two, can be done in the same way as the two-dimensional problems. The main difference is that you can no longer illustrate the result with nice plots. Instead, you can view the data at various two-dimensional projections. It is also possible to look at how the data is distributed among the classes. This may be done using the commands in the Neural Networks package as illustrated next.

Load the Neural Networks package and the data.

In[1]:=<<NeuralNetworks`

In[2]:=

The input patterns are placed in the matrix x and the output in y.

Check the dimensions of the data.

In[3]:=

Out[3]=

Out[4]=

There are 40 data samples. The input matrix x has three columns, which means that the data is in a three-dimensional space. The output y also consists of three columns, which means that the perceptron should have three outputs, one for each class.

By analogy to the data in Section 3.4.1, Classification Problem Example this data could correspond to (scaled values of) the age, weight, and height of children from three different groups.

The main difference compared to two-dimensional problems is that you cannot look at the data in the same way. It is, however, possible to look at projections of the data. To do that you need a projection matrix of dimensions #inputs×2.

Look at a projection of the data.

In[5]:=NetClassificationPlot[x . {{1,0},{0,1},{0,0}},y,SymbolStyle{Hue[.5],Hue[.7],Hue[.9]}]

There are obviously 20 data samples of class two and ten of classes one and three.

You can now train a perceptron with this data set.

Train the perceptron.

In[6]:=

Success of the training depends on the initial weights of the perceptron. If you repeat the command, you will likely obtain slightly different results.

You can use the training record to investigate the training process.

Plot the number of correctly and incorrectly classified data vectors of each class.

In[7]:=

You can also plot the classification during training illustrated with a bar chart. The result is a graphics array.

Check the evolvement of the classifier during the training.

In[8]:=

If you prefer an animation of the training progress, you can load <<Graphics`Animation` and then change the command to Apply[ShowAnimation,NetPlot[fitrecord,x,y,Intervals1, DataFormatBarChart,DisplayFunctionIdentity]].

If you are interested only in the end result, you submit the perceptron instead of the training record.

Look at only the end result.

In[9]:=NetPlot[per,x,y]

If the classification is perfect, then all samples should be on the diagonal of the three-dimensional bar chart. The size of the off-diagonal bars corresponds to the number of misclassified samples.

If you cannot see all the bars properly, you can repeat the command and change the viewpoint. This is most easily done by using the 3D ViewPoint Selector.

Change the viewpoint.

In[10]:=NetPlot[per,x,y,ViewPoint{2.354, -4.532, 6.530}]

If the output y is not supplied, the distribution between the classes according to the perceptron model is given.

Illustrate the classification without any output.

In[11]:=

However, if you do not supply any output data, it cannot be indicated which data samples are correctly and incorrectly classified. Instead, you can see only the distribution over the classes according to the perceptron.