3.4.1 Classification Problem Example

This subsection describes a small classification problem. This two-dimensional example is instructive because the data and the classifier can be visualized very easily. The data used in this example is stored in the file twoclasses.dat in which the input is stored in the matrix x and the output in y. To help understand the problem, assume that the input data is the age and weight of several children, and the output data represents the class to which each child belongs. There are two possible classes in this example.

Load the Neural Networks package and the data.

In[1]:=<<NeuralNetworks`

In[2]:=

Because there are two possible classes, the output can be stored in one column, with a 1 or 0 indicating the class to which a child belongs. For a general discussion of data format, see Section 3.2.1, Data Format.

View the output data.

In[3]:=

Out[3]=

Plot the data, setting the axes' labels according to the names of the measured values.

In[4]:=NetClassificationPlot[x,y,FrameLabel{"Age","Weight"},SymbolStyle{Hue[0.6],Hue[0.8]},RotateLabelFalse]

The plot clearly shows that the data is divided into two classes and that it should be possible to divide the groups with a curve. This curve is found during the training process, and the successfully trained neural network classifier will return the correct group to which a child belongs, given the child's weight and age.

A measure of fit, or performance index, to be minimized by the training algorithm, must be chosen for the training to proceed. For classification problems, the criterion is set to the number of incorrectly classified data samples. The classifier has correctly classified all data when the criterion is zero.

A perceptron is the simplest type of neural network classifier. It is used to illustrate the training in this example. You can initialize a perceptron with InitializePerceptron and then use PerceptronFit to train the perceptron. However, perceptron initialization will take place automatically if you start with PerceptronFit.

Initialize and train a perceptron classifier using the data.

In[5]:=

Note that you will usually obtain slightly different results if you repeat the training command. This is due to the random initialization of the perceptron, which is described in Section 4.1.1. As a result of this, the parametric weights of the perceptron will also be different for each evaluation and you will obtain different classifiers.

During the evaluation of PerceptronFit, a separate notebook opens and displays the progress of the training. At the end of the training, a summary of the training process is shown in the plot of summed squared error (SSE) versus iteration number. The preceding plot is the summary of the training process for this example. You can see that the SSE tends toward 0 as the training goes through more iterations.

The first output argument of PerceptronFit is the trained perceptron per in this case. The second output argument, equal to fitrecord in this example, is a training record that contains information about the training procedure. See Section 7.8 for a description of how the training record can be used to analyze the quality of training. See Chapter 4, The Perceptron, for options that change the training parameters and plot.

The perceptron's training was successful, and it can now be used to classify new input data.

Classify a child of age six and weight seven.

In[6]:=

Out[6]=

You can also evaluate the perceptron on symbolic inputs to obtain a Mathematica expression describing the perceptron function. Then you can combine the perceptron function with any Mathematica commands to illustrate the classification and the classifier.

Obtain a Mathematica expression describing the perceptron.

In[7]:=

Out[8]=

Note that the numerical values of the parameter weights will be different when you repeat the example.

NetPlot can be used to illustrate the trained network in various ways, depending on the options given. The trained classifier can, for example, be visualized together with the data. This type of plot is illustrated using the results from the two-dimensional classifier problem. For this example, a successful classifier divides the two classes with a line. The exact position of this line depends on what particular solution was found in the training. All lines that separate the two clusters are possible results in the training.

In[9]:=

NetPlot can also be used to illustrate the training process by applying it to the training record, the second argument of PerceptronFit.

Illustrate the training of the perceptron.

In[10]:=

The plot shows the classification of the initial perceptron and its improvement during the training.

The perceptron is described further in Chapter 4 and you can find a list of neural networks that can be used for classification problems in Section 2.1, Introduction to Neural Networks.