6.3 Classification with RBF Networks

If you have not done so already, load the Neural Network package.

Load the Neural Networks package.

In[1]:=

Load data consisting of three classes divided into two clusters each. The data is represented by inputs x and their individual classifications by output y. The data format is described in Section 3.2, Package Conventions.

Load the data vectors and output indicating class.

In[2]:=

Look at the data.

In[3]:=

In classification problems it is important to have a nonlinearity at the output of the RBF network model. The purpose of the nonlinearity is to ensure that the output value stays within the range indicating the different classes. This is done by using the option OutputNonlinearity. Its default is None. Set it to Sigmoid so that its saturating values are 0 and 1, exactly as the output data indicating the classes.

Initialize an RBF network with eight basis functions.

In[4]:=

Out[4]=

The initialized network can now be trained by using NeuralFit.

Train the initialized RBF network.

In[5]:=

As usual, the reduction in RMSE over the seven iterations is displayed in a plot. Often, you will also get a warning from the program that the minimum was not reached. It is quite normal that the minimum is not reached in neural network training. Neural network models often contain so many parameters that it is extremely difficult to determine them precisely. Instead you should inspect the RMSE plot to determine whether more training is needed to converge to a reliable minimum. If more training is needed, you can use NeuralFit to continue the training from where you left off. Since the result also depends on the randomness of the initialization, it might be worthwhile to repeat the training with different initial models.

Find some information about the RBF network model.

In[6]:=

Out[6]=

The trained RBF network classifier may now be used to classify new input vectors.

Classify two data vectors.

In[7]:=

Out[7]=

The data vectors are classified to the class with the largest output value. If several outputs give large values, or if none of them do, then the classifier is highly unreliable for this data.

The result can be illustrated in several ways using NetPlot.

Plot the classification borders together with the data.

In[8]:=

This option can, of course, only be used in two-dimensional classification problems.

Look at the functions.

In[9]:=

This option can be used for problems with one or two input signals. The result is given as a graphics array. The classification boundaries are defined where the functions take the value 0.5.

By giving the option BarChart, you can obtain bar charts showing the classification performance. Correctly classified data is found on the diagonal and the misclassified data corresponds to the off-diagonal bars. Notice that a data vector may be assigned to several classes or to no class at all.

In[10]:=

The RBF classifier with a sigmoid on the output gives outputs in the open interval {0,1}. This can be interpreted as the "degree" of membership or probability of membership. However, it is often interesting to have a discrete answer of 0 or 1. This can be obtained in NetPlot, by specifying the option OutputNonlinearityUnitStep, which will replace the output sigmoid with a discrete unitstep. Compare the resulting plot of the classifier function with the preceding one.

In[11]:=

In contrast to FunctionPlot and Classifier, the BarChart option can be used for classifiers of any dimensions.

So far you have evaluated the end result of the training, the obtained RBF network. Consider the training record.

In[12]:=

Out[12]=

The first component is just a copy of the RBF network. The second component contains several items with information about the training. Section 7.8, The Training Record, shows you how to extract the information from the training record. Here you will see how this information can be illustrated in different ways using NetPlot and depending on which DataFormat option is chosen.

Look at how the parameters change during the training.

In[13]:=

The evolution of the classifier as a function of training iterations can be displayed using the option Classifier. As before, you can display snapshots at prescribed iteration intervals using the option Intervals.

Plot a classifier with a frequency three times the value of ReportFrequency.

In[14]:=