3.4.2 Function Approximation Example

This subsection contains a one-dimensional approximation problem solved with a feedforward network. Higher-dimensional problems, except for the data plots, can be handled in a similar manner.

Load the Neural Networks package.

In[1]:=<<NeuralNetworks`

Load data and Mathematica's matrix add-on package.

In[2]:=<<onedimfunc.dat;
<< LinearAlgebra`MatrixManipulation`

The file onedimfunc.dat contains the input and output data in the matrices defined as x and y, respectively. It is assumed that the input-output pairs are related by y=f(x), where f is an unspecified function. The data will be used to train a feedforward network that will be an approximation to the actual function f.

To motivate the use of a neural network, imagine that both x and y are measured values of some product in a factory but that y can be measured only by destroying the product. Several samples of the product were destroyed to obtain the data set. If a neural network model can find a relationship between x and y based on this data, then future values of y could be computed from a measurement of x without destroying the product.

Plot the data.

In[4]:=

This is a very trivial example; the data was generated with a sinusoid. A feedforward network will be trained to find such an approximation. More information on this kind of neural network can be found in Chapter 5, The Feedforward Neural Network.

Initialize a feedforward network with three neurons.

In[5]:=

Out[5]=

Train the initialized network.

In[6]:=

Note that you will usually obtain slightly different results if you repeat the initialization and the training command. This is due to the partly random initialization of the feedforward network, which is described in Section 5.1.1, InitializeFeedForwardNet.

As with the previous example, the improvement of the fit is displayed in a separate notebook during the training process. At the end of the training, the fit improvement is summarized in a plot of the RMSE in the neural network prediction versus iterations. NeuralFit options allow you to change the training and plot features. At the end of the training, you will often receive a warning that the training did not converge. It is usually best to visually inspect the RMSE decrease in the plot to decide if more training iterations are needed. How this can be done is illustrated in Section 5.2.1, Function Approximation in One Dimension. Here we will assume that the network has been successfully trained, though later we will plot the model to compare it to the data.

The first output argument of NeuralFit is the trained feedforward network. The second argument is a training record containing information about the training procedure. See Section 7.8, The Training Record, for a discussion of how to use the training record to analyze the quality of training.

The trained neural network can now be applied to a value x to estimate y = f(x) .

Produce an estimate for y when x=3.

In[7]:=

Out[7]=

To obtain a Mathematica expression describing the network, apply the network to symbolic input. Then you can use Mathematica commands to plot and manipulate the network function.

Obtain a Mathematica expression describing the network.

In[8]:=

Out[9]=

The special command NetPlot illustrates the trained neural network in a way indicated with the option DataFormat. For one- and two-dimensional problems you can use it to plot the neural network function.

Plot the function estimate together with the data.

In[10]:=

Depending on the option DataFormat, NetPlot uses different Mathematica plot commands, and you may submit any options of these commands to change the way the result is displayed. For example, in the illustrated plot the color was changed.