|
8.2.1 Introductory Dynamic Example
In this first example you will see how the different commands of the Neural Networks package can be used to identify and evaluate dynamic models. You do not obtain exactly the same result if you repeat the example. This is due to the randomly chosen input signal and the random initialization of the neural network weights.
Read in the Neural Networks package and two standard add-on packages.
In[1]:=
In[2]:=<<Statistics`ContinuousDistributions`
<<Graphics`MultipleListPlot`
Generate a data set using a function defining the true system.
In[4]:=Ndata=600;
u=RandomArray[NormalDistribution[0,3], {Ndata,2}];
x=FoldList[Function[{xs,uin},{(uin[[1]]+uin[[2]]+0.6*xs[[1]]+0.8*xs[[2]])/(1+xs[[3]]^2),0.7*xs[[2]]+uin[[2]],xs[[1]]}],{0,0,5}, Drop[u,-1]];
y=x[[All,{1,2}]];
The input data is placed in u and the output data in y.
In a real situation the data is measured and an approximation of the unknown true function generating the data is estimated using a dynamic neural network. This situation is now imitated and a neural network approximating the data generating function is estimated.
Check the dimensions of the data.
In[8]:=
Out[8]=
Out[9]=
There are 600 data samples available, and the plant has two inputs and two outputs. It is a good idea to look at the data before you start to estimate a model. From a plot you can see if the data look strange in some way which makes the training of a network hard.
Look at the first input signal.
In[10]:=
Look at the second input signal.
In[11]:=
Look at the first output signal.
In[12]:=
Look at the second output signal.
In[13]:=
The first half of the data set is used for identification and the second half for validation of the model.
Divide the data into identification and validation data.
In[14]:=
In this example the true function is known and the regressor should be chosen to x(t)={ (t-1), (t-2), (t-1)  (t-1), (t-1)}, which is obtained by choosing  ={2,1},  ={1,1}, and  ={1,1} as described in Section 2.6, Dynamic Neural Networks. In real situations, when the generating function is unknown, you usually have to find the best regressor by trail-and-error.
It is always good to start with a linear model. This is obtained by using a FF network without hidden neurons, as described in Section 5.1.1, InitializeFeedForwardNet. The performance of the linear model gives a quality measure which you want your nonlinear neural network model to beat.
Estimate a linear model.
In[18]:=
Find some information about the model.
In[19]:=
Out[19]=
The command NetComparePlot is very convenient for evaluating dynamic models. Depending on which value you choose of the option PredictionHorizon the model can be a simulator or a predictor. For each output the model output is displayed together with the true output signal and the root-mean-square error is given in the plot title. This type of test is only fair if you use fresh data, that is, the validation data for the comparison.
Obtain the one-step-ahead prediction with the linear model and compare it with the true output signal.
In[20]:=
By including the whole data set in the call, and then indicating the validation data with the option ShowRange, you avoid transients in the beginning of the plotted prediction.
From the plot you see that the second output is described almost perfectly by the linear model but there are inconsistencies in the first output. This is not surprising if you take a closer look at the true function generating the data, the second output can be described by a linear model but the first output cannot. To model the first output better you need to make the neural network nonlinear by including some neurons. This will be done later in this section.
You can also use the command NetSimulate or NetPredict to perform simulations and predictions. They give you the simulated and predicted outputs in the same format as the original output signal. The obtained simulation and prediction can then, for example, be plotted with the true output.
Simulate the linear model and plot the first 100 values of the validation data together with the true output.
In[21]:=
In[22]:=
Estimate a nonlinear model based on a FF network with 4 neurons and with the same regressor as the linear model.
In[23]:=
Note that if you repeat the example the result will turn out differently due to randomness in data and in the network initialization. You might need more training iterations, or you might get caught in a local minimum.
Compare the one-step-ahead prediction with the true output.
In[24]:=
Compare this with the prediction of linear model. The first output is much better predicted but the second is slightly worse.
The difference between the two models can be illustrated by comparing the prediction errors in common plots.
Compute and plot the prediction errors using the linear and nonlinear models.
In[25]:=
In[27]:=
As you see from the plots the errors of the linear model dominate the first output but for the second output the nonlinear model gives the largest error. The scales of the prediction errors of the two ouputs are however very different.
An analytic expression of dynamic neural network models is obtained by evaluating the neural network placed at the first position of the neural ARX model on a vector with length equal the number of regressors.
Express the linear ARX model analytically.
In[29]:=
Out[29]=
The expression of the nonlinear model is much more complicated.
Express the nonlinear neural ARX model analytically.
In[30]:=
Out[30]=
The symbolic expressions may be useful if you want to use general Mathematica commands to manipulate the neural network expression.
|
