7.6 Separable Training

Separable training can be used when the neural network model is linear in some of the parameters. The separable algorithm gives a numerically better-conditioned minimization problem that is easier to solve. Therefore, by using a separable algorithm, the training is likely to converge to the solution in fewer training iterations. If the neural network has several outputs, the computational burden per iteration will also be less, which will speed up the training further.

You can indicate if the separable algorithm should be used with the option Separable. The default value is Automatic, which means that the separable algorithm will be used whenever possible for all training algorithms except the backpropagation algorithm. In the following cases, the separable algorithm cannot be used:

If there is a nonlinearity at the output; that is, if OutputNonlinearity is set to anything else but None.

If some of the parameters are fixed in the training.

The separable algorithm is illustrated with two examples. The first example is very simple. It has only two parameters, so that the result can be illustrated in a surface plot. The second example is of a more realistic size.