10.4 Change Step Length and Neighbor Influence

There is a default function guiding the step length in the training of unsupervised networks, and another function for the neighbor strength. These defaults were described in connection with the function UnsupervisedNetFit, and they have been chosen to suffice for a large variety of different problems. Both have been chosen so that the step length and the neighbor strength becomes smaller when the number of training iterations increase. Sometimes, however, depending on the data vectors and the chosen size of the unsupervised network, it might be advantageous to apply other functions than the defaults. The easiest way to modify the step length and neighbor strength is to submit numerical values in the two options StepLength and NeighborStrength. That gives constant values to these parameters, and that might suffice in many cases. In the following description are more advanced changes involving functions for these options.

Read in the Neural Networks packages and load a data set.

In[1]:=

In[2]:=

The step length can be chosen to be any function that takes an integer indicating the number of the training iteration and delivers a step size in the interval {0,1}.

Define a new step length function.

In[3]:=

Out[3]=

The neighbor strength function and the neighbor matrix together dictate how much the neighbors of the winning codebook vector are changed. With a strong neighbor strength, the codebook vectors will be kept close to one another, while loose coupling will let them adapt more independently of each other. The neighbor strength function enters the algorithm as a negative exponent, so that a larger value indicates a weaker neighbor coupling (see Section 10.1.2, UnsupervisedNetFit). The function takes the training iteration number as an input argument and returns a positive real value. Typically, the neighbor strength should have a low value at the beginning of the training so that the codebook vectors obtain the correct orientation with respect to one another. Then, as the training progresses, the coupling strength between the codebook vectors should become weaker, which can be achieved if the neighbor strength function returns a larger value.

Here, a neighbor strength function is defined to return twice the iteration number. This yields a much faster decrease in the coupling between the neighbors than the default.

Define a new function guiding the neighbor strength.

In[4]:=

Out[4]=

With the neighbor option you can specify the codebook vectors, or neurons, that are close to each other. Using the default option NonSymmetric, or the other prespecified alternative Symmetric, makes UnsupervisedNetFit create a neighbor matrix automatically with the correct dimensions. A user-specified neighbor matrix should have 2c-1 components in each dimension, where c is the number of codebook vectors in the corresponding dimension. The center element, which corresponds to the winning codebook vectors, should be zero. Then the winning codebook vector will be updated with the step length given by the step length function. The other codebook vectors will also be updated, and their change is influenced by the neighbor strength function; if the neighbor strength function gives larger values, then the neighbors are changed less, as described in Section 10.1.2, UnsupervisedNetFit.

Specify a neighbor matrix for a SOM network with three codebook vectors in one direction and four in the other. This means that the network has 12 codebook vectors. The elements are chosen to become larger in all directions from the winning codebook vectors, which means that distant codebook vectors are influenced less than those that are close to the winner.

Define a neighbor matrix for a SOM network with 3 × 4 codebook vectors.

In[5]:=

The new choices of step length, neighbor strength, and neighbor matrix can now be submitted as options to UnsupervisedNetFit.

Train the SOM network with the modified algorithm.

In[6]:=

The trained SOM network can be evaluated in the same way as shown earlier in the other examples.