9.2.2 Discrete-Time Classification of Letters

In this example some letters will be automatically classified.

Load the Neural Networks package.

In[1]:=<<NeuralNetworks`

Generate patterns of the letters A, I, and Y. They are stored as matrices with 1 indicating black and -1 indicating white.

Generate the letters A, Y, and I in a list x.

In[2]:=x=-{{{1,-1,1},{1,1,1},{1,-1,1},{1,1,1}},
{{-1,1,-1},{-1,1,-1},{-1,1,-1},{-1,1,-1}},
{{-1,1,-1},{-1,1,-1},{1,-1,1},{1,-1,1}}};

Look at the letters.

In[3]:=xg=Map[ListDensityPlot[#,DisplayFunctionIdentity,FrameTicksNone]&,x];
Show[GraphicsArray[xg]]

Before you can construct a Hopfield network the matrices have to be transformed into vectors.

Transform the matrices into pattern vectors.

In[5]:=xv = Map[Flatten,x,{1}];

The vectors containing the input pattern representing the three letters can now be used to create a Hopfield network.

Create a Hopfield network.

In[6]:=hopletter = HopfieldFit[xv]

Out[6]=

The obtained Hopfield network can be tested in different ways. Start by determining if it can correctly classify noisy versions of the three letters. Noisy versions are generated where each pixel has a certain probability to change value. Since this as a random process you will receive different disturbed data vectors each time you repeat the following commands.

Create three disturbed data vectors from the three pattern vectors.

In[7]:=<<Statistics`ContinuousDistributions`
y=Sign[xv*RandomArray[UniformDistribution[-0.1,1], {3,12}]];

You can look at the disturbed data vectors, but first they must be converted to matrices.

Look at the disturbed letters.

In[9]:=ym=Map[Partition[#,3] &, y];
yg=Map[ListDensityPlot[#,DisplayFunctionIdentity,FrameTicksNone] &,ym];
Show[GraphicsArray[yg]]

It is now time to evaluate the Hopfield network against the disturbed data vectors. This is done by mapping the Hopfield object containing the Hopfield network on each of the data vectors describing the noisy letters. The result is converted back into matrices and plotted.

Evaluate the Hopfield network on the noisy letters and plot the result.

In[12]:=yh=Map[hopletter[#] &, y];
yh=Apply[Partition[#,3] &, yh, 1];
yhg=Map[ListDensityPlot[#,DisplayFunctionIdentity,FrameTicksNone] &, yh];
Show[GraphicsArray[yhg]]

Is the result satisfactory? You can test other noisy letters by repeating the commands.

With NetPlot you can plot the energy decrease and the trajectories x(t).

Look at the energy decrease during the evaluation.

In[16]:=NetPlot[hopletter,y,DataFormatEnergy]

From the plot you can see that the Hopfield network converged after three discrete time steps.

Look at the state vectors starting at the noisy letters.

In[17]:=NetPlot[hopletter,y]

You can also try the Hopfield networks on some randomly generated patterns.

Generate and look at random patterns.

In[18]:=letterRand=Sign[RandomArray[UniformDistribution[-1,1], {4, 12}]];
letterRandMatrix=Map[Partition[#,3] &,letterRand, 1];
lg = Map[ListDensityPlot[#,DisplayFunctionIdentity,FrameTicksNone] &,letterRandMatrix];
Show[GraphicsArray[lg]]

Apply the network to each of these patterns and look at the patterns to which they converge.

In[22]:=lh=Map[hopletter[#] &, letterRand];
lh=Apply[Partition[#, 3] &, lh, 1];
lhg=Map[ListDensityPlot[#, DisplayFunctionIdentity,FrameTicksNone] &, lh];
Show[GraphicsArray[lhg]]

One can show that the mirror vectors to the pattern vectors also constitute minima to the energy function and, therefore, the mirror vectors are also possible convergence points of the Hopfield network. It is not uncommon for some of the randomly generated data vectors to converge to these inverses of some of the original letters.