9.2.4 Continuous-Time Classification of Letters

Consider classification of the same letters as in the example used with the discrete Hopfield network.

Read in the Neural Networks package.

In[1]:=<<NeuralNetworks`

A continuous Hopfield network will now be used to classify noisy patterns of the letters A, I, and Y. First combine the three letters as three matrices in a list.

Generate three matrices containing the letters A,Y, and I.

In[2]:=x=-{{{1,-1,1},{1,1,1},{1,-1,1},{1,1,1}},
{{-1,1,-1},{-1,1,-1},{-1,1,-1},{-1,1,-1}},
{{-1,1,-1},{-1,1,-1},{1,-1,1},{1,-1,1}}};

Each matrix element contains the gray level of one pixel. The value 1 corresponds to entirely black and -1 to entirely white.

Look at the letters.

In[3]:=xg=Map[ListDensityPlot[#,DisplayFunctionIdentity,FrameTicksNone]&,x];
Show[GraphicsArray[xg]]

Before you can construct a Hopfield network the matrices have to be transformed into vectors.

Transform the matrices to pattern vectors.

In[5]:=

The network is obtained with HopfieldFit

Obtain a continuous-time Hopfield network.

In[6]:=

Out[6]=

To test if the network can correct noisy data vectors, some noisy versions of the three pattern vectors are created. If everything works out the way it ought to, then these three noisy data vectors will be classified correctly.

Create three distorted letters from the three given ones.

In[7]:=<<Statistics`ContinuousDistributions`
y=xv*RandomArray[UniformDistribution[-0.3,1], {3,12}];

The disturbed letters take real values in the interval {-1,1}. To look at the disturbed letters, they must first be retransformed into matrices.

Look at the disturbed letters.

In[9]:=ym=Map[Partition[#,3] &, y];
yg=Map[ListDensityPlot[#, DisplayFunctionIdentity,FrameTicksNone] &, ym];
Show[GraphicsArray[yg]]

The Hopfield network will now be used to "repair" the noisy patterns. The result is transformed into matrices and plotted.

Apply the Hopfield network to the noisy data vectors and plot the result.

In[12]:=yh=Map[hopletter[#] &, y];
yh=Apply[Partition[#, 3] &, yh, 1];
yhg=Map[ListDensityPlot[#, DisplayFunctionIdentity,FrameTicksNone] &, yh];
Show[GraphicsArray[yhg]]

Was the result satisfactory? Since the noisy data vectors were generated randomly, the result may vary from time to time. Repeat the calculations a couple of times with different noisy letters to see the differences.

You can also use NetPlot to illustrate the evaluation on the noisy data vectors.

Check how the trajectories develop with time.

In[16]:=

Check how the energy decreases with time.

In[17]:=

You can also see how the Hopfield network deals with some randomly generated patterns.

Generate and look at random patterns.

In[18]:=

Do any of these random pattern look like any of the three letters? What does the Hopfield network say about them? To which energy minima are these random patterns attracted?

Apply the network to the random patterns and look at the result.

In[22]:=lh=Map[hopletter[#] &, letterRand];
lh=Apply[Partition[#, 3] &, lh, 1];
lhg=Map[ListDensityPlot[#, DisplayFunction Identity,FrameTicksNone]&, lh];
Show[GraphicsArray[lhg]]

Have the randomly generated patterns been classified as any of the letters? If so, do these results make sense; that is, do the original random patterns resemble these letters? As mentioned previously, it is possible at times to get the inverted versions of the letters. They are also attractors of the Hopfield network; that is, their inversions are also possible convergence points of the network. This is an unfortunate feature of Hopfield networks.