hopfield

^{Chalmers Tekniska Högskola, Artificiella neuronnät
HEMUPPGIFT 1, Mahiar Hamedi

MEMORIES BASED ON THE HOPFIELD-MODELLEN
Introduction
I have created an associative memory based on the simple hopfield-model with asynchron updating. The memory is programmed using matlab and the matlab code togehther with som description is available for download at the bottom of this page.

For pratctical reasons the memory is displayed as a 2-D picture where each unit's state -1 or 1 is represented as red or balck pixels.

Capacity of the net
The capacity of the net was inituially tested using N=100. N is the number of units in the net and thus also the size of each memory to be stored.
The number of stored memories p in the net were tested for p=1 to p=17. For each fix p all of the stored memories were executed through the hopfield net to be tested for stability. The following results were obtained:}

^{p/N (N=100)}	^{non convergent memories (%)}	^{Robustness (%)}
^0.01	⁰	⁵⁰
^0.02	⁰	⁴⁰
^0.03	⁰	³⁰
^0.04	⁰	³⁰
^0.05	⁰	^~20
^0.06	³⁰	^~10
^0.07	^~35	^{not robust}
^0.08	^~35	^-
^.09	⁶⁰	^-
^0.10	^~65	^-
^0.011	^~65	^-
^0.012	⁷⁵	^-
^0.013	^~75	^-
^0.014	^~75	^-
^0.015	⁹⁵	^-
^0.016	^~100	^-
^0.017	^~100	^-

^{In this table the procentage of the non convergent memories is calculated as (# non conv. mem.)/tot mem.
The robustnesss is the average of distortion that can be added to each tested memory without loosing the nets convergence for that memory.
We can see that we have a very distinct border at (p/N)=0.05 where the memory starts to be non convergent. Another distinct border can be seen at (p/N)~15
where the net almost no longer converges to any memory.
Some tests were made using N>100, with similar results.

Functionality of the net
Some tests were performed on the net with the number of stored memories fixed at p=4, where the net converges for all the memories.
These were the four stored memories (1,2,3,4):

Here are some examples from the execution of the net with different initial states. Between each picture frame 100 asynchron updates are performed on the net starting from the left to the right. Each execution is stopped when a fixed state is reched.

1. Memory 2 with 30% distotion

2. The inverse of memory 2 with 30% ditortion

3. memory 1 with 40% distorion

4. memory 4 with 40% distortion

5. The Inverse of the superposition of memory 1 ,2 and 3 starting with 30% distortion

6. A spinglass state for the memory converging from 100% distortion.

From examples 1,2 and 4 we can see that each stored mamory actually converges (even with some initial noise). Example 2 shows a convergent state for the inverse of one of the memories, and indeed each memory has a fixpoint for it's inverse. Tests showed that the probability of obtaining the inverse for each memory is equal to obtaining the actual memory.
We can also see from example 5 that the net converges to states that represnt superpostions of the stored memories, these states are less probable.
Example 6 demonstrates a fixed state that was not among the stored memories or their superpositions. Such spinglass stated exist with very little probability.

The matlab code
The code consists of three different parts. Rita.m handles the graphics and is called from the main routine hw1.m, Memory.m handles the definition and storage of the memories.
You can download the code here:
matlab.ZIP

Hosted by www.Geocities.ws}