For a very particular reason (which, of course, is a secret) a few years ago I got really interested
in neural networks.
While studying at the university, I started to develop my first program in this field:
a neural network based on the Back-Propagation algorithm (Neuro1). As my interest kept growing, I wrote
a program to simulate a modular neural network based on the “Mixture of Experts” algorithm (Neuro2),
the subject of
my thesis at the university.
Then, taking a look at how the brain works, I decided it was time to start a new project: it had to be
more realistic than the previous ones and not too computationally heavy; so I decided to adopt the
Spiking Neuron Model (Neuro3).
For a quick introduction about the Spiking Neuron Model, I prepared some
slides;
they are very synthetic, done to be commented in a seminar, but they seem to be better than nothing...
If you need some information about the theoretical aspects of this model, I wrote a brief
paper.
This project is still under development. Here there is a list of what I did so far, and what still remains
to be done (consider that I can work on it during the night and the week-ends only... I would like to dedicate
most of my time to this project; if someone should ever finance me, I surely will!):
Project status:
(2007.10.21)
|
Done:
- Spikes management using either a spike event list or "bit streams".
- Spiking Neuron model.
- Utilities:
- logger;
- generation of data files for further plotting;
- generation of data files to represent the network structure in 3D.
- Low level data structure to contain neurons.
- Low level learning algorithms:
- Synaptic enhancement and depression.
|
|
Doing:
- GUI (as portable as possible: XHTML + XForms + JavaScript).
- Data structure implementation:
- Higher level data structures design (cortical columns).
- Higher level data structures implementation.
- XML file format.
|
|
Todo:
- More data structures to contain neurons.
- Pre-elaborate the input data (e.g. wavelets, DSP, ...).
- Post-process the output data.
- Genetic evolution of the network's structure;
i.e. the kinds of neurons used, their development, connectivity, learning functions and some
parameters should evolve genetically, while the synaptic weights should not
(they are too many; even our DNA could never contain such an amount of data!).
|
This diagram shows the layerd structure of the program, where the levels marked with "ok" are the completed ones:
Standard C++ is the programming language I am using.
The following is a UML class diagram that represents a high level view of the neural network:
The diagrams that follow are obtained from data produced by Neuro3.
They are the result of a simple network made of four neurons, connected as shown in the following diagram:
The presynaptic neuron is shown in red, the postsynaptic neuron in green.
The blue arrows represent external testing signals; in this test, these are strong enough to make the neurons fire.
Presynaptic neuron number 1 and postsynaptic neuron number 2, even if interacting with the others, are not shown.
Each of the following pictures contains two diagrams: the top one represents the membrane potential of
the pre- and postsynaptic neurons, the bottom one plots the synaptic weight as a function of pre- and
postsynaptic activity.
Here you can see the behaviour of a synapse governed by the equation:
where W is a learning window (take a look at the slides for details).
Here it is shown the case where one testing signal is sent to the presynaptic neuron, and another,
slightly delayed one to the postsynaptic neuron. In this way, both neurons are forced to fire in a
short time frame, and as a consequence the synaptic weight is modified:
In this second test, a sequence of testing signals is sent to the presynaptic neuron (with ID = 0),
and one single testing signal is sent to the postsynaptic neuron (with ID = 3) making it fire at t = 0.40 seconds.
In the top diagram of the following picture, the red triangles represent the firing times of each neuron:
Here you can see that the synaptic weight is increased or weakened according to the timings of
the pre- and postsynaptic activity.
Since an image is worth a thowsand words, Neuro can export the neural network's structure as a graph
in the TLP format. The generated file (*.tlp) can be opened with the "Tulip" program that, using the
OpenGL or Mesa3D graphics library, can show something like this:
This is a still picture, but using Tulip you can navigate into its 3rd dimension in real-time just moving the mouse.
You can get Tulip for free from here, an Neuro's data file for this
network from here.
|
