Neural Networks
The Brain
The brain consists of billions of cells called neurons. Though their working is
not known fully as yet it is understood that the neurons have a structure as
shown in the figure. These neurons make a network by connecting to each.
This is the neural network. The dendrites act as inputs to the neuron.
The output of the neuron is available at the Synapses. The soma and the axon are
required for processing. Each neuron can be viewed as a tiny element with rudimentary
decision making capabilities. Billions of such neurons come together forming a network
and that is what gives us our intelligence. As neurons are not normally known to
regenerate it is assumed that our memory comes from the connections maintained by
these neurons. 'Artificial Neural Networks' which is nothing but an effort to
replicate this worder of nature is the topic of this article.
The Concept
Even though I am no expert in the field, this is how I understand the concept underlying
neural networks and their learning. Suppose we have a set of examples ( which we
will call the 'Training Set'). The training set is basically a collection of inputs
and outputs. For example :
| Input |
Output |
| x1 | y1 |
| x2 | y2 |
| x3 | y3 |
| xn | yn |
The table above shows us an input and the corresponding output. Mathematically,
a function f() exists such that,
y1 = f(x1)
y2 = f(x2)
y3 = f(x3)
and
yn = f(xn)
Now that we have the set of inputs and outputs, the only part missing is the
function f() itself. Unless I have f() I cannot find the output yp for
an input xp. This is where neural networks come to my rescue. When I present
the neural network with this training set, the adapt themselves in such a way
that their behaviour approximates the behaviour of the function f(). Thus any
input xp given to the neural network after the training will produce
the output yp (ideally, but there might be some deviation). If the
training set is carefully chosen and network is well designed the neural network
can be quite effective. As we discuss the topic in some depth, you might find a
little mathematics coming up. I am also not a math guy so I have tried to avoid it
as far as possible.
The Basic Artificial Neuron
The natural neuron is a very complex entity and its behaviour is not fully
understood to replicate it accurately. What we can do with the current technology
is to make a basic replica of the neuron. This is shown in the figure. This model
has many inputs, namely (x1, x2, .. , xn). These
are given to the inputs of the neuron which have a certain weight attached to
them (w1, w2, .. ,wn). This weight might be
positive or negative. In the most basic artificial neuron this the product of the
inputs and their corresponding weights are summed together to produce and out
f(x)=(x1w1 + x2w2 + .. + xnwn).
This is passed through a threshold function which then produces an output of zero or on.
The threshold function is given below.
y = 0 if f(x) < h
y = 1 otherwise
Where y is the final output of the neuron and h is the thereshold.
This kind of a structure is also called a Threshold Logic Unit (TLU).
The transfer function for the TLU in the figure is summation though in practice that
is sometimes replaced by other functions such as logical OR, logical AND, min(x),
max(x) etc. The threshold function 'h' in the figure can also be replaced by
other functions such as hyperbolic tangent, sigmoid, linear etc
rather than a simple step. Infact this is generally the case as the step is not
continuously differentiable. The sigmoid is commonly used for the threshold function.
The sigmoid is shown in the figure and follows the equation :
y = 1/( 1 + e-x )
where, x is the input to the sigmoid.
The basic artificial neuron which we saw above forms the basic building block from
which complex neural networks can be built.
Note :
It is important to know that there is no direct relation between the number of neurons
in a network and the intelligence of the network. Infact too many neurons tend to
memorize the training set thereby giving up their flexibility.
In the articles that will be put up on this page soon, topics like Neural Network Learning,
Different kinds of neural networks etc.. Will be discussed.
Hope you found the information on this page useful.
Parth Malwankar
Contact :
[email protected]