"ITSTIME" by Andrey Makushkin
toAndrey(at)yahoo(dot)com
March 18, 2014 (last updated May 26, 2015)
This set of ideas is written to solicit a response from the robotics community. Please forward it to your friends and send me an e-mail with your thoughts at the above address. Some of the ideas are not new. They are centered around Developmental Robotics. Additional value of this document lies in summarizing important concepts and providing links to relevant Wikipedia articles.
The secret to creating a successful robot system is to express sensory inputs in terms of time and constants. As a result, outputs to actuators and internal representations will also be expressed in terms of time. Body provides most constants needed to retain information after sensor readings are converted into events as well as converting events back into actions. Think of these constants as yardsticks for your vision, hearing and tactile sensors. One can also think of these constants as unit vectors needed in order to convert observations from real world into internal event based representation and back. Assuming that the body supplies a set of constants can give us insights into problems such as symbol grounding and Qualia.
Once measurement units are left on the border of your system embedded in the body of the robot in terms of constants your system is left free to compare apples to oranges in terms of time. It does much more than gets rid of units in the system. In biological systems this results in neuroplasticity.
Events should be generated when there is a change in the sensor reading caused by the changes in the environment. Body of the robot is part of that environment. Body movements will cause changes in sensor readings.
Let’s say you have a sensor with an 8 bit ADC. Instead of presenting the value to the learning system as an integer, one needs to convert it to an event that does not have any parameters. Sensor ID combined with sensor value in a range [0-255] becomes an event ID. This event ID needs to be converted to a sparsely coded representation. All that is left to describe the event is to specify the time when it occurred. Time of the event defines it's membership in a set. This membership has to be learned! Some learning algorithm's only goal is to assign a set membersip as in classification. In many systems, wrongly events are manually assigned to a set by buffering based on the granularity of that system. Another class of systems that is even worse are those systems which are not implemented in terms of online algorithms.
Another reason why time is so important is planning. Assume the robot is in a partially observable environment. It might be tempting to steer away from planning into reactive behavior, however it is much easier to remember a recognized pattern than all the inputs that lead to recognition of that pattern. This pattern could be a partial pattern. This is why it is so important it is represented sparsely. How long the system remembers a pattern is at most importance. If a partial pattern is remembered long enough and a new piece of the puzzle comes in to complete it, recognition of a pattern could lead to recall of a sequence.
It might seem counterintuitive to measure values in terms of time however there are many real world examples of this. Some ADCs internally measure time between events (voltage changes) however their output is a scaled number of events per unit of time. Another example is expressing distance in terms of time using a light-year constant.
For the philosophers out there: If we can perceive and act only in a single moment, where does time come from? Is it not the fabric of our mind?
Besides providing constants there is one more reason why having a body is so important for learning. Body provides feedback into the world. Imagine you have setup an experiment to study the world. Body is the only thing that can change the parameters of your experiment.
In an event based model actions are triggered by events. In a state based model actions are triggered when a change in state is detected. To detect state changes, current state has to be compared to a previous state. This means the system has to retain the values of previous states. In other words, state can not cause changes. Only events can cause other events and state changes. For example cold temperature does not cause water to freeze however a drop in temperature does.
Are biological NNs state based or event based? Do neurons maintain state or simply propagate events? It turns out there are more feedback connections in the cortex than there are feedforward. Do extensive feedback connections in a NN provide previous state value in order to detect state changes? Are they there to regulate the competition to perform the same function among the cortical columns? Could it act as a memory system that compares past state values to present? Could it be that the information they carry is not sparsely coded (demultiplexed) and requires more bits to represent?
Artificial neural networks provide the perfect fabric for time to flow through. Every time a pulse propagates from one neuron to the next, a new event is generated. ANNs are usually crippled by the fact that data is fed into an ANN in parallel (current and previous states are presented to input layer together) and backpropagation algorithm is used for learning. Such as in "Time delay neural network" (TDNN). This destroys the partial ordering of the events represented by input data. In addition TDNN, creates a combinatorial explosion as "delay" grows. In order for feed forward ANNs to work, the topology of the network has to change while it is learning. This determines how long it remembers a pattern and preserves the event order.
There are two types of patterns spacial and temporal. Current ANNs are good at recognizing spacial but fail at recognizing temporal patterns. Recognizing temporal patterns is essential to determining causality.
Brain is divided into functional regions and cortical columns. You need more than one algorithm to implement a segmented network like that. One algorithm connects neurons within a segment. Another algorithm connects neurons from one segment to another. This could be a genetic algorithm. Given this hierarchical structure a designer could build reflexes into the ANN by manually connecting segments to other segments. For example, connecting sensory neurons of the cheek to a region in the brain responsible for moving neck muscles in order to create a rooting reflex. Network segmentation also helps battle combinatorial explosion since each algorithm is responsible for connectivity only in its own level of hierarchy. Reflexes could be the answer to the hardest question in robotics: “How do you make a robot want something?” Desire is like a fitness function that constrains the learning system from falling into the highest entropy configuration.
There are reasons to seek alternatives to ANNs. Computers of today are better at moving “sets of data” by using indirect addressing. This way a set of events can be reused without being modified or moving in memory. Brain recognizes patterns and detects changes in sensory input streams. It detects changes in stored patterns against recognized patterns. Provides a focus/attention mechanism. Computers are not very good at learning patterns. They are much better at detecting randomness (noise) in a signal for example by observing its distribution and discarding it. Leaving you with information that needs to be remembered.
BEAM robotics seems to experimentally prove that pulse propagation is the basis to achieving complex behavior. Nervous nets even give us a glimpse at a possible hardware implementation of ANNs. Imagine using DRAM memory cells as RC (Resistor/Capacitor) elements of a Nervous Net. Instructions copy data from cell to cell or perform simple operations to form a network. Timing of the pulse propagating through the network is controlled by the decay of individual memory cells.
Twenty years after "Living Machines" we get IBM's TrueNorth, a spike based or "event driven" processor architechture and Numenta's cortical learning algorithm - HTM.
It might be possible to leverage SOA 2.0 (Event Driven SOA) to implement ANNs. SOA 2.0 provides time synchronization and determines causality. Each homogeneous service can be addressable by ID and implement a single function f(num) where num is a lage number or a vector of bits. Learning algorithms determine which services call what and call parameter value.
Linear Separability of a set is important for Linear Classification by algorithms such as Perceptron. Given a high enough dimention as in sparsely encoded value, points in a set are always separable by a hyperplane.
In order to achieve a complex behavior a robot needs to have a large number of sensors. This is required since sensory inputs serve as building blocks for higher abstractions. Although it is possible to convert an analog signal into digital fairly inexpensive, signals often need to be amplified or filtered in hardware. As an alternative, one can construct optical sensors by using filament to carry light from display to the sensors. Light can be any color, can be polarized or modulated. By changing pixels on the screen, sensor's dynamic range can be adjusted. Sensors can mechanically dim the light or refract to narrow the spectrum. For example passing polarized light through another polarization filter mounted on a rotary shaft. Fibers coming from sensors can terminate on CMOS devices such as mouse sensors which have high frame rates. Tactile, compound eye, joint position and many other types can be constructed in this manner.
This mechanism is similar to our own brains where nerve fibers from peripheral nervous system terminates on the cortex in the bran. Scientists tell us that unfolded cortex looks like a sheet the size of a dinner napkin.
Similarly optic fibers going to the actuators can originate on the display. Digital electronics can be completely iliminated from the system which lowers the learning curve.
The designer does not need to understand the values returned by the sensors. Calibrate them or know what units they are measured in. This is what your learning system is for. Just ensure a high dynamic range of the sensor itself.
Sensor reading are not the same as measurements. Measurements can be composed of successive sensor readings or readings from multiple sensors. Your system has to learn what measurements it can do using the sensor readings and these measurements should not be enumerated / limited by the designer. Traditional computing is based on the idea of enumerating instructions and data representation. Sparce coding moves away from using enumerations in data representation. Feature learning is another technique used to avoid enumerations for data representation. What's left is to move away from enumerating instructions.
When it comes to actuator technology, variable compliance is the deciding factor. Hydraulic system is one of the solutions that is extremely difficult to implement for hobbyists. Ideally one would just have a hydraulic transmission: a variable pressure pump connected to a single rotary actuator. Multiple pumps would be running on one axel creating a multi-actuator system.