Computer-Aided Ethology
the ethology of ethologists

How do humans perceive and analyze animal behavior?
Can computers simulate these functions?

Computer-aided ethology aims to simulate human ethological functions which employ visual observations and cognitive analysis.
In order to apply technologies that automate certain ethological functions, we must analyze and understand the complex behavior of ethologists

How do animals see other animals and extrapolate behavior ?

My Contribution :

As a pH.D. project ,  I captured the (X, Y) coordinate* and Surface Area,  of 4 mice,  during 4 hours,  (to a data file), every 40 millisecond  (25 data points (images) / second).  Each, of the four motion files contains approx. 360,000 records!

3 principle componets  :   Input, Data and Behavior Analysis.

The problem, as I've defined it, is divided into 3 principle componets, 

Various technologies are available with many more under development.  For example :

The technology associated with each componet can be improved using new, developing technologies. 

Computer vs. Human Vision

This is an image of just one basin with the mouse, the water and the food distributor.

Fundamental Differences  : From a machine vision perspective the image contains pixels with gray levels.  We recognize water, food, sides, floor, ...  We can defined objects at fixed coordinates and track objects which contrast from other objects.  However, it is difficult to imply impression, i.e. "near the food". The subject (i.e. mouse) surely sees all this from another perspective.

(* Note : the X,Y coordinate is the location of the berry center of the animal)

How to Validate?

Graphical representation of Basin and Motion Data

How to Present Data?   This diagram of motion data provides certain information but eliminates sequence and time.  Replacing and eliminating data could provide more information for humans but limit a computer's potential to resolve problems. This illustrates the mouse's activity and indicates area of interest, to the mouse!

Once behavior is assigned by a computer, how can humans verify the machines evaluation? Display and control programs can aid humans verify computer generated behavior. The data can be displayed and manipulated (indexed, sorted, graphed, ...) in order to obtain and verify information.

The right side image is a graphic representation of the basin, it's landmarks, the mouse's position and calculated motion data (i.e. speed, distance, etc.).  I developed the program, SUIVI, in order to explain and demonstrate "The Problem" and to aid verify certain results. 

General Vs Specific Problems

The problems encountered in this example are similar to those found in many HUMAN AND ANIMAL  motion / behavior studies.   Basically, there are two types of problem, specific (i.e. mice) and general (intelligent machine vision).

Although we're using mice in this example,  we are interested in the process and not the subject. 

We can adapt the technology to :
- animal tracking
- autonomic robots
- consumer browsing and shopping behavior studies
- sports and athletic performance aids
- security application

The software which we are developing for such applications must be very intuitive, adaptive, perform self-evaluations and optimize their functioning.

        The Specific Problem !

The General Problem !

Data Interpretation

Can programs be trusted to find behavior in all the data?  What does the data for sleep or a jump look like?   Or....?  Can the same programs find the same behaviors in other mice? What other behaviors can humans or computers find, or not find?

Rapid Changes in Data (jump?) This graph represents the change in surface area over time. It shows a rapid increase and decrease in surface area in a period of .8 seconds. The video cassette confirms that the mouse jumps up toward the camera causing the mouse to increase rapidly in perceived size and surface area.

* Very little delta on all three axis during periods lasting between 5 and 10 minutes.  Experiences has shown that even sleeping animals "move" when seen through machine vision systems.

Group and Analyze Data

Regions or areas can be subdivided into geographical regions which are labeled (i.e. Aa, Ab,.. or 1, 2, 3,..).  The creation and labeling of regions depends upon analysis to be performed. The relation between regions can be of interest.   The time an animal occupies or traverses a region,  their transitions, .. Certain behaviors (i.e. drinking, sleeping, ) can be associated with a region (water distributor, floor area).

Conclusion

There are many who are  working to apply machine vision and automatic behavior analysis tools. To see links to other sites click here : Ethology Links

We have a long way to go.

Most animals have developed the ability to observe other animals and obtain behavior.  For example, eat or be eaten behavior can be initiated by a sequences of visual stimuli combined with learned responses. 

But, how can machines find behavior in motion data?  Fortunately animal have more optimized computing power and developed responses.

Innovative techniques are required to develop software which is modular with long-term possibilities and can clearly communicate data analysis results.

For further technical details click here : User's Manual of Suivi

For a down load copy of SUIVI click here : Down Load Suivi

STILL UNDER CONSTRUCTION

Last updated 10/26/00 10:05:09