Reflections and Hypothesis
HOME	LITERATURE REVIEW	REFLECTIONS & HYPOTHESIS	TEMPORAL PROCESSING	STUDY DESIGN	REFERENCES	ARTICLE SUMMARIES	HOTLINKS

 

	Reflections on multiple dimensions		Reflections on real and virtual time
	Reflections on the dimensional domains of semiotic systems		Reflections on simultaneity
	Reflections on two-dimensional notation		Reflections on sequence
	A hypothesis

 
 
 

Reflections on multiple dimensions

 
We have evolved to live in a four dimensional universe.  As we live our lives, we are moving constantly through both space and time, and our brains have therefore evolved to cope with the complex spatial/temporal computations necessary for any action we take.

We have also evolved the capacity to operate in the one-dimensional universe of time alone.  Time, uniquely among the dimensions, is uni-directional. One-dimensional space is bi-directional; multi-dimensional space is omni-directional. But the uni-directional quality of time means that communication systems based on time are intrinsically suited to the bearing of sequential information, and therefore (by definition?) to the bearing of language, in which symbols can be assembled sequentially in such a way as to transmit complex ideas.

We may therefore have two different modes of dealing with time and sequence; one in the four dimensional visual/kinaesthetic domain of action; and one in the uni-dimensional domain of language.  This web page is devoted to an exploration of the kinds of processing we engage in when we deal in different dimensional domains, and how differences in processing preferences and abilities may account for some of the variance we see in reading, writing, mathematical, spatial and motor skills.
 
 
 

 

While symbols are not confined to the uni-directional uni-dimensional domain of time, it is a characteristic of symbols that they tend to more parsimonious in their dimensionality than either the domain in which they are enacted, or the domain they signify.  Language occupies the uni-dimensional temporal domain; painting occupies just two spatial dimension; sculpture three. And yet through the use of these dimensionally parsimonious symbols we can conceptualise entities in both fewer and more dimensions than that occupied by the symbolic medium itself.

Our symbolic behaviour allows us to conceptualize goals, which, although necessarily enacted all four dimensions, may occupy fewer, more, or none.

A drawing, whether on paper, or on a cave wall, results from the execution of a two-dimensional concept, even if the surface we draw it on encroaches on a third, and we execute it in a fourth. Moreover, it may represent  three- or four-dimensional subject matter, like this bison hunt painting at Lascaux.
 
 
 
 

Cave paintings at Lascaux
two dimensional painting on a three dimensional surface
http://sunsite.queensu.ca/memorypalace/parlour/caves/
 

Or it may be a purely decorative treatment of the second dimension; or give a visual illusion of multiple dimensions; or portray an abstract idea that is beyond dimensions.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Bridget Riley: Ra2, 1981
http://www.paceprints.com/contemporary/riley/riley-ra2-657-002.asp

Buildings are three dimensional objects.  But although they may be conceived in  three dimensions, they are built and used in four.  Moreover, the fourth dimension may be encoded in the concept of the building.  The arc de triomphe in Paris, for example, is a design conceived in 3-dimensional form that encodes the four-dimensional concept of a triumphal procession - although in Lalou's two-dimensional painting, the four-dimensional concept is countermanded by the static three-dimensional procession of trees. 
 
 
 
 
 
 
 
 
 
 

 two dimensional painting of a four-dimensional building
Galien Lalou: L'Arc de Triomphe
http://www.mezzo-mondo.com/arts/mm/laloue/LEG007.html

Computer icons, like the dancing pencil at the top of this page are attempts to create symbols that embrace all four dimensions.  But, as the paintings above testify, multi-dimensionality is not essential to richness or flexibility in a semiotic system, and indeed may be antithetical to clarity. Our richest and most effective semiotic system - language - is also one of the most parsimonious dimensionally; oral language, as we have seen, exists in a single dimension, and its visual translation - writing -  in a bare two (writing being esssentially linear).

Yet it is the one-dimensional medium of language that can enable us to transcend the bounds of our four dimensional universe - to think, literally, "outside the box"".

Our mental representations of imagined or remembered things are themselves signs, standing in place of the thing itself; they have only virtual dimensions, and are unbound by either space and time.  It is only when we execute the actions needed to bring the imagined entity to reality, that we are constrained by the limits of our four-dimensional universe.
 
 
 
 
 
 
 
 
 
 
 
 

Artemisia Gentileschi: Self Portrait as the Allegory of Painting, 1630
http://www.mystudios.com/women/fghij/gentileschi_self.html

 
 

The dimensional domains occupied by various communication media are tabulated below:

	1 dimension	2 spatial dimensions	3 dimensions  (1 temporal,  2 spatial)	3 spatial dimensions	4 dimensions
Time	speech, music, Morse		movies,		dance,
Space		drawing, painting photography,	animation	sculpture	gesture,
Space	WRITTEN MORSE	WRITING, MUSICAL NOTATION, CHOREOGRAPHY, MATHEMATICAL SYMBOLS, ARCHITECTURAL DRAWINGS	internet		theatre, kinetic sculpture,
Space					teaching physical skills by demonstration

Speech, music, and Morse code exist essentially only in time, and are perceived in time.

Drawings, paintings and photographs exist and are perceived in two linear dimensions, although the spatial relationship between elements can imply a third.

Similarly, in movies, although four-dimensional information is compressed into three, the two spatial dimensions on a flat screen, together with extra information provided by the temporal dimension (via the spatial movement of the subjects or the camera), serves to imply a fourth.  Graphic techniques do the same for animations.

Sculpture exists in three spatial dimensions, and arguably also in the fourth; it is perceived in both three and four: temporal changes in light, patina, and the stance of the observer all serve to pull the three-dimensional form into the fourth dimension.  Its surface may also function as a two-dimensional surface for graphic information; and it may represent an entity that is abstract or dimensionless.
 
 

Barbara Hepworth: Pelagos
http://www.artchive.com/artchive/H/hepworth.html

Lastly, live performances of many kinds, including musical performances, exist in all four dimensions, and are perceived in all four, although in musical performance, the temporal dimension may (or may not) be dominant.

Our brains may not have evolved to watch movies, but they evolved to deal with the problems of hunting in a spatially complex environment.  They may or may not have evolved to contemplate or create sculpture, but they evolved to orient us in our habitat, and to make tools and shelters.  Brains may or may not have evolved to listen to music, but music's close analogue with speech may be what provides music with its apparently direct channel to our perceptions.

The communication systems printed in lower case letters in the table above might be termed primary codes - codes that are interpreted in the samee dimensional domain as they are transmitted - and they tend to speak to us very directly.

Not all semiotic systems are so direct. Some communication systems, involve the notation of a semiotic code in a different and secondary dimensional domain to that occupied by the primary code itself. These are the communication media printed in capitals in the table above, and represented below:
 

1 spatial dimension	2 spatial dimensions
WRITTEN MORSE	WRITING, MUSICAL NOTATION, CHOREOGRAPHY, MATHEMATICAL SYMBOLS, ARCHITECTURAL DRAWINGS

Typically, notational systems are in two spatial dimensions, and represent a primary code expressed in a different dimensional domain. One of the simplest notations is that of Morse: as a primary code, Morse exists in the temporal domain of sound; when Morse is notated, a single linear dimension is substituted for the single temporal dimension, making Morse transcribable into virtually any linear medium from knotted string to binary code.

Similarly, in written forms of the primary codes of speech and music, a linear dimension is substituted for time.  However, in these, the linear dimension representing time shares with a second linear dimension the function of indicating timbre or pitch (both of which, incidentally, being functions of frequency, are also temporal components).  In both these forms of notation, what the eye sees must be translated by the brain back into temporal form - whether audible, as in reading out loud, or playing music, or inaudible, as in the silent reading of either text or score.

Thus, in both written text and musical notation, the notational system employs two spatial dimensions to represent one temporal dimension.

However, in architectural drawings and choreography, the inverse is the case: information from a three-or four-dimensional primary code has to be condensed into two.  In choreography, one linear dimension represents time, but shares with the second linear dimension the task of representing the three spatial dimensions.  And in architectural or engineering drawings, two dimensions have to do the work of three, by means of conventions by which the third dimension can be inferred from spatial relationships between elements drawn in two.

Notational forms of unidimensional temporal primary codes, eg writing, are thus different from notational forms of multi-dimensional primary codes, such as architectural drawings or choreography.  Moreover, notational forms of temporal primary codes, eg writing and choreography, are also different from notational forms of purely spatial primary codes, eg. architectural drawings. The differences are tabulated below:

 

Notational forms of uni-dimensional temporal primary codes eg speech	Notational forms of multi-dimensional primary codes, eg, architecture; dance.
A single dimension is represented by two	Multiple dimensions are collapsed into two
	spatial/temporal primary codes eg dance	spatial primary codes eg architecture
Involves translation from  uni-directional domain, in which sequence is unambiguous, to an iconic two-dimensional one in which sequence is a matter of convention		The iconic signifiers can be perceived in any sequence without jeopardizing comprehension.

These observations suggest that very different types of processing will be required, depending on whether the reader needs to draw from the two-dimensional page:

• a single linear thread or a multi-dimensional image

• a uni-directional sequence, or an image that can be perceived from any angle

To complicate matters, most notational systems require both approaches to be co-ordinated. A transparent orthography like Spanish or German demands a primarily linear approach to interpretation, although each individual letter needs to be perceived as a two-dimensional icon representing a single sound. Ideographic written systems, like Chinese, or hieroglyphic systems like those of ancient Egypt, have a greater proportion of temporal information encoded iconically - morphemes, rather than phonemes, are ttreated iconically, although the sequence of the morphemes is treated via a linear dimension.  But not all written texts are even organised linearly: the tables in this documents need to be read in two dimensions, rather than one; in this web site, the navigation bar at the top of each page enables the site to be explored in a nonlinear fashion; links within each page also allow for tunnelling against the grain of the text.  And some linear texts, like comic strips, require that iconic material is "read" sequentially.

In musical notation, the systems are also mixed.  The horizontal dimension represents sequence, rather than time itself, duration being represented iconically via the form of each note. There is a largely one-to-one correspondence between the vertical dimension and pitch, but again, this is modified by icons such as clefs, sharps and flats.

In technical drawings, too, the systems are mixed: some graphics represent actual surfaces to scale, with one-to-one correspondence between signified and signifying dimension; others are merely symbols, icons serving almost as words; and most drawings include written annotations.

Finally, mathematical notation primarily represents not dimensional information, but logical relationships.  In math, sequence is merely one of a series of possible  relationships, and is not necessarily represented by a linear ordering on the page.  Some logical relationships have directionality; others do not.  Mathematically notated relationships may or may not have a procedural (ie temporal) component;  1+2+3= ? represents a procedure, in which the linear dimension happens to represent the normal sequence of the procedure, although the precise sequence is not critical. However, 4-2=? represents a procedure, where the sequence is critical;  a minus sign has a directionality which a plus sign does not.   1+?+3=6 represents a procedure in which the linear dimension does not represent the sequence of the procedure. 1+2+3=6 is a statement of a logical relationship.  Its linear dimension is bi-directional, and it does not represent a procedure at all.

Moreover, 4³ could represent a three-dimensional cube, four wide by four high by four deep.  But four whats wide, high and deep? Can the cube implied by the notation be said to have "dimensions" at all? And what of 4⁵? or 4^-2? or 4^0.3?  Like language, mathematical notation, if we can handle it, has the capacity, to take us beyond the boundaries of our four-dimensional universe. Indeed, perhaps mathematical notation could defined as a non-linear language.

 
In order to learn to talk, children must be able to imitate speech sounds.  There is evidence that babies do this from soon after birth, as they become adjusted to the phonemes of their native language (Pinker, 1994).  Clearly there is a brain mechanism that processes speech input, and translates into output, with the phonemes in the correct order. Recurrent ear infections are known to interfere with this process, and result in language delay.  It is also possible that translation mechanism may not always work perfectly.  My own son tended to imitate words back to front - last in first out.  So cup was puc, snake was kay, and - his best effort - crocodile was li-co-co.  His errors were evidence of the phonemic processing that was going on internally - the analysis of the phonological input into a sequence, followed by a reproduction of the sequence - in his case with the order reversed.

In order to learn to walk, and explore their environment, children must be able to interpret four dimensional space - to encode their environmental goals in mental representations of space/time. The child learns, partly by trial and error, how to co-ordinate the speed and direction of her actions in order to achieve her goals.

If all this works well, the child grows up with an integrated sense of space/time, a sense of how long a question takes to ask before she asks it; a sense of how far she can move without losing her balance; a sense of the temporal sequencing and timing involved in brushing her teeth.

But is the temporal processing involved in using language the same as the processing of time and sequence when executing an physical action?  Certainly, physical action is involved in speech, and language processing is often involved during the learning of a physical task - but once the actions are automatised, is phonological temporal processing different from what I will call kinaesthetic temporal processing? And if they are different, what is the relationship between them?
 

A hypothesis

There would seem at least to be three kinds of "simultaneous" mental representations, which may be formed from memory or from imagination:
 

• Phonological, in which temporal elements are stored in sequence, but experienced in virtual, not real, time

• Kinaesthetic, in which the temporal frame for an action is stored as an intrinsic aspect of the memory of the action

• Visual/spatial, in which the dimensional elements are stored independently of real space/time, and all angles of view are perceived simultaneously, as a whole.

The first two of these kinds of "simultaneous" memories involve temporal processing; the third does not.  When the first two are notated in the domain of the third, there is thus potential for perceptual problems. When dealing with visual communication systems, or even the visual, static world, we are faced with a non-temporal medium which is antithetical to sequence. In the uni-directional dimension of time, items are held in sequence, like beads in a narrow tube; in two or three dimensional form, items can all too easily slip out of place.  Most of us, nonetheless, learn to impose sequence the spatial world.  Some, however with a strong capability for simultaneous visual/spatial perceptions, may to find it hard to nail the spatial world into order; to those people notational systems may present particular problems.

(In parenthesis: It is also striking that even kinetic visual events rarely have the temporal impact of phonological events.  I have a button on my metronome that changes the beat from a click to a flashing light.  Most musicians find this useless - it is quite remarkably difficult to get a sense of rhythmic impulse from a flashing light.  This phenomenon may also explain why silent films required a cinema organist to make any impact, and why conducting is so much more difficult than it looks. Our brains seem peculiarly resistant to inferring temporal information from the purely visual world.)

The table below shows what might be considered characteristics of sequencing, timing, and the phenomenon of simultaneity, in relation to the uni-dimensional, uni-directional,  phonological domain; the multi-dimensional but uni-directional kinaesthetic domain; and the multi-dimensional, omni-directional visual/spatial domain:

	sequencing	timing	simultaneity
Phonological	is intrinsic to the dimensional domain; items cannot "escape sideways"; phonological memory	additive	phonological memory preserves sequence
Kinaesthetic	is intrinsic to direction of movement; kinaesthetic memory	geometric, harmonic	kinaesthetic memory preserves action in a temporal frame
Visual/spatial	is not intrinsic, items can easily "escape sideways"	not powerful - hard to make rhythmic statement with visuals alone	visual spatial memory preserves 3-D information about object

 
In phonological linguistic processing, sequence is paramount; if the sequence of sounds is corrupted, meaning is lost.  Sequence therefore takes precedence over absolute time.  When you conceive a sentence, for example "Do you think this amazing weather has anything to do with global warming?", you have foreknowledge of the ordering of the phonemic and morphemic units that make up the sentence, and yet in execution, depending on the emphasis of the delivery, the sentence may take anything from four to nine seconds to say.

In phonological processing, therefore, temporal knowledge is additive, and relative - the passage of time is measured as the sum of discrete components of varied, and varying, length.  The end of your sentence is defined as the end of the sequence, not as a pre-ordained moment in time.
 

Kinaesthetic temporal processing, on the other hand, being multi-dimensional, requires temporal precision.  If speed and distance are not precisely calculated, the hand will miss the ball.  The sequence of discrete actions may be less important.  It may make sense to get the cup off the shelf before filling it with juice, but there may be no good reason for filling it on the shelf instead, then picking it up.

The kinaesthetic requirement for precise timing is therefore likely to require something more like a clock than the adding machine of phonological memory - something that divides time into geometric divisions.  A possible candidate for such a clock may lie in the kinaesthetic experience itself.  As gravity-bound creatures consisting of hinged limbs, and flexible spines, we are subject to the physical laws of harmonic motion.  An arm of given length and weight will swing at a given frequency.  The apparent effortlessness of good athletes stems in part from the efficiency with which they use their own momentum and natural harmonic rhythms.  It may be feedback from our own bodies in action that provides us with the temporal information that we need for precise motor timing.  Alternatively, or additionally, our brains may incorporate internal clocks that are capable of measuring present, past and future time precisely.

However, when motor skills are being learned, they are assembled sequentially from discrete units - actemes? - which, as mastery is achieved, become fused into a single fluid action.  It is therefore possible that in the early stages of learning a motor skill, a quasi-phonological approach to temporal processing is used, which prioritizes sequence over temporal precision, and that what marks the acquisition of a motor skill is the changeover to from sequence-priority processing to time-priority processing.

The postulated differences between the two modes of temporal processing are shown in the table below:
 
 

Linguistic temporal processing	Kinaesthetic temporal processing
sequence has priority over absolute time	timing has priority over sequence
time measured arithmetically by the addition of discrete durations, which may be flexible in length	time measured geometrically by harmonic motion
phonological memory, in which sequence is largely intrinsic to the memory form	kinaesthetic memory, which, having spatial dimensions as well as a temporal one, may be less powerfully sequential.

In practice, however, it is likely that the two modes will frequently overlap, and that processing in one mode will also have some relevance to the other.  Talking itself is a motor skill; words are symbols, and sequence may not always be paramount - as in remembering items on a mental shopping list, where the names of the items are treated iconically, rather than syntactically.  And sequencing our actions logically is also sometimes important, as I tell my son when he tries to put his socks on over his shoes.

To summarize the postulates of this hypothesis:

• Phonological and kinaesthetic processing both involve temporal processing.

• Phonological temporal processing prioritizes sequence over absolute time, or "tempo".

• Kinaesthetic processing prioritizes temporal precision, or "tempo" over sequence.

• Proceduralizing motor skills involve may change in priority from sequencing to timing.

• Visual/spatial processing is marked by its resistance to sequencing.

If there is any truth in these postulations, competence in either verbal or performance fields will depend on the appropriate form of temporal processing being engaged.  The strengths of an individual's preference for visual/spatial processing may also have an impact on the success of the interface between temporal media and visual notational systems.

There a number of points at which things could go wrong:
 

• If phonological memory is poor, it follows that recalling the sequence of sounds in a word or sentence will be difficult.

• If kinaesthetic memory is poor, it follows that it will be harder to learn to proceduralize motor skills.  It may also be more difficult to translate the contents of phonological memory into real time motor performance.

• And if an individual has a strong preference or ability to order his/her visual perceptions spatially, rather than sequentially, this preference may interact with the effectiveness of their phonological and kinaesthetic memory in producing a characteristic profile of learning strengths and weaknesses.

Below is the hypothetical model of these interactions:

A three-dimensional model of temporal/spatial processing  and its relationship to learning strengths and weaknesses

 

Bias towards sequential organization of spatial information		Kinaesthetic memory
		Good	Poor
Phonological memory	Good	GOOD at reading, writing, playing music, memorizing, following verbal directions, acquiring motor skills; dance, gymnastics, computation	GOOD at reciting poetry, computation, singing Clumsy at writing, playing instrument,  May learn procedural skills best by instruction or mnemonics
	Poor	GOOD at gymnastics, physical routines POOR at reading, writing, may learn procedural skills best by sequential demonstration	GOOD at ordering, tidying up.  POOR at reading, writing, may learn procedural skills best by sequential demonstration and repetitive practice

 

Strong bias towards simultaneous visual/spatial capacity		Kinaesthetic memory
		Good	Poor
Phonological memory	Good	GOOD at playing music from memory, memorizing, drawing, including engineering drawing, plastic arts, dance, climbing, conceptual math.  May learn procedural skills well by observation and experiment. If strong bias against sequential ordering of spatial information, may have difficulties with tracking in reading, organising written output, spelling, computation.	GOOD at reciting poetry, conceptual math, singing POOR at computational math Clumsy at writing, playing an instrument.  May have tracking difficulties in reading. May learn procedural skill best by observation, or by phonological mnemonics
	Poor	GOOD at climbing, conceptual math, drawing, including engineering drawing, plastic arts. May learn procedural skills well by observation and experiment. POOR at reading, spelling, computation.	GOOD at conceptual math.  POOR at reading, memorizing by rote, writing, computation, spelling, may learn procedural skills best by observation.

To summarize: Good phonological memory will tend to predict good aural memorization skills, good reading skills; and, combined with good kinaesthetic memory, will predict good writing skills, including good spelling.

However, a strongly visual/spatial preference for dealing with spatial information may interfere with the acquisition of both reading and writing skills, including spelling, especially if combined with weaknesses in either phonological or kinaesthetic memory.

A strongly visual/spatial preference will also tend to predict good conceptual math skills; however, if combined with weaknesses in phonological or kinaesthetic memory, may predict weakness in computation.

Good kinaesthetic memory will tend to predict good motor skills; however when combined with a strongly visual/spatial preference, may mean that these skills are learnt most effectively by observation and experiment rather than by sequential teaching.  Poor kinaesthetic memory will predict poor motor skills, including writing.

 

To test this hypothetical model would require a large study that involved measurements of phonological memory, kinaesthetic memory, and visual/spatial ability, with measurements of learning ability as the outcome variable.

However, music may provide a small shortcut. Music is a hybrid form, requiring the sequential temporal processing of language, and the geometric temporal processing of kinaesthetic experience. The tension between these two elements - the rhetorical and the metrical - is at the heart of most music, from baroque opera to hip hop. The sequencing of rhythmic patterns requires sequential processing; the keeping of a steady pulse requires geometrical processing. It is therefore possible that measurement of these musical skills may provide an indicator of at least two of the variables under study.

And it is further possible, if the model shown above has any justification in reality, that musical training may prove to be a useful intervention in the remediation of a range of learning difficulties.