Spatially Non-adjacent areas on retina emerge as Coherent Patterns
Herd is a collective not spatially adjacent
Single Deer masked by dense autumn leaves
Extraction (Enacting) of Pattern:
A way to distinguish and highlight multiple dynamic patterns
Or: What is the function of the human neurology that produces Apparent Motion? Another way of stating the computational level issues is to ask what is the function of the human neurology that enables the perceptual experiences we call apparent motion. Surely evolution did not teleologically aim for enabling movies, video, and computer animation? What use is the neurology that Thomas Edison took advantage of when he invented movies?
Algorithmic Level Analysis
The flow of differences in a Boolean net
Phase relations among two flows in a boolean net
Or (across the chasm):
Some unspecified neural algorithm: The process that generates apparent motion
Implementational Level Analyses
Boolean process in a PC in 2004
Or (across the chasm)
Human processes (CNS)
The fact that apparent motion phenomena exist presupposes that the neurology and processing algorithm exist
Phase Relations between System Dynamics
and Representation:
A Review Basins and Sub-basins in N, K, Boolean Systems
Example: Two-node dynamic system (Figure 3) Historical trace
Time: the horizontal axis
Node 1
Node 2
Iteration # ==>
Figure 3. Historical trace of sixteen iterations of a simple two-node system that has basin length L=4. The repetition of the basin pattern (length 4) is emphasized by alternating shading in the Iteration # row.
Note: {This is an impossible Boolean System made up to illustrate Apparent Motion ideas. It requires (or presumably would have) a third node to have a deterministic flow.}
Basin Length L=4
the pattern of black and white squares repeats every four iterations
Node 1 accounts for L=4
Sub-basin is created by Node 2
Sub-basin Length: SubL=2
Alice:
Stories First!
Explanations take such a dreadful time
Extracting Basins and Sub-basin Patterns
from a Dynamic Universe
Suppose that Figures 1 a through d represent frames from a movie. These frames are shown in rapid succession. The dark rectangle (top row) and bottom rectangle (bottom row) will appear to move across the frame from left to right. Timing is important in this effect; when the frames are shown at around 25 frames per second (fps) the effect is maximal.
Figure 1a
Figure 1b
Figure 1c
Figure 1d
==========>Time ===========>
(about 40 msec per frame)
Figure 1. If frames a through d are shown in rapid succession, about 25 frames per second, the objects in the frames will appear to move forward, from left to right and (in 1d) off the frame of reference.
The motion, of course, can be seen to be going backwards (here represented by right to left).
Figure 2a
Figure 2b
Figure 2c
Figure 2d
==========>Time ===========>
(about 40 msec's per frame)
Figure 2. If frames a through d are shown in rapid succession, about 25 frames per second, the objects in the frames will appear to enter the blank frame (2a) and move backwards (from right to left).
Representing Sets of Successive Iterations Simultaneously
in a Single Window
Usual Procedure: historical trace is outputting the state (black or white) of each node on each iteration at the moment that that iteration occurs.
1. Let the system run for X iterations
2. Then print the set of X iterations at once in a window
3. Let the system run for X more iterations
4. Output those X new iterations on the screen
right over the output of the previous X iterations.
5. And so on.
The representation will now become a window showing the pattern generated by X iterations. This pattern is over-written as soon as the next X iterations of the system are complete.
Such a window will in effect be like a frame on movie film:
It will show one pattern (of length X iterations) then the next pattern of X iterations then the next pattern of X iterations, and so on. Differences in the positions (relative to the window as frame of reference) of various elements of a pattern on successive windows will generate apparent motion (and apparent stability) effects. Let's examine how that happens.
Backwards Apparent Motion:
Window Size Greater than Basin Length
Node 1
Node 2
Iteration # ==>
Figure 4. A window of size 5 iterations imposed on sixteen iterations of the the historical trace of a simple two-node system that has basin length L=4. The basin patterns (length = 4) are emphasized by alternating shading in the Iteration # row.
Note: {This is an impossible Boolean System made up to illustrate Apparent Motion ideas. It requires (or presumably would have) a third node to have a deterministic flow.}
Forward Apparent Motion:
Window Size Less than Basin Length
Node 1
Node 2
Iteration # ==>
Figure 5. A window of size 3 imposed on the historical trace of sixteen iterations of a simple two-node system that has basin length L=4. The basin patterns (length = 4) are emphasized by alternating shading in the Iteration # row.
Note: {This is an impossible Boolean System made up to illustrate Apparent Motion ideas. It requires (or presumably would have) a third node to have a deterministic flow.}
Apparent Stability:
Window Size Equal to Basin Length
Node 1
Node 2
Iteration # ==>
Figure 6. A window of size 4 imposed on the historical trace of sixteen iterations of a simple two-node system that has basin length L=4. The basin patterns (length = 4) are emphasized by alternating shading in the Iteration # row.
Note: {This is an impossible Boolean System made up to illustrate Apparent Motion ideas. It requires (or presumably would have) a third node to have a deterministic flow.}
Finding Sub-basins.
Apparent Stability of Subsets of Nodes:
Window Size Equal to Sub-Basin Length.
Node 1
Node 2
Iteration # ==>
Figure 7. A window of size 2 imposed on the historical trace of sixteen iterations of a simple two-node system that has basin length L=4. The basin patterns (length = 4) are emphasized by alternating shading in the Iteration # row.
Note: {This is an impossible Boolean System made up to illustrate Apparent Motion ideas. It requires (or presumably would have) a third node to have a deterministic flow.}
1. The neural circuitry that produces Apparent Motion Illusions Is capable of adjusting phase relations
and therefore
extracting dynamic patterns from a dynamic universe.
(Evolutions wasn't teleologically aiming for video and movies)
2. Periodic and Aperiodic Phase Shifts
are ALL
that is required for Dynamic Form Perception
Leaping the Chasm:
As
Window Size adjusts the Phase Relations
between
E42 Boolean Dynamics
and
Screen Representational Dynamics
So Can
Neural Processing adjust Phase Relations
between
Retinal Image Dynamics
And
Representational Dynamics
Enactment
This is not a passive reception of info
It is an active generation of Emergent, Self-Organized Form
1. Apparent Stability
The Apparently stable forms that emerge don't have to be stable in the Universe; they can be ongoing processes
2. Arbitrary Units
Coherent Extracted Pattern
does not have to correspond
a System's formal characteristics
(as described in a dynamic systems or other theory)
A. Not Formal. That is, Extracted Patterns don't have to be basins or sub-basins or other formal characteristics of the Universe
qua Universe
B. Regular. Extracted Patterns simply need to be
(arbitrary) regularities
in the environment
that can be stabilized by a creature
who is changing phase relations
in a perceptual system
C. Contextual Utility. Only Arbitrary Units that have utility will continue to be selected (one hopes)
Cultural Relativity in how the Universe is broken up into Arbitrary Units.
3. Attentional ? Shifts
Sudden, holistic, changes
in dynamic pattern
can emerge (self-organize)
from changes of RELATIONSHIP
between
two dynamic systems
(particularly from changes in
phase relationships)
Morphogenesis--the coming into being of form as a dynamic system evolves over time--is Relational
4. Coordinated Systems of Utterly different Kind
Physiological process (electro-chemical)
is nothing like
photons or pressure differences
A. Nothing is REPRESENTED
in a camera/tape recorder sense
B. Entrained Systems
One system of dynamic patterns (photons) sets up resonances in another system of dynamic patterns
Via COUPLING (ENTRAINMENT)
{This is another discussion}
This coupling between Universe and CNS
provokes dynamic patterns in the body(CNS)
Not Passive Reception. The Extracted Pattern
(in the CNS) is
an active co-generation by the CNS
VIA processes like shifting phase relations (within the CNS)
5. Movement: A fundamental Dynamic
We live in world of complex dynamic patterns whose perception is fundamentally tied to motion.
Physiological Nystagmus
Tremors which shift the eye less than one degree of visual angle (which is about the distance between two foveal cones) around 10 times a second
Saccades
Micro-Saccades
Movement of creature through environment
Movement of objects
The Boolean
Phase Relation Model
is an explicit model
of how the dynamic patterns
can be extracted
from all this motion
Summary Overview of Model
Meta-principles in a Batesonian Epistemology
Bateson's verbal descriptions of differences in networks as fundamental to a theory of knowledge
The mapping of difference-based epistemology onto Boolean Evolutionary models
The model metaphor
(in the logic of dreams)
:
AS
Phase relations
between E42 processes
and
screen representational processes
produce dynamic pattern
SO MIGHT
Phase relations
between
Retinal processes and
representational processes
extract dynamic pattern.
A simple, explicit model of
1. The Apparent Stability of "Things"
in a world of relational process
2. The ability to make a shift (attentional?) that extracts wildly different forms from the same dynamics
Physiological Nystagmus. [Note 3 ] Visually static objects in everyday life require the constant movement of our eyes (physiological nystagmus) in order to be seen. Physiological nystagmus is produced by tremors in the eye muscles. These tremors prevent the light from stable objects from falling on the same retinal cells from moment to moment. They do that by shifting the eye less than one degree of visual angle (which is about the distance between two foveal cones) around 10 times a second. Experiments with stabilized images (see Palmer, 1999, p. 521ff) indicate that if an image moves synchronously with these tremors, the perception of that image disappears after a few seconds. Bateson (1979, pp. 90, 91) uses nystagmus as a fundamental example in arguing that difference is the basis of mental process. A static relation between a static object and a static eye would produce no differences and therefore no knowledge; this is in full concordance with the stabilized image results. The perception disappears under such circumstances. Nystagmus assures that the eye receives a continuous flow of differences from environmentally static objects.
Enacting a World, Enacting a Mind. We are proposing that the universe be conceived of as a web of dynamic relations some of which set up a field of dynamic relations on the neural receptors of the retina. This, however, is not a passive receiving, as the word "receptors" taken in its strongest sense might connote. A living being has eye muscles that actively twitch the eyes (nystagmus) so as to create motion relative to the context in which it is enmeshed. Add to this other actions (e.g., saccades and micro-saccades, to say nothing of a being's movement through the world). All these movements, taken in relation to a dynamic (moving) universe, co-create the retinal image. That is, the image is not simply received from the world by the eye; the eye (i.e., the being) is an active partner in creating the retinal image. The model we prefer is of dynamically active eyes AND a dynamically active nervous system enacting a receptive field on the retina in relation to dynamic relations ongoing in the universe. Varela, Thompson and Rosch (1993, p. 9) put it this way: "We propose the term enactive to emphasize the growing conviction that cognition is ... the enactment of a world and a mind on the basis of ... actions that a being in the world performs."
What is proposed here is a more general and macro hypothesis than those relating to physiological nystagmus, saccades, and micro-saccades, though those phenomena form a basis for the structure of the model. The proposal includes not just the co-creation of the retinal image but also what happens after the retinal image is enacted. The proposal is that after the retinal image is co-created there is a mechanism that can adjust phase relations between the dynamics of the retinal image and a later processing, including a person's experience. That is, we are proposing that a fundamental mechanism of pattern perception is the ability to create and adjust phase relations. Nystagmus is simply an early example of this process, an example that influences the retinal image itself. And we are proposing that the neural mechanisms that adjust these phase relations produce (as an artifact as it were) those phenomena so-called Apparent Motion.
But the mechanism of nystagmus introduces a puzzle. To keep sending signals neurons must be repeatedly stimulated; to gain this repeated stimulation, nystagmus moves the light sensitive neurons back and forth (for example) across the edge of stable object in the environment. As we mentioned, that is one basis for proposing that difference is a fundamental of sensation and therefore knowledge. The puzzle is that we don't see the edge of objects trembling back and forth as two adjacent neurons are stimulated. Static objects appear just that--static. How can they appear static when their perception requires that they move relative to retinal nerves? Our answer here is a model in which the phase relations between differing flows of neural activity is adjusted so as to appear stable in much the same way as happened in Applet that could freeze sub-basins and basins of a dynamic system. We propose that that which is actually dynamically created moment by moment has "apparent stability" in that sense. We propose that the sort of Boolean model used as a basis of those demonstrations is a useful way of thinking about such apparent stability in phase relations; indeed, right now, it is the only we model know of to think rigorously and to communicate explicitly and clearly about how an adjustment of phase relations might produce apparent stability in the perception of form. We are proposing that solid environmental edges that nystagmus would transform into dynamic flows of difference could be re-stabilized by such a mechanism as the one we are proposing. We are not proposing that Boolean systems are how the nervous systems work; we are not proposing that Boolean nets are a "replica" of the nervous system; rather we are proposing that Boolean systems are an idealized and much reduced model that allows us to think about what the operating characteristics of the nervous system might be.
That is, nystagmus creates a periodic motion in the retinal image at a rate of about 10 frames (in our terms) per second. Therefore the fact that objects are seen as stable is not explainable by nystagmus itself since it is creating change not stability. Nystagmus by its nature assures that a stable object is transformed into a dynamic object and thus that some other mechanism is required to produce "apparent stability."
Two biological puzzles: This is a working document, so we shall repeat another time: A biological puzzle is provoked by the fact that a neuron needs repeated input to produce repeated output and we know that stable objects disappear if they stimulate the same neuron (or sets of neurons) across time. The solution to this puzzle is to produce the required dynamics by eye tremors that produce periodic (probably cycle length = 2) inputs from environmentally stable objects to a neuron. This solution produces a new puzzle: Stable objects are perceived as stable not as trembling (perhaps alternating signals from one cone and an adjacent cone). We propose that a phase relation model can solve this second puzzle. We also use the nystagmus example as a way of motivating the model, as a way of persuading the reader that it makes sense that a phase relation mechanism could be an early, built-in perceptual function.
But the nystagmus example is is a minor example of apparent stability, macro patterns are also extracted from the retinal image and are of more interest to us. Let's discuss those cases.
In our Java applets we have given explicit control to users so that they can change the phase relations between the dynamics of a simulated universe and its representation on the screen. We are proposing that in an analogous way phase relation adjustments might occur between the retinal image and experienced representation. We found that adjusting such phase relations produced striking examples of the extraction of patterns. There were a least three different kinds of such pattern extractions--basins, sub-basins, and emergent, dynamic patterns encapsulated in time.
We propose that the neural system has a mechanism that performs adjustments of phase relations as a fundamental process for pattern perception. We do not propose that the neural system has spatially defined "windows" whose size can be adjusted. Rather we propose (that since what our "windows" did was adjust phase relations) that the neural system has mechanism that can and do adjust phase relations so that patterns that could exist on the retinal image actually do come to exist. What do we mean by that? We mean that if we take a static image, see Figure 11, from Exemplar 2, we cannot discern the images we CAN perceive by dynamically adjusting phase relations (e.g., window size = 75, 77, 83). Those patterns emerge through the action of adjusting phase relations. In a similar way, we propose that a retinal pattern of simulation, supposing we could photograph it, would not show the patterns that a person experiences. Rather, the patterns that a person perceives are, in part at least, due to processes that shift phase relations between retinal image and experienced representation in way a similar to the way that our window size adjustments changed phase relations. [Note 4]
Marr's Computational Level Analysis Revisited. Why does human (and perhaps other) neurology enable the "illusion" of apparent motion? Did evolution teleologically aim for movies and video games? Obviously (to us) not. We propose that the illusion of apparent motion is possible because it is based on neural mechanisms which routinely adjust the kind of phase relations that exist in apparent motion experiments and that are the basis of a broad (yet unspecified) range of dynamic pattern perception phenomena.
What and Why. In David Marr's terms, we want to specify the what and why of our proposed pattern perception mechanism. As we stated above, a being needs to be able extract many distinct dynamic patterns at once. Some of these are spatially coherent, others are spread out spatially. Some are unmoving and stable, others are spatially disconnected flows of movement (like hints of a current though a rapidly flowing river mostly obscured but appearing here and there in the visual field of the kayaker). How do we form coherent patterns from these fragmented dynamics on the retina?
Computational Constraints. We began this web page with some broad constraints for a mechanism for the extraction of dynamic pattern. For various reasons we would like our proposed mechanism to be able to extract patterns of motion that are not spatially adjacent. We would also like to be able to shift between different patterns of motion. In our example, these spatially separated patterns of motion may be a collective like a herd or a single coherent object, say a deer, that is masked by something irregular like leaves. In short a useful requirement is that a coherent pattern does not have to be coherently adjacent on the retinal image. The experiments with the window size mechanism demonstrate the power of phase relation adjustments in extracting patterns with these general constraints.
A second design constraint we mentioned at the outset is that there be a way of distinguishing dynamic patterns, that there be a way of highlighting one dynamic pattern and then shifting to another dynamic pattern and highlighting it, or, even better, a way of having multiple highlights each pointing at different dynamic patterns in the same scene. Again, the experiments with adjusting phase relations through window size demonstrate the power of a putative neural mechanism that could make analogous adjustments of phase relations.
Marr's Algorithmic Level of Analysis: E42 and Boolean networks in general.
Summary
Finding the patterns (basins, sub basins, and general pattern) in the world around us is nontrivial and important. We have shown that adjusting phase relations between a visual representation and the dynamics of a system is a way to extract many different patterns within the system's dynamics. We propose that the apparent motion phenomenon presupposes that neurology has a way of of adjusting phase relations between the dynamics of the retinal image and the final representation as experienced. Since that mechanism is in place, we propose it as a candidate for the extraction of complex patterns from complex systems.
Note 1. Our approach is to frame perception as co-created by the perceiver and what is perceived. Knowledge is neither here (in the perceiver) nor there (in the object). Knowledge is the relationship between processes in the universe and processes within a subset of the universe (the perceiver). It's neither in one nor the other but rather in the relationship between the two.
Often the information processing approach is characterized as linear: the world to be a stable entity which supplies input to the perceiving system and the perceiving system then extracts features of the world. The flow of the arrows of influence are in one direction. In contrast, the co-created view assumes that in general there is a feedback loop from the perceiver to the world: the world supplies input which is transformed by a system and output in a way that transforms the world which then supplies more input. In this case, the arrows of influence are in both directions. See for example the issues raised by Varela, Thompson, and Rosch (1993,p. 9, p. 134ff). See also, our comments in the Discussion and Conclusions section of this page.
Note 2. The starting points that define the sets of iterations {1, 2, 3, 4}, {5, 6, 7, 8}, {9, 10, 11, 12} and {13, 14, 15, 16} in Figure 3 are arbitrary. We could just as well have started the cycle on iteration 3 generating a set of iterations defining a basin = {3, 4, 5, 6}. The TAO Tool establishes an arbitrary standard for starting a basin (a standard that is not used in Figure 3).
Note 3. "Physiological nystagmus is a high-frequency oscillation of the eye (tremor) that serves to continuously shift the image on the retina, thus calling fresh retinal receptors into operation. If an image is artificially fixed on the retina it disappears, but physiological nystagmus causes every point of the retinal image to move approximately the distance between two adjacent foveal cones in 0.1 seconds. Physiological nystagmus actually occurs during a fixation period, is involuntary and generally moves the eye less than 1°." Reference: (http://www.diku.dk/~panic/eye gaze/node16.html)
Note 4. Within this Framework the Physical Analysis of Stimuli Must Fail. Take your experience with the static image in Figure 11 and compare it with your experience extracting patterns, dynamic and stable, from Exemplar 2, L=50. The sort of dynamic patterns that emerge are not to be found in a static physical image (see Figure 11), rather these patterns are the emergent results of shifting phase relations among various dynamics.
