Psy/Orf 322 Human-Machine Interactions

Spring 2004

Prof. Alain L. Kornhauser


Human Vision: The System and its Behavior

Web References:  (good link on questions of vision and color.  (interesting link on tetrachromats)



Cellular Model of Biological Visual Processing (The human eye):

        Image Data (Photons)


        Captured by Receptors (Rods & Cones)  (Retina)


        Pass Thru layers of cells (Preprocessing of the Data) (Retina)


        Transmitted over distance (Ganglia (optic nerve))


        Higher – level processing (Visual Cortex)


Retina and Phototransduction:



       Retina is derived from neuroectoderm; (cellular structure that gives rise to the brain)


Retina is sufficiently complex to be a small brain

                           (not a photographic plate, rather operates on contrast and differences)



Cellular Structure of Retina:

5 classes of neurons (Figure);  organized in 3 physical layers, producing 2 distinct synaptic interactions :


Rods & Cones (Photoreceptors, ~108) Densities

Bipolar (flat cone & rod varieties); 11 varieties

Horizontal cells (couple many rods & cones) 

Amacrine (couple many Bipolars to a single ganglia)

Ganglion (off-center and on-center varieties, ~106]

Physical Layers:

Outer plexiform layer:

Receptors (Rods & Cones), bipolar, horizontal

Light must pass through other cellular layers to reach them  

Inner plexiform layer: Bipolar, Amacrine, ganglia

Ganglia layer: Ganglion

Synaptic interactions (Information preprocessing)

direct route: receptor, bipolar, ganglion

        Splits visual signals into 2 separate channels (successive ON & OFF pathways):

a.    lighter than background

b.   darker than background

integration route: horizontal cells integrate and transfer info from distant receptors to bipolar-ganglion cell pathway

        Creates simultaneous contrast of visual objects (forms a receptive field with a center contrasted to an inhibitory surround)



                Continual movement in a sequence of saccades

              Each takes about 30 msec to jump to a new point of regard

              Dwells there for 230 [70~700] msec

       Low resolution, Peripheral vision ~ 180 deg.

       High resolution, Fovea ~ 2 deg.

Fovia; intervening neural layers are pealed back

Blind spot: nasal to fovea, where optic nerve fibers leave retina: no photoreceptor


What a Frog’s Eye tells the Frog’s brain



2 major information flow pathways:



Cones: detect color.  3 varieties: RGB, not very sensitive, day vision, Details of color vision

Rods: low light gray-scale receptors(0.64x107).  20 times as many as cones, (1.2x108)


Cones: less sensitive, faster response, directionally sensitive, connected to more individualized neural channels leads to higher acuity.

Rods: longer photoreceptors allow them to capture more light (more sensitive), are affected by more scattered light and many rods synapse on the same bipolar cell, thus, they have poor spatial resolution; “firing rate” changed upon achieving a threshold of stimulus.


They transduce light into electrical signals through synaptic activity:

Surprisingly: light inhibits and darkness excites photo-receptor cells! 

Most cells in visual system show continued synaptic activity (discharge) even in the absence of illumination.  Nominal synaptic rate of .001 sec.

Appropriate stimuli modulates background synaptic activity, (increasing or decreasing)


Rods & cones make direct synaptic contact with bipolar cells. Horizontal and amacrine cells mediate between lateral interactions between receptors and ganglion.  Ganglion cells project to the lateral geniculate nucleus and the superior colliculus in the visual cortex.


A single cone synapses on two separate bipolar cell channels: one is excited by light activation (on-center (depolarizing)) and the other is inhibited by light activation (off-center (hyper-polarizing)). 


Horizontal Cells: deliver to the ganglion response of more distant photoreceptors than those directly connected to the bipolar.

This leads to Center-Surround Antagonism; Direct light on the center of a receptive field is antagonized by direct light on the surround of its receptive field. This is important in determining borders (27-7)


Ganglion cells : on-center and off-center variety

Also have morphologically and functionally different subsets that serve the same photo receptors in parallel

        X ganglion:  medium sized cell bodies, narrow dendritic fields: high visual acuity (28-7) (80%)

        Y ganglion: largest cell bodies, large dendritic arborization: respond to large targets, perform initial crude evaluation (10%)

        W ganglion: small cell bodies, large arborization, project to the superior colliculus: are involved in head and eye movement (10%)



Anatomy of the Central Visual Pathways

        See Fig 28-1 for visual fields,

        Lens inverts the image on the retina (28-2),

        Nasal portion of the visual field projects on the temporal portion of the retina, etc.

Left optic track contains complete representation of right hemifield of vision. (28-3)

Injury to portions of brain lead to defects in visual fields (28-10)


Processing form and movement:

Retina is mapped in the lateral geniculate nucleus and the visual cortex.(29-1, 29-4)

Lateral geniculate nucleus enhances the antagonism between the center and its surround. (29-5)

Receptive field of neurons in the visual cortex have the following 3 features:

1.      correspond to a specific retinal position

2.      have discrete excitatory and inhibitory  zones 

3.      have specific axis of orientation (see fig. 29-7)

 Appearance of an object depends not on the intensity of the light source but on the contrast between the object and its surround (29-6)


Fig 29-8  clearly describes the orientation of the receptive field of a simple cell in the primary visual cortex.

Basic anatomy, Feedback pathways




       Edge extension

Kanizsa Triangle





        Depth1, depth2


Lightness Perceptions

        2 shades of green


        Old/young womana. 6Beers






        Animated Illusions (Do these yourself at home)

        Count the dots



        Bending lines


MC Escher



Views of Viewing:

 Computational Modeling Vision and Visual Cognition

  A. Kornhauser


Model the process of vision and visual cognition in two sequential phases:

       Low-level  & High-level vision


Low-level Vision

·      1st stage of processing; input is array of intensity levels; available are past arrays.

·      Objective of low-level:   Segmentation and decomposition of a scene into constituent independent parts/objects.

·      Need to find what belongs with what

·      Recovers properties of physical environment

Surface orientation
Depth of field, Stereopsis
Material properties/Textures/Color

Motion in space (Relative motion train moving or station moving?)

·      Seems to be executed independent of domain and task

Given a scene on two different occasions with different tasks/goals, same low-level process applied

·      Process applied is mostly independent of specific object knowledge

Motion perception uses assumption about the continuity and rigidity of objects but not that it is a pen or a hat. The use of rigidity in the perception of motion is present at the age of 5 months.

·      Done in parallel / simultaneously in large portions of the visual field

·      Process can be modeled as sub-processes (Marr):

3 stage process:

       a.  Primal sketch (edge detection) (input)

       b.  Modular processes of  Shape extraction, Texture, Color, Binocular, Motion

       c.  2.5 D sketch (view-centered representation)



Given:   Array of intensity values.

1st step is to normalize the values (eye works on relative intensity)

next steps (going on in parallel if possible, somewhat independent)

·      Edge detection: 

find zero crossing of 2nd derivative of image intensity,

need convolution of filter to “smooth out noise”


·      Texture segmentation:

       Can be viewed either as

Statistical property (variation of spatial intensity levels), or

A class of primitive elements (template matching): textons


·      Binocular Stereo:

    Must solve the correspondence problem:  what matches with what?

Important Principles: 

·      If corresponding primitives can be found: triangularization computes depth

·      Epipolar line constraints: A primitive in the left eye can only be matched with primitives lying on the (epipolar) line on the right eye. (fig 1.6b)

·      Direction of Gaze (relative orientation of the eyes)can also compute depth

·      Changes in  Direction of Gaze:

·      changes in direction of gaze changes the complexity of correspondence!  Correspondence is simplest if eyes are fixating directly on object. (fig 1.9)

·      Color:

Color constancy under varying illumination levels: 

Perceived color of a patch is largely independent of illumination (intensity), but does depend on the color of neighboring patches.  (Red London bus shade or sun)

Implications on shape, depth

·      Motion:

Solve the correspondence problem over a temporal sequence of images

   1. Motion Measurement: construct a Velocity Field,  2. Recovery of Structure

Long Range motion correspondence:  Minimal mapping:  finds the correspondence that minimizes a distance function while ensuring that there will be at least one match. 


Short Range motion correspondence: 

Match neighboring point: “easy”

Match neighboring contours:  Aperture problem:

get good information on velocity field normal to the contour,

get bad/no information on velocity field tangential to the contour (1.14)

              (good for collision avoidance)


Higher-Level Vision:

       Pattern Recognition: problems

Independent of scale

Independent of orientation

Partially occluded

Representing Images:

Nonaccidental properties:  1.Smooth Continuation, 2.Cotermination,       3. Parallelism, 4. Symmetry


Complex entities invite decomposition into simple parts.

Recognition-by-components / Geons

24 geons, 4 attribute x-sections,

108 viewpoint invariant relations

3 relative aspect ratios (larger, smaller, equal)

Then: 3 geons give 109  combinations

Extension of scene perception, Transversality principle: matching vertices


Object Recognition:

       Parallel distributed processing

Visual attention