Psy/Orf 322 Human-Machine Interactions

Spring 2004

Prof. Alain L. Kornhauser

Human Vision: The System and its Behavior

Web References:

http://webvision.med.utah.edu/

http://www.driesen.com/visual_system.htm

http://www.cis.rit.edu/mcsl/faq/faq1.shtml (good link on questions of vision and color.

http://www.redherring.com/mag/issue86/mag-mutant-86.html (interesting link on tetrachromats)

Cellular Model of Biological Visual Processing (The human eye):

Image Data (Photons)

Captured by Receptors (Rods & Cones) (Retina)

Pass Thru layers of cells (Preprocessing of the Data) (Retina)

Transmitted over distance (Ganglia (optic nerve))

Higher – level processing (Visual Cortex)

Retina and Phototransduction:

Retina:

Retina is derived from neuroectoderm; (cellular structure that gives rise to the brain)

Retina is sufficiently complex to be a small brain

(not a photographic plate, rather operates on contrast and differences)

Cellular Structure of Retina:

5 classes of neurons (Figure); organized in 3 physical layers, producing 2 distinct synaptic interactions :

Neurons:

Rods & Cones (Photoreceptors, ~10⁸) Densities

Bipolar (flat cone & rod varieties); 11 varieties

Horizontal cells (couple many rods & cones)

Amacrine (couple many Bipolars to a single ganglia)

Ganglion (off-center and on-center varieties, ~10⁶]

Physical Layers:

Outer plexiform layer:

Receptors (Rods & Cones), bipolar, horizontal

Light must pass through other cellular layers to reach them

Inner plexiform layer: Bipolar, Amacrine, ganglia

Ganglia layer: Ganglion

Synaptic interactions (Information preprocessing)

direct route: receptor, bipolar, ganglion

Splits visual signals into 2 separate channels (successive ON & OFF pathways):

a. lighter than background

b. darker than background

integration route: horizontal cells integrate and transfer info from distant receptors to bipolar-ganglion cell pathway

Creates simultaneous contrast of visual objects (forms a receptive field with a center contrasted to an inhibitory surround)

Operation:

Continual movement in a sequence of saccades

Each takes about 30 msec to jump to a new point of regard

Dwells there for 230 [70~700] msec

Low resolution, Peripheral vision ~ 180 deg.

High resolution, Fovea ~ 2 deg.

Fovia; intervening neural layers are pealed back

Blind spot: nasal to fovea, where optic nerve fibers leave retina: no photoreceptor

What a Frog’s Eye tells the Frog’s brain

2 major information flow pathways:

Cones: detect color. 3 varieties: RGB, not very sensitive, day vision, Details of color vision

Rods: low light gray-scale receptors(0.64x10⁷). 20 times as many as cones, (1.2x10⁸)

Cones: less sensitive, faster response, directionally sensitive, connected to more individualized neural channels leads to higher acuity.

Rods: longer photoreceptors allow them to capture more light (more sensitive), are affected by more scattered light and many rods synapse on the same bipolar cell, thus, they have poor spatial resolution; “firing rate” changed upon achieving a threshold of stimulus.

They transduce light into electrical signals through synaptic activity:

Surprisingly: light inhibits and darkness excites photo-receptor cells!

Most cells in visual system show continued synaptic activity (discharge) even in the absence of illumination. Nominal synaptic rate of .001 sec.

Appropriate stimuli modulates background synaptic activity, (increasing or decreasing)

Rods & cones make direct synaptic contact with bipolar cells. Horizontal and amacrine cells mediate between lateral interactions between receptors and ganglion. Ganglion cells project to the lateral geniculate nucleus and the superior colliculus in the visual cortex.

A single cone synapses on two separate bipolar cell channels: one is excited by light activation (on-center (depolarizing)) and the other is inhibited by light activation (off-center (hyper-polarizing)).

Horizontal Cells: deliver to the ganglion response of more distant photoreceptors than those directly connected to the bipolar.

This leads to Center-Surround Antagonism; Direct light on the center of a receptive field is antagonized by direct light on the surround of its receptive field. This is important in determining borders (27-7)

Ganglion cells : on-center and off-center variety

Also have morphologically and functionally different subsets that serve the same photo receptors in parallel

X ganglion: medium sized cell bodies, narrow dendritic fields: high visual acuity (28-7) (80%)

Y ganglion: largest cell bodies, large dendritic arborization: respond to large targets, perform initial crude evaluation (10%)

W ganglion: small cell bodies, large arborization, project to the superior colliculus: are involved in head and eye movement (10%)

Anatomy of the Central Visual Pathways

See Fig 28-1 for visual fields,

Lens inverts the image on the retina (28-2),

Nasal portion of the visual field projects on the temporal portion of the retina, etc.

Left optic track contains complete representation of right hemifield of vision. (28-3)

Injury to portions of brain lead to defects in visual fields (28-10)

Processing form and movement:

Retina is mapped in the lateral geniculate nucleus and the visual cortex.(29-1, 29-4)

Lateral geniculate nucleus enhances the antagonism between the center and its surround. (29-5)

Receptive field of neurons in the visual cortex have the following 3 features:

1. correspond to a specific retinal position

2. have discrete excitatory and inhibitory zones

3. have specific axis of orientation (see fig. 29-7)

Appearance of an object depends not on the intensity of the light source but on the contrast between the object and its surround (29-6)

Fig 29-8 clearly describes the orientation of the receptive field of a simple cell in the primary visual cortex.

Basic anatomy, Feedback pathways

Illusions

Edge extension

Perspective

Depth1, depth2

Front/back

Lightness Perceptions

2 shades of green

Recognition

Old/young womana. 6Beers

Animated Illusions (Do these yourself at home)

Eyetricks.com

Views of Viewing:

Computational Modeling Vision and Visual Cognition

A. Kornhauser

Model the process of vision and visual cognition in two sequential phases:

Low-level & High-level vision

Low-level Vision

· 1^st stage of processing; input is array of intensity levels; available are past arrays.

· Objective of low-level: Segmentation and decomposition of a scene into constituent independent parts/objects.

· Need to find what belongs with what

· Recovers properties of physical environment

Edges/Boundaries

Surface orientation

Depth of field, Stereopsis

Material properties/Textures/Color

Motion in space (Relative motion train moving or station moving?)

· Seems to be executed independent of domain and task

Given a scene on two different occasions with different tasks/goals, same low-level process applied

· Process applied is mostly independent of specific object knowledge

Motion perception uses assumption about the continuity and rigidity of objects but not that it is a pen or a hat. The use of rigidity in the perception of motion is present at the age of 5 months.

· Done in parallel / simultaneously in large portions of the visual field

· Process can be modeled as sub-processes (Marr):

3 stage process:

a. Primal sketch (edge detection) (input)

b. Modular processes of Shape extraction, Texture, Color, Binocular, Motion

c. 2.5 D sketch (view-centered representation)

Given: Array of intensity values.

1^st step is to normalize the values (eye works on relative intensity)

next steps (going on in parallel if possible, somewhat independent)

· Edge detection:

find zero crossing of 2^nd derivative of image intensity,

need convolution of filter to “smooth out noise”

· Texture segmentation:

Can be viewed either as

Statistical property (variation of spatial intensity levels), or

A class of primitive elements (template matching): textons

· Binocular Stereo:

Must solve the correspondence problem: what matches with what?

Important Principles:

· If corresponding primitives can be found: triangularization computes depth

· Epipolar line constraints: A primitive in the left eye can only be matched with primitives lying on the (epipolar) line on the right eye. (fig 1.6b)

· Direction of Gaze (relative orientation of the eyes)can also compute depth

· Changes in Direction of Gaze:

· changes in direction of gaze changes the complexity of correspondence! Correspondence is simplest if eyes are fixating directly on object. (fig 1.9)

· Color:

Color constancy under varying illumination levels:

Perceived color of a patch is largely independent of illumination (intensity), but does depend on the color of neighboring patches. (Red London bus shade or sun)

Implications on shape, depth

· Motion:

Solve the correspondence problem over a temporal sequence of images

1. Motion Measurement: construct a Velocity Field, 2. Recovery of Structure

Long Range motion correspondence: Minimal mapping: finds the correspondence that minimizes a distance function while ensuring that there will be at least one match.

Short Range motion correspondence:

Match neighboring point: “easy”

Match neighboring contours: Aperture problem:

get good information on velocity field normal to the contour,

get bad/no information on velocity field tangential to the contour (1.14)

(good for collision avoidance)

Higher-Level Vision:

Pattern Recognition: problems

Independent of scale

Independent of orientation

Partially occluded

Representing Images:

Nonaccidental properties: 1.Smooth Continuation, 2.Cotermination, 3. Parallelism, 4. Symmetry

Complex entities invite decomposition into simple parts.

Recognition-by-components / Geons

24 geons, 4 attribute x-sections,

108 viewpoint invariant relations

3 relative aspect ratios (larger, smaller, equal)

Then: 3 geons give 10⁹combinations

Extension of scene perception, Transversality principle: matching vertices

Object Recognition:

Parallel distributed processing

Visual attention