Vision is not straightforward • The complexity of the problem was completely overlooked because

The Art and Science of Depiction

– The problem is so difficult – The human visual system is so efficient

Introduction to Visual Perception Fredo Durand and Julie Dorsey MIT- Lab for Computer Science

Intro to Visual Perception

Vision and pictures

Beware of the El-Greco Fallacy

• • • •

• El-Greco, elongated characters • Were supposed due to astigmatism • However, pictures and real people would have been stretched equally • Almost as fallacious as assuming painting should be inverted because our eyes invert what we see

Explain Inspire Malfunction & art Technical simplification – Cinema, Color, JPG

• Pictures can challenge or simplify perception • Emphasize or eliminate cues or channels

2

– Time, color

Intro to Visual Perception

3

However…

Intro to Visual Perception

4

Textbooks

• Monet had a cataract operation • Cataract makes vision blurry and yellowish

Before operation Intro to Visual Perception

After operation 5

Intro to Visual Perception

6

1

Plan

Eye: optics

• Eye • Low-level processing • Different pathways

• Image is inverted (mainly by cornea) • Lens makes the focus

• • • •

Organization High-level Focus, attention Color

Cornea Lens

Intro to Visual Perception

7

Intro to Visual Perception

8

Eye: visual angle

Retina

• Corresponds to size of the projection on retina • Depends on real size and distance

• Layer of photoreceptors • Light->neural signal • Optic nerve

Visual Cornea Lens angle

Retina

Visual angle

Optic nerve Intro to Visual Perception

9

Intro to Visual Perception

10

Photoreceptors

Photoreceptors

• Rod: night vision • Cone: bright, color vision

• • • •

100M rods 5M cones Variable density Fovea: most acuity, cone only

Retina

Retina

Fovea Blind spot

Optic nerve Intro to Visual Perception

Fovea

11

Intro to Visual Perception

Optic nerve

12

2

Field of view

Field of view

• Fovea=2-5 degrees

Intro to Visual Perception

• Fovea=2 degrees

13

Intro to Visual Perception

Summary

Overview of pathway

• Light is transformed into 100M neural signals • But… optic nerve has only 1M nerve fibers

• Input from both eyes is dispatched • Left brain : right part of visual field

Intro to Visual Perception

15

Intro to Visual Perception

Visual processing

Contrast processing

• First step in the retina itself

• We are sensitive to contrast, not to absolute luminance • Useful because contrast is more invariant (it depends less on illumination)

Intro to Visual Perception

17

Intro to Visual Perception

14

16

18

3

Contrast processing

Contrast processing

• Receptors are wired to other neurons • Center-surround organization

• Receptors are wired to other neurons • Center-surround organization

Light

Light Receptors

Receptors -

-

+ +

-

Bipolar Cell

0

+ -

Intro to Visual Perception

19

-

+

Bipolar Cell

Intro to Visual Perception

Contrast processing

Contrast processing

• Receptors are wired to other neurons • Center-surround organization

• Receptors are wired to other neurons • Center-surround organization

20

Light Receptors -

+

-

+ +

Bipolar Cell

Intro to Visual Perception

21

Hermann Grid

Intro to Visual Perception

22

Hermann Grid

- ----- --------++-+------------+-+----------- - -

- ----- --------++-+------------ +-+------- ----- Intro to Visual Perception

23

Intro to Visual Perception

24

4

Vasarely, Supernovae

Mach Bands • Contrast is enhanced at region boundaries

e Intro to Visual Perception

25

d

c

b

a

Intro to Visual Perception

Relation with photo and painting

Visual processing

• Low contrast is not that much a problem • A photo can be brighter/lighter than the original

• First step in the retina itself • … • Next step: visual cortex area V1

Intro to Visual Perception

27

Intro to Visual Perception

Edge detection

Edge detection: Multi-resolution

• Similar to center-surround • Measured using micro-electrodes

• Edge of different sizes

Intro to Visual Perception

29

Intro to Visual Perception

26

28

30

5

Edge detection: not so simple

Retinotopic

• Edges are only a special case • Patterns

• Close optical stimulus map to close parts of V1 • A monkey is shown A • Radioactive tracer • His V1 area is shown in B

Intro to Visual Perception

31

Intro to Visual Perception

Retinotopic

Relation with line drawing

• Close optical stimulus map to close parts of V1 • But not complete correspondence

• The information is ~ the same • Drawing simplifies edge detection

32

• Some neurologist believe that line drawing nicely excites areas of the brain Intro to Visual Perception

33

Intro to Visual Perception

Optical art

Higher-level visual processing

• Op’ Art directly exploits low-level vision

• More complex • Less understood or “measured” • Different pathways

Intro to Visual Perception

35

Intro to Visual Perception

34

36

6

Dorsal vs. Ventral pathways

Different visual channels

• Ventral pathway: What? – Object recognition

• Dorsal Pathway: Where? – Location

• Study on monkeys with damaged brain Intro to Visual Perception

37

Intro to Visual Perception

Different visual channels

Relation to visual arts

• • • •

• Same elements:

Quite complex interactions Not sequential Not one-way Not strictly separate

38

– Color – Form – Layout – Texture

Some interconnections in the Monkey brain Intro to Visual Perception

39

Intro to Visual Perception

Relation to visual arts

Relation to visual arts

• Same elements:

• Same elements:

– Color – Form – Layout – Texture

– Color – Form – Layout – Texture

• Selective treatment

• Selective treatment

– Focus in brain

40

– Focus in brain

• Orchestra metaphor Intro to Visual Perception

41

Intro to Visual Perception

42

7

Form and color

Intro to Visual Perception

Lines

43

Absence of color, contrast

Intro to Visual Perception

44

Shape

45

Duet: shape and texture

Intro to Visual Perception

Intro to Visual Perception

Intro to Visual Perception

46

Symphony

47

Intro to Visual Perception

48

8

Plan of the few next sessions

Assignments

• • • • •

• • • •

Stepping back Organization, Gestalt Perceiving shape and objects Focus, attention Color vision

Intro to Visual Perception

49

Feedback Image Reading Piranesi

Intro to Visual Perception

Reading

Assignment

• Do not forget Gombrich…

• Piranesi tutorial

50

– Demo version on the class web page – Non-photorealistic rendering – Tutorial 1 to 3 – Skip 2.4

Intro to Visual Perception

51

Intro to Visual Perception

52

Talk • Decision next week • Either come with a subject • Or look on the class web page for suggestions

Intro to Visual Perception

53

9