Today’s Topics Written assignment due today Today: More on edge detection • Edge detection using zero-crossings • Scale-space techniques • Case study: Intelligent Scissors

Advanced Edge Detection Techniques Motivating example #1 • How do we extract edges corresponding to different levels of image detail? Input image

Detected edges (high image detail)

Detected edges (low image detail)

1

Advanced Edge Detection Techniques Motivating example #2 • How do we extract edges in the presence of noise in the input image? Input images

Low-noise

High-noise

Edge-enhanced image w/ noise comphensation

Edge-enhanced images

Last Time: The Image Gradient • Examples  ∂I  ∇I =  0,   ∂c 

 ∂I  ∇ I =  ,0   ∂r 

φ

 ∂I ∂I  ∇I =  ,   ∂r ∂c 

• Gradient magnitude: gives the “strength” of the edge 2

 ∂I   ∂I  ∇I =   +    ∂r   ∂c 

2

• Gradient orientation: gives the orientation of the edge  ∂I    φ = tan −1 ∂r   ∂I     ∂c 

2

Detecting Edges of Any Orientation • Algorithm: 1. Compute the gradient of an image 2. Compute the gradient magnitude at every pixel 3. Label as “edge pixels” all pixels whose gradient magnitude is above a pre-determined threshold • Difficulties: How do we determine the right threshold to use?? How do we control sensitivity to noise?

Advanced Edge Detection Techniques • We will now consider 3 ideas: • Using zero-crossings of 2nd-derivative to detect edges • “Smoothing” images using Gaussian masks • Defining the notion of “scale” for processing images at various levels of detail

3

Today’s Topics More on edge detection • Edge detection using zero-crossings • Scale-space techniques • Case study: Intelligent Scissors

Edge Enhancement as Image Differentiation

zero crossings

• 1st derivative: – positive or negative in the neighborhood of an edge – zero in image regions of constant intensity • 2nd derivative: – crosses zero in the neighborhood of an edge – advantage: can be used to detect edges without relying on a threshold

4

Edge Enhancement as Image Differentiation • Discrete approximation to a 1D 1st derivative:

d I( x ) ≅ I( x + 1) − I( x ) dx • Discrete approximation to a 1D 2nd derivative:

d2 dx 2

I( x ) ≅

d d I( x ) − I( x − 1) dx dx

≅ (I( x + 1) − I( x ) )− (I( x ) − I( x − 1) )

≅ I( x + 1) − 2I( x ) + I( x − 1) • 2nd derivative mask:

1

-2

1

Edge Detection in 1D Using 2nd Derivatives 1. Apply the 2nd derivative mask to the input 1D image 1

1

1

50

50

50

60

60

1

1

0

49 -49

0

10 -10 -59 59

0

1

2. Find all zero-crossings in the resulting edge-enhanced image 49 -49

10 -10 -59 59

5

The Image Laplacian • How do we generalize the notion of the zero-crossing of a 2nd derivative to 2D images? Option #1: Compute all 2nd-order partial derivatives 2nd order

2

∂ I ∂ 2I ∂ 2I ∂ 2I , , , 2 2 ∂r∂c ∂c∂r ∂r ∂c

1st order

 ∂I ∂I  ∇I =  ,   ∂r ∂c 

Option #2: Compute Laplacian 2nd order

Properties of Laplacian: • Single scalar value for every pixel • Zero when both 2nd derivatives are zero

2

∇ I=

∂ 2I ∂r 2

+

∂ 2I ∂c 2

The Image Laplacian • What is the 2D mask corresponding to the image Laplacian? 1D case

1

-2

1

Laplacian

2

∇ I= 2nd derivative along a row

0

0

0

1

-2

1

0

0

0

∂ 2I ∂r 2

+

2nd derivative along a column

+

0

1

0

0

-2

0

0

1

0

∂ 2I ∂c 2 Laplacian mask (sum of the 2 derivatives)

=

0

1

0

1

-4

1

0

1

0

6

Edge Detection in 2D Using Laplacian 1. Apply the Laplacian mask to the input 2D image

2. Find all zero-crossings in the resulting edge-enhanced image 5

10

0

-2

-2

1

5

5

10

• In general, pixels in the Laplacian image may not be exactly zero • A pixel is labelled as zero crossings if one of its 8 neighbors has opposite sign

Zero-crossing Detection Example Input image

Zero crossings

Q: How can we eliminate most of the noise-induced clutter in the image?

“Strong” zero crossings

7

Dealing with Noisy Images Ideally, we would like to remove noise from the input image before doing edge enhancement/detection Low noise

Edge-enhanced images

High noise

General Approach Steps: • Given input image I, compute a new, “smoothed” image I’

• Apply edge enhancement/edge detection to the smoothed image Compute zero crossings of the Laplacian of I’

8

Noise Removal (a.k.a Image Smoothing) How can we smooth noisy images? • Simplest idea: Replace pixel with the average of its neighbors • Averaging mask: 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9

Degree of smoothing can be controlled by choosing masks of different sizes

Noise Removal Using Gaussians The averaging mask gives the same weight to every point in a pixel’s neighborhood An entirely different smoothing operation can be defined by assigning larger weights to pixels that are closer to the center pixel In 1D, one such smoothing operation is given by the Gaussian function (usually m assumed to be 0):

G(x) =

1 2πσ2

−

e

(x − m )2 2σ 2

9

Gaussians in 1D and 2D

1D Gaussian

2D Gaussian

G(x ) =

G ( r , c) = G ( r )G (c) =

2πσ

2πσ

−

1 2

−

1

e

2

e

(x − m )2 2σ 2

(r − m r )2 + (c − m c )2 2σ 2

Noise Removal Using Gaussians Degree of smoothing controlled by parameter σ2 (variance)

How can we create a mask for Gaussian smoothing? 1. Sample the Gaussian at regular intervals on a square grid 2. Choose a mask size that captures most of the “mass” of the Gaussian (usually sufficient to choose masks whose elements span the range [-3σ,3σ]) G(-3) G(-2) G(-1) G(0) G(1) G(2) G(3) (mask for σ=1)

10

Normalized vs. Unnormalized Masks What happens if we apply to an image a smoothing mask whose elements sum to a value α ≠ 1? G(-3) G(-2) G(-1) G(0) G(1) G(2) G(3)

1

1

1

1

1

1

1

1

...

α

α

α

...

1

1

1

1

Ans: The resulting image will appear brighter (α > 1) or darker (α < 1) than the original Normalized Gaussian mask (σ = 1): G(-3) G(-2) G(-1) G(0) G(0) G(1) G(2) G(3)

1 * α

Laplacian of the Gaussian Idea: Rather than smooth image first using a Gaussian & then computing the Laplacian of the result, we can derive a mask that does both operations simultaneously G ( r , c)

∇ 2 G ( r , c) =

Laplacian of Gaussian (L.O.G.)

Note similarity to 3x3 Laplacian mask

0

1

0

1

-4

1

0

1

0

∂ 2G ∂r

2

+

∂ 2G ∂c 2

L.O.G. mask computed by sampling the L.O.G. function on a discrete grid (mask size chosen to be identical to that of the corresponding Gaussian mask)

11

Marr-Hildreth Edge Detection Algorithm Given: An input image I and a value for the variance σ2 Steps: 1. Compute the L.O.G. mask corresponding to the desired value of σ2 2. Apply the L.O.G. mask to image I 3. Compute zero crossings of the result 4. Label as edges all zero crossing pixels 5. Optional step: label as edges only those zero-crossings whose transition from + to - or from - to + is greater than a threshold Input image

zero crossings σ=1.0

zero crossings σ=5.0

One Remaining Question... How do we choose the right value for σ ?

σ is generally referred to as the scale parameter

12

Today’s Topics More on edge detection • Edge detection using zero-crossings • Scale-space techniques • Case study: Intelligent Scissors

Choosing the “Right” Scale?? Rather than trying to decide which scale to use, we can process images at all scales & examine how the resulting image changes with increasing values of σ

Basic principles of scale-space approaches • No single scale is “special” • Treat s as a continuous parameter • As s increases, we should obtain “simpler” images, i.e., no “new structures” should be created • “Structures” that remain present for a broad range of scales signify “important” features in the image

13

Scale Space in 1D

smoothed signal (high σ)

smoothed signal (low σ)

input signal (σ=0)

Scale Space in 1D Resulting Scale-Space Surface σ

x

14

Zero Crossings in 1D Zero crossings of 2nd image derivative = inflections of the intensity function smoothed signal (high σ)

smoothed signal (low σ)

input image (σ=0)

Idea: As the 1D image gets increasingly smoothed, the position of zero crossings moves along the smoothed curve, creating trajectory curves

Scale Space in 1D Idea: As the 1D image gets increasingly smoothed, the position of zero crossings moves along the smoothed curve, creating trajectory curves

Trajectory of a single zero-crossing • For the Gaussian function, trajectories form nested, upside-down U’s • The Gaussian smoothing function is the only function that ensures no new zero-crossings are created as σ increases • Therefore, it is the only function for which zero crossing trajectories have the nesting property

15

Scale Space in 2D • Zero crossing contours form surfaces in (r,c)-σ space • No new zero-crossings created as σ increases

Today’s Topics More on edge detection • Edge detection using zero-crossings • Scale-space techniques • Case study: Intelligent Scissors

16

Motivation Question: How easy would it be for an image analysis system to automatically find the person’s outline (or that of a vase in the picture) from the output of an edge detector?

Motivation: Image Scissoring By scissoring portions of one or more images & pasting them together we can create new, composite images

source images

composite image

17

Image Scissoring: Requirements • Interactive operation • “User is always right” Scissoring system must allow a user to select arbitrary regions • Scissoring operations must be performed efficiently • Scissoring interface must be simple & easy to use

Image Scissoring • Brute-force approach User chooses every single image pixel that defines the region boundary • Here: Intelligent Scissors approach developed by Eric Mortensen & presented at SIGGRAPH’95

18

Intelligent Scissors Basic operation • User loads an image & specifies a “seed” point on the boundary that must be outlined • User then positions the mouse close to the object boundary • System automatically creates a “live-wire” that connects seed & current mouse position & that follows the region boundary as much as possible • Live wire is updated whenever the mouse moves

Intelligent Scissors: Basic Idea Approach answers one basic question: • How should we define a path from seed to mouse that follows an object boundary as closely as possible?

• Answer: Define a path that is as close as possible to image edges

19

Intelligent Scissors: Basic Idea Approach taken in intelligent scissors attempts to exploit user interaction while avoiding the need to detect edges corresponding to object boundaries very accurately

Intelligent Scissors: Basic Idea Solution: • Assign an “edge-ness” weight to every pixel in the image (computed using an edge-enhancement operation) • Weights defined so that edge pixels have very low weights • To connect seed & mouse positions, choose the path that minimizes the total weight of pixels along the path

20

Intelligent Scissors: Basic Idea Two questions must be answered to fully-specify the algorithm: • How do we assign weights to pixels? • How do we find the lowest-cost path between any two image points?

Intelligent Scissors: Weight Assignment Path: a sequence of adjacent pixels in the image Link: a pair of pixels along the path In the scissoring algorithm, weights are actually assigned to links, not individual pixels Given a link defined by pixels p & q, the link weight is defined to be l(p, q) = 0.43f z (q ) + 0.43f D (p, q ) + 0.14f G (q ) f z (q ) = 0 if Laplacian=0 at p f z (q ) = 1 otherwise

f G (q ) = 1 −

term that penalizes links where the gradient orientation at p and q is very different

| ∇I | max(∇I)

largest possible value in image

21

Intelligent Scissors: Path Optimizer Path optimization formulated as a graph search algorithm that computes the minimum-cost, n-link path from seed to all other image pixels Step 1

Step 2

Step 3

Weights

Step n

Intelligent Scissors: Results

22

Intelligent Scissors: Results

Intelligent Scissors: Results By scissoring portions of one or more images & pasting them together we can create new, composite images

source images

composite image

23

Today’s Topics More on edge detection • Edge detection using zero-crossings • Scale-space techniques • Case study: Intelligent Scissors Next time: Another image primitive--Regions • Drawing polygonal regions (polygon filling, etc) • Segmenting real images into regions

24