IMAGE ENHANCEMENT IN THE SPATIAL DOMAIN Digital Image Processing, Chapter 3 Gonzalez Woods

Digital Image Processing In Life-Science Sefi addadi 21-3-2012

WHAT DID WE LEARN LAST TIME?

WHAT WILL BE TODAY? Image enhancement in the spatial domain • Gray level transformation – Negative, Log and power law • Histogram processing – Equalization – Matching – Local enhancement • Spatial filtering – Smoothing filters (mean, Gaussian, median) – Sharpening filters – Edge detection – Filter combinations • Background correction – Acquisition (a priori) correction – Retrospective (a posteriori) correction • Rolling ball background correction

IMAGE ENHANCEMENT IN THE SPATIAL DOMAIN • The principal objective of enhancement – “Process an image so that the result is more suitable than the original image for a specific application”.

• “Spatial domain refers to the image plane itself, and approaches in this category are based on direct manipulation of pixels in an image”.

• There is no general theory of image enhancement. • When an image is processed for visual interpretation, the viewer is the ultimate judge of how well a particular method works. Gonzalez Woods Chapter 3

MATHEMATICAL REPRESENTATION General expression

Origin

Neighbourhood

y

g(x, y) = T{f(x, y) }

x

Target

x

(x, y)

Image f (x, y)

y

Image g(x, y)

GRAY LEVEL TRANSFORMATIONS • Among the simplest of all image enhancement techniques.

• Generally denoted by r – values before transformation s – Values after transformation T – the performed

transformation s = T(r)

POWER- LAW TRANSFORMATIONS • S= cr where c and  are positive constants • Power-law curves with fractional values of  map a narrow range of dark input values into a wider range of output values. • The opposite being true for highs. Many devices used for image capture, printing, and display respond according to a power law – Gamma correction

HISTOGRAM PROCESSING • Histogram in digital images is defined as a discreet function • Where

h(rk) = nk rk is the gray level of k and nk is the number of pixels of that value

Histogram normalization Performed by dividing each value by the total number of pixels in the image.

p(rk) = nk / n • p(rk) gives an estimate of the probability of occurrence of gray level rk . • The sum of all components of a normalized histogram is equal to 1.

Dark image

Bright image

Low contrast image

High contrast image

Histogram Equalization p(rk) = nk / n histogram equalization

Each pixel with level rk is mapped into a corresponding pixel with level sk in the output image. histogram linearization

“Automated” process does not require additional input

HISTOGRAM MATCHING • The method used to generate a processed image that has a specified histogram shape. • In contrast to the aim of histogram where we perform automatic uniform histogram.

HISTOGRAM MATCHING 1. Obtain the histogram of the given image. 2. Precompute a mapped level sk for each level rk.

3. Obtain the transformation function G from the given pz(z) 4. Precompute zk for each value of sk using the iterative scheme defined in connection 5.

For each pixel in the original image, if the value of that pixel is rk, map this value to its corresponding level sk; then map then map level sk into the final level zk.

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

HISTOGRAM MATCHING

Original

Histogram equalized image

Using mappings from curve

HISTOGRAM SUMMARY • Useful for assessment of acquisition quality. • Critical for image presentation • Used as basis for subsequent analysis steps such as thresholding etc’.

NEIGHBORHOOD OPERATIONS • Neighborhood is defined as any area bigger than the single pixel.

Origin

x

• May be of any size or shape. • Most used rectangle are around the central pixel • Might be referred to as filters, kernels, templates, or windows.

(x, y)

Neighbourhood

y

Image f (x, y)

NEIGHBOURHOOD OPERATIONS • For each pixel in the origin image, the outcome is written on the same location at the target image. Origin

Neighbourhood

y

x

(x, y)

Image f (x, y)

Target

SIMPLE NEIGHBOURHOOD OPERATIONS Simple neighbourhood operations example: • Min: Set the pixel value to the minimum in the neighbourhood

• Max: Set the pixel value to the maximum in the neighbourhood

SIMPLE NEIGHBOURHOOD OPERATIONS

Max Filter Radius = 3

Original

http://www.biologyimagelibrary.com/

Min Filter Radius = 3

135 142 142 142 142 142 142 142 140 129 113 98 80 80 80 80 71 64 52 31

135 142 142 142 142 142 142 142 140 135 129 124 107 81 80 80 80 71 64 55

115 135 142 142 142 142 142 140 140 135 129 126 124 110 102 96 86 77 75 74

82 115 135 142 142 142 140 140 135 135 129 127 127 127 127 126 119 115 111 108

29 66 103 132 140 140 140 135 135 129 128 141 144 145 145 145 144 144 144 145

R=3

48 82 115 135 142 121 86 63 52 53 72 77 55 32 14 2 2 7 4 4

4 29 66 103 132 140 121 98 70 55 62 73 69 61 45 28 7 10 5 1

5 5 9 33 82 124 135 129 108 83 66 63 69 80 71 53 35 25 7 0

2 1 1 12 43 80 108 125 125 113 98 77 62 63 70 67 64 52 31 14

0 2 0 0 9 30 67 102 117 123 124 107 81 72 74 73 69 64 55 43

1 1 1 1 9 33 52 52 52 52 32 14 2 2 2 2 1 0 0 0

0 0 0 0 0 9 30 52 52 52 32 14 2 2 2 2 0 0 0 0

0 0 0 0 0 0 7 30 52 52 52 32 14 2 2 2 0 0 0 0

0 0 0 0 0 0 0 7 30 52 53 55 32 14 2 2 0 0 0 0

0 0 0 0 0 0 0 1 7 35 55 62 61 45 28 7 7 0 0 0

115 135 142 142 142 142 142 129 108 83 77 77 77 80 71 53 35 25 10 7

82 115 135 142 142 142 140 140 129 113 98 77 80 80 80 71 64 52 31 14

48 82 115 135 142 140 140 134 134 129 124 107 81 80 80 80 71 64 55 42

6 29 65 103 132 140 134 134 129 125 126 124 110 102 96 86 77 75 74 69

6 5 12 43 82 124 134 129 125 126 126 126 127 127 126 119 115 111 108 100

R=1.5

48 82 115 135 142 121 86 63 52 53 72 77 55 32 14 2 2 7 4 4

4 29 66 103 132 140 121 98 70 55 62 73 69 61 45 28 7 10 5 1

5 5 9 33 82 124 135 129 108 83 66 63 69 80 71 53 35 25 7 0

2 1 1 12 43 80 108 125 125 113 98 77 62 63 70 67 64 52 31 14

0 2 0 0 9 30 67 102 117 123 124 107 81 72 74 73 69 64 55 43

3 3 9 33 82 63 52 52 52 52 52 32 14 2 2 2 2 2 0 0

2 1 1 9 33 80 63 52 52 52 53 55 32 14 2 2 2 1 0 0

0 1 0 0 9 30 67 63 52 53 55 62 55 32 14 2 2 0 0 0

0 0 0 0 0 7 30 67 70 55 62 62 62 61 45 28 7 7 0 0

0 0 0 0 0 0 7 30 67 83 66 62 62 62 63 53 35 25 7 0

114 154 181 181 186 186 186 164 124 89 72 59 27 11 13 13 13 6 4 4

123 154 180 186 186 186 186 186 164 116 83 74 59 27 13 13 12 13 6 4

123 154 181 186 185 186 186 186 170 134 102 92 79 50 27 13 13 13 6 4

122 151 181 186 186 186 186 186 171 136 120 102 92 79 49 30 22 14 8 5

119 137 159 181 186 186 186 174 170 150 133 109 92 82 55 52 50 39 29 16

R=2

62 80 106 133 147 125 89 55 42 26 14 9 7 6 3 0 0 0 0 1

82 91 115 154 181 163 125 89 71 47 24 11 7 2 0 0 0 1 1 0

91 105 123 151 181 186 164 116 80 69 59 27 7 7 2 13 6 0 0 4

87 105 119 137 159 174 170 133 102 84 74 50 27 13 4 7 3 0 0 0

56 80 100 115 130 148 158 135 120 102 92 79 49 20 8 1 0 0 0 0

62 62 62 80 89 55 42 26 14 9 7 2 0 0 0 0 0 0 0 0

62 62 62 80 89 55 42 26 14 9 7 2 0 0 0 0 0 0 0 0

56 56 80 91 115 89 55 42 24 11 7 2 0 0 0 0 0 0 0 0

19 19 33 55 79 98 80 69 47 24 7 7 2 0 0 0 0 0 0 0

5 5 5 14 32 51 82 80 69 50 27 7 4 1 0 0 0 0 0 0

THE SPATIAL FILTERING PROCESS Origin

x

Simple 3*3 Neighbourhood

y

e

3*3 Filter

Image f (x, y)

a

b

c

d

e

f

g

h

i

Original Image Pixels

*

j

k

l

m

n

o

p

q

r

Filter (w)

eprocessed = n*e + j*a + k*b + l*c + m*d + o*f + p*g + q*h + r*i

Performed for each pixel in the original image to generate the filtered image

SMOOTHING SPATIAL FILTERS Simple spatial filter Average all of the pixels in a neighbourhood around a central value according to selected mask. Especially useful in removing noise from images Also useful for highlighting gross detail

1/

9

1/

9

1/

9

1/

9

1/

9

1/

9

9

1/

9

1/

9

1/

SMOOTHING SPATIAL FILTERING Origin

x

Simple 3*3 Neighbourhood

1/ 100 1/ 108 1/ 104 9 9 9 199 1/ /9 106 9

1/ 98 9

195 /9

1/ 85 9

1/ 90 9

3*3 Smoothing Filter

104 100 108

1/

99 106 98 95

90

85

Original Image Pixels

y

*

9

1/

9

1/

9

1/

9

1/

9

1/

9

1/

9

1/

9

1/

9

Filter

Image f (x, y)

e = 1/9*106 + 1/9*104 + 1/9*100 + 1/9*108 + 1/9*99 + 1/9*98 + 1/ *95 + 1/ *90 + 1/ *85 = 98.3333 9 9 9

MEAN FILTER SMOOTHING AVERAGING FILTER Original

Mean R=1

Mean R=2

Mean R=4

As R grows Fine details and noise vanish

WEIGHTED AVERAGE • Created by defining different weight for different pixels around the center pixel. • Pixels closer to the center pixel are contribute more to the average • Size of objects in the image should taken into account. a

g ( x, y ) 

b

1/ 2/

1/

16

2/

16

4/

16

2/

16

1/

16

16

2/

16

16

1/

16

  w(s, t ) f ( x  s, y  t )

s   at   b

a

b

  w(s, t )

s   at   b

General annotation for weighted average filtering an MxN image

Comparison of Mean and Median R=2

Mean Filter

Median Filter

GENERAL FILTERING REMARKS • There is no such thing as GOOD FILTER but a SUITABLE filter for a specific need / task. • Spatial filters can and should be applied in combination for optimal results. • Spatial filters change the “numbers” of signal

intensities and might change the area of an object – should be remembered in quantification GO BACK TO THE ORIGINAL

SHARPENING SPATIAL FILTERS “The principal objective of sharpening is to highlight fine detail in an image or to enhance detail that has been blurred, either in error or as a natural effect of a particular method of image acquisition.”

•Sharpening filters are based on spatial differentiation Averaging ≈ Integration = Blurring Gonzalez Woods Chapter 3

Sharpening ≈ spatial differentiation

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

SPATIAL DIFFERENTIATION Differentiation measures the rate of change of a function. For a simplified explanation we will use a 1 dimensional example

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

SPATIAL DIFFERENTIATION

A B

1ST DERIVATIVE The formula for the 1st derivative of a function is as follows:

f  f ( x  1)  f ( x) x Calculates difference between subsequent values and measures the rate of change of the function

1ST DERIVATIVE (CONT…)

f(x)

8 7 6 5 4 3 2 1 0

5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7 0 -1 -1 -1 -1 0 0 6 -6 0 0 0 1 2 -2 -1 0 0 0 7 0 0 0 8 6 4

f ’(x)

2

0 -2 -4 -6 -8

2ND DERIVATIVE The formula for the 2nd derivative of a function is as follows:

 f  f ( x  1)  f ( x  1)  2 f ( x) 2  x 2

Takes into account the values both before and after the current value

1ST AND 2ND DERIVATIVE 8 6

f(x)

4 2 0 8 6

f’(x)

4 2 0 -2 -4 -6 -8 10

f’’(x)

5 0 -5 -10 -15

2ND DERIVATIVES FOR IMAGE ENHANCEMENT The 2nd derivative is more useful for image enhancement  Stronger response to fine detail  Simpler implementation The approach  Define a discrete formulation of the second-order derivative.  Construct a filter mask based on that formulation. We are interested in isotropic filters  Independent of the direction of the discontinuities in the image  Rotation invariant Laplacian - the first sharpening filter we will look at

 One of the simplest sharpening filters  Isotropic  Linear

THE LAPLACIAN •The Laplacian is defined as follows:

 f  f  f  2  2  x  y 2

2

2

Partial 1st order derivative in the x :

 f  f ( x  1, y )  f ( x  1, y )  2 f ( x, y ) 2  x 2

In the y direction as follows:

 f  f ( x, y  1)  f ( x, y  1)  2 f ( x, y ) 2  y 2

THE LAPLACIAN (CONT…) The sum is given by:

 f  [ f ( x  1, y)  f ( x  1, y)  f ( x, y  1)  f ( x, y  1)]  4 f ( x, y ) 2

A filter built based on this 0

1

0

1

-4

1

0

1

0

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

THE LAPLACIAN (CONT…) •Applying the Laplacian to an image we get a new image that highlights edges and other discontinuities

Original Image

Laplacian Filtered Image

Laplacian Filtered Image Scaled for Display

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

LAPLACIAN IMAGE ENHANCEMENT

Original Image

= Laplacian Filtered Image

Sharpened Image

In the final sharpened image edges and fine detail are much more obvious

g ( x, y)  f ( x, y)   f 2

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

LAPLACIAN IMAGE ENHANCEMENT

Original channel

Laplace filter

Subtraction result

http://www.biologyimagelibrary.com/Image - 21481_0_Miller_28_BIL27090

SHARPENING SUMMARY • “A derivative operator is proportional to the degree of discontinuity of the image at the point at which the operator is applied.

• Thus, image differentiation enhances edges and other discontinuities (such as noise) and deemphasizes areas with slowly varying gray-level values”

Gonzalez Woods 2nd eddition Chapter 3

EDGE DETECTION AND FILTER COMBINATIONS

Original

Gaussian blur

Laplace

Gradient magnitude

X

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

COMBINING SPATIAL ENHANCEMENT METHODS • Applying a single spatial operation is usually not enough • A combination of a range of techniques in will usually result in a better final result.

• This example will focus on enhancing the bone scan to the right

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

COMBINING SPATIAL ENHANCEMENT METHODS (CONT…)

(a) Laplacian filter of bone scan (a)

(b) Sharpened version of bone scan (c) achieved by subtracting (a) and Sobel filter of bone scan (a) (b)

(d)

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

COMBINING SPATIAL ENHANCEMENT METHODS (CONT…)

The product of (c) and (e) which will be used as a mask

Result of applying a power-law trans. Sharpened image to (g) which is sum of (a) (g) and (f)

(e)

Image (d) smoothed with a 5*5 averaging filter

(f)

(h)

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

COMBINING SPATIAL ENHANCEMENT METHODS (CONT…) •Compare the original and final images