Lecture 6

Video Data Analysis Filtering in the frequency domain

1. Basics of filtering in the frequency domain 2. Correspondence between filtering in the frequency and spatial domains 3. Smoothing frequency domain filters 4. Sharpening frequency domain filters 5. Homomorphic filtering

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Filtering in the frequency domain Reading: 1:4.2

The Fourier transform of the output image (multiplication on an element-by-element basis : The filtered image is obtained by taking the inverse Fourier transform: Video Data Analysis, B-IT

Marina Kolesnik

G (u, v) = H (u, v) F (u, v)

g ( x, y ) = ℑ−1 [G (u , v)]

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Filtering in the frequency domain Reading: 1:4.2

Filtering in the frequency domain comprises the following steps: 1.

Multiply the input image by (-1)x+y to center the Fourier transform;

2.

Compute F(u,v), the DFT of the image from (1).

3.

Multiply F(u,v) by a filter function H(u,v).

4.

Compute the inverse DFT of the result in (3).

5.

Obtain the real part of the result in (4).

6.

Multiply the result in (5) by (-1)x+y.

Filters are usually real, whereas F is complex, therefore each component of H multiply both real and imaginary components of F. The inverse DFT of G should be real if f(x,y) and H are both real. In reality due to round-offs errors the inverse DFT contains parasitic imaginary components which are ignored. Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Basic filters Reading: 1:4.2

Notch filter is defined by setting F(0,0)=0:

⎧0 if (u , v) = ( M / 2, N / 2) H (u , v) = ⎨ ⎩1 otherwise Q: What is the average value of the processed image?

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Basic filters Reading: 1:4.2

Lowpass filter attenuates high frequencies while passing low frequencies.

Highpass filter has the opposite characteristic.

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Basic filters Reading: 1:4.2

By adding a constant to the Highpass filter, one increases the number of smooth gray-level details in the image ( because the term F(0,0) is not completely eliminated).

Image (left) filtered by the “highpass+constant” filter (right).

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

The link between filtering in the spatial and frequency domains Reading: 1:4.2

The link is established via the convolution theorem and substituting the pulse function instead of f(x,y):

ℑ[ f ( x, y ) ∗ h( x, y )] = F (u , v) H (u , v) (1) ℑ[ f ( x, y )h( x, y )] = F (u , v) ∗ H (u , v) The convolution of a function with an impulse copies the value of that function at the location of the impulse :

The sifting property of the impulse function

M

N

∑∑ s( x, y)δ ( x − x , y − y ) = s( x , y ) 0

x =0 y =0

0

0

0

The Fourier transform of the unit impulse at the origin is a real constant : 1 F(u,v) = MN

M

N

1

∑∑ δ ( x, y) exp[− j 2π (ux / M + vy / N )] = MN

( 2)

x =0 y =0

Let f ( x, y ) = δ ( x, y ), then the convolution : 1 f ( x, y ) ∗ h ( x, y ) = MN

M

N

1

∑∑ δ (m, n)h( x − m, y − n) = MN h( x, y)

(3)

x =0 y =0

By combining (1), (2) and (3), we obtain : ℑ[δ ( x, y ) ∗ h( x, y )] = ℑ[δ ( x, y )]H (u, v) 1 1 ℑ[h( x, y )] = H (u, v) ⇒ h( x, y ) ⇔ H (u , v) MN MN

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

The link between filtering in the spatial and frequency domains Reading: 1:4.2

Conclusion: Using the properties of the impulse function and the convolution theorem we have established that filters in the spatial and frequency domain constitute a Fourier transform pair: given a filter in the frequency domain, one can obtain the corresponding filter in the spatial domain by taking the inverse DFT of the filter. The reverse is true. The size of both filters are the same, but the spatial filter can be constructed in the smaller window. The process of filter construction: - Specify the filter in the frequency domain (more intuitive) - Take the inverse Fourier transform - Use the filter obtained as a guide for constructing a smaller spatial filter mask. Spatial filtering using a small mask is faster Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Filter construction

Lecture 6

Construction of filters based on a Gaussian function

Reading: 1:4.2

Both the forward and inverse DFT of a Gaussian function are real Gaussian functions: H (u ) = A exp[−u 2 / 2σ 2 ] h( x) = 2π σA exp[−2π 2σ 2 x 2 ]

For derivation see a handbook on mathematics of Erwin Kreyszig: “Advanced engineering mathematics”, p. 613.

These functions behave reciprocally: when H(u) has a broad profile (large σ) then h(x) has a narrow profile and vise versa. When :

σ →∞ then : H (u , v) → const; and

h ( x, y ) → δ ( x, y )

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Gaussian-based filters Reading: 1:4.2

Construction of a high-pass filter as a difference of two Gaussians: H (u ) = A exp[−u 2 / 2σ 12 ] − B exp[−u 2 / 2σ 22 ] with A ≥ B; σ 1 > σ 2 h( x) = 2π σ 1 A exp[−2π 2σ 12 x 2 ] − 2π σ 2 B exp[−2π 2σ 22 x 2 ]

Some enhancement tasks that would be exceptionally difficult or impossible to formulate directly in the spatial domain become almost trivial in the frequency domain. Once we have selected a specific filter via experimentation in the frequency domain, the actual implementation of the method is usually done in the spatial domain. Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Smoothing frequencydomain filters

Lecture 6

Reading: 1:4.3 Rule: - construct the filter in the frequency domain, which is much easier. - compute the spatial filter and perform the filtering in the spatial domain. Smoothing frequency-domain filters: The objective is to find H(u,v) that yields G(u,v) by G (u, v) = H (u , v) F (u , v) attenuating high frequency components of F(u,v):

Ideal lowpass filters:

Video Data Analysis, B-IT

Marina Kolesnik

⎧1 if D(u, v) ≤ D0 H (u, v) = ⎨ ⎩0 if D(u, v) > D0

[

D(u, v) = (u − M / 2) 2 + (v − N / 2) 2

]

1/ 2

http://www.fit.fraunhofer.de/~kolesnik/

Smoothing frequencydomain filters

Lecture 6

Reading: 1:4.3

D0 is a cut-off frequency Compute circles that enclose specified amount of total image power PT: M −1 N −1

PT = ∑∑ P(u , v) u =0 v =0

where P(u , v) is the Fourier power spectrum The circle with radius r encloses α percent of the power : ⎡

⎤

α = 100⎢∑∑ P(u, v) PT ⎥

⎣u v ⎦ where (u, v) lie inside the circle : D(u, v) < r

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Smoothing frequencydomain filters Reading: 1:4.3

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Ideal lowpass filters Lecture 6 Reading: 1:4.3

Why “rings” occur? G (u , v) = H (u, v) F (u , v) The corresponding blurring in the spatial domain : g ( x, y ) = h ( x, y ) ∗ f ( x, y ) h( x, y ) is obtained as : 1. H (u, v)(−1) u + v 2.Compute the inverse DFT of (1) 3.Multiply the real part of the inverse DFT in (2) by (−1) x + y

Diagonal cross-section through the image (c) Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Butterworth low-pass filters

Lecture 6

Reading: 1:4.3

H (u , v) =

1 2n 1 + [D(u , v) / D0 ]

A butterworth filter of order 1 has no ringing. Ringing is imperceptible in filters of order 2, and can become significant in filters of higher order.

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6 Reading: 1:4.3

Butterworth low-pass filtering results Results of applying the BLPF with n=2. No visible ringing occurs

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Gaussian low-pass filters

[

H (u, v) = exp − D 2 (u , v) / 2 D02

]

Reading: 1:4.3

No “rings” occur!

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Example of lowpass filtering

Filtering using the Gaussian lowpass filter with D0=80.

Printing applications: - Processing of a text with poor resolution to fill small gaps. - “cosmetic” processing to produce a smoother, “softerlooking” result.

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Sharpening by highpass filtering Reading: 1:4.4

Image sharpening is achieved by high-pass frequency domain filters, which attenuates lowfrequency components of the DFT and does not change their highfrequency components:

H hp (u , v) = 1 − H lp (u , v)

Top: Ideal highpass; Center: Butterworth highpass; Bottom: Gaussian highpass; Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Spatial representation of the high-pass filters Reading: 1:4.4

Spatial representation of a frequency domainfilter is obtained by : 1. multiplying H (u , v) by (−1)u + v 2. computing the inverse DFT 3.multiply the real part of the inverse DFT by (−1) x + y

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Highpass filters

Lecture 6

Reading: 1:4.4

Ideal Highpass ⎧0 if D(u , v) ≤ D0 H (u , v) = ⎨ ⎩1 if D(u , v) > D0

[

D(u , v) = (u − M / 2) 2 + (v − N / 2) 2

Butterworth highpass

H (u , v) =

]

1/ 2

1 2n 1 + [D0 / D (u , v)]

[

H (u, v) = 1 − exp − D 2 (u, v) / 2 D02

No ringing occurs for a Gaussian highpass filter.

Ringing is severe for an ideal highpass filter with low cutoff frequency value Video Data Analysis, B-IT

Gaussian highpass

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

]

The Laplacian in the frequency domain

Lecture 6

Reading: 1:4.4

It can be shown that :

For derivation see a handbook on mathematics of Erwin Kreyszig: “Advanced engineering 1 ℑ[ f ( x)] = mathematics”, p. 616. 2π

⎡ d n f ( x) ⎤ n ℑ⎢ ⎥ = ( ju ) ℑ[ f ( x)] ⎢⎣ dx ⎥⎦ From this it follows that :

[

∞

∫ f ( x )e

−iux

dx

−∞

]

⎡ ∂ 2 f ( x, y ) ∂ 2 f ( x, y ) ⎤ 2 2 2 2 ( ) ( , ) ( ) ( , ) ( ) F (u, v) = ju F u v + jv F u v = − u + v ℑ ∇ f ( x, y ) = ℑ⎢ + ⎥ 2 2 ∂y ⎢⎣ ∂x ⎥⎦ 2

H (u, v) = −(u 2 + v 2 ) After centering the filter function : H (u, v) = −[(u − M / 2) 2 + (v − N / 2) 2 ] ∇ 2 f ( x, y ) ⇔ −[(u − M / 2) 2 + (v − N / 2) 2 ]F (u, v)

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6 Reading: 1:4.4

The Laplacian in the frequency domain

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Homomorphic Filtering Reading: 1:4.5

f ( x, y ) = i ( x, y ) r ( x , y ) Because : ℑ{ f ( x, y )} ≠ ℑ{i ( x, y )}⋅ ℑ{r ( x, y )}

The image can be expressed as the product we use a logarithm : z(x,y) = ln( f ( x, y )) = ln(i ( x, y )) + ln(r ( x, y )) of illumination and reflectance components. ℑ{ln( f ( x, y ))} = ℑ{ln(i ( x, y ))}+ ℑ{ln(r ( x, y ))}

⇒ ⇒ Z (u , v) = Fi (u , v) + Fr (u , v)

After filtering using H(u,v) : S(u, v) = H (u , v) Z (u , v) = H (u , v) Fi (u , v) + H (u , v) Fr (u, v) In the spatial domain by letting : i ' ( x, y ) = ℑ−1{H (u , v) Fi (u , v)}

r ' ( x, y ) = ℑ−1{H (u , v) Fr (u , v)}

we obtain : s ( x , y ) = i ' ( x, y ) + r ' ( x , y ) As z ( x, y ) is the logarithm of the original image f(x, y), the inverse exponential operation gives the desired enhanced image g(x,y) : g ( x, y ) = e s ( x , y ) = e i '( x , y ) ⋅ e r '( x , y ) = i0 ( x, y ) ⋅ r0 ( x, y ) where i0 ( x, y ), r0 ( x, y ) are the illumination and reflectance components of the output image

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

Homomorphic Filtering Reading: 1:4.5

The homomorphic filtering approach:

The illumination component of an image is characterized by slow spatial variations. The reflectance component tends to vary abruptly, particularly at the functions of dissimilar objects. Therefore low frequency of the Fourier transform of the logarithm of an image are associated with illumination, and the high frequencies with reflectance. This is a rough but useful approximation.

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Homomorphic Filtering

Lecture 6

Reading: 1:4.5

The homomorphic filter tends to decrease the contribution made by the low frequencies (illumination) and amplify the contribution of high frequencies (reflectance). The result is simultaneous dynamic range compression and contrast enhancement. The homomorphic filter is given by:

[

H (u, v) = (γ H − γ L ) 1 − e − c⋅( D

2

( u ,v ) / D02 )

]+ γ

L

with

γ L < 1 and γ H > 1 c - controls the sharpness of the slope at the transition between γ L and γ H

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Homomorphic Filtering

Lecture 6

Reading: 1:4.5

Result of the homomorphic filtering with γl=0.5 and γh=2.0 reveals the details within the track originally hidden in the glare of the outside sun.

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Homomorphic Filtering

Lecture 6

Reading: 1:4.5

The original RGB color space is converted to YUV space. The homomorphic filtering is applied to Y only. Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/

Lecture 6

The combination does Reading: 1:4.4

High frequency emphasis filter is given by:

H hfe (u , v) = a + bH hp (u , v) for

a ≥ 0; b > a;

a b c d a: The original x-ray image of a chest; b: Result of high-pass filtering using a Butterworth filter of order 2 and value of D0 equal to 5% of the image vertical dimension; c: Result of high-emphasis filtering with a=0.5 and b=2.0 shows more gray-level tonality due to the low frequency components was not lost; d: Result of performing histogram equalization on the previous image.

Video Data Analysis, B-IT

Marina Kolesnik

http://www.fit.fraunhofer.de/~kolesnik/