Fast Fourier Transform: Why? How? Shamgar Gurevich UW Madison

May 13, 2012

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

1 / 26

Joint work with:

Ronny Hadani (Math, Austin)

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

2 / 26

Motivation: Discrete Fourier Transform 2πi 1 DFT = p e N τω N

Shamgar Gurevich (UW Madison)

0 τ,ω N 1

Fast Fourier Transform: Why? How?

May 13, 2012

3 / 26

Motivation: Discrete Fourier Transform 2πi 1 DFT = p e N τω N

0 τ,ω N 1

Compute:

b f = DFT [f ];

Shamgar Gurevich (UW Madison)

0

1 f (0) B C . B C C; . f =B B C @ A . f (N 1)

Fast Fourier Transform: Why? How?

Fast!!

May 13, 2012

3 / 26

Motivation: Discrete Fourier Transform 2πi 1 DFT = p e N τω N

0 τ,ω N 1

Compute:

b f = DFT [f ];

0

1 f (0) B C . B C C; . f =B B C @ A . f (N 1)

Fast!!

Cooley–Tukey (1965): O (N log(N )) operations! Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

3 / 26

Solution: (I) Heisenberg Group Representation

H = C(ZN ) — Hilbert space of digital signals.

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

4 / 26

Solution: (I) Heisenberg Group Representation

H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N

Shamgar Gurevich (UW Madison)

1g ! C.

Fast Fourier Transform: Why? How?

May 13, 2012

4 / 26

Solution: (I) Heisenberg Group Representation

H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N

1g ! C.

Basic operations

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

4 / 26

Solution: (I) Heisenberg Group Representation

H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N

1g ! C.

Basic operations Time shift:

τ 2 ZN ,

Lτ : H ! H , Lτ [f ](t ) = f (t + τ ), t 2 ZN .

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

4 / 26

Solution: (I) Heisenberg Group Representation

H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N

1g ! C.

Basic operations Time shift:

τ 2 ZN ,

Frequency shift:

Lτ : H ! H , Lτ [f ](t ) = f (t + τ ), t 2 ZN .

ω 2 ZN ,

Mω : H ! H , 2πi Mω [f ](t ) = e N ωt f (t ).

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

4 / 26

Solution: (I) Heisenberg Group Representation

H = C(ZN ) — Hilbert space of digital signals. 1g ! C.

f : f0, ...., N

Basic operations τ 2 ZN ,

Time shift:

Frequency shift:

Lτ : H ! H , Lτ [f ](t ) = f (t + τ ), t 2 ZN .

ω 2 ZN ,

Mω : H ! H , 2πi Mω [f ](t ) = e N ωt f (t ).

Note: Mω

Lτ = e

2πi N

ωτ

Shamgar Gurevich (UW Madison)







Heisenberg commutation relations

Fast Fourier Transform: Why? How?

May 13, 2012

4 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

Shamgar Gurevich (UW Madison)

2πi N

f 21 ωτ +z g

Fast Fourier Transform: Why? How?



Lτ .

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

2πi N

f 21 ωτ +z g



Lτ .

Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

1 τ ω ). 2 τ0 ω0

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

2πi N

f 21 ωτ +z g



Lτ .

Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +

1 τ ω ). 2 τ0 ω0

Question. How to think on this?

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

2πi N

f 21 ωτ +z g



Lτ .

Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +

1 τ ω ). 2 τ0 ω0

Question. How to think on this? Answer. Heisenberg group:

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

2πi N

f 21 ωτ +z g



Lτ .

Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +

1 τ ω ). 2 τ0 ω0

Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V

Shamgar Gurevich (UW Madison)

ZN ; |{z} Z

Fast Fourier Transform: Why? How?

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

2πi N

f 21 ωτ +z g



Lτ .

Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +

1 τ ω ). 2 τ0 ω0

Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V

(v , z )

(v 0 , z 0 )

Shamgar Gurevich (UW Madison)

ZN ; |{z} Z

= (v + v 0 , z + z 0 +

1 2

v ); v0

Fast Fourier Transform: Why? How?

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

2πi N

f 21 ωτ +z g



Lτ .

Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +

1 τ ω ). 2 τ0 ω0

Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V

ZN ; |{z} Z

v ); v0 (0, 0) (v , z ) = (v , z ) (0, 0) = (v , z );

(v , z )

(v 0 , z 0 )

Shamgar Gurevich (UW Madison)

= (v + v 0 , z + z 0 +

1 2

Fast Fourier Transform: Why? How?

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Combine:

τ, ω, z 2 ZN π (τ, ω, z ) = e

2πi N

f 21 ωτ +z g



Lτ .

Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +

1 τ ω ). 2 τ0 ω0

Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V

ZN ; |{z} Z

v ); v0 (0, 0) (v , z ) = (v , z ) (0, 0) = (v , z ); (v , z ) ( v , z ) = ( v , z ) (v , z ) = (0, 0).

(v , z )

(v 0 , z 0 )

Shamgar Gurevich (UW Madison)

= (v + v 0 , z + z 0 +

1 2

Fast Fourier Transform: Why? How?

May 13, 2012

5 / 26

Heisenberg Rep’n, cont. Summary: Heisenberg Rep’n 8 π : H ! GL(H); < 2πi 1 π (τ, ω, z ) = e N f 2 ωτ +z g Mω Lτ ; : π (h h0 ) = π (h ) π (h0 ) — homomorphism.

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

6 / 26

Heisenberg Rep’n, cont. Summary: Heisenberg Rep’n 8 π : H ! GL(H); < 2πi 1 π (τ, ω, z ) = e N f 2 ωτ +z g Mω Lτ ; : π (h h0 ) = π (h ) π (h0 ) — homomorphism.

De…nition A representation of a group H on a complex vector space H is a homomorphism π 0

:

H ! GL(H),

π (h h ) = π (h ) π (h 0 ).

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

6 / 26

Heisenberg Rep’n, cont. Summary: Heisenberg Rep’n 8 π : H ! GL(H); < 2πi 1 π (τ, ω, z ) = e N f 2 ωτ +z g Mω Lτ ; : π (h h0 ) = π (h ) π (h0 ) — homomorphism.

De…nition A representation of a group H on a complex vector space H is a homomorphism π 0

:

H ! GL(H),

π (h h ) = π (h ) π (h 0 ). Question. DFT ? Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

6 / 26

(II) Representation Theory De…nitions We say that (π 1 , H, H1 ), (π 2 , H, H2 ) are equivalent, π 1 ' π 2 , if

9 α : H1 ! f H2

Intertwiner,

such that for every h 2 H

H1 ? ?α y H2

π 1 (h )

! H1 ? ?α y

π 2 (h )

! H2

i.e., α π 1 (h ) = π 2 (h ) α.

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

7 / 26

(II) Representation Theory De…nitions We say that (π 1 , H, H1 ), (π 2 , H, H2 ) are equivalent, π 1 ' π 2 , if

9 α : H1 ! f H2

Intertwiner,

such that for every h 2 H

H1 ? ?α y H2

π 1 (h )

! H1 ? ?α y

π 2 (h )

! H2

i.e., α π 1 (h ) = π 2 (h ) α.

Example: DFT is an intertwiner! Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

7 / 26

Rep’n Theory, cont. H=V

Z = ZN |{z} T

Shamgar Gurevich (UW Madison)

ZN |{z} W

ZN |{z} Z

Fast Fourier Transform: Why? How?

May 13, 2012

8 / 26

Rep’n Theory, cont. H=V

Z = ZN |{z} T

Time representation:

ZN |{z} W

ZN |{z} Z

HT = C(T )

π T : H ! GL(HT ); 2πi 1 π T (τ, ω, z ) = e N f 2 ωτ +z g Mω

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

Lτ .

May 13, 2012

8 / 26

Rep’n Theory, cont. H=V

Z = ZN |{z} T

Time representation:

ZN |{z} W

ZN |{z} Z

HT = C(T )

π T : H ! GL(HT ); 2πi 1 π T (τ, ω, z ) = e N f 2 ωτ +z g Mω Frequency representation:

HW = C(W )

π W : H ! GL(HW ); 2πi 1 π W (τ, ω, z ) = e N f 2 ωτ +z g Mτ

Shamgar Gurevich (UW Madison)

Lτ .

Fast Fourier Transform: Why? How?

L

ω.

May 13, 2012

8 / 26

Rep’n Theory, cont. H=V

Z = ZN |{z} T

Time representation:

ZN |{z} W

ZN |{z} Z

HT = C(T )

π T : H ! GL(HT ); 2πi 1 π T (τ, ω, z ) = e N f 2 ωτ +z g Mω Frequency representation:

Lτ .

HW = C(W )

π W : H ! GL(HW ); 2πi 1 π W (τ, ω, z ) = e N f 2 ωτ +z g Mτ

L

ω.

Fact: DFT where

Shamgar Gurevich (UW Madison)

π T (h ) = π W (h ) DFT ,

1 DFT [f ](w ) = p N



f (t )e

2πi N

wt

.

t 2 ZN

Fast Fourier Transform: Why? How?

May 13, 2012

8 / 26

(III) FFT Algorithm Idea:

HT ? ? slow yDFT HW

Shamgar Gurevich (UW Madison)

fast

! HΛ

fast



Fast Fourier Transform: Why? How?

May 13, 2012

9 / 26

(III) FFT Algorithm Idea:

HT ? ? slow yDFT HW

fast

! HΛ

fast



More representation theory:

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

9 / 26

(III) FFT Algorithm Idea:

HT ? ? slow yDFT HW

fast

! HΛ

fast



More representation theory:

De…nition A rep’n (π, H, H) is irreducible if @ 0 6= H0

H

such that π (h ) H 0 Shamgar Gurevich (UW Madison)

H0 ,

8h 2 H.

Fast Fourier Transform: Why? How?

May 13, 2012

9 / 26

FFT Algorithm: more RT Theorem (Stone–von Neumann) If (π j , H, Hj ), j = 1, 2, are irreducible representations of the Heisenberg group H = V Z , with π j (z ) = e

2πi N

z

IdHj , 8z 2 Z ,

then π 1 ' π 2 are equivalent, i.e., 9 α : H1 ! f H2 such that α π 1 (h ) = π 2 (h ) α, 8h 2 H.

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

(1)

May 13, 2012

10 / 26

FFT Algorithm: more RT Theorem (Stone–von Neumann) If (π j , H, Hj ), j = 1, 2, are irreducible representations of the Heisenberg group H = V Z , with π j (z ) = e

2πi N

z

IdHj , 8z 2 Z ,

then π 1 ' π 2 are equivalent, i.e., 9 α : H1 ! f H2 such that α π 1 (h ) = π 2 (h ) α, 8h 2 H.

(1)

Theorem (Schur’s lemma) If π 1 ' π 2 equivalent irreducible representations, and if α, α0 satisfy equation (1), then α = c α0 , for some scalar c. Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

10 / 26

FFT Algorithm: Models of Heisenberg Rep’n

Examples (1) Time model: V

T = f(t, 0); t 2 ZN g π T : H ! GL(HT ).

(2) Frequency model: V

W = f(0, w ); w 2 ZN g π W : H ! GL(HW ).

Corollary DFT : HT ! HW — unique (up to a scalar) intertwiner between π T and πW .

Shamgar Gurevich (UW Madison)

Fast Fourier Transform: Why? How?

May 13, 2012

11 / 26

FFT Algorithm: Models of Heisenberg Rep’n Examples (3) W –invariant model: Space: 8 f : H ! C, < W f (w h ) = f (h ), 8 w 2 W , h 2 H, H = : 2πi f (z h ) = e N z f (h ), 8 z 2 Z , h 2 H.

Action:

π W : H ! GL(HW ), [π W (h0 )f ](h) = f (h h0 ).

Remark: We have a natural identi…cation HT = HW , given by

Shamgar Gurevich (UW Madison)

2πi f 7! e f (wtz ) = e N z f (t ). Fast Fourier Transform: Why? How?

May 13, 2012

12 / 26

Think on W-invariant functions as functions on T A function f (t ) on T = f(t, 0, 0); t 2 Z/52 g

0 .2

0 .1 5

0 .1

0 .0 5

0

-0 .0 5

-0 .1

-0 .1 5

-0 .2 25 20

25 20

15 15

10 10 5

5 0

0

W

Shamgar Gurevich (UW Madison)

T

Fast Fourier Transform: Why? How?

May 13, 2012

13 / 26

FFT Algorithm: Models of Heisenberg Rep’n Examples (4) T –invariant model: Space: 8 g : H ! C, < T g (t h ) = g (h ), 8 t 2 T , h 2 H, H = : 2πi g (z h ) = e N z g (h ), 8 z 2 Z , h 2 H. Action:

π T : H ! GL(HT ), = g (h h 0 ).

[π T (h0 )g ](h)

Remark: We have a natural identi…cation HW = HT , given by

Shamgar Gurevich (UW Madison)

g 7! ge (twz ) = e

2πi N

z

g (w ).

Fast Fourier Transform: Why? How?

May 13, 2012

14 / 26

Think on T-invariant functions as functions on W A function g (w ) on W = f(0, w , 0); w 2 Z/52 g

0 .2 0 .1 5 0 .1 0 .0 5 0 -0 .0 5 -0 .1 -0 .1 5 -0 .2 25

25 20

20 15

15 10

10 5

W

Shamgar Gurevich (UW Madison)

5 0

0 T

Fast Fourier Transform: Why? How?

May 13, 2012

15 / 26

FFT Algorithm: The Arithmetic Model Examples (5) Arithmetic model: N = p 2 Lagrangian: V = Z/p 2

Z/p 2

Λ = f(p a, p b )g

Space:

HΛ = Action:

8