Fast Fourier Transform: Why? How? Shamgar Gurevich UW Madison
May 13, 2012
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
1 / 26
Joint work with:
Ronny Hadani (Math, Austin)
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
2 / 26
Motivation: Discrete Fourier Transform 2πi 1 DFT = p e N τω N
Shamgar Gurevich (UW Madison)
0 τ,ω N 1
Fast Fourier Transform: Why? How?
May 13, 2012
3 / 26
Motivation: Discrete Fourier Transform 2πi 1 DFT = p e N τω N
0 τ,ω N 1
Compute:
b f = DFT [f ];
Shamgar Gurevich (UW Madison)
0
1 f (0) B C . B C C; . f =B B C @ A . f (N 1)
Fast Fourier Transform: Why? How?
Fast!!
May 13, 2012
3 / 26
Motivation: Discrete Fourier Transform 2πi 1 DFT = p e N τω N
0 τ,ω N 1
Compute:
b f = DFT [f ];
0
1 f (0) B C . B C C; . f =B B C @ A . f (N 1)
Fast!!
Cooley–Tukey (1965): O (N log(N )) operations! Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
3 / 26
Solution: (I) Heisenberg Group Representation
H = C(ZN ) — Hilbert space of digital signals.
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
4 / 26
Solution: (I) Heisenberg Group Representation
H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N
Shamgar Gurevich (UW Madison)
1g ! C.
Fast Fourier Transform: Why? How?
May 13, 2012
4 / 26
Solution: (I) Heisenberg Group Representation
H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N
1g ! C.
Basic operations
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
4 / 26
Solution: (I) Heisenberg Group Representation
H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N
1g ! C.
Basic operations Time shift:
τ 2 ZN ,
Lτ : H ! H , Lτ [f ](t ) = f (t + τ ), t 2 ZN .
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
4 / 26
Solution: (I) Heisenberg Group Representation
H = C(ZN ) — Hilbert space of digital signals. f : f0, ...., N
1g ! C.
Basic operations Time shift:
τ 2 ZN ,
Frequency shift:
Lτ : H ! H , Lτ [f ](t ) = f (t + τ ), t 2 ZN .
ω 2 ZN ,
Mω : H ! H , 2πi Mω [f ](t ) = e N ωt f (t ).
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
4 / 26
Solution: (I) Heisenberg Group Representation
H = C(ZN ) — Hilbert space of digital signals. 1g ! C.
f : f0, ...., N
Basic operations τ 2 ZN ,
Time shift:
Frequency shift:
Lτ : H ! H , Lτ [f ](t ) = f (t + τ ), t 2 ZN .
ω 2 ZN ,
Mω : H ! H , 2πi Mω [f ](t ) = e N ωt f (t ).
Note: Mω
Lτ = e
2πi N
ωτ
Shamgar Gurevich (UW Madison)
Lτ
Mω
—
Heisenberg commutation relations
Fast Fourier Transform: Why? How?
May 13, 2012
4 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
Shamgar Gurevich (UW Madison)
2πi N
f 21 ωτ +z g
Fast Fourier Transform: Why? How?
Mω
Lτ .
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
2πi N
f 21 ωτ +z g
Mω
Lτ .
Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
1 τ ω ). 2 τ0 ω0
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
2πi N
f 21 ωτ +z g
Mω
Lτ .
Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +
1 τ ω ). 2 τ0 ω0
Question. How to think on this?
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
2πi N
f 21 ωτ +z g
Mω
Lτ .
Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +
1 τ ω ). 2 τ0 ω0
Question. How to think on this? Answer. Heisenberg group:
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
2πi N
f 21 ωτ +z g
Mω
Lτ .
Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +
1 τ ω ). 2 τ0 ω0
Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V
Shamgar Gurevich (UW Madison)
ZN ; |{z} Z
Fast Fourier Transform: Why? How?
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
2πi N
f 21 ωτ +z g
Mω
Lτ .
Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +
1 τ ω ). 2 τ0 ω0
Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V
(v , z )
(v 0 , z 0 )
Shamgar Gurevich (UW Madison)
ZN ; |{z} Z
= (v + v 0 , z + z 0 +
1 2
v ); v0
Fast Fourier Transform: Why? How?
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
2πi N
f 21 ωτ +z g
Mω
Lτ .
Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +
1 τ ω ). 2 τ0 ω0
Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V
ZN ; |{z} Z
v ); v0 (0, 0) (v , z ) = (v , z ) (0, 0) = (v , z );
(v , z )
(v 0 , z 0 )
Shamgar Gurevich (UW Madison)
= (v + v 0 , z + z 0 +
1 2
Fast Fourier Transform: Why? How?
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Combine:
τ, ω, z 2 ZN π (τ, ω, z ) = e
2πi N
f 21 ωτ +z g
Mω
Lτ .
Identity: π (τ, ω, z ) π (τ 0 , ω 0 , z 0 ) = π (τ + τ 0 , ω + ω 0 , z + z 0 +
1 τ ω ). 2 τ0 ω0
Question. How to think on this? Answer. Heisenberg group: H = ZN ZN | {z } V
ZN ; |{z} Z
v ); v0 (0, 0) (v , z ) = (v , z ) (0, 0) = (v , z ); (v , z ) ( v , z ) = ( v , z ) (v , z ) = (0, 0).
(v , z )
(v 0 , z 0 )
Shamgar Gurevich (UW Madison)
= (v + v 0 , z + z 0 +
1 2
Fast Fourier Transform: Why? How?
May 13, 2012
5 / 26
Heisenberg Rep’n, cont. Summary: Heisenberg Rep’n 8 π : H ! GL(H); < 2πi 1 π (τ, ω, z ) = e N f 2 ωτ +z g Mω Lτ ; : π (h h0 ) = π (h ) π (h0 ) — homomorphism.
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
6 / 26
Heisenberg Rep’n, cont. Summary: Heisenberg Rep’n 8 π : H ! GL(H); < 2πi 1 π (τ, ω, z ) = e N f 2 ωτ +z g Mω Lτ ; : π (h h0 ) = π (h ) π (h0 ) — homomorphism.
De…nition A representation of a group H on a complex vector space H is a homomorphism π 0
:
H ! GL(H),
π (h h ) = π (h ) π (h 0 ).
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
6 / 26
Heisenberg Rep’n, cont. Summary: Heisenberg Rep’n 8 π : H ! GL(H); < 2πi 1 π (τ, ω, z ) = e N f 2 ωτ +z g Mω Lτ ; : π (h h0 ) = π (h ) π (h0 ) — homomorphism.
De…nition A representation of a group H on a complex vector space H is a homomorphism π 0
:
H ! GL(H),
π (h h ) = π (h ) π (h 0 ). Question. DFT ? Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
6 / 26
(II) Representation Theory De…nitions We say that (π 1 , H, H1 ), (π 2 , H, H2 ) are equivalent, π 1 ' π 2 , if
9 α : H1 ! f H2
Intertwiner,
such that for every h 2 H
H1 ? ?α y H2
π 1 (h )
! H1 ? ?α y
π 2 (h )
! H2
i.e., α π 1 (h ) = π 2 (h ) α.
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
7 / 26
(II) Representation Theory De…nitions We say that (π 1 , H, H1 ), (π 2 , H, H2 ) are equivalent, π 1 ' π 2 , if
9 α : H1 ! f H2
Intertwiner,
such that for every h 2 H
H1 ? ?α y H2
π 1 (h )
! H1 ? ?α y
π 2 (h )
! H2
i.e., α π 1 (h ) = π 2 (h ) α.
Example: DFT is an intertwiner! Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
7 / 26
Rep’n Theory, cont. H=V
Z = ZN |{z} T
Shamgar Gurevich (UW Madison)
ZN |{z} W
ZN |{z} Z
Fast Fourier Transform: Why? How?
May 13, 2012
8 / 26
Rep’n Theory, cont. H=V
Z = ZN |{z} T
Time representation:
ZN |{z} W
ZN |{z} Z
HT = C(T )
π T : H ! GL(HT ); 2πi 1 π T (τ, ω, z ) = e N f 2 ωτ +z g Mω
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
Lτ .
May 13, 2012
8 / 26
Rep’n Theory, cont. H=V
Z = ZN |{z} T
Time representation:
ZN |{z} W
ZN |{z} Z
HT = C(T )
π T : H ! GL(HT ); 2πi 1 π T (τ, ω, z ) = e N f 2 ωτ +z g Mω Frequency representation:
HW = C(W )
π W : H ! GL(HW ); 2πi 1 π W (τ, ω, z ) = e N f 2 ωτ +z g Mτ
Shamgar Gurevich (UW Madison)
Lτ .
Fast Fourier Transform: Why? How?
L
ω.
May 13, 2012
8 / 26
Rep’n Theory, cont. H=V
Z = ZN |{z} T
Time representation:
ZN |{z} W
ZN |{z} Z
HT = C(T )
π T : H ! GL(HT ); 2πi 1 π T (τ, ω, z ) = e N f 2 ωτ +z g Mω Frequency representation:
Lτ .
HW = C(W )
π W : H ! GL(HW ); 2πi 1 π W (τ, ω, z ) = e N f 2 ωτ +z g Mτ
L
ω.
Fact: DFT where
Shamgar Gurevich (UW Madison)
π T (h ) = π W (h ) DFT ,
1 DFT [f ](w ) = p N
∑
f (t )e
2πi N
wt
.
t 2 ZN
Fast Fourier Transform: Why? How?
May 13, 2012
8 / 26
(III) FFT Algorithm Idea:
HT ? ? slow yDFT HW
Shamgar Gurevich (UW Madison)
fast
! HΛ
fast
HΛ
Fast Fourier Transform: Why? How?
May 13, 2012
9 / 26
(III) FFT Algorithm Idea:
HT ? ? slow yDFT HW
fast
! HΛ
fast
HΛ
More representation theory:
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
9 / 26
(III) FFT Algorithm Idea:
HT ? ? slow yDFT HW
fast
! HΛ
fast
HΛ
More representation theory:
De…nition A rep’n (π, H, H) is irreducible if @ 0 6= H0
H
such that π (h ) H 0 Shamgar Gurevich (UW Madison)
H0 ,
8h 2 H.
Fast Fourier Transform: Why? How?
May 13, 2012
9 / 26
FFT Algorithm: more RT Theorem (Stone–von Neumann) If (π j , H, Hj ), j = 1, 2, are irreducible representations of the Heisenberg group H = V Z , with π j (z ) = e
2πi N
z
IdHj , 8z 2 Z ,
then π 1 ' π 2 are equivalent, i.e., 9 α : H1 ! f H2 such that α π 1 (h ) = π 2 (h ) α, 8h 2 H.
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
(1)
May 13, 2012
10 / 26
FFT Algorithm: more RT Theorem (Stone–von Neumann) If (π j , H, Hj ), j = 1, 2, are irreducible representations of the Heisenberg group H = V Z , with π j (z ) = e
2πi N
z
IdHj , 8z 2 Z ,
then π 1 ' π 2 are equivalent, i.e., 9 α : H1 ! f H2 such that α π 1 (h ) = π 2 (h ) α, 8h 2 H.
(1)
Theorem (Schur’s lemma) If π 1 ' π 2 equivalent irreducible representations, and if α, α0 satisfy equation (1), then α = c α0 , for some scalar c. Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
10 / 26
FFT Algorithm: Models of Heisenberg Rep’n
Examples (1) Time model: V
T = f(t, 0); t 2 ZN g π T : H ! GL(HT ).
(2) Frequency model: V
W = f(0, w ); w 2 ZN g π W : H ! GL(HW ).
Corollary DFT : HT ! HW — unique (up to a scalar) intertwiner between π T and πW .
Shamgar Gurevich (UW Madison)
Fast Fourier Transform: Why? How?
May 13, 2012
11 / 26
FFT Algorithm: Models of Heisenberg Rep’n Examples (3) W –invariant model: Space: 8 f : H ! C, < W f (w h ) = f (h ), 8 w 2 W , h 2 H, H = : 2πi f (z h ) = e N z f (h ), 8 z 2 Z , h 2 H.
Action:
π W : H ! GL(HW ), [π W (h0 )f ](h) = f (h h0 ).
Remark: We have a natural identi…cation HT = HW , given by
Shamgar Gurevich (UW Madison)
2πi f 7! e f (wtz ) = e N z f (t ). Fast Fourier Transform: Why? How?
May 13, 2012
12 / 26
Think on W-invariant functions as functions on T A function f (t ) on T = f(t, 0, 0); t 2 Z/52 g
0 .2
0 .1 5
0 .1
0 .0 5
0
-0 .0 5
-0 .1
-0 .1 5
-0 .2 25 20
25 20
15 15
10 10 5
5 0
0
W
Shamgar Gurevich (UW Madison)
T
Fast Fourier Transform: Why? How?
May 13, 2012
13 / 26
FFT Algorithm: Models of Heisenberg Rep’n Examples (4) T –invariant model: Space: 8 g : H ! C, < T g (t h ) = g (h ), 8 t 2 T , h 2 H, H = : 2πi g (z h ) = e N z g (h ), 8 z 2 Z , h 2 H. Action:
π T : H ! GL(HT ), = g (h h 0 ).
[π T (h0 )g ](h)
Remark: We have a natural identi…cation HW = HT , given by
Shamgar Gurevich (UW Madison)
g 7! ge (twz ) = e
2πi N
z
g (w ).
Fast Fourier Transform: Why? How?
May 13, 2012
14 / 26
Think on T-invariant functions as functions on W A function g (w ) on W = f(0, w , 0); w 2 Z/52 g
0 .2 0 .1 5 0 .1 0 .0 5 0 -0 .0 5 -0 .1 -0 .1 5 -0 .2 25
25 20
20 15
15 10
10 5
W
Shamgar Gurevich (UW Madison)
5 0
0 T
Fast Fourier Transform: Why? How?
May 13, 2012
15 / 26
FFT Algorithm: The Arithmetic Model Examples (5) Arithmetic model: N = p 2 Lagrangian: V = Z/p 2
Z/p 2
Λ = f(p a, p b )g
Space:
HΛ = Action:
8