Partial functional correspondence Michael Bronstein

University of Lugano

Intel Corporation

Lyon, 7 July 2016 1/59

Microsoft Kinect 2010

(Acquired by Intel in 2012)

4/59

Different form factor computers featuring Intel RealSense 3D camera 7/59

Deluge of geometric data

3D sensors

Repositories

3D printers

8/59

Applications

Deformable fusion

Motion capture

Motion transfer

Texture mapping

Dou et al. 2015; Sumner, Popovi´ c 2004; Faceshift; Cow image: Moore 2014 9/59

Shape correspondence problem

Isometric

10/59

Shape correspondence problem

Isometric

Partial

10/59

Shape correspondence problem

Isometric

Partial

Different representation 10/59

Shape correspondence problem

Isometric

Different representation

Partial

Non-isometric 10/59

Computer Graphics Forum SGP 2016

Computer Graphics Forum SGP 2016 Best paper award

11/59

Outline

Background: Spectral analysis on manifolds Functional correspondence Partial functional correspondence Non-rigid puzzles

12/59

Riemannian geometry in one minute Tangent plane Tm M = local Euclidean representation of manifold (surface) M around m

Tm M m

M

13/59

Riemannian geometry in one minute Tangent plane Tm M = local Euclidean representation of manifold (surface) M around m

Tm M m m0

Riemannian metric h·, ·iTm M : Tm M × Tm M → R

Tm0 M

depending smoothly on m

13/59

Riemannian geometry in one minute Tangent plane Tm M = local Euclidean representation of manifold (surface) M around m

Tm M m m0

Riemannian metric h·, ·iTm M : Tm M × Tm M → R

Tm0 M

depending smoothly on m Isometry = metric-preserving shape deformation

13/59

Riemannian geometry in one minute Tangent plane Tm M = local Euclidean representation of manifold (surface) M around m

Tm M m

v

expm (v)

Riemannian metric h·, ·iTm M : Tm M × Tm M → R depending smoothly on m Isometry = metric-preserving shape deformation Exponential map expm : Tm M → M ‘unit step along geodesic’

13/59

Laplace-Beltrami operator m

f

Smooth field f : M → R

14/59

Laplace-Beltrami operator m f ◦ expm

f

Smooth field f ◦ expm : Tm M → R

14/59

Laplace-Beltrami operator Intrinsic gradient

m f ◦ expm

∇M f (m) = ∇(f ◦ expm )(0) Taylor expansion f

(f ◦ expm )(v) ≈ f (m) + h∇M f (m), viTm M

14/59

Laplace-Beltrami operator Intrinsic gradient

m f ◦ expm

∇M f (m) = ∇(f ◦ expm )(0) Taylor expansion f

(f ◦ expm )(v) ≈ f (m) + h∇M f (m), viTm M Laplace-Beltrami operator ∆M f (m) = ∆(f ◦ expm )(0)

14/59

Laplace-Beltrami operator Intrinsic gradient

m f ◦ expm

∇M f (m) = ∇(f ◦ expm )(0) Taylor expansion f

(f ◦ expm )(v) ≈ f (m) + h∇M f (m), viTm M Laplace-Beltrami operator ∆M f (m) = ∆(f ◦ expm )(0) Intrinsic (expressed solely in terms of the Riemannian metric)

14/59

Laplace-Beltrami operator Intrinsic gradient

m f ◦ expm

∇M f (m) = ∇(f ◦ expm )(0) Taylor expansion f

(f ◦ expm )(v) ≈ f (m) + h∇M f (m), viTm M Laplace-Beltrami operator ∆M f (m) = ∆(f ◦ expm )(0) Intrinsic (expressed solely in terms of the Riemannian metric) Isometry-invariant

14/59

Laplace-Beltrami operator Intrinsic gradient

m f ◦ expm

∇M f (m) = ∇(f ◦ expm )(0) Taylor expansion f

(f ◦ expm )(v) ≈ f (m) + h∇M f (m), viTm M Laplace-Beltrami operator ∆M f (m) = ∆(f ◦ expm )(0) Intrinsic (expressed solely in terms of the Riemannian metric) Isometry-invariant Self-adjoint h∆M f, giL2 (M)=hf, ∆M giL2 (M)

14/59

Laplace-Beltrami operator Intrinsic gradient

m f ◦ expm

∇M f (m) = ∇(f ◦ expm )(0) Taylor expansion f

(f ◦ expm )(v) ≈ f (m) + h∇M f (m), viTm M Laplace-Beltrami operator ∆M f (m) = ∆(f ◦ expm )(0) Intrinsic (expressed solely in terms of the Riemannian metric) Isometry-invariant Self-adjoint h∆M f, giL2 (M)=hf, ∆M giL2 (M) ⇒ orthogonal eigenfunctions

14/59

Laplace-Beltrami operator Intrinsic gradient

m f ◦ expm

∇M f (m) = ∇(f ◦ expm )(0) Taylor expansion f

(f ◦ expm )(v) ≈ f (m) + h∇M f (m), viTm M Laplace-Beltrami operator ∆M f (m) = ∆(f ◦ expm )(0) Intrinsic (expressed solely in terms of the Riemannian metric) Isometry-invariant Self-adjoint h∆M f, giL2 (M)=hf, ∆M giL2 (M) ⇒ orthogonal eigenfunctions Positive semidefinite ⇒ non-negative eigenvalues 14/59

Discrete Laplacian j wij

αij βij i ai

i

Undirected graph (V, E) X (∆f )i ≈ wij (fi − fj )

αij

Triangular mesh (V, E, F ) (∆f )i ≈

(i,j)∈E

wij =

1 X wij (fi − fj ) ai (i,j)∈E  cot αij +cot βij (i, j) ∈ Ei  2  1 2

  −

cot P αij k6=i wik

0

(i, j) ∈ Eb i=j else

ai = local area element Tutte 1963; MacNeal 1949; Duffin 1959; Pinkall, Polthier 1993 15/59

Fourier analysis (Euclidean spaces) A function f : [−π, π] → R can be written as Fourier series X 1 Z π f (x) = f (ξ)eiωξ dξ e−iωx 2π −π ω

= α1

+ α2

+ α3

+...

16/59

Fourier analysis (Euclidean spaces) A function f : [−π, π] → R can be written as Fourier series X 1 Z π f (x) = f (ξ)eiωξ dξ e−iωx 2π −π ω | {z } fˆ(ω)=hf,e−iωx iL2 ([−π,π])

= α1

+ α2

+ α3

+...

16/59

Fourier analysis (Euclidean spaces) A function f : [−π, π] → R can be written as Fourier series X 1 Z π f (x) = f (ξ)eiωξ dξ e−iωx 2π −π ω | {z } fˆ(ω)=hf,e−iωx iL2 ([−π,π])

= α1

+ α2

+ α3

+...

Fourier basis = Laplacian eigenfunctions: ∆e−iωx = ω 2 e−iωx

16/59

Fourier analysis (non-Euclidean spaces) A function f : M → R can be written as Fourier series XZ f (m) = f (ξ)φk (ξ)dξ φk (m) k≥1 | M {z } fˆk =hf,φk iL2 (M)

=

f

α1

+

φ1

α2

+

φ2

α3

+ ...

φ3

Fourier basis = Laplacian eigenfunctions: ∆M φk = λk φk

17/59

Outline

Background: Spectral analysis on manifolds Functional correspondence Partial functional correspondence Non-rigid puzzles

18/59

Point-wise correspondence

t

n

m

M

N

Point-wise maps t : M → N

19/59

Functional correspondence

g f F (M)

F (N ) T

Functional maps T : F(M) → F(N )

Ovsjanikov et al. 2012 19/59

Functional correspondence

f

↓ T ↓

g

Ovsjanikov et al. 2012 20/59

Functional correspondence

f

≈ a1

+ a2

+ · · · + ak

≈ b1

+ b2

+ · · · + bk

↓ T ↓

g

Ovsjanikov et al. 2012 20/59

Functional correspondence

≈ a1

f

g

↓

↓

T ↓

C> ↓

≈ b1

+ a2

+ · · · + ak

Translates Fourier coefficients from Φ to Ψ

+ b2

+ · · · + bk

Ovsjanikov et al. 2012 20/59

Functional correspondence

≈ a1

f

↓ T ↓

g

+ a2

+ · · · + ak

↓ ≈

Ψk C>Φ> k ↓

≈ b1

Translates Fourier coefficients from Φ to Ψ

+ b2

+ · · · + bk

g> Ψk = f > Φk C where Φk = (φ1 , . . . , φk ), Ψk = (ψ 1 , . . . , ψ k ) are truncated Laplace-Beltrami eigenbases Ovsjanikov et al. 2012 20/59

Functional correspondence in Laplacian eigenbases

For isometric simple spectrum shapes C is diagonal since ψ i = ±Tφi 21/59

Computing functional correspondence

Ovsjanikov et al. 2012 22/59

Computing functional correspondence

f1

f2

···

fq

g1

g2

···

gq

Given ordered set of functions f 1 , . . . , f q on M and corresponding functions g1 , . . . , gq on N (gi ≈ Tf i )

Ovsjanikov et al. 2012 22/59

Computing functional correspondence

f1

f2

···

fq

g1

g2

···

gq

Given ordered set of functions f 1 , . . . , f q on M and corresponding functions g1 , . . . , gq on N (gi ≈ Tf i ) C found by solving a system of qk equations with k 2 variables G> Ψk = F> Φk C where F = (f 1 , . . . , f q ) and G = (g1 , . . . , gq ) are n × q and m × q matrices Ovsjanikov et al. 2012 22/59

Key issues

How to recover point-wise correspondence with some guarantees (e.g. bijectivity)? How to automatically find corresponding functions F, G? Near isometric shapes: easy (a lot of structure!) Non-isometric shapes: hard Does not work well in case of missing parts and topological noise

23/59

Partial Laplacian eigenvectors

ζ2

ζ3

ζ4

ζ5

ζ6

ζ7

ζ8

ζ9

ψ2

ψ3

ψ4

ψ5

ψ6

ψ7

ψ8

ψ9

φ2

φ3

φ4

φ5

φ6

φ7

φ8

φ9

Laplacian eigenvectors of a shape with missing parts (Neumann boundary conditions)

Rodol` a, Cosmo, B, Torsello, Cremers 2016 24/59

Partial Laplacian eigenvectors

Functional correspondence matrix C

Rodol` a, Cosmo, B, Torsello, Cremers 2016 25/59

Perturbation analysis: intuition

∆M

φ1

∆M

φ1

φ2

φ3

M ¯ M

∆M ¯

φ2

φ¯1

φ3

φ¯2

φ¯3

Ignoring boundary interaction: disjoint parts (block-diagonal matrix) Eigenvectors = Mixture of eigenvectors of the parts Rodol` a, Cosmo, B, Torsello, Cremers 2016 26/59

Perturbation analysis: eigenvalues 8.00

·10−2 M

6.00

4.00 r

k

2.00

0.00

10

20

30

40

N

50

eigenvalue number

Slope

r k

≈

|M| |N |

(depends on the area of the cut)

Consistent with Weil’s law Rodol` a, Cosmo, B, Torsello, Cremers 2016 27/59

Perturbation analysis: details

∆M M ¯ M

∆M+tDM

tE>

tE

∆M ¯ +tDM ¯

∆M ¯

a, Cosmo, B, Torsello, Cremers 2016 Rodol` 28/59

Perturbation analysis: details

∆M M ¯ M

∆M+tDM

tE>

tE

∆M ¯ +tDM ¯

∆M ¯

“How would the Laplacian eigenvalues and eigenvectors of the red part change if we attached a blue part to it?” Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59

Perturbation analysis: details

M ¯ M

PM n×n

P n×n ¯ DM

E

“How would the Laplacian eigenvalues and eigenvectors of the red part change if we attached a blue part to it?” Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59

Perturbation analysis: details ¯Λ ¯Φ ¯ > , Φ = Φ(0), and Denote ∆M + tPM = Φ(t)Λ(t)Φ(t)> , ∆M ¯ =Φ Λ = Λ(0). Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues is given by   d 0 0 > λi = φi PM φi PM = 0 DM dt

Rodol` a, Cosmo, B, Torsello, Cremers 2016 29/59

Perturbation analysis: details ¯Λ ¯Φ ¯ > , Φ = Φ(0), and Denote ∆M + tPM = Φ(t)Λ(t)Φ(t)> , ∆M ¯ =Φ Λ = Λ(0). Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues is given by   d 0 0 > λi = φi PM φi PM = 0 DM dt

¯ j for Theorem 2 (eigenvectors) Assuming λi 6= λj for i 6= j and λi 6= λ all i, j, the derivative of the non-trivial eigenvectors is given by n n ¯ ¯ X X φ> φ> d i P φj ¯ i PM φj φi = φj + φ ¯ dt λi − λj λi − λj j j=1 j=1

 P=

0 0 E 0



j6=i

Rodol` a, Cosmo, B, Torsello, Cremers 2016 29/59

Perturbation analysis: boundary interaction strength

20 10

Value of f Eigenvector perturbation depends on length and position of the boundary R d Perturbation strength k dt ΦkF ≤ c ∂M f (m)dm, where

f (m) =

2 n  X φi (m)φj (m) λi − λj i,j=1 j6=i

Rodol` a, Cosmo, B, Torsello, Cremers 2016 30/59

Laplacian perturbation: typical picture

Plate

Punctured plate Figure: Filoche, Mayboroda 2009 31/59

Partial functional maps Model shape M Query shape N Part M ⊆ M ≈ isometric to N Data F, G Partial functional map

T M N

TG ≈ F(M ) M

a, Cosmo, B, Torsello, Cremers 2016 Rodol` 32/59

Partial functional maps Model shape M Query shape N Part M ⊆ M ≈ isometric to N Data F, G Partial functional map

C M N

G> ΨC ≈ F(M )> Φ M

a, Cosmo, B, Torsello, Cremers 2016 Rodol` 32/59

Partial functional maps Model shape M Query shape N Part M ⊆ M ≈ isometric to N Data F, G Partial functional map

C v N

G> ΨC ≈ F> diag(v)Φ v ∈ F(M) indicator function of M

M

a, Cosmo, B, Torsello, Cremers 2016 Rodol` 32/59

Partial functional maps Model shape M Query shape N Part M ⊆ M ≈ isometric to N Data F, G Partial functional map

C v N

G> ΨC ≈ F> diag(η(v))Φ v ∈ F(M) indicator function of M

M

η(t) = 21 (tanh(2t − 1) + 1) Optimization problem w.r.t. correspondence C and part v min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v) C,v

Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59

Partial functional maps min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v) C,v

Rodol` a, Cosmo, B, Torsello, Cremers 2016 33/59

Partial functional maps min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v) C,v

Part regularization  Z ρpart (v) = µ1 |N | −

2 Z η(v)dm + µ2

M

where ξ(t) ≈ δ η(t) −

1 2

ξ(v)k∇M vkdm

M




Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

Partial functional maps min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v) C,v

Part regularization  2 Z Z ρpart (v) = µ1 |N | − η(v)dm + µ2 ξ(v)k∇M vkdm M | M {z } | {z } area preservation

where ξ(t) ≈ δ η(t) −

1 2

Mumford−Shah




Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

Partial functional maps min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v) C,v

Part regularization  2 Z Z ρpart (v) = µ1 |N | − η(v)dm + µ2 ξ(v)k∇M vkdm M | M {z } | {z } area preservation

where ξ(t) ≈ δ η(t) −

1 2

Mumford−Shah




Correspondence regularization X X ((C> C)ii − di )2 ρcorr (C) = µ3 kC ◦ Wk2F + µ4 (C> C)2ij + µ5 i6=j

i

Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

Partial functional maps min kG> ΨC − F> diag(η(v))Φk2,1 + ρcorr (C) + ρpart (v) C,v

Part regularization  2 Z Z ρpart (v) = µ1 |N | − η(v)dm + µ2 ξ(v)k∇M vkdm M | M {z } | {z } area preservation

where ξ(t) ≈ δ η(t) −

1 2

Mumford−Shah




Correspondence regularization X X ρcorr (C) = µ3 kC ◦ Wk2F + µ4 (C> C)2ij + µ5 ((C> C)ii − di )2 | {z } i i6=j slant {z } | {z } | ≈ orthogonality

rank≈r

Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59

Structure of partial functional correspondence

4

2

0

C

W

C> C

0

20

40

60

80 100

singular values

Rodol` a, Cosmo, B, Torsello, Cremers 2016 34/59

Alternating minimization C-step: fix v∗ , solve for correspondence C min kG> ΨC − F> diag(η(v∗ ))Φk2,1 + ρcorr (C) C

v-step: fix C∗ , solve for part v min kG> ΨC∗ − F> diag(η(v))Φk2,1 + ρpart (v) v

Rodol` a, Cosmo, B, Torsello, Cremers 2016 35/59

Alternating minimization C-step: fix v∗ , solve for correspondence C min kG> ΨC − F> diag(η(v∗ ))Φk2,1 + ρcorr (C) C

v-step: fix C∗ , solve for part v min kG> ΨC∗ − F> diag(η(v))Φk2,1 + ρpart (v) v

Iteration 1

2

3

4

Rodol` a, Cosmo, B, Torsello, Cremers 2016 35/59

Example of convergence Time (sec.) 1010

0

5

10

15

20

25 C-step v-step

109

Energy

108 107 106 105 104

0

20

40

60

80

100

Iteration Rodol` a, Cosmo, B, Torsello, Cremers 2016 36/59

Examples of partial functional maps

Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

Examples of partial functional maps

Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

Examples of partial functional maps

Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

Examples of partial functional maps

Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59

Partial functional maps vs Functional maps 100 100

% Correspondences

80

150

50

PFM Func. maps

60

40 50 20

0

100 150

0

0.05

0.1

0.15

0.2

0.25

Geodesic error

Correspondence performance for different rank values k Rodol` a, Cosmo, B, Torsello, Cremers 2016 38/59

Partial correspondence performance Cuts

Holes

% Correspondences

100

80

60

40

20

0 0

0.05

0.1

0.15

0.2

0.25

0

Geodesic Error

PFM

0.05

0.1

0.15

0.2

0.25

Geodesic Error

RF

IM

EN

GT

SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, B, Torsello, Cremers 2016 (PFM); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 39/59

Partial correspondence performance Cuts

Holes

Mean geodesic error

1

0.8

0.6

0.4

0.2

0 20

40

60

80

20

Partiality (%)

PFM

40

60

80

Partiality (%)

RF

IM

EN

GT

SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, B, Torsello, Cremers 2016 (PFM); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 40/59

Deep learning + Partial functional maps

Correspondence

0.1

0.0

Correspondence error Boscaini, Masci, Rodol` a, B 2016 41/59

Deep learning + Partial functional maps

Correspondence

0.1

0.0

Correspondence error

Boscaini, Masci, Rodol` a, B 2016 42/59

Outline

Background: Spectral analysis on manifolds Functional correspondence Partial functional correspondence Non-rigid puzzles

43/59

Litani, BB 2012

Partial correspondence

Rodol` a, Cosmo, B, Torsello, Cremers 2016 45/59

Non-rigid puzzle

Litani, Rodol` a, BB, Cremers 2016 45/59

Partial Laplacian eigenvectors

Functional correspondence matrix C

Rodol` a, Cosmo, B, Torsello, Cremers 2016 46/59

Key observation M

N

N

M

CN N slant ∝

|N | |N |

CM M slant ∝

|M | |M|

Litani, Rodol` a, BB, Cremers 2016 47/59

Key observation M

N

N

M

CN M = CN N CN M CMM slant ∝

|N | |M| |N | |M |

Litani, Rodol` a, BB, Cremers 2016 47/59

Key observation M

N

N

M

CN M = CN N CN M CMM slant ∝

|N | |N | |M| = |N | |M | |M|

Litani, Rodol` a, BB, Cremers 2016 47/59

Non-rigid puzzles problem formulation Model

Input Model M Parts N1 , . . . , Np Output Segmentation Mi ⊆ M Located parts Ni ⊆ Ni Correspondences Ci Clutter Nic Missing parts M0

Parts

M1 M2

C1 C2

N2c N2

N1 N1c N1

M0

M

N2

Litani, Rodol` a, BB, Cremers 2016 48/59

Non-rigid puzzles problem formulation Model

Data Fi , Gi Model basis Φ, Φ(Mi ) Part bases Ψi , Ψi (Ni ) Data term

Parts

M1 M2

C1 C2

N2c N2

N1 N1c N1

> F> i Φ(Mi ) ≈ Gi Ψi (Ni )Ci

M0

M

N2

Litani, Rodol` a, BB, Cremers 2016 48/59

Non-rigid puzzles problem formulation

min

p X

Ci Mi ⊆M,Ni ⊆Ni

> kG> i Ψ i (Ni )Ci − Fi Φ (Mi )k2,1

i=1

+ λM

p X

ρpart (Mi ) + λN

i=0 p X

+ λcorr

p X

ρpart (Ni )

i=1

ρcorr (Ci )

i=1

s.t. Mi ∩ Mj = ∅ ∀i 6= j M0 ∪ M1 ∪ · · · ∪ Mp = M |Mi | = |Ni | ≥ α|Ni |,

Litani, Rodol` a, BB, Cremers 2016 49/59

Non-rigid puzzles problem formulation

min

Ci ui ,vi

p X

Ψi Ci − F> Φk2,1 kG> i diag(η(ui ))Ψ i diag(η(vi ))Φ

i=1

+ λM

p X

ρpart (η(vi )) + λN

i=0 p X

+ λcorr

p X

ρpart (η(ui ))

i=1

ρcorr (Ci )

i=1

s.t.

p X

η(ui ) = 1

i=1 a> M ui

> = a> N vi ≥ αaNi 1

Litani, Rodol` a, BB, Cremers 2016 49/59

Convergence example

Outer iteration 1

Litani, Rodol` a, BB, Cremers 2016 50/59

Convergence example

Outer iteration 2

Litani, Rodol` a, BB, Cremers 2016 50/59

Convergence example

Outer iteration 3

Litani, Rodol` a, BB, Cremers 2016 50/59

Convergence example 30

80

32

90

34

100

36

110

Time (sec) 38

120

40

42

130

Iteration number

44

140

46

150

48

160

Litani, Rodol` a, BB, Cremers 2016 51/59

“Perfect puzzle” example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) Isometric No No Dense (SHOT)

Litani, Rodol` a, BB, Cremers 2016 52/59

“Perfect puzzle” example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) Isometric No No Dense (SHOT)

Segmentation Litani, Rodol` a, BB, Cremers 2016 52/59

“Perfect puzzle” example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) Isometric No No Dense (SHOT)

Correspondence Litani, Rodol` a, BB, Cremers 2016 52/59

Overlapping parts example Model/Part Transformation Clutter Missing part Data term

Synthetic (FAUST) Near-isometric Yes (overlap) No Dense (SHOT)

Segmentation Litani, Rodol` a, BB, Cremers 2016 53/59

Overlapping parts example Model/Part Transformation Clutter Missing part Data term

Synthetic (FAUST) Near-isometric Yes (overlap) No Dense (SHOT)

Correspondence Litani, Rodol` a, BB, Cremers 2016 53/59

Overlapping parts example Model/Part Transformation Clutter Missing part Data term

Synthetic (FAUST) Near-isometric Yes (overlap) No Dense (SHOT)

0.1

0.0

Correspondence error Litani, Rodol` a, BB, Cremers 2016 53/59

Missing parts example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) Isometric Yes (extra part) Yes Dense (SHOT)

Litani, Rodol` a, BB, Cremers 2016 54/59

Missing parts example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) Isometric Yes (extra part) Yes Dense (SHOT)

Segmentation Litani, Rodol` a, BB, Cremers 2016 54/59

Missing parts example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) Isometric Yes (extra part) Yes Dense (SHOT)

Correspondence Litani, Rodol` a, BB, Cremers 2016 54/59

Scanned data example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) / Scan Non-Isometric No No Sparse deltas

Litani, Rodol` a, BB, Cremers 2016 55/59

Scanned data example Model/Part Transformation Clutter Missing part Data term

Synthetic (TOSCA) / Scan Non-Isometric No No Sparse deltas

Segmentation Litani, Rodol` a, BB, Cremers 2016 55/59

Non-rigid puzzle vs Partial functional map

Non-rigid puzzle

Partial functional map (pair-wise)

Rodol` a, Cosmo, B, Torsello, Cremers 2016; Litani, Rodol` a, BB, Cremers 2016 56/59

Summary

New insights about spectral properties of Laplacians Extension of functional correspondence framework to the partial setting Practically working methods for challenging shape correspondence settings Code available (SGP Reproducibility Stamp) Some over-engineering - can be done simpler! (stay tuned...)

57/59

E. Rodolà

A. Bronstein

O. Litany

L. Cosmo

A. Torsello

D. Cremers

Supported by

58/59

Thank you!

59/59