DTU Compute
Introduction to Medical Image Analysis Rasmus R. Paulsen DTU Compute
[email protected] http://www.compute.dtu.dk/courses/02511 http://www.compute.dtu.dk/courses/02512
DTU Compute
Lecture 10 – Registration 9.00
Lecture
Exercises
2
12.00 – 13.00
Lunch break
13.00 –
Exercises
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
What can you do after today?
3
Describe the clinical use of image registration Describe the different types of landmarks Annotate a set of image using anatomical landmarks Describe the objective function used for landmark based registration Compute the objective function when the transformation is given Compute the optimal translation between two sets of landmarks Use the rigid body transformation for image registration Use the similarity transformation for image registration
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Image Registration The act of adjusting something to match a standard Align images
4
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Image registration Monitoring of change in the individual Fusion of information from different sources in a clinically meaningful way Comparison of one subject with others Comparison of groups with others Comparing with an atlas
5
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Data fusion Same patient – two scans
MR
6
DTU Compute, Technical University of Denmark
CT
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Change detection Patient image before and after operation What has changed? Images need to be aligned before comparison
Before operation 7
DTU Compute, Technical University of Denmark
After operation Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Reference and template image The reference image R Template image T Transform the template so it fits the reference
R
T
T 8
transformed
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
The transformations Translation Rotation
Scaling
9
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Similarity measures The aim is to transform the template so it looks like the reference Looks like = Similarity measure Image similarity – Subtract the two images and see “what is left”
Landmark similarity – Landmarks from the two images should be “close together”
10
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Landmark Based Registration Landmarks placed on both reference and template image The landmark should have correspondence
11
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Point correspondence Landmarks are numbered Each landmark should be placed the same place on both images
3
2 1
3 2 1
12
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Landmark types Anatomical landmark – a mark assigned by an expert that corresponds between objects in a biologically meaningful way
Mathematical landmark – a mark that is located on a curve according to some mathematical or geometrical property
Pseudo landmark – a mark that is constructed on a curve based on anatomical or mathematical landmarks
13
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Landmarks
𝑎𝑖 Reference image
14
𝑏𝑖
R
DTU Compute, Technical University of Denmark
Template image
Introduction to Medical Image Analysis
T 13/4/2016
DTU Compute
The aim of registration Find a transformation that maps the coordinates of the reference to the coordinates of the template – Why not the template to the reference? Sampling of template image:
Backward mapping -> inverse transform
15
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
The Transformation Transforms point p Into point p’ T is for example a – – – –
16
DTU Compute, Technical University of Denmark
Translation Rotation Rigid body transform Similarity transform
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
The Transformation Transforms points from the reference
𝑎𝑖 17
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
The parameters The parameters is a vector with p elements
parameters
18
DTU Compute, Technical University of Denmark
– The type of transformation determines the number of parameters – Translation p = 2 – Rotation p = 1 – Scaling p = 1
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Rigid body transform How many parameters?
A) 1 B) 2 C) 3 D) 4 E) 5
22
4
A 19
DTU Compute, Technical University of Denmark
2
0
B
C
Introduction to Medical Image Analysis
D
0
E 13/4/2016
DTU Compute
Similarity transform How many parameters?
A) 1 B) 2 C) 3 D) 4 E) 5
19
1
1
A 20
DTU Compute, Technical University of Denmark
2
0
B
C
Introduction to Medical Image Analysis
D
E 13/4/2016
DTU Compute
Objective function 𝑁
𝐹=
𝐷 𝑇 𝑎𝑖 , 𝑏𝑖
2
The objective function measures how well two point sets match
𝑖=1 Points from the template image
Transformed points from the reference image
22
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Objective function The objective function measures how well two point sets match
𝑁
𝐹=
𝐷 𝑇 𝑎𝑖 , 𝑏𝑖
2
𝑖=1 Distance between points
b5
D T (a5 ) 23
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Objective function 3
𝐹=
𝐷 𝑇 𝑎𝑖 , 𝑏𝑖
2
= 𝐷12 + 𝐷22 + 𝐷32
𝑖=1
b1
b2
D1
D2 T (a1 ) 24
T (a2 )
DTU Compute, Technical University of Denmark
T (a3 )
Template
b3
D3 Transformed reference Introduction to Medical Image Analysis
13/4/2016
DTU Compute
MedTek: Laver jeg Matlab øvelser A) Ingen af dem B) Få af dem C) Halvdelen D) Mange af dem E) Dem alle
7 6
2 1
1
A 25
DTU Compute, Technical University of Denmark
B
C
Introduction to Medical Image Analysis
D
E 13/4/2016
DTU Compute
Minimization / Optimization 𝑁
𝐹=
𝐷 𝑇 𝑎𝑖 , 𝑏𝑖 𝑖=1
27
DTU Compute, Technical University of Denmark
2
Find the set of parameters that minimizes the objective function
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Minimization – pure translation F = D 12 + D 22 + D 32
b1
b2
D1
D2 T (a1 ) 28
T (a2 )
DTU Compute, Technical University of Denmark
T (a3 )
Template
b3
D3 Transformed reference Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Minimization – pure translation F = D 12 + D 22 + D 32
Decreased!
b1
b2
D1
D2
T (a1 )
T (a2 )
T (a3 )
Template
b3
D3 Transformed reference
29
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Minimization – pure translation F = D 12 + D 22 + D 32
Decreased! Overall minimum lengths!
b1
b2
D1
D2
T (a1 )
T (a2 )
T (a3 )
Template
b3
D3 Transformed reference
30
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Objective function A) 600 B) 50 C) 100 D) 900 E) 300 16
5 1
A 31
1
0
B
C
D
E
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Translation Simple shift of coordinates
parameters
34
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Translation Simple shift of coordinates
parameters
35
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Objective function for translation 𝑁 Objective function
𝐹=
𝐷 𝑇 𝑎𝑖 , 𝑏𝑖
2
𝑖=1 Translation
Objective function for translation
36
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Optimal function value 𝑁
𝐹=
𝐷 𝑇 𝑎𝑖 , 𝑏𝑖
2
𝑖=1
37
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Optimal translation Objective function
Parameters
Optimal translation
38
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Optimal translation
Original landmarks
39
DTU Compute, Technical University of Denmark
Reference points translated
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Decrease in F A) 1 B) 2 C) 3 D) 4 E) 5
20
1
A
40
1
0
B
0
C
D
E
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Rigid body transformation Translation and rotation Rigid body = stift legeme Angles and distances are kept
42
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Rigid body transformation objective function Transformation
Rotation matrix
Objective function
43
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Optimal rigid body transformation The minimum of the objective function can be found in several ways Always start by matching the centre of masses The rotation can be found by singular value decomposition In 2D it can also be found in simpler ways
44
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Rigid body transformation Comparing images of the same patient – Images taken with the same scanner – Compensate for different positioning in the scanner
45
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Similarity transformation Translation, rotation, and isotropic scaling Angles are kept
Solution found using SVD
46
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Similarity transform Same patient – different scanners – Different pixel/voxel sizes
Same patient over time – Growing
Different patients Atlas to patient
47
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Niveau af forelæsninger A) Meget let at følge med B) Let at følge med C) Tilpas D) Svært at følge med E) Meget svært at følge med
19
5
4
2
0
A 48
DTU Compute, Technical University of Denmark
B
C
Introduction to Medical Image Analysis
D
E 13/4/2016
DTU Compute
Image transformation The found transformation maps the coordinates of the reference to the coordinates of the template We want to transform the template? Backward image mapping!
49
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Backward mapping Run through all the pixel in the output image Use the inverse transformation to find the position in the input image Use bilinear interpolation to calculate the value Put the value in the output image
Here – output image = transformed template image – Input image = template image – Inverse transform = the computed transform From reference to template 50
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Hvad er forkert A) 1 B) 2 C) 3 D) 4 E) 5 18
3 0
0
A 51
DTU Compute, Technical University of Denmark
0
B
C
Introduction to Medical Image Analysis
D
E
13/4/2016
DTU Compute
What can you do after today?
53
Describe the clinical use of image registration Describe the different types of landmarks Annotate a set of image using anatomical landmarks Describe the objective function used for landmark based registration Compute the objective function when the transformation is given Compute the optimal translation between two sets of landmarks Use the rigid body transformation for image registration Use the similarity transformation for image registration
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Exercise Registration – Matching of hands using landmarks – Cancer treatment follow up in a brain
54
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Next week Hough Transformation and Path Tracing
Hough transform
-400
-200 0 200 400 -50
55
0
DTU Compute, Technical University of Denmark
50
Introduction to Medical Image Analysis
13/4/2016
DTU Compute
Exercises
? 56
DTU Compute, Technical University of Denmark
Introduction to Medical Image Analysis
13/4/2016
