SPRING 2016

MEDICAL IMAGE COMPUTING (CAP 5937)

LECTURE 7: Medical Image Segmentation (I) (Radiology Applications of Segmentation, and Thresholding)

Dr. Ulas Bagci HEC 221, Center for Research in Computer Vision (CRCV), University of Central Florida (UCF), Orlando, FL 32814. [email protected] or [email protected]

1

2

Outline • Introduction to Medical Image Segmentation, type of segmentation methods, and definitions – Recognition & Delineation

• Simplest Segmentation Method(s): Thresholding – – –

Otsu Thresholding Parametric Method PET Image Thresholding Methods •

ITM (Iterative Thresholding Method)

3

Motivation for Image Segmentation In the last 20 years the computer vision and medical imaging communities have produced a number of useful algorithms for localizing object boundaries in images.

4

Motivation for Image Segmentation • • • • • • • • • • •

Content based image retrieval Machine Vision Medical Imaging applications (tumor delineation,..) Object detection (face detection,…) 3D Reconstruction Object/Motion Tracking Object-based measurements such as size and shape Object recognition (face recognition,…) Fingerprint recognition, Video surveillance …

5

Segmentation Tools in Radiology Applications • 3D views to visualize structural information and spatial anatomic relationships is a difficult task, which is usually carried out in the clinician’s mind.

6

Segmentation Tools in Radiology Applications • 3D views to visualize structural information and spatial anatomic relationships is a difficult task, which is usually carried out in the clinician’s mind. • Image-processing tools provide the surgeon with interactively displayed 3D visual information.

7

Segmentation Tools in Radiology Applications

Credit: Kaus, et al. Radiology 2001.

8

Segmentation Tools in Radiology Applications • Determination of the volumes of abdominal solid organs and focal lesions has great potential importance (liver, spleen, …). • Monitoring the response to therapy and the progression of neoplastic disease and preoperative examination of living liver donors are the most common clinical applications of volume determination.

(credit: Farraher, et al. Radiology 2005)

9

Segmentation Tools in Radiology Applications • Gross Tumor Volume in CT/MRI • Metabolic Tumor Volume in PET/SPECT/ – Surgery/Therapy Planning • Planning Tumor Volume (PTV)

– Tumor characterization

• Texture Extraction requires segmentation to be done • Shape analysis

Credit: Manniesing, et al, Radiology 2008

Segmentation Tools in Radiology Applications • There is a strong interest in automatic and reproducible techniques for detection and quantification of vascular disease • A first step toward an effective vessel analysis tool is segmentation of the vasculature. axial

MIP: maximum intensity Projection image of cerebral vessels (in CTA)

coronal

sagittal

10

11

Segmentation Tools in Radiology Applications • MR volumetry of the hippocampus can help distinguish patients with AD (Alzheimer’s Disease) from elderly controls with a high degree of accuracy (80%–90%).

12

Segmentation Tools in Radiology Applications • MR volumetry of the hippocampus can help distinguish patients with AD (Alzheimer’s Disease) from elderly controls with a high degree of accuracy (80%–90%).

amygdala Credit: Colliot et al, Radiology 2008.

hippocampus

13

Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation.

14

Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation.

Segmentation of an image entails the division or separation of the image into regions of similar attribute.

15

Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation.

Segmentation of an image entails the division or separation of the image into regions of similar attribute.

The most basic attributes: -intensity -edges -texture -other features…

16

Image Segmentation Definition: Partitioning a picture/image into distinctive subsets is called segmentation.

Purpose:

To extract object information and represent this as a hard/fuzzy geometric structure.

Recognition:

Determining the object’s whereabouts in the scene. (humans > computer)

Delineation:

Determining the object’s spatial extent and composition in the scene. (computers > humans)

17

Recognition - Example

Model is induced

No Model is induced (slice credit: J. Kim et al, Signal Processing 2007)

18

Approaches to Recognition • Model-based • Knowledge-based • Atlas-based

-

Non-interactive

• Human-assisted

-

Interactive

19

Approaches to Recognition • Model-based • Knowledge-based • Atlas-based

-

Non-interactive

• Human-assisted

-

Interactive

- They all originate from human knowledge. - Their relative efficacy is unknown.

20

Approaches to Delineations pI (purely image-based) approaches • Rely mostly on information available in the given image only. • Recognition: manual

21

Approaches to Delineations pI (purely image-based) approaches • Rely mostly on information available in the given image only. • Recognition: manual SM (shape model-based) approaches • Employ models to codify object family shape info. • Recognition: model-based/manual

22

Approaches to Delineations pI (purely image-based) approaches • Rely mostly on information available in the given image only. • Recognition: manual SM (shape model-based) approaches • Employ models to codify object family shape info. • Recognition: model-based/manual Hybrid approaches • Combine among pI and SM approaches. • Recognition: model-based, automatic.

23

Classification of Methods Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets

24

Classification of Methods Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets Region-based (RpI): • clustering – kNN, CM, FCM • graph cut • fuzzy connectedness • MRF • watershed • optimum partitioning • (Mumford-Shah)

25

Classification of Methods Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets Region-based (RpI): • clustering – kNN, CM, FCM • graph cut • fuzzy connectedness • MRF • watershed • optimum partitioning • (Mumford-Shah)

SM Approaches • manual tracing • live wire • active shape/appearance • M-reps • atlas-based

26

Classification of Methods Boundary-based (BpI): • optimum boundary • active boundary • live wire • level sets Region-based (RpI): • clustering – kNN, CM, FCM • graph cut • fuzzy connectedness • MRF • watershed • optimum partitioning • (Mumford-Shah)

SM Approaches • manual tracing • live wire • active shape/appearance • M-reps • atlas-based

Hybrid Approaches • • • • • •

BpI + BpI RpI + RpI BpI + RpI BpI + SM RpI + SM SM + SM

27

Classification of Methods pI Approaches + Where image info is good, accuracy is good; - Bad where it is poor/absent; - Need recognition help; + Can determine degree of match of model to image well; - Lack obj shape & geographic info;

28

Classification of Methods SM Approaches - Even where image info is good, accuracy suffers; + Where bad, model helps; + Can help in recognition; - Need best match info;

+ Good models embody obj shape & geographic info;

29

Purely Image Based Segmentation Methods

30

Thresholding – Simple Segmentation • Image binarization – mapping a scalar image I into a binary image J

J(x, y) =

(

0 1

if I(x, y) < T otherwise.

31

Thresholding – Simple Segmentation • Image binarization – mapping a scalar image I into a binary image J

J(x, y) =

(

0 1

if I(x, y) < T otherwise.

32

Thresholding – Simple Segmentation Brighter objects

Darker objects

33

Thresholding – Simple Segmentation Brighter objects

Darker objects

DIFFICULTIES 1. The valley may be so broad that it is difficult to locate a significant minimum 2. Number of minima due to type of details in the image 3. Noise 4. No visible valley 5. Histogram may be multi-modal

34

Example: CT Scan

35

Example: CT Scan

36

Example: CT Scan

37

Example: CT Scan

38

Example: CT Scan

39

Thresholding Methods • • • • • • • • • • •

Huang Intermode Isodata Li MaxEntropy Mean MinError Otsu Percentile RenyiEntropy Moments

40

Thresholding Methods • • • • • • • • • • •

Huang Intermode Isodata Li MaxEntropy Mean MinError Otsu Percentile RenyiEntropy Moments

41

Thresholding Methods • • • • • • • • • • •

Huang Intermode Isodata Li MaxEntropy Mean MinError Otsu (non-parametric) Percentile RenyiEntropy Moments

PET Imaging Fixed Thresholding Adaptive Thresholding Iterative Thresholding

42

Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance).

43

Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance). • Otsu’s algorithm selects a threshold that maximizes the 2 between-class variance b . In the case of two classes, 2 b

= P1 (µ1

µ)2 + P2 (µ2

µ)2 = P1 P2 (µ1

µ2 ) 2

44

Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance). • Otsu’s algorithm selects a threshold that maximizes the 2 between-class variance b . In the case of two classes, 2 b

= P1 (µ1

µ)2 + P2 (µ2

µ)2 = P1 P2 (µ1

µ2 ) 2

• where P1 and P2 denote class probabilities, and μi the means of object and background classes.

45

Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance).

P1 =

u X

u

p(i)

ı=0

P2 =

GX max

ı=u+1

p(i)

u

46

Otsu Thresholding • Definition: The method uses the grey-value histogram of the given image I as input and aims at providing the best threshold in the sense that the “overlap” between two classes, set of object and background pixels, is minimized (i.e., by finding the best balance).

P1 =

u X

p(i)

µ1 =

P2 =

ı=u+1

p(i)

ip(i)/P1

ı=0

ı=0

GX max

u X

µ2 =

GX max

ip(i)/P2

ı=u+1 CLASS MEANS

47

Otsu Thresholding-Algorithm

P1 cI (u)

P2 1

cI (u)

c indicates cumulative histogram, and P1 and P2 can be approximated well with cumulative density function.

48

Otsu Thresholding-Algorithm

2 b

P1 cI (u)

= P1 (µ1

µ)2 + P2 (µ2

P2 1

cI (u)

c indicates cumulative histogram, and P1 and P2 can be approximated well with cumulative density function.

µ)2 = P1 P2 (µ1

µ2 ) 2

49

Otsu Thresholding-Algorithm

P1 cI (u)

P2 1

cI (u)

c indicates cumulative histogram, and P1 and P2 can be approximated well with cumulative density function.

50

Otsu Thresholding-Algorithm

P1 cI (u)

P2 1

cI (u)

c indicates cumulative histogram, and P1 and P2 can be approximated well with cumulative density function.

51

Otsu Thresholding-Algorithm

P1 cI (u)

P2 1

cI (u)

c indicates cumulative histogram, and P1 and P2 can be approximated well with cumulative density function.

52

Otsu Thresholding-Algorithm

P1 cI (u)

P2 1

cI (u)

c indicates cumulative histogram, and P1 and P2 can be approximated well with cumulative density function.

53

Otsu Thresholding-Algorithm

P1 cI (u)

P2 1

cI (u)

c indicates cumulative histogram, and P1 and P2 can be approximated well with cumulative density function.

optimal

54

Parametric Method for Optimal Thresholding • Assuming again a two-class problem and assuming that the distribution of gray levels for each class can be modeled by a normal distribution with mean and variance

55

Parametric Method for Optimal Thresholding • Assuming again a two-class problem and assuming that the distribution of gray levels for each class can be modeled by a normal distribution with mean and variance • the overall normalized intensity histogram can be written as the following mixture probability density function:

56

Parametric Method for Optimal Thresholding • Assuming again a two-class problem and assuming that the distribution of gray levels for each class can be modeled by a normal distribution with mean and variance • the overall normalized intensity histogram can be written as the following mixture probability density function:

where P1 and P2 are class probabilities. The optimal threshold (T) can be found as solving the quadratic equation à

57

Parametric Method for Optimal Thresholding

58

Parametric Method for Optimal Thresholding

In case, variances of both classes are equal, then->

59

Parametric Method for Optimal Thresholding

In case, variances of both classes are equal, then->

60

Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014)

61

Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014)

Fixed Thresholding

Adaptive Thresholding

Iterative Thresholding

62

Fixed Thresholding Methods

• Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014)

63

Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014)

Fixed Thresholding

Adaptive Thresholding

Phantom Based

Iterative Thresholding

Image Quality metrics based

Adaptive Thresholding

64

65

Thresholding methods for PET Image Segmentation • Due to the nature of PET images (i.e., low resolution with high contrast), thresholding-based methods are suitable – because the local or global intensity histogram usually provides a sufficient level of information for separating the foreground (object of interest) from the background. (Foster, Bagci, et al., CBM 2014)

Fixed Thresholding

Adaptive Thresholding

Phantom Based

Iterative Thresholding

Image Quality metrics based

66

Iterative Thresholding Method (ITM) S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes.

67

Iterative Thresholding Method (ITM) S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes.

68

Iterative Thresholding Method (ITM) S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes. The measured S/B ratios of the lesions are then estimated from PET images, and their volumes are iteratively calculated using the calibrated S/B-threshold-volume curves

69

Iterative Thresholding Method (ITM) S/B: Source to background ratio. The method is based on calibrated threshold-volume curves at varying S/B ratio acquired by phantom measurements using spheres of known volumes. The measured S/B ratios of the lesions are then estimated from PET images, and their volumes are iteratively calculated using the calibrated S/B-threshold-volume curves The resulting PET volumes are then compared with the known sphere volume and CT volumes of tumors that served as gold standards.

70

ITM Example Result on PET Images/Lung

71

Another Example for PET Thresholding

ITM for tumor segmentation/FDG PET

72

Another Example for PET Thresholding

73

Further Thresholding Example – CT Bones

74

Further Thresholding Example – CT Bones

75

Head-Neck CT – Thresholding for Skull Modeling

Segmentation of the skull and the mandibula in CT images using thresholding. (a) Original CT image of the head. (b) Result with a threshold value of 276 Hounsfield units. The segmented bony structures are represented in color. (c) 3D rendering of the skull shows a congenital growth deficiency of the mandibula in this 8-year-old patient. This information was used preoperatively to plan a repositioning of the mandibula. (Slice Credit: P.Seutens)

76

Multiple Thresholds – MRI Thresholding

Thresholding can be done interactively and separates the image into different regions. Valleys in the histogram indicate potentially useful threshold values Credit: Toeonies, K.

77

Summary of today’s lecture • Introduction into the Medical Image Segmentation • Recognition and Delineation concepts in Segmentation • Simplest Segmentation method: Thresholding – Otsu – Parametric method for optimal thresholding – PET Image thresholding • ITM, fixed thresholding, etc.

78

Slide Credits and References • Jayaram K. Udupa, MIPG of University of Pennsylvania, PA. • P. Suetens, Fundamentals of Medical Imaging, Cambridge Univ. Press. • • • • • • •

Foster, B., et al. CBM, Review paper, 2014. Kaus, et al. Radiology 2001. Toeonies, K., Medical Image Analysis. Farraher, et al., Radiology 2005 Zaidi, H., Quantitative Analysis in Nuclear Medicine Imaging. Bailey et al. Positron Emission Tomography, Springer. Dawood, M., et al. Correction Techniques in Emission Tomography