Segmentation in Ultrasound Imaging

Segmentation in Ultrasound Imaging Xiaowei Zhou

Dept. of ECE, HKUST 2009-06-09

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

1 / 35

Segmentation in Ultrasound Imaging

Outline 1

Introduction

2

General methods for segmentation An overview MRF Deformable models

3

Customized segmentation methods for ultrasound images Section overview Variation reduction Parametric images Shape prior Sequential data

4

Future work Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

2 / 35

Segmentation in Ultrasound Imaging Introduction

Why we need segmentation Objective of segmentation Segmentation is to separate the image into different regions which are meaningful to special tasks. Segmentation techniques are important to structural and anatomical analysis, which helps the health checking and therapy. e.g, computing the ejection power of the heart from the LV volume change.

Manual delineation is labor consuming and dependent on human expertise. Automatic and accurate segmentation by computer is needed. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

3 / 35

Segmentation in Ultrasound Imaging Introduction

Basis for segmentation Definition Image segmentation means partitioning an image into nonoverlapping regions that are homogeneous with respect to some features. What is a good image for segmentation? (1) Homogeneous within a single region (2) High contrast between regions

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

4 / 35

Segmentation in Ultrasound Imaging Introduction

Difficulties in ultrasound imaging segmentation What’s the case in ultrasound imaging? speckle High frequency variation attenuation Low frequency variation low contrast Undistinguishable regions orientation sensitive Missing edges along wave propagation direction other artefacts Occlusion, shadow, noises ...

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

5 / 35

Segmentation in Ultrasound Imaging Introduction

Organization of the following discussion

A systematic overview of general segmentation methods. A summary of methods customized for ultrasound image segmentation in the unified frameworks. A perspective of what can be done in future.

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

6 / 35

Segmentation in Ultrasound Imaging General methods for segmentation An overview

An overview of general segmentation methods (1) The output of segmentation can be classification of pixels with region labels or the location of boundaries separating different regions. Accordingly, the segmentation methods can be divided to two categories: a Pixel classfication 1 thresholding 2 clustering: k-means, Markov Random Fields (MRF) 3 supervised classifiers: ANN, SVM ...

b Edge based methods 1 Feature detection: spatial filters, morphological operations, Hugh

transform ... 2 Energy minimization: Graph cut, Deformable models

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

7 / 35

Segmentation in Ultrasound Imaging General methods for segmentation An overview

An overview of general segmentation methods (2) The MRF and the Deformable models are two of the most advanced and preferred methods for image segmentation. Markov Random Fields Bayesian framework Optimization: Posterior probability Variable: region labels Deformable models Energy minimization framework Optimization: Energy function Variable: Location of the contour Both are convenient to integrate all kinds of information. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

8 / 35

Segmentation in Ultrasound Imaging General methods for segmentation MRF

MRF: Clustering - Maximum likelihood The basis for MRF is clustering: Given image intensity y , classify pixels into k regions with region label x = 1, 2, . . . , k. Optimization formulation for clustering: ML x ∗ = arg max ln p(y |x, θ)

(1)

x

where p(y |x, θ) is the p.d.f of intensity given the class, θ is the parameter vector of the distribution . For classical k-means, p(y |x, θ) is a Gaussian distribution: Optimization function for k-means P P xs ln p(y |x, θ) = ln p(ys |xs ) ∝ {ln(σ xs ) + ( y√s −µ )2 } 2σ xs s

(2)

s

where µxs = µj (j = xs = 1, 2, . . . , k) and σ xs = σ j (j = xs = 1, 2, . . . , k) are the cluster mean and variance. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

9 / 35

Segmentation in Ultrasound Imaging General methods for segmentation MRF

MRF: MRF methods - Maximum a posterior k-means only considers the intensity of pixels without spatial constraint. MRF introduces a prior distribution of x describing the spatial continuity of regions: Optimization formulation for MRF: MAP x ∗ = arg max ln p(y |x, θ)p(x)

(3)

x

where: p(x) =

1 exp{−E (x)} Z

(4)

Z is for normalization. E (x) =

X i

V1 (xi ) +

X

V2 (xi , xj )

(5)

i,j

The first order potential V1 reflects the prior knowledge about relative likelihood of different region types, and the second order potential V2 describes correlation of neighboring pixels. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

10 / 35

Segmentation in Ultrasound Imaging General methods for segmentation MRF

MRF: Clique potential For example, V2 can be defined as: Second order potential  V2 (xi , xj ) =

−β, if xi = xj +β, if xi 6= xj

(6)

β>0 i,j are neighbors.

Figure: second-order clique Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

11 / 35

Segmentation in Ultrasound Imaging General methods for segmentation MRF

MRF: Solve the optimization How to optimize Equation (3)? x is to be estimated θ is a latent variable Expectation-Maximization initialize x old = x0 while x new 6= x old do θ = arg maxθ ln p(y |x old , θ) x new = E [ln p(y |x, θ)p(x)] =⇒ idea behind k-means Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

12 / 35

Segmentation in Ultrasound Imaging General methods for segmentation Deformable models

Deformable models: traditional snake Snake: A contour X (s) = [x(s), y (s)] imagined to be put on a landscape with height being the minus gradient of the original image. Energy function of the snake Z ˛ ˛2 1 ˛˛ 0 ˛˛2 E (X (s)) = (α X (s) + β ˛X 00 (s)˛ ) − |∇I (X )|2 ]ds 2

(7)

c

Figure: Zhou, Xiaowei (Dept. of ECE)

Deformable snake. McInerney and Terzopoulos 1996

Segmentation in Ultrasound Imaging

2009-06-09

13 / 35

Segmentation in Ultrasound Imaging General methods for segmentation Deformable models

Deformable models: Generalization Generalized energy function Z (Eint + Eext )ds

E (X (s)) =

(8)

c

Eint : keeping smoothness of the contour Eext : attracting the contour to desired features External energy: defined according to any objective Eext = EC + Eω + Eω¯

(9)

8 R < EC = RC PC (X (s))ds Eω = Rω Pω (x, y )dxdy : Eω¯ = ω¯ Pω¯ (x, y )dxdy

(10)

C : the contour; ω: inside the contour; ω ¯ : outside the contour. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

14 / 35

Segmentation in Ultrasound Imaging General methods for segmentation Deformable models

Deformable models: External energy function

There are two examples for Eext by defining the potential function in Equation (10): Gradient information based Eext PC (x, y ) = −|∇[Gσ (x, y ) ∗ I (x, y )]|2

(11)

where Gσ is a Gaussian filter and ∗ means convolution. Region homogeneity based Eext 

Pω (x, y ) = |I (x, y ) − µω |2 Pω¯ (x, y ) = |I (x, y ) − µω¯ |2

(12)

where µω and µω¯ are the region mean inside and outside the contour.

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

15 / 35

Segmentation in Ultrasound Imaging General methods for segmentation Deformable models

Deformable models: Solve the optimization The optimization of the energy function in Equation (7) is a variational problem (functional optimization). Solution: Euler-Lagrange equation Fint + Fext = 0 where:

(

Fint = Fext =

∂Eint ∂X ∂Eext ∂X

(13) 2

∂ ∂Eint ∂ ∂Eint − ∂s + ∂s 2 ∂X 00 ∂X 0 ∂ ∂Eext ∂ 2 ∂Eext − ∂s + ∂s 2 ∂X 00 ∂X 0

(14)

Above PDE can be solved by FDM or FEM. Another solution: Dynamic simulation µ

Zhou, Xiaowei (Dept. of ECE)

∂2X ∂X +γ = Fint + Fext ∂t 2 ∂t Segmentation in Ultrasound Imaging

(15)

2009-06-09

16 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Section overview

Ultrasound image segmentation An overview 1 2 3 4

Segmentation of ultrasound images are a challenging task due to the reasons discussed previously. Thousands of works have been done to improve the results, customized for ultrasound data. The segmentation frameworks behind these approaches are mostly based on the Bayesian clustering and the deformable models. Efforts are made on: 1 2 3 4

Variation reduction Segmentation on parametric images Using shape prior Using sequential data

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

17 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Inhomogeneity correction

Inhomogeneity correction: Preprocessing (1)

Motivation: to remove granular speckle patterns, random noises, attenuation and other artefact. Conventional image enhancement techniques: spatial filtering, temporal averaging, and wavelet based methods ... Difficulty: tradeoff between noise reduction and image information loss.

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

18 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Inhomogeneity correction

Inhomogeneity correction: Preprocessing (2) Xiao et al., 2002: Intensity inhomogeneity correction in a Bayesian framework. Basic idea of Xiao et al, 2002. Add a variable d in the MRF model representing the distortion field, and estimate x and d using EM. y = y∗ + d (16) The likelihood term in Equation (3) was changed to: ln p(y |x, θ) = ln p(y − d|x, θ),

(17)

The EM iteration was initialized at d = 0. When estimating d in the following iterations, a low pass filter was applied on the result to impose spatial smoothness on the distortion field. Correct the slow intensity variation caused by attenuation instead of speckle and other high frequency variation

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

19 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Inhomogeneity correction

Inhomogeneity correction: Adaptive clustering Some papers claimed: underlying intensity distribution in the same region in Equation (2) was not single mode. Adaptive clustering based on (Pappas et al., 1992) was used. Basic idea of adaptive clustering Compute µxs in Equation (2) by local statistics in a sliding window. The energy function in Equation (2) was improved to: U(x|y ) =

X s

ys − µxs {ln(σsxs ) + ( √ xss )2 } + E (x) 2σs

(18)

The subscript of µxs s and σsxs means that the parameters are varying from pixel to pixel. Multi-scale: iteration from a large window to a small window. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

20 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Inhomogeneity correction

Inhomogeneity correction: Multi-scale strategy Basic idea of multi-scale strategy Coarse to fine optimization based on a Gaussian pyramid. Results at higher level initialize the optimization at lower levels. Advantages are as follows: 1 Make the algorithm more robust to noise: the high level image is filtered by Gaussian mask. 2 In ultrasound images, the intensity distribution is not strictly Gaussian. However, in the high level of Gaussian pyramid, it is nearly Gaussian according to Central Limit Theorem: A relatively accurate result can be obtained at high levels and refined at lower levels (Lin at al., 2003). 3 Optimization in a coarse to fine procedure can reduce computational complexity. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

21 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Parametric images

Parametric images: Pase based edge detection In Mulet-Parada’s paper, it was suggested that local phase information be more appropriate and reliable as edge feature than intensity gradient. Basic idea of Mulet-Parada et al., 2000 Theory fundamental: In fourier analysis, the edge is corresponding to the symmetric point of the local area, where the expanded fourier components are exactly in phase.

Methods: The phase congruency was detected by a quadrature pair of oriented Gabor filters. Advantages: Reliable output invariant to image amplitude and contrast. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

22 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Parametric images

Parametric images: Texture features Definition Texture features: features estimated from local intensity models based on B-mode signals. All kinds of texture parameters have been extracted for tissue characterization in ultrasound imaging. An example: the scatter density can be estimated from parameter estimation of the Nakagami distribution. Nakagami distribution pdf =

2mm x 2m−1 exp(−mx 2 /Ω)U(x) Γ2 (m) · Ωm

(19)

m: related with the scatter density in the resolution cell. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

23 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Parametric images

Parametric images: Acoustic features Definition Acoustic features: features estimated from spectral estimation based on RF signals. All kinds of texture parameters have been extracted for tissue characterization in ultrasound imaging. Two examples: IBS and MCF IBS =

X

PSD(f )

(20)

BW

MCF =

X f · PSD(f ) IBS

(21)

BW

where PSD is the power spectral density, which can be computed by FFT. BW means the summation is performed on a limited bandwith. Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

24 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Shape prior

Shape prior

Motivation: poor quality; missing edges. The prior shapes are usually constructed by: 1 2 3

prior knowledge offline training result of segmentation on the coarse-scale image: Lin et al.

The way to incorporate prior shapes: the deformable models with a regularization term in the energy function in Equation (8).

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

25 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Shape prior

Shape prior: an example

Chen et al., 2002: Active contour with shape prior. Basic idea of Chen et al., 2002 Constrain the deformation of the contour in the affine space: Z

{−|∇[Gσ ∗ I (X (s))]|2 + λd 2 (µMX (s) + T , X ∗ (s))}ds

E (X (s), µ, M, T ) =

(22)

C

where µ, M and T are deformation parameters describing scaling, rotation and translation. d(a, b) is the distance between corresponding points of the active contour X (s) and the prior shape X ∗ (S).

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

26 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Sequential data: Overview Motivation: 1 2

Ultrasound imaging: relatively high temporal resolution. Straightforward to thinking about taking advantage of spatial information.

How to make use of temporal information? temporal smoothness constraint Temporal smoothing or adding a temporal smoothness regularization term to the deformable models. spatiotemporal edge detection Continuity of motion makes the contour expanded to a continuous surface while the speckle and random noise patterns may decorrelate along time. contour tracking in 2D+T Propagating the segmentation results from frame to frame. motion as a feature Segmentation on the velocity/strain image.

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

27 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Motion as a feature

Motion information is first estimated, then used as a feature for segmentation. Motivation: Different tissues may have similar acoustic characteristics, while their motion properties are distinctive: 1 2

fast flowing blood vs. static vessel soft normal tissue vs. stiff cancer mass

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

28 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Motion as a feature: Example

Figure: RF image of a phantom (frame 1) Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

29 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Motion as a feature: Example

Figure: RF image of a phantom (frame 2) Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

30 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Motion as a feature: Paper 1 Basic idea of Gronningsaeter et al, 1994. The temporal cross-correlation coefficient ρ: relatively low at regions corresponding to rapid flowing blood while be near 1 at regions of stationary vessel wall. P I (t) · I ∗ (t + 1) ROI rP ρ = r P (23) 2 2 |I (t)| |I (t + 1)| ROI

ROI

ROI means the region of interest with a tuned size. I (t) and I (t) represents consecutive frames. Then the image was segmented based on the image of ρ(x, y ). Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

31 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Motion as a feature: Paper 2

Basic idea of Dydenko et al., 2003 The estimated velocity field will be relatively homogeneous in muscle area highly variational in fast and randomly flowing blood area due to the motion-feature decorrelation phenomenon and noise The local variation of the estimated velocity field was used as feature for segmentation.

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

32 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Motion as a feature: Paper 3

Definition MCC: the Maximum Correlation Coefficient optimized with each searching window in correlation based speckle tracking methods. Basic idea of Yan et al., 2007 The segmentation was based on MCC over the image: MCC in blood area was lower (< 0.5) due to irregular motion MCC in myocardium was higher(> 0.7)

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

33 / 35

Segmentation in Ultrasound Imaging Customized segmentation methods for ultrasound images Sequential data

Motion as a feature: Remarks

Paper 1: reasonable to separate the rapid moving object from the static object, but problematic when all objects are moving which make the the correlation coefficients are low all over the image. Paper 2 and 3: both using computation error of motion as segmentation features on RF images: the high variance of estimation or the low MCC value. Hardly any paper directly use motion estimated as features for segmentation. The reason is speckle tracking in ultrasound imaging is hard due to the motion-feature decorrelation.

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

34 / 35

Segmentation in Ultrasound Imaging Future work

Future work What if the motion can be estimated accurately? Ideal case: If the motion information can be obtained accurately, a complete velocity image (D(x)) or strain image D(x)0 can largely improve the segmentation results. Difficulty: in the correlation based motion estimation, the computation window needs a sufficiently large size which may average the boundary, and the single motion assumption within the window is not true near the boundary. Possible solution 1 2

the relatively accurate motion image + the gray level image locating the boundary and estimating the motion at the same time?

Zhou, Xiaowei (Dept. of ECE)

Segmentation in Ultrasound Imaging

2009-06-09

35 / 35