Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS) Peter Weber *, Kostas Konstantinos, Stergios Stegiopoulos, Georg Sakas, Stefan Wesarg, Daniel Speicher, Andreas Freibert, Matthias Noll * Fraunhofer IBMT, Dept. Technical Ultrasound Ensheimerstraße 48, 66386 St. Ingbert, Germany Tel: +49 6894 980 227, Fax: +49 6894 980 234 [email protected]

ABSTRACT PUDIS is a Portable 4Dimensional Ultrasound Diagnostic Imaging System with a single 2D phased arrayprobe. A 3D adaptive beamforming procedure is implemented in an advanced ultrasound computing architecture. Up to 20 volumes per second are recorded and displayed to give a real time view inside the body at a full opening angle of 80 degrees (azimuth and elevation). An easy to use user interface in combination with a decision-support process provides the possibility for a rapid and automated diagnosis of internal injuries like bleeding or facilitates image guided surgery. The objective of this paper is to describe the technological implementation of PUDIS concerning the overall operating procedure, the computing architecture, the probe with multiplexer, the adaptive beamforming method and the automated diagnosis procedure.

1.0 INTRODUCTION The rapid diagnosis of invisible internal injury in an austere and hostile front-line operational environment remains a challenge for medical, search and rescue personnel. The availability of a portable 4D-ultrasound imaging system with a single probe, providing high image resolution and deep penetration, is considered by medical practitioners and their military health services counterparts as exceedingly helpful, if not essential in supporting triage and medical decisions to save lives. However, portable and easy-to-use 4D non-invasive medical imaging systems are not yet commercially available, primarily because of unresolved major technological and engineering challenges. Available portable ultrasound systems only provide 2D images, requiring a medical professional to mentally integrate multiple images to develop a 3D impression of the scanned objects. This practice is time-consuming, inefficient, and requires a highly skilled operator to administer the scanning procedure. On behalf of Defence R&D Canada (DRDC) we develop a Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS, figure 1) to address the above-mentioned challenges.

Figure 1: 3D/4D PUDIS (left), 2D array probe with cable (right),

STO-MP-HFM-239

8-1

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS)

2.0 PUDIS - OVERALL OPERATING PROCEDURE With PUDIS a conical volumetric segment is imaged with an opening angle of 80x80 degrees to a maximum depth of 24 cm (sample volume) with an angular resolution of 0.5 degrees and a rate of 20 volumes per second (Vps). A 2D-phased array probe with 32 x 32 single elements working at a centre frequency of 2.5 MHz is used. All 1024 elements are active during the transmit phase. During the receive phase a group of 256 elements are connected to the 256 electronic channels of the system. Four shots are necessary to acquire the whole volume of interest according to figure 2 with all 1024 elements. Switching of the elements between transmit and receive phase and the correct choice of the relevant receive elements is done by a multiplexer which is integrated into the probe. The 256 received A-Scans of one shot are digitized (sampling rate 25 MHz, resolution 14 bit, length 4096 samples).

Figure 2: Volume acquisition using 4 snapshots.

After four shots, the 1024 A-Scans of one sample volume are processed (filter, adaptive beamforming, scan conversion) and transmitted to a monitor, where they are displayed in a 3D representation. This is done 20 times per second so that a quasi-real-time 4D imaging is provided (4D = 3D spatial + 1D temporal). The monitor shows also the graphical user interface and the result of the automated detection of free fluid (intraperitoneal free fluid, blood) inside the sample volume. In this first implementation, the automated diagnosis is restricted to detect free fluid in a characteristic region called Morrison-Pouch, which is the space that separates the liver from the right kidney (figure 3). If a person is lying on the back this is the deepest region inside the abdominal cavity where fluid collects in the case of an internal injury with bleeding.

Figure 3: Morrison-Pouch (left), with free fluid.

Due to its 4D capability, use of the FAST procedure (Focused Assessment with Sonography for Trauma, [7]) and the adaptive beamforming, PUDIS only requires placing the ultrasound probe at fixed anatomical positions, which is easy to learn even for people without special knowledge in ultrasound diagnosis. The 8-2

STO-MP-HFM-239

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS) relevant positions are displayed in the graphical user interface and if no reliable detection of Morrison-Pouch is possible, the user is guided by the system to change the placement of the probe appropriately. As soon as a reliable detection is possible, the detection of free fluid is done automatically. Beyond the functional advantages of PUDIS (easy to use, automated detection of free fluid), the system is characterized by its technical features compared to other existing systems, as will be described below.

3.0 PUDIS – COMPUTING ARCHITECTURE To provide a high level of flexibility and accuracy, the computing architecture implemented in PUDIS utilizes a fully digital data acquisition and data processing logic. During the transmit phase, the digital-to-analog converter (DAC) generate a complex set of 256 engineered signals that are transmitted into the human body. In four consecutive transmit-receive cycles the 256 analog-to-digital (ADC) converters collect the data from all 1024 elements of the connected probe. During the acquisition cycle, the system has to collect and process a volume of 140 MB raw data per second. In order to create a 3D volume, the collected raw data is processed. According to the decomposed 3D adaptive beamforming algorithm (5.0), in the frequency domain, beams are calculated along both dimensions of the 2D array of the probe. After calculating the beams in column direction of the array (‘Csteer’), the results are sorted and distributed for calculation of beams in the second dimension (‘Rsteer’). The output of the Rsteer is compiled and displayed as a 3D volume on a screen. Figure 4 picture illustrates the processes.

Figure 4: Illustration of data processing. The collected data is processed (‘steered’) in the frequency domain.

To facilitate the implementation of the above process, PUDIS uses a highly parallel computing architecture built around eight Field Programmable Gate Arrays (FPGA). Four Virtex 6 FPGAs (XC6VLX240, Xilinx) interface to the 256 DACs and 256 ADCs and implement the processing from acquisition to Csteer. The Rsteer and remaining IFFT are implemented on four Virtex 6 FPGAs (XC6SLX315, Xilinx). The graphical display and user interface are facilitated by using an embedded computer module (PC). To provide a fast exchange of data between the eight FPGAs, sixteen high speed links connect each of the four FPGAs in the first stage with each FPGA in the second stage, delivering a bandwidth of over 60 Gbis/s. A data exchange between FPGAs in the same stage is not required. The connection between the second stage and the embedded PC are four single (x1) PCEexpress-Interface lanes (PCIe). A schematic representation of the system with its different stages is shown in figure 5.

STO-MP-HFM-239

8-3

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS)

Figure 5: Schematic overview of the processing stages and their connections.

As depicted in figure 5, the first stage consists of four identical acquisition units. For the PUDIS prototype, these units are realised by four modules of printed circuit boards that provide the high voltage (HV) transmit stage and the signal conditioning during the acquisition phase. The printed circuit board for the second stage is located on top of the above mentioned four modules and connects them electrically and mechanically. It also provides the PCIexpress connection for the embedded PC and the system’s power supplies. A picture of the PUDIS computing architecture is shown in Figure 6.

Figure 6: Picture of PUDIS' Computing Architecture.

Due to its modularity, PUDIS’ computing architecture is flexible and scalable. The processing units (FPGAs) are In System Programmable (ISP) and, based upon the application, the number of modules can be extended or reduced.

8-4

STO-MP-HFM-239

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS)

4.0 PUDIS – 2D PLANAR ARRAY The 2D planar probe is a phased array probe with 32 x 32 elements according to lambda-half approach. Figure 7 shows the acoustic block after structuring and the connection layer, sputtered with gold before the ceramic layer (PZT-layer) has been attached (glued) to it. Structuring has been done by sawing. The element pitch, i.e. the distance between the middle of two neighbouring elements, is 308 µm (half of the wavelength in water at 2.5 MHz), the gap between two neighbouring elements, i.e. the cutting width, is 50 µm. Thus, the active aperture of the probe is 1 x 1 cm (figure 8, the black square in the picture on the right).

Figure 7: 2D planar array: left, acoustic block with flex-tapes, right, sputtered connection layer.

The cable of the probe consists of 300 lines used for driving the elements (256) and for controlling the multiplexer (figure 8, left). The multiplexer consists of eight boards and the connection carrier for the cable. The overall size of the hand piece of the probe is 8 x 12 x 13 cm. The housing of the first implementation was 30 % bigger for a future implementation further reduction by using ASIC based components is planned.

Figure 8: 2D planar array: left, multiplexer and cabling, right, housing with aperture.

The advantage of the adaptive beamforming algorithm used for PUDIS is, that compared to conventional beamforming the resolution of the image increases by a factor of two. For the same resolution, a system with conventional beamforming needs a probe that has twice as much elements, i.e. 64 x 64 instead of 32 x 32 [1]. The requirements of the algorithm to the PUDIS probe are: -

Lambda-half-pitch (see above).

-

Cross talk between two neighbouring elements as good as possible.

-

Bandwidth as much as possible.

STO-MP-HFM-239

8-5

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS) For the cross talk -40 dB have been specified for the array, the current probe set-up is at – 35 dB (figure 9) which is much better than reached by a conventional probe . The average bandwidth of the probe fulfils the requirements of 70 % of the centre frequency (figure 10, from 1.6 MHz to 3.4 MHz). The cable of the probe is connected to the system by a conventional zero injection force connector (Cannon ITT 0418, ZIF) shown in figure 11.

Figure 9: Measurement of the cross coupling between neighbouring elements with a laser vibrometer.

Figure 10: Measurement of the bandwidth with pulse echo measurements. The selected element of the array has a bandwidth of 75 % at 2.5 MHz.

Figure 11: ZIF connector with opened housing.

8-6

STO-MP-HFM-239

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS)

5.0 PUDIS – ADAPTIVE BEAMFORMING In ultrasound imaging systems the angular resolution provided by conventional beamformers is determined by the size of the aperture, and by the frequency of the received signals. Since the operating frequency is usually fixed, only the aperture size can be eventually increased by a higher number of elements. This leads to more complex hardware and software implementations. The alternative is to employ an adaptive beamforming method (figure 12). Adaptive beamformers are designed to maximize signal detection, while minimizing the beam-width, and suppressing the side-lobes. The convergence time of the specific method adopted allows for real-time imaging. The method uses a combination of the Sub-Aperture pre-processing scheme and a space-time statistic to reduce the degrees of freedom required by the algorithm [6]. The beamforming method used in PUDIS is based on the class of Linear Constrained Minimum Variance (LCMV) adaptive beamformers [2], [3], [4], [5]. The way how it is implemented in PUDIS is proprietary to DRDC. The theoretical background and the decomposition of the complex 3D beamforming algorithm into two linear steps, that can be implemented on an appropriate parallelized computational architecture, are described in [6].

Figure 12: 3D image reconstruction from the output of DRDC’s proprietary beamformer for a planar 2D probe with 32 x 32 elements (simulated data). Left hand side is for a conventional beamformer and right hand side is for the 3D adaptive beamformer. The improvement in image resolution is equivalent with that provided by an equivalent larger planar array by a factor of 4.

Beyond the hardware requirements (parallel computing architecture) there are special requirements concerning the probe. High bandwidth and low cross talk are necessary to keep the convergence time short. PUDIS has both, the appropriate computing architecture (3.0) and an appropriate 2D planar probe (4.0).

6.0 PUDIS – AUTOMATED DIAGNOSIS The key feature for its use in an austere and hostile front-line operational environment by non-specialists is the guided probe positioning and the automated diagnosis of free fluid in the Morrisson-Pouch. For PUDIS the intraperitoneal free fluid detection is performed in several pipeline steps (see figure 14). To show the feasibility of automated diagnosis on ultrasound images using the FAST exam protocol (7.0) the focus was set on the perihepatic (RUQ) view. Here the right liver lobe and kidney as well as the diaphragm and pleura are visible on the exam recording (figure 13). To collect high quality images with fewer shadow artefacts previously developed shadow detection method for ultrasound was employed [8]. The guided probe placement can be adjusted by the operator based on the shadow detection findings. The development of the automatic free fluid detection resulted in two entirely different methods. One method utilizes rather global image information like the image intensity values and the result of the shadow detection. The methods result STO-MP-HFM-239

8-7

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS) is a shadow and fluid confidence map, which can be seen in figure 13.

Figure 13: Axial and sagittal cut of the 3D ultrasound (top) and the corresponding confidence map. The result of the shadow and fluid detection shows tissue (white), fluid (light grey) and shadow (grey) locations.

The second method is based on the visible image features for the chosen FAST view. The typical image features are generated by the organ structures through ultrasound reflection, refraction, absorption and so on. Detecting the organ locations can determine the image regions associated with fluid accumulation. For the RUQ view this is the Morrison-Pouch, which is located between the surfaces of the kidney and the liver. Therefore, a kidney and liver region detection was developed to determine the Morrison-Pouch’s position adjacent to both organs. The kidney detection can further be utilized to determine the correct probe placement for the RUQ view (figure 14). The X- and Z-axis centre should be considered as an optimal kidney recording position during probe placement, as all surrounding organ structures as well as the Morrison-Pouch are included in one single 3D acquisition.

Figure 14: Free fluid detection pipeline.

The actual kidney detection was performed on a downscaled version of the recorded image to speed up the detection by over 90%. To find the kidney position the renal cortex and renal sinus are utilized as detectable kidney features. The feature extraction is performed by applying a slice by slice scan line algorithm on the 3D data. Here, searching for bright and dark tissue transitions, results in the identification of all renal cortex 8-8

STO-MP-HFM-239

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS) candidates. All implausible e.g. single transition candidates are removed. Following, all remaining candidates are accumulated to determine the kidney position as the candidates’ centre of gravity. Additionally, a multi-step kidney segmentation was developed to determine the kidney surface in liver direction. For now this algorithm can only be applied on 2D images. In the pipeline it is used on the candidate slice with the highest intensity variance. The second organ delimitating the Morrison-Pouch is the liver. The liver region was extracted using the highly visible vessel structures as features. The vessel highlighting algorithm of Sato et al. was employed [9] to generate a likeliness representation of all vessel structures in a so called vesselness image. An enhancement of Drechsler et al. [10] was included in the vesselness generation to combine different vessel scales, resulting in an improved vessel highlighting. Following, the binary vessel segmentation is carried out on the vesselness image. Here, an iterative connected component analysis (CCA) is used to achieve the segmentation. The required seed point placement for the CCA is performed by applying a windowing technique on the vesselness values to insert the initial seeds only in high vesselness areas. The segmentation results are further enhanced by the use of a level set method (LSM) as described in [11]. To extract the liver parenchyma, which is entangled by the vessel structures, the convex hull on the vessel segmentation is generated by applying the quickhull algorithm [12]. As a result we obtain a segmentation of a large portion of the visible liver.

Figure 15: Detected organ regions. Kidney (red, bottom), liver (white, top) with vessel structures (red).

After the successful detection of both organ positions it is possible to detect intraperitoneal free fluid in the Morrison-Pouch between both partial organ segmentations (see figure 15). This can be achieved by applying a basic thresholding in the organ boundary restricting region or by the comparison of fluid regions in the previously generated confidence map with the detected Morrison-Pouch location.

7.0 PUDIS – USER INTERFACE AND MOBILITY The three main concepts of PUDIS are: -

It is mobile (front line operational-environment).

-

It can be used by operators that do not have special skills in ultrasound diagnosis.

-

The detection of free fluid inside the body is done automatically.

STO-MP-HFM-239

8-9

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS) The mobility of the system is given by its size, weight and robustness of assembly, which were not the focus of the development until now. The automated detection of free fluid is described under 6.0. To make the system easy to use and to provide the possibility of a remote, wireless communication of the system, e.g. with a hospital, the TeleConsult software has been implemented. It provides the graphical user interface (GUI) for PUDIS and is an advanced telemedicine software combining 4D diagnostic ultrasound imaging features with additional telemedicine capabilities, although this is not yet used by PUDIS (figure 16). Putting the focus on usability, TeleConsult has been developed by following a human-centered design process. With its intuitive and easy to use GUI and the FAST (Focused Assessment with Sonography for Trauma, [7]) workflow support, TeleConsult enables even operators not familiar with ultrasound and without advanced diagnostic skills to acquire 4D ultrasound images for fast and reliable detection of internal bleeding.

Figure 16: The TeleConsult GUI. Left, screenshot of the interface with the ultrasound data and the operating panel. Right, the user guide for the correct placement of the probe.

8.0 PUDIS – CONCLUSION AND OUTLOOK PUDIS is a 4D portable ultrasound system that can be used in an austere and hostile front-line operational environment for the diagnosis of free fluid inside the body. The implementation of a guided easy-to-useconcept (FAST) together with the ability to detect free fluid inside the body make it a valuable instrument even for people without deep skills in ultrasonic imaging. The highly parallelized hardware in combination with a 2D planar phased array enables the implementation of a 3D adaptive beamforming algorithm that improves the imaging performance compared to existing systems with the same probe configuration. Until now the system has been developed and tested in a laboratory environment, the next step is its evaluation during clinical tests. Further improvements in future activities will be the reduction of the size of the probe (implementation of ASIC based multiplexer) and further support of mobility (telemedical communication, battery based operation).

8 - 10

STO-MP-HFM-239

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS)

9.0 ACKNOWLEDGEMENT PUDIS was developed on the basis of DRDC’s proprietary 3D adaptive beamforming method. The hardware was developed by Indecexon (computational architecture), Fraunhofer IBMT (probe and multiplexer), Medcom (TeleConsult) and Fraunhofer IGD (automated diagnosis). The project was funded by DRDC under the contract number W7719-115038.

10.0 REFERENCES [1]

S. Stergiopoulos, “Digital Ultrasound 3D/4D Imaging Technology”, Chapter in 2nd Edition, Handbook on Advanced Signal Processing for Sonar, Radar and Non-Invasive Medical Diagnostic Systems, Editor: S. Stergiopoulos, CRC Press LLC, Boca Raton, FL, USA, pg: 457-498, March 2009

[2]

Stergios Stergiopoulos and Amar Dhanantwari, "High Resolution 3D Ultrasound Imaging System Deploying a Multi-Dimensional Array of Sensors and Method for Multi-Dimensional Beamforming Sensor Signals" Assignee: Defence R&D Canada, US Patent: 6,482,160, issued 19 Nov. 2002.

[3]

Stergios Stergiopoulos and Amar Dhanantwari, "High Resolution 3D Ultrasound Imaging System Deploying a Multi-Dimensional Array of Sensors and Method for Multi-Dimensional Beamforming Sensor Signals" Assignee: Defence R&D Canada, US Patent: 6,719,696, issued 13 April 2004.

[4]

S. Stergiopoulos, “Advanced Beamformers”, Chapter in 2nd Edition, Handbook on Advanced Signal Processing for Sonar, Radar and Non-Invasive Medical Diagnostic Systems, Editor: S. Stergiopoulos, CRC Press LLC, Boca Raton, FL, USA, pg: 79-146, March 2009.

[5]

Stergiopoulos S., "Implementation of Adaptive and Synthetic Aperture Beamformers in Sonar Systems", Proc. of the IEEE , 86 (2), 358-396, Feb. 1998.

[6]

A. Dhanantwari, S. Stergiospoulos, “Adaptive 3D Beamforming for Ultrasound Systems Deploying Linear and Planar Phased Array Probes”, IEEE Conference Proceedings, IEEE International Ultrasonics Symposium, Honolulu Hawaii, October 5-8, 2003

[7]

http://www.sonoguide.com/FAST.html

[8]

Noll Matthias, Puhl Julian, Wesarg Stefan: Enhanced Shadow Detection for 3D Ultrasound, Bildverarbeitung für die Medizin 2014; 234-239

[9]

Sato, Yoshinobu ; Nakajima, Shin ; Atsumi, Hideki ; Koller, Thomas ; Gerig, Guido; Yoshida, Shigeyuki ; Kikinis, Ron: 3D Multi-Scale Line Filter for Segmentation and Visualization of Curvilinear Structures in Medical Images. Version: 1997. In: Troccaz, Jocelyne (Hrsg.) ; Grimson, Eric (Hrsg.) ; Mösges, Ralph (Hrsg.): CVRMed-MRCAS’97 Bd. 1205. Springer Berlin / Heidelberg, 1997. – DOI 10.1007/BFb0029240, 213–222

[10] Drechsler, Klaus ; Oyarzun Laura, Cristina: A Novel Multiscale Integration Approach for Vessel Enhancement. In: Dillon, Tharam et al. (Hrsg.) ; IEEE Computer Society Technical Committee on

STO-MP-HFM-239

8 - 11

Portable 3D/4D Ultrasound Diagnostic Imaging System (PUDIS) Computational Medicine (Veranst.): Twenty-Third IEEE Symposium on Computer-Based Medical Systems 2010 - CBMS 2010 [11] Keil Matthias: Ultraschallbasierte Navigation für die minimalinvasive onkologische Nieren- und Leberchirurgie, TU Darmstadt: 2013 [12] C. Bradford Barber and David P. Dobkin and Hannu Huhdanpaa: The Quickhull algorithm for convex hulls, ACM TRANSACTIONS ON MATHEMATICAL SOFTWARE, 22,1996; 469-483

8 - 12

STO-MP-HFM-239