VISIBLE HUMAN DATASET COLOR IMAGE HISTOGRAMS: 3D WEB JUGGLER S. Bonacinaa,b, F. Menegonib, M. Masserolia,b, and F. Pincirolia,b,c a

Dipartimento di Bioingegneria, b Politecnico di Milano, Milan, Italy c Istituto di Ingegneria Biomedica, Consiglio Nazionale delle Ricerche, Milan, Italy [email protected] Abstract: In front of the relevant number of successful researches involving the Visible Human Dataset (VHD) data, today, we still have to deal with the best ways for a completely automatic detection of the anatomical structures present in the VHD thousands of images. In contouring the anatomical structures (on a single VHD slice, or on a chosen slice set) we believe that there is no way to obtain good results if the color characteristics of the images are not known in deep, with quantified details. A colorimetric characterization of the all about nine thousand VHD cryosectional cross-section color images of the Female and Male Datasets is described here. An on-line three-dimensional immersive environment to navigate through our colorimetric characterization is also introduced. Real-time analysis of color component characteristics of a user defined set of VHD images is now possible. This is a resource potentially useful to many developers working on the VHD raw data, and it could be used in medical education too. Introduction The Visible Human Dataset (VHD) consists of digital images of two cadavers, a male and a female. Male and Female datasets contain over one thousand of digital magnetic resonance (MR) images, about two thousand of computed tomography (CT) images and color pictures of anatomic serial sections: more than 5,000 anatomic cross-sectional images are in Female dataset, more than 3,700 - "classical" and the higher resolution - images are in Male dataset [1]. The contouring of relevant anatomic structures present in an image is one of the top problems regarding any anatomic image. For such a purpose, a number of different and complementary algorithmic approaches may be considered. For example, in [2] an image segmentation algorithm is described which segments anatomical structures independent of the variation in color contained in the anatomical structure. The generation of pseudo-radiographic images from the 24bit digital color images is also described. A color edge detection algorithm was proposed in [3]. It address the aspect of differential geometric. A semi-automated color segmentation algorithm for the lung extraction was developed in [4], it is a first attempt to attack the

complex issue of outlining fine anatomical structures in color medical images. It is based on repeatedly dividing an image into regions and classifying the regions based on experimental classification statistics. The technique combines an iterative automated approach using Voronoi diagrams with optional human refinement to extract anatomical structures from the Visible Human color cryosection data. This color-based method facilitates segmentation of soft tissues. The three hundred images involving lungs has been considered. A method for fine blood vessel 3D segmentation is presented in [5]. Authors found some radiometric inhomogeneities on the anatomical cryosection images of the pulmonary region. Discontinuous, inter-slice radiometric variations were corrected through the use of an adaptive correction propagated across a series of parallel slices. This study used the only 160 VHD images of the pulmonary region. The availability of preliminarily quantitative characterizations of the images can help in targeting better the contouring task. Sets of three red, green, and blue intensity histograms for each VHD color image, stored on a server and made accessible like the VHD images themselves, represent a useful potentially resource to many developers working on the VHD raw data. Furthermore, navigating through series of such histograms in a virtual three-dimensional (3D) immersive environment could augment understanding of color component VHD image characteristics. In this study we first describe an histogram-based colorimetric characterization of the all VHD 24-bit color images, that is our meta-database implementation. Then, we introduce the Dynamic Histogram Visualizer (DHV), an on-line interactive 3D environment to navigate through piled histograms obtained from VHD cross-section anatomic color images. Also DHV could be used in medical education. Materials and Methods The VHD anatomic color images In the Visible Human Male dataset there are 1,878 anatomic axial sections. Numbering was started at 1,001 and proceeded to 2,878. The anatomic slices were named as a-vmXXXX, where XXXX indicates the section number. They are spaced at 1.00 mm intervals

from head to toes. Anatomic slices missing due to kerf loss were indicated by the use of empty files as place holders [6]. Thus, there are 1,871 anatomic color images. Each of these images is composed by 2,048 x 1,216 pixels, with each pixel constituted by 24 bits of colors – a byte for each red, green, and blue component - and with square pixel size of 0.33 mm. Each image takes 7.125 megabytes, thus the whole set of images occupies approximately 13 gigabytes. In August 2000, the seventy millimeter film taken during the original data collection phase has been digitized at a resolution of 4,096 x 2,700 pixels, 24 bits of colors [1], and square pixel size of 0.144 mm. Also these images are spaced at 1.00 mm intervals from head to toes and are 1,871. Such images are referred as “higher resolution images” and takes about 32 megabytes each. They were named as XXXX, where XXXX indicates the section number. The whole higher resolution dataset requires about 60 gigabytes. Globally, in Male dataset the anatomic photographic images take 73 gigabytes. The Visible Human Female data set has the same characteristics as the Visible Human Male one with one exception. Its axial anatomical images were obtained at 0.33 mm intervals instead of 1.0 mm intervals. This resulted in 5,189 anatomical images [1]. Numbering was started at 1,001 and proceeded to 2,734. The anatomic slices were named as a-vfXXXXi, where XXXX indicates the section number, and the index i can be ‘a’, ‘b’, or ‘c’. Each of these images is composed by 2,048 x 1,216 pixels, with each pixel constituted by 24 bits of colors – a byte for each red, green, and blue component - and with square pixel size of 0.33 mm. Again, anatomic slices missing due to kerf loss were indicated by the use of empty files as place holders. Globally, in Female dataset the anatomic photographic images take more than 36 gigabytes. The Histogram meta-database design As any color image, each VHD color image can be represented according to a color space. A color space is a specification of a coordinate system and a subspace, within that system, where each color is represented by a single point [7]. The VHD color images are stored using the Red-Green-Blue (RGB) raw format. Histograms are the basis for numerous spatial domain processing techniques. The information inherent in histograms also is quite useful in other image processing applications, such as image segmentation. Histograms are simple to calculate in software and also lend themselves to economic hardware implementations, thus making them a popular tool for real-time image processing [7]. From each VHD color image, we obtained three histograms, one for each color component. To perform colorimetric characterization, i.e. color histogram construction, of the VHD color images, we have used the method shown in Figure 1. First, we downloaded all the color image raw files from the VHD Milano Mirror Site® ftp server [8] and stored them locally on a PC.

We developed a software agent that: first, automatically it detects VHD image size and format, the Realizer step; second, it loads a color image in the computer memory using dynamic memory allocation features, the Loader step; third, it divides each color image in three monochrome images, one for each color component, and then it performs the histogram computation, the Calculator step; fourth, it stores histograms on the computer hard disk, the Storer step. Histograms were stored on hard disk in two formats: binary to be read quickly by other programs, and eXtensible Mark-up Language (XML) format. Storing sequentially in a file the number of pixels for each intensity level Binary format was obtained.

Figure 1: The steps followed to perform VHD color image meta analysis. We named Histogrammer the method used to compute and store color component histogram from VHD color images. The XML format was obtained storing in a file into XML tags the number of pixels for each intensity level. For each image six files – three in binary and three in XML format – for a total of 48 KBytes were generated. The Dynamic Histogram Visualizer design The Dynamic Histogram Visualizer (DHV), is a Java-based software application using the Java 2 SDK [9], the Java 3D API [10], and the open source Visualization for Algorithm Development (VisAD) library [11]. Using Java Applet technology and HTML together with PHP Hypertext Preprocessor (PHP) scripting language an interactive Internet version of DHV has been implemented and published on the web. Results The Histogram meta-database The meta-database produced is intended as service for developers of applications based on the VHD data [12]. The time needed to perform the second and the third steps of the colorimetric characterization algorithm

explained above, varied from 15 to 30 seconds per image. The memory occupation of each histogram file was of 1 Kbyte in binary format, and of 13 Kbytes in XML format. We have maintained the original color image name in the dataset to naming histograms. Thus, the generic histogram is labeled by a tag containing the section number, the gender (male or female), the color component (red, green, or blue). For example, a_vm1301.raw_r_ist indicates the red color component histogram of the a_vm1301.raw male image; 1500.rgb_b_ist indicates the blue color component histogram of the 1500.rgb high resolution image of the male; avf1632a.raw_g_ist indicates the green color component histogram of the avf1632a.raw female image. All calculated histogram were upload to the VHD Milano Mirror Site® ftp server.

Java Virtual Machine, Java 3D API and VisAD library installed. Our system is easily maintainable and portable on the most common operating systems. Additionally it can work on widespread generation computer platforms, including computers with no special graphics hardware.

The Dynamic Histogram Visualizer The developed DHV tool is able to visualize hundreds of histograms at a time, see Figure 2. It provides features for viewpoint moving, manipulating, zooming, changing position and orientation of piled histograms in a 3D environment. By pressing and dragging with the left mouse button the user can move the histogram pile around, obtaining translations and rotations of the scene. By shift-clicking and moving the mouse up and down the user can zoom in and out the scene, so peak and valley positions become better understandable, see Figures 3 and 4. A web version of DHV, uniformly visualized on web browsers with different display abilities, is available at URL http://nestore.bioing.polimi.it/~DHV/.

Figure 2: Green component piled histograms for thirty section images, from the 1,001 to the 1,301 section with step 10, of the Visible Human Male dataset.

Discussion Histograms of each Red, Green, and Blue monochromatic component of each VHD photo image are the basic way to start, thus we performed a meta analysis of all VHD anatomic color images. For each image six files – three in binary and three in XML format – for a total of 42 KBytes were generated. We used C++ programming language and we used dynamic memory allocation and garbage collector features to maintain computational time limited. We make some attempts using an image processing software, but we obtained bad results: it took some of nine our to process an higher resolution images. Further the developed software agent is able to verify if the dimensions of image file to process are exactly those expected using the file name format. Histogram peak and valley 3D exploring provides significant information for further image characteristic studies, such as segmentation or color cluster discovery [13]. In contouring the anatomical structures of VHD images we believe that there is no way to obtain good results if the color characteristics of the images are not known in deep, with quantified details. The developed stand-alone application has been successfully tested on a variety of hardware and software platforms with a

Figure 3: The same histogram pile shown in Figure 2 has been rotated and translated. To better understand peak and valley positions Slices and Intensity axes have been scaled.

[3] ILOVICI I. (1999), ‘Detecting color edges in the visible human data-set’. Proc. of the 12th IEEE Symposium on Computer-Based Medical Systems. Stamford, CT, 1999, pp. 254-258 [4] IMIELINSKA C., DOWENS M., HOSAKERE S., W. YUAN (2000): ‘Semi-automated color segmentation of anatomical tissue’, Comput Med Imaging Graph, 24, n.3, pp. 173-180 [5] MÁRQUEZ J., SCHMITT F. (2000): ‘Radiometric homogenization of the color cryosection images from the VHP Lungs for 3D seg-mentation of blood vessels’, Comput Med Imaging Graph, 24, n.3, pp. 181-191

Figure 4: The same histogram pile shown in Figure 3 has been rotated and zoomed. Pixels axis has been scaled, so peaks are better visualized. Conclusions

[6] SPITZER V., ACKERMAN M.J., SCHERZINGER A.L., WHITLOCK D. (1996): ‘The Visible Human male: a technical report’. J Amer Med Informatics Assoc, 3, n. 2, pp. 118-130 [7] GONZALEZ, R.D., WOODS, R.E., (2002): ‘Digital image processing’. 2nd ed. (Addison-Wesley Publishing Company, Reading, MA, USA)

We have performed colorimetric characterization, by means of color histograms, of all the VHD color images. An amount of about 27 thousand histograms was generated, and are available as binary files and XML files to be downloaded from the VHD Milano Mirror Site®, for a VHD licensed user. Also, we implemented a Dynamic Histogram Visualizer tool able to let manage and navigate through color histogram-based knowledge of many VHD anatomic pictures at a time. A stand-alone version and a prototype web version, satisfying all visualization and portability requirements, are available. Intended as a developer oriented tool, our 3D environment provides capabilities for data mining of still VHD anatomic pictures, but it represents also an interactive training tool.

[8] The Visible Human Dataset Milano Mirror Site® [Online] 1997. Internet site address: http://vhdmms.cilea.it. Last access: February 2, 2004

References

[12] BONACINA S., MASSEROLI M., MENEGONI F., QUATTRONE G., PINCIROLI F. (2003): ‘A colorimetric characterization of the raw digital data of the Visible Human Dataset images’. In: MUSEN M.A., FRIEDMAN C.P., TEICH J.M., editors. American Medical Informatics Association 2003 Symposium Proceedings [CD-ROM], Omnipress, Madison, WI, p. 793

[1] ACKERMAN M.J. The United States National Library of Medicine. The Visible Human Project. [Online] 1995. Internet site address: http://www.nlm.nih.gov/research/visible/visible_hu man.html. Last access: February 2, 2004 [2] STEWART J.E., JOHNSON J.H., BROADDUS W.C. (1996): ‘Segmentation and reconstruction strategies for the Visible Man’. In: BANVARD R.A., editor. Proc. of The 1st Visible Human Project Conference [CD-ROM]; 1996 Oct 7-8; Bethesda, MD. National Library of Medicine (US), Office of High Performance Computing and Communications, Bethesda, MD, 1996

[9] Sun Microsystems. Java 2 Platform, Standard Edition. Internet site address: http://java.sun.com/j2se/. Last access: February 2, 2004 [10] Sun Microsystems. Java 3D API Home Page. Internet site address: http://java.sun.com/products/ja va-media/3D/. Last access: February 2, 2004 [11] VisAD Library Home Page. Internet site address: http://www.ssec.wisc.edu/~billh/visad.html. Last acc ess: February 2, 2004

[13] CHENG H.D., JIANG X.H., SUN Y., J. WANG (2001): ‘Color image segmentation: advances and prospects’, Pattern Recogn, 34, n. 12, pp. 2259-2281