CARS 2002 – H.U. Lemke, M.W. Vannier; K. Inamura, A.G. Farman, K. Doi & J.H.C. Reiber (Editors)  CARS/Springer. All rights reserved.

1

Detailed Image Classification Code for Image Retrieval of medical Images (IRMA) Berthold B. Weina, Thomas Lehmannb, Daniel Keysersc, Henning Schuberta, Michael Kohnena a Department of Diagnostic Radiology, Medical Faculty b Institute of Medical Informatics, Medical Faculty c Computer Science VI, Computer Science Department Aachen University of Technology (RWTH), Aachen, Germany

Abstract To support the automated classification of medical images an easy to use, tree-based, detailed examination code was developed. It consists of three parts: 1. technical code, 2. anatomical code, and 3. orientational code. The code was applied to about 6000 radiological images from daily routine, creating a well indexed database for testing of classifications. The code has been shown to be sufficient for the description of the images concerning image content. Keywords: Image retrieval, medical images, detailed classification code.

1. Introduction For real life medical use of Image Retrieval in Medical Applications (IRMA) different actions have to be undertaken in order to receive medical meaningful computer assisted analysis of images. The first of them are the classification of the image, the analysis of image patterns, the reasoning of deviations of the isolated patterns from normal, all under taking into account the location of the body region and the imaging technique. The last item will be the basis for the comparison of different feature vectors, resulting from the image recognition procedures, returning the most likely reference images. In daily routine this has to be done by an image analyzing program. The process of automated classification of medical images, which will result in the computerized extraction of the necessary information out of the image, should return a description code readable by man. This description code has to be as complete as possible and should be easy to use. To reach this goal, the reverse approach has to be performed, where the code is assigned to images by human readers and is used for assessment and improvement of the analyzing computer programs.

2. Methods The methods describe the principals of the codification scheme and the rough layout of the supporting software-tool to manually enter the code data.

CARS 2002 – H.U. Lemke, M.W. Vannier; K. Inamura, A.G. Farman, K. Doi & J.H.C. Reiber (Editors)  CARS/Springer. All rights reserved.

2 2.1 The classification code Starting with a systematical analysis of medical imaging procedures the technical code (part one of the classification code) was developed. It was specifically refined for radiological images. The description of this first part consists of a hierarchical code with a depth of 5 steps, naming the imaging system, modality, technique, subtechnique, and a modulator item. All of those technical items have an impact on image analysis or image interpretation. Every item of the code is represented by one digit or character.

Fig. 1: A webbased system for assigning the classification code to the images in the cases database. For easy choise the pop-up menues are installed helping to compose the appropriate code. The radiological report is also available to judge for specific comments.

The next part of the code shows the anatomical codification. These codes have to consider that both an antomical region and a functional anatomical structure have to be identified for the examination. The region is derived from subsegmentation of the human body, the functional code shows hierarchical ordering of the organic systems of the body. Both techniques together allow systematic and complete description of the point of interest for the imaging procedure.

CARS 2002 – H.U. Lemke, M.W. Vannier; K. Inamura, A.G. Farman, K. Doi & J.H.C. Reiber (Editors)  CARS/Springer. All rights reserved.

3 The third part of the code consists of orientation, functional add-ons and external markers. This code was created in view of the existing DICOM orientations, but also of the variety of radiological examinations. This code is mainly used for the radiological examinations. Other medical disciplines have to create their own settings. This part of code allows to estimate deviations of the normal projections and may also explain the problems of the automated image classification. 2.2 The webbased system Secondarily digitized images were converted to readable icons of about 200*200 pixels size and were labeled with the appropriate code by two professional readers, i.e. board certified radiologists. This was necessary for the assessment of the quality of computerized image categorisation and also to create a gold standard. The labelling tool was based on an SQL-database (postgres), an interface to a webserver (apache with builtin php-interpreter) and on client-site on an internet browser. All images were listed by their numbers, showing up one by one. The appropriate code was entered and remarks of the readers were collected (see Fig.1). The code could be entered directly into the appropriate field, if known by heart, or could be composed – making extensive use of javascripting in the HTML-file to enable modifying pop-up-menues according to the decision made at higher level (on lefthand side). This resulted in a stepwise refinement of the code. The appropriateness and feasibility of the classification code was estimated on the base of the listed remarks.

3. Results In the following examples for the code representations are given. The complete code will be available by the author. The technical part of the five steps classification code has now 8 representations on the first level: 1. X-Rays, 2. Ultrasound, 3. Magnetic Resonance, 4. Nuclear Medicine, 5. Optical Techniques, 6. Biophysical Techniques, 7. Varia, 8. Secondary Captured Images. The first step is subdivided into 6 further entities, which distinguish different X-raymodalities: 1. projection radiography, 2. fluoroscopy, 3. angiography, 4. computed tomography, 5. quantitative dual energy absorption measurement, 6. megavolt images (radiation therapy). The projection radiography is subdivided in the next step into digital and analog projection radiographs, X-ray stereometry and –graphy, and tomosynthesis. The digital projection radiography shows the following subdivisions: 1. tomography, 2. high-kV-beam, 3. low-kV-beam, 4. high-distance-imaging, 5. original size, 6. dual energy imaging. As modifier only small focus is available. Defined as such, the complete technical code consists of maximum 5 digits. At any position ‘0’, i.e. ‘not further specified’, can be used or omitted. The organ system is also hierarchically organized: starting with the main topological order 1. total body, 2. head, 3. spine, 4. upper extremity, 5. chest, 6. breast, 7. abdomen, 8. pelvis, and 9. lower extremity further refinement is given in nearly every topic. Chest is subdivided into 1. bone, 2. lung, 3. hilum, 4. mediastinum, 5. heart, and 6. diaphragm.

CARS 2002 – H.U. Lemke, M.W. Vannier; K. Inamura, A.G. Farman, K. Doi & J.H.C. Reiber (Editors)  CARS/Springer. All rights reserved.

4 Bone is further subsegmented into 1. clavicula, 2. sternoclavicular joint, 3. sternum, 4. upper ribs, 5. lower ribs. Nearly each level shows a further drill down possibility like 4. upper extremity Æ 4.6. shoulder Æ 6.6.3 acromioclavicular joint. This code also allows anatomically mixed regions like 4.4 ellbow, regions, which practically exist quite often in plain radiography. This three-step approach guarantees a sufficient resolution of the anatomical code to be fine enough for the definition of imaging needs. Similarily the hierarchical order is organized for the functional structures. Starting on top with the division of the body into the main organisational components: 1. cerebrospinal system, 2. cardiovascular system, 3. respiration system, 4. gastrointestinal system, 5. uropoetic system, 6. reproductive system, 7. musculosceletal system, 8. endocrinal system, 9. immuncellular system, and 10. dermal system. Subdivision will be shown at the example of the gastrointestinal system, presenting 1. oropharynx, 2. oesophagus, 3. stomach, 4. small bowel, 5. large bowel, 6. appendix, 7. rectum/anus, 8. liver, 9. gallsystem, a. salivary gland. As third hierarchic step the small bowel is subdivided into duodenum, jejunum, ileum, and terminal ileum. All of those have their impact concerning diseases or radiological specificities. This hierarchical code could easily be extended towards substructures and subfunctionalities to even fulfil higher requirements concerning functional resolution or anatomical details. The third code section describes the orientation, functional technique, and external markers. The orientation code has to be given in two signs, the first will be 1. coronal, 2. sagittal, 3. transversal, 4. oblique, and 5. RSA, the second will be a more detailed description of the first one. For example: sagittal orientation will be divided into the suborientations 1. “lateral, left first”, 2. “lateral, right first”, 3. “mediolateral”, 4. “lateromedial”. If no further specification is used, the number ‘0’ has to be taken to fill up the two places completely. The orientation will be accompanied by functional descriptors, which will describe the position of the patient regarding the examination technique and the action taken by the patient or the examinator. This position is an open list with 18 instances at the moment, where are 1. inspiration, 2. exspiration, 3. valsalva manoever, 4. Hitzenberg’s sniffing, and 8. supine position, 9. prone position, a. lateral decubitus, and more. External markers will explain use of external forces like 1. external stress – subdivided to manual and mechanical stress – 2. drug application for heart frequency increase, 3. contrast administration. Reviewing the complete code, it consists of four parts, combining digits or characters in each section and separation the sections by ‘-‘. The example (Fig. 1) is represented by the following complete code: “1122– 500 – 3 – 1110” meaning “high-kV conventional plain radiograph, depicting at the location chest the respiratory system, with posterior-anterior coronal view in inspiration without additional external markers”, describing a normal paview of the chest. The coding support was easy to use in so far on about 6000 single images, the loading time for the small images (max 200 pixels either direction) was short, the support of the javascripts in case of not known combinations of the codification scheme was helpful.

CARS 2002 – H.U. Lemke, M.W. Vannier; K. Inamura, A.G. Farman, K. Doi & J.H.C. Reiber (Editors)  CARS/Springer. All rights reserved.

5 It turned out that rarely (less than 0,1%) the regional code is too limited to describe the body parts depicted on a radiograph. This is especially true for long extremity overviews. The extremity codes gives only the chance to make a very general or a highly specific localisation of a combination of two nearby regions.

4. Conclusion Up to 6000 images have been classified so far with the complete detailed image classification code and are used to test classification programs for correctness of the prediction of the examination parameters. It could be shown that the code is sufficient to describe more than 99% of the X-ray images completely. Due to the tree structure of the code extensions are manageable, if a new method shows up, resulting in new parametric images. The code was an excellent help in assessment of the classification methods developed and used in the IRMA-Project.

5. References For references see the URL of the IRMA-project: http://www.klinikum.rwth-aachen.de/webpages/MIB/mbv/projects/irma/irma.html

Acknowledgements This paper was granted by the Start-project / Aachen and the Deutsche Forschungsgemeinschaft / Bonn.