IMAGING & THERAPEUTIC TECHNOLOGY

313

RadioGraphics

AAPM/RSNA Tutorial on Equipment Selection: PACS Equipment Overview General Guidelines for Purchasing and Acceptance Testing of PACS Equipment1 Ehsan Samei, PhD ● J. Anthony Seibert, PhD ● Katherine Andriole, PhD Aldo Badano, PhD ● Jay Crawford, BS ● Bruce Reiner, MD ● Michael J. Flynn, PhD ● Paul Chang, MD A picture archiving and communication system (PACS) is a comprehensive computer system that is responsible for the electronic storage and distribution of medical images in the medical enterprise. The system is highly integrated with digital acquisition and display devices and is often related closely to other medical information systems, such as the radiology information system or hospital information system. In the past few years, there has been continuous growth in clinical implementation of PACS to reduce costs and improve patient care, a trend that is expected to continue. However, a PACS is complex and costly to acquire, replace, maintain, and repair. To select a system that best meets their requirements, purchasers of PACS equipment need to be aware of the key characteristics and differing features of the various products. After the PACS has been installed, the user should perform technical and clinical acceptance testing to ensure that the system meets expectations. ©

RSNA, 2004

Abbreviations: CCD ⫽ charge-coupled device, CR ⫽ computed radiography, CRT ⫽ cathode-ray tube, DICOM ⫽ Digital Imaging and Communications in Medicine, DR ⫽ digital radiography, HIS ⫽ hospital information system, LCD ⫽ liquid crystal display, PACS ⫽ picture archiving and communication system, PCR ⫽ performance-to-cost ratio, RAID ⫽ redundant array of independent disks, RIS ⫽ radiology information system, TFT ⫽ thin-film transistor Index terms: Computers ● Images, storage and retrieval ● Picture archiving and communication system (PACS) RadioGraphics 2004; 24:313–334 ● Published online 10.1148/rg.241035137 1From

the Departments of Radiology, Physics, and Biomedical Engineering, Duke University, DUMC Box 3302, Durham, NC 27710 (E.S.); the Department of Radiology, University of California–Davis, Sacramento (J.A.S.); the Department of Radiology, University of California, San Francisco (K.A.); the Medical Imaging and Computer Applications Branch, Office of Science and Technology, Center for Devices and Radiological Health, Food and Drug Administration, Rockville, Md (A.B.); the Department of Radiology, Medical University of South Carolina, Charleston (J.C.); the Departments of Radiology, Veterans Affairs Maryland Healthcare System and University of Maryland School of Medicine, Baltimore (B.R.); Radiology Research, Henry Ford Health System, Detroit, Mich (M.J.F.); and the Department of Radiology Informatics, University of Pittsburgh Medical Center, Pittsburgh, Pa (P.C.). From the AAPM/RSNA Tutorial on Equipment Selection at the 2002 RSNA scientific assembly. Received May 28, 2003; revision requested July 16 and received August 27; accepted September 8. Address correspondence to E.S. (e-mail: [email protected]).

©

RSNA, 2004

RadioGraphics

314

January-February 2004

Introduction A picture archiving and communication system (PACS) is an inter- and intrainstitutional computational system that manages the acquisition, transmission, storage, distribution, display, and interpretation of medical images. As such, the system is highly integrated with the imaging operation of the radiology department and with image-based clinical practice. In the past few years, there has been continuous growth in the implementation of PACS, mainly due to potential advantages such as improved work flow, improved throughput and productivity, rapid, remote, and simultaneous access to image data, electronic archiving, the possibility of image manipulation, and cost-effectiveness, leading to overall better patient care. The growth has also been fueled by the transition from analog to digital in many acquisition modalities, mainly the radiographic modalities; reduction in the cost of computers; improvements of computational communications including networks and protocols; reduced cost and increased speed of electronic archival devices; and standardization efforts, most noteworthy those of Digital Imaging and Communications in Medicine (DICOM), a joint initiative by the American College of Radiology and the National Electrical Manufacturers Association for developing standards pertaining to PACS technology (medical.nema.org). Many technological and implementation hurdles for a fully integrated electronic practice have already been tackled. Although PACS can now be considered a mature technology, not all of the key issues have been resolved. Currently, a large number of vendors offer PACS products with different features and capabilities. PACS are often complex and costly to acquire, replace, maintain, or repair. Furthermore, the performance of a PACS can directly affect patient care and clinical flow. Thus, careful attention should be paid to the selection of a system that meets the needs and requirements of the user. As many medical centers are now switching from analog

RG f Volume 24

●

Number 1

practice (with film acquisition, printing, distribution, and storage) to PACS and electronic practice, the user needs to be aware of the key characteristics and differing features of various products. Furthermore, after the selection has been made and the product has been installed, the user should evaluate the actual performance of the system to ensure that the system meets expectations (1). The purpose of this article is first to describe the key specifications and the desired options and features of a modern PACS, including those related to the acquisition, archival, and display systems; radiology information system (RIS) connectivity; and enterprise distribution. Some key practical issues that should be considered in a PACS acquisition will be discussed. The article further describes methods for acceptance testing of PACS and concludes with a review of computer technology changes and their effect on PACS.

Key Components of a PACS A PACS has a number of physical components (Fig 1). At the front end, the digital acquisition devices are the sources of digital images that are distributed and stored by the PACS. They can include all digital modalities such as computed tomography (CT), magnetic resonance (MR) imaging, nuclear medicine, ultrasonography (US), digital radiography, digital angiography, digital fluoroscopy, and digital mammography. The first four modalities often output images in digital form. Projection x-ray radiography, traditionally performed with analog screen-film detectors, has made a relatively recent transition to digital; thus, in many installations, a transition to PACS parallels a switch from screen-film radiography to digital radiography. As such, digital radiography is discussed in more detail later. Digital angiography, digital fluoroscopy, and digital mammography are three modalities that are just starting to be incorporated into PACS. The network is the highway that provides the data connection between the various components of the PACS. Some common network technologies are listed in Table 1, which gives the maximum image transfer performance possible for

●

Number 1

Samei et al

315

RadioGraphics

RG f Volume 24

Figure 1.

Schematic of the physical components of a typical PACS. ICU ⫽ intensive care unit.

Table 1 Various Network Technologies, Bandwidths, and Typical Maximum Transfer Times Transfer Times Network Technology Modem T1 Ethernet Fast Ethernet ATM‡ Gigabit Ethernet

Bandwidth

Chest Radiograph*

Chest CT Scan†

56 kbits/sec 1.54 Mbits/sec 10 Mbits/sec 100 Mbits/sec 155 Mbits/sec 1 Gbit/sec

20 min 43 sec 6.7 sec 0.7 sec 0.4 sec 0.07 sec

2h 4.3 min 40 sec 4 sec 2.6 sec 0.4 sec

*8.4 Mbytes. †50 Mbytes. ‡ATM ⫽ asynchronous transfer mode.

RadioGraphics

316

January-February 2004

transferring typical digital images. Sometimes also associated with the network are intermediary computers for transferring images from modalities to the PACS or transferring information to and from the RIS. Although such devices might be necessary at this point to translate images from a certain vendor’s proprietary format into DICOM objects, they impose cost, space, and maintenance burdens and should ideally be eliminated. The database server, sometimes referred to as the image server, is the “brain” of a centralized PACS. The server, often a central computer with multiple high-speed central processing units (CPUs) and a large amount of random-access memory (RAM) and cache memory, is responsible for keeping track of the information, images, image attributes, and image locations. Some PACS implementations have two such units providing redundant functionalities to minimize possible technical or upgrade downtime. In a less common decentralized or distributed PACS architecture, the functionality of the image database server is distributed among multiple computers. The archival system is an important component of a PACS and is responsible for electronic archiving of image data. It often has multiple components of its own for short-term (and rapid-access), long-term (often slower access), and duplicate (often off-site for disaster recovery) storage of image data. The division of the system into short-term and long-term archives is primarily due to economic considerations, as the technology for the rapid access necessary for recently acquired images has been too expensive to allow use on a long-term basis. However, with the rapid drop in the cost of rapid-access storage technologies, the division between short- and long-term archives is diminishing. The archival system, which is a key purchase consideration when acquiring a PACS, is discussed in more detail later. The RIS is the system responsible for maintaining patient demographics, scheduling and financial information, and interpretations of examination results. (Interpretations of examination results are sometimes handled by a separate entity, the dictation system, which manages this information and provides it to the RIS.) Many RISs

RG f Volume 24

●

Number 1

were and are being developed and used independently of PACS primarily based on Health Level 7 (HL7), a standard communication language for medical information developed by the Healthcare Information and Management Systems Society (HIMSS) (www.himss.org). To avoid data redundancy and, consequentially, data inconsistencies, most PACS rely on the RIS as the primary source of patient information. As a result, the PACS does not work efficiently without a robust connection to the RIS. This connection is often made through a “broker,” where the relevant data are exchanged and translated (from HL7 to DICOM or vice versa). This connection is not ideal, as data inconsistencies can still occur and most image interpretation functions will require seamless functionality of two different systems. Intimately integrated PACS and RISs, a need recognized by many users, are now offered by several suppliers. Another key purchase consideration when acquiring a PACS, the RIS and its PACS connection, is discussed in more detail later. The soft-copy display workstation or system, the last element of the imaging chain, is perhaps the most visible “face” of a PACS. The soft-copy displays provide a dynamic and changeable presentation of the image data to the clinician. A softcopy display workstation has two major components: software and hardware. At the software level, the graphical user interfaces (GUIs) implemented by the manufacturer enable various functionalities necessary for viewing an image such as image query and retrieval, image display, and various image manipulation functions such as window-leveling and zooming. At the hardware level, two technologies are currently used for softcopy display of medical images: the cathode-ray tube (CRT) and the liquid crystal display (LCD). The soft-copy display station is a critical component of a PACS that can affect its acceptability and efficiency. Various display technology options and qualities are discussed later. The term PACS was originally thought to be synonymous with filmless. However, many PACS operations have been unable to eliminate the use of film; digital images are often printed on film laser printers for various reasons ranging from lack of soft-copy displays to the discomfort of some clinicians with the new technology. High-resolution digital printers are now available that use “wet” or newer “dry” technologies. Perhaps a

●

Number 1

Samei et al

317

RadioGraphics

RG f Volume 24

Figure 2. Schematic of a CR imaging system, screen, and scanner. The CR cassette is like a screen-film cassette. After the x-ray exposure, the imaging plate is processed in a reader to extract the latent image, scale the signal, and contrast enhance the output image. The latent image is erased and the plate is returned for reuse. ADC ⫽ analogto-digital converter.

necessity for now, film printing should be controlled and eventually eliminated to allow PACS to maintain its cost-effectiveness. Easy remote access to medical images by referring physicians is important for the success of a PACS and is essential if a fully filmless operation is desired. This is often provided by a Web server that can display patient images on Internet browser applications normally installed on desktop computers. When the intranet network services commonly available within a medical center are used, access to images with a Web client is relatively rapid and secure. However, use outside the medical center by using the Internet is currently limited by security and patient privacy considerations (encryption is a must), speed (bandwidth limited), image quality issues (display quality and possible degradation due to lossy compression), and the medical-legal implications of telemedicine. The options for remote access are further discussed in a separate section.

Digital Detectors: Computed Radiography and Digital Radiography Projection radiography generates 60%–70% of the imaging volume in a typical hospital-based

diagnostic radiology department. Thus, a key consideration for implementation of a PACS is the purchase and deployment of digital radiography devices. There are a variety of detectors available for acquiring large-area digital projection radiographs for diagnostic medical imaging. These are traditionally split into two broadly defined categories called computed radiography (CR) and digital radiography (DR). (This traditional distinction is rather misleading, as CR is also a digital radiography technology.) CR uses a photostimulable detector housed in a cassette that mimics a screen-film cassette. After x-ray exposure, a separate “reader” scans the imaging screen with a red laser beam, which stimulates the emission of blue light photons under the excitation of the laser beam (Fig 2) (2). A digital processor in the CR reader converts the captured luminescence into a corresponding digital image. Whereas the most common implementation of CR technology involves detector handling between two sequential stages of exposure and data

January-February 2004

RG f Volume 24

●

Number 1

RadioGraphics

318

Figure 3. Schematic of a DR imaging system based on a “flat-panel” detector. The “self-contained” detector acquires the transmitted x rays and electronically processes and contrast enhances the output image without user intervention. After readout and data transmission, the detector is ready for reuse. ADC ⫽ analog-to-digital converter, CCD ⫽ charge-coupled device, CMOS ⫽ complementary metal oxide semiconductor, ms ⫽ millisecond, s ⫽ second, TFT ⫽ thin-film transistor.

acquisition, DR represents a class of digital detectors that produce an image without subsequent detector handling, in which the latent x-ray image is converted to a digital image within a short interval after exposure (Fig 3) (3,4). The DR detector is further classified into “indirect” and “direct” acquisition types based on the x-ray to digital signal conversion process (Fig 4). Indirect DR devices initially convert x rays into light photons by using a scintillator and then into electrons via a light-sensitive photodiode to generate the digital signal. Direct DR devices convert the x rays directly into electrons and then to a digital signal by using a photoconductive material. Examples of indirect DR devices include scintillators optically coupled to charge-coupled device (CCD) cameras, tiled arrays of complementary metal oxide semiconductor (CMOS) sensors, or flat-panel thin-film transistor (TFT) arrays, in which the x-ray information is stored in an array of minute capacitors that are electronically scanned to produce a digital image (Fig 4). Table 2 lists the digital radiography technologies currently available. A wide range of system capabilities, functionality, and costs exist within each type of digital radiographic detectors.

CR is the most flexible digital detector in terms of positioning and use. Multiple rooms can be serviced with a single reader; however, at a busy site more than one reader is recommended for backup and redundancy. For high-throughput requirements, a system with an external stacker or automatic multiple-cassette reader or both should be considered. In general, DR has a higher throughput capability than CR because the image extraction from the detector occurs without user intervention (although some automatic “cassetteless” CR systems can achieve a similar throughput). Higher x-ray detection efficiency is achieved by DR systems, particularly the flat-panel systems with structured phosphors or thick photoconductors, which allow reduced patient dose for similar image quality. Unlike analog screen-film detectors, which are contrast limited in operation, digital acquisition devices are signal-to-noise ratio limited, which means that the image quality is usually dependent on the quantum statistics of the image formation process combined with contrast and spatial resolution enhancement methods. The inherent performance of a radiographic detector can be described by the concept of the detective quantum efficiency (DQE), the ability of a detector to yield high image signal-to-noise ratio per unit exposure. All digital radiography systems aim for high

RadioGraphics

RG f Volume 24

●

Number 1

Samei et al

319

Table 2 Current Digital Radiography Systems System

X-ray Converter

Detector

Computed radiography

BaFBr photostimulable phosphor

Cassette based (replacement for screen-film cassette) or cassetteless automated system

Direct radiography Indirect CCD Indirect CCD array Indirect CMOS* Indirect TFT Direct TFT

Gd2O2S phosphor CsI or Gd2O2S phosphor Gd2O2S phosphor CsI or Gd2O2S phosphor Amorphous selenium photoconductor

Optically coupled CCD camera Directly coupled, linear CCD array slot-scan Directly coupled, tiled CMOS array Directly coupled TFT array Directly coupled TFT array

*CMOS ⫽ complementary metal oxide semiconductor.

Figure 4. Acquisition methods of DR systems: cross section of a direct flat-panel system (a), cross section of an indirect flat-panel system (b), a lens-coupled CCD system (c), a fiberoptic-coupled CCD system (d), and a fiberoptic-coupled scanning array (e).

RadioGraphics

320

RG f Volume 24

January-February 2004

●

Number 1

Table 3 General System Attributes for CR and DR for Radiographic Applications System Attribute Relative cost Room integration System integration PACS integration* Image and patient throughput Positioning flexibility Replacement for screen-film radiography Ease of use by technologists Postexposure handling Targeted applications and use Zero frequency DQE† (%) Equivalent x-ray speed‡ Frequency with 0.1 MTF§ (mm⫺1) Detector element dimension㛳 (␮m) Detector element fill factor# (%)

CR

DR

DR (CCD)

Inexpensive to moderate Simple add-on Limited Good Low High Excellent

Expensive Complex to robust Full Excellent High Limited Good

Inexpensive to moderate Range of integration Limited to full Good to excellent Intermediate Limited Limited

Good Significant Portable, general and Bucky positioner 20–40 100–400 2.6–4.7 100–200 100

Excellent Little or none Bucky positioner

Excellent Little or none Bucky positioner

40–80 200–800 3.5–6.2 100–200 30–80

10–30 50–200 ⬃2.6 50–200 100

Note.—These are general descriptions of the technology as a whole and might not correctly categorize a specific imaging system in either a positive or negative way as described. *Related to issues pertaining to the DICOM DX object versus CR object. †DQE ⫽ detective quantum efficiency. At zero frequency, DQE is a measure of the information collection efficiency (ie, dose efficiency) of an imaging system. A larger value is better. ‡For a typical adult procedure such as chest radiography. Speed is the relative measure of screen-film sensitivity (eg, a 400-speed rare earth phosphor screen). A higher number means that a lower dose is required. §MTF ⫽ modulation transfer function. MTF is a measure of the ability of an imaging detector to reproduce image contrast from subject contrast at various spatial frequencies (ie, resolution). Reported values are spatial frequencies typically measured with 0.1 MTF. 㛳Length of the side of a square area over which information is averaged. #Ratio of the area of the detector element over which information is actively collected divided by the total area of the detector element. Areas that contain electronics (transistors, storage capacitors) do not capture and cause a loss of information. A larger value is better.

DQE and thus high spatial and contrast resolution at low radiation doses. Some digital systems do this better than others (4 – 6); however, cost, portability, handling, throughput, and practicality of use are issues that must be considered prior to purchase and implementation. The bottom line is that there is no single “one detector type does all” technology at this time. Generalized system attributes are compared in Table 3. Note, however, that generalization can be misleading as technologies continue to advance. Notable are improvements in detective quantum efficiency of CR systems with “dual readout” and with “structured” photostimulable phosphor materials. CR systems are available with faster scanning by using line-scan readout

technology with resultant high throughput capability, competitive with DR systems, and the CR system footprint continues to shrink, approximating DR form factors. “Portable DR” systems have also recently been introduced, which allow remote bedside imaging. In addition, system costs continue to fall in parallel with improved manufacturing yields and lower computer costs. Some key issues that should be considered in the purchase of digital acquisition systems are listed in Table 4, and the functional issues are detailed in Table 5. In terms of interfaces and integration into the PACS, digital detectors must be supported by (a) modality work list functionality (getting patient demographic information from the RIS); (b) scheduled work flow and technologist quality control image verification and correction capabilities; (c) DICOM information object descriptions (IODs) to transfer acquisition

RadioGraphics

RG f Volume 24

●

Number 1

Samei et al

321

Table 5 Desired Options and Functions for CR and DR Equipment Option or Function Modality work list Modality “performed procedure step” DICOM DX object Dose monitoring tools Quality control phantom Quality control software

PACS-RIS integration

Purposes Sends RIS patient demographic information to the modality; allows the technologist to improve work flow at the modality Allows modification of the work list to reflect actual performed examinations and changes during the imaging procedure; redirects and redistributes images as necessary when sending to the PACS Provides a robust description of image acquisition parameters via an interface to the x-ray generator; provides a more flexible DICOM structure Identifies under- or overexposed images on the basis of analysis of collected image data for feedback to the technologist and radiologist Allows image acquisition for evaluation of digital system performance with a radiographic phantom tuned for digital analysis Analyzes the digitally acquired phantom image to determine the optimal functionality of the digital radiographic system with automated software, logging of historical data, and “yes/no” operational status Provides seamless interoperability of the modality with patient scheduling, work flow, data acquisition, image display, and the image archive

Table 4 Issues to Consider before Purchasing Digital Radiographic Detectors Type(s) of digital detector(s) Number of systems needed Basic equipment costs Peripheral components necessary for integration with the x-ray generator Peripheral components necessary for PACS integration Considerations necessary to verify optimal performance Preventive maintenance needs Expected longevity Contingency backup plan for possible specific system malfunctions Specific applications* Robust PACS functionality† *For example, trauma, orthopedics multiple film studies, dental Panorex imaging, the operating room arena, and mammography. †For example, modality work list management interface, quality control workstation for technologists, DICOM conformance, and tight RIS integration.

technique parameters (such as kilovolt peak, milliampere seconds, and exposure time); (d) assistance to set up acquisition techniques on the xray generator; and (e) application of examinationspecific image processing. (Note: Scheduled work flow is a concept developed by the Integrating the Healthcare Enterprise [IHE] initiative of the Radiological Society of North America [RSNA],

DICOM, and the Healthcare Information and Management Systems Society [HIMSS]. This concept involves a seamless flow of information that supports efficient patient care in a typical imaging encounter. It specifies transactions that maintain the consistency of patient information from registration through ordering, scheduling, image acquisition, storage, and viewing.) Continuous optimal system calibration and operation require implementation of a quality control program that can be performed by technologists. A quality control phantom and evaluation software that allows simple image acquisition and verification should be provided. Regardless of the digital system to be implemented, it is imperative that the purchase includes a quality control phantom that provides a simple yet robust check of the imaging parameters, with automatic analysis of system functionality to minimal standards initially determined by baseline measurements. Because digital detectors can “hide” incident exposures, the provision of exposure information and database capabilities are extremely important considerations. In addition, preventive maintenance and local support service should be investigated and verified prior to signing a commitment for purchase. Overall, with current technology offerings, CR is less expensive; however, direct digital radiographic systems have distinct advantages in terms

January-February 2004

RG f Volume 24

●

Number 1

RadioGraphics

322

Figure 5.

Storage cost (a) and storage capacity (b) of magnetic disks over time. GB ⫽ Gbytes, MB ⫽ Mbyte.

of detection efficiency and throughput. Digital solutions are likely best accomplished with a complementary mix of technologies, with DR used in high-throughput rooms such as dedicated chest imaging rooms and CR providing flexibility for general radiography applications and portable imaging. Currently, the major hurdle is that of cost.

Archival Systems The cost and inefficiencies of image storage for PACS were once thought of as impediments to moving toward digital imaging departments (7). Trends in archival technology have shown the cost of digital storage media decreasing and the capacity increasing at increasing rates (Fig 5), whereas analog devices such as paper and film show little change or even an increase in cost (8). Improvements in storage devices, along with the use of intelligent software, have removed digital archiving as a major stumbling block to implementing PACS. Digital image archiving can be more efficient than the manual data storage of the traditional film file room (9). It is generally less people intensive and, therefore, less expensive and less subject to the errors in filing and lost images that often plague film stores. Electronic archives can improve the security of stored image data and related records and typically have an intelligent patient-centric system database, enabling easy retrieval of results of imaging examinations. Digital archiving also enables multiplicity of archiving at different sites, which is invaluable for possible disaster recovery. The digital storage media used in PACS today include computer hard drives or magnetic disks, which are often configured as a redundant array of independent disks (RAID); optical disks, magneto-optical disks, and digital video disks

Figure 6. Retrieval time versus capacity for magnetic disks, optical disks, and tape. MB ⫽ Mbytes.

(DVDs), which are often integrated for automated access by using a jukebox device; and various magnetic tape cartridge devices. The standard computer hard drive or magnetic disk, also known as a direct-access storage device, is the fastest medium from which to retrieve data. Presently available devices have a capacity of 10 –200 Gbytes and a rapidly decreasing cost. Although magnetic disk storage has typically been used for workstation local storage and short-term archive storage, continued decreases in cost indicate that magnetic disks may be used for long-term storage as well. When configured as a RAID, these devices can offer redundancy and lessen the potential for data loss. Optical disks and magneto-optical disks of the removable storage media class are typically stored in an automated media-movement device or jukebox, giving them total device storage amounts equal to hundreds of times the medium capacity. Optical disks and magneto-optical disks have higher capacity per cost than magnetic disks, typically gigabytes to tens of gigabytes yielding hun-

RadioGraphics

RG f Volume 24

●

Number 1

Samei et al

323

Table 6 Archival Technologies for Medical Image Storage, Timing Performance, Capacity, and Cost Timing Performance*

Technology Magnetic disk Optical disk Tape

1–50 msec Seconds to minutes 24 sec to minutes

Digital video disk RAID

Seconds 100–300 msec

Capacity

Cost† (dollars/ Gbyte)

Hundreds of megabytes to tens of gigabytes Gigabytes to tens of gigabytes, yielding terabyte devices Tens to hundreds of gigabytes, yielding tens of terabytes devices Gigabytes to tens of gigabytes, yielding terabyte devices

1.00 0.40 0.2

Tens to hundreds of gigabytes, yielding tens of terabytes devices

8.00

0.8

*Time for retrieval of a 10-Mbyte file. †Based on the technologies and costs in 2003.

Table 7 Key Considerations for Designing Medical Image Archives Features

Local Site Specifications

DICOM compliance Capacity

Procedural volume Amount of data per study type, number of images per study Performance (read/write Requested number of specifications) relevant priors Cost Legal storage life requirement Redundancy or backup Centralized or distributed architecture Database Tolerable compression levels ... Scalability

dreds of gigabytes to tens of terabytes total device storage, and have lower cost per capacity than RAID. Optical disks are also known as write-once, read-many (WORM) disks, with data permanently written on the disks. Magneto-optical disks are erasable reusable platters. Because of their slower performance and lower cost per capacity, optical disks and magneto-optical disks have been used for long-term permanent storage. Tape is also a removable storage medium typically kept in a jukebox or tape library. It has very high capacity, tens to hundreds of gigabytes or many terabytes per tape library, and low cost. However, it is a slower medium than optical disks (eg, in terms of random retrieval times) because of its sequential nature. Nevertheless, the performance of tape is competitive with that of optical disks for retrieval of large files; it is best used for disaster backup as well as for long-term permanent storage.

Newer technologies, such as digital video disks, which store gigabytes of data per disk, appear promising but have failed to move significantly into the medical arena because of their high cost and lack of a uniform standard. An emerging new digital video disk initiative, Blu-ray Disc technology, which uses blue-violet lasers, is promising much greater capacity (27 Gbytes vs 4.7 Gbytes for a standard 12-cm disk). Figure 6 and Table 6 summarize the performance, capacity, and cost specifications for each of the available archival media used today for medical image storage. To tailor an archive system to the needs and specifications of an individual healthcare enterprise, one must consider the expected procedural volume and the data per study type, along with the capacity, performance, and cost of various storage devices. If the PACS operates with a cached architecture in which data are automatically distributed to and stored at the display station, then the online storage capabilities should include space for maintaining all of the pertinent examination results for a given episode of current care (ie, 3 days for outpatients and 6 days for inpatients). Space for prefetched relevant historical examination results should also be included in the anticipated storage requirements. If the PACS operates as a cacheless centralized system, then it is important to have enough capacity to store the results of a patient’s clinical encounter on the server and ideally to have an expanding size to permanently store all data. For a centralized storage architecture, the local storage at each display station is typically minimal. Table 7 summarizes the key issues to consider when designing a medical imaging archival system.

RadioGraphics

324

January-February 2004

RG f Volume 24

●

Number 1

Table 8 Key Specifications with Available Ranges for PACS Display Systems Key Specifications Ergonomics Size (diagonal of active screen) Weight Support Physical specifications Pixel pitch Pixel array Refresh rate Geometrical distortion Antireflection treatment Interface Optical characteristics Luminance range Maximum luminance Luminance nonuniformity Gray-scale bit depth Chromaticity*

Large-area contrast ratio Phosphor (CRTs) Defective pixel count (LCDs)

Typical Range or Type 380–611 mm (15–24 inches) 20–60 kg Tilt base, swivel base, wall mounted 0.125–0.325 mm 1,024 ⫻ 1,280 to 2,048 ⫻ 2,560 50–75 Hz 1%–10% Multiple-layer coating, conductive layer Digital-analog, digital video interface 250:1, 800:1 250–850 cd/m2 10%–30% 8–10 bits (0.280, 0.304) for CRT phosphor P104 (0.256, 0.311) for CRT phosphor P45† (0.27–0.30, 0.28–0.31) for active-matrix LCDs‡ 250–850 P45, P104 5–30

*Expressed as (X, Y) CIE color coordinates. (CIE ⫽ Commission Internationale de l’Eclairage.) †Source.—Clinton Electronics, Loves Park, Ill (“CRT Design Guide”). ‡The chromaticity of an active-matrix LCD varies greatly with viewing angle. Within a 45° cone, the variations in (X, Y) CIE coordinates can amount to 0.1. The acceptance angle of the color-measuring device therefore plays a significant role in the interlaboratory reproducibility of the colorimetry.

Display Systems The paradigm shift from film-based to filmless imaging has redefined clinicians’ expectations for the processes of image display and interpretation. Image display in traditional screen-film operation is a fixed and static process, with the components consisting of the screen-film image and the view box. Clinicians have little input into the process of hard-copy image display, with limited capability for image enhancement after the acquisition process has been completed. Conversely, image display in PACS is a flexible and dynamic process whereby radiologists directly interact with the soft-copy image, which is displayed on a computer workstation.

The hardware component of a display system for PACS is composed of a display device and a display controller. The specifications given for a system are valid only for that particular combination of device and controller. In a PACS environment, display systems are used for different purposes including primary reading, consultation, and review. For the purpose of defining requirements, it is useful to identify the display system as a primary or secondary class device on the basis of its usage (10,11). In a general sense, key display specifications can be grouped into three areas: ergonomics, physical characteristics, and optical characteristics (Table 8). Some vendors provide optional features that can improve the usability and lifetime of the display system within a PACS environment. Such features include brightness stabilization, a self-recognizing format and scal-

●

Number 1

Samei et al

325

RadioGraphics

RG f Volume 24

Figure 7. Typical emissive structure of a medical CRT monitor shows the components involved in the generation and transport of light. Energetic electrons excite the cathodoluminescent phosphor, generating light, which scatters multiple times before it emerges to form the image in the screen. AR ⫽ antireflective. Figure 8. Cross section of an active-matrix LCD with an in-plane switching pixel design. The liquid crystal molecules rotate under the influence of the electric field (E) but always remain in the display plane. This arrangement improves the viewing angle performance but sacrifices brightness, which can be compensated for with a more powerful backlight. The thin-film transistor (TFT) is the switching element for the addressing of each pixel individually. a-Si:H ⫽ hydrogenated amorphous silicon, ITO ⫽ indium tin oxide.

ing, an internal sensor for automatic or remote calibration, power management tools, and an ambient light sensor (external or internal). Current display offerings are based on two competing technologies (12): the cathode-ray tube (CRT) and the active-matrix liquid crystal display (LCD). The former is a mature technology half a century old, based on the excitation of cathodoluminescent phosphors by energetic electron beams. Light is generated in an emissive structure, where it diffuses in a controlled manner until it emerges toward the viewer, forming the displayed image (Fig 7). During the past 25 years, great advances have been made in display devices based on liquid crystals. As opposed to the CRT

emissive technology, LCDs are light-modulating devices that form the visible image by affecting the transparency of display pixels (Fig 8). This technology, in its basic arrangement, is limited by dramatic variations of luminance and contrast with viewing angle. However, during the past 10 years, LCD designs with a more uniform luminance and contrast profile within a larger viewing angle cone have been introduced, including inplane switching, multiple subpixel domains, and vertically aligned liquid crystal modes (12). The angular variation in luminance and contrast is one of the many performance issues

RadioGraphics

326

RG f Volume 24

January-February 2004

●

Number 1

Table 9 Performance Issues in Medical Display Systems Key Performance Issues

Example

Lifetime Viewing angle (LCDs) Small-spot contrast ratio Diffuse reflection coefficient Specular reflection coefficient Amplifier bandwidth (response time) Resolution-addressability ratio (CRTs) Pixel size at 5% of maximum (CRTs) Resolution Noise

20,000 h to service 130°–170° for contrast ratio ⬎ 10:1 120:1 to 800:1 0.002–0.064 cd/m2/lux 0.002–0.040 25–40 msec 0.9–1.1 3 ⫻ full width at half maximum Modulation transfer function* Noise power spectrum†

*The resolution properties of CRT systems vary across the screen, whereas the modulation transfer function of active-matrix LCDs is spatially uniform. †Noise includes phosphor granularity (in CRTs), subpixel structure (in LCDs), and temporal noise.

associated with display systems (Table 9). An aspect of display performance that merits attention is the ability of the system to achieve a large small-spot contrast ratio (13). This capability ensures that dark areas of the screen with subtle image features are not affected by brighter areas elsewhere in the screen. The small-spot contrast is dominated by veiling glare in CRTs (14) and by electronic crosstalk in active-matrix LCDs (12,15). Table 10 summarizes the current technology offerings for CRT and active-matrix LCD monitors. In terms of the graphical user interface (GUI) and display workstation capabilities, the processes of image display and interpretation with PACS can be subdivided into several individual tasks, all of which should ideally be customizable to the individual preferences of the clinician. The ultimate goal is to simultaneously improve radiologists’ productivity and interpretation accuracy. Productivity is enhanced if the interface enables work flow optimization software that automates time-consuming manual tasks. Interpretation accuracy can be improved by using decision support and computational tools to deconstruct complex processes in image perception and diagnostic reasoning. Workstation capabilities that can enhance clinicians’ productivity are listed in Table 11. This focus on radiologists’ productivity takes on greater importance as imaging data sets are becoming increasingly larger and more complex. An example of this transformation can be seen

Table 10 Novel Technology Offerings in Medical Monitors Technology CRTs

LCDs

Offering Dynamic focusing of electron beam Antistatic panel Antireflective panel External and/or internal sensor for automatic black level setting and gray-scale calibration In-plane switching liquid crystal mode Multiple-domain pixels Vertically aligned liquid crystal mode Sensor for backlight adjustment 10-bit gray scale via subpixel or temporal frame modulation

with CT examinations, which previously consisted of 80 –100 images and now can comprise 1,000 images with the adoption of multisection CT. This has led to dramatic evolutionary changes in image display with PACS, which began with use of the frame mode and now involves use of two- and three-dimensional multiplanar reformation. In the future, advanced volumetric data renditions will place even greater evolutionary demands on the diagnostic workstation.

RIS and RIS Connection Radiology departments face an issue that is common to many areas in high-technology medicine. An increase in the amount of data acquired, a

RadioGraphics

RG f Volume 24

●

Number 1

Samei et al

327

Table 11 Desired Functionalities of Display Workstations Option or Function

Purpose

Prefetch algorithms

Software programs that automatically retrieve historical examination results and reports for correlation with the current study Intelligent image display guidelines based on the anatomic region, examination type, technique, and pathologic condition; customized based on the preferences of individual radiologists and linked to the user’s sign-on Specialized processing algorithms used to enhance specific anatomic features or types of pathologic conditions (ie, disease-specific processing); incorporated into the workstation via keyboard presets to enhance radiologists’ productivity Diagnostic aids to assist in soft-copy interpretation, such as software for computeraided detection, segmentation, and textural analysis and additional artificial intelligence techniques to reduce the “human weaknesses” of bias, fatigue, and inconsistency

Hanging protocols

Image processing

Decision support tools

Table 12 Required Data Elements at a PACS Workstation Data Element

Originating Data Source

Patient name Study identifier Patient identifier Patient age Reason for study Signs and symptoms, history Referring physician Patient location Organ system or body part Modality identifier Examination date and time Report status

HIS RIS HIS HIS RIS RIS RIS HIS PACS or RIS Modality Modality Dictation system

growing reliance on these data, and disconnected legacy systems to manage this influx of data raise issues that point toward the need for better integration and better interfaces. There are at least three and up to five information systems in a radiology department that are required to handle the data associated with an examination (16). These systems include the hospital information system (HIS), the RIS, the PACS, the voice recognition system, and an electronic teaching file system. Integration with other information systems and the ability to display relevant data on the desktop are critical for the success of PACS. Current technology solutions offer possible ways to address many of these issues.

Table 13 Options for RIS Connectivity Technology Integration Method Broker based

Brokerless

Advantages and Disadvantages Flexible in a multivendor environment Supports wide range of HIS-RIS combinations Less technology bias Fewer vendors required Simpler implementation Potential for greater accuracy in mapping

The connection between the RIS and the PACS is most critical. At the acquisition modality, the Modality Work List provides a direct point of entry of patient demographic information into the system. At the workstation level, the images are usable only if they are associated with the patient information. Table 12 lists some of the data elements that need to be available at the workstation. In general, two types of RIS-HIS integrations are now offered, one type via a broker and another without a broker (Table 13, Fig 9). The brokerless technology is preferred, as there is less opportunity for data inconsistencies. Regardless of the technology, a number of issues need to

RadioGraphics

328

January-February 2004

RG f Volume 24

●

Number 1

Figure 9. Schematics of the broker-based (a) and brokerless (b) HIS-RIS integration methods. HL7 ⫽ Health Level 7.

be considered: They include the current technology state of the modalities, the capability of the modalities for Modality Work List functionality, the current working relationship between the RIS and PACS companies, prior validation of the RIS-PACS connections, and partnerships and acquisitions facilitating the RIS-PACS integration. When functions are combined, the line between study management and image management blurs, as does the line between RIS and PACS responsibility. Ultimately, it is preferable to have a common platform that incorporates image management and study management capabilities. This method ensures timeliness and accuracy of data and provides greatest enhancement of work flow. True integration goes beyond simple RISPACS communication. Full integration is a complex task that requires cooperation from PACS, RIS, and modality vendors. Incorporation of the work flow enhancements gained from interfaces and integration can be done incrementally. For example, DICOM Modality Work List can be installed on CT and MR imaging units as it becomes available before incorporation into nuclear medicine or US. Date validation may occur long before image distribution through an independent results repository. Taken in steps, integration is complex but manageable and certainly an essential step for appropriate management of work flow.

Remote Access For PACS to achieve its fullest potential, there must be a strategy for the distribution of diagnostic images throughout the healthcare enterprise.

One cannot stop printing film without an acceptable mechanism for the electronic distribution of images outside the radiology department. Indeed, PACS implementations can truly be justified economically only when PACS can provide effective image distribution and display throughout the institution, not just for the radiology department. The enterprise-wide distribution of medical images is a significantly different and much more challenging problem compared with traditional radiology-centric PACS models. The requirements for image delivery to users outside the radiology department can differ significantly compared with those for the radiology department. Users outside the radiology department can be more unpredictable compared to radiologists with respect to what images they require, where they need to see images, and when they need to see them. These users can be even more demanding compared to radiologists with respect to image delivery performance. This creates a technological challenge, as network and computer infrastructure is usually much more modest throughout the enterprise compared to what is usually present within the radiology department. Consequently, the specific requirements for enterprise-wide image distribution are somewhat more demanding than those for traditional PACS. Some of those requirements are as follows: 1. Acceptable proxy or replacement for film: Any strategy for enterprise-wide medical image distribution must provide for the needs of a wide variety of users, including those who require highquality images outside the radiology department. Enterprise electronic image distribution must be a completely acceptable replacement for film; otherwise, one will still have to print film for clinicians.

RadioGraphics

RG f Volume 24

●

Number 1

Samei et al

329

Table 14 Remote Access Technologies and Their Advantages and Disadvantages Technology PACS extension Thin client Web-based Thin client just-in-time CD-ROM*

Advantages

Disadvantages

Speed, reliability Relatively low cost, speed, ease of implementation Speed, reliability Low cost, ease of implementation

Costly, requiring dedicated equipment Possible image quality degradation if compression is applied for speed Relatively costly, requiring dedicated equipment Slow, low throughput

*CD-ROM ⫽ compact disk, read-only memory.

2. Timing performance: Radiologists do not like to wait for studies when working with PACS. Clinicians can be even more demanding of delivery performance, at least some of the time. Whether they are working in a trauma setting, are in an acute situation in the intensive care unit, or are simply trying to make it through a hectic busy outpatient workload, clinicians will occasionally demand almost immediate access to imaging studies. 3. Scalability and cost-effectiveness: Traditional radiology-centric PACS models can make great demands on network, server, and workstation resources. Expensive dedicated networks, large database and archive servers, and high-fidelity workstations are not unusual components of any PACS. Current radiology-centric systems can use this “brute force” approach, since most radiology departments are composed of a relatively small number of centralized reading rooms. However, the traditional PACS model without modification will not scale when attempting to provide high-performance ubiquitous distribution of fullfidelity images throughout a large, expanding enterprise. Providing high-bandwidth network access as well as distributing expensive workstations to every desktop throughout the enterprise is not financially viable for the vast majority of institutions. The enterprise PACS must be able to provide high performance with respect to both image fidelity and access or delivery time to the desktop and modest workstations using limited network bandwidth. 4. Image quality: Providing the images at a quality level necessary for the specific use of the image is paramount. There is a need for a comprehensive system-wide quality control program that ensures adequate image quality throughout the clinic. Such a program should take into account the varying ambient lighting conditions of different clinical areas. Particularly challenging areas are orthopedic clinics, where the fidelity of the images (in terms of gray scale and resolution) is of particular importance, and operating rooms, where the presence of a generally high level of

ambient lighting can considerably affect the grayscale presentation of the images. Strategies should be considered to reduce the level or effect of ambient lighting by paying particular attention to the level and orientation of lighting, as well as the intrinsic reflection characteristics of the display device. Display devices with more favorable reflection characteristics and wider luminance ranges may be considered for areas within which the level of ambient lighting cannot be reduced. 5. Integration of the electronic medical record (EMR): The radiology department is not the sole generator of medical images; even a casual tour of the endoscopy suite, cardiology department, or pathology laboratory will confirm this. Many of these entities are generating large and complex high-resolution image and video data sets. Clinicians integrate information derived from imaging studies with other clinical data to make patient care decisions. Clearly, what the clinician requires is a single workstation with an interface that provides an integrated multimedia electronic medical record, including the ability to present and manipulate medical image data. The traditional PACS that supports only dedicated “thick client” workstations that display images alone does not address this integration requirement; the enterprise PACS must. A true enterprise PACS must also provide delivery of these data sets. 6. Security: The enterprise PACS must have much more robust security relative to the traditional radiology-centric PACS given the requirement for ubiquitous and flexible distribution and accessibility of clinical images throughout the enterprise. Four specific strategies are currently used for enterprise-wide image distribution (Table 14). The PACS extension approach uses the traditional radiology-based PACS as the foundation of the enterprise PACS. This strategy views the enterprise as “extensions” of the radiology department. The same network, server, and workstation infrastructure solutions are used to provide image

RadioGraphics

330

January-February 2004

distribution to “designated remote” areas of the enterprise (the intensive care unit, operating room, emergency room, etc). This approach can also be used with other means to extend the network beyond its physical reach (ie, shared or dedicated wide area network [WAN] extensions including satellite connections). The thin client Web-based approach is an increasingly popular strategy for enterprise-wide image distribution. This approach avoids the problems with distributing large image data sets (and the very demanding infrastructure requirements of a traditional PACS network and workstations) by distributing “subsampled” (ie, compressed) representations of the images. Most vendors who use this strategy deliver these subsampled images using thin client (usually Web-based) applications, which usually make no strong assumptions or requirements of the user workstation. The thin client just-in-time method is a relatively new approach that addresses the problem of delivering large data sets using modest resources in a different way: Instead of globally subsampling the entire image data set prior to transmission and display, this strategy uses a “just in time” delivery model to deliver that portion of the image data set to the user just when it is required at arbitrary fidelity, including full fidelity. Finally, compact disk, read-only memory (CD-ROM) data delivery is a method that allows the distribution of images without the need for a network infrastructure. It can be very useful for the delivery of images “outside the firewall,” especially to “foreign” institutions.

Acceptance Testing of PACS As complex systems, PACS are prone to technical problems and operational shortcomings. Problems can arise if a new system is not installed properly or if demands made on the system are beyond those originally anticipated. Often, the problems cannot be solved by simple replacement or repairs without significant inconvenience, given the extreme cost of a PACS and its usually wide integration with routine clinical operation. However, most potential problems can be prevented by rigorous, complete acceptance testing before putting the system into clinical use. A wellthought-out, carefully implemented protocol can

RG f Volume 24

●

Number 1

ensure that the system will meet the user’s technical, functional, and operational requirements and that it is ready for “safe” clinical use (17,18). Acceptance testing is meaningful only when it is directly tied to the agreement between the user and the vendor, which usually takes the form of a request for proposal (RFP). An RFP should set out detailed performance specifications and functionality requirements and should include specific acceptance testing procedures and criteria. If the RFP is carefully written, acceptance testing is simply the validation of previously agreed-on performance. Payments for the system should be linked to timely completion of the tests and satisfactory resolution of problems discovered during the evaluation. In this manner, the respective roles of the user and the vendor are defined and the interests of both parties are protected. Although the vendor might perform some testing during installation, the user should verify those results and test the system independently. The procedure is best performed by a team of individuals with a range of expertise, including (minimally) a computer specialist, a medical physicist, a radiologic technologist familiar with the work flow in the department, a radiology informatics specialist, and a radiologist. Acceptance testing is best performed in two sequential phases: technical testing and clinical testing. Technical acceptance testing evaluates the performance and specifications of the components as well as the functional and operational performance of the system, so that potential problems can be addressed by the vendor before the system is used clinically. Clinical acceptance testing evaluates the functional and operational utilities of the system and its reliability in the early stages of actual clinical use.

Technical Acceptance Testing In this phase, all the components of the system should be evaluated individually and collectively to ensure conformance with expected technical performance requirements. The exact tests and requirements will vary according to the architecture and specifics of each system. However, in general, technical acceptance testing can be organized into seven different elements: network performance, installation and basic configuration, single-component performance, functionality, functionality under load, overall image quality, and fault tolerance.

RadioGraphics

RG f Volume 24

●

Number 1

Network Performance.—During or before PACS installation, the user should ensure that the network provides adequate bandwidth and reliability for acceptance testing as well as for robust PACS operation. The network system, including cable, switches, and the configuration, can be tested by using network diagnostic tools. Some performance characteristics to consider include bandwidth, number of collisions per unit time, and speed. Testing should also encompass shared network configurations. Any shortcomings should be addressed prior to system-wide testing or clinical operation. Installation and Basic Configuration.—The basic setup and architecture of the installed system should be inspected initially. All components of the system as noted in Figure 1 should also be checked against the purchase order and their specifications verified. Some characteristics that should be verified include central processing unit (CPU) storage and speed, memory, and network connection speed for all computational components of the system. Single-Component Performance.—After verification of the proper installation and configuration, acceptance testing should focus on evaluating the single components of the system. These include all of the components listed in Figure 1, with a primary focus on the acquisition, display, and archival devices. The performance of computed radiography systems should be evaluated according to the guidelines of Task Group 10 of the American Association of Physicists in Medicine (AAPM) (19). The recommended tests include dark noise, linearity, exposure response calibration, throughput, erasure thoroughness, resolution, noise, and laser beam function. A subset of these tests can be adopted for other digital radiography systems. The performance assessment of any digital radiography system should also include the default image processing settings, initially for the most common imaging protocols. The acceptability of the settings should be verified in consultation with radiologists and customized to their preference. At this stage, that task can be undertaken with images of anthropomorphic phantoms. A full customization of processing settings should be performed by using clinical images as part of the clinical acceptance testing.

Samei et al

331

It is also important to verify that the new and existing acquisition devices are optimally interfaced to the PACS. The tests should include the bandwidth of the connection, the utilization of the DICOM communication protocol, and complete data transfer (in terms of bit depth and number of sections) to the PACS. It is preferable that all of the CR and DR images are transferred as DICOM DX objects. All display workstations should be evaluated according to the guidelines of AAPM Task Group 18 (10,11). Both the primary (so-called diagnostic) as well as secondary (so-called clinical) displays should be included. The main recommended tests include luminance response, resolution, noise, veiling glare, reflection, chromaticity, and viewing angle performance (for LCDs). All display quality tests should be performed via the graphical user interface (GUI) of the display application to test the display system (hardware and software) as a whole. The workstation functions should also be included in the acceptance testing protocols. Printing should also be tested as a mode of display. The testing should encompass direct printing from the modality as well as printing from the PACS to each printer. Printer-specific tests may include speed and throughput, density uniformity, density consistency, bit-depth resolution, true-size printing, spatial interpolation settings, and gray-scale settings. It is important to ensure that the gray-scale settings of both the soft- and hard-copy displays comply with the DICOM Gray-Scale Standard Display Function. Functionality.—Once the performance of single components is evaluated, the entire system should be tested for system-wide functionality. Tests should include all key functions including modality routing, Web server routing, autoarchiving, prefetching, query-retrieve capabilities, printing capabilities, backup archive retrieval (if applicable), teaching file or folder capabilities (if applicable), PACS quality control protocols, system security (including Web encryption), connectivity, and timing performance with all imaging modalities connected to the PACS. The system should be tested for the most common tasks, such as querying a patient folder,

RadioGraphics

332

RG f Volume 24

January-February 2004

●

Number 1

Table 15 Acceptance Checklist for RIS-PACS Connectivity Functions Verification

Routing Prefetching Report notification Report accessibility

Questions Are new studies verified and passed to the PACS? Are patient name, account number, and medical record number populated correctly? Is the procedure code field populated properly? Are study results routed to the proper review stations after verification? Are relevant priors prefetched and routed to the appropriate workstations? Is the status changed as preliminary and finalized reports are created? Are old reports available? Are new reports available? Is the format correct (eg, word wrap)? Are all fields populated correctly? Report status Signer Reason for study Patient location Examination modifiers

sending images from one location to another, correcting a misentered patient demographic, and common workstation functions including work list management. The PACS interface should not be too cumbersome to allow convenient execution of the common tasks. Otherwise, the vendor should provide alternative solutions. For the RIS-PACS connectivity test, a checklist such as that in Table 15 may be used. It is also important to ensure that for the broker-based systems, the broker database is updated every time a relevant change is made in the RIS. In implementations in which the PACS has a direct link with the HIS, the HIS-PACS interface should also be examined. DICOM compliance is one of the important aspects of system functionality. The system should conform to current DICOM standards, given that DICOM is a changing standard and a previous compliance might be outdated. Conformance should be verified against the vendor’s DICOM Conformance Statement, preferably with non–vendor-specific tools. It is also important to test that the system can readily export images in the DICOM format. System-wide settings should also be verified, including routing, prefetching, compression, and encryption. The routing and prefetching rules should be examined closely and possibly customized to optimize work flow. The image compression settings for transmission, storage, and Web access should comply with the user directives. It is also important to ensure that the image data are transmitted within and out of the system with the

desired bit-depth resolution. The security settings of the system should be verified to ensure firewall protection and patient confidentiality in accordance with state and federal (eg, the Health Insurance Portability and Accountability Act [HIPAA]) requirements. Finally, the timing performance of the system should be tested. The user should establish timing benchmarks for common processes. The timing tests might include common tasks such as image and study transfer times from the acquisition system to display and to archive, RIS report retrieval, single image retrieval (from the cache and the short-term and long-term archives), patient folder retrieval (from the short-term and longterm archives), switching from one study to another, marking a study as “dictated,” log-out and log-in functions, system reboots, and Web-display retrievals (from the server and from the archive). Functionality under Load.—The tests described up till now will provide an overall evaluation of the system performance. However, the results are indicative of the best conditions, where the load on the server is low and bandwidths are underused. Thus, there is a need to test the system performance (particularly the timing tests) under load. Such tests may be performed with the archive and database full to 70% with parallel activities in the system by using a script-driven artificial load. Overall Image Quality.—Although the image quality of acquisition and display devices may be independently assessed, as described earlier, it is important to evaluate the overall image quality in the PACS as well. This task may be accomplished by acquiring phantom images at the modalities,

●

Number 1

RadioGraphics

RG f Volume 24

Figure 10. Change in the PCR for computers over time. E ⫽ exponent, MIPS ⫽ million instructions per second.

sending the images to the PACS, and evaluating the quality of the images on a PACS display (including the primary, secondary, and Web displays). The tests should include (but not be limited to) verification of the correct window-level setting transfers to display stations, matching hard-copy and soft-copy displays, the correct order of images for multi-image modalities (eg, CT and MR imaging), preset window-level settings, and true-size hard-copy and soft-copy functionality. Fault Tolerance.—While performing the technical acceptance testing procedures, the user should consider reliability and uptime as important performance factors under evaluation. Any unexpected failure, system freezes, or auto-reboots should be viewed critically and further investigated. It is also important to test that in the event of a system failure, backup strategies would allow some critical level of operation. To test the fault tolerance, a schema might include fixing wrong manual entries; fixing mismatched studies; and trying to operate with the RIS down, with the RIS broker down, with the image or database server down, with a modality interface down, and even with partial power or with the uninterruptable power supply (UPS) down.

Clinical Acceptance Testing Once technical acceptance testing is successfully completed and the potential problems are resolved, the system can be put into clinical service. However, because of limited time and scope, technical acceptance testing cannot possibly test all the possible ways that the system may be used. Thus, it is important that the user maintain some discretion in accepting or rejecting the system at the early stages of clinical use. To do so, the system should be systematically examined during the first few months of operation. Toward the end of

Samei et al

333

this period, the data stored in the archive and the server database will have grown. This provides an opportunity for more operationally relevant testing of the system functionality (notwithstanding the functionality under load tests described earlier). At this phase, the timing performance and the configurational settings of the system should be more fully evaluated and customized for optimum operation. Seamless operation of the PACS with all peripheral devices and systems (eg, all acquisition devices, the RIS and HIS, the dictation system) should be further verified. In addition, this period provides the user with an invaluable opportunity to more fully examine the reliability (ie, uptime) and fault tolerance of the system.

Technology Change Although someone purchasing PACS equipment at the present time might rely on the options and testing recommendations outlined herein, the PACS technology will continue to change. An understanding of historical changes and current trends in computer technology and their effect on PACS may prove invaluable in planning for future PACS deployment and upgrades. The essential elements of medical PACS were well established 15 years ago, with prototype systems operating at several facilities. However, the cost and performance of computers, storage devices, and network components made these early systems impractical. Continued rapid improvement in the performance-to-cost ratio (PCR), also called the price-performance ratio (PPR) in information technology circles, has led to a presently favorable return on investment for PACS technology when a filmless operation is considered. Even more notable is the expectation that computer and network technology will continue to see further rapid improvements in the PCR in the coming decade. These improvements should be considered when designing and purchasing a new or replacement PACS. Improvements in the PCR for computer devices are most well known. In 1982, a computer with a 5-MHz processor, 8 Mbytes of memory, and 900 Mbytes of magnetic disk storage cost about $250,000 (20). Only about 20 years later, a system with dual 3-GHz processors, 2 Gbytes of memory, and 240 Gbytes of magnetic disk storage cost about $4,000. Moravec (21) recently examined the cost and computing power of numerous systems available throughout the 20th century. His results are summarized in Figure 10,

RadioGraphics

334

January-February 2004

which shows that the PCR improved by a factor of 1,000 over the 20 years from 1970 to 1990 (ie, a doubling time of about 2 years) and at a somewhat faster rate in the past decade. This is consistent with the improvement in transistor density seen for Intel (Santa Clara, Calif) processors and often referred to as Moore’s law. For PACS, this suggests that the cost of small computers used by referring physicians and more powerful computers used by radiologists will continue to rapidly drop to allow rapid deployment in high volume, even for locations that see occasional use. As noted previously and shown in Figure 5, the cost per megabyte for magnetic disk storage devices has been declining by about a factor of 2 per year for the past several decades. Presently, highspeed small computer system interface (SCSI) disks in a RAID configuration can be purchased for about $8,000/Tbyte. Given that a large medical center may typically require about 5 Tbytes of storage for image data per year (with lossless compression assumed), the present price makes it possible to consider the use of a centralized magnetic disk storage device. The declining cost suggests that an archive should be configured such that capacity is added annually with a supply cost that will be very small within a few years. Further improvements are likely to favor remotely located disaster recovery magnetic disk archives that can rapidly maintain service in the event of major damage to a central device. The PCR for network components has similarly improved, and some forecast that network capacity will increase faster than computing, memory, and storage in the coming decade. Networks that deliver data to workstations at a rate of 1 Gbit/sec are now available at low cost, and most new desktop systems provide connection at this speed. Similarly, high-speed network service for wide area network (WAN) operations has notably dropped in cost in most regions. This favors a move toward more centralized network architectures using grid computing models and thin client services, a trend that is increasingly being seen in the newer PACS implementations.

References 1. Honeyman JC, Frost MM, Staab EV. PACS component testing: beta and acceptance testing. Proc SPIE 1997; 3035:405– 412. 2. Sonoda M, Takano M, Miyahara J, Kato H. Computed radiography utilizing scanning laser stimulated luminescence. Radiology 1983; 148:833– 838. 3. Yaffe MJ, Rowlands JA. X-ray detectors for digital radiography. Phys Med Biol 1997; 42:1–39.

RG f Volume 24

●

Number 1

4. Maidment ADA. Quality control issues for computed and direct radiography. In: Reiner B, Siegel E, Carrino J, eds. Quality assurance: meeting the challenge in the digital medical enterprise. SCAR University Primer 3. Great Falls, Va: Society for Computer Applications in Radiology, 2002; 9 –28. 5. Samei E, Flynn MJ. An experimental comparison of detector performance for computed radiography systems. Med Phys 2002; 29:447– 459. 6. Samei E, Flynn MJ. An experimental comparison of detector performance for direct and indirect digital radiography systems. Med Phys 2003; 30: 608 – 622. 7. Pratt HM, Langlotz CP, Feingold ER, et al. Incremental cost of department-wide implementation of a picture archiving and communication system and computed radiography. Radiology 1998; 206: 245–252. 8. Chunn T. Tape storage for images. Imaging World 1996; 5:1–3. 9. Horii S, Levine BA, Groger G, et al. A comparison of case retrieval times: film versus picture archiving and communication systems. J Digit Imaging 1992; 5:138 –143. 10. Samei E. New developments in display quality control. In: Reiner B, Siegel E, Carrino J, eds. Quality assurance: meeting the challenge in the digital medical enterprise. SCAR University Primer 3. Great Falls, Va: Society for Computer Applications in Radiology, 2002; 71– 82. 11. Samei E, Shepard J, Fetterly KA, Roehrig H, Kim HJ, Flynn MJ. Clinical verification of TG18 methodology for display quality evaluation. Proc SPIE 2003; 5029:484 – 492. 12. Badano A. Display systems. RadioGraphics 2004 (in press). 13. Badano A, Flynn MJ, Kanicki J. Accurate smallspot luminance measurements. Displays 2002; 23:177–182. 14. Badano A, Flynn MJ, Muka E, Compton K, Monsees T. Veiling glare point-spread function of medical imaging monitors. Proc SPIE 1999; 3658: 458 – 467. 15. Libsch FR, Lien A. Understanding crosstalk in high-resolution color thin-film-transistor liquid crystal displays. IBM J Res Devel 1998; 42:467– 479. 16. Honeyman JC. Information systems integration in radiology. J Digit Imaging 1999; 12:218 –222. 17. Richardson NE, Thomas JA, Lyche DK, Romlein J, Norton GS, Dolecek QE. The philosophy of benchmark testing a standards-based picture archiving and communications system. J Digit Imaging 1999; 12:87–93. 18. Lewis TE, Horton MC, Kinsey TV, Shelton PD. Acceptance testing of integrated picture archiving and communications systems. J Digit Imaging 1999; 12:163–165. 19. Samei E, Seibert JA, Willis CE, Flynn MJ, Mah E, Junck KL. Performance evaluation of computed radiography systems. Med Phys 2001; 28:361– 371. 20. Goble GH, Marsh MH. A dual processor VAX 11/780. Purdue University Technical Report TR-EE 81-31. West Lafayette, Ind: Purdue University, 1981. 21. Moravec H. When will computer hardware match the human brain? J Evol Technol [online journal] 1998; 1. Available at: www.jetpress.org/volume1/ moravec.htm.