ROBOTIC systems with master slave control were introduced

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 55, NO. 11, NOVEMBER 2008 2593 Optimization of a Pneumatic Balloon Tactile Display for Robot-Assis...
Author: Naomi Jackson
1 downloads 0 Views 423KB Size
IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 55, NO. 11, NOVEMBER 2008

2593

Optimization of a Pneumatic Balloon Tactile Display for Robot-Assisted Surgery Based on Human Perception Chih-Hung King*, Martin O. Culjat, Miguel L. Franco, James W. Bisley, Erik Dutson, and Warren S. Grundfest

Abstract—Robot-assisted surgery is characterized by a total loss of haptic feedback, requiring surgeons to rely solely on visual cues. A compact, flexible, and lightweight pneumatic balloon tactile display has been developed suitable for mounting on robotic surgical master controls. The tactile display consists of a molded polydimethylsiloxane substrate with cylindrical channels and a spin-coated silicone film that forms the array of balloons. Human perceptual studies were conducted to determine the optimal diameter, spatial resolution, and temporal resolution of the balloon actuator design. A balloon diameter of 3.0 mm provided the highest average accuracy (≥95%) while offering five detectable inflation levels. Spatial accuracy in a two-actuator discrimination task reached 100% with 1.5 mm edge-to-edge spacing, and the accuracy of determining the order of two successive stimuli was greater than 90% when the time separation was 100 ms. Design optimization based on the results from this study enables the described tactile display to provide the effective tactile feedback that is otherwise unavailable during robotic surgery. Index Terms—Haptic feedback, haptic perception, robot tactile systems.

I. INTRODUCTION OBOTIC systems with master–slave control were introduced to surgery in the early 1990s, providing advanced features such as stereoscopic vision, tremor reduction, additional degrees of freedom, increased precision, and teleoperation capabilities [1]–[5]. Currently, the only such robotic surgical system approved by the U.S. Food and Drug Administration for use in abdominal, pelvic, and cardiothoracic surgery is the da Vinci Surgical System (Intuitive Surgical, Inc., Sunnyvale, CA). The da Vinci system features a control console that allows

R

Manuscript received August 13, 2007; revised April 15, 2008. First published June 10, 2008; current version published October 31, 2008. This work was supported in part by the Telemedicine and Advanced Technology Research Center (TATRC)/Department of Defense under Award W81XWH-05-2-0024. The work of J. W. Bisley was supported by the Alfred P. Sloan Research Fellowship and a Klingenstein Fellowship Award. Asterisk indicates corresponding author. *C. H. King is with the Department of Biomedical Engineering, University of California, Los Angeles (UCLA), Los Angeles, CA 90095 USA, and also with the Center for Advanced Surgical and Interventional Technology (CASIT), Los Angeles, CA 90095-1600 USA (e-mail: [email protected]). M. O. Culjat and E. Dutson are with the Department of Surgery, University of California, Los Angeles (UCLA) School of Medicine, Los Angeles, CA 90095 USA, and also with the Center for Advanced Surgical and Interventional Technology (CASIT), Los Angeles, CA 90095-1600 USA. M. L. Franco is with the University of California, Los Angeles (UCLA), Los Angeles, CA 90095 USA, and also with the Center for Advanced Surgical and Interventional Technology (CASIT), Los Angeles, CA 90095-1600 USA. J. W. Bisley is with David Geffen School of Medicine, University of California, Los Angeles (UCLA), Los Angeles, CA 90095 USA. W. S. Grundfest is with the University of California, Los Angeles (UCLA), Los Angeles, CA 90095 USA. Digital Object Identifier 10.1109/TBME.2008.2001137

a surgeon to manipulate robotic arms remotely and a robotic apparatus with three arms (or four arms) that hold a stereoscopic endoscope and detachable laparoscopic surgical tools. The surgeon controls the arms by manipulating two master controls with the index finger and thumb of both hands with each finger inserted into a Velcro support strap. A major limitation of robotic surgery is the complete absence of haptic feedback [6], [7]. During robotic surgery, surgeons are completely isolated from the patient and rely solely on visual cues. Previous studies have suggested that force feedback may be beneficial to robotic surgery [8]–[11]. Kitagawa et al. [12] showed that the force feedback may facilitate the performance of surgical knot tying using sensory substitution (visual and audio cues) in surgical robots. However, the majority of previous research has focused on force (kinesthetic) rather than tactile (cutaneous) feedback. The addition of a tactile display, or tactile feedback actuator array, to the master controls, when integrated with a tactile sensor array mounted onto the tips of robotic graspers, may enable surgeons to “feel” tissue characteristics, appropriately tension sutures, identify pathologic conditions, and enable expansion of robotic minimally invasive surgery (MIS) to other surgical procedures. The size of the finger pad and limited mounting space on the robotic surgical master controls require a tactile display that is compact, flexible, and lightweight. In the case of the da Vinci Surgical System, the available mounting area is approximately 1.0 cm × 1.8 cm. Various tactile display technologies, including electromagnetic, electrocutaneous, electrostatic, motor, piezoelectric, rheological fluid, and shape-memory alloy (SMA) actuators, have previously been explored [13]–[19]. Pneumatic tactile actuators are capable of producing high displacement and large output force [20]; however, those that have been developed are not practical for surgical robotics due to bulky size, awkward geometries for mounting, or slow dynamic response [21]–[24]. We have developed a modular and scalable pneumatic actuator designed to provide haptic feedback to the fingers using an array of balloons formed from a spin-coated silicone film placed over a molded substrate. This design incorporates the advantages of low mass, compact size, large force output, and deflection, and can be integrated with the master controls of surgical robotic systems. The elastic balloon films have the ability to conform to the shape of the fingers, allowing uniform distribution of force. The balloon actuators are designed to provide haptic input into the human sensory system by stimulating the mechanoreceptors through skin deformation [25]. Previous mechanical characterization tests on our balloon actuators have demonstrated a

0018-9294/$25.00 © 2008 IEEE Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

2594

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 55, NO. 11, NOVEMBER 2008

Fig. 1. Cyclic actuation of a balloon with 3.0 mm diameter at 500, 1500, 5000, 10000, and 15000 cycles.

high linearity between inflation pressure and balloon deflection (R2 ≥ 0.93), negligible hysteresis effects between inflation and deflation over 15 000 cycles (see Fig. 1), and consistent performance among actuators from various fabrication batches [26]. Furthermore, preliminary perceptual testing using a prototype balloon actuator suggested that balloon-based haptic feedback may be an effective means to provide tactile information to the finger, provided the balloon diameter is greater than 1.0 mm [27]. However, prior to application of the actuator onto a surgical robotic system, it is critical to ensure that it provides effective tactile feedback with high resolution. The aim of this study was to address this issue by optimizing the balloon diameter, balloon spacing, and temporal response of the actuator. We did this by examining the human perceptual responses to various actuator inputs, and then, optimizing the tactile display based on the statistical analyses of these data. II. MATERIALS AND METHODS A. Pneumatic Balloon Tactile Display A detailed description of the manufacturing and operating process has been presented elsewhere [26]. Briefly, each pneumatic balloon actuator consists of a polydimethylsiloxane (PDMS, General Electric RTV615) substrate and a spin-coated silicone film. The PDMS substrates were constructed using aluminum molds that were machined with horizontal channels for tubing placement and vertical channels for the air chambers. The silicone film was fabricated by spin coating SORTA Clear 40 (Smooth-On, Inc.) silicone in liquid form on a silicon wafer. SORTA Clear was selected over PDMS due to its superior elasticity and durability, allowing large balloon deformation without breakage. A spin coater (Laurell Technologies WS-400B6NPP-LITE) was used to control the thickness and uniformity of the film. The PDMS substrate was placed atop the SORTA Clear film immediately following spin coating. As the SORTA Clear film cured, it cross-linked and bonded with the PDMS substrate without the use of additional bonding agents. The balloon array elements were formed from the vertical balloon channels while the horizontal side channels remained open to provide pressure inputs for each of the newly formed balloons. This process was used to fabricate balloon actuators with diameters of 1.5, 2.0, 2.5, 3.0, and 4.0 mm. Prior mechanical

Fig. 2. Examples of tactile displays used for perceptual testing, with tactile display with multiple edge-to-edge spacings (2.0, 1.5, 1.0, and 0.5 mm) at top, and tactile display with multiple diameters (2.0, 1.5, 1.0, and 0.75 mm) at bottom.

Fig. 3.

Block diagram of the control system.

characterization tests of balloon film thicknesses and balloon diameters indicated that 300-µm-thick films are appropriate for 3.0–4.0-mm-diameter balloons, and 200 µm films are appropriate for 1.5-, 2.0-, and 2.5-mm-diameter balloons [26]. These thickness–diameter combinations did not show any noticeable hysteresis and were selected for all actuators used in this study. Various balloon diameters or spacings were fabricated on single PDMS substrates for use in the psychophysics experiments. Tactile displays with examples of multiple spacings and multiple balloon diameters are shown in Fig. 2. The balloon actuation patterns were generated with a control system consisting of a microcontroller and associated electronics, pressure regulators, air source, and pneumatic tubing and fittings (see Fig. 3). The firmware of the microcontroller (Microchip PIC16F877A) was programmed using C language under Microchip MPLAB IDE. The microcontroller interacted with a computer via a serial interface to: 1) receive the control instructions and testing parameters from the operator; 2) pseudorandomly select an actuation pattern from all possible combinations; and 3) generate the corresponding analog control signals (0–5 V) to the electropneumatic pressure regulators (SMC ITV0010–2MMS), which, in turn, inflated each of the balloon elements with the proportional pressures (0–15 psi). The air source to the electropneumatic regulators was supplied from a compressed air tank using 1/4" PVC tubing and the input pressure was limited to 30 psi, using a series of manual regulators. The output port of the electropneumatic regulator was fitted with 4 mm polyurethane tubing, which was converted to 2 mm tubing using quick-release fittings. The 2 mm tubing was

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

KING et al.: OPTIMIZATION OF A PNEUMATIC BALLOON TACTILE DISPLAY FOR ROBOT-ASSISTED SURGERY

connected to the side channels of the actuator, and a silicone adhesive sealant (GE RTV-108) was applied across the contact area between the tubing and side channels. One regulator provided inflation pressure for each balloon. The electropneumatic regulator, which consisted of control electronics, supply and exhaust solenoid valves, and pressure sensors, linearly translated the electrical input signal (0–5 V) into a proportional inflation pressure (0–15 psi) using closedloop pneumatic control. The control electronics monitored the output pressure through the pressure sensors and adjusted the switching of supply or exhaust solenoid valves to maintain the linear relationship between the control signal and the output pressure. This regulator was compact in size (1.5 cm × 5.0 cm × 9.0 cm) with a response time of less than 100 ms. B. Psychophysics Experiments To optimize the actuator array for use on a surgical robotic system, human performance was studied using a series of balloon diameter tests, spatial resolution tests, and temporal resolution tests. Balloon diameter tests were intended to determine the optimal balloon size that can be effectively detected by the human finger while still being compact enough to form a functional array. Spatial resolution tests were performed to study the minimum balloon spacing such that the subjects could identify two adjacent actuations as separate stimuli. Temporal resolution tests were devised to examine the optimal time separation required for human subjects to determine the order of two successive stimuli. A total of nine subjects participated in the studies; the protocol was approved by the University of California, Los Angeles (UCLA) Institutional Review Board and the Human Research Protection Office of the U.S. Medical Research and Materiel Command. Six of the nine subjects (age range 23–40 years), including at least one female and one surgeon, participated in each test, each of which was composed of three sessions. The subjects were given a brief training session prior to each test to ensure familiarity. The subjects placed the upper portion of their index finger in contact with the balloon actuators with the tactile display placed on a benchtop. The same finger was used for the duration of each test session. In each session, the subjects were asked to perform five two-alternative forced choice trials for each possible combination, comparing two sequentially presented stimuli. A minimum break of 15 min was given between test sessions. Three test sessions from the six subjects provided 90 trials for each possible combination. The accuracy was calculated based on the percentage of total correct responses divided by the total number of trials. Human perceptual tests were conducted for each of the existing balloon diameters to determine the optimal size that could be effectively detected. Prior mechanical characterization of the actuators demonstrated that balloons with 4.0 mm diameter had large variations in inflation deflection given a consistent pressure input, and therefore, were not included in the study [27]. Balloons with 3.0, 2.5, 2.0, and 1.5 mm diameters were tested with four inflation levels (Table I). Inflation levels were chosen based on the linear relationships between pressure and the amount of deflection [26].

2595

TABLE I INFLATION LEVELS OF EACH BALLOON SIZE IN THE FOUR-LEVEL TESTS

During the four-level balloon diameter test, the control system sequentially actuated a single balloon to one of the four inflation levels. The stimuli were presented over a period of 1 s, and were separated by a 1 s delay. Since there are 12 possible actuation combinations among four inflation levels, the subject was asked to perform 60 trials in each session. After each trial, the subject had to inform the operator whether the first or second stimulus had a higher pressure. With 90 trials of each combination, the four-level inflation test of each balloon diameter contained a total of 1080 trials. Five-level tests were performed between the two balloon sizes that yielded no statistical differences during the four-level tests. The five levels were chosen to be 0, 3, 6, 11, and 15 psi. Instead of testing all the possible combinations, only combinations with one inflation level difference were tested: 0–1, 1–0, 1–2, 2–1, 2–3, 3–2, 3–4, and 4–3. (For instance, 2–1 represents the actuation sequence from 6 psi inflation to 3 psi inflation.) with eight possible actuation combinations, the subjects were asked to perform 40 trials in each session for a total of 720 trials for each balloon size. Array spacing tests were performed using the balloon diameter that was determined to be optimal in performance and human perception as a result of the diameter tests. Actuator array spacing of previous tactile displays ranged between 0.5 and 3.75 mm [13]–[23]. For comparison, Braille dots are spaced between 2.0 and 2.7 mm apart [28]. In the balloon array spacing tests, four edge-to-edge element spacing parameters were selected: 0.5, 1, 1.5, and 2 mm. During actuator fabrication, spacing parameters of 0.5 and 1.0 mm were eliminated from consideration due to mechanical limitations. When actuators were built with 0.5 mm spacing, the thin PDMS wall separating two adjacent channels acted as a membrane and extended under pressure and experienced breaking during fabrication or during actuation. Actuators with 1.0 mm spacing exhibited a lesser degree of channel expansion, but revealed occasional membrane disassociation from the PDMS base due to the limited contact area between channels. Spacing parameters of 1.5 and 2.0 mm did not exhibit either effect. For each spacing parameter, the control system generated sequential inflations of one or two balloons, providing four possible combinations of sequential actuation: A–A, B–B, A–B, and B–A. (For instance, B–A represents the actuation sequence from balloon B to balloon A.) Balloons were inflated to inflation level 1 indicated in Table I. The stimuli were presented over a

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

2596

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 55, NO. 11, NOVEMBER 2008

period of 1 s, were separated by 500 ms delay, and took less than 100 ms to inflate. After each trial, the subject had to inform the operator whether the same or different balloons were actuated. There were 90 trials for each combination and a total of 360 trials for each spacing parameter that was tested. Temporal tests were conducted using actuators with the optimal balloon diameter and spacing from the study. Prior studies have suggested that the temporal threshold for determining the order of two successive stimuli is 20–60 ms [29], [30]. The control system generated inflations of two adjacent balloons in pseudorandom sequences with time separation of 100, 80, 60, and 40 ms. The subjects were asked to determine which of the balloons was actuated first. There were total of 180 trials for each time separation. Data were analyzed by fitting a standard psychometric function, the Weibull function [31], which resembles a sigmoid function with the detection accuracy displayed on the ordinate and the time separation on the abscissa  β  (1) W (x) = 0.5 + 0.5 1 − 2−(x/α ) where x was the time separation; α, the centering parameter of the function; and β, the steepness parameter of the function. The asymptotic minimum was fixed at 0.5 (50%) and the asymptotic maximum was fixed at 1.0 (100%). Threshold was calculated as the time separation corresponding to 75% correct. III. RESULTS Detection accuracy for each balloon diameter in the fourlevel tests is shown in Fig. 4 with the highest detection accuracy observed with the 3.0 and 2.5 mm balloons. Accuracies of all combinations for 3.0 mm balloons were above 95% [see Fig. 4(a)], and those for 2.5 mm balloons were above 90% [see Fig. 4(b)]. Detection accuracies of the 2.0 mm [see Fig. 4(c)] and 1.5 mm balloon diameters [see Fig. 4(d)] were reduced with each displaying multiple combinations with accuracies below 90%. Accuracies of many combinations improved as the balloon diameter increased. For the 3.0 and 2.5 mm balloons, there were no significant differences in performance between combinations of ramp-up and ramp-down sequences (p > 0.2, χ2 −tests); however, average performance with the 2.0 and 1.5 mm balloons was significantly better for ramp-down than for ramp-up sequences (p < 0.01, χ2 −tests). Average accuracy was also proportional to inflation level differences with the lowest average accuracy for one-level differences and the highest average accuracy for three-level differences. These results were consistent across the subjects whether male, female, surgeon, or nonsurgeon. All the subjects showed a deficit in accuracy with the 1.5-mm-diameter balloons and the best performance with the 3.0-mm-diameter balloons. Therefore, we believe that there is little or no difference of tactile perception capability between surgeon and nonsurgeon. Furthermore, the addition of the tactile feedback may also be used in surgical training to benefit surgical residents and medical students who have little or no experience in robotic surgery. Average accuracy of the four-level tests is shown as a function of balloon diameter in Fig. 5. The average accuracy increased as the balloon diameter increased. The smallest balloon diam-

Fig. 4. Detection accuracy of each balloon diameter and each combination of the sequential actuation in the four-level tests. (a) 3.0 mm balloon diameter. (b) 2.5 mm balloon diameter. (c) 2.0 mm balloon diameter. (d) 1.5 mm balloon diameter.

Fig. 5. tests.

Average detection accuracy of each balloon diameter in the four-level

eter (1.5 mm) produced the poorest performance, which was significantly worse than performance for the three larger diameter balloons (p  0.001, χ2 −tests). Likewise, performance on the 2.0 mm balloons was significantly worse than performance on the 2.5 and 3.0 mm balloons (p  0.001, χ2 −tests). There was no statistical difference in performance between 3.0 and 2.5 mm balloon diameters (p = 0.084, χ2 −test). However, in the debriefing, the subjects indicated that 3.0 mm balloons were easier to detect than the 2.5 mm balloons. To determine whether human tactile perception was significantly better with the 3.0-mm-diameter balloon, five-level inflation tests were conducted on the 3.0- and 2.5-mm-diameter balloons. From the first experiment, we knew that the subjects could detect steps of two levels, so in this experiment, we used smaller step sizes and only tested one-level step changed (see Section II for details). The results of this experiment are shown in Fig. 6. The average accuracy from both sets of balloons

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

KING et al.: OPTIMIZATION OF A PNEUMATIC BALLOON TACTILE DISPLAY FOR ROBOT-ASSISTED SURGERY

2597

Fig. 6. Detection accuracy of 2.5–3.0 mm balloon diameters and each combination of the five-level tests. (∗) indicates p < 0.003, χ 2 −tests.

Fig. 8. Optimized balloon tactile display with attached Velcro support strap and 2-mm-diameter pneumatic tubing. The tactile display (1.0 cm × 1.8 cm × 0.4 cm) has a 3 × 2 element array of 3.0 mm balloons with 1.5 mm spacing.

Fig. 7. Average detection accuracy of each time separation in temporal tests. Data are fitted with a psychometric Weibull function (R 2 = 0.998).

was significantly lower than the accuracy in the four-level tests (p  0.001, χ2 −tests). This shows that this experiment was a good assessment of near-threshold performance. In four of the eight combinations tested, performance was significantly better for the 3.0 mm balloons (p < 0.003, χ2 −tests) whereas performance was never significantly better for the 2.5 mm balloons. Overall, the average accuracy of the 2.5 mm balloons (82.8%) was significantly lower than the accuracy of the 3.0 mm balloons (p = 2e − 13, χ2 −tests); indeed, 3.0 mm balloons maintained high-detection accuracy (95%) in the five-level tests, achieving above 90% accuracy in all combinations except combination 0–1. Based on the earlier experiments, we determined that the 3.0-mm-diameter balloons elicited optimal performance. These balloon actuators were used in the spacing tests with edge-toedge distances of 1.5 and 2.0 mm. Performance for both spacing resulted in 100% detection accuracy from all subjects in all sessions. This shows that the ideal performance can be obtained with the minimal spacing required for structural integrity (1.5 mm). The temporal tests were performed on the optimal array as determined by the previous experiments; 3.0-mm-diameter balloons were positioned with 1.5 mm spacing. These tests revealed that there was a marked decrease in detection accuracy with decreased time separation (see Fig. 7) and these data were fit closely by the psychometric function (R2 = 0.998), giving a threshold of 62.4 ms. With a time separation between two adjacent balloon actuations of 100 ms, the subjects detected the order of inflation in 92% of trials. This decreased to 85% accu-

racy with an 80 ms time separation, 73% accuracy with a 60 ms time separation, and 61% accuracy with a 40 ms time separation. The low accuracy of the 40 ms time separation eliminated the need for further testing of shorter time separations. The psychometric Weibull function reached ideal performance (> 98%) at 125 ms and good performance (> 90%) at 92.6 ms. These data indicate that the timing of an actuator running at 8 Hz could be interpreted almost perfectly, and one running at 10 Hz would be viable with less than 8% loss of temporal information. Together, the results presented here suggest that an optimal actuator array can operate at a frequency of 8–10 Hz and should have 3-mm-diameter balloons with 1.5 mm spacing. Given these dimensions and the size limitations of the finger pad, it was determined that an ideal array would be composed of 3 × 2 elements. An example of an optimized tactile display is shown in Fig. 8 with a Velcro mounting strap attached to the rear of the substrate using silicone adhesive. The substrate thickness of 4.0 mm provides sufficient structural support with a minimal footprint, while allowing insertion of 2 mm tubing. This display is suitable for mounting on the master controls of the da Vinci Surgical System (see Fig. 9). IV. DISCUSSION The number of accurately detectable inflation levels for a given balloon diameter is an important factor in the determination of optimal balloon actuator architecture for haptic feedback. Ideally, an actuator that can generate a higher number of discrete levels will deliver more dynamic haptic feedback, allowing the tactile information to more closely resemble the actual sense of touch. Overall, the balloon diameter tests showed that the number of detectable inflation levels was related to the balloon diameter with smaller balloons delivering reduced accuracy. The 1.5 mm balloons delivered lowest average accuracy (85.3%) in the four-level tests, and the accuracy of combination 1–2 and 2–3 was similar to guessing. The smaller balloon also suffered from the reduced operational pressure range. The 1.5 mm

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

2598

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 55, NO. 11, NOVEMBER 2008

Fig. 9. Optimized balloon tactile displays mounted on the master control of the da Vinci Surgical System. The tactile displays were mounted at the thumb and index finger contact points of the master control.

balloon required 8 psi for the subject to feel the initial balloon actuation, suggesting that balloons with diameters smaller than 1.5 mm cannot effectively deliver four levels of reliable tactile sensation in the existing system configuration. The average performance with the 2.0 and 1.5 mm balloons was significantly worse for ramp-up than for ramp-down sequences, suggesting that the subjects may have been unable to accurately characterize and remember the lower pressure stimuli from these small balloons. This result confirms that these two diameters were suboptimal. Although 2.5 mm balloons achieved high accuracy in the four-level tests, the accuracy in the five-level tests was reduced significantly. This result indicates that balloons with diameters smaller than 2.5 mm were less effectively in delivering five levels of reliable tactile sensation. Diameter test results also suggested that larger balloons were easier to detect when the same pressure difference was applied. The five-level test results showed that when the 3–0 psi combination was applied, the 3.0 mm balloons achieved 90.8% accuracy, which was significantly higher than the detection accuracy of the 2.5 mm balloons (68.9%). Furthermore, subjects indicated that increased concentration was needed for smaller balloon diameters. Previous characterization tests demonstrated that the pressure-deflection profiles of the 2.5–3.0 mm balloons were similar [26]. Therefore, the increased accuracy may be due to the increased number of mechanoreceptors that were stimulated since larger balloons provided more contact area with the finger. Since it achieves high accuracy in the four-level and five-level tests, 3.0 mm was selected as the optimal diameter to provide five distinct levels of effective tactile information. Prior characterization of the actuators demonstrated that balloons with 4.0 mm diameter could not reproduce inflation deflections given a constant pressure input, and therefore, were not included in the study [26]. Using the current materials and fabrication technique, the 1.5 mm spacing was considered to be optimal for 3.0-mmdiameter balloons. This spacing provided 100% detection accu-

racy, did not exhibit channel extension or membrane disassociation, and allowed for dense array patterns. Denser spacing may be achieved with stiffer PDMS bases and stronger membrane adhesion techniques developed in future fabrication trials. The temporal tests indicated that the threshold for accurate detection of two time separated adjacent balloon actuations was 62.4 ms (for 3.0 mm balloons with 1.5 mm spacing). This value is slightly higher than the usual range of 20–60 ms, and is probably due to the fact that these stimuli provide less indentation at lower velocities than stimuli usually used to test temporal order judgments. To achieve optimal performance, we found that time separations of 93 ms would be adequate, and 125 ms would be ideal. These data show that this system running at 8 Hz would be ideal and frequencies as high as 10 Hz would still be viable. The study described here has enabled optimization of the actuator array, ready to be mounted on a clinical robotic surgical system. To fully evaluate the efficacy of tactile feedback in robotic surgery, a complete tactile feedback system will be required. This complete system will include force sensors mounted onto the “fingers” of robotic tools to detect force, magnitude, and location, actuators that supply tactile information to the surgeon, and a control system that translates the sensor inputs to proportional actuator outputs. This system may enable the surgeons to apply less force to tissues, monitor dynamic changes in grasping, and detect change in tissue hardness. V. CONCLUSION The described studies have demonstrated that the pneumatic balloon actuator, with accompanying pneumatics and control system, is a viable means to provide tactile feedback to the finger. The perceptual tests showed that detection accuracy is related to the balloon diameter, and that 3.0 mm diameter is the optimal balloon size for our array. An edge-to-edge spacing of 1.5 mm and time separation above 92 ms provided high accuracy in detecting multiple balloon stimuli. Together these results enable construction of an optimal 3 × 2 tactile display. While this tactile display design is under development for robotic surgery, it can also be adopted for other applications needing sensory substitution, including general robotic manipulation, prosthetic rehabilitation, virtual-reality-based gaming, surgical training, and flight controls and orientation. ACKNOWLEDGMENT The authors would like to thank A. T. Higa for her contributions to this project, Dr. E. Carmack Holmes and Prof. Gregory P. Carman for their support of this project, and to all the subjects in the experiment for their commitment and contribution to this study. REFERENCES [1] R. M. Satava, “Surgical robotics: The early chronicles: A personal historical perspective,” Surg. Laparoendosc. Endosc. Percutan. Tech., vol. 12, pp. 6–16, 2002. [2] G. H. Ballantyne, “Robotic surgery, telerobotic surgery, telepresence, and telementoring. Review of early clinical results,” Surg. Endosc., vol. 16, no. 10, pp. 1389–402, 2002.

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

KING et al.: OPTIMIZATION OF A PNEUMATIC BALLOON TACTILE DISPLAY FOR ROBOT-ASSISTED SURGERY

[3] Y. Munz et al., “The benfits of stereoscopic vision in robotic-assisted performance on bench models,” Surg. Endosc., vol. 18, no. 4, pp. 611– 616, 2004. [4] K. Moorthy et al., “Dexterity enhancement with robotic surgery,” Surg. Endosc., vol. 18, no. 5, pp. 790–795, 2004. [5] J. Marescaux, J. Leroy, M. Gagner, F. Rubino, D. Mutter, M. Vix, S. E. Butner, and M. K. Smith, “Transatlantic robot-assisted tele-surgery,” Nature, vol. 413, no. 6854, pp. 379–380, 2001. [6] J. Marescaux et al., “Telerobotic laparoscopic cholecystectomy: Initial clinical experience with 25 patients,” Ann. Surg., vol. 234, pp. 1–7, 2001. [7] A. R. Lanfranco, A. E. Castellanos, J. P. Desai, and W. C. Meyers, “Robotic surgery—A current perspective,” Ann. Surg., vol. 239, pp. 14–21, 2004. [8] C. R. Wagner, N. Stylopoulus, and R. D. Howe, “The role of force feedback in surgery: Analysis of blunt dissection,” in Proc. Symp. HAPTICS, 2002, pp. 68–74. [9] G. Tholey, J. P. Desai, and A. E. Castellanos, “Force feedback plays a significant role in minimally invasive surgery: Results and analysis,” Ann. Surg., vol. 241, no. 1, pp. 102–109, 2005. [10] M. Kitagawa, A. M. Okamura, B. T. Bethea, V. L. Gott, and W. A. Baumgartner, “Analysis of suture manipulation forces for teleoperation with force feedback,” in Proc. Int. Conf. Med. Image Comput. Comput. Assisted Intervention, 2002, vol. 2488, pp. 155–162. [11] C. R. Wagner and R. D. Howe, “Force feedback benefit depends on experience in multiple degree of freedom robotic surgery task,” IEEE Trans. Robot., vol. 23, no. 6, pp. 1235–1240, Dec. 2007. [12] M. Kitagawa, D. Dokko, A. M. Okamura, and D. D. Yuh, “Effect of sensory substitution on suture-manipulation forces for robotic surgical systems,” J. Thorac. Cardiovasc. Surg., vol. 129, pp. 151–158, 2005. [13] T. Fukuda et al., “Micro resonator using electromagnetic actuator for tactile display,” in Proc. Int. Symp. Micromechatronics. Human Sci., 1997, pp. 143–148. [14] H. Kajimoto, N. Kawakami, T. Maeda, and S. Tachi, “Electrocutaneous display as an interface to a virtual tactile world,” in Proc. IEEE Virtual Reality, 2001, pp. 289–290. [15] H. Tang and D. J. Beebe, “A microfabricated electrostatic haptic display for persons with visual impairments,” IEEE Trans. Rehabil. Eng., vol. 6, no. 3, pp. 241–248, Sep. 1998. [16] C. R. Wagner, S. J. Lederman, and R. D. Howe, “A tactile shape display using RC servomotors,” in Proc. HAPTICS, 2002, pp. 354–355. [17] Y. Ikei, K. Wakamatsu, and S. Fukuda, “Texture presentation by vibratory tactile display-image based presentation of a tactile texture,” in Proc. IEEE Virtual Reality Ann. Int. Symp., 1997, pp. 199–205. [18] P. M. Taylor, A. Hosseini-Sianaki, and C. J. Varley, “An electrorheological fluid-based tactile array for virtual environments,” in Proc. IEEE Int.Conf. Robot. Autom., 1996, pp. 18–23. [19] D. A. Kontarinis, J. S. Son, W. Peine, and R. D. Howe, “A tactile shape sensing and display system for teleoperated manipulation,” in Proc. IEEE Int.Conf. Robot. Autom., 1995, pp. 641–646. [20] K. Sato, E. Igarashi, and M. Kimura, “Development of non-constrained master arm with tactile feedback device,” in Proc. IEEE Int. Conf. Adv. Robot., 1991, pp. 334–338. [21] M. B. Cohn, M. Lam, and R. S. Fearing, “Tactile feedback for teleoperation,” in Proc. SPIE Teleman. Tech., 1992, pp. 240–255. [22] D. G. Caldwell, N. Tsagarakis, and C. Giesler, “An integrated tactile/shear feedback array for stimulation of finger mechanoreceptor,” in Proc. IEEE Int. Conf. Robot. Autom., 1999, pp. 287–292. [23] G. Moy, C. Wagner, and R. S. Fearing, “A compliant tactile display for teletaction,” in Proc. IEEE Int. Conf. Robot. Autom., 2000, pp. 3409–3415. [24] F. Vidal-Verdu, M. J. Madueno, and R. Navas, “Thermopneumatic actuator for tactile displays and smart actuation circuitry,” Proc. SPIE-Int. Soc. Opt. Eng., 2005, vol. 5836, pp. 484–492, Jul. 2005. [25] A. W. Goodwin, V. G. Macefield, and J. W. Bisley, “Encoding of object curvature by tactile afferents from human fingers,” J. Neurophysiol., vol. 78, no. 6, pp. 2881–2888, 1997. [26] C. H. King, M. Franco, M. Culjat, J. Bisley, E. Dutson, and W. Grundfest, “Fabrication and characterization of a balloon actuator array for haptic feedback in robotic surgery,” ASME J. Med. Devices, to be published. [27] C. H. King et al., “A pneumatic haptic feedback actuator array for robotic surgery or simulation,” in Proc. Med. Meets Virtual Reality, 2007, vol. 125, pp. 217–222. [28] (2008). [Online]. Available: Tiresias. org, http://www.tiresias.org/research/ reports/braille_cell.htm [29] I. J. Hirsh and C. E. Sherrick, “Perceived order in different sense modalities,” J. Exp. Psychol., vol. 62, no. 5, pp. 423–432, 1961.

2599

[30] S. Shibuya, T. Takahashi, and S. Kitazawa, “Effects of visual stimuli on temporal order judgments of unimanual finger stimuli,” Exp. Brain Res., vol. 179, no. 4, pp. 709–721, 2007. [31] F. A. Wichmann and N. J. Hill, “The psychometric function: I. Fitting, sampling, and goodness of fit,” Percept. Psychophys., vol. 63, pp. 1293– 1313, 2001.

Chih-Hung King (M’07) received the B.S. degree in electrical engineering and the M.S. and Ph.D. degrees in biomedical engineering in 2000, 2006, and 2008, respectively, from the University of California, Los Angeles (UCLA). Since 2005, he has been a Graduate Student Researcher at the UCLA Center for Advanced Surgical and Interventional Technology (CASIT). His current research interests include tactile display, telepresence, haptic system design and analysis, haptic perception, and haptic real-time control.

Martin O. Culjat (M’07) received the B.S. degree in bioengineering from the University of California, San Diego, in 2000, and the M.S. and Ph.D. degrees in biomedical engineering from the University of California, Los Angeles (UCLA), Los Angeles, in 2002 and 2005, respectively. From 2005 to 2007, he was a Postdoctoral Fellow in the Department of Surgery, UCLA. Since 2006, he has been the Engineering Research Director of the Center for Advanced Surgical and Interventional Technology (CASIT), Los Angeles, where he is currently an Assistant Research Engineer in the Department of Bioengineering and Surgery, UCLA. He is also an Adjunct Assistant Professor in the Department of Electrical and Computer Engineering, University of California, Santa Barbara (UCSB), Santa Barbara. His current research interests include ultrasound transducers and imaging, surgical robotics, rehabilitation engineering, haptic feedback, and terahertz imaging.

Miguel L. Franco received the B.A. degree in physics from Whittier College, Whittier, CA, in 2005, and the M.S. degree in biomedical engineering from the University of California, Los Angeles (UCLA), in 2008. Since 2006, he has been a Graduate Student Researcher at the Center for Advanced Surgical and Interventional Technology (CASIT), Los Angeles. His current research interests include haptic feedback, surgical robotics, medical instrumentation, prosthetics, and pneumatic systems.

James W. Bisley received the Ph.D. degree in neuroscience from the University of Melbourne, Melbourne, Vic., Australia, in 1998. He was a Postdoctoral Fellow at the University of Rochester, National Institutes of Health, and Columbia University. He then became an Assistant Professor of Neurobiology at the David Geffen School of Medicine, University of California, Los Angeles (UCLA). His current research interests include the somatosensory system, the neural mechanisms underlying the cognitive processes of short-term memory, and the allocation of spatial attention and visual perception.

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

2600

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 55, NO. 11, NOVEMBER 2008

Erik Dutson received the M.D. degree from the Eastern Virginia Medical School, Norfolk, in 1995. In 2000, he was trained in general surgery at the Eastern Virginia Medical School, and in 2003, he was further trained in advanced laparoscopy at the University of Louis Pasteur, Strasbourg, France. Since 2003, he has been the Co-Director at the Center for Advanced Surgical and Interventional Technology, Los Angeles, CA. He is also an Assistant Professor of surgery in the School of Medicine, University of California, Los Angeles (UCLA), Los Angeles.

Warren S. Grundfest received the M.D. degree from College of Physicians and Surgeons, Columbia University, New York, in 1980. In 1985, he was trained in general surgery at the University of California, Los Angeles (UCLA), and Cedars-Sinai Medical Center, Los Angeles. He is currently a Professor in the Department of Bioengineering, Electrical Engineering, and Surgery, UCLA. His current research interests include excimer lasers for medical applications, optical diagnostic procedures, minimally invasive surgical tools, haptic feedback, and ultrasound imaging. Dr. Grundfest was elected as a Fellow of the American Institute of Medical and Biological Engineers (AIMBE), for pioneering development and dissemination of minimally invasive surgery in 1996 and a Fellow of the Society of Photo-Optical Instrumentation Engineers (SPIE), for his distinguished and valuable contributions to the field of optical engineering in medicine and biology.

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on May 04,2010 at 21:55:40 UTC from IEEE Xplore. Restrictions apply.

Suggest Documents