Automation in Construction 17 (2008) 907–914

Contents lists available at ScienceDirect

Automation in Construction j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / a u t c o n

Remote control of backhoe at construction site with a pneumatic robot system Takahiro Sasaki ⁎, Kenji Kawashima Tokyo Institute of Technology, Precision and Intelligence Laboratory, 4259, Nagatuta, Midori-ku, R2-46,Yokohama, 226-8503, Japan

a r t i c l e

i n f o

Article history: Accepted 12 February 2008 Keywords: Remote control Construction machines Pneumatic artificial rubber muscle Disaster Robot arm

a b s t r a c t In this paper, remote control of backhoes is achieved at construction sites with a pneumatic robotic system we have developed. The system mainly consists of pneumatic robot arms actuated by pneumatic rubber muscles (PARM), CCD cameras and a control box containing power source, PC and wireless LAN boards. The remote control system was applied to two types of backhoe, one small with bucket size of 0.025 [m3] and another medium-sized bucket with 0.28 [m3]. Field tests were conducted at local construction sites to confirm the effectiveness of the system. The remote control operations were achieved with the working efficiency of more than 50% compared with that of the direct operation. The effectiveness of the system has been determined. © 2008 Published by Elsevier B.V.

1. Introduction In disaster sites, the remote control of construction machines is essential in order to minimize the injuries and loss of life. There is a substantial literature dealing with the development of remote control systems for manipulating the functional tasks of construction machines in disaster sites [1–6]. However, the ordinary systems are large and heavy so that the transportation takes time and is troublesome. The installation of robot to ordinary construction machinery is more effective for practical use in the opinion of transportation and quickness of the activity. A pneumatic system has a high weight–power ratio and compliance and clean energy to drive. Therefore, pneumatic robot systems that consist of pneumatic actuators were applied to the remote control of construction machinery [7]. We have developed a remote control system using pneumatic rubber muscle (PARM) as the actuator of the robot system [8,9]. The effectiveness of the system was demonstrated with a small backhoe. However, there are demands to make the system more compact to install more ordinary machines. In this research, we have newly developed a 6-DOF (degree of freedom) pneumatic module arm which consists of three 2-DOF modules. The modules make the system simpler and more portable. In addition, we also newly developed a control box to make the system compact for easy installation to the operation room of middle type backhoes. Two CCD cameras are installed on the pneumatic robot system in order to monitor the synchronized activities of the pneumatic robot system and backhoe machines from different perspective positions [10,11]. Images from the cameras are transported into the master side through a wireless LAN, and then display on a Laptop PC. The wireless

⁎ Corresponding author. E-mail address: [email protected] (T. Sasaki). 0926-5805/$ – see front matter © 2008 Published by Elsevier B.V. doi:10.1016/j.autcon.2008.02.004

LAN and PC boards, and the power source are built in the control box. The entire weight of the pneumatic robot system is only 40 kg. The system was installed on a small type backhoe whose bucket size is 0.025 [m3] and a middle type that of 0.28 [m3]. The effectiveness of the system was confirmed by some field tests at construction sites. 2. 6-DOF pneumatic module arm 2.1. Pneumatic artificial rubber muscle Pneumatic artificial rubber muscle is a noble actuator that has a high weight–power ratio [12–17]. As shown in Fig. 1, PARM is composed of rubber tube and fibers bladed around it. Driving system of single joint using PARMs is shown in Fig. 2. Two PARMs are connected to a link in parallel. A 5-port servo valve (MYPE-

Fig. 1. Pneumatic artificial rubber muscles (PARM) (Upper side: McKibben type, Lower side: fiber knitted type).

908

T. Sasaki, K. Kawashima / Automation in Construction 17 (2008) 907–914

Fig. 5. 2-DOF module (CAD Image).

Fig. 2. Joint driving system.

the amplitude is 10 [°] and each feedback gains are Kp = 35 [V/rad], KI = 20 [V·s/rad], Kp = 0.2 [V/(s·rad)] and TD = 0.02 [s]. The frequency responses were obtained by experimental results and were summarized in a bode diagram as shown in Fig. 4. As the delay cannot be observed at the frequency of 1 [Hz], it is enough for operating the lever of backhoes. 2.2. 2-DOF arm module

Fig. 3. Single position PI-D feedback control block diagram.

We have developed 2-DOF module using pneumatic artificial rubber muscles. Fig. 5 shows the CAD image of the module and Fig. 6 is photograph of the developed module. The operating range of each angle is 20 [°] and 40 [°] as shown in Table 1. The weight of the module is only about 1 [kg]. It can be seen from Fig. 6 that the driving part of the module is connected by ball joints with four PARMs. Fig. 7 shows the movement of 2-DOF arm module. Two pairs of BMDS realize 2DOF motions. Experimental results of joint driving at 1 [Hz] are shown in Fig. 8. Controlled joint ϕ1, ϕ2 is smoothly driven. Even there is a delay about 0.1 [s], both of joints were well controlled. 2.3. 6-DOF pneumatic robot arm Pneumatic robot arms coupled three 2-DOF modules are shown in Fig. 9. The size of upper arm and forearm are 490 [mm] and 400 [mm], respectively. The weight of the arm is only about 3 [kg] and 20 [N] force can be generated at the tip position. As each module is controlled two 5port servo valves, the arm is controlled with six servo valves. Rotary encoders were set to each joint and the rotation angles were measured. 3. Remote control system 3.1. Control box A control box was developed for the pneumatic robot systems as shown in Fig. 10. The control box is divided into three small units,

Fig. 4. Bode diagram of single position control of PARM.

5-M5-010B: FESTO) controls supply and exhaust air to PARMs. Then, the difference of contraction quantities of two PARMs creates joint torque that is called BMDS (Bi-Muscular Driving System). A rotary encoder was set to each joint and the rotation angle was measured and controlled. 1-DOF position control experiment was conducted using single loop position control as shown in Fig. 3. For experimental condition,

Fig. 6. Photograph of 2-DOF module.

T. Sasaki, K. Kawashima / Automation in Construction 17 (2008) 907–914

909

Table 1 Experimental results of achievement time No.

1 2 3 Ave.

Subject 1

Subject 2

Subject 3

Without camera [s]

With Without camera [s] camera [s]

With Without camera [s] camera [s]

With camera [s]

75 65 67 69

63 61 58 61

56 56 61 58

64 62 60 62

62 54 61 59

65 62 65 64

namely valve unit, PC unit and power source unit, considering the limitation of space in the cockpit of ordinary backhoes. The valve unit contains pneumatic servo valves and regulators. The PC unit includes

Fig. 9. Developed 6-DOF module arm.

CNT boards, an A/D board, a D/A board and a wireless LAN board. The outer size of the two box is W: 430 [mm] × D: 400 [mm] × H: 200 [mm]. The electric power source unit contains battery for valves and PC and its outer size is W: 430 [mm] × D: 200 [mm] × H: 200 [mm]. The total weight of the control box is only about 30 [kg]. All three units in the control box have the same width and height. This provides options and compositions for easy installation to a number of backhoe machines. Fig. 7. 2-DOF rotation experiment.

Fig. 8. Experimental results of 2-DOF arm module control.

Fig. 10. Control box.

910

T. Sasaki, K. Kawashima / Automation in Construction 17 (2008) 907–914

Fig. 11. Remote control system mounted on two types of backhoe.

3.2. Installation on to two type backhoes The system was mounted on two types of backhoe. Fig. 11-(a) depicts the remote control system mounted on a small type backhoe (PS-03 Manufactured by KOMATSU Ltd.), the bucket size of which is 0.025 [m3]. The units of

the control box were combined and were mounted on the back side of the arms. The system is ready to be controlled in 30 min by two persons. Two CCD cameras are also set in the front and the side of the system. The system was also mounted on a medium-sized backhoe (SR-60 Manufactured by KOBELCO Construction Machinery Co., Ltd.), the

Fig. 12. Remote control architecture.

T. Sasaki, K. Kawashima / Automation in Construction 17 (2008) 907–914

911

Fig. 13. Camera visions.

bucket size is 0.28 [m3] as shown in Fig. 11-(b). Compared with the small backhoe, the space is limited to install the system (control boxes and robot arms). However, the control boxes can be contained into the operation room piece by piece. Therefore, the system can be smoothly set to the backhoe and the setting time is only about an hour. 3.3. Remote control architecture Fig. 12 shows a remote control system. An operator uses joysticks to input the target tip position of the robot arms from the master side (right side in Fig. 12). Then a laptop PC takes the input values from the joysticks and transmits the values to the control box at the slave side (left side in Fig. 12) via wireless LAN (IEEE 802.11b). Then the robot arms, grasping the levers of the backhoe, manipulate the levers according to the transmitted value. As a result, the joystick movement and backhoe lever movement are synchronized. We have confirmed that the response delay is less than 200 [ms] which is acceptable for remote control. Two CCD cameras put on the robot are directly cableconnected with the access point which is contained in the electric power source unit. The access point relays signal from the CCD cameras directly to the laptop PC. The access point also relays command signal from an operator, who controls the robot arm remotely with joysticks, to the PC installed in the PC Unit.

An operation of a backhoe is composed of two main actions. One is the movement of the backhoe itself and another is the movement of the arm of the backhoe. Therefore, two particular visions are available for the operator to ensure the control of movements. Two CCD cameras are put on the robot system as shown in Fig. 13. The camera at the top center of the system mainly supports the movement of the backhoe. In addition, the operator can understand the direction of the front area of the backhoe. Another camera put on the side of the system supports the operator to watch the bucket and the excavating area. 4. Remote control experiment 4.1. Remote control experiments using small type backhoe Remote control experiment was carried out using a small and a medium-sized backhoe machines as shown in Fig. 11. The purpose of the experiments is to remotely drive the backhoe machines to carry out the following tasks: (a) (b) (c) (d)

Excavate the earth in front of the backhoe. Rotate the backhoe 180° to the right. Empty the load from the bucket to a defined area Rotation the backhoe 180° to the left. (e) Repeat (a)–(d) tasks (See Fig. 14)

To ensure the effectiveness of our system, (a)–(e) tasks were repeated three times, and the operation time was measured for each cases.

Fig. 14. Schema of remote control experiment.

Fig. 15. Schema of remote control experiment.

912

T. Sasaki, K. Kawashima / Automation in Construction 17 (2008) 907–914 Table 2 Experimental results of achievement time

Fig. 16. Schema of remote control experiment with a run.

Table 1 shows a summary of the achievement time of the tasks (a)–(e) when pneumatic robot system was installed on the small backhoe machine. The first column of numbers in Table 1 represents the measured time when the operator, without using the cameras, watched the synchronized activities with a face to face contact. With the cameras, the operator watched the synchronized activities from the two CCD cameras. The total time to achieve the task was summarized in Fig. 15. The average time for completing the tasks is smaller with the use of the CCD cameras. Moreover, it is clear that the repeatability of the remote control is higher using the CCD cameras. The effectiveness of the developed pneumatic robot system was confirmed.

No.

Exp. 1 [s]

1 2 3 Average

76 72 69 72

Exp. 2 [s] 110 93 113 105

Exp. 3 [s] 75 74 75 75

Next, the remote control experiment including the run of a small type backhoe was conducted. Fig. 16 illustrates the schema of remote control experiments and the details are as follows: (a) (b) (c) (d) (e) (f)

Move backhoe forward. Excavate the ground in front of backhoe. Rotate backhoe 90° to the right. Empty the load from the bucket to a defined area. Rotate backhoe 90° to the left. Move backhoe backward (See Fig. 16)

In this experiment, an operator controlled the backhoe from two different places where the operator was affected in operation of the backhoe. Experiment (1) The operator manipulates the backhoe watching directly with his/her eyes from the left side of the backhoe as shown in Fig. 17-(a). In this situation, an operator controlled the backhoe without the visions from the cameras because the operator can see the whole view of the backhoe and the excavating area. Experiment (2) The experiment was conducted in the situation that the operator and the backhoe are facing each other as shown in Fig. 17-(b). The operator watched the front side of the backhoe. In this situation, the operator tends to operate inversely especially at the rotation. Experiment (3) The experiment was conducted in the same situation as Experiment (2), but the operator is supported with visions from the CCD cameras set on the robot system. Experimental results are shown in Table 2 and in Fig. 18. The experiment was conducted three times in the situations described above. In Experiment (1), the operation was well done because the vision is clear. On the other hand, it takes additional time in Experiment (2) compared with Experiment (1). In Experiment (2), the control of the backhoe is little harder in the process (a) and (c). In Experiment (3), an operator can go through the task without mistake, so it takes little more time than Experiment (1) because of the confirmation of vision from the cameras. However, it is much faster than Experiment (2). It is clear that an operator can control the backhoe in any situation with the visual support. 4.2. Advanced remote control experiments using a middle type backhoe Finally, we conducted the remote control experiment using a middle type backhoe at a construction site. The experimental process

Fig. 17. Remote control experiment with a small backhoe.

Fig. 18. Average operation time with a run.

T. Sasaki, K. Kawashima / Automation in Construction 17 (2008) 907–914

913

Table 3 Experimental results with medium-sized backhoe Exp. no. A-1 A-2 A-3

Situation Remote control Remote control Direct operation

V [m3] 11.9 18.0 16.0

WE [m3/h]

T [h] −1

4.44 × 10 5.35 × 10− 1 2.88

26.8 33.6 55.7

operation time T [h] were measured in each experiment. Then, working efficiency WE was derived as follows: WE ¼ V=T

 3  m =h :

The working efficiency was compared with the remote control and with the direct operation. Table 3 shows the experimental results. The remote control operations were well achieved with the average of working efficiency of more than 50% compared with that of the direct operation. 5. Conclusion Fig. 19. Experimental process.

is to excavate 4 × 10 [m2] area, including the backward movement as shown in Fig. 19. Photographs during the experiments are shown in Fig. 20. Experiments were conducted with the remote control and with the direct operation. The excavated volume V [m3] and the

A remote control system for construction machinery was developed using pneumatic robot arms. We developed a robot arm module having 2 DOFs, a control box to make the system compact and a vision system to help the task. The high portability was realized since the weight of the system is only 40 kg. Then, the remote control system was applied with two types of backhoe, one small with bucket size of 0.025 [m3] and another medium-sized with 0.28 [m3]. The remote control experiments were successfully conducted with support of vision from the cameras. The remote control operations were well achieved at a construction site with the medium-sized backhoe. The working efficiency of the remote operation was more than 50% compared with the direct operation. Disasters caused by environmental factors and man-made actions are increasing worldwide. The safety of rescue workers and investigative officials in disaster areas is a growing concern. The system herein provides an opportunity to improve safety. Acknowledgment This research was supported in part by the “Development of Advanced Robots and Information Systems for Disaster Response” project of the Ministry of Education, Culture, Sports, Science and Technology in Japan, by FUJITA Corporation and by KOBELCO Construction Machinery Co., Ltd. References

Fig. 20. Experiment with a medium-sized backhoe.

[1] H. Yamada, S. Mukota, D. Zhao, T. Muto, Construction telerobot system with virtual reality (Development of a bilateral construction robot), Proc. of VSMM98, vol. 1, 1998, pp. 152–157. [2] M. Minamoto, K. Matsunaga, Tele-presence information and remote-controlled task execution, IEEE/RSJ International Conference on Intelligent Robots and Systems, 1998. [3] H. Hasunuma, et al., A tele-operated humanoid robot drives a lift truck, Proc. IEEE Int. Conf. on Robotics and Automation, 2002, pp. 2246–2252. [4] K. Yokoi, K. Nakashima, M. Kobayashi, H. Mihune, H. Hasunuma, Y. Yanagihara, T. Ueno, T. Gokyuu, K. Endou, A tele-operated humanoid robot drives a backhoe in the open air, IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003. [5] Ha. Quang, M. Santos, N. Quang, D. Rye, H. Durrant-Whyte, Robotics and automation in construction, IEEE Robotics and Automation Magazine 9-1 (2002) 4–6. [6] P. Saeedi, P.D. Lawrence, D.G. Lowe, P. Jacobsen, D. Kusalovic, K. Ardron, P.H. Sorensen, An autonomous excavator with vision-based track-slippage control, IEEE Transaction on Control Systems Technology 13 (1) (Jan. 2005) 67–84. [7] T. Tanaka, Y. Sakai, Y. Konomi, K. Chayama, M. Minamoto, K. Matsunaga, Development of a portable teleoperated robot for the manipulation of a backhoe shovel for the restoration of disaster stricken sites, Proc. of 12th Int. Symp. of the Int. Society for Optical Engineering, Telemanipulator and Telepresence Technologies, vol. VI, Sep. 1999, pp. 125–132. [8] K. Kawashima, T. Sasaki, T. Miyata, N. Nakamura, M. Sekiguchi, T. Kagawa, Development of robot using pneumatic artificial rubber muscles to operate construction machines, Journal of Robotics and Mechatronics 16 (1) (2004) 8–15.

914

T. Sasaki, K. Kawashima / Automation in Construction 17 (2008) 907–914

[9] T. Sasaki, K. Kawashima, Development of remote control system of construction machinery for rescue activities with pneumatic robot, Advanced Robotics 20 (2) (2006) 213–232. [10] K. Fukaya, T. Nakamura, S. Uumezaki, J. Lu, Y. Egawa, Development of excavator simulator and characteristic of operator, Proceedings of the 41st SICE Annual Conference, vol. 5, 2002, pp. 2815–2818. [11] M. Minamoto, K. Matsunaga, Tele-presence information and remote-controlled task execution, Proceedings of the 1998 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Victoria, B.C., Canada, 1998. [12] C. Chou, B. Hannaford, Measurement and modeling of McKibben pneumatic artificial muscles, IEEE Transactions on Robotics and Automation 12 (1996) 90–103. [13] G.K. Klute, J.M. Czerniecki, B. Hannaford, Artificial muscles: actuators for biorobotic systems, The International Journal of Robotics Research 21 (Apr. 2002) 295–309.

[14] T. Noritsugu, Y. Tsuji, K. Ito, Improvement of control performance of pneumatic rubber artificial muscle manipulator by using electroreological fluid damper, IEEE/ ASME Transactions on Mechatronics 2 (4) (1997) 259–267. [15] S. Tasagarakis, J. Canderle, D.G. Caldwell, Enhanced modeling and performance in braided pneumatic muscle actuators, The International Journal of Robotics Research 22 (3–4) (2003) 213–227. [16] S. Davis, N. Tsagarakis, J. Canderle, D.G. Caldwell, Enhanced modelling and performance in braided pneumatic muscle actuators, The International Journal of Robotics Research 22 (3–4) (2003) 213–227. [17] D.B. Reynolds, D.W. Repperger, C.A. Phillips, G. Brandry, Modeling the dynamic characteristics of pneumatic muscle, Annals of Biomedical Engineering 31 (3) (March 2003) 310–317.