Assessment of laser range finders in risky environments Jos´e Pascoal, Lino Marques and An´ıbal T. de Almeida Institute for Systems and Robotics University of Coimbra 3030-290 Coimbra, Portugal {zefranc, lino, adealmeida}@isr.uc.pt Abstract—This paper characterizes four commercial Laser Range Finders (LRF) while operating under adverse conditions, namely: low visibility, and multiple types of target surfaces, including different optical properties, angles and radiant surfaces. The study considered two 2D LRF commonly used in mobile robotics: the Sick LMS200 and the Hokuyo URG-04LX, and two industrial punctual LRFs: the Ifm Effector O1D100 and the Sick DT60. Based on the results obtained, a set of conclusions and recommendations are taken considering the utilization of LRF in mobile robots operating in risky and adverse environments, like firefighting applications.

I. INTRODUCTION Autonomous mobile robot navigation can only be achieved if a robot can accurately sense its environment in order to estimate its localization and the position of the obstacles around it. This problem is currently addressed by Simultaneous Localization And Mapping (SLAM) algorithms, but in order to be effective, these methods require accurate range data. In optimal operating conditions, Laser Range Finders (LRF) are an excellent choice to use in mobile robots to provide this type of data [2]. A LRF is a device which uses a laser beam in order to determine the distance to a reflective object. The most common form of laser range-finder operates on the time-of-flight principle by sending a laser pulse in a narrow beam towards the object and measuring the time taken by the pulse to be reflected off the target and returned to the sender. For optimal operation, these sensors need environments with high visibility and target surfaces with good reflectivity for any orientation (ideally white Lambertian surfaces), becoming frequently unusable when this is not the case. The performance of commercial LRFs has already been characterized by others. For example, Hebert and Krotkov [3] characterized the range and angular accuracy and precision of two 3D LRFs: an Erim and a Perceptron. Luo and Zhang [4] characterized an Acuity AccuRange 4000 in terms of the influence to environmental light level, and target surface optical properties and orientation. Ye and Borenstein [5] characterized a Sick LMS200 and Alwan et al. [1] characterized a Hokuyo PBS-03JN. This paper expands the previous works with a broader sample of models - two scanning LRFs and two punctual LRFs - and a broader sample of testing conditions particularly relevant for firefighting applications including smoking environments and radiant surfaces. II. EXPERIMENTAL SETUP Table I describes some of the major characteristics of the sensor used in this test. Several environments were set up in

Fig. 1.

Picture showing the four sensors used in these tests.

order to characterize the performance of the multiple range sensors evaluated in this study. This study considered two 2D LRF commonly used in mobile robotics: the Sick LMS-200 and the Hokuyo URG-04LX, and two punctual industrial LRF: the Ifm Effector O1D100 and the Sick DT60 (see Figure 1 for a picture of the four LRFs tested in this study). The experimental tests were made inside an enclosed testing space with 4x3x0.5 m3 . The punctual LRFs were connected to a 14 bit resolution NI USB 6009 data acquisition board and the measured data was obtained with the sensors configured to 4 − 20 mA current output mode using a 251,01Ω resistor with 20 ppm/◦C. All data was recorded using Matlab and the Data Aquisition Toolbox. The scanning LRFs have been connected to a computer using RS232 protocol and the data was recorded in a log file using Player-Stage software as a server. This software takes approximately 9 samples per second using the LMS200 driver with 180◦ scan and angular resolution of 0.5 degree increments. The URG-04LX driver was configured to take approximately 10 samples per second and 3 degrees of scanning angle. For static tests, the target surfaces were placed 2 meters away from the lasers in the top of a translation

TABLE I L ASER R ANGE F INDERS ’ S PECIFICATIONS

0,50 m

2,00 m

Hokuyo URG-04LX 50x50x70 160 240 0.36◦ 0.2 - 4 ±10 or 1% of range 785 USB/RS232 550mA @5V -10 to +50 2500

B

0.16 m

D A

Fig. 2.

C

Setup to test the sensors at multiple accurate distances.

axis, 16 centimetres above the ground (see Figure 2). For the linearity and sensitivity tests, a target surface (A) was placed 2 meters away from the LRFs, a precision translation axis (D) was used with the LRFs (B) placed 16 centimetres above the ground on the top of its actuator (see Figure 2). The translation rail (Micro-Controle GV 88) has a range of 1200 millimetres and is actuated by a stepping motor (C) that provides 0.1 millimetres per 1 step (Micro-Controle UE72). This stepping motor was controlled by a Technosoft Intelligent Servo Drive IDM680-8EI able to provide up to 256 microsteps per step. The environmental and the sensor surface temperatures were monitored with Texas Instruments TMP102 temperature sensors. These sensors interface to a higher level host through SMBus (in our case a Gumstix1 single board computer running Linux) and provide temperature measurements with 0.0625◦C resolution and ±0.5◦C accuracy in the range of −25◦C to +85◦C. The tests were made in an approximately constant luminosity of 200 lux (measured with a ISO-TECH ILM350 light meter). Unless otherwise specified, 2000 samples were gathered for each test performed. It was found that this number of measurements was sufficient to characterize the statistics of all the sensors analysed in this study. A. Warm-up time When a range sensor is turned on, its internal temperature increases until equilibrium between the power consumption and thermal dissipation to the environment is reached. During this process, called warm-up, the average output of a LRF can drift several millimetres (see Figures 3 to 6). This family 1 http://gumstix.com

Sick DT60 38x99x104 202 punctual – 0.2 to 5.3 ±10 655 4 − 20 mA; 0 − 10V < 170 mA @ 11 to 30V -25 to +55 750

IFM O1D100 42x52x59 203 punctual – 0.2 to 10 ±15 650 4 − 20 mA; 0 − 10V < 150 mA @ 18 to 30V -10 to +60 250

of measurements was made during about 2 hours for each sensor inside an acclimatized laboratory with an approximately constant temperature of about 20◦C. A temperature sensor was used to monitor the environmental temperature near the LRF and another temperature sensor was thermally coupled to the LRF case under test. A white MDF2 target surface located 2 meter far from the LRF was used. All the lasers were maintained disconnected from power for at least 5 hours before the beginning of these tests. Figure 3 shows that the URG-04LX reaches a stationary state about 40 minutes after starting the test. During this time, the sensor output drifts almost 2 cm. This time is far better than the warm-up time for the LMS200 that takes about 2 hours to stabilize and drifts more that 2 cm during that period (see Figure 4). Distance Measurement Distance (m)

Sick LMS200 155x210x156 4500 180 1/0.5/0.25◦ 80 ±7.5 905 RS422/RS232 650mA@24V 0 to +50 4000

2

1.95

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Samples

4

x 10

Laser case and environment temperature 26

Temperature (oC)

Sensor Dimensions (mm) Total Weight (g) Scanning Range (◦ ) Angular Resolution Range (m) Max. Error (mm) Wavelength (nm) Interface Power consumption Operating temperature (◦ C) Approximated Cost (e)

LASER CASE ENVIRONMENT

25 24 23 22 21 20

0

10

20

30

40

50

60

70

Time (min)

Fig. 3.

URG-04LX range drift during warm-up.

The punctual LRFs are much faster to stabilize (about 10 minutes) and show less drift (about 1 cm). A curiosity in the two LRFs tested is their opposing behaviour during warm-up: while the DT60 increases its output, the O1D100 decreases (see Figures 5 and 6). A major conclusion of this set of tests is that all LRFs suffer from drift of 1 to 2 cm during a warm-up time that can range from 10 minutes to 2 hours. 2 Medium

Density Fiberboard

TABLE II L INEARITY OF RANGE VALUES Distance Measurement Distance (m)

2.11 2.1 2.09 2.08 2.07 2.06

0

1

2

3

4

5

6

Samples

4

x 10

Laser case and environment temperature o

Temperature ( C)

26

24 LASER CASE

1.5

Mean StDev

1.5251 0.0018

Mean StDev

1.5811 0.0031

Mean Voltage Mean StDev

2.0248 1.5000 0.0054

Mean Voltage Mean StDev

1.6055 1.5000 0.0943

ENVIRONMENT

22

20

18

Distance

0

20

40

60

80

100

1.7 1.9 URG-04LX 1.7157 1.9138 0.0020 0.0020 LMS200 1.7810 1.9823 0.0038 0.0019 DT60 2.1889 2.3517 1.7000 1.9000 0.0066 0.0065 O1D100 1.6854 1.7662 1.6995 1.9007 0.0910 0.0899

2.1

2.3

2.5

2.1161 0.0026

2.3187 0.0018

2.5138 0.0020

2.1804 0.0043

2.3792 0.0038

2.5778 0.0038

2.5150 2.1001 0.0070

2.6783 2.3000 0.0069

2.8409 2.5000 0.0067

1.8454 2.1010 0.0908

1.9228 2.2978 0.0887

2.0049 2.5000 0.0865

120

Time (min)

Fig. 4.

LMS200 range drift during warm-up.

2.4

Measured distance (m)

Distance Measurement

Voltage (V)

2.45

2.44

2.43

2.42

2.41

0

0.5

1

1.5

2

2.5

2.2

2

1.8

3

Samples

5

x 10

1.6

Laser case and environment temperature 25 24

1.8

2

2.2

2.4

2.6

True distance (m)

Fig. 7.

22

URG-04LX range and standard deviation for the linearity test

21 0

5

10

15

20

25

30

35

40

45

50

Time (min)

Fig. 5.

B. Linearity of range values

DT60 range drift during warm-up.

Distance Measurement 1.9

Voltage (V)

1.6

23

20

1.85

1.8

1.75

1.7

0

0.5

1

1.5

2

2.5

3

Samples

5

x 10

Laser case and environment temperature 26 LASER CASE ENVIRONMENT

25

o

Temperature ( C)

1.4 1.4

LASER CASE ENVIRONMENT

o

Temperature ( C)

26

24 23 22 21 20

0

5

10

15

20

25

30

35

40

45

Time (min)

Fig. 6.

O1D100 range drift during warm-up.

50

Accurate displacements inside the range of the positioner allow an estimation of the sensors non-linearity and infinitesimal range increments allow to estimate their sensitivity. Using a Micro-Controle GV88 translation rail and the UE72 stepping motor it was possible to estimate the precision of the range measurements and identify the error in range readings. Each sensor was moved from a distance of 1.5 metres to 2.5 metres in 20 cm steps. In each stopping position 2000 measurements were taken. The results from these tests are shown in Table II. Figures 7, 8, 9 and 10 show the linearity of the range values and the standard deviation multiplied by a factor of 10, in order to be more representative. As we can see in Figure 10, the standard deviation from the O1D100 laser should be taken into account, since it is far bigger than the values for the other sensors tested in this paper. This test helped us to convert the punctual lasers voltage to distance, using Matlab interpolation function with extrapolation, for each of the distances. Using a linear regression for the output of the analog sensors using the values

TABLE III I NTERPOLATION VALUES FOR DISTANCE MEASUREMENT OF PUNCTUAL LASERS 2.6

Distance (m) 1.5 1.7 1.9 2.1 2.3 2.5

Measured distance (m)

2.4

2.2

2

O1D100 Mean Voltage 1.6055 1.6854 1.7662 1.8454 1.9228 2.0049

DT60 Mean Voltage 2.0248 2.1889 2.3517 2.5150 2.6783 2.8409

1.8

1.6

of table II, the following transfer functions are obtained:

1.4 1.4

1.6

1.8

2

2.2

2.4

dDT 60 = 6, 1349693 ×V − 11, 423313

2.6

True distance (m)

Fig. 8.

LMS200 range and standard deviation for the linearity test

(1)

dO1D100 = 12, 658228 ×V − 19, 316456

(2)

The interpolation for the following tests took in consideration the Table III values. C. Range accuracy To determine the range precision, the tests were made at 0.50, 1, and 2 meters away from the white MDF surface using a Edmund Scientific rail with 2.10 meters and mountings from the same company, 10.5 centimetres above the ground. The distance to the target surface was carried with a tape measure.

3

Measured voltage (V)

2.8

2.6

2.4 2.2 2.2

2

1.8

1.8 1.4

1.6

1.8

2

2.2

2.4

Measured distance (m)

2

2.6

True distance (m)

Fig. 9.

DT60 range and standard deviation for the linearity test

1.6

1.4

1.2

1

0.8

0.6

0.4 0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

True distance (m)

Fig. 11. test

Measured voltage (V)

2.2

2

URG-04LX range and standard deviation for the range accuracy

As we concluded in II-B, the IFM O1D100 presents excessive standard deviation.

1.8

D. Influence of target colour and surface properties 1.6

1.4

1.2 1.4

1.6

1.8

2

2.2

2.4

2.6

True distance (m)

Fig. 10.

O1D100 range and standard deviation for the linearity test

In order to analyse the influence of the target surface optical properties in the output of the LRFs, a set of eleven different surfaces was chosen. The colours and materials used were white MDF, a mirror, blue, red, yellow, white and black matte coloured cardboard, black velvet and aluminium foil, black plate and grey plate of aluminium, respectively. For a better characterization of the LRFs behaviour for multiple surfaces, Lambertian and specular surfaces were chosen with different

TABLE IV TARGET COLOR AND SURFACE INFLUENCE ON SENSORS 2.2

Surface

Color

2

Aluminium Black

Measured distance (m)

1.8

1.6

1.4

1.2

Cardboard

1

0.8

0.6

0.4 0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

True distance (m)

Fig. 12.

LMS200 range and standard deviation for the range accuracy test

MDF Mirror Velvet

Mean StDev Foil Mean StDev Gray Mean StDev Black Mean StDev Blue Mean StDev Red Mean StDev White Mean StDev Yellow Mean StDev White Mean StDev Mean StDev Black Mean StDev

URG04LX 2.0151 0.0027 2.0212 0.0041 2.0234 0.0045 2.0293 0.0045 2.0058 0.0028 2.0129 0.0022 2.0164 0.0022 2.0172 0.0022 2.0184 0.0022 2.0465 0.0049 4.0000 0

LMS200 DT60

O1D100

2.0776 0.0016 2.0774 0.0037 2.0407 0.0013 2.0814 0.0059 2.0837 0.0035 2.0837 0.0041 2.0764 0.0011 2.0777 0.0036 2.0795 0.0028 2.0700 0.0038 2.0798 0.0038

1.9991 0.0918 2.0057 0.0931 2.0065 0.0801 2.0194 0.0929 2.0100 0.0909 1.9988 0.0912 1.8064 0.0357 1.8068 0.0367 1.8063 0.0362 0.8765 0.0278 1.9164 0.2679

1.9945 0.0056 1.9986 0.0055 1.9891 0.0061 1.9994 0.0069 1.9943 0.0059 1.9913 0.0059 1.9985 0.0056 1.9990 0.0055 1.9996 0.0052 0.1048 0.0054 5.3246 0.0059

2

Measured distance (m)

1.8

TABLE V S TANDARD DEVIATION FOR TARGET ROTATION MEASUREMENTS

1.6

1.4

LRFs URG04LX LMS200 DT60 O1D100

1.2

1

0.8

0◦ 10◦ 20◦ 30◦ 40◦ 50◦ 60◦ 0.0020 0.0024 0.0029 0.0034 0.0037 0.0040 0.0040 0.0027 0.0013 0.0020 0.0036 0.0042 0.0055 0.0055 0.0061 0.0058 0.0055 0.0056 0.0058 0.0057 0.0058 0.0929 0.0922 0.0908 0.0921 0.0913 0.0917 0.0917

0.6

0.4 0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

True distance (m)

Fig. 13.

DT60 range and standard deviation for the range accuracy test

3

Measured distance (m)

2.5

2

E. Angular influence

1.5

1

0.5

0

−0.5 0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

True distance (m)

Fig. 14.

reflectivity for the LRFs wavelength. The LRFs were placed perpendicular to the target surface. As can be seen from the results obtained and presented in Table IV, the output distance is not independent of the surface optical characteristics. Although the output for all sensors is not changing significantly with the target colour, there is a noticeable influence according to the specular characteristics of the surfaces. An exception is black velvet that had a very strong influence on the measurements of the URG-04LX laser.

O1D100 range and standard deviation for the range accuracy test

The influence to different target orientations was tested and the target was the white MDF. A 60 degree rotation was carried, 10 degrees per turn, using a UE 30 Micro-Controle stepping motor with 0.01 degrees per step, by the target surface (see Figure 15). As shown in Figure 16, the target rotation modified the values of the distance to target. It becomes noticeable that the range values increase like if the target surface was moving away from the sensor. On the other hand, the variance is not affected by this circumstance as shown on Table V.



 







Fig. 17.

Fig. 15.

Setup for visibility test

to 2 meters away from white surfaces. The results of these tests can be seen in Figures 18 to 21. From the tests it can be observed that all sensors perform poorly in smoky conditions, but scanning sensors perform worse that the punctual ones. It can also be seen that at 2 metres distance, the sensors tested allow a reduction in visibility from 10 to 20%.

Target surface rotation system

Voltage on the Infrared sensor

2.025

URG−04LX

2.02

LMS200 DT60

Distance (m)

2.015

O1D100

2.01

2.005

Infrared sensor data 3 2.5 2 1.5 1 0.5 0

0

1

2

3

4

5

6

Samples

4

x 10

Distance Measurement

2

1.995

1.99 0

10

20

30

40

50

60

Rotation Angle (o)

Distance (m)

4

3

2

1

0

0

1000

2000

3000

4000

5000

6000

Samples

Fig. 16.

Lasers angular influence

Fig. 18.

URG-04LX measurements for reduced visibility environment

Infrared sensor data 3 2.5 2 1.5 1 0.5 0

0

1

2

3

4

5

6

7

8

Samples

9 4

x 10

Distance Measurement 10

Distance (m)

To evaluate the behaviour of the LRFs under reduced visibility, the sensors (3) were placed 2 meters away from a target white MDF surface (1) inside an enclosed chamber with a small opening to the exterior (see Figure 17). The visibility was successively reduced by the injection of smoke produced by a Magnum 800 smoke machine (4) and the smoke was homogenized by means of a ventilator (5). An opacity sensor made with an infrared emitter and an infrared photodetector was monitoring the opacity inside the testing area (2). Each test started without smoke during the first minute and then some smoke was injected during the next minute. During the remaining time of the test (28 minutes) the smoke was allowed to escape slowly by the opening, increasing the visibility inside the testing chamber. This setup allowed to identify the behaviour of each LRF in smoky environments and to identify the threshold of visibility that allows to measure distances up

Voltage on the Infrared sensors

F. Reduced visibility

8 6 4 2 0

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

Samples

Fig. 19.

LMS200 measurements for reduced visibility environment

2.5

Radiant surface disconnected Distance (m)

2 1.5 1 0.5 1

2

3

4

5

6

2.08 2.07

Samples

200

400

2

1000

1200

1400

1600

1800

2000

1400

1600

1800

2000

1400

1600

1800

2000

1600

1800

2000

1600

1800

2000

1600

1800

2000

1600

1800

2000

1600

1800

2000

1600

1800

2000

Radiant surface at 100V 2.1

µ=2.080241 σ= 0.003254

2.09 2.08 2.07

1.5

0

200

400

600

800

1000

1200

Samples

Radiant surface 220V

1 0.5 0

800

Samples

4

x 10

2.5

0

1

2

3

4

5

6

Samples

Fig. 20.

7 4

x 10

2.2

µ=2.110415 σ= 0.011547

2.15 2.1 2.05

0

200

400

600

800

1000

1200

Samples

DT60 measurements for reduced visibility environment Fig. 22.

Radiant influence on URG-04LX.

Infrared sensor data 3 2.5 2 1.5 1

Radiant surface disconnected

0.5 0

0

1

2

3

4

5

6

Samples

7 4

x 10

Distance Measurement

Distance (m)

Voltage on the Infrared sensors

600

7

Distance Measurement Distance (m)

µ=2.077786 σ= 0.002389

2.09

0 0

Distance (m)

0

Distance (m)

Voltage on the Infrared sensors

Infrared sensor data 3

2.14 2.13 2.12 2.11

0

200

400

800

1000

1200

1400

Samples Distance (m)

6 4 2 0

600

Radiant surface at 100V

8 2.16

µ=2.127687 σ= 0.005569

2.14 2.12 2.1

0

200

400

600

800

1000

1200

1400

Samples 0

1

2

3

4

5

Samples

6

Radiant surface at 220v

7 4

x 10

O1D100 measurements for reduced visibility environment

Distance (m)

Distance (m)

10

Fig. 21.

µ=2.125028 σ= 0.003587

11

µ=8.190000 σ= 0.000000

10 9 8 7

0

200

400

600

800

1000

1200

1400

Samples

G. Radiant surfaces

III. CONCLUSIONS Four commercial Laser Range Finders widely used in mobile robotics were tested in extreme environmental conditions, particularly relevant for firefighting applications. In optimal

Fig. 23.

Radiant influence on LMS200.

Distance (m)

Radiant surface disconnected 2.12

µ=2.071690 σ= 0.006243

2.1 2.08 2.06 2.04

0

200

400

600

800

1000

1200

1400

Samples Distance (m)

Radiant surface at 100V 2.12

µ=2.074359 σ= 0.005920

2.1 2.08 2.06 2.04

0

200

400

600

800

1000

1200

1400

Samples

Radiant surface at 220V Distance (m)

A domestic radiant heater was used as a target radiant surface. As with the previous tests, the target was placed 2 meters away from the sensors. The temperature of the heater was adjusted with a variable transformer and a set of tests were made at three temperatures: environmental temperature, a medium hot temperature obtained with the heater supplied at about 100 Volts, (tests made with a thermal radiation sensor gave an average temperature of 200 ◦C, but this temperature should be interpreted as the average temperature of the heater surface, that includes the metallic protection grid), and a high temperature with the heater supplied at full 230 V voltage and with the radiation elements presenting a bright orange colour. Figures 22 to 25 show the behaviour of the sensors to these testing conditions. It can be seen that the output of the sensors degrades with the surface temperature, becoming unusable when the surfaces are very hot.

5.36

µ=5.324981 σ= 0.006568

5.34 5.32 5.3

0

200

400

600

800

1000

1200

1400

Samples

Fig. 24.

Radiant influence on DT60.

Distance (m)

Radiant surface disconnected µ=2.061085 σ= 0.093849

2.4 2.2 2 1.8

0

200

400

600

800

1000

1200

1400

1600

1800

2000

1600

1800

2000

1600

1800

2000

Samples Distance (m)

Radiant surface at 100V µ=2.056915 σ= 0.094154

2.4 2.2 2 1.8

0

200

400

600

800

1000

1200

1400

Samples Distance (m)

Radiant surface at 220V 15

µ=3.051879 σ= 2.804618

10 5 0

0

200

400

600

800

1000

1200

1400

Samples

Fig. 25.

Radiant influence on IFM

conditions, namely good visibility, no interfering sources, and surfaces with good reflectivity in the sensor direction, LRFs are excellent range sensors, particularly after warming-up, in terms of linearity and accuracy. But in adverse environments, when the previous conditions are not met, LRFs provide erroneous or saturated outputs, becoming unusable as a range sensor for robotics. The results published in this paper, although extensive, representing the culminate of several full days setting up different testing environments and gathering measurements, are seen as a starting work in terms testing range sensors in extreme environments. In the future, the authors intend to test the behaviour of other types of sensors, like sonars and microwave radars under this type of environments. ACKNOWLEDGMENTS This work was partially supported by the Portuguese Science and Technology Foundation (FCT/MCTES) by project RoboNose, contract POSI/SRI/48075/2002 and by project GUARDIANS contract FP6-IST-045269. R EFERENCES [1] M. Alwan, M. Wagner, G. Wasson, and P. Sheth. Characterization of infrared range-finder PBS-03JN for 2-d mapping. In Proc. IEEE Int. Conf. on Robotics and Automation, 2005. [2] M.C. Amann, T. Bosch, M. Lescure, R. Myllyla, and M. Rioux. Laser ranging: a critical review of usual techniques for distance measurement. Opt. Eng., 40(1):10–19, 2001. [3] M. Hebert and E. Krotkov. 3D measurements from imaging laser radars: how good are they? Image and Vision Computing, 10(3):170–178, 1992. [4] Xiujuan Luo and Hong Zhang. Characterization of acuity laser range finder. In 8th intl. Conf. on Control, Automation, Robotics and Vision. IEEE, 2004. [5] Cang Ye and Johann Borenstein. Characterization of a 2-D laser scanner for mobile obstacle negotiation. In Proc. IEEE Int. Conf. on Robotics and Automation, 2002.