IEEE/RSJ IROS’04 – Sendai, Japan TTP4 : Tutorial on Advanced Visual Servoing September the 28th
Applications in visual servoing Professor Philippe Martinet IFMA – LASMEA Clermont-Ferrand, France
[email protected] http://wwwlasmea.univ-bpclermont.fr/Philippe.Martinet/Welcome.html Philippe Martinet
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
Outline of the presentation
Manipulator robot Visual servoing : 2D and 3D approach, Stereo Turning around a complex object Force/Vision control Mobile Robot Automatic Guided vehicule in agricultural context. Automatic Guided vehicule in indoor context. Parallel Robot Vision based control using legs observation Visual servoing toolbox On line Demonstration Philippe Martinet
2
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
Manipulator robot : 2D visual servoing
T = .L .(S (t ) S *) T+ S
S*
-
Control law
+
T
inverse Jacobian
q&
Power Controller
S (t )
Joint feedback
f (t )
Features Extraction
Visual feedback
Sensor of 2D features
Philippe Martinet
3
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Turning around a complex object
Motyl [92], Urban[90] Coupling a camera with laser stripes
ASEA IRB5
Polyhedral object
Spherical object
Secondary task Philippe Martinet
4
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Turning around a complex object
Berry [99] Turning around a complex object by visual servoing Trajectories generation and control in image space Translation - Rotation - Helicoid Trajectory generation
Trajectory Control
e = C.(S (t ) S * (t ) )
3D desired Trajectory
S * (t )
T = L ..(S (t ) S * (t ) )+ L . T+
T+
S
S
Philippe Martinet
5
d S * (t ) dt
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Turning around a complex object
Berry [99] Trajectories generation and control in image space Experimental context
Translation & Rotation linkage
Philippe Martinet
6
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Turning around a complex object
Berry [99] Turning around a complex object by visual servoing
Turning around Linkage
Philippe Martinet
7
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
Manipulator robot : 3D visual servoing T = .L .(S (t ) S *) T+ S
S*
-
Control law
+
T
Inverse Jacobian
q&
Power Controller
S (t )
Joint feedback
Pose Estimation
f (t )
Features Extraction
Visual feedback
Sensor of 3D features
Philippe Martinet
8
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
3D
… 3D Visual servoing
Martinet [96,99] Positioning task face to a tetrahedron
Positioning task
Philippe Martinet
9
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
3D
… 3D Visual servoing
Positioning task face to an unknown object
Dhome, Jurie
Positioning task
Philippe Martinet
10
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
ST
… Stereo Visual servoing
Robotic Plateform (Jaume I university, Castellon, Spain) Manipulator Robot with 7 ddl : Mitsubishi PA-10
Mobile robot : Nomad 200 Philippe Martinet
11
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
ST
… Stereo Visual servoing
Robotic Plateform
b
Color CCD Cameras Video Signal Cognachrome board Cognachrome board
S erial link
PC Win98
S erial link
Image : 200*250
Fu=300 Fv=450 b=118mm
Te = 33ms Philippe Martinet
12
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
ST
… Stereo Visual servoing Cervera [99,01,02,03] Comparaison study : 3D point 2D Stereo point 2D image point + depth 2D image point + disparity Case study for grasping: Oriented blob
Philippe Martinet
13
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
FV
… Force and vision coupling Force feedback Control
LASMEA IFMA-SOCRATES Robot control By vision/force coupling
Visual servoing
External Position/force
Hybrid Position Force
External Vision/force
Hybrid Vision Force
Hybrid Vision Force : case study Philippe Martinet
14
Done by M. Prats from Castellon, Spain
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
Outline of the presentation
Manipulator robot Visual servoing : 2D and 3D approach, Stereo Turning around a complex object Force/Vision control Mobile Robot Automatic Guided vehicule in agricultural context. Automatic Guided vehicule in indoor context. Parallel Robot Vision based control using legs observation Visual servoing toolbox On line Demonstration Philippe Martinet
15
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles
Debain [96] Control laws for agricultural machines Combine-harvester
Harvesting work Philippe Martinet
16
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles
Debain [96]
Control
Guiding on a flat ground
Philippe Martinet
17
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles
Debain [96]
Control
Guiding on a sloping ground
Lateral offset Driving sideways
Philippe Martinet
18
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles
Debain [96]
Control
Guiding on a sloping ground
Philippe Martinet
19
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles : target tracking
Clady [02] Fixed focal length camera Wide angle
Sensor
Our Vision sensor
Decision Module
Actuators
PTZ Controlled Camera
Philippe Martinet
20
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles : target tracking
Clady [02]
Philippe Martinet
21
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles : visual memory
Blanc [04] Ait Ader[04] Navigation using visual memory
Philippe Martinet
22
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
2D
… Automatic Guided Vehicles : visual memory
Blanc [04] Ait Ader[04] Navigation using visual memory
Philippe Martinet
23
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
Outline of the presentation
Manipulator robot Visual servoing : 2D and 3D approach, Stereo Turning around a complex object Force/Vision control Mobile Robot Automatic Guided vehicule in agricultural context. Automatic Guided vehicule in indoor context. Parallel Robot Vision based control using legs observation Visual servoing toolbox On line Demonstration Philippe Martinet
24
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
Parallel robot Andreff, Marchadier [04]
Stewart plateform
Parallel robot ?
Control of parallel robot using legs observation Philippe Martinet
25
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
Outline of the presentation
Manipulator robot Visual servoing : 2D and 3D approach, Stereo Turning around a complex object Force/Vision control Mobile Robot Automatic Guided vehicule in agricultural context. Automatic Guided vehicule in indoor context. Parallel Robot Vision based control using legs observation Visual servoing toolbox On line Demonstration Philippe Martinet
26
IEEE/RS J IROS ’04 – S endai, Japan TTP4 : Tutorial on Advanced Visual S ervoing Tuesday, S eptember the 28th
