Formation Control of Nonholonomic. Mobile Robots with Omnidirectional Visual Servoing and Motion. Segmentation. Johns Hopkins University
Omid Shakernia Shankar Sastry Department of EECS, UC Berkeley
René Vidal Center for Imaging Science Johns Hopkins University
Formation Control of No...
Omid Shakernia Shankar Sastry Department of EECS, UC Berkeley
René Vidal Center for Imaging Science Johns Hopkins University
Formation Control of Nonholonomic Mobile Robots with Omnidirectional Visual Servoing and Motion Segmentation
Safety in numbers Decreased aerodynamic drag Higher traffic throughput Applications in defense, space exploration, etc
Formation control is ubiquitous
Motivation
2
Generalization of string stability to a planar mesh
Structure of interconnections and amount of information communicated affects ISS
Differential geometric conditions on feasibility of formations under kinematic constraints of mobile robots
Leader position estimated by vision; Formation control in task space
Vision-based formation control [Das et.al. TAC02]
Feasible formations [Tabuada et.al. ACC01]
Input-to-state stability [Tanner et.al ICRA02]
Mesh stability of UAVs [Pant et.al. ACC01]
Formation can become unstable due to error propagation
String stability for line formations [Swaroop et.al. TAC96]
Formation control has a very rich literature (short list):
Previous Work
3
Distributed formation control (no explicit communication) Formation specified in image plane of each follower Multi-body motion segmentation to estimate leader position Followers employ tracking controller in the image plane Naturally incorporate collision avoidance by exploiting geometry of omni-directional images
Our Approach
4
Central panoramic cameras, back-projection ray Central panoramic optical flow equations Multi-body motion segmentation
Leader-follower dynamics in image plane Feedback linearization control design Collision avoidance using navigation functions
Motion segmentation of robots in real sequence Vision-based formation control simulations
Efficiently compute back-projection ray associated with each pixel in image
Catadioptric camera is lens-mirror combination Central panoramic: single effective focal point
Central Panoramic Camera
6
Central Panoramic Optical Flow
Optical flow induced by a planar camera motion with velocities and
Central Panoramic Panoramic Projection Projection Model Model Central
Central Panoramic Optical Flow
7
independent motions
Number of independent motions is obtained as:
Given
Optical flows of pixels in frames live in a 5 dimensional subspace. Optical flows can be factorized into structure and motion
Central Panoramic Motion Segmentation
8
Project onto a subspace of dimension 6 Apply GPCA: fit and differentiate a polynomial
Independent motions live in 5 dimensional subspaces of a higher-dimensional subspace Motion segmentation can be solved using Generalized Principal Component Analysis
Central Panoramic Motion Segmentation
9
Recover leader velocity using optical flow of background
10
Write as drift-free control system:
Inputs Leader position in follower’s camera Central panoramic panoramic leader-follower leader-follower dynamics dynamics Central
Kinematic model
Image Leader-Follower Dynamics
Polar coordinates
Controlling in Cartesian coordinates, leader trajectory intersects circle Controlling in polar coordinates, follower mostly rotates Trajectory passing through inner circle is a collision
Cartesian coordinates
Omnidirectional Visual Servoing
11
Leader position
, desired leader position
due to geometry of central panoramic cameras Corresponds to horizon points at infinity
due to nonholonomy of mobile robot Robot can not move sideways instantaneously
Degenerate configurations
Feedback Linearization Linearization Control Control Law Law in in Polar Polar Coordinates Coordinates Feedback
Omnidirectional Visual Servoing
12
13
However, the formation is only Input-to-State Stable (ISS) Can easily modify control law to achieve collision avoidance by using a Navigation Function
Can avoid degenerate configurations with pseudofeedback linearizing control law
Omnidirectional Visual Servoing
Multi-body motion segmentation
Experimental Results
14
Green follows red Blue follows red
Wedge Formation
15
Distance to leader (in pixels)
Wedge Formation
Angle to leader (in degrees) 16
Green follows red Blue follows green
String Formation
17
Distance to leader (in pixels)
String Formation
Angle to leader (in degrees) 18
Generalize formation control to UAV dynamics Hybrid theoretic formation switching control Implement on BEAR fleet of UGVs and UAVs
Future work
A framework for distributed formation control in the omni-directional image plane An algorithm for multi-body motion segmentation in omni-directional images
