Multi-Objective Mission Flight Planning in Civil Unmanned Aerial Systems by Paul P.-Y. Wu BEng(Elec&CompEng)(Hons), MEng(Comp&Comn)
Australian Research Centre for Aerospace Automation School of Engineering Systems Queensland University of Technology
Submitted for the degree of Doctor of Philosophy Supervisors: Associate Professor Duncan Campbell Professor Rodney Walker Dr Torsten Merz (CSIRO)
2009
In loving gratitude to my father and my mother.
ii
Statement of Authorship The work contained in this thesis has not been previously submitted to meet requirements for an award at this or any other higher education institution. To the best of my knowledge and belief, the thesis contains no material previously published or written by another person except where due reference is made.
..................................... Paul P.-Y. Wu
..................................... Date
iii
Abstract Unmanned Aerial Vehicles (UAVs) are emerging as an ideal platform for a wide range of civil applications such as disaster monitoring, atmospheric observation and outback delivery. However, the operation of UAVs is currently restricted to specially segregated regions of airspace outside of the National Airspace System (NAS). Mission Flight Planning (MFP) is an integral part of UAV operation that addresses some of the requirements (such as safety and the rules of the air) of integrating UAVs in the NAS. Automated MFP is a key enabler for a number of UAV operating scenarios as it aids in increasing the level of onboard autonomy. For example, onboard MFP is required to ensure continued conformance with the NAS integration requirements when there is an outage in the communications link. MFP is a motion planning task concerned with finding a path between a designated start waypoint and goal waypoint. This path is described with a sequence of 4 Dimensional (4D) waypoints (three spatial and one time dimension) or equivalently with a sequence of trajectory segments (or tracks). It is necessary to consider the time dimension as the UAV operates in a dynamic environment. Existing methods for generic motion planning, UAV motion planning and general vehicle motion planning can not adequately address the requirements of MFP. The flight plan needs to optimise for multiple decision objectives including mission safety objectives, the rules of the air and mission efficiency objectives. Online (in-flight) replanning capability is needed as the UAV operates in a large, dynamic and uncertain outdoor environment. This thesis derives a multi-objective 4D search algorithm entitled MultiStep A* (MSA*) based on the seminal A* search algorithm. MSA* is proven to find the optimal (least cost) path given a variable successor operator (which enables arbitrary track angle and track velocity resolution). Fur-
iv
v thermore, it is shown to be of comparable complexity to multi-objective, vector neighbourhood based A* (Vector A*, an extension of A*). A variable successor operator enables the imposition of a multi-resolution lattice structure on the search space (which results in fewer search nodes). Unlike cell decomposition based methods, soundness is guaranteed with multi-resolution MSA*. MSA* is demonstrated through Monte Carlo simulations to be computationally efficient. It is shown that multi-resolution, lattice based MSA* finds paths of equivalent cost (less than 0.5% difference) to Vector A* (the benchmark) in a third of the computation time (on average). This is the first contribution of the research. The second contribution is the discovery of the additive consistency property for planning with multiple decision objectives. Additive consistency ensures that the planner is not biased (which results in a suboptimal path) by ensuring that the cost of traversing a track using one step equals that of traversing the same track using multiple steps. MSA* mitigates uncertainty through online replanning, Multi-Criteria Decision Making (MCDM) and tolerance. Each trajectory segment is modeled with a cell sequence that completely encloses the trajectory segment. The tolerance, measured as the minimum distance between the track and cell boundaries, is the third major contribution. Even though MSA* is demonstrated for UAV MFP, it is extensible to other 4D vehicle motion planning applications. Finally, the research proposes a self-scheduling replanning architecture for MFP. This architecture replicates the decision strategies of human experts to meet the time constraints of online replanning. Based on a feedback loop, the proposed architecture switches between fast, near-optimal planning and optimal planning to minimise the need for hold manoeuvres. The derived MFP framework is original and shown, through extensive verification and validation, to satisfy the requirements of UAV MFP. As MFP is an enabling factor for operation of UAVs in the NAS, the presented work is both original and significant.
Keywords Path planning, heuristic algorithms, multi-objective decision making, multiobjective optimisation, expert systems, unmanned aerial vehicles.
vi
Acknowledgments The author would like to express his sincere gratitude to the patient help and wise direction of A/Prof. Duncan Campbell. Without his guidance, and the kindly and knowledgeable assistance of Dr Torsten Merz and Prof. Rodney Walker, this research would not have been possible. Special thanks are also extended to Prof. Gilles Coppin for providing an unique opportunity to study with his research group at Telecom Bretagne. The author is grateful for the amiable assistance and the insight provided by himself, A/Prof. Patrick Meyer and Dr Frederic Cadier. Finally, the author would like to express thanks to the fellow researchers at the Australian Research Centre for Aerospace Automation (ARCAA) - without their help and camaraderie, the PhD would not have been survivable.
vii
Contents Statement of Authorship
iii
Abstract
iv
Keywords
vi
Acknowledgments
vii
List of Figures
xvii
List of Tables
xviii
List of Abbreviations
xix
1 Introduction 1.1 Motivation and Significance . . . . . . . . . 1.2 Mission Flight Planning Task . . . . . . . . 1.3 Research Questions . . . . . . . . . . . . . . 1.3.1 Research Objectives . . . . . . . . . 1.3.2 Research Outcomes and Application 1.4 Research Methodology . . . . . . . . . . . . 1.5 Research Contributions . . . . . . . . . . . . 1.6 Publications . . . . . . . . . . . . . . . . . . 1.7 Format of Thesis . . . . . . . . . . . . . . .
. . . . . . . . .
1 2 5 8 10 11 11 13 14 15
2 Review of Motion Planning Theory 2.1 Planning Space . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Sampling Based Motion Planning . . . . . . . . . . . . . . . . 2.2.1 Sampling Methods . . . . . . . . . . . . . . . . . . . .
17 19 22 23
viii
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
CONTENTS . . . . . . . . . . .
25 26 31 34 36 36 38 42 43 47 48
3 Review of Expert Pilot Decision Making 3.1 Overview of Human Cognition . . . . . . . . . . . . . . . . . . 3.2 Cognitive Models . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Recognition Primed Decision Making . . . . . . . . . . 3.2.2 Rasmussen’s Model . . . . . . . . . . . . . . . . . . . . 3.3 Cognitive Processes . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Articulation of Elementary Strategies . . . . . . . . . . 3.3.3 Decision Making as a Search for a Dominance Structure 3.4 Decision Making Under Uncertainty . . . . . . . . . . . . . . . 3.5 Implementation of IF THEN Rules . . . . . . . . . . . . . . . 3.5.1 Fuzzy Logic . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Multi-Attribute Utility Theory . . . . . . . . . . . . . 3.6 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51 52 54 55 55 58 58 59 61 62 64 64 67 68
4 Mission Flight Planning Design Approach 4.1 Review of Autonomous Vehicle Motion Planning . . . . . 4.1.1 Methods with High Resolution Track Angles . . . 4.1.2 Multi-Objective Planning Algorithms . . . . . . . 4.1.3 Findings . . . . . . . . . . . . . . . . . . . . . . . 4.2 Synthesised Approach . . . . . . . . . . . . . . . . . . . 4.2.1 Motion Planning . . . . . . . . . . . . . . . . . . 4.2.2 Expert Cognition . . . . . . . . . . . . . . . . . . 4.2.3 Summary of Findings and Synthesised Approach .
71 71 72 75 78 78 78 80 82
2.3
2.4 2.5 2.6
2.2.2 Collision Detection . . . . . . . . . . . . 2.2.3 Incremental Sampling and Searching . . 2.2.4 Roadmap Based Planning . . . . . . . . 2.2.5 Cell Decomposition and Multi-Resolution Graph Search Algorithms . . . . . . . . . . . . . 2.3.1 Dynamic Programming . . . . . . . . . . 2.3.2 Best First Search . . . . . . . . . . . . . 2.3.3 Other Search Algorithms . . . . . . . . . Planning with Uncertainty . . . . . . . . . . . . Computational Complexity . . . . . . . . . . . . Findings . . . . . . . . . . . . . . . . . . . . . .
ix . . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . .
. . . . . . . . . . .
. . . . . . . .
. . . . . . . .
x
CONTENTS
5 Mission Flight Planning Framework 5.1 Multi-Step A* (MSA*) . . . . . . . . . . . . . . 5.1.1 Multi-Step 3D Successor Operator . . . . 5.1.2 Extending the Successor Operator to 4D 5.1.3 Search Algorithm . . . . . . . . . . . . . 5.1.4 Cost Optimality of MSA* . . . . . . . . 5.2 Multi-Resolution Lattice Structure . . . . . . . 5.2.1 3D Lattice . . . . . . . . . . . . . . . . . 5.2.2 3D Multi-Resolution Lattice . . . . . . . 5.2.3 4D Multi-Resolution Lattice . . . . . . . 5.3 Architecture . . . . . . . . . . . . . . . . . . . . 5.3.1 Planner Subsystem . . . . . . . . . . . . 5.3.2 Self-Scheduling Replanning Architecture 5.4 Summary . . . . . . . . . . . . . . . . . . . . . 6 UAV Flight Planning Application 6.1 Decision Objectives . . . . . . . . . . . . . . . 6.1.1 Safety Objective . . . . . . . . . . . . 6.1.2 Rules of the Air . . . . . . . . . . . . . 6.1.3 Mission Efficiency Objective . . . . . . 6.1.4 Summary . . . . . . . . . . . . . . . . 6.2 Experimental Setup . . . . . . . . . . . . . . . 6.2.1 MCDM Cost Function . . . . . . . . . 6.2.2 Test Algorithms . . . . . . . . . . . . . 6.2.3 Dynamic Constraints . . . . . . . . . . 6.2.4 Simulation World . . . . . . . . . . . . 6.3 Simulations and Results . . . . . . . . . . . . 6.3.1 Computation time . . . . . . . . . . . 6.3.2 Total path cost . . . . . . . . . . . . . 6.3.3 Selected Scenarios . . . . . . . . . . . . 6.3.4 Illustration of Replanning Architecture 6.4 Findings . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
84 85 85 87 89 92 94 94 97 98 99 101 104 107
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
108 . 109 . 109 . 114 . 116 . 117 . 117 . 118 . 118 . 121 . 121 . 123 . 126 . 129 . 130 . 137 . 140
7 Analysis and Extension 141 7.1 Special Planning Cases . . . . . . . . . . . . . . . . . . . . . . 141 7.2 Online Replanning . . . . . . . . . . . . . . . . . . . . . . . . 143
CONTENTS 7.3 7.4 7.5 7.6
Lattice Structure Path Cost . . . . Uncertainty . . . Findings . . . . .
xi . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
8 Conclusion 8.1 Key Contributions . . . . . 8.2 Significance and Originality 8.3 Future Work . . . . . . . . . 8.4 Concluding Remarks . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
145 154 156 159
. . . .
161 . 164 . 165 . 167 . 168
Bibliography
168
A MACBETH Value Functions
185
B Example Planning Scenarios B.1 Scenario 1 . . . . . . . . . . B.2 Scenario 2 . . . . . . . . . . B.3 Scenario 3 . . . . . . . . . . B.4 Scenario 4 . . . . . . . . . . B.5 Scenario 5 . . . . . . . . . . B.6 Mountain Wind Scenario . . B.7 Replanning Scenario . . . .
188 . 188 . 191 . 193 . 195 . 198 . 201 . 201
C Publications
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
203
List of Figures 1.1 1.2 1.3 1.4
UAV intelligent control architecture Levels of autonomy [10]. . . . . . . A mission flight plan. . . . . . . . . Standard flight profile [18]. . . . . .
2.1
Construction of convex region using the intersection of halfplanes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Classification of regions in world space for the purposes of path planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Illustration of different sampling methods showing Voronoi lines: (a) a Sukharev grid, and (b) uniform random sampling. Examples of bug traps (local minima): (a) single bug trap and its effect on a unidirectional search, (b) single bug trap and how a bidirectional search can overcome it, and (c) double bug trap and multi-directional search. . . . . . . . . . . . . . . . . (a) Potential field contour plot and, (b) skeleton [37]. . . . . . Rapidly exploring Dense Tree (RDT) graph [12]. . . . . . . . . Difference between (a) a grid/raster map, (b) a probabilistic roadmap, and (c) cell decomposition map. Note the reduced number of sample nodes in a roadmap and cell decomposition. Example roadmap derived from learning phase in Probabilistic RoadMap (PRM) planning, showing a set of two disjoint, connected components. Note that in going from A to B, even though a straight line path exists, the structure of the graph results in a longer, suboptimal path containing a turn. . . . . A* algorithm. s0 is the start node, sg is a goal node and Γ is the successor or neighbourhood operator. . . . . . . . . . . . .
2.2 2.3 2.4
2.5 2.6 2.7
2.8
2.9
xii
[8]. . . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
3 4 6 7
20 22 24
27 28 30
31
32 39
LIST OF FIGURES
xiii
2.10 Computational complexity classes [12]. . . . . . . . . . . . . . 47 3.1
3.2 3.3 3.4 3.5 4.1
4.2 4.3 5.1
5.2
5.3 5.4 5.5 5.6
5.7
Cognitive continuum [97]. Note that RPD refers to a Recognition Primed Decision, and MAUA refers to Multi-Attribute Utility Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . Rasmussen’s [104] three layer model. . . . . . . . . . . . . . . 3T Intelligent Control Architecture [16]. . . . . . . . . . . . . Fuzzy versus crisp set for a given Universe of Discourse (UoD). Singleton fuzzification. . . . . . . . . . . . . . . . . . . . . . .
53 56 57 65 65
Vector neighbourhood based successor operator used by [75], (a) shows the discretisation of track angles and (b) the full successor operator with smooth track angle transitions. Track angles are at 0◦ , 26.6◦ , 45◦ . . . . . . . . . . . . . . . . . . . . . 73 Lattice structure of Yahja et al. [77]. . . . . . . . . . . . . . . 73 MFP task in the context of Rasmussen’s model [104]. . . . . . 81 Successor operator illustration, (a) a single 2D trajectory segment and the corresponding cell sequence, and (b) an example 2D successor operator showing individual trajectory segments. Note that the trajectory segment terminates in the centre of the successor cell, which is not necessarily adjacent to the source cell. . . . . . . . . . . . . . . . . . . . . . . . . . . . . MSA* Pseudocode. Note that at line 12, Succ (Γs (s)) extracts the set of successor nodes s0 ∈ S 0 where s0 is the last cell in each cell sequence γs0 ∈ Γs (s). . . . . . . . . . . . . . . . . . General lattice structure, (x, y, z) dimensions shown . . . . . Top (x − y) view of lattice structure showing trajectory segments for (a) lattice position (0, 0), (b) (0, 2). . . . . . . . . Constrained vertical angle lattice structure. . . . . . . . . . Multi-resolution lattice with Λx = Λy = 3 on the left and Λx = Λy = 6 on the right. Selected trajectory segments are shown for the sake of clarity. . . . . . . . . . . . . . . . . . . Boskovic’s UAV intelligent control architecture [38]. . . . . .
. 86
. 90 . 95 . 95 . 96
. 98 . 100
xiv
LIST OF FIGURES 5.8
Planner flow diagram. Green represents input data, red output data, yellow represents data and states associated with the search itself, and blue represents data and states associated with the monitoring process. . . . . . . . . . . . . . . . 5.9 Outer feedback control loop data flow diagram. Green represents input data, red output data, data processes and data coloured in yellow relate to the deliberative layer, orange for the sequencing layer and blue the control layer in a three tiered intelligent control architecture [16]. . . . . . . . . . . . . . . 5.10 Inner aircraft feedback control loop data flow diagram. Again, green represents input data, red output data, yellow represents aircraft systems and blue represents real world truths. . . . . 5.11 Illustration of start node s0 prediction using A* and goal set defined by e. Consider the scenario where given the existing trajectory segment (si−1 , si ), si is now unreachable due to obstacles (shaded ovals). The current position is s. In the first iteration (a), e is too small and no candidate s0 is found. At the next iteration (b), e is increased and both s0 and a local trajectory from s to s0 is found. . . . . . . . . . . . . . . . . 6.1 6.2 6.3 6.4 6.5 6.6
6.7
6.8
. 102
. 105
. 105
. 107
Plot of the horizontal component of the density field β for different values of R and σr . . . . . . . . . . . . . . . . . . . . 112 Illustration of sampling grid for density field β showing horizontal Euclidean distance r for cells with dimensions δx = δy. 113 Cruising levels rule CAR173 [7] obtained from [148]. . . . . . . 115 Airspace classes example. . . . . . . . . . . . . . . . . . . . . . 115 Fuel value function. . . . . . . . . . . . . . . . . . . . . . . . . 119 Judgment matrix and weights for decision criteria. Note that neighbtlvl refers to total flight time and clvls the cruising levels rule. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Chosen Γ0 successor operator, showing source cell (centre) and successor cells joined by trajectory segments. Λx = 3, Λy = 3, Λz = −∞, Ntl = {1, 2, 3, 4}min. . . . . . . . . . . . . . . . . 120 Example wind map. Note the increase in wind magnitude with altitude and slight variation in direction simulating wind shear. 123
LIST OF FIGURES 6.9
6.10 6.11 6.12 6.13 6.14
6.15 6.16 6.17
6.18 6.19
Example planning scenario showing no fly zones and other aircraft at t = 120s. Note for aircraft and weather, the inner cylinder represents the separation zone/storm cell extents (around the expected position) and the outer cylinder is the 2σ uncertainty boundary (which grows with time). Note also that a red X marks the goal position. . . . . . . . . . . . . . Example planning scenario showing a close up of no fly zones and other aircraft at t = 320s. Compare with Fig. 6.9. . . . Example planning scenario risk (represented by normalised population density) map. . . . . . . . . . . . . . . . . . . . . Example planning scenario wind map xy slice plane at z = 3500ft. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Normalised frequency histogram of computation time ratio between Vector A*, MSA*1 and MSA*2. . . . . . . . . . . . . Statistical box plot of computation time for inflated heuristic variants of the three test algorithms. Note that the complete interquartile range lies within one second and the red crosses denote ‘outliers’. . . . . . . . . . . . . . . . . . . . . . . . . Cumulative histogram of relative path cost between MSA*1, MSA*2 and Vector A*. . . . . . . . . . . . . . . . . . . . . . Cumulative histogram of relative path cost between inflated heuristic MSA*1, MSA*2 and admissible Vector A*. . . . . . Planning scenario where there is a turn angle of greater than 90◦ at the last trajectory segment. Note that the allowable track angles at 3500 and 5500ft are 0◦ to 179◦ , at 4500ft are 180◦ to 359◦ . Because the goal node is on a general NorthEasterly heading from the start node, a turn of greater than 90◦ occurs in ensuing the track angle of the final track lies in the acceptable range. . . . . . . . . . . . . . . . . . . . . . . Planning scenario where Vector A* returns a straight line path showing no fly zones and other aircraft at t = 1250s. . . . . . Planning scenario where Vector A* returns a straight line path showing (a) Normalised Population Density (NPD) and (b) wind in the xy plane at 5500ft. . . . . . . . . . . . . . . . .
xv
. 124 . 125 . 125 . 126 . 127
. 128 . 129 . 130
. 131 . 131
. 132
xvi
LIST OF FIGURES 6.20 Aircraft and no fly zones plot for planning scenario with the greatest cost (for Vector A*, MSA*1 and MSA*2) out of all simulation scenarios. Note the need to fly ‘backwards’ initially due to terrain and no fly zone restrictions. . . . . . . . . . . . 133 6.21 Plot of risk for scenario with greatest path cost. . . . . . . . . 134 6.22 Wind at (a) 5000ft and at (b) 11500ft for planning scenario with greatest path cost—note the slight differences in direction and greater magnitude at high altitudes. . . . . . . . . . . . . 134 6.23 Risk plot of planning scenario with greatest difference in cost between MSA*2 and Vector A*. . . . . . . . . . . . . . . . . . 135 6.24 Plot of other aircraft, storm cells and no fly zones for a computationally intensive planning scenario (one that is in the 99th percentile of all simulations in terms of the number of search iterations). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 6.25 Risk plot for scenario depicted in Fig. 6.24. . . . . . . . . . . 136 6.26 Replanning scenario showing initial path at time 17.5min. . . 137 6.27 Replanning scenario showing a changed environment (new aircraft information) and inflated heuristic solution at t = 18min. 138 6.28 Replanning scenario showing the optimal path for the changed environment, which is enacted at t = 21min. Note how the chosen path descends below the other aircraft which is climbing.138 6.29 Population risk for replanning scenario. . . . . . . . . . . . . . 139 6.30 Wind plot at 3500ft. . . . . . . . . . . . . . . . . . . . . . . . 139 7.1 7.2 7.3 7.4
7.5 7.6 7.7 7.8
Single bug trap case. . . . . . . . . . . . . . . . . . . . . . . Double bug trap case. . . . . . . . . . . . . . . . . . . . . . . Mountain wind simulation. The solution is found in 5.14s. . Number of nodes in the lattice for different values of Λ. Note that Λz = −∞ corresponds to the constrained vertical track angle case described in Section 5.2.1. . . . . . . . . . . . . . Computation time ratio of MSA*3 to Vector A* and MSA*1. Cost ratio of MSA*3 to Vector A*. . . . . . . . . . . . . . . Planning scenario where Vector A*, MSA*1 and MSA*3 return a straight line path. . . . . . . . . . . . . . . . . . . . . Planning scenario with greatest cost where MSA*3 returns the lowest cost path. . . . . . . . . . . . . . . . . . . . . . . . .
. 142 . 142 . 143
. 146 . 149 . 150 . 151 . 151
LIST OF FIGURES Risk plot of planning scenario with greatest difference in cost between MSA*3 and Vector A*. . . . . . . . . . . . . . . . . 7.10 Plot of path found using MSA*3 showing other aircraft, storm cells and no fly zones for a computationally intensive planning scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.11 Histogram of trajectory segment cost across all MSA*1 simulations. Note the rarity of segments with a cost of more than 0.2 - out of 13968 segments, 13 exceed a cost of 0.2. . . . . . 7.12 Illustration of a cell in the cell sequence for a given trajectory segment AB. . . . . . . . . . . . . . . . . . . . . . . . . . .
xvii
7.9
A.1 Aircraft separation risk value function. . . . . . . . . . . . . A.2 Cruising levels rule value function. . . . . . . . . . . . . . . A.3 Value function for the time criterion. Note the use of cumulative time levels to ensure additive consistency. . . . . . . . . A.4 Normalised population risk value function. . . . . . . . . . . A.5 Storm cell risk value function. . . . . . . . . . . . . . . . . .
. 152
. 153
. 157 . 158 . 185 . 186 . 186 . 187 . 187
List of Tables 2.1
Suitability of different Probabilistic RoadMap (PRM) planners to different classes of search space. Note that an open region refers to one that is predominantly free space [35]. . . . 35
3.1
Comparison of intuitive and analytic decision making. . . . . . 54
4.1
Chosen design approach for UAV MFP. . . . . . . . . . . . . . 83
6.1
Computation time and loop count for Vector A*, MSA*1 and MSA*2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ratio of computation time between MSA*1, MSA*2 and Vector A*. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computation time and loop count for Vector A*, MSA*1 and MSA*2 with an inflated heuristic. . . . . . . . . . . . . . . . Path cost ratio between MSA*1, MSA*2 and Vector A*. . . Path cost ratio between inflated heuristic MSA* and admissible Vector A*. . . . . . . . . . . . . . . . . . . . . . . . . . . Turn angle statistics for Vector A*, MSA*1 and MSA*2. . .
6.2 6.3 6.4 6.5 6.6 7.1 7.2 7.3
Computation time for special case test scenarios. . . Frequency of different track traversal times ntl . . . . Horizontal track velocity in m/s for different track times ntl . . . . . . . . . . . . . . . . . . . . . . . .
xviii
. 126 . 127 . 128 . 129 . 130 . 131
. . . . . . 142 . . . . . . 148 traversal . . . . . . 148
List of Abbreviations AD* ADP ADS-B AGL AMSL ARA* ARCAA AU AUD AUV
Anytime Dynamic A*. Approximate Dynamic Programming. Automatic Dependent Surveillance Broadcast. Above Ground Level. Above Mean Sea Level. Anytime Replanning A*. Australian Research Centre for Aerospace Automation. Addition of Utilities. Addition of Utility Differences. Autonomous Underwater Vehicle.
C-space CAR CASSY CoG CON
Configuration space. Civil Aviation Regulation. Cockpit ASsistant SYstem. Centre of Gravity. CONjunctive.
DIS dof DOM DSS
DISjunctive. degrees of freedom. DOMinance. Decision Support System.
EBA ELOS
Elimination By Aspects. Equivalent Level of Safety.
FIFO
First In First Out. xix
xx
LIST OF ABBREVIATIONS FILO FPA
First In Last Out. Flight Path Angle.
GIS GPS GRIB GTA
Geographic Information System. Global Positioning System. GRIdded Binary. Ground Track Angle.
HAZMAT
HAZardous MATerials.
IAS IDA* IFR ISE
Indicated AirSpeed. Incremental Deepening A*. Instrument Flight Rules. Incremental Search Engine.
LADAR LEX LIDAR LUT
LAser Detection And Ranging. LEXicographic. LIght Detection And Ranging. Look-Up Table.
MACBETH MAUT MBH MCDM MDP MF MFP MISO MNA MSA*
Measuring Attractiveness by Categorical Based Evaluation Technique. Multi-Attribute Utility Theory. Moving Basis Heuristic. Multi-Criteria Decision Making. Markov Decision Process. Membership Function. Mission Flight Planning. Multiple Input Single Output. Maximising Number of Attributes. Multi-Step A*.
NAS NDM NPD
National Airspace System. Naturalistic Decision Making. Normalised Population Density.
LIST OF ABBREVIATIONS
xxi
NURBS
Non-Uniform Rational B-Splines.
PDF POMDP
Probability Density Function. Partially Observable Markov Decision Process. Probabilistic RoadMap.
PRM RAWFS RBFS RDT RPDM RPV RRT RSR RVSM
Reduction, Assumption, Weighing, stalling, Suppressing. Recursive Best First Search. Rapidly exploring Dense Tree. Recognition Primed Decision Making. Remotely Piloted Vehicle. Rapidly exploring Random Tree. Route Surveillance Radar. Reduced Vertical Separation Minima.
SDS SRTM
Search for a Dominance Structure. Shuttle Radar Topography Mission.
UAV UoD
Unmanned Aerial Vehicle. Universe of Discourse.
VFR
Visual Flight Rules.
Fore-
Chapter 1 Introduction There is no heavier burden than a great potential Charles Schulz
I
N recent times, Unmanned Aerial Vehicles (UAVs) have emerged as a viable platform in applications such as surveillance, communications, environmental and disaster monitoring, agriculture and defence. UAVs provide an unparalleled degree of flexibility in mobility and response time. This contributes towards lower mission costs (compared to manned aircraft and satellites) and enables acquisition of information (or delivery of a package) in a time frame not previously possible [1]. It is expected that by 2013, the worldwide UAV market will be worth $USD10.6 billion [2]. An integral part of UAV operation is the design of a flight path (or mission plan) that achieves the objectives of the mission [2]. Mission Flight Planning (MFP) ensures that the UAV operates in a safe and efficient manner. For civilian applications, the flight path must additionally conform with the rules of the air. Currently, UAVs are used predominantly in military applications [3]. However, there are a large number of potential civil applications including atmospheric observation, transportation and search and rescue [1, 2, 4]. One application that is common to both the civilian and military domains is a medical package delivery mission. In this mission, a small, low cost and field deployable UAV, such as the RQ-7A Shadow, is used to deliver the package to a remote location [3]. Low cost UAVs, which are generally smaller in size, are especially suited to civil applications due to the sensitivity to cost in the civil environment [2]. 1
2
1.1
CHAPTER 1. INTRODUCTION
Motivation and Significance
One of the key impediments to widespread adoption of UAVs for civil applications is the inability to gain access to the National Airspace System (NAS) [2, 5]. The successful integration of UAVs within the NAS is contingent on the satisfaction of three major requirements [6]. Firstly, the UAV must demonstrate an Equivalent Level of Safety (ELOS) to that of human piloted aircraft. In the absence of adequate see and avoid technology (a requirement for the unsegregated operation of UAVs alongside other aircraft, Civil Aviation Regulation (CAR) 163A [7]), strategic mission flight planning plays an important role in the risk management, and subsequent approval, of UAV operations in the NAS. As well, the UAV must operate under the existing rules and regulations of aviation and must appear as a ‘transparent’ user of the NAS. Automation of MFP onboard the UAV can help address a number of the requirements for integration of UAVs within the NAS. As UAVs operate in an unpredictable and dynamic outdoor environment, it is not possible to guarantee satisfaction of the three integration criteria outlined above with a static flight plan. Instead, it is necessary to combine offline (pre-flight), strategic planning with online (in-flight) tactical replanning. It is advantageous to perform online replanning onboard the UAV as this can ensure continued conformance with the NAS operating requirements in the event of an outage in the communications link. There is significant time pressure on tactical replanning due to aircraft velocities and the additional constraint, for fixed wing UAVs, of constant motion (maintaining a minimum stall speed). In order to meet the requirements of NAS integration, the MFP system needs to find motion plans that conform with the ‘rules of the air’, avoid collision scenarios, and maximise mission efficiency and goal attainment. Furthermore, the plans can not exceed the performance limitations of the aircraft. There are complex trade-offs that must be made with regard to mission goals, mission efficiency and safety objectives, and the ‘rules of the air’. MFP is a deliberative task that sits in the topmost layer in an intelligent control architecture as shown in Fig. 1.1 (for more details, refer Section 3.2.2). The flight plan generated by an MFP system is enacted through a manoeuvre generator by the aircraft control systems. This MFP system takes as
1.1. MOTIVATION AND SIGNIFICANCE
3
Figure 1.1: UAV intelligent control architecture [8].
input, a model of the operating environment (obtained from sensor systems), and the goals (or target waypoints) and planning objectives (provided by the human operator) [2]. Currently, the majority of UAVs are operated as a Remotely Piloted Vehicle (RPV) [2]. In this mode, the human operator performs the MFP task and interfaces directly to the manoeuvre generator in Fig. 1.1. RPV flight requires constant human supervision and is very fatiguing to the human operator [9]. Automation of the MFP process reduces the operator’s cognitive workload and enables the operator to focus on the mission objectives. This potentially enables a scenario where one operator controls multiple UAVs or where a mission is conducted by a ‘mission specialist’ without a need for a human pilot [2]. Thus, a high degree of autonomy can aid in reducing not just the operator’s workload, but also reduce operational costs, and reduce the dependence on a constant communications link [1, 2]. Consider the definition of different degrees (or levels) of autonomy as proposed by Parasuraman, Sheridan et al. [10] (refer Fig. 1.2). Level one
4
CHAPTER 1. INTRODUCTION 10
Computer ignores the human
9
Computer reports only if it wants to
8
Computer only reports if asked
7
Computer executes, then reports to human
6
Human can veto computer’s decision within time frame
5
Computer executes suggestion with approval
4
Computer suggests one alternative
3
Computer chooses a set of alternatives
2
Computer computes complete set of alternatives
1
Human makes all decisions Figure 1.2: Levels of autonomy [10].
corresponds to a RPV operating scenario where the operator interfaces directly with aircraft control systems and sensor systems. The intermediary levels from two to five describe different operating scenarios for a Decision Support System (DSS) where the computer derives candidate solutions based on available situational awareness information. For the scenario where there is one operator commanding multiple UAVs, a level of autonomy from six to eight would be desired due to the cognitive workload involved [1, 2]. Finally, in the event of an outage in the communications link, the UAV operates at the ninth or tenth level of autonomy in Fig. 1.2. This corresponds to fully autonomous flight. It can be seen that an automated MFP system can enable a high level of autonomy for a variety of operating scenarios. Fully autonomous operation (a level of autonomy of nine or ten) requires the MFP system to be situated onboard the UAV. For other operational scenarios, the MFP system can also be situated at the ground station or third party locations. The significance of automated MFP is that it plays a crucial role in satisfying the requirements for operating UAVs in the NAS for a number of different operating scenarios (with differing levels of autonomy). In addition, automated MFP helps to reduce the human operator’s workload and thus aids in reducing operational costs. This enables operating scenarios such as those involving one human operator controlling multiple UAVs or the scenario where the operator is a mission specialist (i.e. non-pilot).
1.2. MISSION FLIGHT PLANNING TASK
1.2
5
Mission Flight Planning Task
UAVs are characterised by: (i) operation in large, outdoor environments, (ii) movement in three dimensions (x, y, z), (iii) uncertain and dynamic operating environment, (iv) presence of environmental forces that affect motion (winds or currents), and (v) differential constraints on movement [1]. Because of (ii) and (iii), the planning space must be four dimensional (three spatial and one time dimension). Note that a dynamic environment refers not only to moving obstacles, but also to changing weather conditions. The proposed work mitigates the uncertainty inherent in a dynamic environment through online replanning and incorporation of tolerances in the planning process. The motion plan is constrained by vehicle dynamics (such as maximum climb/ascent rate), environmental constraints (e.g. static and dynamic obstacles and wind) and the rules of the air. In addition, the planned path must satisfy multiple, possibly conflicting objectives such as fuel efficiency and flight time. Due to the ‘curse of dimensionality’ [11], it is not computationally feasible to plan in a high dimensional search space consisting of all the aforementioned decision variables. It is common, instead, to plan the path in the world space (x, y, z, t) [12] by aggregating the decision variables into a single, non-binary cost term [11]. This planning problem is a type of weighted region path planning [13]. One of the unique characteristics of UAV operation listed above is wind. The wind velocity vector constrains vehicle movements and affects travel time and fuel consumption. In the presence of wind, it is especially important to have high track angle resolution as low track angle resolution can result in suboptimal paths that contain spurious turns [14, 15]. This is a shortcoming of conventional search grids as the track angle is limited to increments of 45◦ . Note that 4D motion planning as described here should not be confused with trajectory planning, which finds a path expressed in terms of the degrees of freedom of the vehicle and velocity/angle rates [12]. Instead, a 4D motion plan comprises a geo-referenced sequence of 3D waypoints and the desired track velocities between them. This is illustrated in Fig. 1.3. In this work, such tracks are also equivalently referred to as trajectory segments. Typically, this form of planning resides in the topmost deliberative layer in conventional intelligent control architectures [16]. However, it is still necessary to incorporate an approximation of vehicle dynamics (which feature
6
CHAPTER 1. INTRODUCTION
Figure 1.3: A mission flight plan.
in the lower layers of an intelligent control architecture) to ensure that the generated paths are physically realisable. McManus [4] outlines the necessary characteristics for an MFP algorithm. He argues that a global planner is needed as local planners may get stuck in local minima, may fail to reach the goal and may be inefficient (producing paths that are locally optimal instead of globally optimal) [4]. However, McManus asserts that local planners are useful for other systems such as emergency collision avoidance. Scherer, Singh et al. [17] concur on a global planning approach and have successfully implemented this approach in real life helicopter experiments. Even though McManus [4] does not implement a method for multi-objective planning, he recognises this need by stressing the importance of considering flight time, distance, fuel efficiency and conformance with aviation regulations. The MFP task is non-trivial due to the need to optimise for multiple decision objectives such as safety objectives, the rules of the air and mission objectives. It is important to differentiate between an objective and a constraint. For example, the safety objective might be evaluated according to a midair collision risk criterion and a risk presented to third parties criterion. The degree of satisfaction of the safety objective is obtained by aggregating (but not necessarily with equal weighting) the constituent criteria. On the other hand, a constraint refers to limits imposed on individual decision criteria (or decision variables) such as the maximum allowable risk. Conformance with the rules of the air is a requirement for operation of UAVs in the NAS. There are two sets of regulations governing flight, namely
1.2. MISSION FLIGHT PLANNING TASK
7
Figure 1.4: Standard flight profile [18].
Visual Flight Rules (VFR) and Instrument Flight Rules (IFR) [7]. Many civil UAV applications and general aviation flights (e.g. crop dusting) are operated under VFR [4]. Typically, MFP is concerned with finding a path for the en-route component (between the departure and descent point) of the standard flight profile as shown in Fig. 1.4. Note that the destination point may not necessarily be a place for landing; for some applications, the mission tasks (e.g. spray crops, perform surveillance) are conducted at the destination point. There is more scope for flight planning under VFR as IFR flight is mostly constrained to pre-established flight routes. As a result, it is assumed throughout this thesis that the UAV operates under VFR. An important consideration in the design of a suitable MFP algorithm and framework is the type and format of input variables used for planning. Typically, the external environment is modeled using a discrete, grid based format. Geographic data obtained from a Geographic Information System (GIS), for instance, is often structured as a raster map [19]. Terrain elevation data, such as that obtained from the Shuttle Radar Topography Mission (SRTM), is encoded as a 2D grid [20]. Wind information is similarly encoded in a grid format known as GRIdded Binarys (GRIBs) [21]. Sensor based information such as LAser Detection And Ranging (LADAR) and LIght Detection And Ranging (LIDAR) is similarly transcoded into a discrete, probabilistic evidence grid (like in [17]). Another important consideration in MFP is online (or in-flight) replanning. A plan that is optimal when it is generated can become invalidated or suboptimal by changes to assumptions in the flight plan [22–25]. For example, unanticipated wind conditions can increase fuel consumption, it may take an unexpectedly long time to reach a waypoint, and there may be changes to mission goals as new information becomes available [23, 24].
8
CHAPTER 1. INTRODUCTION
However, time delays and limitations in communications bandwidth or communications outages can limit opportunities for operator intervention [23]. The ability to modify a flight plan onboard the UAV is therefore important to help provide ELOS and continue the mission. Finally, it is important to ensure that the MFP algorithm is deterministic in terms of computation time and solution cost. For example, heuristic search algorithms have a theoretically proven upper bound on computational (time and memory) complexity [12, 26]. This is essential for addressing the need for real time computation stated earlier. In addition, given identical planning scenarios, the planner should produce identical flight plans. Determinism is essential in obtaining certification under existing aviation software certification standards such as DO-178B [27]. Of course, as DO178b is quite dated, it is possible that determinism requirements will be relaxed in the future. Therefore, it is not imperative to have determinism, however, it is highly desirable. It can be surmised that an MFP system must be able to: – evaluate multiple objectives – handle uncertainty – be computationally efficient Additionally, it is desirable for the planner to be deterministic. These four criteria are used in the literature review to select appropriate methods and approaches to implementing an MFP motion planning algorithm.
1.3
Research Questions
The MFP problem is concerned with finding a path (or motion plan) that links a specified initial state and goal state. These states are four dimensional (three spatial and one time dimension) due to the presence of dynamic elements (e.g. other aircraft, weather). It is assumed that the mission parameters (i.e. start and goal states) are provided by either the human operator or a higher level automated scheduling system. There are two forms of MFP both of which are currently performed (with proficiency) by the human operator [28]. Strategic planning, which occurs before take-off, takes a priori information about the operating environment and the mission goals and constructs a path that optimises for the given
1.3. RESEARCH QUESTIONS
9
decision objectives. These include the ‘rules of the air’, mission efficiency objectives and safety objectives. On the other hand, tactical planning involves re-evaluation and re-generation of a flight plan during flight based on updated information about the goals and operating environment. The generated plan should be as close as possible to the optimal plan given available planning information. This leads to the first research question: 1.
Can a mission flight plan be automatically generated that optimises for multiple mission objectives for predefined mission goals?
Due to the sheer breadth of potential UAV applications, it is not possible to devise a universal set of decision criteria for all applications. Instead, the research develops a framework (defined as a collection of tools) for MFP. This framework comprises a multi-objective motion planner that can be adapted to different decision criteria, different aircraft dynamics and different sized world space. An accompanying research question relates to quantitative measures for the verification and validation of such an MFP framework. Candidate measures of performance include the optimality of the solution path and computation time. Finding an optimal path is difficult because trade-offs have to be made when there are conflicting objectives. In the field of motion planning, an optimal path is typically defined as the least cost path [12]. In the presence of multiple decision criteria, a new definition of optimality is used, typically based on the least aggregated cost path and constraints imposed by different objectives [11]. Verification of planner optimality and planner predictability is important for certification under existing aviation regulations. This leads to the next research question: 2.
How can optimality be defined for mission flight planning and what methods for verification are applicable to this problem?
MFP is a complex task that requires the evaluation of multiple, possibly conflicting objectives. Furthermore, there is uncertainty in the decision variables used for planning as UAVs operate in an outdoor environment. The complex issues surrounding the evaluation of multiple decision criteria, which
10
CHAPTER 1. INTRODUCTION
are used to assess the plan with respect to mission objectives, leads to the next research question: 3.
How can different and possibly conflicting decision objectives be satisfied when constructing a mission flight plan with uncertain information?
Finally, in designing an MFP system, it is necessary to consider how such a system would function within an intelligent control architecture onboard the UAV. The MFP system needs to interface with higher level deliberative systems and the human operator, as well as lower level control systems. The operation of the planner, and how it integrates with other systems for offline and online planning, is encapsulated in the final research question: 4.
How can an automated mission flight planner be integrated into the UAV and what are the implications on the UAV from an operational perspective?
It is beyond the scope of the research to validate the effectiveness of an integrated intelligent control architecture. Instead, the research is concerned with the development of a framework (or methodology) that addresses the requirements of UAV MFP. An important part of the framework is the motion planning algorithm.
1.3.1
Research Objectives
In light of the research questions posed in the previous section, the research objectives for this project can be defined as: 1.
The primary objective of this research is to develop a framework that enables UAV developers and operators to create an automated mission flight planning system capable of dealing with multiple objectives with uncertain information in a computationally efficient manner.
2.
The secondary objective is to apply this research to case study scenarios to evaluate the performance of such a framework.
1.4. RESEARCH METHODOLOGY
1.3.2
11
Research Outcomes and Application
The outcomes for the research stem directly from the research objectives above. Given the primary research objective, the major outcome of the research is: 1.
A framework (or methodology) for creating mission flight planning systems.
A major component in the verification of the proposed framework is the validation, both analytically and through simulation, of the proposed motion planning algorithm. This requires the development of suitable case study test scenarios. Such a scenario also serves as a reference implementation for the MFP framework for future UAV operators and developers: 2.
The research also develops an example implementation of the mission flight planning framework for a case study UAV scenario. This serves to demonstrate the applicability of the framework to UAV missions
Finally, it is important to disseminate and share the research outcomes: 3.
1.4
The research also produces a variety of publications in the form of conference proceedings and journal articles that serve as a stimulus for valuable discussion with industry and academia.
Research Methodology
This section presents a methodology for delivering research outcomes and meeting research objectives. The overall research can be divided into three major phases, namely, review of existing work, algorithm development, and verification and validation. The first phase comprises a survey of relevant literature in the fields of motion planning (including UAV and vehicle motion planning) and human cognition (since MFP is a task performed proficiently by human experts). The outcomes of this research are presented in chapters 2, 3 and 4.1 and summarised in Section 4.2. These findings provide a guideline for the development of the MFP framework and associated motion planning algorithm. Using the design approach summarised in Table 4.1, a framework for UAV MFP is presented in Section 5. This framework, the primary outcome
12
CHAPTER 1. INTRODUCTION
of the design phase, comprises a motion planning algorithm and the interface definition between that algorithm and decision objectives, mission goals, and UAV control systems. The final phase of the research is to verify and validate the proposed framework. Firstly, it is necessary to verify that the proposed framework meets the requirements of MFP as described in the research objectives (Section 1.3). Recall that the primary research objective stipulates an algorithm that incorporates (i) multiple decision objective, (ii) uncertainty and (iii) computational efficiency. Furthermore, the algorithm should be deterministic. Determinism can be easily be verified based on the chosen approach. However, these requirements are quite general. In order to validate the proposed framework it is necessary to use specific and quantifiable performance measures. Validation of the framework against research objective (i) takes on two aspects. Firstly, as expert pilots tend to find a satisficing path that meets multiple decision constraints (refer Section 3), one of the subobjectives of satisfying (i) is to enforce path constraints (e.g. maximum risk, maximum velocity). This can be verified through inspection of the planning algorithm. Secondly, the optimality of the path can be quantified based on the total path cost. This can be validated through theoretical proof (i.e. show that the algorithm finds the least cost path). The optimality can also be evaluated through Monte Carlo simulations. One method for evaluation is to compare the total path cost of the proposed algorithm with a benchmark algorithm. This can help determine the effect of different sampling structures where both algorithms use the same cost function and find the least cost path. For example, it is instructive to compare a multi-resolution search with a uniform resolution search to determine the speed-up and effect on path cost (like in [14]). Similarly, a test algorithm with fewer sample points can be benchmarked against a high resolution Sukharev grid. It is straightforward to verify that the framework incorporates uncertainty (ii). Due to the computational complexity of planning with uncertainty (refer Section 2.4), it is common to adopt an approach where uncertainty is mitigated through heuristics (refer Section 3.4), path tolerance and online replanning. In such an approach, one way to quantitively validate (ii) is to
1.5. RESEARCH CONTRIBUTIONS
13
measure the degree of tolerance of the planned path. This can be represented with a distance tolerance (e.g. tolerance of 100m). The total planning time is another metric that can be used to validate the feasibility of online replanning for mitigating uncertainty (ii) given the computational efficiency requirement (iii). This can be done through Monte Carlo simulations like those used to validate (i). Again, benchmarks can be used to determine the speed-up of the proposed algorithm over the benchmark. One of the major challenges in running Monte Carlo simulations is the infinity of possible planning scenarios in a natural environment [29]. McManus [4] adopted a simulation approach where a small number of case studies based on real world scenarios were simulated. It is difficult to capture the expected performance of the algorithm with such a small sample size. Instead, an approach based on a large number of randomly generated simulation scenarios (i.e. Monte Carlo simulations) using real world parameters is adopted. This approach is often used in motion planning [14, 15, 30, 31]. In addition, a small number of special case scenarios (such as local minima) can also be simulated as done in [32]. By using a combination of theoretical analysis and practical simulations, it can be shown that the proposed algorithm meets the objectives of the research.
1.5
Research Contributions
This section summarises the major contributions of the research. These stem from the first research outcome, and are demonstrated and published as per the second and third research outcomes respectively. The primary contribution of this research is the Multi-Step A* (MSA*) search algorithm and associated multi-resolution lattice search methodology. This 4 Dimensional (4D), multi-objective motion planning algorithm can incorporate multiple decision objectives such as safety, regulatory (rules of the air) and mission efficiency objectives. MSA* uses a novel variable successor operator to provide variable track velocity and track angles with arbitrarily selectable resolution (refer Section 5.1). These parameters are key to path optimisation under non-negligible wind conditions. Existing vehicle motion
14
CHAPTER 1. INTRODUCTION
planning methods do not simultaneously consider variable track velocities and angles. In addition, a variable successor operator enables the imposition of a lattice sampling structure which can be used to reduce computation time (refer Section 5.2). MSA* is able to find paths of equivalent cost (mean cost difference of 0.5%) to conventional, vector neighbourhood based A* (Vector A*) but in a third of the computation time (on average). Note that existing Vector A* had to be extended as it does not meet the requirements of MFP (for example, it does not consider multiple planning objectives). The proposed algorithm is shown to be computationally efficient (equivalent complexity to A*) and suitable for online replanning. This is the primary contribution. The proposed variable successor operator is mapped to a cell sequence in 4D space. Thus, the planned path comprises a sequence of 4D waypoints that lie within a corridor of cells. This cell sequence is shown to provide a quantifiable tolerance to uncertainty and is another major contribution. Another important contribution stemming from this research is the discovery of a need for additive consistency (refer Section 7.4). This property relates to the Multi-Criteria Decision Making (MCDM) cost function, and prevents path bias (which results in suboptimal paths) in the presence of variable length trajectory segments. It is shown that additive consistency is especially important in a multi-resolution search. Finally, the research proposes a novel feedback based architecture for online replanning that integrates MSA* with other UAV systems (refer Section 5.3). Expert pilot decision making strategies are used to help reduce the complexity and increase the cognitive compatibility of the MFP system.
1.6
Publications
Publications stemming from this research are listed in chronological order below: P. Narayan, P. Wu, D. Campbell, and R. Walker, “An intelligent control architecture for unmanned aerial systems (UAS) in the national airspace system (NAS),” in 22nd International Unmanned Air Vehicle Systems Conference, Melbourne, Australia, 2007.
1.7. FORMAT OF THESIS
15
P. Wu, R. Clothier, D. Campbell, and R. Walker, “Fuzzy multi-objective mission flight planning in unmanned aerial systems,” in IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making, Honolulu, Hawaii, 2007. P. Narayan, P. Wu, and D. Campbell, “Unmanning UAVs - addressing challenges in on-board planning and decision making,” in Humans Operating Unmanned Systems, Brest, France, 3-4 September 2008. P. Wu, D. Campbell, and T. Merz, “On-board multi-objective mission planning for unmanned aerial vehicles,” Big Sky, Montana, 7-14 March 2009. P. P.-Y. Wu, D. Campbell, and T. Merz, “Multi-objective 4D vehicle motion planning in large dynamic environments,” Submitted1 . IEEE Trans. Systems, Man and Cybernetics B.
1.7
Format of Thesis
This thesis presents a framework for MFP using Multi-Step A* (MSA*), a method for 4D vehicle motion planning based on variable length, angle and velocity trajectory segments. A number of fundamental motion planning methods are reviewed in Chapter 2 – Review of Motion Planning Theory given the requirements described in Chapter 1 – Introduction. As MFP is a task that is performed proficiently by expert pilots, a review of human decision strategies is presented in Chapter 3 – Review of Expert Pilot Decision Making to inform the design of an MFP framework. Based on the outcomes of the literature review and existing methods for UAV and vehicle motion planning, a methodology for designing and evaluating an MFP framework is presented in Chapter 4 – Mission Flight Planning Design Approach. This is followed by the presentation and theoretical evaluation of the proposed motion planning algorithm (MSA*) and the proposed feedback based replanning architecture (Chapter 5 – Mission Flight Planning Framework). An example medical package delivery application is discussed in Chapter 6 – UAV Flight Planning Application along with simulation results. Analysis of the research outcomes can be found in 1
To be re-submitted as per editor instructions - original paper was too long.
16
CHAPTER 1. INTRODUCTION
Chapter 7 – Analysis and Extension. The conclusions of the thesis are given in Chapter 8 – Conclusion and followed by a number of appendices. These appendices contain the value functions used in simulation, data tables for example planning scenarios, and a reproduction of selected publications.
Chapter 2 Review of Motion Planning Theory All truths are easy to understand once they are discovered; the point is to discover them Galileo Galilei
T
HIS chapter presents an overview of fundamental theories in the field of motion planning. These core concepts form the basis of existing vehicle motion planning algorithms and are used in the design of the proposed Mission Flight Planning (MFP) algorithm. The term motion planning is often used in robotics and encompasses a wide range of methods for planning where the robot operates in a continuous space. Motion planning is a form of path planning, a term used in the fields of robotics, control theory and artificial intelligence (AI). Path planning is defined as “finding a sequence of actions that transforms some initial state into some desired goal state. In path planning, the states are agent locations and transitions between states represent actions the agent can take, each of which has an associated cost” [33]. An Unmanned Aerial Vehicle (UAV) for example operates in a three dimensional Euclidean space. This Euclidean space is also referred to as a ‘workspace’ (in the field of manipulator planning) or a ‘world space’ (for mobile robot planning) [12, 34]. An optimal path is defined as one that incurs the least cost across all possible paths. This contrasts with a feasible path, which is a plan that arrives at the goal state, regardless of efficiency [12]. A planning algorithm is said to be complete if it will always find a path in finite time where one exists [12, 33]. However, completeness is sometimes traded for practically 17
18
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
achievable computation time [35]. In these situations, weaker notions of completeness such as resolution completeness and probabilistic completeness may be tolerated (refer Section 2.2). It should be noted that path planning for mobile robots is significantly different from path planning for robot manipulators. There are three major differences [36]. Firstly, mobile robots operate in an uncertain natural environment whereas manipulators operate in a known, controlled environment. Additionally, the mobile robot typically only negotiates a path once compared to a manipulator which repeats the same motions many times. Therefore, there is significant time pressure on the planning process - a suboptimal solution delivered on time is preferable to an optimal solution that is late. Furthermore, there should be a safety margin between the path and obstacles for a mobile robot. This is important for safe navigation under uncertainty and helps to avoid the problem of occlusion of far away objects by nearby obstacles. Hence, methods for manipulator planning are not necessarily suited to mobile robot path planning. Planning algorithms are often classified as either local or global [37]. Global planning is a two step process that first constructs a roadmap (i.e. a graph) that represents the connectivity of the free space and which also contains the start and goal state. A search is then performed on this roadmap to find the least cost path [12]. Example global planning methods include roadmap based techniques and cell decomposition based techniques. Local planning methods, on the other hand, do not create a roadmap - instead a search is performed on the world space directly (typically by putting a search grid over the entire world space). The resultant path is not necessarily of least cost. A common approach is to combine an approximate global planner (which ignores vehicle dynamics, also referred to as differential constraints) with an accurate local planner [14]. Selection of a suitable method for planning should be based on factors such as computational efficiency, response to dynamic changes, robustness to initial assumptions and choice of optimisation criteria [38]. A theoretical measure of computational efficiency is the computational complexity of a planning algorithm. This is presented in Section 2.5. Note that logic based planning techniques [12] are not considered here as path planning in logic space is computationally intractable (NP-hard, refer Section 2.5).
2.1. PLANNING SPACE
2.1
19
Planning Space
In motion planning, it is commonly assumed that the world space can be segregated into two types of regions: free space, and space that is occupied by obstacles. This section evaluates different methods for geometric modeling and the resultant search spaces. A common approach is to approximate all obstacles as convex polygons (or polyhedra in three dimensional space). All non-convex polyhedra are first converted into convex polyhedra as shown in [12]. Consider a set of points X in the world space; X is convex “if and only if, for any pair of points in X, all points along the line segment that connects them are contained in X” [12]. Such a polyhedron can be described with either a boundary or solid representation. In the boundary represented case, polyhedra are defined according to the position of faces, edges and vertices. On the other hand, a solid representation is defined with half planes (or half spaces in 3D) which divide the search space into two subsets. This provides a direct means for collision detection. Consider a plane in three dimensional space which has an equation ax + by + cz + d = 0. Let f be a function such that: f (x, y, z) = ax + by + cz + d (2.1) Note that f (x, y, z) > 0 for all points on one side of the plane and f (x, y, z) < 0 on the other side, hence the term half space. An object (the robot or an obstacle) can be represented as a conjunction of half spaces H: O = H0 ∧ H1 ∧ . . . ∧ Hn
(2.2)
where Hi is defined for each point (x, y, z) in the search space (i.e. world space) W : Hi = { (x, y, z) ∈ W, W | fi (x, y, z) ≤ 0} (2.3) The construction of polyhedrons from half planes is illustrated for the twodimensional case in Fig. 2.1. An advantage of a half space based representation is the ease of extension to non-linear half spaces (i.e. non-linear functions f ). The class of semialgebraic models for example correspond to the case where f is a polynomial in x, y and z. A polyhedral representation is a special case of a semi-algebraic
20
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
Figure 2.1: Construction of convex region using the intersection of half-planes.
model (i.e. a first degree polynomial). Even though a higher order polynomial (or a more complex half space equation f - refer [12]) can provide a better approximation of an object, this must be balanced against the need for increased computational expense. The drawback with a low-order semialgebraic model is the need for many half planes (and hence computational complexity) for objects of non-regular shape. Another approach to geometric representation is the bitmap (or grid). In this approach, the search space is divided into rectangloid cells and a numerical value is associated with each cell. In a binary representation, ‘1’ denotes an obstacle and ‘0’ free space. This can be generalised to all values on the interval (0, 1) which is the case for a ‘grey-scale’ map or occupancy grid [12]. In an occupancy grid, the numerical value represents the probability that an obstacle exists. Alternatively, the numerical value can also represent the difficulty of traversing a cell. This latter interpretation is often used for planners operating on the weighted region problem [13, 14, 30, 33, 36, 39]. As noted in Section 1.2, sensor and map data about the UAV operating environment is typically in this format. Such a representation trades computational efficiency (memory look-up at run time) with memory usage (which is exponential in dimensionality). An alternative method for geometric modeling is 3D triangles. This method is widely used in computer graphics, however, it is difficult to determine whether a point lies inside or outside a 3D triangle [12]. This is a similar shortcoming of Non-Uniform Rational B-Splines (NURBS). Unlike 3D triangles, NURBS are computationally expensive to evaluate, much like generalised cylinders and cones [40]. For instance, the computational complexity of generalised cones in 3D space has a lower bound of O(n3 ) and upper bound O(n4 ) [12, 40]. The previously described methods for geometric representation are used to model the world space W . A common approach in robotics is to transform
2.1. PLANNING SPACE
21
the problem from W into Configuration space (C-space). Each point (referred to as a configuration q) in C-space describes not just the position but also the pose of the robot. Hence, C-space is of the same dimensionality as the number of degrees of freedom as the robot. The purpose of C-space is to provide a level of abstraction that allows for application of generic planning algorithms to planning problems with different geometries and kinematics [12]. However, it is computationally expensive to transform obstacles in W to C-space. Furthermore, C-space is of higher dimensionality than W and an increase in dimensionality results in an exponential increase in the number of search states. [12, 34] Uncertainty is ignored in the previously discussed geometric models. One way for capturing uncertainty is to transform the problem from W into information space. Information space captures uncertainty in the robot’s position by augmenting W with the observation space (defined for all the robot’s sensor actions). This way, there is no uncertainty associated with states in information space which can then be searched using a conventional deterministic planning algorithm. [12] The primary shortcoming of both C-space and information space is the exponential increase in the memory and computation time with dimensionality. Even though this may be tolerable for a small search space, an increase in dimensionality can make planning intractable for mobile robots due to a large outdoor operating environment. As a result, a large number of recent works in mobile robot planning (including UAV planning) plan in the world space [21, 30, 32, 41–47]. Dale and Amato [35] present a nomenclature for classifying different regions of the world space: • • • • •
free cluttered narrow passage blocked passage blocked (region completely filled with obstacles)
These are illustrated in Fig. 2.2, where A and E is free space, B is at an entrance to a narrow passage, C is at the entrance to a blocked passage, D is cluttered and F is inaccessible space. Note that the terms ‘free’, ‘cluttered’ and ‘narrow’ are relative; if the robot is small and manoeuvrable compared
22
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
D
B A
F C
E
Robot Bounding Box Figure 2.2: Classification of regions in world space for the purposes of path planning.
to the environment, then the workspace appears more open. This is often the case with fixed wing aircraft operating in a relatively obstacle-free sky.
2.2
Sampling Based Motion Planning
There are two major classes of planning algorithms in motion planning, sampling based planning and combinatorial planning [12]. Sampling based techniques approximate the continuous space motion planning problem with a discrete representation (typically in the form of a graph). On the other hand, a combinatorial algorithm uses an exact representation of the original problem. These approaches are complete and optimal, but are computationally impractical for many real world applications [12]. This is because the general motion planning problem has been proven to be PSPACE-hard which implies NP-hard (i.e. computationally intractable, refer Section 2.5). As a result, combinatorial methods are ignored in this literature survey. Sampling based motion planning algorithms attain computational tractability by sacrificing strict completeness for a weaker definition such as resolution completeness or probabilistic completeness. The ensuing sections discuss a variety of sampling based motion planning techniques and their applicability to the UAV MFP problem. Note that the majority of these motion planning algorithms are derived for a Boolean search space where a sample point s is either an obstacle or free space.
2.2. SAMPLING BASED MOTION PLANNING
2.2.1
23
Sampling Methods
Sampling based methods try to select a finite number of points (samples) that are representative of W (the continuous world space). Note that a unique sample set can be constructed from a given sample sequence but there are many possible sample sequences for a given sample set. In practice, the information available for UAV MFP is itself a sampling of the real world. [12] A sample sequence is said to be dense if for every element s in W , it is possible to find a sample point that is of arbitrary proximity to s. Conversely, a sample sequence is dense if a sample set is dense. If probabilistic sampling is employed, then the resultant set is probably dense. Typically, a probabilistic sampling sequence employs a uniform distribution to obtain ‘even’ coverage across the state space (refer Fig. 2.3b). [12] Another property used to evaluate a sample set or sequence is the dispersion. The dispersion refers to the largest uncovered (or unsampled) area in the world space [12]. A method for deterministic sampling based on the minimisation of dispersion is the dispersion grid. If the resolution can be arbitrarily increased (corresponding to a decrease in dispersion), then the planner is resolution complete. If a solution exists, than it will be found in finite time but if it does not, then an infinite amount of time may be expended. A Sukharev grid (Fig. 2.3a) is a type of dispersion grid that achieves 1 optimal dispersion by dividing W into k n cubes where n is the number of dimensions and k is the resolution (total number of sample points). Note that 1 if k n is not an integer, then there may be leftover sample points that can be arbitrarily placed without changing the dispersion. The number of sample points in a Sukharev grid is thus exponential in the number of dimensions. A multi-resolution grid can be used to help reduce the number of sample points although this results in non-optimal dispersion. A problem with grid based sampling is that in certain applications, these grids sometimes produce unexpected behaviour because a row of points may align with a free space corridor. Sometimes a solution is obtained with very few samples but other times, a large number of samples are necessary. Such scenarios arise because of the limited connectivity between adjacent sample points. For example, in 2D, the angle of the vector formed by connecting a grid sample point with its (8-connected) neighbour is confined to incre-
24
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
(a)
(b)
Figure 2.3: Illustration of different sampling methods showing Voronoi lines: (a) a Sukharev grid, and (b) uniform random sampling.
ments of 45◦ [12]. Alignment problems increase the sensitivity of grid based sampling to the alignment of obstacles and results in large variation in computation times. They can also result in paths with spurious turns [14, 15]. One method for addressing this shortfall is to use a low-discrepancy sampling method [12]. There are three major approaches: (i) Halton/Hammersley sampling, (ii) (t, s)-sequences and (iii) lattices. The Halton algorithm is a computationally efficient method based on prime integers but its performance degrades exponentially (in terms of dispersion and discrepancy [12]) with the number of dimensions. This method has been used as a quasi-random form of sampling in a probabilistic roadmap based motion planner [29]. Both (t, s)sequences and lattices come from the field of numerical integration. The term lattice refers to a property of the sample points such that all neighbouring points can be obtained by adding or subtracting a vector. In a general lattice, the vectors linking a node to its neighbours (known as a generator) are not necessarily orthogonal. Such a lattice is effectively a ‘skewed grid’ [29]. Dispersion grids and low discrepancy sampling are suited to UAV motion planning because they are deterministic and thus easier to certify under existing aviation software regulations [27]. The Sukharev grid is especially suited because the sample structure reflects the raster structure of Geographic Information System (GIS), Shuttle Radar Topography Mission (SRTM) elevation maps, and occupancy grid based sensor information. Using a Sukharev grid minimises the need for data pre-processing. The problem with a Sukharev
2.2. SAMPLING BASED MOTION PLANNING
25
grid lies in the exponential increase in sample points with dimensionality and the problems of alignment (which can be resolved with a lattice, i.e. non-orthogonal generator vectors). Additionally, a multi-resolution grid can help alleviate the problem of memory complexity. Note that if a constant (or minimal) dispersion is imposed on the sample set, then the sample size always grows exponentially with dimensionality regardless of the sampling method [29].
2.2.2
Collision Detection
An important part of planning is the checking for collisions between sample points and/or graph edges with obstacles. In the ensuing discussion, all edges are assumed to be straight lines unless otherwise specified. Collision detection is modeled with a collision function φ(q) (where q is a configuration in C-space) which yields TRUE if q ∈ Cobs (where Cobs is the set of obstacles) and FALSE otherwise. The minimum distance between all configurations in the obstacle set Cobs and the robot A can be used to model φ. If Cobs ∩ A 6= ∅ , then this minimum distance is 0 and there is a collision. In the na¨ıve approach, every point in A is checked. A better approach is to use a hierarchical tree which is based on successively smaller bounding boxes. Incremental checking is also better than the na¨ıve approach and works on the assumption that the robot only moves a small amount between calls to the collision checker. In order to check an edge for collisions, each point in A needs to be checked at each sample point along the edge. This is very inefficient and sensitive to the step size (which is typically determined experimentally). Consider a robot positioned at configuration q which has a minimum distance of d (in the world space) from the nearest obstacle. If the robot moves (translation only) by a distance less than d to a new configuration q 0 , then the segment from q to q 0 is collision free since every point in A only moves a distance of d. However, for rotation, it is important to find the point furthest away from the origin for rotation and calculate the distance traversed by that point (which corresponds to the greatest distance traversed by a point in A). Such a collision checking strategy helps reduce the number of sample points to check. Further computational savings can be obtained by selecting an uniform sampling sequence (such as with a Van der Corput sequence) [12].
26
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
2.2.3
Incremental Sampling and Searching
Incremental sampling and searching methods are a form of local planning which, as the name suggests, check for collisions at runtime. Even though local planning is not directly applicable to UAV MFP, these methods can be used to help escape traps, or connect start/goal nodes to a global roadmap [12]. An incremental sampling and search approach to motion planning is based on the general procedure listed below: 1. Initialisation: Create a search graph G(V, E) where V contains at least one vertex and E contains no edges. Typically, V contains the initial and goal states (or configurations) though in general, V may also contain other points in the free space as seeds for a multi-directional search for instance. 2. Vertex selection method: Choose a vertex v ∈ V for expansion
3. Local planning method: For some v 0 that may or may not be a part of V , try to construct a collision-free path segment from v to v 0 . If this fails, return to step 2. 4. Insert the successful path segment as an edge into graph G. If v 0 ∈ / V, insert v 0 into V
5. Check for a solution: Determine whether G contains a solution path from the start to the goal 6. Return to step 2 and iterate until some termination condition, in which case report failure
Note that in the first step, if V begins with just one vertex, then the algorithm is classified as a unidirectional search. A bidirectional search is where there are two starting vertices (or seeds) and a multi-directional search is where there are two or more. A multi-directional search can be used to help the algorithm escape local minima (or ‘bug traps’). Normally, a best first algorithm tends to ‘fill up’ a local minima before it can escape it - the number of points is typically O(100n ) where n is the number of dimensions [12]. A multi-directional search can potentially overcome this shortcoming but it is much more computationally complex to determine which connected components (subgraphs stemming from each seed) to join and when to join them. The effect of bug traps (local minima) on the search process is illustrated in Fig. 2.4. A common approach with incremental sampling and searching techniques is to discretise the world space using a grid (such as a Sukharev grid) [12].
2.2. SAMPLING BASED MOTION PLANNING
27
(a)
(b)
(c)
Figure 2.4: Examples of bug traps (local minima): (a) single bug trap and its effect on a unidirectional search, (b) single bug trap and how a bidirectional search can overcome it, and (c) double bug trap and multi-directional search.
It is necessary to select a suitable sampling resolution as oversampling (too many sample points) leads to wasted computation cycles and undersampling (insufficient sample points) leads to loss of completeness. If the resolution can not be selected with a priori knowledge, then it can be determined by iteratively refining the resolution until a solution is found. Alternatively, a continuous planning algorithm could be used such as potential field or rapidly exploring dense tree (refer following sections). In the discrete approach, a search graph is constructed from the sample set by connecting each node with neighbouring nodes. In a 2D grid, neighbouring nodes are selected using a 4-connected or 8-connected neighbourhood [12]. When there are n dimensions, the maximum number of grid neighbours is 3n − 1
(2.4)
28
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
(a)
(b)
Figure 2.5: (a) Potential field contour plot and, (b) skeleton [37].
A path is established by linking successive neighbouring nodes starting from the initial node (or initial/start state) and terminating with the goal node. Potential Fields Potential field based path planning is a form of iterative planning based on an attractive potential to the goal and a repulsive potential from obstacles. Barraquand, Langlois et al.’s [37] seminal work is briefly discussed here. The Randomised Path Planner (RPP) by Barraquand, Langlois et al. [37] is based on a skeleton representation of free space using potential fields. The first step involves taking an uniform grid (or bitmap) sampling of the search space where each cell is classified as either an obstacle or free space. 4neighbour connectivity is assumed (i.e. four neighbours in 2D and six in 3D space). A distance based potential field d, referred to as W-potentials in the workspace and C-potentials in configuration space, is calculated for each cell in the workspace. Cells that lie on the boundaries of obstacles are given a d value of 0. Wavefront expansion is applied to determine the potential field value for all the cells in free space. This process has a computational complexity that is linear in the number of free cells. Note that different results can be obtained with a different field, such as an electrostatic field (where field strength varies with the square of the distance). The W-potentials are then used to create a skeleton representation of the free space by extracting the points where waves collide (refer Fig. 2.5). A skeleton based path is of maximal distance from obstacles, much like a generalised Voronoi diagram and skeletons in image processing (morphology) [37].
2.2. SAMPLING BASED MOTION PLANNING
29
The goal cell is connected to the skeleton by tracing a path along the neighbouring cells with the largest d value until the skeleton is reached. An attractive potential towards the goal is propagated along this augmented skeleton using the same distance metric and wavefront propagation. By establishing a potential field with a global minimum at the goal, a path can be found by following along the (maximal) negative gradient. The computational complexity of forming this potential field is linear in the number of points in the bitmap. As can be seen in Fig. 2.5a, there are several regions in the workspace that correspond to local minima - essentially a basin in the contour plot. The problem of getting stuck in local minima (since the gradient all around is positive) is a well documented shortcoming of potential field based methods [12,34,37,48]. There are three major approaches for overcoming local minima [34, 37]. One approach is to use a navigation function, which is a potential field where the goal is a global minimum and there are no local minima [12]. In general, this is not feasible [34], hence some method for escaping local minima is still needed. A listing of example navigation functions can be found in [12, 49]. Another method for escaping local minima is to use a best first search (refer Section 2.3) to ‘fill up’ the local minima when one is encountered. This method is complete but can be very computationally expensive as discussed previously. The final method for escaping local minima is to execute a random walk (or random motion) when one is encountered [34]. By using randomised techniques to escape local minima, RPP has been shown to be very efficient for robots with 3 to 31 degrees of freedom (dof) in both simulations and practical experiments [37, 48, 50]. However, RPP tends to fail in the presence of narrow passages, or local minima near narrow passages, or where the start and goal nodes lie within local minima (double bug trap in Fig. 2.4b). As a result, RPP is not complete [37, 48, 50]. In general, potential field based path planning is unsuitable to UAV MFP due to the problem of local minima. Rapidly Exploring Dense Trees Rapidly exploring Dense Trees (RDTs) are a class of local planning algorithms that create paths by growing a tree structure from one or more seed
30
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
Figure 2.6: Rapidly exploring Dense Tree (RDT) graph [12].
points. Note that a tree is an acyclic simple graph; if there are n nodes, then there are n − 1 edges. RDT algorithms are dense in the limit which means it is possible to get arbitrarily close to every point with a probability of 1 - refer Fig. 2.6. Again, G(V, E) is the search graph and let S (called the swath) denote all points that can be reached with G. The general algorithm begins with a seed configuration q0 (or equivalently a point in the world space). At each iteration i, a new configuration α(i) is chosen (randomly or otherwise) and is joined to S by finding the vertex qn that is closest to α(i). This can be done with exact or approximate solutions (such as the Kd-tree [12]). RDTs that use approximate or random methods are called Rapidly exploring Random Trees (RRTs). Note that, as with other incremental sampling and search approaches, it is necessary to check for collisions when adding the edge (α(i), qn ). If α(i) lies within an obstacle region, than the nearest point to α(i) on the line (α(i), qn ), labeled qs is chosen. If qs is distinct from qn , then this vertex is added to G. Note that it is possible to have bi-directional or multi-directional RDTs but the difficulty arises in deciding when to connect trees to each other and when to ‘explore’. It is possible to encode aircraft dynamics into the planning algorithm by changing the geometry of the branches. However, due to the nature in which branches are created, the resultant plan can include many spurious turns. This was found to be a problem in helicopter path planning [25]. However, RDT is suited as a local planning method to help connect a node to a roadmap and could even be used as an aid to help escape local minima.
2.2. SAMPLING BASED MOTION PLANNING
31
(a)
(b)
(c)
Figure 2.7: Difference between (a) a grid/raster map, (b) a probabilistic roadmap, and (c) cell decomposition map. Note the reduced number of sample nodes in a roadmap and cell decomposition.
2.2.4
Roadmap Based Planning
A roadmap based approach to motion planning is a type of global planning based on a graph (the roadmap). There are two stages in roadmap based planning: (i) pre-processing (roadmap construction) and (ii) query (graph search) [12]. Typically, the nodes in the roadmap are chosen to minimise the total number of sample points whilst capturing the connectivity of free space. Note that, by this definition, virtually all global sampling based motion planning methods follow this general methodology. The key distinction between individual planning methods is in the method for graph construction. This is illustrated for a grid, cell decomposition and probabilistic roadmap in Fig. 2.7. The Probabilistic RoadMap (PRM) is a widely used method in roadmap based planning. PRM was developed separately by Kavraki and Latombe [51] and Overmars [52]. This algorithm is probabilistically complete and
32
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
A B
Figure 2.8: Example roadmap derived from learning phase in Probabilistic RoadMap (PRM) planning, showing a set of two disjoint, connected components. Note that in going from A to B, even though a straight line path exists, the structure of the graph results in a longer, suboptimal path containing a turn.
computationally efficient. Probabilistic completeness is where the planner is guaranteed to find a solution if one exists given an infinite number of iterations [34, 35]. The pre-processing stage (generally the most computationally expensive stage) of PRM planning comprises two steps: learning, and expansion. In the learning phase, the planner generates collision free configurations (in C-space, or equivalently in world space) at random using a uniform distribution. An attempt is made to connect each node to K neighbouring nodes to create a forest graph structure. Note that a forest is a disjoint union of trees (referred to as connected components) - i.e. the maximum number of edges is less than n − 1. This is illustrated in Fig. 2.8. The next step, entitled expansion, attempts to identify ‘difficult’ regions in C-space such as narrow corridors and tries to create more sample points in these regions. One method for selecting nodes to expand is to use a weighting scheme based on the number of successful connections to neighbouring nodes [48]. For each selected node q, M new nodes are created in the vicinity of q and connected, as in the learning step, to its K closest neighbours not in the same connected component. In the query stage, the initial (start) and goal nodes are attached to the same connected component using a local planner (e.g. RPP, RDT, or a simple straight line planner [48]). Then, a simple breadth first search is conducted to find the shortest path that links the start node to the goal node. As the
2.2. SAMPLING BASED MOTION PLANNING
33
roadmap is a forest graph, the query stage is comparatively inexpensive (in terms of computational effort). The parameters and design considerations for a PRM planner are described below. 1. N , the number of nodes. Increased N reduces query time and improves path quality but increases pre-processing time [51]. Typically N is in the order of a few thousand nodes for many degrees of freedom [48]. 2. K, the number of nearest neighbours. Kavraki [48] suggests that K should be in the order of 30. 3. M , (used in the expansion step). Kavraki [48] again recommends a M value close to N to give good results. 4. Choice of local planner to construct edges and check for collisions. Kavraki [48] used a straight line planner which is fast but likely to be less successful than a slower and more sophisticated local planner (such as one based on RPP [37]). Because density is important in a roadmap, it has been empirically shown that it is better to use a fast local planner with more nodes than a slower one with fewer nodes [48]. 5. Method for collision checking. Kavraki [51] employed a bitmap based collision detection algorithm. An alternative but more computationally intensive method is to use iterative distance to obstacle based collision checking [12, 53]. 6. Choice of distance function. Kavraki [48] uses the Euclidean distance metric. It has been shown that the probability of a PRM planner failing when a solution exists is dependent on: (i) the distance between sample nodes and obstacles, (ii) the distance between the start and goal nodes, and (iii) the number of sample points N . The probability of success increases exponentially with N and is linear in the distance from start to goal. Note that further roadmap expansion can be performed (followed by another query) in the case of failure [12, 48]. [35, 54] PRM has been employed successfully in a wide variety of applications in 2D and 3D world spaces, and also in helicopter motion planning [25]. The PRM employed in [25] uses a straight line local planner with 5000 nodes to achieve 97% map coverage. A PRM is typically faster than grid based planners as fewer nodes are placed in free space regions [54]. Unfortunately,
34
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
because the roadmap is a forest (i.e. acyclic), the resultant path can be suboptimal and contain spurious turns [48]. This problem is highlighted in Fig. 2.8 and is a shortcoming of all acyclic roadmaps. As a result, PRM planning is not suited to UAV MFP, even though the methodology itself is useful. There are a large number or PRM variants. For example, Obstacle-Based PRM (OBPRM) is designed for cluttered workspaces with long narrow corridors between obstacles [55]. Lazy PRM seeks to reduce overall planning time by foregoing the collision checking step of roadmap construction. An even lazier variant is where the roadmap is defined implicitly thus foregoing the learning phase altogether; the sample nodes are instead generated during the search process [29]. Branicky, LaValle et al. [29] present a lazy PRM variant that uses the Hammersley-Halton sampling sequence. They found that the imposition of a lattice neighbourhood structure does not degrade performance in terms of discrepancy and dispersion. It is shown that lowdiscrepancy sampling finds paths more quickly and using a fewer number of sample nodes than PRM in some selected simulations. Dale and Amato [35] offer a classification of PRM algorithms (and include RRT) with respect to the ‘degree of clutter’ of the operating environment (refer Table 2.1). Note that the classification is a guide only as it is difficult to quantify the ‘degree of clutter’. Additionally, the shape and orientation of obstacles also influence planner performance. Of the existing PRM variants, the quasi-random lattice based lazy PRM proposed by [29] is best suited to UAV MFP as the roadmap is not acyclic (i.e. no spurious turns). Unfortunately, due to this, the query stage is more computationally complex.
2.2.5
Cell Decomposition and Multi-Resolution
Cell decomposition is a widely used methodology for path planning that divides the workspace into regions called cells. A path comprises a sequence of cells that link the start state to the goal state. The primary motivation behind cell decomposition is to be able to reduce the number of cells used to represent the world space or C-space by increasing the sampling resolution of cluttered regions and lowering the resolution in open regions [34]. This leads to a smaller search graph which reduces computation time and memory
2.2. SAMPLING BASED MOTION PLANNING
35
Table 2.1: Suitability of different Probabilistic RoadMap (PRM) planners to different classes of search space. Note that an open region refers to one that is predominantly free space [35].
Algorithm
Open
Clutter Less
Basic PRM [51] Fuzzy PRM [56] Lazy PRM [57] RNG [58] OBPRM [55]
MAPRM [61] Ariadne’s Clew [62] RRT [63]
Most
×
×
×
×
×
×
×
×
×
× ×
Gauss PRM [59] Visibility Roadmap [60]
More
× ×
× × × × ×
Dilated Spaces [64] Closest VE (surfaces) [65] User Input [66]
×
× ×
×
×
×
×
×
× ×
× ×
storage. The graph is generated by treating each cell as a node and linking that with the immediate neighbours of that cell. The ‘pre-processing’ stage of cell decomposition generates a binary classification of cells such that each cell is either empty (free space) or full (obstacle). This can be done using an exact decomposition (where cells match precisely the shape of obstacles) or approximate decomposition (where cells are of regular shape, e.g. cubes). A planner based on an exact decomposition is complete whereas one based on approximate decomposition is resolution complete. However, exact decomposition is computationally intractable for outdoor environments as obstacles are often of non-regular shape [12]. In approximate decomposition, cells can be empty, full or mixed (region partially occupied by an obstacle). Mixed cells are decomposed as much as possible into free and full cells. If there is insufficient resolution (i.e. the granularity is too coarse), then the solution path may contain mixed cells which makes the planner unsound [67]. That is, even though a path is found, it may not be possible for the robot to physically traverse a mixed cell because
36
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
there is no information as to the size and shape of obstacles inside a mixed cell. One commonly used method for approximate cell decomposition is to partition a cell into 2m equally sized cells. When m = 2 this is called quad-tree decomposition (in 2D) and when m = 3 this is called octree decomposition (3D) [36]. Quad-trees offer a compact, non-linear representation of a 2D environment by modeling large constant-cost regions as single cells [14]. However, if there are obstacles of irregular shape and orientation relative to the cell structure, a large number of decompositions are needed [4, 34]. One way to overcome this problem is to set a threshold based on the fraction of a cell occupied by obstacles, however, this can result in a path that is unsound. The primary difficulty of using cell decomposition for UAV MFP is that weighted region cells are not conducive to a binary classification. Furthermore, many existing methods for decomposition require a polygonal obstacle representation. As there are multiple decision variables, there may not be many (if any) regions of constant weight. A similar approach to cell decomposition is multi-resolution sampling. Both these methods subdivide the search space into regular shaped cells of different sizes to reduce the total number of cells. However, instead of creating a sampling structure based on obstacles, the cells in multi-resolution planning are specified beforehand using a priori knowledge. Methods for vehicle motion planning that employ multi-resolution search are discussed in Section 4.1.
2.3
Graph Search Algorithms
All sampling based motion planning algorithms create a graph (also referred to as a roadmap) that models the connectivity of the search space. This section describes the search algorithms that can be used to find a nearoptimal or optimal path on this graph.
2.3.1
Dynamic Programming
Virtually all optimal search algorithms are derived from the principle of dynamic programming. Pioneered by Bellman in the 1940s, dynamic programming is founded on the concept of dividing a problem or process to be opti-
2.3. GRAPH SEARCH ALGORITHMS
37
mised into stages. At each stage, the optimal decision is made such that the objective is optimised globally over the total number of stages. The principle of optimality is: “From any point on an optimal trajectory, the remaining trajectory is optimal for the corresponding problem over the remaining number of stages or time intervals initiated at that point” [68]. Mathematically, the optimal cost J over the interval [k, K] is ∗ J(k,K) (x) = inf
uk ∈Ak
∗ ∗ J(k,K) (x, uk , U(k+1,K−1) )
(2.5)
∗ is the optimal sequence of actions u for the interval (k + where U(k+1,K−1) 1, K − 1), Ak is the set of possible actions at stage k, x is the state and inf is the infimum operator which returns the argument that gives the minimum expression [68]. If the objective function (i.e. costs) are additive across stages and independent of k (implying that the system is time invariant), then (2.5) simplifies to G∗k+1 = inf [c(x, u) + G∗k (f (x, u))] (2.6) u∈A
where G is the optimal cost, f (x, u) is the state arrived at by applying action u at state x and c is the transition cost [68]. Fuzzy dynamic programming is a variation of Bellman’s principle first put forward by Bellman and Zadeh [69]. The defining feature of fuzzy dynamic programming is the use of conjunctive operators to determine the overall utility of a plan. This is suited to implementing a satisficing solution (one that meets minimum constraints) as the total evaluation is the logical conjunction of decision outcomes at each stage [70]. This overall utility is defined as a Membership Function (MF) µG where given a N stage plan [71] µGN −i (xN −i ) = max µCN −i (uN −i ) ∧ µGN −i+1 (xN −i+1 ) uN −i
(2.7)
where µC is the constraint MF at each stage, ∧ is a conjunctive t-norm, and xN −i+1 is the state arrived at by applying action uN −i at state xN −i : xN −i+1 = f (xN −i , uN −i ). By using MFs, it is possible to extend (2.7) to model state uncertainty and state transition uncertainty but doing so requires significantly more computation. The key shortcoming of fuzzy dy-
38
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
namic programming is that current methods for finding a solution need to solve N simultaneous equations for a system containing N nodes. This is clearly intractable for large search spaces and further exacerbated by the limited computational capacity onboard small UAVs. [71] Another variant of dynamic programming that incorporates state and state transition uncertainty is Approximate Dynamic Programming (ADP). The utility is evaluated with a value function V [11]: V n (X) = max c(X, u) + E V n−1 (Xk+1 ) u
(2.8)
where, as before, c is the cost of executing action u at state X and E is the expected value. In practice, it is difficult to obtain an algebraic expression for V , hence this is approximated, typically with a linear or piecewise linear function of system input variables. Unfortunately, ADP is also significantly more computationally expensive than conventional dynamic programming. [11] Due to the relative computational simplicity of Bellman’s principle of dynamic programming (refer equation (2.5)), existing graph search algorithms are predominantly derived from this principle. These are discussed in the following section.
2.3.2
Best First Search
Best first search refers to a class of graph search algorithms which expand nodes that are deemed to be the most promising by some evaluation function. Examples of best first search include Dijkstra’s algorithm [72], A* [73] and variants of A* (e.g. D* Lite [30]). A brief overview of key search algorithms is provided as a background to the proposed MFP search algorithm. The A* algorithm [73] (pronounced A-star) is a systematic, heuristic search that finds the optimal (or least cost) path [12]. A heuristic is “any rule of thumb or strategy that is used to limit the time required to search for solutions” [74]. A* belongs to the class of forward label correcting search algorithms and is a special form of dynamic programming [12, 33]. The search procedure, as shown in Fig. 2.9, makes use of a queue to determine the order in which to expand nodes.
2.3. GRAPH SEARCH ALGORITHMS 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22:
39
for all s ∈ S do g(s) ← ∞ end for Queue ← ∅ s ← s0 g(s) ← 0 Queue.Insert(s) while s 6= sg do s = Queue.P op() if s = sg then return end if S 0 ← Γ(s) for all s0 ∈ S 0 do gˆ(s0 ) = g(s) + c(s, s0 ) ˆ 0 , sg ) fˆ(s0 ) = gˆ(s0 ) + h(s 0 0 if gˆ(s ) < g(s ) then g(s0 ) ← gˆ(s0 ) Parent(s0 ) ← s Queue.Insert(s0 ) end if end for end while
Figure 2.9: A* algorithm. s0 is the start node, sg is a goal node and Γ is the successor or neighbourhood operator.
Beginning with the start node, A* successively expands nodes in increasing order of f (the evaluation function), where f is the sum of the cost to come g (from the start node) and a heuristic estimate of the cost to go h (to the goal node). Thus, f represents the total path cost. The cost to come can be calculated using the principle of dynamic programming (compare with equation (2.6)) g(s0 ) = g(s) + c(s, s0 ) (2.9) where c is the cost of a transition (i.e. edge cost in the graph) from node s to a successor node s0 . At each iteration, A* evaluates the cost to reach successor (or neighbour or child) nodes S 0 via a path through s. Γ(s) = S 0 is said to be the successor operator. For a graph derived from a 2D grid, Γ could be a simple 8-connected neighbourhood. If a lower cost path to s0 is found, then s is set as the parent to s0 ; at algorithm completion, the path
40
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
is reconstructed by following the parent pointers from the goal node back to the start node. The heuristic is used as part of the evaluation function to guide the search. If this estimate is close to the actual cost to go, it allows the algorithm to prune candidate paths and investigate the most promising paths first. Hence, it can be more efficient than a less informed algorithm. A* finds the least cost path if this heuristic never overestimates the actual cost to go. Such an estimate is said to be admissible [73]. The Euclidean distance metric for instance is a commonly used heuristic estimate for world space based graphs [12, 14]. A* has also been shown to be computationally optimal [73] in that an equally informed algorithm can not find a path in fewer steps than A*. As a result of this optimality (optimal computation time and path cost), A* and its variants are widely used in mobile robotics (e.g. [14, 30, 39, 41, 47, 75–77]). Dijkstra’s algorithm is a special case of A* where h = 0 [12]. Additionally, in Fig. 2.9, if instead of using the evaluation function f , a First In First Out (FIFO) queue is used, then the algorithm degenerates into a breadth first search. Similarly, if a First In Last Out (FILO) queue is used, then a depth first search is obtained. These search algorithms are not suited for finding least cost paths in general, unless all edge costs are equal [12]. A number of heuristic search algorithms build upon A* to optimise its performance for specific planning scenarios. Dynamic A*, such as D* (Focused Dynamic A*) [78] and D* Lite [30] reduce online replanning time by reusing the results of previous searches. D* and D* Lite perform a backward search (from the goal) since, if a forward search was employed, it would be necessary to update the plan every time the robot moved. It has been shown that D* and D* Lite use substantially less time for replanning than A* since A* plans from scratch (i.e. disregards all previous planning efforts). These algorithms assume that the changes to the environment are small and are reflected as changes to edge costs (the c term in (2.9)). However, D* and D* Lite are less efficient than A* if there are significant changes to edge costs, or if the changes are close to the goal. This is because all affected nodes are processed twice. In UAV MFP, the presence of dynamic obstacles and dynamic weather conditions limit the advantages of D* and D* Lite as they alter the cost for large regions of the search space.
2.3. GRAPH SEARCH ALGORITHMS
41
Anytime Replanning A* (ARA*) [79] and Anytime Dynamic A* (AD*) [76] are anytime variants of A* and dynamic A* respectively that quickly find an initial (possibly highly suboptimal) path, and then iteratively improve this solution (terminating with the optimal path). These algorithms are also referred to as a form of weighted A* [80]. Anytime algorithms achieve computational efficiency by inflating an admissible heuristic by a factor > 1. Inflation of the heuristic value can considerably reduce search time, even in cluttered environments [79]. Each iteration in the main loop uses successively smaller values of (terminating with = 1) to generate potentially more optimal paths. It has been proven, however, that the total path cost is no more than times the optimal path cost [79]. ARA* makes use of three queues, one for states that are yet to be processed called OPEN, one for states for which an optimal cost has been calculated called CLOSED, and another one for states that have become inconsistent called INCONS. Unlike A*, when a state becomes inconsistent (i.e. a new path is found that is of lower cost), it is not placed back into the OPEN queue but is instead put onto the INCONS queue. This also reduces search time because states are not revisited. At the next execution of ARA*, the nodes in INCONS are put back onto the queue to ensure eventual convergence onto the optimal path. As a result of placing inconsistent nodes onto a separate queue, ARA* takes more time to find an optimal path, but can quickly return a navigable path. However, this is not always the case because an inflated heuristic results in a more greedy search, which tends to spend more time exploring local minima than A* with an admissible heuristic [79]. The running time of A* has been shown to be strongly dependent on the heuristic [33, 79]. A method to reduce the sensitivity of A* to the heuristic is to evaluate the first K nodes in the queue at each iteration. Known as K-Best-First Search [80], this method is useful in scenarios where the cost function is non-monotonic and A* would struggle to overcome any incorrect (but admissible) heuristics [80]. Note however, that the original A* derivation explicitly assumes a monotonic cost function [73]. Nevertheless, K-BestFirst Search represents a generalisation of best first search that is useful for planning scenarios with non-monotonic cost functions. A shortcoming of the aforementioned best first search algorithms is the need for PSPACE memory [81]. One method to overcome this is to employ
42
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
node retraction – when the search runs out of memory, the children of the worst node in memory are deleted and the parent updated to be the minimum cost of the children [82]. Linear memory search algorithms also address this shortcoming and are discussed along with alternative search algorithms in the following section.
2.3.3
Other Search Algorithms
Value iteration is a common search method that works off the same dynamic programming principle as A*. Typically cast as a backward search, the optimal cost to go at each stage is G∗ (s) = min {c(s, u) + G∗ [f (s, u)]} u
(2.10)
where G is the cost to go, s is the current node, u is equivalent to the successor operator1 described previously such that f (s, u) is a successor (neighbour) node [12]. Note the similarity to equation (2.9). Value iteration can be modeled as A* except at each iteration, all the nodes on the queue are expanded (not just the topmost node). Conceptually, this is similar to the propagation of cost waves [4]. This approach is extensible to decision theoretic planning but requires O(KV M ) time where K is the number of stages, V the number of nodes and M the number of neighbours to each node. In comparison, A* 3 and related heuristic search algorithms have a complexity of O(N 2 ) where N is the number of nodes [26]. Both A* and value iteration require PSPACE memory. The class of linear memory search algorithms overcome this requirement by trading computation time for memory usage. These are based on a depth first search where the search graph is conceptually represented as a tree. Any finite problem space can be transformed into a tree so that a single state in problem space becomes multiple nodes in the search tree [83]. Note that this tree may be infinite even with a finite problem space. Such a tree is characterised by an asymptotic branching factor b (a ratio of the number of child nodes to a parent node in the limit) and maximum depth d. A breadth first search for example requires O(bd ) memory and O(d2 ) computation time whereas a depth first search requires O(d) memory and O(bd ) computation time [84]. 1
As this is a backward search, technically, this is a predecessor operator
2.4. PLANNING WITH UNCERTAINTY
43
Incremental Deepening A* (IDA*) is an example of a linear memory search [85]. At each iteration, IDA* expands the child nodes of all parent nodes to a depth d determined by a cost limit f (s) = g(s) + h(s) like in A*. Like A*, the optimal path is found if an admissible heuristic is used. If the goal is not found at this iteration, then the cost threshold for the next iteration is set to the minimum cost that exceeded the current threshold. This procedure is repeated until the goal is found. The difference between A* and IDA* is that A* does not re-expand duplicate nodes in the tree because all nodes are tracked in the problem space. This is not the case for IDA* which, as a result, is more computationally expensive (but more memory efficient). Another linear memory search algorithm is Recursive Best First Search (RBFS) [83]. This algorithm is based on a similar methodology to IDA* but improves on the efficiency of iterative deepening by using local cost thresholds instead of a global threshold for each iteration. RBFS has a memory complexity of O(bd) and time complexity of O(bd ) (assuming uniform edge costs) and O b2d−1 (non-uniform edge costs and successive nodes are in different subtrees). As edge costs in MFP are unlikely to be of uniform cost, it can be seen that RBFS is also less efficient than A*. An alternative approach to memory efficiency is to use A* but only store nodes that are on the queue. Entitled ‘Frontier Search’ [81], the search component of this algorithm is identical to that of A* except that path reconstruction is difficult and much more computationally intensive. It can be seen that linear memory search algorithms are in general more computationally expensive than A* and are not suited to UAV MFP (which requires online replanning).
2.4
Planning with Uncertainty
Uncertainty is a pervasive element in the operation of UAVs. Saffiotti [86] describes four sources of uncertainty: (i) a priori knowledge of the world is incomplete, (ii) sensor data is unreliable and uncertain, (iii) there are dynamic obstacles and (iv) control actions do not necessarily produce the desired state transition. An alternate classification of uncertainty is described by LaValle and Sharma [87] where: (i) there is uncertainty in the perceived configu-
44
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
ration of the robot (configuration sensing), (ii) in the future configuration of the robot (configuration predictability), (iii) uncertainty in the perceived state of the environment (environment sensing), and (iv) in the future state of the enviroment (environment predictability). Note that the state of the environment includes all possible robot configurations. The wind, for example is an unpredictable element that can drastically affect the current state and predicted future states of the aircraft. The aircraft state is itself composed of the decision variables used to evaluate mission objectives such as position, velocity, time, fuel and risk. There are two major approaches to planning under uncertainty [12]. A typical approach is to obtain a best estimate of the state first, and then employ deterministic2 planning algorithms (such as those described in Section 2.2). Uncertainty is mitigated through provision of tolerance and/or online (in-flight) replanning. Alternatively, the uncertainty can be incorporated into the planning process itself - this is known as decision theoretic planning. The former approach is widely used in UAV motion planning (e.g. [21, 41– 43, 45, 88]) as generally, unlike the latter, it is tractable for a large search space [12]. Planning algorithms based on the former approach are discussed in Section 4.1. This section provides a brief overview of the latter approach. An alternative method for incorporating uncertainty in the decision variables (but not in the planning process itself) is with fuzzy logic as described in Section 3.5.1. Decision theoretic planning models uncertainty in (i) the transition from one state to another given a chosen action u, and (ii) uncertainty in the state itself. Consider the first case where there is only uncertainty in the state transition. A set of actions U are associated with a probability distribution P (u) for each action u ∈ U . The problem is cast as a game where the robot executes some action u and then nature executes a random action θ independent of u. As a result, the cost c of a state transition is c(u, θ). There are two major approaches for addressing this type of uncertainty. One is the non-deterministic approach where the robot has no information about nature. A worst case analysis is employed where it is assumed that nature always chooses the action that will lead to the maximum possible transition cost. It has been found that such an approach is often too pes2 Note that a deterministic algorithm in this case refers to one that does not incorporate uncertainty. It does not refer to determinism in planning time or solution cost.
2.4. PLANNING WITH UNCERTAINTY
45
simistic and can lead to highly suboptimal paths [12]. Furthermore, even though it is guaranteed to find a solution if one exists, it will not be able to find a better solution even if one exists. An alternative approach is to use a probabilistic model which uses statistics describing the distribution of θ to find an optimal path. One problem with this Bayesian based approach is that it is very difficult to estimate P (θ). Many independent trials are required and for UAVs operating in an outdoor environment, the prior probabilities are either not available, erroneous or delayed [89]. One way to overcome this lack of information is to use Laplace’s principle of insufficient reason and assume a uniform distribution for θ. However, it is possible to get different results by posing the problem in a different way which makes the method sensitive to interpretation. In scenarios where it is possible to observe nature prior to choosing an action, another problem is the difficulty of estimating the conditional probability of an observation given a nature action. Therefore, it can be seen that a shortcoming of the probabilistic approaches lies in the difficulty of accurately estimating probabilities. Without accurate probabilities, the plans can be suboptimal [12]. The above discussion relates to a single decision under uncertainty. The process of creating a plan requires a sequence of such decisions – this is often referred to as a Markov Decision Process (MDP). Based on the Markovian assumption, the cost after k steps is c(xk , uk , θk ) and [12] P ( θk+1 | x1 , . . . , xk , u1 , . . . , uk ) = P ( θk+1 | xk , uk ).
(2.11)
There are many methods for solving MDPs of which popular methods include the probabilistic value iteration method, policy iteration and probabilistic Dijkstra’s algorithm. Probabilistic value iteration is inefficient as time is wasted re-evaluating states with no new information. Policy iteration is infeasible for systems with large state spaces as for N states, N simultaneous equations have to be solved at each iteration. Probabilistic Dijkstra’s algorithm, which is virtually identical to conventional Dijkstra’s algorithm (refer Section 2.3), then remains as the only tractable alternative. [12] The preceding discussion ignores uncertainty in the state itself. In this scenario, the decision space is augmented by an observation space Y which represents the robot’s efforts to estimate the true state. Uncertainty in the estimate of the true state is modeled by an interfering nature sensing action.
46
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
Such a problem is referred to as a Partially Observable Markov Decision Process (POMDP). Solving a POMDP is difficult because the number of states in information space is exponential in the number of states (or graph nodes) with the number of sensor states as the base. This makes the planning problem computationally intractable for many mobile robotic operations [47, 90]. Gonzalez and Stentz [47] however present a novel method for planning with state uncertainty (effectively position uncertainty) that combines a deterministic heuristic algorithm (A* [73]) with the state estimation aspects of a POMDP. This algorithm uses a grid to sample the world space and optimises for a single, monotonic objective function. The algorithm has a worst case complexity of O(Nx Ny Nu ) where Nx , Ny , Nu are the dimensions for the x, y and uncertainty axis u respectively. Another method for addressing the problem of state uncertainty is with ‘pinch points’ [90]. This algorithm is developed for a single objective 2D roadmap using a deterministic heuristic search. Each cell has associated with it a probability of being untraversable (similar to an occupancy grid). A pinch point is defined as a region where the combined probability for the cell and its neighbours exceed a given threshold. The rationale behind pinch points is that in the event that a cell turns out to be non-traversable, it is likely that traversing the cells around it would also be difficult. Ferguson and Stentz [90] look for such pinch points in the map and plan around them using a heuristic search algorithm to help overcome the computational complexity of POMDPs. Bakker, Zivkovic et al. [91] describe a different approach based on MDPs and value iteration. The key to their method is that planning is done hierarchically with maps that range from low detail (high level planning) to high detail (low level planning) maps. Higher level maps are generated from lower level maps through clustering. The plans created at the higher levels can guide the planning process at the lower levels. However, the proposed method is still computationally expensive compared to deterministic algorithms with a complexity of O(KN 2 ) where K is the size of the neighbourhood and N the total number of nodes [91]. It can be seen that incorporating uncertainty into the planning process greatly increases the computational complexity. Non-deterministic methods
2.5. COMPUTATIONAL COMPLEXITY
47
Figure 2.10: Computational complexity classes [12].
can lead to highly suboptimal solutions and probabilistic methods tend to be computationally expensive and require probability estimates which are very difficult to obtain. Inaccurate estimates of the probabilities can lead to suboptimal solutions.
2.5
Computational Complexity
An important measure of an algorithm’s usefulness for UAV MFP and especially online replanning is the computational complexity. Complexity analysis of a planning algorithm is based on the notion of a problem (or language). For example, the class-P problem comprises all languages for which algorithms with polynomial running time exist; i.e. O(nk ) time for integer k. An algorithm with polynomial running time is deemed as efficient. In practice, if k is very large, than the algorithm is not practical. If a polynomial time algorithm does not exist, then a problem is deemed intractable (i.e. problem class-NP or harder). The NP class of problems (NPcomplete) require a non-deterministic Turing machine polynomial time to solve. On the other hand, the PSPACE class of problems require polynomial storage space during execution. Finally, EXPTIME is the set of languages k which are decided in O(2n ) time (for integer k). It is known that EXPTIME is larger than P, but the boundaries for NP and PSPACE are not known with certainty. These classes are illustrated in Fig. 2.10. [12] Let X denote P, NP, PSPACE or EXPTIME. A language A is classified as X-hard if every language B in class X is polynomial-time reducible to A. That is, in polynomial time, every instance in B can be translated to an instance
48
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
in A, and the decisions can then be translated back in polynomial time. If it is known that a language is X-hard and also a member of X, then it is referred to as X-complete. For example, the general motion planning problem defined in [12, Formulation 4.1] is PSPACE-complete. It has been shown that planning for a robot under uncertainty for a 3D polyhedral environment is NEXPTIME-hard which is even harder than EXPTIME [12]. In analysing complexity, a lower and upper bound is usually given. The existence of an algorithm typically provides the upper bound as it proves that no more time than that needed by the algorithm can solve the problem. Lower bounds on the other hand are proven through analysis on the problem itself, typically with artificially complex scenarios. This is useful as it is not possible to make an algorithm any more efficient than the lower bound. For randomised algorithms, the expected running time becomes the worst-case upper bound. For the general motion planning problem, the upper bound has been shown using Canny’s Silhouette method and randomised planning. Let n be the number of dimensions, m the number of polynomials (or half planes for polyhedral representations) to represent obstacles and d the degree of the 4 2 polynomials. The upper bound is thus mn log(m)dO(n ) and mn log(m)dO(n ) for Canny and randomised planning methods respectively. As noted previously, many polyhedrons may be required to model an outdoor operating environment leading to a large mn term in the above upper bound. By comparison, graph search algorithms have significantly lower complexity. For example, the popular Dijkstra’s algorithm has an upper bound of O (V log(V ) + E) where V is the number of graph verticesandE the number 3 of edges [12]. This compares with an upper bound of O V 2 for A* based algorithms on a grid [26]. A* and its variants are widely used in mobile robot motion planning due to this efficiency (e.g. [14, 15, 21, 30, 31, 39, 41, 42, 75, 77, 92]).
2.6
Findings
UAV MFP is a global motion planning problem based on the weighted region problem where each sample point has a non-binary weight calculated from multiple decision variables. This chapter reviewed some fundamental mod-
2.6. FINDINGS
49
els and algorithms in motion planning and identified suitable methods for modeling the UAV MFP task. These methods are generic and not directly applicable to MFP; none of the reviewed methods incorporate multiple decision objectives in a computationally efficient manner (as required in research objective one). Motion planning is a difficult problem that has been shown in general to be PSPACE hard [12]. Because of the time critical nature of online replanning, it is necessary to adopt a near-optimal, resolution complete (or probabilistically complete) approach. It is not feasible to plan in seven dimensional C-space (six dof for the UAV and time) due to a large operating environment. The path must instead be planned in a four dimensional world space (three spatial dimensions and time dimension). Because the aircraft is small relative to obstacles and distances between obstacles, it is possible to assume that the UAV is a point object. Obstacles and the environment in general can be modeled using semi-algebraic (including polyhedral) and grid based representations. For example, gridded GIS data is suited to a bitmap representation whereas regions of airspace and separation zones, which are often of regular shape, are suited to semi-algebraic representation. It was found that a deterministic, sampling based approach to motion planning was best suited to UAV MFP because of computational tractability and resolution completeness. Additionally, deterministic sampling enables a priori determination of sampling density, discrepancy and vector angles between neighbours (with a vector/lattice neighbourhood). This is not possible with random sampling. Candidate deterministic sampling techniques include the Sukharev grid, lattice, multi-resolution grid and quasi-random sampling sequences. The PRM motion planning methodology was found to be well suited to UAV MFP, even though a PRM in itself is not suited as the solution path may contain spurious turns (which stem from an acyclic graph). A lazy PRM approach, where collisions are checked at the query stage or the even lazier variant where the roadmap is implicitly defined, is especially applicable. Such an approach avoids unnecessary pre-processing of unvisited regions in a large search space. A lazy PRM is similar to a local planner as there is no explicit roadmap building; however, unlike local planning, the resultant
50
CHAPTER 2. REVIEW OF MOTION PLANNING THEORY
path is globally optimal. Local planning methods like RDT and the simple straight line planner [48] can be used to connect start/goal nodes to the roadmap or aid in escaping local minima. An evaluation of existing graph search algorithms for the query stage of PRM planning revealed that the A* algorithm, which is based on the principle of dynamic programming, is best suited to UAV MFP. A* has been shown to find the least cost path in an optimal manner such that any equally informed algorithm cannot find the solution in fewer iterations than A* [73]. By inflating an admissible heuristic by a constant factor (as is the case for anytime algorithms), it is possible to reduce computation time and produce a path that is no more than times the cost of the least cost path. Thus, A* is well suited to the online replanning component of MFP. In comparison, other search algorithms (e.g. D*, ADP, RBFS) are less efficient than A*, especially in the presence of dynamic obstacles and global changes to the environment (such as changing wind conditions). Given that the memory capacity of modern computing platforms are in the order of gigabytes, it is possible to store the entire search graph, even if there are millions of sample nodes. Fuzzy dynamic programming, especially the deterministic variant, is a viable alternative approach. Unlike A*, which uses the logical disjunction (i.e. additive operator), a fuzzy dynamic programming solution is based on the logical conjunction of individual decisions. However, existing methods for solving the fuzzy dynamic programming recurrence relation are computationally intractable for large search spaces. Finally, existing methods for decision theoretic planning were found to be unsuitable for UAV MFP. Non-deterministic approaches can produce highly suboptimal paths, and probabilistic approaches tend to be intractable for large world spaces. Also, it is difficult to obtain accurate estimates of the necessary probability distributions. Instead, the majority of existing vehicle motion planning algorithms adopt an approach of planning with a best estimate of the operating environment. Online replanning and tolerances in the motion plan are used to mitigate uncertainty. If changes occur during mission execution or if planning assumptions are violated, then a replan is performed. Online replanning is a critical part of UAV MFP given the unpredictability of the operating environment [4].
Chapter 3 Review of Expert Pilot Decision Making I can stand brute force, but brute reason is quite unbearable. There is something unfair about its use. It is hitting below the intellect. Oscar Wilde
D
ESPITE the memory and time complexity of the motion planning problem as described in Section 2, Mission Flight Planning (MFP) is a task that is performed with proficiency by human expert pilots [28]. Replication of the decision strategies (as opposed to direct replication of human knowledge which is difficult [93]) used by human experts can help increase the efficiency of the planner. Additionally, this helps to provide a higher degree of cognitive compatibility which increases the system’s usefulness in terms of design and operation [94]. The study of human thought processes and decision making is known as cognitive science; in the context of MFP, it is “how pilots think in operational situations” [93]. This differs subtly from the definition of decision making which is “the ability of a pilot to respond to cues from the environment, evaluate the situation, come to conclusions and act on these conclusions (possibly the only reason for keeping a human in future aircraft)” [93]. Cognitive science and the study of human decision making is a diverse field in itself. This chapter focuses on the cognitive processes of expert pilots and how they handle uncertainty. In addition, a number of methods for replicating these decision strategies are discussed. This chapter does not consider such aspects as the psychiatric study of human cognition in general, human foibles and 51
52
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
cognitive training. For a more comprehensive overview of decision theory and cognition, refer [28].
3.1
Overview of Human Cognition
There are two major approaches to the study of human decision making. The classical approach, known as the normative approach, prescribes the manner in which a rational decision maker arrives at a decision. In comparison, the descriptive approach studies human decision makers in a practical context to determine the cognitive strategies involved without prescribing a preset procedure [95]. The former assumes a rational decision maker whilst the latter questions this assumption. First put forward by Bernoulli in 1738, the rationality principle makes four key assumptions: • the decision maker can generate all decision scenarios (or alternatives), • the decision maker can evaluate the attractiveness of all these scenarios, • the decision maker can aggregate all local evaluations into a global one, and • the decision maker chooses the alternative with the most favourable global evaluation. In other words, the problem is assumed to be exhaustively described including all problem parameters and possible alternatives. Suitable objective functions are defined that directly relate the attractiveness of each alternative to the problem definition. Therefore, using this classical approach, the decision maker will ultimately arrive at the optimal solution [95,96]. The classical approach has been deemed unrealistic by such influential researchers as Bernard Roy. It is argued that this approach requires an unrealistic amount of computational effort from the human decision maker. Indeed, even if sufficient time exists, it has been found that pilots and human experts in general will still revert to ‘non-rational’ or naturalistic decision making methods (see Section 3.2) [97]. The descriptive approach rejects the rationality principle and adopts instead an assumption of bounded rationality [98]. It is proposed that humans do not consider all attributes for each decision alternative; typically, humans focus on only three to four categories of attributes. This is especially
3.1. OVERVIEW OF HUMAN COGNITION
53
Figure 3.1: Cognitive continuum [97]. Note that RPD refers to a Recognition Primed Decision, and MAUA refers to Multi-Attribute Utility Analysis.
the case for decision making under duress (i.e. time pressure). Studies show that there is no added accuracy in using a decision model with more than ten variables [99]. Furthermore, humans only process a few alternatives rather than all possible alternatives. It can be seen that for the purposes of determining how human pilots (i.e. experts) currently perform the MFP task, it is inappropriate to survey studies based on the normative approach. Thus, the following discussion is constrained to studies based on the descriptive approach for expert pilots. Generally speaking, an expert is someone who can “make sense out of chaos” [95]. Experts are capable of using a relatively small amount of information to make a decision but can do so rapidly and competently [95,100]. In addition, they are able to respond to unique problems under stressful conditions with novel demands [93]. Furthermore, experts are are able to convince others using their experience in the field of expertise [95]. Adam [93] suggests that one of the key areas of differentiation between experts and novices is the speed and accuracy of problem solving using both long term and short term memory. It is observed that experts process information in short term memory, but compile strategies for solving problems in long term memory [101]. The administration of memory is paramount to the form of cognition known as intuition. Intuition can be defined as “knowledge based on experiences (i.e. memory) and acquired through sensory contact” [93]. This is a characteristic trait of expert pilots who transform a procedural decision making task into a cued recall task [28, 93, 97]. It has been said by Nobel laureate Herbert Simon that “expertise and intuition are aspects of the same thing” [93]. Cognitive processes can be plotted on a continuum between purely intuitive and purely analytical decision making (Fig. 3.1). Known as cognitive continuum theory, it is stipulated that the location on the continuum for a particular task can help identify which cognitive processes will be activated.
54
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING Table 3.1: Comparison of intuitive and analytic decision making.
Intuitive Decision Making
Analytic Decision Making
Sequential generation and evaluation of options
Concurrent generation and evaluation of options
Bounded rationality
Unbounded rationality
Use of fast and frugal heuristics to reduce search time
Search for a solution is a constrained optimisation problem
For example, tasks that requires the processing of large amounts of information in a short time span lie towards the intuitive side. Obviously, the locus on the continuum for a task can change with changes in the task conditions (e.g. more time). The continuum can also be used to predict the likelihood and impact of errors. Analytical decision making typically produces fewer errors than intuitive methods, but these errors are more severe. It has been found, contrary to common perception, that employing intuitive methods under intuitive task conditions produces more accurate outcomes then those arrived at using analytical methods. This is especially important to the replanning phase of MFP as expert pilots must deal with a highly complex situation in a short time frame. It has been shown that aviation experts predominantly employ intuition based decision making. The major differences between the two extremes are summarised in Table 3.1. [97] It can be surmised that a human expert possesses domain specific knowledge and can rapidly adapt this to the situation at hand. However, extraction of expert knowledge from human experts is a non-trivial process because experts are not necessarily adept at expressing their own knowledge. Instead, it is better where possible to mirror the decision strategies that experts employ and these are discussed in the following sections [95].
3.2
Cognitive Models
In recent times, research in the descriptive approach to human cognition has focused on Naturalistic Decision Making (NDM) [102]. NDM is characterised by the observation that humans tend to find a satisficing solution (one that addresses relevant decision criteria) rather than an optimal solution [102]. Studies have shown that Recognition Primed Decision Making
3.2. COGNITIVE MODELS
55
(RPDM) provides a good model of expert pilot decision making [97]. Additionally, Rasmussen’s model of cognition has been used as a basis for an aviation Decision Support System (DSS) [103]. This section discusses these two cognitive models in NDM and how they can inform the design of an MFP framework.
3.2.1
Recognition Primed Decision Making
RPDM emerged from the study of real world ‘experts’. Effectively an intuitive form of diagnosis and prediction, there are three stages (or variants) to RPDM [97, 102]. The first stage, referred to as recognition, is where the expert employs pattern matching (intuition) to match actions directly to a given scenario. In scenarios where the situation is unclear, the decision maker moves to the second stage of RPDM known as serial evaluation. In this stage, the decision maker generates situational awareness by mentally simulating the events leading up to the current situation. Finally, the third stage of RPDM (simulation) is where the decision maker mentally simulates the future consequences of a chosen action (without necessarily considering a large number of alternatives). It has been observed that RPDM reflects the decision making process of expert pilots [97]. The observation is that experts choose from only a small set of alternatives and are constrained by time pressures, level of expertise, uncertainty and possibly ill-defined goals. Expert pilots employ pattern matching using experience honed cues (effectively a form of a priori knowledge) to structure the decision process [93, 97]. This corresponds to the first stage of RPDM and can be represented by a series of IF THEN rules. A shortcoming of RPDM is that meta-cognitive and analytic processes are not explicitly modeled. Expert pilots are aware of and control their own cognitive processes (known as meta-cognition), are capable of adapting their problem solving method and seek out more information to reduce uncertainty [94, 97].
3.2.2
Rasmussen’s Model
Rasmussen’s [104] three layer model of cognition provides a holistic framework for capturing all aspects of expert decision making (Fig. 3.2). In this
56
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
Figure 3.2: Rasmussen’s [104] three layer model.
model, the three levels (skills, rules and knowledge) correspond to increasing degrees of conscious control over cognitive activities. The skills level is where the human executes the task with virtually no conscious monitoring. This is the case for well practiced physical tasks like checking instruments as part of a pre-defined procedure. At the rules level, a series of IF THEN rules are enacted through pattern recognition. This level corresponds to the recognition stage of RPDM. It is observed that an expert often operates at the rules or skills level, where actions are executed quickly and effortlessly with little feedback [28, 93, 97, 104]. Finally, the knowledge level represents significant conscious control of cognitive activities (i.e. meta-cognition) to assess the situation, select and simulate an action and review the chosen action. This level of cognitive activity reflects the cognitive control of a novice or an expert faced with a novel situation [93, 97, 104]. Knowledge level cognition is an analytic process that is slow, arduous and requires considerable feedback. The implication of meta-cognition is that the human pilot possesses a means for managing his/her own expert behaviours along the three layers of Rasmussen’s model [104]. Pilots are capable of changing the importance of different rules, attributes and even the search cues depending on the overall situational awareness. It can be seen that Rasmussen’s model provides a suitable representation for modeling expert pilot decision making. There is also significant overlap between this model and RPDM especially with respect to the rules/recognition level and mental simulation at the knowledge level. The Cockpit ASsistant SYstem (CASSY) (an aviation decision support system) for commercial airliners is based on Rasmussen’s model of cognition [103].
3.2. COGNITIVE MODELS
57
Figure 3.3: 3T Intelligent Control Architecture [16].
It is interesting to note the similarities between this model and the standard three layer intelligent control architectural based on the seminal work of Bonasso et al. [16] (Fig. 3.3). For a comprehensive review of intelligent control architectures, refer [8]. The bottommost layer of 3T contains fast, real time reactive control loops which are similar to the sensorimotor patterns in the skills level. At the intermediate level of 3T, a series of predefined actions are sequenced for execution. There is some similarity between the sequencer and the rules level as actions are selected for execution according to some set of rules. Finally, the deliberative level in 3T, like the knowledge level, is made up of slow, deliberative processes. Like Rasmussen’s model, sensory input is escalated to higher levels if an appropriate action (i.e. decision outcome) can not be reached at a lower level. Therefore, it can be seen that the generic three layered intelligent control architecture is conceptually similar to Rasmussen’s model. This is perhaps a direct consequence of the nature of the human mind and the need for cognitive compatibility – humans can better design and maintain systems with a high level of cognitive compatibility [94, 95]. Typically, an MFP system resides in the topmost deliberative layer of an intelligent control architecture (including existing Unmanned Aerial Vehicle (UAV) architectures [8]) [16]. However, the planning task itself can be further analysed with respect to RPDM and Rasmussen’s model as the planning problem is in effect a sequential decision making problem [12]. Conceptually speaking, the imposition of constraints, such as vehicle dynamic constraints and risk limits correspond to skills level decision making. The evaluation function used to calculate path costs is rule based, reflecting the rules level
58
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
of Rasmussen’s model. Finally, the selection of an evaluation function and scheduling of planning activities (such as in an anytime framework [79]) mimics the knowledge level. The ramifications of RPDM and Rasmussen’s model on the UAV MFP framework are discussed in Section 4.2.2.
3.3
Cognitive Processes
The preceding sections identified the cognitive models that would be relevant to the MFP problem. This section examines the cognitive processes that are relevant to UAV MFP.
3.3.1
Heuristics
Heuristics refer to a category of cognitive processes that feature widely in NDM. The primary role of a heuristic is to reduce the search space and thus guide the decision maker onto a satisficing or possibly optimal solution in a short space of time. Some of these heuristics are directly related to the search strategy. For example, the satisfaction heuristic introduced by Simon stops the search process once a solution is found that satisfies some minimal criterion. Another example is the as-if heuristic which applies to the recognition phase of cognition and assumes that all cues are of equal importance. Thus, decisions are made based on the total number of data cues as opposed to the reliability or importance of the cues themselves. This saves computational cost. The adjustment and anchoring heuristic reduces computational cost with a combination of search space reduction and use of satisficing. An initial guess at the solution is used to seed the search process, which then adjusts the solution based on the situational awareness. The quality of the initial guess and degree of adjustment is crucial as a bad initial guess and insufficient adjustment can result in a highly suboptimal solution. [70, 102] A number of heuristics reduce computation time by using simplifying assumptions on the search problem which may or may not be true. For example, the representativeness heuristics is a judgment strategy where the decision maker estimates event probabilities based on its similarity to the parent population. This can result in ignorance of the basic laws of Bayesian probability (such is the case with the Gambler’s Fallacy) [70]. This is a
3.3. COGNITIVE PROCESSES
59
similar flaw for the availability heuristic where the ease with which an event can be recalled is associated with the probability of occurrence of the event. These heuristics, along with bias based heuristics are unsuitable for MFP as incorrect assumptions invalidate the optimality of the solution. It can be seen that the adjustment and anchoring heuristic is applicable to MFP as flight plans tend to follow the standard flight profile (Fig. 1.4). Additionally, due to the time pressures of online replanning, the satisficing heuristic could be used to quickly find a negotiable path rather than deliver an optimal solution that is late.
3.3.2
Articulation of Elementary Strategies
The use of elementary strategies to solve complex decision problems stems from the observation that a simple system can exhibit complex behaviour in a complex environment [105]. An elementary strategy is defined on a given problem representation; this representation is based on the selection of suitable attractiveness scales for each attribute. Attractiveness scales include numerical, ordinal, semi-ordinal (where a difference threshold is associated with each attribute), bipartite or difference scales. The representation also depends on the commensurability of the attributes, that is, whether attributes are pair-wise comparable or not. Finally, the representation also depends on the existence of lexicographic ordering. Note that in the ensuing discussion, an attribute is equivalent to a criterion. [95] A lexicographic ordering is where alternatives are ranked according to the most important attribute. If there is no unique ‘best’ solution, the alternatives are ranked again according to the second most important attribute and so on until a solution is found. This is known as the LEXicographic (LEX) rule [95]. When human pilots are under significant time pressures, they tend to employ this decision rule [93] as it allows the decision maker to quickly focus on the most critical aspects of flight (such as avoiding a collision) to find a satisficing solution. Another common elementary strategy is known as the DOMinance (DOM) rule which also employs a semi-ordinal scale with no commensurability and no lexicography. This strategy is concerned with finding the pareto-optimal set of alternatives [12]. An alternative A is chosen over A0 if A is more attractive than A0 on at least one attribute and not worst than A0 on all other
60
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
attributes [95]. If there are a large set of alternatives, then it can be timeconsuming to determine the pareto-optimal set; if this set is large, it is also difficult to choose the ‘best’ solution [106]. Thus, a dominance rule based strategy would be more suited as part of an analytical approach. The greatest difference in attractiveness rule is an elementary strategy that employs difference scales with commensurability but no lexicography. In this rule, the chosen alternative maximises the difference in attractiveness for each attribute; this maximises the difference in multi-dimensional solution space between the ideal (chosen) solution and other solutions (especially the nadir solution). Such an approach requires pair-wise comparison between individual solutions which can be computationally expensive [95]. The CONjunctive (CON) rule is where an alternative A is chosen if the attractiveness of A exceeds minimum attractiveness scores on all attributes. In comparison, the DISjunctive (DIS) rule is where A is selected if the attractiveness exceeds the minimum for only one attribute. Elimination By Aspects (EBA) is where alternatives which do not meet the minimum attractiveness score on the most important attribute are excluded. This process is repeated for the next most important attribute and so on until only one alternative remains. The Maximising Number of Attributes (MNA) rule selects A over A0 if A differs favourably from A0 on a greater number of attributes then the number on which A0 differs favourably from A. The Addition of Utilities (AU) rule is where the alternative with the greatest sum of weighted attractiveness values is chosen. In comparison, the Addition of Utility Differences (AUD) rule finds the sum of the difference of criterion values between A and A0 . If this sum is positive, A is selected, otherwise A0 is selected. The DOM, CON, DIS, LEX and EBA are non-compensatory decision rules, i.e. a poor attractiveness rating on one attribute cannot be compensated by more positive ratings on other attributes. On the other hand, MNA, AU, and AUD are compensatory rules [107]. Note that selection of a suitable rule is contingent on the search methodology. In Section 2.3, it was found that virtually all search algorithms use a single cost value in the search process. Determination of this cost value is a judgmental task (evaluate the cost/utility of an alternative) rather than a choice task (selection of an alternative) [101].
3.3. COGNITIVE PROCESSES
3.3.3
61
Decision Making as a Search for a Dominance Structure
Decision making as a search for a dominance structure refers to a class of cognitive processes that search for a problem representation (in terms of attributes, attractiveness scales and thresholds) such that one or more alternatives become dominant. The chosen problem representation (or structure) is dependent on the decision scenario, however, it has been shown that for human experts, this structure is ‘stabilised’ [95]. A seminal example of this type of decision making is the Search for a Dominance Structure (SDS) [107, 108] model. This is a four step process: 1. Pre-editing: the decision problem is simplified by selecting suitable alternatives and attributes to represent the decision situation. Elementary decision rules used in this step include CON, EBA. 2. Finding a promising alternative: under duress or high stakes, there may be strong directionality where the decision maker is heavily committed to one alternative. Decision rules used in this step include DIS, LEX and EBA. 3. Dominance testing: comparison between alternatives using DOM rule. 4. Dominance structuring: the decision maker tries to neutralise or counterbalance disadvantages for the promising alternative to make it dominant. Note that this form of dominance is not the same as pareto dominance. There are four major strategies: • De-emphasise the disadvantages by arguing that the probabilities are
low or can be controlled or avoided somehow. Decision rules used include LEX, AU and AUD.
• Bolster the advantages (which indirectly de-emphasises disadvantages) using decision rules such as DIS and CON.
• Cancellation operation - relate some advantage to disadvantages to cancel it using decision rules like MNA, AUD.
• Collapsing two or more attributes into single new attribute (AU decision rule).
The overall effect of the model is to produce a dominance strategy for problem representation such that application of dominance rules (or production rules [93]) produce a single solution. SDS is very similar to the Moving Basis Heuristic (MBH) of [100] except that SDS does not necessarily consider all attributes with regards to dominance. MBH was originally pre-
62
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
sented with a polynomial based choice process model. However, this type of cognitive process is difficult to utilise for UAV MFP because the output is a choice rather than a judgment.
3.4
Decision Making Under Uncertainty
The preceding sections have dealt with the basics of cognitive decision making theory without considering uncertainty. This section reviews the literature to identify strategies employed by human experts when presented with uncertainty. There are three major approaches to the treatment of uncertainty: Bayesian probabilities, pattern matching, and treating uncertainty as a problem solving task [94]. Perhaps the most common approach is to incorporate uncertainty using Bayesian probabilities. In this approach, the unknown probability of a hypothesis or event is expressed in terms of other probabilities that are more easily estimated and/or assessed. Therefore, the decision maker must be able to generate an exhaustive set of mutually exclusive hypotheses and numerically assess these hypotheses using available evidence. It has been shown that real-world decision makers consistently fall short of this probabilistic model (which is a normative model) [94]. Probabilistic handling of uncertainty has poor cognitive compatibility [94]. Cognitive compatibility is defined along three key criteria for a DSS. Firstly, the inputs that the DSS requires should be inputs which the decision maker has confident and precise intuitions. For probabilistic evaluations, this requires the human decision maker to have knowledge of the problem model (including numerical information) which requires unrealistic time and effort for numerical evaluation. Secondly, the operations applied by the DSS to the inputs needs to be understandable and plausible. Consider the case where two sources contradict each other (which is likely in multi-objective decision making). The decision maker does not want a statistical average, but instead wants to know why the sources disagree. Finally, the DSS needs to produce as outputs judgments or choices that the decision maker cannot confidently make, but which he/she requires. The shortcoming of Bayesian methods with respect to this criterion is that the outcomes of the hypotheses
3.4. DECISION MAKING UNDER UNCERTAINTY
63
are accepted or rejected - hence, there is no facility to handles degrees of knowledge underlying these conclusions. Another reason for the failure of real-world decision makers to embrace the probabilistic method is the existence of biases and heuristics [94]. Decision makers frequently make use of heuristics to deal with uncertainty. Studies have shown that experts commonly employ the Reduction, Assumption, Weighing, Forestalling, Suppressing (RAWFS) heuristic [109]: 1. Reducing uncertainty: the decision maker attempts to reduce uncertainty by seeking more information or through prioritisation of information. 2. Assumption based reasoning: use a priori knowledge and imagination to fill in the gaps or to comprehend factual information; these assumptions can be evaluated using mental simulation. 3. Weighing pros and cons: essentially a rough approximation of the subjective expected utility model. 4. Forestalling: prepare a course of action in anticipation of an undesirable outcome (such as worst case planning). 5. Suppressing uncertainty: ignore the uncertainty.
A discerning characteristic of experts is their ability to generate superior situational awareness through information gathering – this was found in both aviation [93] and fire fighting [109]. Experts were observed to step through the RAWFS heuristic in addressing uncertainty. However, the last two steps in RAWFS (forestalling and suppressing) were not observed in human expert fire fighters. Like pilots, fire fighters rely heavily on learned procedures, pattern matching (intuition) and mental simulation to achieve the goals of the mission [109]. Cohen and Freeman [94] propose an alternate model of decision making under uncertainty based on recognition/meta-cognition (R/M). The methodology comprises four components the first of which is the identification of evidence/conclusion pairs called arguments. This is performed through the use of cues and inferences based on these cues (which is very similar to the process of RPDM). The second step is the critiquing process where the evidence/conclusion pairs are scrutinised for incompleteness, unreliability and/or conflict. Following this is the correcting step which includes alteration, deletion and generation of new evidence/conclusion pairs. Finally, there is the quick test step which is effectively a meta-cognitive component
64
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
that regulates the critiquing and correcting steps. The decision to terminate a decision process is based on time constraints, cost of error and degree of uncertainty in the situation. This recognition/meta-cognition based model treats uncertainty as a problem solving exercise. It can be deduced that heuristics and meta-cognition play important roles in the handling of uncertainty. For the purposes of MFP, the RAWFS heuristic provides a structured approach to replicating the cognitive processes in human expert pilots. The reduction step could be replicated through the use of prioritisation and the assumption step through the use of schema. Additionally, a weighted sum could be used to replicate the weighting step.
3.5
Implementation of IF THEN Rules
This section provides a brief overview of two candidate methods for implementing IF THEN rules to emulate the recognition component of RPDM and the skills/rules levels of Rasmussen’s model. Fuzzy logic was selected as one candidate because it provides a direct method for encoding IF THEN rules using linguistic variables [110]. The second candidate, Multi-Attribute Utility Theory (MAUT) is used in existing multi-objective motion planning applications (refer Section 4.1).
3.5.1
Fuzzy Logic
First introduced in 1965 by Zadeh [111], fuzzy logic provides a method for mapping input space to output space in a non-linear, rule based fashion. Fuzzy logic is a form of soft (or approximate) computing that allows for better tolerance to imprecision through use of fuzzy sets which are also called fuzzy Membership Functions (MFs). It is possible to easily model complex systems with fuzzy logic as unlike Boolean logic based systems, there is no need to accurately define every precondition and every consequent [112]. Each point in the Universe of Discourse (UoD) (i.e. variable space) has a degree of membership to a fuzzy set on the interval (0, 1). This is unlike a Boolean crisp set where the membership is either 0 or 1 (Fig. 3.4). A fuzzy MF can also be used to model uncertainty as a possibility distribution [113].
3.5. IMPLEMENTATION OF IF THEN RULES
65
Figure 3.4: Fuzzy versus crisp set for a given Universe of Discourse (UoD).
Figure 3.5: Singleton fuzzification.
There are three stages in a fuzzy system, which evaluates disjoint IF THEN rules of the form: IF antecedent THEN consequent In the first stage, entitled fuzzification, a crisp input is matched onto antecedent MFs [114]. By far the most popular method is singleton fuzzification where the degree of membership for a given input value is found by finding the intersection point between a singleton MF at that value and the antecedent MF. This is shown in Fig. 3.5. A series of rules are then used to infer the output given the ‘fuzzified’ input values. For a Multiple Input Single Output (MISO) fuzzy system, the rules are phrased as: IF X1 is A1 AND . . . AND Xp is Ap THEN Y is Bq where Xp is the pth input variable, Ap is the corresponding input MF, Y is the output variable and Bq the q th output MF. Mathematical evaluation of a fuzzy rule base was first described using Zadeh’s compositional rule of inference [110]: B 0 = A0 ◦ R (3.1)
66
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
The first step in evaluating equation (3.1) is to extend the fuzzified input variable A0 from the input UoD X to the Cartesian product of the input and output UoD XY . This results in µA00 , which is then intersected with a fuzzy rule (or relation) to get µA00 (x, y) ∧ µR (x, y) where ∧ is a t-norm (or conjunctive) operator. Note that each relation is itself a conjunction of an input MF and an output MF. The result is projected onto the output UoD by taking the supremum of the intersection: µB 0 (y) = sup {µA00 (x, y) ∧ µR (x, y)}
(3.2)
x∈X
Consider the scenario where there are three relations R1 , R2 and R3 where R1 and R2 are tightly coupled, but R3 is independent. In this case, the resultant rule is R = (R1 ∧ R2 ) ∨ R3 where ∨ is the t-conorm or s-norm or disjunction operator. Example t-norm operators include the min and product operators, and example s-norm operators include the max and addition operators. A common method for inferencing that is based on the compositional rule of inference is Mamdani inferencing [114]. In Mamdani inferencing, the implicated output MF B 0 on UoD y is derived using 0 B (y) = sup min A (x), max [min (µAi (x), µBi (y))] 0
i=1,...,p
(3.3)
where x is the input UoD and µA and µB are the input and output MFs respectively. Note that Mamdani inferencing uses min as the t-norm and max as the s-norm operators (compare with equation (3.2)). The final stage, entitled defuzzification, converts the implicated output MFs into a crisp value. A commonly used method for defuzzification is the Centre of Gravity (CoG), which is calculated as shown [110] Z
y= Z
yB 0 (y) dy (3.4) 0
B (y) dy
A linguistic label can be associated with each MF. For example, IF the temperature is cold, THEN the heat is high. In addition, as each MF can be treated as a possibility distribution, fuzzy logic inherently provides a model for uncertainty. Uncertainty in the input can be modeled with non-singleton
3.5. IMPLEMENTATION OF IF THEN RULES
67
fuzzification [115, 116]. Furthermore, uncertainty in the rule base and in the definition of the possibility functions can be modeled with type 2 MFs [117]. The primary shortcoming of fuzzy logic is the computational cost, which is especially high for type 2 fuzzy systems. Without hardware acceleration (e.g. [118]), this can undermine the usability of fuzzy logic in MFP. Additionally, there are no standardised guidelines for the design of MFs and rules [114].
3.5.2
Multi-Attribute Utility Theory
Multi-Attribute Utility Theory (MAUT) is a methodology for modeling decision maker preferences based on binary relations between alternatives. For a given pair of alternatives X and X 0 , if the decision maker prefers X to X 0 (i.e. X X 0 ), then the overall utility value U (X) ≥ U (X 0 ). The output of a MAUT based system is this utility value U . Hence, MAUT can be used to implement a judgmental decision task. Note that a utility (assumed to be on (0, 1)) can be converted to a cost by subtracting the utility from one. There are also three steps in the implementation of MAUT [119]. The first step in MAUT is to construct value (or utility) functions for each attribute. Because attributes are typically incommensurate (e.g. 2L of fuel is not comparable to 2 fatalities per flight hour of risk), a value function ui is needed to map each attribute onto a commensurate scale. Attributes are also referred to as decision criteria [119]. A number of methods can be used to guide the design of value functions, such as the popular Measuring Attractiveness by Categorical Based Evaluation Technique (MACBETH) package [120]. The second step in MAUT is the selection of an aggregation function F and learning the parameters for F from the human expert. The overall utility is found by aggregating the utility values for each attribute xi U (X) := F (u1 (x1 ), . . . , un (xn ))
(3.5)
Selection of U depends on the Multi-Criteria Decision Making (MCDM) problem at hand. If there is mutual preferential independence among the decision criteria, then U is the classical additive model, i.e. the weighted sum U (x) :=
n X i=1
wi ui (xi ), ∀x = (x1 , . . . , xn ) ∈ X
(3.6)
68
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
In this scenario, it is only necessary to learn n weights wi from user preferences. Note that the cardinality of objects (i.e. number of alternatives) used to elicit expert preferences is usually small (less than 20) [121]. Therefore, this set of objects should cover as much of the decision space as possible. The weightings for a weighted sum can also be obtained from MACBETH. If however, there is dependence among the decision criteria, then the Choquet integral [122] is needed. A Choquet integral of an alternative X, represented by the vector (x1 , . . . , xn ) with respect to a capacity µ (equivalent of a ‘weight’) is defined by Cµ (x) =
n X i=1
xσ(i) µ Aσ(i) − µ Aσ(i+1)
(3.7)
where σ is a permutation on the attributes such that xσ(1) ≤ . . . ≤ xσ(n) . Additionally, Aσ(i) := {σ(i), . . . , σ(n)} for all i ∈ {1, . . . , n} and Aσ(n+1) := ∅. Interaction between attributes is captured in the capacities, for example µ(1) is the ‘weight’ for attribute 1 by itself, and µ(1, 2) denotes the ‘weight’ of attributes 1 and 2 combined. Where there is interaction between at most k attributes, the capacity is known to be k-additive. Thus, the weighted sum is the case where k = 1. From equation (3.7), it can be seen that there is significantly more computation involved due to the need to sort attributes and manipulate a set of capacities for each attribute. Furthermore, the set of capacities µ can be very large. For a k-additive capacity (k ≤ n), the capacity is completely defined with m coefficients where ! k X n m= (3.8) l l=1 Learning these capacities from expert preferences is known as capacity identification. A review of methods for capacity identification is provided in [119].
3.6
Findings
This section summarises the key findings on expert pilot decision making, highlighting relevant cognitive models and strategies and identifying candidate MCDM algorithms for replicating them.
3.6. FINDINGS
69
A number of cognitive models in the field of NDM were identified that could describe the decision making process of expert pilots [70]. Experts have bounded rationality and consider a small set of relevant attributes and alternatives to find a satisficing solution. Studies have shown that expert pilots predominantly employ a RPDM process with extensive use of intuition and pattern matching. This can be modeled with IF THEN rules in the framework of Rasmussen’s skills, rules and knowledge model. Rasmuseen’s model captures the pattern matching (intuitive), mental simulation (analytical) and meta-cognitive aspects of expert decision making. Such an approach was adopted in the development of the CASSY [103] DSS. MAUT and fuzzy logic were identified as candidate methods for implementing IF THEN rules as part of the rules level of Rasmussen’s model (or first stage of RPDM). The advantage of MAUT over fuzzy logic is the existence of standard methods/guidelines (e.g. MACBETH and capacity identification methods [119]) for eliciting expert preferences. In the case of a weighted sum, a MAUT based approach is also more computationally efficient. However, in order to model general decision preferences (without independence among criteria), the computationally intensive Choquet integral is required. This shortcoming can be partially addressed with a k-additive capacity where k is small. In addition to the cognitive model, a number of cognitive processes were identified that could aid in reducing the complexity of MFP. For complex planning and scheduling problems, experts employ heuristics to introduce simplifying hypotheses, prune the search space or start of with a ‘reasonable’ initial solution [96]. Heuristics that are pertinent to MFP were found to be the as-if (equal weighting), adjustment and anchoring and satisficing heuristic. Additionally, the RAWFS heuristic was found to be a more accurate model of how human experts deal with uncertainty than the Bayesian approach. Even though forestalling and suppression were not observed in expert firefighters (and are not applicable to the MFP task), the first three steps (reduction, assumption and weighing) can inform the design of an MFP framework.
70
CHAPTER 3. REVIEW OF EXPERT PILOT DECISION MAKING
Finally, a number of elementary decision strategies were identified. As shown in Section 3.3.3, such strategies can be used to implement more complex decision models such as SDS. Even though it is desirable to obtain the ‘optimal’ solution, it is difficult to define metrics for optimality for a command and control problem like MFP [97, 123]. This is due to the dynamic nature of the operating environment and the embedding of the decision task within overarching tasks and goals. For online replanning, a solution that is ‘good enough’, robust and delivered on time is preferable to one that requires long computation time – especially if changes to planning assumptions invalidate the ‘optimal’ solution [123].
Chapter 4 Mission Flight Planning Design Approach How many legs does a dog have if you call the tail a leg? Four. Calling a tail a leg doesn’t make it a leg. Abraham Lincoln
T
HIS chapter revisits the research objectives in light of the literature review outcomes (on motion planning and expert cognition) and adds to this a review of pertinent vehicle motion planning methods. From these research findings, a methodology for the design and evaluation of a Mission Flight Planning (MFP) framework is proposed.
4.1
Review of Autonomous Vehicle Motion Planning
Much of the previous work for vehicle planning has focused on techniques in computational geometry using a grid as described in Section 2.2.1 [14,15,21, 30,31,39,41,42,75,77,92,124]. However, a shortcoming of many grid-based approaches (e.g. [30,39,47,92,124]) is that the resultant path is confined to track angles that are multiples of 45◦ . As a result, the path can be suboptimal and may contain spurious turns [14, 15]. A lack of regular high resolution track angles also affects methods based on Voronoi graphs (e.g. [42, 44, 125, 126]), methods that use probabilistic sampling (e.g. [32,43,45]), and generally methods where path angles are not considered (e.g. [9,29,88]). A review of vehicle motion planning algorithms is provided below for methods that address the 71
72 CHAPTER 4. MISSION FLIGHT PLANNING DESIGN APPROACH requirements of Unmanned Aerial Vehicle (UAV) MFP vis a vis the track angle problem, wind/current effects and multi-objective optimisation.
4.1.1
Methods with High Resolution Track Angles
A number of grid based methods determine the track angle in continuous space (effectively infinite resolution) instead of sampling from predefined track angles. Field D* [14] and 3D Field D* [31] for example interpolate the costs of adjacent cells to determine the track angle. This allows for arbitrary track angles but assumes that cell costs are known a priori. For vehicle motion planning, this approach is not feasible given a 4D search space (three spatial and one time dimension) as the cost is dependent on the track angle. It is shown, however, that a multi-resolution grid greatly reduces memory usage and computation time. Theta* [15] works in a similar vein but unlike Field D*, a child or successor cell is not necessarily adjacent to the parent/source cell. Instead, the parent is chosen as either the start cell or a cell as close as possible to the start cell such that the resultant trajectory does not intersect any obstacles. Even though the method enables continuous angle resolution, it clearly does not find a path that optimises for the weighted region problem [13]. A similar method to Theta*, named A3D is presented by Kim et al. [41] for UAV trajectory planning. A3D also uses straight line projections from the current cell to the goal cell in constructing a path. A shortcoming of this methodology is the tendency to find paths that follow the boundaries of obstacles or threat zones. Like Theta*, A3D does not find the least cost path - rather it finds a path that does not exceed given risk thresholds. Pivtoraiko and Kelly [75] present an alternate method that provides regular, high resolution track angles by defining a successor operator with predetermined successors at selected track angles. Like Theta*, parent cells are not necessarily adjacent to child cells, hence the notion of a vector neighbourhood [12]. This is shown in Fig. 4.1. Note that greater track angle resolution requires tracks of greater length as the successor cell is further away on the grid. Unfortunately, this method is formulated for 2D vehicle planning with no consideration for winds/currents and is focused on satisfying vehicle dynamic constraints (i.e. trajectory planning [12] instead of path planning).
4.1. REVIEW OF AUTONOMOUS VEHICLE MOTION PLANNING 73
(a)
(b)
Figure 4.1: Vector neighbourhood based successor operator used by [75], (a) shows the discretisation of track angles and (b) the full successor operator with smooth track angle transitions. Track angles are at 0◦ , 26.6◦ , 45◦ .
Figure 4.2: Lattice structure of Yahja et al. [77].
Like [75], Yahja et al. [77] also present a method with pre-defined, discrete track angles. This method employs framed quadtrees and octrees for 2D and 3D planning respectively. Nodes are placed on the boundaries of each quad/octree decomposed cell without increasing the overall resolution of the search space (refer Fig. 4.2). This memory efficient sampling scheme enables high resolution track angles when traversing from one border of a cell to another. Unfortunately, transitions between adjacent border cells are again limited to increments of 45◦ . Additionally, [77] does not consider wind effects.
74 CHAPTER 4. MISSION FLIGHT PLANNING DESIGN APPROACH Another method for modeling tracks at different angles is to use polygon shaped cells [127]. For example, a hexagon enables 60◦ track angle increments. However, it is non-trivial to convert a square shaped raster map (such as a Geographic Information System (GIS) map) into arbitrary polygon shaped cells. Note that even though [127] is presented for UAV motion planning in the presence of probabilistic threat zones, track angles are not considered at all. A number of existing methods for incorporating wind/current effects use a constant weighting value for polygonal (or polyhedral) shaped regions of constant wind/current [128–131]. These methods exploit the geometric properties of such a representation to reduce the number of nodes in the search space [128,129]. However, these methods do not consider multiple objectives nor a dynamic environment. Furthermore, it is not possible in general to guarantee regions of constant weight given multiple objectives. Another method for incorporating wind/current effects in the planning process is presented by Petres et al. [124] for Autonomous Underwater Vehicle (AUV) motion planning. The algorithm is a heuristic directed method based on a fast marching algorithm (a search methodology that assumes constant traversal cost between nodes). This assumption does not hold for a multiobjective weighted region problem. In the UAV field, Rubio and Kragelund [21] incorporate wind effects to optimise fuel consumption for long distance flights. They combine linearly interpolated grid based wind forecasts with a planner based on artificial evolution. Uncertainty in the wind forecast is mitigated by online replanning. The presented method, however, does not consider other decision objectives such as flight time and the rules of the air. There are a number of planners based on artificial evolution (e.g. [21, 32, 45]) that find an obstacle free path (or in the probabilistic case, one that does not exceed risk thresholds) whilst incorporating vehicle dynamic constraints. With the exception of [21], these planners do not incorporate wind/current effects. Since the search space is continuous, the track angle resolution is infinite. The primary shortcoming for these evolutionary methods, however, is the inability to specify bounds on computation time or solution quality [132]. This can be problematic for online replanning (due to the time constraints imposed on planning) and for applications where determinism is a regulatory requirement (e.g. DO-178B [27] for aviation software).
4.1. REVIEW OF AUTONOMOUS VEHICLE MOTION PLANNING 75
4.1.2
Multi-Objective Planning Algorithms
None of the previously quoted methods explicitly address the requirements of multi-objective planning though many incorporate multiple motion constraints (e.g. water currents and vehicle dynamics in [124]). Examples of explicit multi-objective planning algorithms can be found in the study of HAZardous MATerials (HAZMAT) transportation due to the need for fuel efficient and low risk (presented to the population) paths [106]. These algorithms combine a multi-objective decision function (typically a weighted sum) with a graph search algorithm (such as A* or Dijkstra’s algorithm) on a grid [106, 133–135] (refer [115] for a description). This methodology is also used by Gu [42] for a bi-objective (risk and fuel objectives) UAV motion planner based on Voronoi diagrams. Unfortunately, the plans generated by [42] tend to be globally suboptimal as the path follows Voronoi lines. Additionally, the aforementioned HAZMAT planning methods are confined to 2-dimensional ground vehicles and none incorporate the effects of wind. One example of HAZMAT planning that is of particular interest is described in [133]. This algorithm models graph edges with a weight that is associated with a probability distribution. Planning is done by calculating the expected value of the weights through use of quadratic utility functions. By using quadratic utilities, it is possible to do planning based on only the mean and variances of the Probability Density Function (PDF). The algorithm returns an optimal path that is of the least expected cost; however, it is of of exponential complexity (in terms of the number of nodes). This makes it infeasible for UAV MFP. A related algorithm for linear or exponential utility functions is also presented that is of O(V 2 ) complexity but it can not handle multiple objectives [133]. Nembhard and White [136] present another approach based on Multi-Attribute Utility Theory (MAUT) but the algorithm is limited to acyclic graphs. These multi-objective planning algorithms (e.g. [42,106,133–135]) almost universally adopt a global planning methodology where the track cost is calculated online much like a lazy probabilistic roadmap [12]. Furthermore, the roadmap itself (comprising nodes and the tracks/trajectories that link nodes) can be represented implicitly to obtain an even lazier algorithm [29]. This is necessary because it is computationally expensive to compute a priori
76 CHAPTER 4. MISSION FLIGHT PLANNING DESIGN APPROACH all possible trajectory costs for all nodes in the search space especially when given a complex multi-objective decision function. Another example of multi-objective planning is presented by Soltani and Fernando [137] for construction site path planning. Fuzzy sets were used to transform input values onto a commensurate scale but the transformed values are aggregated using weighted sum aggregation. Dijkstra’s algorithm is then used to find a least cost path based on this aggregated cost. This approach is therefore effectively no different from that used in HAZMAT planners discussed previously. An alternative approach to multi-objective path planning is to use a multi-objective search algorithm like [125, 138]. However, these algorithms are computationally expensive and in the case of [138], restricted to acyclic graphs; note that graphs derived from grids are cyclic. A similar, direct approach to multi-objective path planning is logic based planning. Two candidate approaches include the Hierarchical Task Network (HTN) [139] and Temporal Action Logic (TAL) [140]. An application of HTN to indoor robot navigation is described in [141]. However, logic based motion planning is generally computationally expensive and the resultant plan is typically non-optimal [12, 139]. The Incremental Search Engine (ISE) [39] is a grid based, multi-objective heuristic search algorithm based on D* Lite [30]. Tompkins et al. [39] classify decision variables as either independent or dependent. An independent variable such as terrain elevation has the same value for a given node regardless of the path chosen. On the other hand, a dependent variable changes in value depending on the path chosen. Fuel is such an example in a 2D search. ISE returns results identical to A* but, like D* Lite, is more efficient in replanning. Tompkins, Stentz et al. [39] present an application of ISE for the Mars rover using a four dimensional search (two spatial, one time and one ‘energy’, i.e. fuel dimension). There are three decision objectives: minimisation of distance travelled, energy minimisation and arrival within specified time bounds. The energy and time dimensions are combined into a single cost function through aggregation to reduce computation. ISE also uses heuristic inflation (refer Section 2.3.2) to help reduce computation time. Total planning time is approximately 60 seconds with a search rate of 29000 states per
4.1. REVIEW OF AUTONOMOUS VEHICLE MOTION PLANNING 77 second [39]. Unfortunately, the path planned by ISE is constrained to track angles in increments of 45◦ and does not consider uncertainty. Snorrason and Norris [142] present a motion planner for the Mars rover based on computational morphology and A*. Even though Snorrason and Norris label their method as a form of depth first A*, their method is actually a form of anytime A* (refer Section 2.3.2) as they reduce computation time by inflating the heuristic. They assume that the search space is mostly obstacle free (true for MFP) and inherently assume no local minima. An approach to multi-objective route planning for a land vehicle is presented by Suzuki, Araki et al. [92]. This algorithm is based on a GIS raster map of the environment that is discretised to a resolution of 100m in the x and y and 250m in the z dimension. The planner considers as decision criteria obstacles, roads, (ground) slope and rainfall. Like the previously discussed multi-objective planners, this one also aggregates decision variables into a single cost value which is used in a heuristic search algorithm. However, Suzuki et al. [92] describe a novel method for calculating the traversal cost. For each input variable, singleton fuzzification (refer Section 3.5.1) is used to find the degree of membership to each input Membership Function (MF). The rain criterion for example is represented by the MFs light rain, moderate rain and heavy rain. IF THEN rules are then used to implicate the degree of membership to the output MF on the mobility (i.e. difficulty of traversal) Universe of Discourse (UoD). Unlike a conventional fuzzy system, the degree of membership to the input MF is translated directly to the output MF. Once this is done, Depmster-Shafer’s rule of combination is used to aggregate the outcomes of each rule. In the final step, entitled ‘data interpretation’, a belief interval is calculated from the aggregated result. The lower bound on the interval is used as the cost value in a heuristic search. Even though Dempster-Shafer theory is used for rule aggregation, this method does not incorporate uncertainty as the input values are crisp. Additionally, the track angles are constrained by the grid. Pfeiffer, Battas et al. [88] describe a method for bi-criteria UAV flight planning in the presence of probabilistic threat zones. The planned path minimises the risk of detection and minimisation of flight time through the
78 CHAPTER 4. MISSION FLIGHT PLANNING DESIGN APPROACH pareto-optimal definition of optimality. However, this approach is developed for pre-flight planning and is inherently unsuited to online replanning.
4.1.3
Findings
It can be seen that existing methods can not fully address the multi-objective vehicle motion planning problem. None of the work described previously even consider dynamic obstacles and the effects of different vehicle velocities. However, there are a few points to note. Firstly, it was found that a ‘lazy PRM’ based global planning approach is needed due to the intractability of computing trajectory costs a priori between all parent and child nodes. Secondly, all of the deterministic planning algorithms described above are based on a discrete search space. This is logical as wind data and obstacle maps are often formatted using a grid structure [21]. Thirdly, a vector neighbourhood was identified as a suitable method for overcoming the limited track angle resolution of a grid based search space [75]. It is desirable to have high track angle resolution to ensure path optimality [14], especially given unpredictable wind/current vectors. Fourthly, uncertainty is often mitigated with online replanning. Finally, given the complexity of planning [12], a multi-resolution search space can be used to increase memory efficiency and decrease computation time [14, 31, 77].
4.2
Synthesised Approach
The primary research objective is to develop an MFP framework that can incorporate multiple objectives with uncertain information in a computationally efficient manner. This section highlights the key findings of the two major facets of an automated MFP system, namely expert cognition and motion planning. These findings are used to synthesise a suitable design approach for the proposed MFP framework and motion planning algorithm.
4.2.1
Motion Planning
The general motion planning problem was found to be PSPACE hard in the lower bound and exponential in the upper bound. This complexity is further compounded by the presence of multiple decision objectives in MFP
4.2. SYNTHESISED APPROACH
79
which results in a weighted region motion planning problem. A near-optimal solution needs to be adopted to meet the requirements of online replanning. A lazy Probabilistic RoadMap (PRM) based global planning framework is suited to MFP. This approach is widely adopted in HAZMAT route planning and vehicle motion planning (Section 4.1). Typically, the search is conducted on a (Sukharev) grid as the environment is represented in a bitmap format (which is the case for GIS data and sensor occupancy grids). A multiresolution lattice variant of the grid was identified as a promising approach to achieving memory and time efficiency and high resolution track angles (Section 2.2.1 and 4.1). Local planning techniques as described in Section 2.2.3 can be used to aid the global planning process for joining start and goal nodes to the roadmap. For the query phase of the PRM methodology, it was revealed that the A* and Anytime Replanning A* (ARA*) algorithms were suited to MFP. Based on dynamic programming, A* finds the least cost path in optimal time in that an equally informed algorithm can not find a solution in less time than A*. This performance is not compromised by the presence of a dynamic environment where there can be large changes to the roadmap. A* has been shown to be efficient in a wide variety of planning applications (Section 4.1) due to the use of a heuristic. Therefore, an MFP motion planning algorithm should be based on A* and incorporate the use of heuristic inflation as per ARA* for online replanning. An alternative search method for the query phase is fuzzy dynamic programming. The advantage of fuzzy dynamic programming (compared to A* which is based on conventional dynamic programming) is that the solution is a logical conjunction of individual decision outcomes. In A*, a particularly costly choice can be mitigated by successive lower cost choices such that the final path is still of least cost since the additive operator is a disjunctive operator [143]. This problem is resolved by using a conjunctive t-norm operator. However, existing methods for solving the fuzzy dynamic programming are intractable for UAV MFP. The vast majority of existing vehicle motion planning algorithms do not consider multiple decision objectives even though many consider multiple dynamic constraints. A number of examples of multi-objective path planning were found in the field of HAZMAT. These algorithms aggregate the deci-
80 CHAPTER 4. MISSION FLIGHT PLANNING DESIGN APPROACH sion variables (not necessarily with equal weighting) into a single cost value. Candidate methods for doing so included the weighted sum, fuzzy logic and the Choquet integral. Such an aggregation based approach is almost universally adopted in existing multi-objective planning algorithms due to the exponential increase in memory and computation time with dimensionality. Finally, a survey of the literature revealed the difficulty of incorporating uncertainty into the planning process. Existing methods are computationally inefficient (Section 2.4) and often require assumptions of probability distributions. An alternative approach is proposed where uncertainty is mitigated through online replanning, multi-objective decision making and incorporation of tolerance into the planned path. This reflects to a degree the cognitive strategies observed in experts when dealing with uncertainty. As shown in Section 2 and 4.1, none of the existing work in generic, vehicle or UAV motion planning adequately address the simultaneous requirements of multi-objective decision making, uncertainty and computational efficiency for UAV MFP.
4.2.2
Expert Cognition
MFP is a task that is presently performed with proficiency by expert pilots. Replicating the cognitive strategies of expert pilots can help address the complexity problem of MFP and help to create a framework with higher cognitive compatibility. A review of the literature on expert pilot decision making revealed that expert cognition can be modeled using Recognition Primed Decision Making (RPDM) and Rasmussen’s [104] three layered model. Experts have bounded rationality in that they identify key attributes in a small set of alternatives to find the best solution. They also make extensive use of intuition, effectively a form of pattern matching based on IF THEN rules at the rules level of Rasmussen’s model (or stage one of RPDM). Rasmussen’s model captures fast, intuition based decision making (skills and rules levels) and slower, analytic decision making (knowledge level). The Cockpit ASsistant SYstem (CASSY) Decision Support System (DSS) is based on this model. It is proposed that the MFP task can also be analysed in the context of Rasmussen’s model (refer Fig. 4.3).
4.2. SYNTHESISED APPROACH
81
MFP Decision Making Identification
Decision of task realisation
Scheduling
Planning mode f(s),c(s) Recognition
Task / state association
Memorised rules
Decision variables, e.g. fuel, weather, terrain, risk} Discriminating Features Constraints, e.g. max risk, max climb rate
Planning variables, mission parameters
Memorised (sensorimotor) patterns
Sequence of waypoints (x,y,z,t)
Figure 4.3: MFP task in the context of Rasmussen’s model [104].
In the context of MFP, each attribute (equivalently referred to as a decision criterion) has associated with it a constraint and ideal value. For example, there is an ideal (most fuel efficient) cruise velocity and a maximum and minimum airspeed limit. Constraints can be enforced with simple boundary checking and correspond to the skills level of cognition (Fig. 4.3). Optimisation of the selected path, however, requires evaluation of multiple decision rules as per the rules level. The scheduling of planning activities, specification of cost function c and heuristic function h (resulting in a new evaluation function f , refer Section 2.3.2) corresponds to knowledge level cognition. For example, in an anytime planning framework, multiple planning instances are executed with a decreasing heuristic inflation factor . This is similar to the adjustment and anchoring heuristic as a large inflation value for a distance based heuristic biases the solution to a straight line profile (shortest path) [79]. Heuristics play an important role in achieving computational tractability. In addition to the aforementioned adjustment and anchoring heuristic, other useful heuristics include the as-if heuristic and the satisficing heuristic. Satisficing (returning the first solution that meets path constraints) is especially important for online replanning. In addition, the Reduction, Assumption,
82 CHAPTER 4. MISSION FLIGHT PLANNING DESIGN APPROACH Weighing, Forestalling, Suppressing (RAWFS) heuristic was found to provide a viable methodology with high cognitive compatibility for addressing uncertainty. It can be seen that a heuristic search algorithm is suited to implementing expert cognition due to the incorporation of a heuristic term in the evaluation function (refer Fig. 2.9). Furthermore, it was shown in the literature (refer Section 4.1.2) that a computationally tractable solution can be obtained by combining a heuristic search with a Multi-Criteria Decision Making (MCDM) cost function. MAUT and fuzzy logic were identified to be suitable candidates for implementing such a cost function as the calculation of a cost is a judgmental task. A MAUT based approach is especially suited given the computational efficiency of a k-additive Choquet integral (where k is small) and established knowledge elicitation strategies. In comparison, a fuzzy approach is significantly more computationally expensive.
4.2.3
Summary of Findings and Synthesised Approach
Based on the research findings described above, a summary of the chosen approach to UAV MFP is listed in Table 4.1. This table also contains links to the sections where each design approach was reviewed. It can be seen that the design choices address the primary research objective of multi-objective MFP under information uncertainty in a computationally efficient manner. In addition, a number of the research questions in Section 1.3 are answered. The first research question regarding the feasibility of automating UAV MFP is addressed through the literature review. MFP is a motion planning task currently performed proficiently by expert pilots. Existing multi-objective motion planning algorithms can address some aspects of this task, but not all aspects simultaneously (i.e. multi-objective decision making and uncertainty and efficiency). The optimality of a solution as per research question two can be defined as a least cost path using the dynamic programming principle. This can be verified through theoretical proof. The third research question regarding evaluation of multiple decision objectives under uncertainty can be addressed through an understanding of expert cognition. Experts employ IF THEN rules and heuristics (as described in RPDM and Rasmussen’s model [104]) to evaluate multiple objectives and reach a satisficing solution. This can be replicated using MAUT and heuristics such as
4.2. SYNTHESISED APPROACH
83
Table 4.1: Chosen design approach for UAV MFP. Section Design Choice
Justification
Reviewed
Use global, sampling based planning
Globally optimal solution
2
Lattice based sampling on a grid
Enable regular arbitrary resolution track angles
2.2.1, 4.1
Use lazy PRM methodology
Computational efficiency
2.2.4
Base framework on Rasmussen’s model
Mimic expert pilot decision making as they are proficient, provide cognitive compatibility
3.2
Use A*
Efficient search even with large dynamic changes
2.3
Use MAUT to calculate aggregated cost
Computationally efficient method for incorporating multiple objectives
3.5.2
Use heuristic inflation for online replanning
Reduce computation time, mimic adjustment and anchoring heuristic and satisficing
2.3, 4.2
Consider fuzzy dynamic programming
Enables logical conjunction
2.3.1
Use tolerance, online replanning and RAWFS to handle uncertainty
Computational tractability cognitive compatibility
and
2.4, 3.4
adjustment and anchoring and RAWFS to find a solution that satisfies multiple objectives under uncertainty. Finally, the fourth research question can be resolved as an MFP system sits in the deliberative layer in an intelligent control architecture.
Chapter 5 Mission Flight Planning Framework The Russians will never be able to get their missiles through the dense protective layer of delayed flights circling over the United States in complex, puke-inducing holding patterns Dave Barry
A
framework for Unmanned Aerial Vehicle (UAV) Mission Flight Planning (MFP) is presented with a view to providing automated MFP capability onboard a UAV. The framework defines the interfaces for a motion planning algorithm capable of planning in 4D (three spatial and time dimension). It is necessary to plan in four dimensions to ensure completeness due to the presence of a dynamic environment (e.g. changing wind conditions) and dynamic obstacles. Note that the proposed motion planning algorithm, Multi-Step A* (MSA*) is presented as a generic 4D vehicle motion planning algorithm to ensure that it is extensible to a wide variety of UAV applications. Thus, even though the research is presented for UAV MFP, the motion planning algorithm itself is extensible to a wide range of 4D vehicle motion planning problems such as Autonomous Underwater Vehicle (AUV) motion planning. This section presents MSA*, followed by an application of MSA* to 4D multi-resolution lattice structures. The resultant framework is then incorporated into a self scheduling replanning architecture for in-flight UAV MFP.
84
5.1. MULTI-STEP A* (MSA*)
5.1
85
Multi-Step A* (MSA*)
The planning task is defined as finding a path P through a roadmap S starting at node s0 and terminating at node sG . Each node s ∈ S is located at the centre of a 4D rectangloid cell defined in the world space W (x, y, z, t). Assuming a regular grid sampling of the search space, each node s maps uniquely to a cell in W . Thus s refers simultaneously to both the cell and the node located in the centre of each cell. The global planning approach described in Section 4.1 is adopted whereby trajectory segments are evaluated online. Furthermore, the roadmap is defined implicitly through a successor (or neighbourhood) operator Γ where for a given source (or parent) node s, Γ(s) denotes a set of cell sequences γs0 ∈ Γ(s) which begin at s and terminate at the successor (or child) node s0 ∈ S 0 . Consider the modeling of a vector neighbourhood like [75] where s0 does not necessarily lie adjacent to s. The successor operator is assumed to denote a linear trajectory segment (or track) connecting the centre of cell s to the centre of cell s0 (refer Fig. 5.1). It is assumed that turns (possibly required between tracks) have negligible impact on the overall path in terms of flight time and fuel consumption. For each successor s0 , the trajectory intersects a sequence of cells between s and s0 γs0 = {sj+m−1 = s0 ksj = s, . . . , sj+m−2 }
(5.1)
where sj , . . . , sj+m−1 are a sequence of m cells and k denotes a conditional dependence, i.e. sj+m−1 ksj , . . . , sj+m−2 is interpreted as cell sj+m−1 via a sequence of cells sj , . . . , sj+m−2 . In the ensuing sections, Γ is derived for a three dimensional world (Γ3D ∈ Γ, Section 5.1.1) and then extended to four dimensions (Section 5.1.2). This is possible as the search dimensions are orthogonal.
5.1.1
Multi-Step 3D Successor Operator
In Fig. 5.1b, it is possible to determine the horizontal track angle θ and slope from the endpoints of each trajectory segment. The vertical track angle φ and slope can be similarly determined for a 3D trajectory segment. In aviation, the horizontal and vertical track angles are referred to as the ground
86
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK Successor Cell Trajectory Segment
y3=ny y2
Cell Sequence
y1 y0=0 x0=0 x1=nx Source Cell
(a)
(b)
Figure 5.1: Successor operator illustration, (a) a single 2D trajectory segment and the corresponding cell sequence, and (b) an example 2D successor operator showing individual trajectory segments. Note that the trajectory segment terminates in the centre of the successor cell, which is not necessarily adjacent to the source cell.
track angle and flight path angle respectively. Note that the vertical slope can be used to calculate the climb/descent rate. These angles can then be used to determine the cells that intersect the track using a variant of Bresenham’s pixel drawing algorithm [144]. Note, because the vehicle controller has non-zero trajectory tracking error, it is necessary to include cells whose edges/corners touch the trajectory line. This prevents the possibility of the vehicle squeezing through an infinitesimally small gap or “brushing past” an obstacle. Doing so provides an intrinsic tolerance to navigational and controller uncertainty as there is always a safety margin between the trajectory segment and the boundaries of a cell. Using Bresenham’s [144] line drawing concept, a 3D cell sequence is determined based on the displacement (nx , ny , nz ) of the successor cell from the source in terms of the number of cells in the x, y, and z dimensions respectively. This sequence is invariant to the physical dimensions of each cell but assumes regular cells. The 3D line equations are y= x = x= y =
ny x, z nx
=
nz x nx
nx y, z ny
=
nx z, y nz ny z, x nz
=
nz y ny ny x nx
=
nx y ny
if |nx | ≥ |ny |, |nx | ≥ |nz | if |nx | < |ny |, |ny | ≥ |nz |
(5.2)
if |nx | ≥ |ny |, |nx | < |nz | if |nx | < |ny |, |ny | < |nz |
Note that, as is done in [144], line symmetry properties are exploited to avoid slopes greater than one. The cells in the sequence are determined by selecting and then applying the appropriate equation in (5.2) for each
5.1. MULTI-STEP A* (MSA*)
87
successor (nx , ny , nz ). The equation is evaluated at the midpoints between cells, i.e. 0.5, 1.5, . . . , n − 0.5 cell widths. If the midpoint lies on an edge, cells that share that edge are included in the cell sequence. If the midpoint intersects a corner point, all cells that share that corner point are included. This produces a cell sequence that has Manhattan stepping with non-zero spacing between the trajectory segment and cell boundaries. The horizontal and vertical track angles, θ and φ respectively can be calculated from the displacement as shown in (5.3) and (5.4) respectively. θ = arctan
ϕ = arctan q
nx δx ny δy
(5.3)
nz δz 2
2
(nx δx) + (ny δy)
(5.4)
where δx, δy and δz correspond to the x, y and z dimensions of each cell respectively. Note that as Γ is specified a priori, there is no need to optimise the cell sequence generation algorithm. Consider the design of Γ3D . From (5.3) and (5.4), it can be seen that arbitrary track angles are possible but this can result in successors that are displaced by a great cell distance (nx , ny , nz ) from s. It is possible to reduce the physical track distance by increasing the cell resolution, but this also increases the computation time. Thus, the design of Γ3D is dependent on the available computation time, desired track length and angle resolution for a specific application.
5.1.2
Extending the Successor Operator to 4D
Consider the extension of Γ3D to four dimensions where each cell has dimensions (δx, δy, δz, δt); δt specifies a duration of time spent inside a 3D cell. The vector neighbourhood concept extends to the time dimension in that s0 lies at a discretised time level s0tl that is displaced from stl by ntl time levels; ntl corresponds to the track traversal time. In 4D vehicle motion planning, the track velocity plays an important role in evaluating the attractiveness (or cost) of each trajectory segment. This is because of dynamic elements such as dynamic obstacles. For instance, selection of a faster or slower velocity may avoid a collision given the same 3D
88
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
track. It is not possible to predefine a set of cruise velocities for each successor in Γ due to the unpredictable nature of wind. Instead, it is better to generate multiple 4D successors s0 with displacement (nx , ny , nz , ntl ) where ntl ∈ Ntl for each 3D successor displacement (nx , ny , nz ). Selection of a suitable set of time level displacements Ntl is conditional on application specific parameters. These include the track length, minimum and maximum cruise velocity of the vehicle and expected wind conditions (i.e. magnitudes). The time level displacement for each cell s in the cell sequence generated using (5.2) can be calculated using nsx s n = n t t l nx l ns nstl = ntl nyy ns = nt nsz tl l nz
if |nx | ≥ |ny |, |nx | ≥ |nz | (5.5)
if |nx | < |ny |, |ny | ≥ |nz | otherwise
where ns is the displacement of cell s from the source node. Given ntl , the vehicle cruise velocity can be derived from the track length via the track velocity v~t . q (s0x − sx )2 + (s0y − sy )2 + (s0z − sz )2 |~ vt | = (5.6) ntl δtl This track velocity is itself a sum of the cruise and wind velocity vectors |~ vt | cos φ
sin θ cos θ
!
= |~ vc | cos φ
sin α cos α
!
+
wx wy
!
(5.7)
where θ and φ are the horizontal and vertical track angles respectively, v~c is the cruise velocity, α is the vehicle heading angle and (wx , wy ) are the horizontal wind magnitudes. All angles are measured from true north in navigational tasks [145]. Note that equation (5.7) is predicated on 2D wind and cruise velocity vectors. This is the case as the horizontal component of the track and track velocity is far greater than the vertical for both UAVs [145] and AUVs [124]. The vertical component is not ignored, and is instead treated as a constraint (like a maximum climb rate). By separating the x and y components from (5.7) and then solving the resultant simultaneous
5.1. MULTI-STEP A* (MSA*)
89
equations, it is possible to get an expression for v~c given |vt | and (wx , wy ). |~ vc | =
q
|~ vt |2 − 2 |~ vt | (wy cos θ + wx sin θ) + wx2 + wy2
(5.8)
Note that we are not interested in the negative root which corresponds to traversal in the opposite direction. This section described a method for generation of a vector neighbourhood using 4D (x, y, z, t) successor operators Γ much like that presented in [75] (which was for 2D x, y). The following section present a multi-step variant of A* [73] that enables the application of a variable successor operator Γs for each node s in the search space. This provides a basis for the implementation of multi-resolution search and imposition of a lattice structure on the search space that is not possible with a static neighbourhood like [75]. This structure is further discussed in Section 5.2.
5.1.3
Search Algorithm
The MSA* algorithm is listed in Fig. 5.2. Note that s0 and sg refer to the start and goal nodes respectively. Like A*, nodes are placed on a priority queue sorted according to the evaluation function f which is itself the sum ˆ In Fig. 5.2, Queue.Insert of the cost to come g and estimated cost to go h. refers to the addition of a node s0 to the queue such that f (s∗ ) ≤ f (s) ∀s ∈ Queue where s∗ is the topmost element in Queue. Queue.P op is the removal of this topmost element. The key distinction between MSA* and A* lies in the cost function c. A* computes the cost as a function of the cells s and s0 , whereas the new cost is a function of multiple cells as defined in Γs . g(s0 ) = g(s) + c(s0 = sj+m−1 ks = sj , . . . , sj+m−2 )
(5.9)
Evaluation of this cost function c can be divided into two steps. First, the decision variables xi (e.g. fuel, risk) are uniquely mapped or calculated from the cell sequence such that xi = ρi (s, . . . , s0 ) where ρi is the mapping function. Traditionally, the decision variable value is either averaged between the source and successor cells or the worst (or best) case value is taken [14]. The problem with taking the average is that the selected path may include
90 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21:
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK for all s ∈ S do g(s) ← ∞ end for Queue ← ∅ s ← s0 Queue.Insert(s) g(s) ← 0 while s 6= sg do s = Queue.P op() if s = sg then return end if S 0 ← Succ (Γs (s)) for all s0 ∈ S 0 do gˆ(s0 ) = g(s) + c(s0 kγs0 ) ˆ 0 , sg ) fˆ(s0 ) = gˆ(s0 ) + h(s 0 0 if gˆ(s ) < g(s ) then g(s0 ) ← gˆ(s0 ) Queue.Insert(s0 ) end if end for end while
Figure 5.2: MSA* Pseudocode. Note that at line 12, Succ (Γs (s)) extracts the set of successor nodes s0 ∈ S 0 where s0 is the last cell in each cell sequence γs0 ∈ Γs (s).
individual cells that exceed constraints for particular decision variables even though the average does not. A better approach is thus to take the worst case value. For variables such as risk, the final variable value (probability) is actually the sum of individual cell values (probability density values) [146]. Note that constraints can be imposed by setting c = ∞ if a particular decision variable exceeds a specified limit (e.g. maximum risk). Thus, the solution path is guaranteed to be satisficing. In the second step, a Multi-Criteria Decision Making (MCDM) cost function is used to transform the decision variables into a single cost term c where c is non-zero and monotonic (i.e. c > 0). This second step could be a weighted sum aggregation (like that used in [106, 133–135]) or a fuzzy mapping (e.g. [115]) as described in Section 3.5. Using a weighted sum approach for example, each decision variable is mapped onto a commensurate scale on the interval (0, 1) using a value function ui (xi ). The final cost is c = w0 u0 (x0 ) + . . . + wn−1 un−1 (xn−1 ) + δc where n is the number of decision criteria and δc is a small positive value to ensure c > 0.
5.1. MULTI-STEP A* (MSA*)
91
MCDM cost functions (like the one described above) can be incorporated into any search algorithm. However, it is non-trivial to determine the values of the decision variables in this cost function as the decision space (made up of individual decision variables) is often of higher dimensionality than the search space (refer section 6.1 for an example). MSA* and the accompanying lattice structure (refer section 5.2) can be used to model decision variables such as track velocity, track angles and track fuel consumption (in the presence of wind) using a 4D search space. This approach is more tractable (computation-wise) than searching a high dimensional decision space with an existing search algorithm (with a MCDM cost function). Note that the difference between A* [73] (Fig. 2.9) and MSA* is in the evaluation of the track traversal cost c (lines 12 and 14 in Fig. 5.2). A common approach for evaluating c when using A* is to place sample points on the track joining s and s0 [12]. The decision variables xi are then determined from these sample points in the same manner as MSA*. This approach typically produces fewer sample points (and hence reduced computational complexity) but does not provide any tolerance to uncertainty (since all sample points lie on the track). If the world space is modeled with a grid, then the cell sequence sampling methodology described in section 5.1.1 can be used with A* to provide a tolerance to uncertainty. In this scenario, A* has equivalent complexity to MSA* as both have the same cost function and the same number of sample points (for a given successor operator). In MSA*, it is necessary to determine which successor operator to use, but this has negligible complexity compared to the MCDM cost function. It can be seen that MSA* is similar to A* in many respects. However, unlike A* (which uses a static successor operator Γ(s)), MSA* makes use of an explicit variable (and multi-step) successor operator Γs (s). This successor operator enables the imposition of a lattice structure on the search space (refer section 5.2) and provides a tolerance to uncertainty (refer section 7.5). As shown in section 7.3, a multi-resolution lattice can greatly reduce the number of nodes in the search space. This results in a significant reduction in computation time over A* (e.g. the computation time ratio of MSA*1, MSA*2 and MSA*3 to A* is 0.484, 0.239 and 0.329 respectively - refer section 6.3 and 7.3).
92
5.1.4
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
Cost Optimality of MSA*
It can be shown that MSA* will find the least cost path given a predefined set of successor operators Γs for each node s ∈ S and MCDM cost function c. As MSA* is derived from A* [73], cost optimality can be shown in a similar manner as well. Lemma 1. Consider any globally optimal path P ∗ = (s0 , . . . , sn ). It can be shown that P ∗ is itself composed of optimal paths. Proof. The optimal path for a given node sik is a path P ∗ = (s0 , . . . , sik ), such that for all possible paths P ∈ Π, g(sik kP ∗ ) < g(sik kP ). Recall that any path is made up of an integer number of trajectory segments K and each segment is represented by a cell sequence of length mj for segment j. Thus, the index to a node (or cell) in P ∗ at the k th trajectory segment is ik = −k +
k−1 X
mj
(5.10)
j=0
For the case where k = K, the lemma is trivially true by definition of Pi∗K as iK = n, Pn∗ = (s0 , . . . , sn ). Consider the case for k = K − 1 trajectory segments, whose cell sequence PiK−1 = (s0 , s1 , . . . , siK−1 ) is a subset of the optimal path P ∗ . If PiK−1 is not a least cost path, then there exists another path Pi0K−1 = s0 , s01 , . . . , s0iK−2 , siK−1 such that g(Pi0K−1 ) < g(PiK−1 ). But, as the K − 1th cell sequence siK−1 , . . . , siK is unchanged, then given (5.9), the cost term c is unchanged. This implies that there exists a path Pi0K = (s0 , s01 , . . . , s0iK−2 , siK−1 , . . . , siK ) such that g(PiK ) < g(Pi∗K ), contradicting the definition of Pi∗K . Therefore, PiK−1 must also be an optimal path. By mathematical induction, any optimal path P ∗ must itself be composed of optimal paths. Theorem 1. If the heuristic is admissible [73], then MSA* will find the ˆ is an estimate of the optimal path if one exists. An admissible heuristic h ˆ j , sg ) ≤ h(sj , sg ). cost to go that is always less than the actual cost to go, h(s Proof. Consider an optimal path P ∗ = (s0 , . . . , sg ) which contains K trajectory segments. On an optimal path, g(sj ) = g ∗ (sj ) for all j = (0, 1, . . . , iK ). ˆ j , sg ) ≤ h(sj , sg ), therefore, f (sj ) ≤ f ∗ (sj ). From line 15, because h(s
5.1. MULTI-STEP A* (MSA*)
93
In the trivial case where K = 0 (i.e. s0 = sg ), MSA* discovers the solution in one iteration. During initialisation (lines 1-7 in Fig. 5.2), s0 is placed on the queue with cost g(s0 ) = g ∗ (s0 ) = 0. Upon expansion of s0 , MSA* terminates. Consider the case where K = 1, i.e. sg ∈ S00 where S00 = Γs0 (s0 ). Let S ∗ denote the set of nodes on the queue that lie on an optimal path. After one iteration (i.e. expansion of s0 ), at least one of the successors s00 is a member of S ∗ . Consider the contrary where none of the nodes s00 ∈ S00 lie on an optimal path and/or are not on the queue. There are two possibilities, one is that none of the successors are reachable (in which case no path exists) or at least one of them lies on a least cost path. Note that if a path exists, then an optimal path also exists. For this latter case, given a node s00 , an optimal path (s0 , . . . , s00 ) must exist that does not contain any nodes in S00 because by Lemma 1, an optimal path (s0 , . . . , sn ) comprises optimal paths (s0 , . . . , sn−1 ), (s0 , . . . , sn−2 ), . . . and none of the nodes in S00 lie on an optimal path (by assumption). This is not possible, hence, at least one node s00 must lie on an optimal path in which case, by line 16, s00 would be added to the queue at the expansion of s0 . Therefore, where a path exists, S ∗ 6= ∅ and sg ∈ S0∗ for the scenario K = 1. The preceding argument can be extended to show that up until algorithm termination, S ∗ 6= ∅. Let Sk0 denote the set of nodes generated by 0 0 Γsik−1 ∈Sk−1 (Γsik−2 ∈Sk−2 (. . . Γs0 (s0 ))) - i.e. all nodes that can be reached in k trajectory segments. From above, all nodes s : s ∈ S ∗ , s ∈ Sk (k = 1) are placed on the queue after the first iteration. If optimal paths of length k + 1 exist, expansion of s : s ∈ S ∗ , s ∈ Sk must yield nodes s0 : s0 ∈ S ∗ , s0 ∈ Sk+1 . Otherwise, as before, there would exist nodes s0 ∈ / Sk that result in optimal paths of length k + 1 trajectory segments which is not possible. Hence, S ∗ 6= ∅. Because Γs is a finite set for all s ∈ S and because trajectory segments incur a non-zero and non-negative cost c, there are only a finite number of nodes such that f (s) ≤ f ∗ (sg ) = f ∗ (s0 ). Therefore, as S ∗ 6= ∅, the nodes on the optimal path P ∗ = (s0 , si1 , . . . , sik = sg ) are expanded (in sequence) in a finite number of iterations, terminating with the expansion of sg . It is not possible to terminate without finding the optimal path if one exists. Consider the scenario where MSA* terminates such that f (sg ) =
94
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
g(sg ) > f ∗ (s0 ). But, by the analysis above, there exists a node s ∈ S ∗ just before termination such that f (s) ≤ f ∗ (s0 ) < f (sg ). Hence, s would be expanded instead of sg , contradicting the assumption that MSA* would have terminated. Therefore, MSA* will find an optimal path (s0 , . . . , sg ) in finite time where such a path exists.
5.2
Multi-Resolution Lattice Structure
This section describes a multi-resolution lattice based search space structure which makes use of the variable successor operator presented in the preceding section. By using MSA*, it is possible to find an optimal path with an admissible heuristic using a variable successor operator. The purpose of the lattice is to reduce the number of sample points and thus reduce computation time. For the purposes of visualisation, a three dimensional lattice structure is presented first which is then extended conceptually to 4D. It is shown that Lattice MSA* reduces the size of the search space without sacrificing track angle resolution or soundness [67].
5.2.1
3D Lattice
An illustration of the proposed 3D lattice is shown in Fig. 5.3. This lattice consists of a series of planes parallel to the x − y, y − z and x − z Cartesian planes at regular intervals of Λx , Λy and Λz (which are defined in terms of the number of cells in x, y and z respectively). A 2D (x − y) cross-sectional view of Fig. 5.3 is provided in Fig. 5.4 to show source-successor trajectory segments. The lattice design methodology is a three step process. Firstly, a series of base 3D successor operators Γ0 are chosen, one for each search space resolution. As described previously, selection of Γ0 requires consideration of the desired track angle resolution and track length; these parameters are both related to sampling density. Γ0 must be such that, in any plane x − y, x − z or y − z, all successors lie on the border of a rectangle centred at the source node (as in Fig. 5.4a).
5.2. MULTI-RESOLUTION LATTICE STRUCTURE
95
x y
z
z y x
Figure 5.3: General lattice structure, (x, y, z) dimensions shown
(a)
(b)
Figure 5.4: Top (x − y) view of lattice structure showing trajectory segments for (a) lattice position (0, 0), (b) (0, 2).
Using Γ0 , it is then possible to define the spacing between planes in the lattice, Λx , Λy , Λz using Λx = sup γ ∈ Γ0 : γnx ≥ γny , γnz γnx
Λy = sup γ ∈ Γ0 : γny ≥ γnx , γnz γny
(5.11)
Λz = sup γ ∈ Γ0 : γnz ≥ γnx , γny γnz
where sup denotes the supremum operator (least upper bound). Note that if the start or goal nodes do not lie on the lattice, it is a simple matter to connect those nodes to one that is on the lattice using a local search technique (refer [12]). Using this lattice structure, it is then possible to define individual Γs operators for each node on the lattice. An example is shown in Fig. 5.4 in 2D for the sake of clarity. As the lattice is regular, nodes that are located at equivalent positions on the lattice share the same successor operator Γp~ for
96
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK x y
z y x
Figure 5.5: Constrained vertical angle lattice structure.
position p~. Two positions are said to be equivalent if and only if they are separated by integer multiples of Λx , Λy and Λz cells. A lattice position p~ can be uniquely defined based on the modulus p~ = (mod(x, Λx ), mod(y, Λy ), mod(z, Λz ))
(5.12)
Referring to Fig. 5.3, the total number of unique lattice positions np is np = Λx Λy + Λx (Λz − 1) + (Λy − 1)(Λz − 1)
(5.13)
Consider the case where the source node is at lattice position (0, 0, 0), i.e. at the intersection of lattice planes (Fig. 5.4a). Successors of the source node are chosen to lie on the border of a rectangle centred on the source node with dimensions of (2Λx , 2Λy , 2Λz ) cells. Therefore, Γp~=(0,0,0) = Γ0 . For cells located at different positions on the lattice (refer Fig. 5.4b), the successors are chosen to maximise the number of identical successors to Γ(0,0,0) . Additionally, where possible, the successors are chosen to terminate at lattice position (0, 0, 0). This ensures that the same track angle can be maintained over consecutive trajectory segments to avert unnecessary turns. For the lattice structure illustrated, the horizontal track angles are at: 0◦ , 18.4◦ , 33.7◦ , 45◦ , 56.3◦ , 71.6◦ , 90◦ , 108.4◦ , 123.7◦ , 135◦ , 146.3◦ , 161.6◦ , 180◦ , 198.4◦ , 213.7◦ , 225◦ , 236.3◦ , 251.6◦ , 270◦ , 288.4◦ , 303.7◦ , 315◦ , 326.3◦ , 341.6. For fixed wing UAVs (and also AUVs), there is a maximum limit on the vertical track angle (i.e. climb rate). The maximum Flight Path Angle (FPA) is small as the horizontal cruise velocities (e.g. 36-57m/s for the RQ-7A Shadow UAV) are much greater than the maximum climb velocity (5m/s) [3]. A constraint on the vertical track angle can be modeled by a constraint on Γ such that if nz 6= 0, then either nx = Λx or ny = Λy .
5.2. MULTI-RESOLUTION LATTICE STRUCTURE
97
Given such a successor operator, Λz = −∞ (from (5.11)) which results in a simplified lattice structure as shown in Fig. 5.5. The maximum vertical track angle is (assuming δx = δy = δ) φmax = arctan
n z δz δ max(nx , ny )
(5.14)
Additionally, the total number of unique lattice positions is greatly reduced: np = Λx + Λy − 1
5.2.2
(5.15)
3D Multi-Resolution Lattice
The purpose of multi-resolution sampling is to reduce the total number of nodes and thus reduce computation time. In many applications, the search space can be divided into regions of fine sampling resolution and regions of coarse sampling resolution [14]. In UAV MFP for example, fine sampling resolution is required at lower altitudes but a coarse sampling resolution can be used for high altitude en-route airspace. It is easy to implement multiresolution search by using multiple base successor operators Γi0 . Consider the division of the search space into a series of N rectangular prism shaped regions each of which has a lattice resolution of (Λix , Λiy , Λiz ), i = 1, 2, . . . , N and a base successor operator Γi0 . Typically, N is small as it is necessary to check the source node against region boundaries at every iteration to determine successor nodes. Each region must have dimensions that are a multiple of Λix , Λiy , Λiz to ensure that all trajectories terminate on and originate from a lattice plane separating the two regions (refer Fig. 5.6). Furthermore, assume that for any two adjacent regions i and j, Λix , Λiy , Λiz is an integer multiple of Λjx , Λjy , Λjz . This way, successors in Γi0 are displaced at integer multiples of those in Γj0 , which ensures that all horizontal track angles (calculated using (5.3)) in region j also exist in region i (see Fig. 5.6). It is not possible to guarantee replication of vertical track angles between regions in general, even though track angles in the x−z and y−z planes are replicated in region i. The successors in Γi0 can also be filtered to limit the increase in the number of successors (and hence computation time per iteration). This can be implemented for example by selecting every k th successor in Γi0
98
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
Figure 5.6: Multi-resolution lattice with Λx = Λy = 3 on the left and Λx = Λy = 6 on the right. Selected trajectory segments are shown for the sake of clarity.
starting with the successor at track angle (0◦ , 0◦ ) (horizontal and vertical). k is an integer such that Λix = kΛjx and similarly in the y and z dimensions. The aforementioned multi-resolution lattice provides a means for fine and coarse sampling corresponding to smaller and larger values of Λ respectively. There is no reduction in track angle resolution for coarsely sampled regions as larger values of Λ enable higher track angle resolutions. Additionally, each track is evaluated at the cell resolution using cell sequences defined in Section 5.1.1. This guarantees path soundness [67] by avoiding the problem of mixed cells (which contains free space and obstacles [12]).
5.2.3
4D Multi-Resolution Lattice
Extension of the previous 3D lattice to four dimensions involves selection of a suitable set of values ntl ∈ Ntl for each 3D successor in Γp~ using the methodology described in Section 5.1.2. The full 4D multi-resolution lattice structure is implicitly defined through variable successor operators where, at each iteration of MSA* search, Γp~ is selected based on (5.12) and the boundaries of each Γi0 region. This approach enables consistent track angle resolution across fine and coarse resolution regions without sacrificing soundness. In addition, the lattice structure reduces memory usage. The underlying cell grid of MSA* contains Nx Ny Nz Ntl nodes where Nx , Ny , Nz and Ntl are the total number of sample points in the x, y, z and t dimensions respectively. The total memory requirement (i.e. number of nodes) N for a lattice can easily be derived by counting the number of nodes in the x, y and z planes
5.3. ARCHITECTURE
99
(refer Fig. 5.3) and then subtracting overlap regions. N = Ntl (Ny Nz αx + Nx Nz αy + Nx Ny αz − αx αy Nz −αx αz Ny − αy αz Nx + αx αy αz )
(5.16)
k j + 1 (and similarly for y and z) and all division operations where αx = NΛx −1 x are integer divisions. Note that for the case where vertical track angle is constrained as in (5.14), the αz term in (5.16) goes to zero. The proposed multi-resolution lattice is suited to efficient practical implementation. Firstly, a regular lattice structure ensures that only a small set of successor operators Γp~ are needed. This enables the encoding of Γp~ as a computationally efficient Look-Up Table (LUT). It is easy to determine which successor operator to use based on equation (5.12) and there is no need to check if the resultant successor lies on the lattice or not. Furthermore, the orthogonality of the lattice provides for easy extension from 3D to 4D.
5.3
Architecture
An important consideration in the design of a UAV mission flight planner is how it interfaces with other systems within the hierarchy of an intelligent control architecture. As shown in Fig. 3.3, path planning resides in the topmost deliberative layer of the standard three layer architecture [16]. A review of existing intelligent control architectures can be found in [8]. Boskovic [38] presents an adaptation of the three layered architecture for autonomous UAV operation where the interaction between path planning and lower level control systems is made explicit (refer Fig. 5.7). The proposed MFP system corresponds to the ‘autonomous path planner’ in Fig. 5.7. Note the interfaces between the MFP system and other UAV systems. An MFP system needs: • Start position and time s0 • Goal position and time sg • MCDM cost function – Decision objectives – Constraints – Decision criteria
100
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
Figure 5.7: Boskovic’s UAV intelligent control architecture [38].
• Heuristics • Environment maps (situational awareness) Note that satisfaction of one or more decision criteria (and constraints) is needed to address a decision objective. For example, the safety objective may be decomposed into a risk criterion, no fly zones criterion and maximum velocity constraint. Additionally, an approximation of the dynamic constraints of the vehicle is required to ensure that the final path is traversable. Note also that due to the existence of multiple decision criteria, the environment map is actually a collection of data maps that describe the operating environment. For example, there may be a risk map (based on a probabilistic occupancy grid for example), a map of no fly zones and a wind vector map. As described in [8, 38] and shown in Fig. 5.7, the output of the MFP is a sequence of absolute waypoints in 3D space. In addition, due to the dynamic nature of the environment, each waypoint is given an expected time of arrival. It is then the role of the trajectory generation layer to design a trajectory in terms of the six degrees of freedom (dof) of the aircraft that follows the given sequence of waypoints. Trajectory generation can be thought of as a form of high resolution local planning whereas MFP is low resolution global planning.
5.3. ARCHITECTURE
101
In the absence of a trajectory generation system, the MFP system can equivalently provide the Ground Track Angle (GTA), FPA and Indicated AirSpeed (IAS) at each waypoint. These can then be passed, via a sequencer, to the onboard autopilot (i.e. flight control system). This approach is feasible as modern autopilots can perform basic navigational tasks using Global Positioning System (GPS) if the distance between waypoints is sufficiently large. In en-route planning, the distance between waypoints is typically in the order of nautical miles [18].
5.3.1
Planner Subsystem
The proposed, self-scheduling replanning architecture1 is presented in Fig. 5.8 and 5.9 in the form of a data flow diagram similar to that used in [147]. Data processes are represented by rounded rectangles and are executed when data is present on all required input ports. Required data is denoted by a filled arrow and optional data with an open arrow. The search state also has a trigger input port (denoted by a filled rectangle line). Data itself is represented by rectangles. This architecture models the offline and online (re-)planning problem for an aircraft in constant motion (which is the case for fixed wing aircraft). The notation used in Fig. 5.8, 5.9 and 5.10 follow that used in the preceding sections. Nodes in the search graph s ∈ S correspond to sample points in the 4D world space where s0 is the start node, sg the goal node, (si−1 , si ) is the current trajectory segment, c(s, s0 ) is the MCDM cost function and h(s) is the heuristic function. The MSA* search algorithm resides in the search process in Fig. 5.8 given the parameters provided on the input ports and by the monitor. Whenever a search is being executed, the PlanFlag (essentially a busy flag) is set to prevent repeated interruption of the search process such that a path is never returned. The role of the monitor is to trigger searches and provide a degree of knowledge level control as per Rasmussen’s model. Replanning occurs if there is a significant change to the data maps (i.e. change in the environment), a change to the cost function c(s, s0 ), to constraints/tolerances and to the goal This architecture adopts the feedback based control architecture first proposed by Dr Torsten Merz (CSIRO). However, it extends upon the original architecture by incorporating expert pilot decision making strategies for MFP such as the adjustment and anchoring heuristic. 1
102
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
Figure 5.8: Planner flow diagram. Green represents input data, red output data, yellow represents data and states associated with the search itself, and blue represents data and states associated with the monitoring process.
sg . Additionally, a replan is also triggered if the start node s0 does not equal the target node (or waypoint) for the current trajectory segment. This occurs if the UAV has deviated from the planned track beyond tolerance bounds. Recall from Section 4.2 that within the planning process, application of path constraints correspond to skills level cognition and evaluation of path cost to rules level cognition. Candidate methods for implementing rules level cognition were identified to be Multi-Attribute Utility Theory (MAUT) and fuzzy logic (although computationally expensive, it directly implements IF THEN rules). In addition, it is found that experts employ heuristics to speed up the search process. A common heuristic identified in experts is the adjustment and anchoring heuristic. This heuristic is applicable to MFP as standard flight profiles typically consist of straight line segments between goal waypoints [18]. For a heuristic search like MSA*, this can be implemented by using a weighted heuristic estimate of the cost to go h(s) based on the Euclidean distance d between s and sg (in 2D or 3D space). This is known as heuristic inflation [79] and is a common approach in the implementation of heuristic search in mobile robotics [14, 33].
5.3. ARCHITECTURE
103
However, in the presence of a MCDM cost function where the resultant cost is on (0, 1), an extra scaling factor (or function) is required as the Euclidean distance is likely many times the actual cost to go. As it is difficult to accurately predict h without further information on the states between s and sg (which would incur extra computation), an alternative simulation based approach is proposed. Prior to flight, a set of Monte Carlo simulations (like those presented in Section 6.3) are run on virtual scenarios using the same cost functions. These virtual scenarios are based on the same operating conditions as those expected for the actual mission, but with some random variation. It is possible to empirically determine a scaling factor α such that h(s) = αd(s, sg ) is always admissible. This is difficult to guarantee across the infinity of all possible planning scenarios, but can be achieved in practice by choosing α conservatively as shown in Section 7.6 (thus making α smaller than it needs to be). Choosing α in this manner produces a relatively weak heuristic but ensures path optimality and minimises unnecessary node expansions as observed in [14]. It also provides a sound reference value for heuristic inflation. Using an admissible Euclidean heuristic, it is a simple matter to implement the adjustment and anchoring heuristic by scaling (inflating) with . h(s, sg ) = αd(s, sg )
(5.17)
The monitor in Fig. 5.8 not only schedules planning/re-planning, but also adjusts the heuristic inflation in a similar vein to anytime planning (refer Section 2.3). In offline planning, represented by an aircraft ground state (stored in the current state data box), the monitor initiates planning with an admissible heuristic. During flight, the monitor continuously compares the current state with the current trajectory segment and determines if the aircraft has strayed beyond the tolerance bounds of the current flight plan. The tolerance bounds include not just the 3D flight path, but could also include time bounds (arriving too early or late) and fuel bounds (using more or less fuel than expected). These bounds are non-zero due to unpredictable wind effects and non-negligible flight controller error. If tolerance bounds are exceeded, then a replan is triggered based on the expected start node s0 . To mimic the adjustment and anchoring heuristic, the initial replan is performed with a large inflation factor on h(s). Even though
104
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
the resultant path is suboptimal, it is still feasible as path constraints are still satisfied. Thus the solution is a satisficing solution. However, as noted in Section 2.3, heuristic inflation results in a greedy search that is less efficient than an admissible search in the presence of local minima. The purpose of the timer is to cut off this initial search if it exceeds a pre-determined maximum search time tcutoff (also determined through Monte Carlo simulation) for a satisficing solution. This approach aids in overcoming inefficiencies with an inflated heuristic (as reported in [33, 79]) by switching to an admissible search. Typically, the initial plan is completed within the order of one second. If the aircraft reaches s0 before a plan is available, it is assumed to execute a hold manoeuvre at s0 . A fast initial solution can help prevent unnecessary hold manoeuvres for fixed wing aircraft. Following the completion of the initial plan, the monitor calculates the time t0 to reach s0 from its current position given the current track velocity assuming a straight line trajectory (effectively a lower bound on the actual time available). If the aircraft is currently executing a hold manoeuvre, then t0 is ∞ or the maximum loiter time. In either case, if t0 exceeds the expected worst case planning time tplan , then the monitor triggers another search but this time with an admissible heuristic. Otherwise, the monitor waits until s0 is such that t0 exceeds tplan . If tplan is less than the minimum trajectory traversal time, then admissible replanning would occur on the first trajectory segment of the satisficing path. Note that tcutoff = ∞ for admissible planning since it has been proven to find a solution in finite time. It can be seen that the monitor module enables a degree of knowledge level cognition and meta-cognition as per Rasmussen’s model.
5.3.2
Self-Scheduling Replanning Architecture
A potential implementation of the self-scheduling replanning architecture is provided for an intelligent control architecture that is absent of a trajectory generation system. The outer control loop is shown in Fig. 5.9 and the standard inner control loop in Fig. 5.10. Note that the aircraft state includes not just the position, orientation and time information, but also variables like fuel state, velocity and local wind estimates. Reactive control and manual control is compatible with the control loop presented in Fig. 5.9 as they are modeled as external set points which are fed
5.3. ARCHITECTURE
105
Figure 5.9: Outer feedback control loop data flow diagram. Green represents input data, red output data, data processes and data coloured in yellow relate to the deliberative layer, orange for the sequencing layer and blue the control layer in a three tiered intelligent control architecture [16]. Environment
Set points
+_
Controller
Aircraft
Aircraft configuration
Aircraft state
Real aircraft state
Aircraft performance
Sensor
Figure 5.10: Inner aircraft feedback control loop data flow diagram. Again, green represents input data, red output data, yellow represents aircraft systems and blue represents real world truths.
directly into the autopilot. Under normal operating conditions, the UAV operator may reject the proposed plan and update the decision function. Doing so triggers a replan as new inputs are available to the planner. The operator can also take direct control of the aircraft via the external set points. In the event of a communications outage, control is switched wholly to onboard planning systems since there is an absence of manual control inputs. As the planner is modeled after expert decision preferences and is guaranteed to find a satisficing (or an optimal) solution, the UAV is able to continue the mission.
106
CHAPTER 5. MISSION FLIGHT PLANNING FRAMEWORK
Emergency reactive control systems (such as emergency collision avoidance systems), when activated, override the planner via the switch. During execution of reactive manoeuvres, the monitor detects if the UAV has deviated beyond tolerable bounds for the planned path. If so, a new plan is computed immediately based on the predicted final state (e.g. position, time, fuel) of the aircraft which corresponds to s0 . Note that s0 is one of the outputs of the control loop and is made available to the local planner. When the manoeuvre is completed, the aircraft is assumed to fly to this predicted start node (under reactive control) and the new plan is enacted. If the reactive manoeuvres do not deviate from the planned path beyond tolerable bounds, then the original plan is resumed (by the sequencer) after reactive systems relinquish control. An important element of the feedback control loop is the predictor. This module runs continuously except when the PlanFlag is set. Setting this flag inhibits the predictor output to prevent interruption of the planning process. Note that even though s0 is constantly updated when PlanFlag is not set, this does not necessarily initiate replanning as the monitor controls this via a trigger. In selecting s0 , the predictor considers the current aircraft state, current target waypoint or node si (stored in the set point) and current situational awareness (data maps, constraints and tolerances). The predictor module contains a local planner (such as those described in Section 2.2.3) as it is necessary to find a nearby s0 that is reachable and collision free. Under normal operation, s0 = si , i.e. the next waypoint passed by the sequencer to the autopilot. This is logical as the planned path is known to meet flight constraints and optimises for multiple decision objectives. The predictor only selects s0 6= si if si is unreachable or if the UAV is no longer in the vicinity of si (due to reactive manoeuvres or operator input for example). A candidate method for selecting s0 when in the vicinity of si is to define a set of target nodes St such that all nodes st ∈ St are within an Euclidean distance of e around si . Using this goal set, A* [73] or even MSA* could be executed on the small, local search space between the current aircraft position and si . A simple, distance based cost function can be used which checks each trajectory segment against path constraints. The search will then return the shortest collision free path that satisfies these constraints if
5.4. SUMMARY
107
(a)
(b)
Figure 5.11: Illustration of start node s0 prediction using A* and goal set defined by e. Consider the scenario where given the existing trajectory segment (si−1 , si ), si is now unreachable due to obstacles (shaded ovals). The current position is s. In the first iteration (a), e is too small and no candidate s0 is found. At the next iteration (b), e is increased and both s0 and a local trajectory from s to s0 is found.
one exists. If not, then e can be incrementally increased until a path is found. This is illustrated in Fig. 5.11.
5.4
Summary
This section derived MSA*, a multi-objective heuristic search algorithm based on A* that finds least cost paths. MSA* enables the use of a variable successor operator which in turn enables a multi-resolution lattice search space. In addition, an architecture for implementing MSA* onboard a UAV is also presented. This architecture integrates deliberative, motion planning activities with reactive control systems and describes a methodology for scheduling (re-)planning using expert pilot cognitive strategies.
Chapter 6 UAV Flight Planning Application And this, ladies and gentlemen, is the very first Fokker airplane built in the world. The Dutch call it the mother Fokker. Custodian at the Aviodome aviation museum, Schiphol airport, Amsterdam
A
REPRESENTATIVE civil Unmanned Aerial Vehicle (UAV) application is selected for the purposes of simulation testing of Multi-Step A* (MSA*). The example mission scenario comprises the delivery of a medical package to a remote outback location using a small UAV. Such a mission is typically conducted under Visual Flight Rules (VFR) [148] due to the low altitude ceiling of small UAVs [3] and non-standard locale. Candidate UAV platforms for the proposed mission include the RQ-7 Shadow (which has a 68NM operating range), the RQ-2 Pioneer (100NM range) and to a lesser extent the Aerosonde UAV (since it has limited payload capacity) [3]. Many of the decision criteria considered for the medical delivery task are applicable to generic en-route flight planning under VFR. Medical package delivery is ideal for evaluating a 4D search algorithm due to the presence of multiple decision criteria, dynamic elements in the operating environment and the significant effect of wind on a small UAV. Both offline and online planning capability is required as it is sometimes necessary to prepare a flight plan for regulatory approval prior to flight and, due to a dynamic and uncertain operating environment, sometimes also necessary to replan during flight. This chapter discusses a set of relevant decision criteria for the UAV medical delivery task. In addition, it describes the implementation of MSA* for 108
6.1. DECISION OBJECTIVES
109
this application including the choice of successor operator and cost function. Finally, it describes the generation of simulation scenarios that mimic the real world operating environment.
6.1
Decision Objectives
This section describes the decision criteria relevant to the medical package delivery task. For each criterion, the potential data source, data representation and impact on the medical package delivery mission is discussed. Recall that each Mission Flight Planning (MFP) objective can be decomposed into individual decision criteria. Satisfaction of an objective is thus conditional on a multi-criteria evaluation of the constituent decision criteria. For the proposed mission scenario, three major decision objectives are considered: (i) safety, (ii) conformance with the rules of the air and (iii) mission efficiency.
6.1.1
Safety Objective
Mission safety is given utmost consideration in the planning process. The two most important safety concerns in the operation of UAVs in the National Airspace System (NAS) are that of midair collision and flight termination in a populated area [149]. Thus, for simulation purposes, the safety objective is modeled with a criterion for collision avoidance, for storm cell avoidance and for risk presented to people on the ground. The collision avoidance criterion was simulated by adopting the aircraft separation requirement for en-route airspace. This requirement comprises a vertical and lateral separation specification. The vertical separation is defined by the Reduced Vertical Separation Minima (RVSM) requirement to be 1000ft (304.8m) [150]. Lateral separation specifications are more complex and vary depending on aircraft flight vectors, equipage and airspace. For the purposes of simulation, a 5NM (9260m) lateral separation is adopted. This is the proposed separation minima for aircraft in conflict using Automatic Dependent Surveillance Broadcast (ADS-B) and is identical to the current standard for flight under Route Surveillance Radar (RSR) [150, 151]. The lateral and vertical separation standards describe a moving cylinder with a radius of 5NM and height of 2000ft. By incorporating a priori knowledge of aircraft movement, it is possible to strategically avoid collision scenar-
110
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
ios without activating emergency collision avoidance systems. Potentially, this information can be obtained from flight plans lodged with the regulatory body. Alternatively, it is also possible to obtain position and velocity information of other aircraft from surveillance systems such as ADS-B. There is uncertainty in the position, velocity and flight direction of other aircraft. Rathbun, Kragelund et al. [45] present a simple method for modeling this uncertainty based on growing position uncertainty. This approximation of the true uncertainty (which can be calculated using techniques described in [152]) is computationally efficient and thus suited to MFP. Note that there are two basic sources of uncertainty; uncertainty in the initial position and uncertainty in the estimated velocity vector. Positional uncertainty can be assumed to be due to Global Positioning System (GPS) error. Modern day GPS systems have a horizontal accuracy of 5-10m (95% confidence) and a vertical accuracy that is approximately 1.4 times the horizontal accuracy [153]. If the cell size is in the order of NM, then this error is negligible. However, the accumulated error due to velocity uncertainty can be much greater. Without further information on the performance and operator intentions of other aircraft, this uncertainty is assumed to be Gaussian. The approximation technique described in [22,45] is extended to 3D using a bivariate Gaussian model on a discrete grid. As the lateral and vertical separation requirements are specified independently, the Probability Density Function (PDF) p for the aircraft position can be expressed as
− 1 e p (r, z, σr , σz ) = 2πσr σz
(z−zc (t))2 r2 2+ 2 2σr 2σz
(6.1)
q where r = (x − xc (t))2 + (y − yc (t))2 and σr and σz are the standard deviations. The expected position (xc (t), yc (t), zc (t)) of the aircraft is assumed to be a piecewise linear function of the form xc (t) = vxi t + ci
(6.2)
where v i is the predicted velocity for the ith segment. yc (t) and zc (t) can be similarly expressed in piecewise linear fashion. The probability density field for the previously described aircraft separation zone can be derived by constructing a gate function on r and z and integrating it over the positional
6.1. DECISION OBJECTIVES
111
uncertainty distribution: r−R z−H Z Z
ΠR (r0 )ΠH (z 0 )p(r0 , z 0 , σr , σz ) dr0 dz 0
z+H r+R
β(r, z, σr , σz ) = Z∞ Z∞
(6.3)
ΠR (r0 )ΠH (z 0 )p(r0 , z 0 , σr , σz ) dr0 dz 0
−∞ −∞
where Π is the gate function, R is the radius (5NM) and H the half-height (1000ft). Equation (6.3) can be approximated using discrete sums
β(r, z, σr , σz ) =
K X L X
ΠR (r0 (i)) ΠH (z 0 (j)) p (r0 (i), z 0 (j), σr , σz ) δr δz
i=0 j=0
M X N X
(6.4) 0
0
0
0
ΠR (r (i)) ΠH (z (j)) p (r (i), z (j), σr , σz ) δr δz
i=1 j=1
where for the numerator, r0 (i) = r + R − iδr , z 0 (j) = z + H − jδz , δr = 2R , L 2(R+nσ ) r . For the denominator, r0 (i) = −iδr , z 0 = −jδz , δr = , δz = δz = 2H K M 2(H+nσz ) . Note that i, j are indices, K, L, M , N control sampling resolution N and n controls the number of standard deviations. As p is assumed to be Gaussian, values of n ≥ 2 are suitable for approximating the denominator in equation (6.3) as that corresponds to two standard deviations (i.e. 95% probability on the normal curve and slightly more than that for β with the addition of R). A plot of the horizontal component of β for different values of R and σr is shown in Fig. 6.1. Note that it is possible to divide β into r and z components due to the assumption of independence. Similar plots can be obtained for the z component. The area under the curve for r ≤ |R + 2σr | is on average 1.00 with a minimum value of 0.97. This empirically verifies the assertion above that a sampling limit of n ≥ 2 is sufficient to capture the majority of the PDF. The resultant density field β, defined in terms of r and z can itself be sampled using a 3D grid for a given time point in the same manner as [22]. This is important as it enables the calculation of approximate probability density values for cells in the world space W upon which the MSA* search is based. Doing so assumes that the cell size is small relative to R and H and
112
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION PDF σr=0.50
PDF σr=0.75
0.8
R=0.0 R=2.5 R=5.0 R=7.5 R=10.0
0.6
0.7
R=0.0 R=2.5 R=5.0 R=7.5 R=10.0
0.6 0.5 0.4
0.4 0.3 0.2
0.2
0.1 0 −20
−10
0
10
20
0 −20
−10
(a)
0
10
20
(b)
PDF σr=1.00
PDF σr=1.25
0.5
R=0.0 R=2.5 R=5.0 R=7.5 R=10.0
0.4 0.3
0.35
R=0.0 R=2.5 R=5.0 R=7.5 R=10.0
0.3 0.25 0.2 0.15
0.2
0.1 0.1 0 −20
0.05 −10
0
(c)
10
20
0 −20
−10
0
10
20
(d)
Figure 6.1: Plot of the horizontal component of the density field β for different values of R and σr .
as a result, that the expected aircraft position is in the centre of a cell. For the selected UAV MFP application, a cell size of 1NM×1NM is used which is small relative to R = 5NM. Consider the task of sampling β using a grid. Firstly, the horizontal Euclidean distance r is calculated between the expected aircraft position and neighbouring cells to a maximum distance of r + nσr . Regardless of the resolution of r, there will be some cells with the same r value as indicated in Fig. 6.2. Let ki be the number of occurrences of a particular discretised value of r. The density value of a cell at distance r is then β(r) . Note that it is only ki necessary to discretise r if the PDF is to be encoded as a pair of Look-Up Tables (LUTs) with r, R, σr as inputs for one and z, H, σz for the other. It is possible to separate the two due to the assumption of independence in equation (6.3).
6.1. DECISION OBJECTIVES
113
Figure 6.2: Illustration of sampling grid for density field β showing horizontal Euclidean distance r for cells with dimensions δx = δy.
The uncertainty in the expected position of other aircraft grows with time. This is approximated by casting the standard deviation σ as a function of time [45]. The growth in σ can be modeled linearly in a similar fashion to equation (6.2) or it could be based on the acceleration characteristics of the aircraft [45] a (6.5) σ(t) = σ(t0 ) + b (t − t0 ) + (t − t0 )2 2 where t is the current time, t0 is the initial time when the aircraft was observed to be at the initial position, b is the linear growth rate and a the acceleration. For the purposes of visualisation, it is useful to show not just the cylindrical separation zone (R = 5NM, H = 1000ft) around the expected position, but also the boundaries beyond which the risk can be deemed negligible. Note that the risk in this case does not correspond to the likelihood of a midair collision; it is instead the likelihood of encroachment on the minimal separation requirement. As shown in Fig. 6.1, the maximum possible risk if the UAV never approaches within a distance of r = R+2σr (and equivalently in the z dimension) of another aircraft is less than 0.03. The resultant shape of the separation zone, when visualised using the r = R + 2σr boundary is cylindrical. This model is suited to modeling storm cells as they too are dynamic and uncertain and can be described with a position and radius. Avoidance of storm cells and the associated air turbulence inside them is paramount to mission safety and is a requirement
114
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
of VFR operation. Forecasts regarding storm cells and their movement are provided by the Bureau of Meteorology in Australia [154]. Finally, it is desirable (though not mandatory) to avoid overflying populous regions to minimise the risk presented to people and property on the ground. The population risk is an important safety criterion that is unique to the operation of UAVs as there are no humans onboard [149]. This risk can be modeled in terms of the expected number of fatalities per flight hour given the kinetic energy of the UAV upon impact [149]. For the purposes of simulation, it is sufficient to approximate this risk value with the population density or Normalised Population Density (NPD). This information can be obtained in raster format from Geographic Information Systems (GISs).
6.1.2
Rules of the Air
The flight plan must also conform with the rules of the air, many of which are safety oriented. In addition to the separation requirements stated earlier, the aircraft must also conform with the cruising levels rule, low flying restrictions and segregated airspace. For aircraft flying on headings from 0◦ to 179◦ , the permissible flight levels are at odd multiples of 1000ft plus 500ft Above Mean Sea Level (AMSL) (e.g. 1500ft, 3500ft, 5500ft AMSL). For headings between 180◦ and 359◦ , the cruise levels are at even multiples of 1000ft AMSL plus 500ft (e.g. 2500, 4500, 6500ft AMSL). This is illustrated in Fig. 6.3. The cruising levels rule is intended to minimise the risk of a head-on collision, and is mandatory above 5000ft. When flying at low altitudes, civil air regulation CAR157 [7] stipulates that the aircraft must maintain an altitude of 500ft Above Ground Level (AGL) over non-populous areas and 1000ft above population centres. The altitude AGL is obtained by simply subtracting the current altitude in AMSL from the terrain elevation. Because of the relatively short range of small UAVs, it is possible to neglect the curvature of the earth [3]. Digital terrain elevation maps such as those obtained from the Shuttle Radar Topography Mission (SRTM) encode elevation information in raster format at a resolution of 90m for many areas of the world [20]. Finally, the UAV must avoid certain classes of segregated airspace (e.g. controlled airspace) in the en-route portion of the flight. These can be modeled simply as no fly zones. In Australia, there are five major categories of
6.1. DECISION OBJECTIVES
115
Figure 6.3: Cruising levels rule CAR173 [7] obtained from [148].
Figure 6.4: Airspace classes example.
airspace, namely class A, C, D, E and G [148]. Regions of airspace, as shown in Fig. 6.4, are defined using altitudes (e.g. en-route airspace, class A and E) or, in terms of altitude and proximity to an aerodrome (class C, and D). Class G airspace covers all regions not defined otherwise. Only in a small number of cases are there more complicated airspace designations (such as special use airspace or military airspace). Airspace charts can be obtained from Airservices Australia [155]. The airspace can be efficiently modeled using polyhedrons, especially cylinders (for class C and D airspace) and rectangular prisms (for class A
116
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
and E). Note that all remaining airspace is class G. For the proposed medical package delivery operated under VFR, it is desirable to avoid class A, C and D airspace. These obstacle regions can be represented as the conjunction of half spaces as shown in Section 2.1. Airspace is best represented with flat (equation (6.6)) and curved (equation (6.7)) surfaces due to the presence of cylindrical regions. fi (x, y, z) = ax + by + cz − d (6.6) fi (x, y, z) = (x − xc )2 + (y − yc )2 − r2
(6.7)
In such a representation, it is only necessary to store parameters a, b, c, d, xc , yc and r—this is much more memory efficient than a grid.
6.1.3
Mission Efficiency Objective
The flight plan also needs to optimise for the objectives of the mission itself (i.e. the delivery task). These objectives include the delivery time (i.e. the time of arrival at the goal node) and fuel consumption. With a 4D search, it is possible to not only find a path that minimises the delivery time, but also to designate a specific delivery time or acceptable time window (like with [39]). For the proposed application, the mission efficiency objective is fulfilled by minimising the total travel time and minimising the fuel consumption. It was shown in Section 5.1.2 that the flight time of a trajectory segment (or track traversal time) is contingent on the wind velocity vector. Wind and weather forecasts in Australia are obtainable from Airservices Australia [155]. For long range flight, wind forecasts are available in GRIdded Binary (GRIB) format [21] with 1 × 1.25◦ (latitude/longitude) resolution. Potentially, high resolution wind maps that incorporate ground effects can be derived from low resolution wind forecasts but this is beyond the scope of the research. The fuel consumption on the other hand is a complex function of the cruise velocity, track traversal time, flight altitude, climb/descent angle, aircraft parameters (e.g. fuel mixture, throttle, propeller pitch), and atmospheric temperature and pressure. A fuel model can be obtained from aircraft simulation software like Aerosim [156] (refer [4]). Alternatively, it can be approximated with a LUT by interpolating the cruise and climb fuel usage data provided by the aircraft manufacturer (e.g. [157]). A LUT based approach is advantageous as it helps to reduce computation time.
6.2. EXPERIMENTAL SETUP
6.1.4
117
Summary
This section described the safety, regulatory and mission efficiency objectives and the decision criteria and variables upon which they are based. A MultiCriteria Decision Making (MCDM) cost function can then be used to judge the cost (or utility) of a trajectory segment given these decision criteria (refer Section 6.2.1). However, it is also necessary to check each trajectory segment against path constraints. If a constraint is violated, then the cost of that trajectory segment is set to infinity. These constraints can typically be modeled as limits on each decision variable. For risk based criteria (under the safety objective), the constraint is simply the maximum tolerable risk. When considering the rules of the air, the constraint is applied when there is non-conformance to a mandatory rule (such as flight below 500ft AGL). A constraint that applies to the time criterion is the maximum flight time and latest time of arrival at the goal (whichever is smaller) and potentially also an earliest time of arrival. For the fuel criterion, the constraint is the total fuel load. In addition, it is also necessary to consider the dynamic constraints of the aircraft. Note that as the dynamic constraints are closely linked to the chosen successor trajectory segments in Γ, these are discussed in Section 6.2.3.
6.2
Experimental Setup
The primary purpose of a simulation based experimental analysis is to compare the computational efficiency and solution path of the proposed algorithm with that of existing algorithms. This section presents a candidate MCDM cost function and describes an implementation of MSA* for the UAV medical package delivery task. In addition, a method for generating Monte Carlo simulation scenarios is described. Note that the optimality of the flight plan is contingent on the accuracy of the planning maps (e.g. terrain maps, predicted flight path of other aircraft). However, inaccuracies in the planning maps can be mitigated with online replanning assuming that new information becomes available.
118
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
6.2.1
MCDM Cost Function
A candidate MCDM cost function is presented for the proposed medical package delivery mission using Multi-Attribute Utility Theory (MAUT) (refer Section 3.5.2). The Measuring Attractiveness by Categorical Based Evaluation Technique (MACBETH) method for designing value (or utility) functions and finding weightings is adopted using the MACBETH software package. Recall from Section 6.1 that the decision variables (or criteria) are: • • • • • • • •
Aircraft separation risk Storm cell risk Population risk (NPD) Cruising levels rule Low altitude flight rule Segregated airspace (no fly zones) Time Fuel
Since the low altitude flight rule and the avoidance of no fly zones are mandatory, these can be modeled purely as constraints. For the purposes of simulation, a weighted sum (i.e. a Choquet integral with 1-additive capacity, refer Section 3.5.2) is used to calculate the path cost based on the remaining variables. Fig. 6.5 shows the preferences used to define the fuel value function and Fig. 6.6 the preference matrix and resultant weightings. The utility functions used for the other decision variables can be found in Appendix A.
6.2.2
Test Algorithms
Two different variants of MSA* are compared against a benchmark algorithm, Vector A*, on 1000 randomly generated Monte Carlo planning scenarios and a number of special case scenarios. In these comparisons, each test algorithm uses a different (set of) successor operator(s) but the same cost function c (a weighted sum) and heuristic function h. Vector A* is a direct extension of A* using a vector neighbourhood (like that used in [75]). However, the method presented in [75] is for a 2D ground vehicle. Existing vector neighbourhood based A* does not incorporate multiple decision objectives (and constraints such as the effects of wind) and only mitigates uncertainty through online replanning.
6.2. EXPERIMENTAL SETUP
119
Figure 6.5: Fuel value function.
Figure 6.6: Judgment matrix and weights for decision criteria. Note that neighbtlvl refers to total flight time and clvls the cruising levels rule.
120
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Figure 6.7: Chosen Γ0 successor operator, showing source cell (centre) and successor cells joined by trajectory segments. Λx = 3, Λy = 3, Λz = −∞, Ntl = {1, 2, 3, 4}min.
The successor operator for the benchmark (Vector A*) is chosen to reflect the base successor operator used in the test algorithms and is shown in Fig. 6.7. Using this successor operator, there are 360 successors for each source node as the traversal time is either ntl = {1, 2, 3}min. or ntl = {2, 3, 4}min. Vector A* is in effect a special case of MSA* where Γs is constant throughout the entire search space. Due to the popularity (and optimality) of A* and related algorithms in robotics [33], Vector A* serves as an ideal benchmark of calculation time and path cost for a deterministic 4D planning algorithm with selectable track angle resolution. Two variants of MSA*, MSA*1 and MSA*2 are tested against the benchmark. MSA*1 uses a single, fine resolution lattice based on Γ0 as shown in Fig. 6.7. MSA*2, on the other hand, uses a multi-resolution lattice where N = 2. A fine resolution lattice based on Γ0 in Fig. 6.7 with Ntl = {2, 4}min is used for altitudes less than 7000ft, and a coarse resolution lattice (Λx = 6, Λy = 6, Λz = −∞, Ntl = {4, 6, 8}min) is used for altitudes above 7000ft. In addition, an inflated heuristic variant of the three test algorithms were also simulated over the same scenarios. This is done to verify the decreased computation time reported in the literature (refer Section 2.3.2) and to validate the proposed architecture (Section 5.3). As
6.2. EXPERIMENTAL SETUP
121
proposed in Section 5.3.1, the heuristic scaling and inflation factor (α and respectively) are determined by sampling a smaller number of simulation scenarios (200 in this case). All experiments were performed on a 3.3GHz Intel Core 2 Duo QX6850 CPU with 4GB of physical RAM running 32-bit Windows XP.
6.2.3
Dynamic Constraints
The dynamic constraints of the aircraft were considered in the selection of Γ0 in Fig. 6.7. Oftentimes, these dynamic constraints are modeled with a minimum turn radius [12]. Assuming a maximum airspeed of 65m/s, a bank angle of 30◦ and a force of one g (9.8m/s2 ), the worst case turn radius is approximately 747m (refer [145, (3.9.10)]). As the turn radius is less than half the cell size, it is possible to execute a 180◦ turn within the bounds of a single cell. However, it is still desirable to minimise the turn angle as it is difficult for the flight controller to execute such a turn with accuracy under strong wind conditions. Turn angles and the impact on the track are further discussed in Section 6.3.2. For UAV operations, it is also necessary to incorporate climb/descent rate constraints and a maximum airspeed constraint (a constant value). The latter can be easily enforced by comparing the cruise velocity calculated using (5.8) with the maximum airspeed. The maximum climb rate, however, decreases with altitude and is zero at the aircraft ceiling [145]. At sea level, the maximum climb rate for a small UAV is limited to approximately 5m/s [3]. This matches the maximum climb rate achieved using Γ0 under no wind conditions: max (nz δz ) nz = 5.08m/s ≈ 5m/s (6.8) min (nt δt ) nt ∈Nt
6.2.4
Simulation World
The simulation worlds are generated randomly to enable a Monte Carlo evaluation of the test algorithms. Each simulation world comprises a terrain map, no fly zones, other aircraft, storm cells, wind map and population density map (as a simple model of risk presented to people and property on the ground). For each world, a number of start and goal pairs were randomly chosen. The mission area for each world was arbitrarily chosen
122
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
to be 50NM×50NM×15000ft×90min with a cell resolution of 1NM×1NM× 1000ft×1min (1NM=1852m, 1ft=0.3048m). Note that the maximum distance of the search area approximately matches the maximum operating range of the RQ-7A Shadow UAV [3]. An artificial terrain map is randomly generated through summation of bivariate Gaussian functions with randomly chosen parameters (A, b, c, σ, n). Population density is also generated using this equation. z (x, y) =
n X
Ai e
−
(x−bi )2 +(y−ci )2 σ2 i
(6.9)
i=0
Airspace can be modeled with prisms and stacked cylinders representing the controlled airspace above aerodromes (refer Section 6.1.2). For prism shaped no fly zones, the parameters used to describe each region are the coordinates of each vertex. For controlled airspace, the only parameter required is the centre position of the cylinder stack as the radius and height of each cylinder in the stack is defined in aviation regulations. These parameters are randomly generated. The probabilistic model described in Section 6.1.1 is used to model other aircraft and storm cells. Parameters for this model, namely the position, velocity, standard deviations and volumetric description (radius and height) are randomly generated. For aircraft, the radius and height is defined by separation standards and the velocity is assumed to range between 50 knots and a speed limit (for flight below 10000ft) of 250 knots (128.6m/s) [7]. For storm cells, an average radius of 25km and height of 15km is assumed [158, Fig. 5]. The rate of movement of a storm cell is assumed to be between 5 to 20m/s for altitudes between 0 and 15000ft [158, Fig. 1]. Finally, a simple algorithm is used to generate wind maps that mimic real world winds. Firstly, a number of seed nodes are randomly generated at different positions (x, y) that contain a position ~vi , a direction φi and a vector of wind magnitudes m ~ z for each altitude level z. For each node s = (x, y, z) in the world space, a weighting vector w ~ is calculated wi = a0 d~i + a1 ∠d~i − φi
(6.10)
6.3. SIMULATIONS AND RESULTS
123
Figure 6.8: Example wind map. Note the increase in wind magnitude with altitude and slight variation in direction simulating wind shear.
where d~i = ~s − v~i and a0 , a1 are weights. The largest element in w ~ is then scaled by a∗ ; this gives the “winning seed” more weighting. For a given node s, the wind magnitude is fm (s) = w ~ ·m ~ z . The wind magnitude for each altitude level is randomly chosen based on average wind speeds from Bureau of Meteorology radiosonde measurements [154]. The direction fd is ~ + σz where σz is a small, random calculated in a similar manner, fd (s) = w ~ ·φ perturbation added to simulate wind shear. An example of a simulated wind map is shown in Fig. 6.8. Even though it is impossible to simulate all possible planning scenarios (which is infinite and impractical), the proposed set of Monte Carlo simulations emulate a range of expected operating conditions for the proposed medical package deliery mission. This, in combination with selected special case test scenarios, allows for a more comprehensive evaluation of the search algorithms than evaluation on a small number of case studies (as done in [4]).
6.3
Simulations and Results
This section discusses the results of the simulation study described in Section 6.2 where MSA* is evaluated against Vector A* (which is based on an existing vector neighbourhood based search algorithm [75]). A Monte Carlo simulation of the three tests algorithms was performed on 1000 randomly generated planning scenarios. The results of these simulations
124
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Figure 6.9: Example planning scenario showing no fly zones and other aircraft at t = 120s. Note for aircraft and weather, the inner cylinder represents the separation zone/storm cell extents (around the expected position) and the outer cylinder is the 2σ uncertainty boundary (which grows with time). Note also that a red X marks the goal position.
are presented below and evaluated with respect to computation time and path cost. In addition, the algorithms were also evaluated on three special case tests scenarios. These were constructed to determine the effect of local minima and to test the adaptability of the planner to situations where the vertical wind velocity, for instance, exceeds aircraft performance. Note that unless otherwise labeled, the results are presented for an admissible heuristic. An illustration of a typical multi-objective planning scenario is provided in Fig. 6.9 and Fig. 6.10 (showing other aircraft and no fly zones), Fig. 6.11 (risk map) and Fig. 6.12 (wind map). The solution path using Vector A*, MSA*1 and MSA*2 are also shown on each of these figures. In Fig. 6.9, all three planners select a path that avoids an aircraft on a converging course by descending and heading in an easterly direction (refer Fig. 6.10). Once a risk of collision is averted, the paths continue in a shortest path fashion towards the goal (marked by a red cross). There are deviations only to avoid terrain (where the paths hop over a mountain in Fig. 6.9) and route around high risk (population density) areas (Fig. 6.11). Note that, as shown in Section 5.1.4, each algorithm finds a path that satisfies all given constraints whilst minimising the overall path cost (which is a multi-objective cost function).
6.3. SIMULATIONS AND RESULTS
125
Figure 6.10: Example planning scenario showing a close up of no fly zones and other aircraft at t = 320s. Compare with Fig. 6.9.
Figure 6.11: Example planning scenario risk (represented by normalised population density) map.
126
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Figure 6.12: Example planning scenario wind map xy slice plane at z = 3500ft. Table 6.1: Computation time and loop count for Vector A*, MSA*1 and MSA*2.
6.3.1
µt
σt
µn
σn
MSA*2
4.46s
2.36s
65501
34250
MSA*1
9.23s
4.93s
161571
88386
Vector A*
19.25s
10.41s
289015
160575
Computation time
The mean and standard deviation for the computation time (µt and σt respectively) and the loop count (µn and σn respectively) is presented in Table 6.1 for each test algorithm. From the results, it can be deduced that a lattice based successor operator can significantly reduce total computation time and that further time savings can be achieved with a multi-resolution lattice. This can be confirmed by examining a normalised histogram of the computation time ratio as shown in Fig. 6.13. The mean µr and standard deviation σr of the computation time ratio between algorithms is listed in Table 6.2. For the test resolution level, Vector A*, MSA*1 and MSA*2 are all suitable for onboard replanning as the computation time is well within the minimum track traversal time of 1min. as specified in Nt .
6.3. SIMULATIONS AND RESULTS
127
Table 6.2: Ratio of computation time between MSA*1, MSA*2 and Vector A*.
µr
σr
MSA*2 to Vector A*
0.239
0.042
MSA*1 to Vector A*
0.484
0.050
MSA*2 to MSA*1
0.496
0.084
Norm. Freq. Plot of Comp. Time Ratio 16
MSA*1 to Vect A* MSA*2 to Vect A* MSA*2 to MSA*1
Normalised Frequency
14 12 10 8 6 4 2 0 0
0.5
Ratio
1
1.5
Figure 6.13: Normalised frequency histogram of computation time ratio between Vector A*, MSA*1 and MSA*2.
Note that both the computation time and loop count can be used to empirically quantify the computational complexity of a search algorithm. Unlike the absolute computation time, the absolute loop count (or number of search iterations) is platform independent. However, the loop count does not capture the complexity of each iteration of the search algorithm which limits the utility when comparing algorithms with substantially different runtime complexity. This can occur due to different successor operators, such as when comparing multi-resolution and uniform resolution planning. The computation time provides a more appropriate indicator of overall complexity; especially the ratio of the computation time to the benchmark (which is platform independent). Using the methodology described in Section 5.3.1, an admissible heuristic scaling factor of α = 0.008 was determined for MSA*1, MSA*2 and Vector A* through trial and error on 200 simulation scenarios. The mean and standard deviation of α for MSA*1, MSA*2 and Vector A* over all 1000 simulations is (0.0118, 0.0033), (0.0114, 0.0022) and (0.0127, 0.0038) respectively.
128
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Computation Time (s)
100
80
60
40
20
0 Vector A*
MSA*1
MSA*2
Figure 6.14: Statistical box plot of computation time for inflated heuristic variants of the three test algorithms. Note that the complete interquartile range lies within one second and the red crosses denote ‘outliers’. Table 6.3: Computation time and loop count for Vector A*, MSA*1 and MSA*2 with an inflated heuristic.
µt
σt
µn
σn
MSA*2
0.235s
3.392s
3856.7
56587
MSA*1
0.168s
1.748s
2888.6
35282
Vector A*
0.697s
6.002s
11023
98643
The effects of heuristic inflation were simulated by running the three test algorithms over the same Monte Carlo simulation scenarios with an inflated heuristic. A large inflation factor of = 4 is arbitrarily chosen for the selected α value since a satisficing solution is desired (as per the proposed replanning architecture). The resultant computation time is recorded in Table 6.3. It can be seen that there is a large spread in computation time which can be verified by examining a statistical box plot as shown in Fig. 6.14. Some of the outliers in Fig. 6.14 use more computation time than the optimal path search. Hence, it is necessary to switch to an admissible heuristic after a designated cut-off time. As the vast majority of test cases require less than one second of computation time (for MSA*1, the 98th percentile is 0.939s), a suitable cut-off time is one second.
6.3. SIMULATIONS AND RESULTS
129
Cumulative Frequency Plot of Cost Ratio Normalised Cumulative Frequency
1
MSA*1 to Vector A* MSA*2 to Vector A*
0.8
0.6
0.4
0.2
0 0
0.5
1 Cost Ratio
1.5
2
Figure 6.15: Cumulative histogram of relative path cost between MSA*1, MSA*2 and Vector A*. Table 6.4: Path cost ratio between MSA*1, MSA*2 and Vector A*.
6.3.2
µc
σc
MSA*1 to Vector A*
0.9891
0.0307
MSA*2 to Vector A*
1.0334
0.0376
Total path cost
The mean µc and standard deviation σc for the ratio of the path cost between MSA*1, MSA*2 and Vector A* is presented in Table 6.4. A cumulative histogram of the ratios is illustrated in Fig. 6.15. As the successor operator Γp~ in MSA*1 is largely similar to Γ in Vector A*, it is unsurprising to find that both return paths of approximately equivalent cost. Of particular note however, is the fact that, on average, MSA*2 finds paths that are only 3.3% more costly than Vector A*. Therefore, it can be seen that MSA* finds paths of equivalent cost but with significantly less computation time. As expected, an inflated heuristic results in a suboptimal solution as shown in Table 6.5. A cumulative histogram of the cost ratio to optimal cost Vector A* is shown in Fig. 6.16. It can be seen that when using an inflated heuristic, MSA*1 provides the best performance with the least computation time and lowest path cost. It is observed that each of the three test algorithms return a solution path that tends to follow the profile of a straight line (shortest path). This is
130
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Table 6.5: Path cost ratio between inflated heuristic MSA* and admissible Vector A*.
µc
σc
MSA*1 to Vector A*
1.121
0.082
MSA*2 to Vector A*
1.181
0.099
Cumulative Frequency Plot of Cost Ratio Normalised Cumulative Frequency
1 MSA*1 to Vector A* MSA*2 to Vector A* 0.8
0.6
0.4
0.2
0 0
0.5
1 Cost Ratio
1.5
2
Figure 6.16: Cumulative histogram of relative path cost between inflated heuristic MSA*1, MSA*2 and admissible Vector A*.
attributable to the minimisation process of A* and the fact that all trajectory segments have a non-zero and non-negative cost value. As a result, the turn angles are typically small. The mean turn angle µdθ and standard deviation in turn angles σdθ is listed in Table 6.6. The turn angles are small even without explicit optimisation of turn angles. Of the simulation scenarios where the maximum turn angle exceeds 90◦ , 90.6%, 96.1% and 81.3% of these cases for Vector A*, MSA*1 and MSA*2 respectively only contain a single turn of greater than 90◦ . This turn occurs on the transition to the last trajectory segment in the path like that shown in Fig. 6.17. A combined turn and altitude change occurs to ensure conformance with the cruising levels rule at the goal altitude.
6.3.3
Selected Scenarios
This section describes a number of selected simulation scenarios from the Monte Carlo simulations to demonstrate how the test algorithms respond to different planning scenarios.
6.3. SIMULATIONS AND RESULTS
131
Table 6.6: Turn angle statistics for Vector A*, MSA*1 and MSA*2.
µdθ
σdθ
Vector A*
11.72◦
19.01◦
MSA*1
12.71◦
22.42◦
MSA*2
16.42◦
21.93◦
Figure 6.17: Planning scenario where there is a turn angle of greater than 90◦ at the last trajectory segment. Note that the allowable track angles at 3500 and 5500ft are 0◦ to 179◦ , at 4500ft are 180◦ to 359◦ . Because the goal node is on a general North-Easterly heading from the start node, a turn of greater than 90◦ occurs in ensuing the track angle of the final track lies in the acceptable range.
Figure 6.18: Planning scenario where Vector A* returns a straight line path showing no fly zones and other aircraft at t = 1250s.
132
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
(a)
(b)
Figure 6.19: Planning scenario where Vector A* returns a straight line path showing (a) Normalised Population Density (NPD) and (b) wind in the xy plane at 5500ft.
Consider firstly a planning scenario where the least cost path found by Vector A* is the shortest path (i.e. straight line path). An example is illustrated in Fig. 6.18 and 6.19. For this scenario, the path cost is 1.08, 1.08 and 1.10 and the computation time is 28.4s, 14.1s and 7.25s for Vector A*, MSA*1 and MSA*2 respectively. MSA*1 returns an identical path to Vector A*, yet MSA*2 returns a path that is very nearly a straight line but flies at a much higher altitude. Inspection of the wind in this particularly rare simulation scenario shows that the headwind at higher altitude is approximately equal to that at lower altitude. MSA*2 chooses a path at this higher altitude as the track traversal time resolution (i.e. velocity resolution) is greater in this coarse resolution region (above 7000ft). Both Vector A* and MSA*1 choose a path with a track velocity of 43.65m/s which is the same as the track velocity achieved at higher altitudes in the path chosen by MSA*2. If the flight were to be conducted at 5500ft using MSA*2, the track velocity would either be 32.7m/s or 65.5m/s which results in longer traversal time and increased fuel consumption respectively. Hence, the path selected by MSA*2 is optimal for the chosen successor operator and the ratio of the path cost of MSA*2 to the other algorithms is 1.0185.
6.3. SIMULATIONS AND RESULTS
133
Figure 6.20: Aircraft and no fly zones plot for planning scenario with the greatest cost (for Vector A*, MSA*1 and MSA*2) out of all simulation scenarios. Note the need to fly ‘backwards’ initially due to terrain and no fly zone restrictions.
This particular scenario shows that even though MSA*2 provides a good approximation of the least cost path returned by Vector A* (and MSA*1), there are rare cases where an otherwise unexpected path is selected due to limited velocity resolution in the fine resolution region. Section 7.3 discusses an improvement in the design of Γ for MSA*2 based on simulation statistics. It is observed that even if the planning scenario produces a high cost path, as shown in Fig. 6.20, 6.21 and 6.22, the resultant path mostly adheres to a straight line where possible. In this planning scenario, Vector A* returns the most expensive path (cost of 2.57) compared to MSA*1 (2.52) and MSA*2 (2.54) despite a need to climb to high altitudes for the path returned by MSA*2. This can be attributed to the fact that Vector A* overflies higher population density regions (Fig. 6.21). Additionally, a greater tailwind at higher altitudes helps to reduce fuel consumption for the path chosen by MSA*2 (Fig. 6.22). The planning scenario that produces the greatest different in path cost between MSA*2 and Vector A* is shown in Fig. 6.23. Even though the overall flight path is quite similar and operated at the same altitude, the path chosen by MSA*2 overflies a higher risk region instead of routing around it and uses more fuel. This can be attributed to the limited velocity resolution of MSA*2 below 7000ft as identified previously. As a result, the 4D trajectory chosen by MSA*2 tends to consume more fuel per NM than the other algorithms. The
134
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Figure 6.21: Plot of risk for scenario with greatest path cost.
(a)
(b)
Figure 6.22: Wind at (a) 5000ft and at (b) 11500ft for planning scenario with greatest path cost—note the slight differences in direction and greater magnitude at high altitudes.
6.3. SIMULATIONS AND RESULTS
135
Figure 6.23: Risk plot of planning scenario with greatest difference in cost between MSA*2 and Vector A*.
path chosen by MSA*2 uses 32.59L of fuel whereas MSA*1, which chooses a longer path, only uses 31.59L. An example of a simulation scenario that is more computationally complex is provided in Fig. 6.24 and 6.25. In Fig. 6.24, all three paths bear to the left of the oncoming aircraft since there is a storm cell on the right. The flight continues in Euclidean shortest path fashion but, due to the high risk (population density) region surrounding the goal node, the aircraft must fly around it as shown in Fig. 6.25. This high population density region acts like a local minima since the Euclidean shortest path intersects the region of highest risk and is blocked. As a result, the search algorithm has to ‘fill up’ the ‘trap’ before escaping it (i.e. find a path around it). In this scenario MSA*1 and MSA*2 find lower cost paths (cost of 1.92 and 1.94 for 533977 and 188932 iterations respectively) than Vector A* (cost of 2.1 and 1050993 iterations). Numerical Tables of planned paths and more selected simulation scenarios can be found in Appendix B.
136
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Figure 6.24: Plot of other aircraft, storm cells and no fly zones for a computationally intensive planning scenario (one that is in the 99th percentile of all simulations in terms of the number of search iterations).
Figure 6.25: Risk plot for scenario depicted in Fig. 6.24.
6.3. SIMULATIONS AND RESULTS
137
Figure 6.26: Replanning scenario showing initial path at time 17.5min.
6.3.4
Illustration of Replanning Architecture
This section presents a case study planning scenario demonstrating the operation of the proposed self-scheduling replanning architecture in Section 5.3.1. The initial scenario and optimal plan, which is computed in 5.015s (88458 iterations) is shown in Fig. 6.26. At t = 17min. 26s, updated information on the status of other aircraft becomes available. Note the appearance of new aircraft in Fig. 6.27 compared to the original planning scenario in Fig. 6.26. The uncertainty region around each aircraft is re-initialised (i.e. set the standard deviation to the initial value) as there is updated information. This update triggers a replan where the initial, suboptimal solution is generated in 0.063s (920 iterations) and is shown in Fig. 6.27. The new path is enacted at t = 18min. As the time available for planning (i.e. time to the next waypoint) is only 18min.−17min. 26s − 0.078s = 33.922s (which is less than tplan = 1min.), admissible planning is not triggered until the aircraft reaches the first waypoint on the suboptimal plan. The start point used for admissible planning is thus the second waypoint on the inflated heuristic solution (at t = 21min.). The optimal plan is then found, taking 1.844s (29278 iterations) and is activated once the aircraft reaches the new start point (Fig. 6.28).
138
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
Figure 6.27: Replanning scenario showing a changed environment (new aircraft information) and inflated heuristic solution at t = 18min.
Figure 6.28: Replanning scenario showing the optimal path for the changed environment, which is enacted at t = 21min. Note how the chosen path descends below the other aircraft which is climbing.
6.3. SIMULATIONS AND RESULTS
Figure 6.29: Population risk for replanning scenario.
Figure 6.30: Wind plot at 3500ft.
139
140
CHAPTER 6. UAV FLIGHT PLANNING APPLICATION
It can be seen in this example that fast generation of an initial, satisficing solution is advantageous as it prevents a fixed wing aircraft from having to enter a hold manoeuvre to wait for a new plan. The updated plan avoids a potential collision scenario with another aircraft by descending (as shown in the optimal path, Fig. 6.28. Note that the near-optimal inflated heuristic path also descends and routes around the separation region of the other aircraft (Fig. 6.27). The static elements of this planning scenario, namely the wind forecast and normalised population density are shown in Fig. 6.29 and 6.30. Data tables for the planned paths can be found in Appendix B.
6.4
Findings
This chapter presented the medical package delivery mission using a small UAV. The decision objectives and constraints involved in operating this mission, such as separation management (with other aircraft), vehicle dynamic constraints and the rules of the air, are applicable to many civil UAV applications. A Monte Carlo simulation experiment is derived for this medical package delivery mission to evaluate the performance of MSA* compared to the benchmark, Vector A*. A fine resolution and multi-resolution lattice based implementation of MSA* (MSA*1 and MSA*2 respectively) are tested against this benchmark. It is shown that MSA*1 and MSA*2 finds paths of equivalent cost with a path cost ratio of 0.989 and 1.033 on average compared to Vector A* respectively. However, MSA*1 and MSA*2 use significantly less computation time with a computation time ratio of 0.484 and 0.239 on average compared to Vector A* respectively. Because MSA* always plans from scratch, validation of admissible and inflated heuristic planning validates the two step planning process proposed in the self-scheduling replanning architecture (refer Section 5.3.1). However, an illustration of the architecture is provided to demonstrate how the MSA* framework could work in practice. Validation of the overall replanning architecture within an intelligent control architecture is beyond the scope of the research. The proposed architecture is novel as it integrates a deliberative activity (planning) into the control loop in a manner suited to an aircraft that is in constant motion.
Chapter 7 Analysis and Extension When you are courting a nice girl an hour seems like a second. When you sit on a red-hot cinder a second seems like an hour. That’s relativity. Albert Einstein
T
HE proposed Multi-Step A* (MSA*) methodology is reviewed with respect to the objectives of the research given the analytical results in Chapter 5 and simulation results in Chapter 6.3. This chapter analyses and extends upon the verification and validation results in terms of the following categories: special planning cases, online replanning, the lattice structure, path cost and path uncertainty.
7.1
Special Planning Cases
It is widely acknowledged that A* and best-first search algorithms (in general) require significantly more computation time in the presence of local minima [12]. This is tested for the single and double bug trap case as shown in Fig. 7.1 and Fig. 7.2 respectively. The computation time is recorded in Table 7.1 for an admissible heuristic solution. It can be seen that even though the absolute computation time is approximately double to 2.5 times the mean obtained in the previous Monte Carlo simulations, the relative computation time between Vector A*, MSA*1 and MSA*2 remains approximately the same as before. A simulation scenario which mimics the presence of strong up/downdrafts in mountainous regions (where the vertical wind velocity can exceed the vehicle’s climb rate) is depicted in Fig 7.3. Even though a variety of wind 141
142
CHAPTER 7. ANALYSIS AND EXTENSION
Figure 7.1: Single bug trap case.
Figure 7.2: Double bug trap case.
Table 7.1: Computation time for special case test scenarios.
Single Bug Trap
Double Bug Trap
Time
Loop Count
Time
Loop Count
MSA*2
9.59s
201197
9.17s
194981
MSA*1
23.45s
515657
22.61s
503871
Vector A*
51.14s
922544
50.65s
926276
7.2. ONLINE REPLANNING
143
Figure 7.3: Mountain wind simulation. The solution is found in 5.14s.
conditions were simulated in the previous Monte Carlo experiment, this experiment specifically studies the effect of wind by setting other decision variables (e.g. no fly zones, other aircraft, storm cells, risk) to zero. As shown in Fig. 7.3, only MSA*2 successfully finds a traversable path that satisfies aircraft climb constraints. The chosen path climbs in a switchback pattern before ‘hopping over’ the mountain. Recall that as the aircraft’s maximum climb rate decreases with altitude, it is necessary to climb to 13500ft to accommodate loss of altitude in the downdrafts region. The reason that MSA*2 successfully finds a path while Vector A* and MSA*1 fail is because of the higher climb and descent rate required by the fine resolution successor operator. The same vertical distance is covered in a shorter time using the fine resolution operator (e.g. 1000ft in 2, 3 or 4min.) compared to the coarse resolution operator (e.g. 1000ft in 4, 6 or 8min.).
7.2
Online Replanning
The primary objective of the simulation experiments presented in Chapter 6.3 is to determine whether the algorithm is fast enough for online replanning. Online replanning is needed to mitigate the uncertainty and unpredictably of an outdoor operating environment. An indicator of the available planning time can be derived from the minimum track traversal time as the motion
144
CHAPTER 7. ANALYSIS AND EXTENSION
plan is made up of discrete trajectory segments (i.e. tracks). For the successor operator selected in Fig. 6.7, the mean planning time for MSA*2, MSA*1 and Vector A* (of 4.46s, 9.23s and 19.25s respectively) was found to be much smaller than the minimum track traversal time (1min). All three tests algorithms were also shown to be capable of finding a path within the time constraints of online replanning for environments containing deep local minima and narrow escape passages (as per Fig. 7.1 and Fig. 7.2). In these experiments, it is found that the proposed methodology offers an approximately two-fold reduction in computation time. Further reductions were obtained by using a multi-resolution lattice. This increased speed is especially significant when planning over a larger search space or on a more difficult search space. For a 100NM×100NM×15000ft×180min. search space, a similar Monte Carlo experiment found that MSA*2 is still able to meet the requirements for online replanning with a mean planning time of 48.25s and standard deviation of 20.62s (which is less than the minimum track traversal time of 2min.). However, Vector A* does not as it takes approximately four times the computation time. Similarly, when presented with local minima, which are known to be difficult to solve for best first search algorithms like A*, the computational efficiency of MSA* (201197 search iterations) over Vector A* (922544 search iterations) is significant in meeting online planning constraints [12]. Furthermore, as is widely reported in the literature, it is observed that an inflated heuristic can drastically reduce computation time. The mean planning time is less than one second for all three test algorithms which makes an inflated heuristic solution suitable for quickly finding a satisficing path. Note that a satisficing path still satisfies all path constraints (such as climb rate constraints, maximum risk constraints) and is on average only 12% and 18% (for MSA*1 and MSA*2 respectively) more expensive than the least cost path found using Vector A*. However, as noted in Fig. 6.14, there are a non-negligible number of scenarios where the computation time is significantly greater than the time required to find an optimal solution. This is a result of state re-expansion due to an inadmissible heuristic as noted in [14]. As a result, it is necessary to switch to an admissible search after some time tcutoff as per the proposed replanning architecture (Section 5.3). From simulation results, a suitable value for tcutoff is 1s.
7.3. LATTICE STRUCTURE
145
The previous discussion of algorithm computation time assumes that in each case, the algorithm plans from scratch. This approach of always discarding previous planning information was adopted because in applications such as Unmanned Aerial Vehicle (UAV) package delivery, online changes can occur anywhere in the search map and affect large swathes of the search space. If a large number of nodes are changed and/or changes occur close to the goal node, replanning algorithms like D* and D* Lite are less efficient than one that plans from scratch [33]. The presence of fast moving aircraft and storm cells for example can affect large areas of the search space that are not necessarily localised around the vehicle’s current position. Hence, it is more efficient to plan from scratch each time. Due to the time critical nature of online replanning, it is preferable to use a fast and near-optimal planner rather than an optimal planner which may be too slow. Under these conditions, MSA*2 is the best candidate for online replanning out of the three test algorithms as it provides a near optimal solution in the shortest time. It was theoretically proven in Section 5.1.4 that MSA* finds a solution in finite time if one exists. The Monte Carlo simulation further shows that MSA* meets the computational efficiency requirement of the research objective for online replanning as the computation time is less than the minimum track traversal time. Even if the goal node is unreachable (i.e. a solution does not exist), only a finite number of iterations (502959, 297686 and 110590 iterations on average for Vector A*, MSA*1 and MSA*2 respectively) are expended to detect this.
7.3
Lattice Structure
The computational efficiency of MSA* compared to Vector A* can be attributed to a smaller, lattice structure based search space (Vector A* uses a 4D grid). With the exception of the successor operator and cell sequence based sampling, Vector A* is virtually identical to MSA*. Hence, as shown in section 5.1.3, both algorithms have virtually the same computational complexity given the same successors and sampling sequence. However, given the same computing platform, it is possible to evaluate 14906 iterations of Vector A* per second (on average) compared to 17400 iterations of MSA*1 per second (on average). This is because the number of successors for each
146
CHAPTER 7. ANALYSIS AND EXTENSION 6
3.5 x 10
Number of Nodes
3 2.5 Λz =1 2
Λz =2 Λz =3
1.5 1 0.5 1
Λz =4 Λz =5 Λz = −∞ Grid 2
3
4
5 6 Λx = Λy
7
8
9
10
Figure 7.4: Number of nodes in the lattice for different values of Λ. Note that Λz = −∞ corresponds to the constrained vertical track angle case described in Section 5.2.1.
position in the lattice (using MSA*) is less than or equal to the number of successors in the base successor operator position (refer Fig. 5.4). Hence, on average, there are fewer successors to evaluate per iteration for MSA* (i.e. less complexity per iteration). The disadvantage of a lattice based successor operator is that the track angles in Γ(0,0,0) are not identical to those at Γp~ where p is some other lattice position. This can potentially result in extra turns as demonstrated by the mean turn angle across all simulations. Note that the mean turn angle is an indirect indicator of both the number of turns and the turn magnitude. For Vector A*, the mean turn angle is 11.8◦ compared to 12.8◦ for MSA*1 and 16.4◦ for MSA*2. Using equation (5.16), a plot of the number of nodes for a search space of dimensions 50×50×15×90 given different values of Λ assuming Λx = Λy is shown in Fig. 7.4. Note that the memory required in a full grid corresponds to the case where Λx = 1, or Λy = 1, or Λz = 1. From Fig. 7.4, it is evident that larger values of Λ produce a lattice structure with fewer sample nodes. This results in a smaller search space and hence fewer search iterations. As noted in Table 6.1, MSA*1 (fine resolution lattice) takes 161571 iterations on average compared to 289015 for Vector A* (full grid) and 65501 for MSA*2 (multi-resolution lattice).
7.3. LATTICE STRUCTURE
147
However, more computation is required to evaluate the longer trajectory segments in coarse resolution regions (in MSA*2) since the cell resolution is unchanged. Furthermore, the coarse resolution successor operator has potentially more successors per node with greater track angle resolution and track length. As a result, fewer search iterations are required but each iteration incurs more computation time. For example, it is possible to evaluate 17400 nodes per second (on average) on the chosen computing platform using MSA*1 whereas only 14750 nodes per second are possible (on average) using MSA*2. It can be seen that the variable successor operator Γs is a crucial, application-specific design parameter that influences the path cost, the traversability of the path and the computation time. For the demonstration UAV application, it has been shown that the selected fine resolution and especially the multi-resolution successor operators are effective at delivering a solution of comparable cost with significant savings in computation time. The use of a coarse resolution successor operator Γs for high altitudes in MSA*2 is especially suited to UAV planning because of the scarcity of obstacles and reduced climb rate at high altitudes (refer Section 7.1). However, there are a small number of simulation scenarios (refer Section 6.3.3) where the MSA*2 successor operator has insufficient velocity resolution given the application and UAV platform. An improved successor operator is proposed based on the one used in MSA*2. By examining the frequency with which different track traversal times ntl are selected by MSA*1 in simulation (Table 7.2), it is found that typically, only two out of the three candidate traversal times (i.e. velocities) in Ntl are selected. Let the improved MSA* variant be known as MSA*3. As with MSA*2, a fine resolution 3D successor operator based on Fig. 6.7 is used below 7000ft and a coarse resolution operator above 7000ft. The traversal time set Ntl for each successor is defined using the results obtained in Table 7.2. Given the horizontal displacement of the successor from the source node, the two trajectory traversal times with the highest frequency in Table 7.2 are selected (highlighted in bold). For example, trajectory segments of horizontal length 2NM occur at track angles of 0◦ , 90◦ , 180◦ and 270◦ . For these trajectory segments, Ntl = {1, 2}min. In this way it is possible to derive the 4D successor operator Γp~ for the fine resolution region since Table 7.2 is derived from MSA*1. The
148
CHAPTER 7. ANALYSIS AND EXTENSION Table 7.2: Frequency of different track traversal times ntl .
ntl (min.) xy track length (NM)
1
2
3
4
1.000
384
0
0
0
2.000
9
18
0
0
2.236
25
915
3
0
3.000
0
717
105
2
3.162
0
2325
484
5
3.606
0
1075
3824
198
4.243
0
5
3427
447
Table 7.3: Horizontal track velocity in m/s for different track traversal times ntl .
ntl (min.) xy distance (NM)
1
2
3
4
1.000
30.9
15.4
10.3
7.7
2.000
61.7
30.9
20.6
15.4
2.236
69.0
34.5
23.0
17.3
3.000
92.6
46.3
30.9
23.2
3.162
97.6
48.8
32.5
24.4
3.606
111.3
55.7
37.1
27.8
4.243
131.0
65.5
43.7
32.7
4D fine resolution lattice thus has Λx = 3, Λy = 3, Λz = −∞, Λt = 1 since ntl can be an even or odd number of minutes. For the coarse resolution region, it is desirable to replicate the 4D lattice of MSA*2 by setting Λx = 6, Λy = 6, Λz = −∞, Λt = 2. This reduces the number of search nodes and aids in reducing computation time. As with the fine resolution region, two candidate traversal times ntl are selected for each 3D successor (as opposed to three in MSA*2). For successors that are at a multiple k = 1, 2, . . . of the distances listed in Table 7.2, Ntl is simply k times the corresponding ntl values chosen for the fine resolution operator. Where the horizontal track length is not a multiple of one of the ones in Table 7.2, a simple heuristic is used to select ntl . This heuristic is based on the observation that the horizontal track velocity for the most frequent ntl
7.3. LATTICE STRUCTURE
149
Normalised Frequency
20
MSA*1 to Vect A* MSA*3 to Vect A* MSA*3 to MSA*1
15
10
5
0 0
0.5
Ratio
1
1.5
Figure 7.5: Computation time ratio of MSA*3 to Vector A* and MSA*1.
values in Table 7.2 correspond to ones that have the lowest track velocity greater than 30m/s (refer Table 7.3). Note that as is the case with a 3D lattice, a trajectory must terminate or originate from a 4D hyper-plane that is on the boundary of the coarse and fine resolution regions. In other words, when transitioning from a fine to coarse resolution region, the successor time level must be an even number of minutes (since Λt = 2). MSA*3 is evaluated on the same Monte Carlo simulations as the three test algorithms. A histogram plot of the computation time ratio is shown in Fig. 7.5 and a plot of the cost ratio in Fig. 7.6. MSA*3 has a mean and standard deviation in computation time of 5.99s and 2.92s respectively and a mean and standard deviation in the number of search iterations of 95019 and 47233 respectively. The mean and standard deviation in the cost ratio to Vector A* is 1.005 and 0.038. The mean computation time ratio compared to Vector A* is 0.329 (standard deviation of 0.069). Thus, MSA*3 is actually slower than MSA*2. This can be attributed to a larger search space (recall that Λt = 2 in the fine resolution region of MSA*2 whereas Λt = 1 in MSA*3). Furthermore, even though there are now only a maximum of 240 successors (or neighbours) per source node compared to 360 previously, it is noted that many of the
150
CHAPTER 7. ANALYSIS AND EXTENSION
Normalised Cumulative Frequency
1
MSA*3 to Vector A*
0.8 0.6 0.4 0.2 0 0
0.5
1 Cost Ratio
1.5
2
Figure 7.6: Cost ratio of MSA*3 to Vector A*.
successors derived from Γ0 in Fig. 6.7 correspond to more extreme track velocities. This is shown in Table 7.3. These successors would only be feasible in scenarios where there is sufficient headwind or tailwind such that the airspeed is within aircraft performance constraints. Hence, even though there is a maximum of 360 successors, the number of feasible successors is not much different to that in MSA*3. Despite the added computational cost, MSA*3 still meets the requirements for online replanning (the mean planning time of 5.99s is much less than the minimum track traversal time of 1min). Furthermore, it still uses less computation time (and search iterations) than MSA*1 and Vector A*. The greatest advantage of MSA*3 over MSA*2 is that it resolves the idiosyncrasies of insufficient velocity resolution described in Section 6.3.3. Furthermore, the mean turn angle for MSA*3 is 14.8◦ which is less than that for MSA*2 (16.4◦ ). Hence, it can be seen that the average turn angle is affected by not just the track angles in the successor operator, but also the track velocity (since the MSA*3 successor operator has identical track angles to that in MSA*2). For the planning scenario depicted in Fig. 6.18, MSA*3 now returns the same straight line path as Vector A* as shown in Fig. 7.7. In the high cost
7.3. LATTICE STRUCTURE
151
Figure 7.7: Planning scenario where Vector A*, MSA*1 and MSA*3 return a straight line path.
Figure 7.8: Planning scenario with greatest cost where MSA*3 returns the lowest cost path.
152
CHAPTER 7. ANALYSIS AND EXTENSION
Figure 7.9: Risk plot of planning scenario with greatest difference in cost between MSA*3 and Vector A*.
planning scenario, MSA*3 still returns a path that climbs to higher altitudes (Fig. 7.8) but is of lower cost (2.50) than Vector A* (2.57) and MSA*1 (2.52). This can be attributed to the fuel efficiency at higher altitudes as described in Section 6.3.3 and the more gradual climb rate describe in Section 7.1. For the scenario depicted in Fig. 6.23, MSA*3 returns a shorter path that consumes less fuel than MSA*1 (30.6L), but still overflies the higher risk region at the end (Fig. 7.9). This is partially due to the low weighting given to population density (refer Fig. 6.6). As shown in Fig. 7.10, MSA*3 finds a path that is of greater similarity to MSA*1 for the computationally expensive scenario depicted in Fig. 6.24. However, MSA*3 uses more iterations than MSA*2 for this scenario (230305 compared to 188932 for MSA*2) and the total path cost is only marginally smaller than MSA*2 (1.93 compared to 1.94). Finally, for the mountain wind scenario depicted in Fig. 7.3, MSA*3 returns the same path as MSA*2. In the single and double bug trap (local minima) scenarios (Fig. 7.1 and 7.2 respectively), MSA*3 takes 15.01s/299909 iterations and 14.34s/292420 iterations respectively. Numerical data tables of planned paths can be found in Appendix B.
7.3. LATTICE STRUCTURE
153
Figure 7.10: Plot of path found using MSA*3 showing other aircraft, storm cells and no fly zones for a computationally intensive planning scenario.
Note that even though the proposed lattice sampling structure enables significantly reduced computation time, it is impossible to escape the “curse of dimensionality” [11]. Regardless of the sampling scheme employed (probabilistic or deterministic), if a constant (or minimal) dispersion is required, then there is always exponential growth in memory and time with dimensionality [29]. Hence the selection of a 4D search space for vehicle motion planning instead of a higher dimensional search space that incorporates the degrees of freedom of the vehicle and decision variables (e.g. heading angle, fuel, risk probability). Unfortunately, this means that it is not possible to find an optimal path whilst enforcing a ‘total fuel’ constraint. Typically, it is assumed that there is sufficient fuel to carry out a mission. It is possible to indirectly enforce this constraint by running a search, checking the total fuel usage and if the limit is exceeded, increase the weighting on the fuel criterion and search again. This is repeated until either a path is found, or until the maximum fuel weighting or maximum number of iterations is reached (refer [159]). The lattice structure presented here is similar to the framed quad/octree presented by [77]. A key improvement in terms of 3D sampling is that the proposed lattice comprises sample planes that are one cell wide (Fig. 5.4) whereas those in framed octree are two cells wide (Fig. 4.2). Hence, there are fewer search nodes, which implies less memory and computation time. Additionally, the track angle in a framed octree is constrained to intervals
154
CHAPTER 7. ANALYSIS AND EXTENSION
of 45◦ when transitioning between quadtree nodes. Finally, the proposed method guarantees path soundness by sampling each trajectory segment at the grid resolution, thus avoiding the problem of mixed cells. Furthermore, unlike [77], no pre-processing (i.e. cell decomposition) is required with the proposed method. Even though lattice based motion planning algorithms have been presented in the past (e.g. [29, 75]), none of them have been applied to a multiobjective 4D motion planning problem. This lattice structure is used as a memory efficient sampling methodology that can encode track angles. The successors for any node on the lattice can be easily determined due to the regularity of the lattice. This regularity also reduces the total number of unique successor operators Γs as shown in equation (5.13) and (5.15). As a result, the successor operator can be stored in a Look-Up Table (LUT) which reduces computation time.
7.4
Path Cost
MSA* was shown to find the least cost path with an admissible heuristic (Section 5.1.4). Through Monte Carlo simulations and special case demonstrations, it was further shown that careful selection of Γs can produce paths of equivalent cost but with significantly less computation than high resolution Vector A*. It is observed that in addition to the requirement for monotonicity, the cost function c must also be additively consistent. This prevents biased trajectory segment costs because of a variable successor operator Γs and variable track lengths for each successor in Γs . An additively consistent cost function c is defined as follows. Given any two nodes s, s0 and intermediary nodes s00i such that Γp~ (s) = {s, . . . , s0 } = Γp~0 (s000 = s), . . . , Γp~k−1 (s00k−1 ) , c(s0 ks, . . . , s0 ) = c(s001 ks000 = s, . . . , s001 ) + . . . + c(s00k ks00k−1 , . . . , s00k = s0 )
(7.1)
Without additive consistency, the search algorithm is unfairly biased to select longer (or shorter) trajectory segments due to the equation for calculating g (5.9). This effect is especially evident in multi-resolution planning.
7.4. PATH COST
155
For the previous Monte Carlo simulation, MSA*2 finds paths with an average cost ratio compared to Vector A* of 1.033 with an additively consistent cost function. Without additive consistency, the cost ratio rises to 1.139 for the case in (7.1) where LHS < RHS (Left Hand Side is less than the Right Hand Side). Inspection of simulation results confirmed the presence of bias as the planner tends to select paths containing longer trajectory segments in the coarse resolution region. Without additive consistency, the ratio of coarse to fine resolution trajectory segments over all simulation scenarios is 0.854, but is only 0.258 with additive consistency. Therefore, additive consistency is paramount to any practical implementation of a planning algorithm where there is significant variation in length between successor trajectory segments. Consider the multi-objective cost function used for the previous UAV flight planning application. The decision variables used to evaluate the cost can be classified as either cumulative or non-cumulative. Cumulative variables are those like fuel and risk where the total value for a trajectory segment is found by integrating the fuel rate over the length of the track or integrating the risk density over the cell sequence. Thus, the total value of the decision variable is always a summation of the value at each cell or increment in the trajectory segment. If the cost function is also a linear function of cumulative variables, it can be seen that (7.1) is always satisfied. In UAV flight planning, the cumulative variables are fuel, aircraft separation risk, storm cells risk and population risk (modeled with population density). On the other hand, it is much more difficult to ensure additive consistency for non-cumulative variables. An example of a non-cumulative variable is the cruising levels variable ccl . At altitudes below 5000ft, if the track does not conform to the cruising levels rule, then a constant penalty value is assigned to ccl ; otherwise, ccl = 0. Given two nodes (s, s0 ), the LHS in (7.1) will be less than the RHS as the effect of ccl is added multiple times on the RHS. A simple method for enforcing additive consistency is to transform all non-cumulative variables into cumulative variables. The cruising levels rule can thus be transformed by multiplying ccl by the length of the track. Another non-cumulative decision variable is the time of arrival at a successor node. Even though both the time of arrival and the transition time can be used to address the decision objective of minimising flight time, the time of arrival has the added benefit of increasing in value as time passes.
156
CHAPTER 7. ANALYSIS AND EXTENSION
Therefore, given a constant weighting (again assuming a weighted sum cost function), it is possible to obtain increasing weighting for the time decision criterion with flight time. This can be used to implicitly enforce maximum flight time and fuel consumption. The time of arrival decision variable can be transformed into a cumulative variable by taking the sum of the time of arrival at each cell for all cells on the cell sequence. Despite the proliferation of multi-resolution planning algorithms and extensive literature on multi-objective planning (refer Section 4.1.2), there is little mention of the effects of aggregating multiple decision criteria in a multi-resolution search. This work, to the best of the author’s knowledge, is the first to identify the need for additive consistency to prevent path bias. However, it is not possible in general to guarantee additive consistency for non-linear cost functions such as the Choquet integral and fuzzy logic. Hence the reason for selection of the weighted sum cost function. Finally, a generic shortcoming of A* and A* related algorithms (e.g. D*, D* Lite) in general is that the solution path is a disjunction of decisions. This is because the overall path cost is determined by taking the disjunction (i.e. sum) of all individual trajectory segment costs on the path as per the dynamic programming principle (refer equation (2.6)). As a result, it is possible (but rare) that the path may contain an undesirable trajectory segment of higher cost which is offset by other lower cost segments. The rarity of this scenario is shown in Fig. 7.11. Of the outliers in Fig. 7.11, the segments of greater cost often come as a result of violation of the cruising levels rule (below 5000ft) on the last trajectory segment (which is the case in Fig. 6.23 and Fig. 6.24 – for more details see Appendix B). One method for overcoming the problem described above is to use a conjunctive cost operator such as that used in fuzzy dynamic programming (Section 2.3.1). However, existing methods for solving the fuzzy dynamic programming equation are computationally intractable for UAV Mission Flight Planning (MFP) (especially on a small UAV).
7.5
Uncertainty
MSA* returns a path comprising a sequence of cells which form a corridor in 4D space around the planned trajectory. This differs from existing vector
7.5. UNCERTAINTY
157
Figure 7.11: Histogram of trajectory segment cost across all MSA*1 simulations. Note the rarity of segments with a cost of more than 0.2 - out of 13968 segments, 13 exceed a cost of 0.2.
neighbour based methods like [75] which do not explicitly associate cells or a volume of space with each trajectory segment. Such a cell sequence provides an inherent tolerance to uncertainty. This approach avoids the intractability of directly incorporating uncertainty into the search space (using methods such as Markov Decision Processes) for a large, high dimensional search space [12]. The level of tolerance can be determined by finding the minimum perpendicular distance d between the track and the cell boundaries for each cell −→ on the trajectory segment. This is shown in Fig. 7.12 where AB is the tra−−→ jectory segment and DC is the perpendicular distance to an exterior corner of a cell on the sequence γs . An exterior corner is one that is not completely enclosed by adjacent cells. The angle θ can be determined by the dot product of vector AB and AC, −→ −→ −→ −→ AB · AC = AB AC cosθ
(7.2)
In 4ADC, as ∠ADC = 90◦ (by definition), then AD = AC cos θ. The position of D can then be determined using a simple vector line equation and line AD −→ AB −→ ~ D = A + −→ AC cos θ (7.3) AB
158
CHAPTER 7. ANALYSIS AND EXTENSION
Figure 7.12: Illustration of a cell in the cell sequence for a given trajectory segment AB.
Because the cell sequence generated using (5.2) only includes cells that intersect the track, if D does not lie within the cell, then that particular corner is ignored. Otherwise, the perpendicular distance DC is −−→ −→ DC = AC sin θ
(7.4)
The value of d is determined by taking the minimum DC value for all corners of all cells in γs . Note that θ = 0 implies that the trajectory intersects a cell edge/corner in which case all adjacent cells were already included in the cell sequence. Hence, this implies a non-exterior cell corner. For the successor operator depicted in Fig. 6.7 and a 3D cell size of 1852m×1852m×304.8m, the minimum 3D tolerance is 50.6m, and the minimum horizontal (xy) tolerance is 256.8m. Note that all transition manoeuvres (i.e. turns) needed to transition between tracks are assumed to be of negligible cost compared to the tracks themselves. These manoeuvres are assumed to stay well within the boundaries of the cell sequence. It is possible to modify the cell sequence returned by the method described in Section 5.1.1 to enforce a minimum tolerance constraint (3D or horizontal only). For each exterior corner of each cell that does not satisfy the distance constraint, it is a simple matter to include all cells adjacent to that corner to increase the minimum tolerance. This procedure is repeated until all exterior corner points satisfy the minimum tolerance. The time level of each added cell can be determined in the same manner as in (5.5). The preceding discussion shows that MSA* provides an arbitrarily adjustable tolerance to uncertainty. In addition, it is shown in Section 7.2 that MSA* is fast enough for online replanning. Online replanning mirrors the re-
7.6. FINDINGS
159
duction step of Reduction, Assumption, Weighing, Forestalling, Suppressing (RAWFS) as it uses new, up to date information to create a new plan. The planning process is itself a form of assumption based reasoning as it involves simulation of a sequence of decisions. Finally, the use of a Multi-Attribute Utility Theory (MAUT) based Multi-Criteria Decision Making (MCDM) cost function provides a means for replicating the weighting aspect of RAWFS. It can be seen that the overall approach to accommodating uncertainty is through heuristics (RAWFS heuristic), tolerance and replanning. This approach is computationally efficient unlike existing methods in decision theoretic planning (Section 2.4) and does not require artificial constraints on the planned path (such as pinch points [90]).
7.6
Findings
The applicability of MSA* to UAV MFP is demonstrated and validated through theoretical proof (Section 5.1.4) and simulation results (Chapter 6.3). Analysis reveals that the computational efficiency of the test MSA* algorithms over Vector A* arises from the lattice structure. A lattice has significantly fewer search nodes than a grid which enables a solution to be found with fewer iterations. As a result, lattice based MSA* is able to find a solution in significantly less time than Vector A*, even with the added complexity of multi-resolution planning. Note that the uniform resolution lattice is shown to have, on average, less complexity per iteration than Vector A*. Such computational efficiency enables the MFP system to meet the time constraints of online replanning for large search spaces and difficult search spaces (such as those containing local minima). Analysis of the results led to an improved successor operator which was implemented for MSA*3. Even though MSA*3 is slower than MSA*2 when compared to Vector A* (computation time ratio to Vector A* of 0.329 on average compared to 0.239 for MSA*2), MSA*3 finds paths of lower cost. The cost ratio compared to Vector A* is 1.005 compared to 1.033 for MSA*2. More importantly, MSA*3 avoids the problems of insufficient velocity resolution found in MSA*2. In depth analysis of simulation results also revealed the need for additive consistency. This property avoids path bias and ensures that the algorithm
160
CHAPTER 7. ANALYSIS AND EXTENSION
finds the least cost path. For example, MSA*2 finds paths with a cost ratio to Vector A* of 1.033 with additive consistency and 1.139 without. Additionally, it is shown that the cell sequence used in MSA* provides a quantifiable tolerance (measured as a 2D or 3D tolerance) to uncertainty. This cell sequence can be modified to provide an arbitrary tolerance. MSA* is just one component (albeit the primary component) of the proposed MFP framework which in turn is just one subsystem in an intelligent control architecture. It is shown that MSA* addresses the requirements of multi-objective decision making, planning uncertainty and computational efficiency (research objective one). This is also the case for the inflated heuristic variant of MSA* as it is identical to admissible MSA* but finds a satisficing solution (Section 7.2) in significantly less time. The UAV MFP framework described here helps to increase the level of onboard autonomy by providing a means for responding automatically to changes in the operating environment. This is especially useful in the event of a communications link failure.
Chapter 8 Conclusion Our scientific power has outrun our spiritual power. We have guided missiles and misguided men. Martin Luther King Jr.
T
HIS research derived Multi-Step A* (MSA*), a method for motion planning using a variable successor operator that finds least cost paths. MSA* provides the basis for an automated, Mission Flight Planning (MFP) framework which is a key enabler in the operation of Unmanned Aerial Vehicles (UAVs) in the National Airspace System (NAS). UAV motion planning is characterised by a large and dynamic outdoor environment, movement in three spatial dimensions, presence of wind and differential constraints on vehicle movement. Additionally, the UAV must conform with the rules of the air. For the purposes of validation, an example medical package delivery mission was simulated with the following decision criteria: • • • • • • • •
Aircraft separation risk Storm cell risk Population risk Low flying rule CAR157 Cruising levels rule CAR173 No fly zones (based on airspace classes) Flight time (function of wind and selected cruise velocities) Fuel
The key difference between the derived search algorithm and methodology MSA* and A* [73] is the use of a variable successor operator. A 161
162
CHAPTER 8. CONCLUSION
variable successor operator enables variable track length, angle and velocity trajectory segments that are modeled using a computer graphics inspired cell sequence. The ability to define arbitrary track angle resolution and define arbitrary cruise velocities is important for 4D vehicle motion planning due to the presence of wind and dynamic obstacles. This capability for variable velocity trajectory segments with multiple decision variables is unmatched in existing work (which typically assume constant track velocity, e.g. [45, 124]). Furthermore, the modeling of trajectory segments with a cell sequence provides an inherent tolerance to uncertainty. The tolerance is measured using the minimum distance between the track and cell sequence boundaries. A variable successor operator also enables the imposition of a multiresolution lattice structure on the search space which drastically reduces the number of search nodes and search time. The proposed lattice structure is more memory efficient than framed quad/octrees [77] and guarantees a sound path (since there are no mixed cells). Extensive simulations for a UAV flight planning task reveal that multiresolution MSA* is approximately three times faster than vector neighbourhood based A* (Vector A*) but returns paths of approximately the same cost (average path cost ratio of 1.005). Even with a uniform, fine resolution lattice, MSA* is still twice as fast as Vector A* with an average path cost ratio of 0.99. Note that existing Vector A* based methods (e.g. [75]) do not address the multi-objective planning requirements of MFP and had to be extended to meet MFP requirements. It is shown that MSA* is suited to online replanning with an average computation time (5.99s for multi-resolution MSA*) that is a fraction of the minimum track traversal time (1min.). MSA* and A* are shown to have comparable complexity given the same successor operator and sampling sequences (for determining the cost of each trajectory segment). It is further shown that lattice based MSA* has less complexity per iteration (on average) than A*. In addition, a multi-resolution lattice structure greatly reduces the number of search nodes which produces a significant improvement in computational performance. This improvement is crucial for meeting the time constraints of online replanning in large MFP environments and difficult planning environments (i.e. those containing local minima).
163 Simulation studies also show that by inflating the heuristic ( = 4), a nearoptimal solution (12% more expensive than the optimal on average) can be obtained in a mean time of 0.168s for MSA*. Such a satisficing solution still meets all the MFP constraints (such as dynamic constraints, risk constraints, constraints imposed by the rules of the air). The purpose of quickly finding a satisficing solution is to minimise the need for hold manoeuvres in cases where the time available for planning is less than the maximum expected planning time. Through a combination of inflated and admissible heuristic search, MSA* meets the requirements for online replanning. Online replanning is itself used to mitigate a dynamic and uncertain operating environment. MSA* was theoretically proven to find least cost paths (given a chosen successor operator) where a Multi-Criteria Decision Making (MCDM) cost function is used to ‘aggregate’ the decision variables into a single cost value. It was shown through a range of case study simulation experiments that MSA* successfully finds paths in a variety of complex scenarios. MSA* was shown to be able to find straight line paths, route around regions of higher risk, avoid dynamic obstacles and select flight altitudes that maximise fuel efficiency. In addition, it was shown to be able to make trade offs between decision variables such as population risk, fuel and time. However, the chosen path never exceeds specified constraints for each trajectory segment since such a segment is given infinite cost. Simulation studies also revealed the need for additive consistency to avoid bias in the selection of trajectory segments, especially for multi-resolution planning. This property is based on the premise that, given a particular cell sequence, the cost of traversal using a single trajectory segment is equal to that of using two or more shorter trajectory segments. As a result of additive consistency, a weighted sum MCDM cost function was chosen over non-linear cost functions such as the Choquet integral and fuzzy logic. Although additive consistency is crucial for multi-resolution planning, especially given multiple decision objectives, this property has not been previously identified in the path planning literature. It can be seen that MSA* addresses the requirements of UAV motion planning as it efficiently finds a path that considers multiple planning objectives under varying wind conditions. As well, the algorithm mitigates uncertainty through tolerance and its suitability to online replanning. It also
164
CHAPTER 8. CONCLUSION
replicates the Reduction, Assumption, Weighing, Forestalling, Suppressing (RAWFS) heuristic observed in human expert decision making under uncertainty through weighted sum MCDM (weighing pros and cons). Existing methods for motion planning fail to address the simultaneous requirements of multi-objective decision making, uncertainty and computational efficiency. A study of expert (as defined in section 3.1) pilots, who are proficient at MFP, revealed that the cognitive strategies used by experts can be modeled using Rasmussen’s [104] skills, rules and knowledge model. Additionally, heuristics can be used to reduce the complexity of the MFP problem. Adjustment and anchoring for instance reduces computation time by refining an initial estimate of the solution and RAWFS provides a means for mitigating uncertainty in a cognitively compatible manner. The proposed replanning architecture incorporates both of these heuristics. Furthermore, a novel interpretation of the MFP task is provided based on Rasmussen’s model. The proposed replanning architecture is specifically targeted at online replanning since there are no time constraints on offline planning. A replan is initiated if the UAV deviates from the planned path (beyond the tolerance bounds as specified by the cell sequence) or if there are significant changes to the environment (e.g. changes in the movement of other aircraft that exceed tolerance bounds). When replanning, an inflated heuristic search is conducted to quickly (within a sub-second time frame) find a satisficing path. Then, as soon as sufficient planning time is available, optimal (or admissible) MSA* is executed to refine the flight plan. Such an approach replicates the adjustment and anchoring heuristic and helps to prevent the need to enter hold manoeuvres for fixed wing aircraft. Note that if the inflated heuristic search exceeds the cut-off time (chosen to be one second through simulation studies), the architecture switches to admissible MSA*. This is done as the inflated heuristic search is less efficient than an admissible search in complex planning scenarios (especially those with local minima).
8.1
Key Contributions
A brief summary of the key contributions of the research is provided below: 1. A search algorithm and methodology, MSA*:
8.2. SIGNIFICANCE AND ORIGINALITY
165
• MSA* is proven to find the least cost path for multiple decision objectives given a variable successor operator. • MSA* can address the requirements for online replanning. For a 50NM×50NM×15000ft×90min world, MSA*1 (fine resolution lattice), MSA*2 (multi-resolution) and MSA*3 (improved multiresolution) finds the least cost paths in 9.23s, 4.46s and 5.99s (on average) respectively. This is significantly less than the minimum track traversal time of 1min. • MSA*, in conjunction with a multi-resolution lattice structure provides significant reductions in computation time without sacrificing path cost. The ratio of the path cost between MSA*1, MSA*2 and MSA*3, and Vector A* (the benchmark) is 0.989, 1.033 and 1.005 respectively. However, the ratio of the computation time is 0.484, 0.239 and 0.329 respectively. 2. Discovery of the additive consistency property, where given the same track, the cost of traversal using one trajectory segment is the same as using multiple trajectory segments. This property is demonstrated to be crucial for multi-resolution, multi-objective planning problems. 3. Provision of tolerance to uncertainty by embedding each trajectory segment within a cell sequence. This tolerance is measured as the distance from the track to the bounds of the cell sequence. Arbitrary tolerance is achievable through selection of a suitable cell sequence when designing the variable successor operator in MSA*. 4. In addition, the research proposed a self-scheduling replanning architecture for implementing MSA* within an intelligent control architecture onboard a UAV. This architecture makes use of expert pilot cognitive strategies using Rasmussen’s [104] skills, rules and knowledge model. The proposed replanning architecture is unique as it incorporates deliberative planning with reactive control elements for a vehicle that is in constant motion.
8.2
Significance and Originality
The outcomes of the research comprise the proposed MFP framework and the MSA* search methodology (Chapter 5), a series of case study scenarios for
166
CHAPTER 8. CONCLUSION
the UAV medical package delivery scenario (Chapter 6 and 6.3) and a variety of publications (refer Appendix C). These match the research outcomes in section 1.3.2. A number of significant and original contributions stem from these research outcomes which were summarised in the previous section. It should be noted that an extensive review of the literature revealed that none of the existing work simultaneously addresses the requirements of multi-objective planning under uncertainty with computational efficiency. For civil UAV MFP, the existing work (i.e. [4]) only optimises for a single objective and does not consider dynamic obstacles. On the other hand, existing vehicle motion planning algorithms predominantly focus on vehicle dynamic constraints (including track angles and fuel) and obstacles, but do not consider regulatory requirements (e.g. rules of the air) and wind effects. This is due to the computational complexity of 4D motion planning for a large search space. Therefore, it can be seen that the research is original. The outcomes of the research address the research problems posed in section 1.3. Clearly, the first research question concerning the feasibility of MFP is proven given the theoretical and simulation results. The second research question concerning the definition of optimality in MFP is also resolved - optimality is defined as the least cost path where the cost is found using a MCDM cost function. Candidate methods for implementing such a cost function include Multi-Attribute Utility Theory (MAUT) and fuzzy logic even though only a weighted sum was shown to be additively consistent. The third research question concerns the satisfaction of multiple objectives under uncertainty. It is shown that a computationally tractable approach is to mitigate uncertainty with tolerance and online replanning. Thus, planning is performed with the best estimate of the current scenario. Finally, the fourth research question, which relates to integration of MFP with other UAV systems is also resolved. The proposed self-scheduling replanning architecture describes a novel methodology for onboard replanning and actuation of planning outcomes. It can be seen that the research outcomes not only resolve the research questions, but also the research objectives (section 1.3.1). The research objectives stipulated that the derived MFP framework must consider multiple objectives under uncertainty in a computationally efficient manner. It is shown that MSA* optimises for multiple objectives such as the rules of the
8.3. FUTURE WORK
167
air, mission safety and mission efficiency objectives. MSA* enables the use of a multi-resolution lattice which is computationally efficient and shown to be significantly faster than existing vector A* based methods. Finally, uncertainty is mitigated through a combination of online replanning, tolerance (in the planned path) and the RAWFS heuristic. The resultant framework addresses the requirements for UAV MFP (as described in the research objectives) and enables a high level of autonomy for a variety of UAV operating scenarios. MFP plays a crucial component in the integration of UAVs in the NAS as it ensures that the flight plan meets safety objectives and conforms with the rules of the air. Furthermore, automated MFP aids in reducing the human operator’s workload and helps to reduce operational costs. For an operating scenario where there is one operator controlling multiple UAVs, a level of autonomy is required where the machine can make decisions and potentially execute them without operator intervention. This corresponds to autonomy levels six to eight in Fig. 1.2. In this scenario, the MFP system can be situated at the ground station or onboard the UAV. However, in the event of a communications outage, fully autonomous operation is required (levels nine and ten in Fig. 1.2) and the MFP system must be situated onboard the UAV. Both of these operating scenarios require an automated MFP system. Therefore, the proposed MFP framework is both significant and original.
8.3
Future Work
This research derived a framework for UAV MFP (MSA*) and an architecture for online replanning. The study of heuristics is one possible avenue for future work. This research presented a number of heuristics observed in human decision making and adopted the adjustment and anchoring and RAWFS heuristic. However, it is conceivable that the computation time could be further reduced through more ‘intelligent’ heuristics. For example, an optimal dispersion sequence such as the Van der Corput sequence [12] could be used in combination with adjustment and anchoring. This could potentially help to reduce the impact of local minima on computation time.
168
CHAPTER 8. CONCLUSION
Another avenue for future research is the theoretical and practical evaluation of stability for a control system that includes a slow (relatively speaking) deliberative element. This is important in obtaining regulatory certification for such an intelligent control architecture. In addition, the methodology proposed in [159] could be adopted where the weights in the cost function are adjusted in-flight based on the aircraft’s current state. This could be useful in (typically rare) scenarios where there is only just enough fuel to reach the destination waypoint. A related topic of research is non-linear cost functions. It was shown that additive consistency is a requirement for a multi-resolution search. In general, it is not possible to guarantee additive consistency with a non-linear cost function. However, a weaker definition of additive consistency may be possible based on MCDM techniques. In this research, an approach is adopted where the system operates independently of the operator. It is assumed that prior to flight, experts are consulted and their preferences are encoded in the MCDM cost function. In this approach, the operator is not necessarily an expert pilot but those who implement the framework are. An alternative approach is where the operator is an expert pilot in which case the system learns from the operator. This would require an online learning algorithm to modify the cost function according to his/her preferences. In this scenario, the proposed MFP framework functions as a Decision Support System (DSS) but becomes an automatic flight controller in the event of a communications outage. Finally, note that MSA* is a generic, 4D motion planning algorithm that could be used in other applications such as Autonomous Underwater Vehicle (AUV) motion planning.
8.4
Concluding Remarks
The research derived a 4D motion planning algorithm and methodology, MSA* and proposed a self-scheduling replanning architecture that implements MSA* onboard a UAV. Such an automated MFP system aids in raising the level of autonomoy of the UAV which enables the UAV to meet the requirements of operation within the NAS – an Equivalent Level of Safety (ELOS), conformance with the rules of the air and transparent operation.
Bibliography [1] S. S. Wegener, S. S. Schoenung, J. Totah, D. Sullivan, J. Frank, F. Enomoto, C. Frost, and C. Theodore, “UAV autonomous operations for airborne science missions,” in AIAA 3rd “Unmanned Unlimited” Technical Conference, Workshop and Exhibit, Chicago, Illinois, 2004. [2] T. Cox, “Civil UAV capability assessment,” NASA, Tech. Rep., 2004. [3] Office of the Secretary of Defense, “Unmanned aircraft systems roadmap: 2005-2030,” Tech. Rep., 2005. [4] I. McManus, “A multidisciplinary approach to highly autonomous UAV mission planning and piloting for civilian airspace,” Ph.D. Thesis, QUT, 2004. [5] M. T. DeGarmo, “Issues concerning integration of unmanned aerial vehicles in civil airspace,” MITRE, Center for Advanced Aviation System Development, Tech. Rep., November 2004. [6] “UAV task-force final report, a concept for european regulations for civil unmanned aerial vehicles (UAVs),” The Joint JAA/EUROCONTROL Initiative on UAVs, Tech. Rep., 11 May 2004. [7] Civil Aviation Safety Authority (CASA), “Civil aviation regulations 1988 (CAR 1988),” August 2003. [8] P. Narayan, P. Wu, D. Campbell, and R. Walker, “An intelligent control architecture for Unmanned Aerial Systems (UAS) in the National Airspace System (NAS),” in 22nd International Unmanned Air Vehicle Systems Conference, Melbourne, Australia, 2007.
169
170
BIBLIOGRAPHY
[9] E. B. Smith and R. Langari, “Fuzzy multiobjective decision making for navigation of mobile robots in dynamic, unstructured environments,” Journal of Intelligent & Fuzzy Systems, vol. 14, no. 2, pp. 95–108, 2003. [10] R. Parasuraman, T. Sheridan, and C. Wickens, “A model for types and levels of human interaction with automation,” IEEE Trans. Systems, Man and Cybernetics, Part A, vol. 30, no. 3, pp. 286–297, 2000. [11] W. B. Powell, Approximate dynamic programming. Wiley-Interscience, 2008.
New Jersey:
[12] S. M. LaValle, Planning Algorithms. New York: Cambridge University Press, 2006. [13] J. S. Mitchell and C. H. Papadimitriou, “Weighted region problem. finding shortest paths through a weighted planar subdivision,” Journal of the Association for Computing Machinery, vol. 38, no. 1, pp. 18–73, 1991. [14] D. Ferguson and A. Stentz, “Using interpolation to improve path planning: the Field D* algorithm,” Journal of Field Robotics, vol. 23, no. 2, pp. 79–101, 2006. [15] A. Nash, K. Daniel, S. Koenig, and A. Felner, “Theta*: Any-angle path planning on grids,” in AAAI Conf. on Artificial Intelligence, 22-26 July 2007. [16] R. Bonasso, D. Kortenkamp, D. P. Miller, and M. G. Slack, “Experiences with an architecture for intelligent, reactive agents,” in International Joint Conference on Artificial Intelligence, 1995. [17] S. Scherer, S. Singh, L. Chamberlain, and S. Saripalli, “Flying low and fast among obstacles,” in IEEE Int. Conf. Robotics and Automation, Rome, Italy, 2007. [18] M. Kayton and W. Fried, Avionics Navigation Systems. John Wiley & Sons, 1997.
New York:
[19] D. Bailey, “Development of an optimal spatial decision-making system using approximate reasoning,” Ph.D. Thesis, QUT, 2005.
BIBLIOGRAPHY
171
[20] NASA Jet Propulsion Laboratory, “Shuttle radar topography mission,” 2006, http://www2.jpl.nasa.gov/srtm/. [21] J. Rubio and S. Kragelund, “The trans-pacific crossing: long range adaptive path planning for UAVs through variable wind fields,” in 22nd Digital Avionics Systems Conference, vol. 2, 2003, pp. 8.B.4–81–12 vol.2. [22] D. Rathbun and B. Capozzi, “Evolutionary approaches to path planning through uncertain environments,” in 1st AIAA Technical Conference and Workshop on Unmanned Aerospace Vehicles. AIAA, 20-23 May 2002. [23] D. Sullivan, J. Totah, S. Wegener, F. Enomoto, C. Frost, J. Kaneshige, and J. Frank, “Intelligent mission management for uninhabited aerial vehicles,” in SPIE - The International Society for Optical Engineering Remote Sensing Applications of the Global Positioning System, vol. 5661, 2004, pp. 121–131. [24] M. Freed, P. Bonasso, K. Dalal, W. Fitzgerald, and R. Harris, “An architecture for intelligent management of aerial observation missions,” in Proceedings AIAA Infotech@Aerospace Technical Conference, Arlington, Virginia, 2005. [25] M. Wzorek, G. Conte, P. Rudol, T. Merz, S. Duranti, and P. Doherty, “From motion planning to control - a navigation framework for an autonomous unmanned aerial vehicle,” in 21st Bristol UAV Systems Conference, 2006. [26] C. Tovey, S. Greenberg, and S. Koenig, “Improved analysis of D*,” in Proc. IEEE Int. Conf. on Robotics and Automation, vol. 3, 2003, pp. 3371–3378 vol.3. [27] RTCA, “Software considerations in airborne systems and equipment certification,” 1992. [28] E. L. Deitch, “Learning to land: A qualitative examination of pre-rlight and in-flight decision-making processes in expert and novice aviators,” Ph.D. dissertation, Virginia Polytechnic Institute and State University, Nov 21, 2001.
172
BIBLIOGRAPHY
[29] M. S. Branicky, S. M. LaValle, K. Olson, and Y. Libo, “Quasirandomized path planning,” in IEEE Int. Conf. Robotics and Automation (ICRA), vol. 2, 2001, pp. 1481–1487 vol.2. [30] S. Koenig and M. Likhachev, “Improved fast replanning for robot navigation in unknown terrain,” in IEEE Int. Conf. Robotics and Automation, vol. 1, 2002, pp. 968–975 vol.1. [31] J. Carsten, D. Ferguson, and A. Stentz, “3D Field D*: Improved path planning and replanning in three dimensions,” in IEEE Int. Conf. Intelligent Robots and Systems, 2006, pp. 3381–3386. [32] I. K. Nikolos, K. P. Valavanis, N. C. Tsourveloudis, and A. N. Kostaras, “Evolutionary algorithm based offline/online path planner for UAV navigation,” IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 33, no. 6, pp. 898–912, 2003. [33] D. Ferguson, M. Likhachev, and A. T. Stentz, “A guide to heuristicbased path planning,” in Int. Conf. on Automated Planning and Scheduling (ICAPS), Monterey, CA, 2005. [34] J.-C. Latombe, Robot Motion Planning. Kluwer Academic Publishers, 1991.
Norwell, Massachusetts:
[35] L. Dale and N. Amato, “Probabilistic roadmaps-putting it all together,” in IEEE Int. Conf. on Robotics and Automation, vol. 2, 2001, pp. 1940–1947 vol.2. [36] S. Kambhampati and L. Davis, “Multiresolution path planning for mobile robots,” IEEE Journal of Robotics and Automation, vol. 2, no. 3, pp. 135–145, 1986. [37] J. Barraquand, B. Langlois, and J.-C. Latombe, “Numerical potential field techniques for robot path planning,” IEEE Trans. Systems, Man and Cybernetics, vol. 22, no. 2, pp. 224–241, 1992. [38] J. Boskovic, R. Prasanth, and R. Mehra, “A multilayer control architecture for unmanned aerial vehicles,” in American Control Conference, vol. 3, 2002, pp. 1825–1830.
BIBLIOGRAPHY
173
[39] P. Tompkins, A. T. Stentz, and W. R. L. Whittaker, “Mission-level path planning for rover exploration,” in 8th Conf. on Intelligent Autonomous Systems (IAS-8), March 2004. [40] R. A. Brooks, “Solving the find-path problem by representing free space as generalised cones,” Massachusetts Institute of Technology, Tech. Rep. 674, 1982. [41] Y. Kim, D.-W. Gu, and I. Postlethwaite, “Real-time path planning with limited information for autonomous unmanned air vehicles,” Automatica, vol. 44, no. 3, pp. 696–712, 2008. [42] D.-W. Gu, W. Kamal, and I. Postlethwaite, “A UAV waypoint generator,” in AIAA 1st Intelligent Systems Conference, Chicago, IL, 20-22 Sep 2004. [43] P. O. Pettersson and P. Doherty, “Probabilistic roadmap based path planning for an autonomous unmanned aerial vehicle,” in ICAPS, 2004. [44] Y.-H. Qu, Q. Pan, and J.-G. Yan, “Flight path planning of UAV based on heuristically search and genetic algorithms,” in IEEE Industrial Electronics Society 32nd Annual Conf., 2005, p. 5. [45] D. Rathbun, S. Kragelund, A. Pongpunwattana, and B. Capozzi, “An evolution based path planning algorithm for autonomous motion of a UAV through uncertain environments,” in Proceedings 21st Digital Avionics Systems Conference, vol. 2, 2002, pp. 8D2–1–8D2–12 vol.2. [46] G. Yang and V. Kapila, “Optimal path planning for unmanned air vehicles with kinematic and tactical constraints,” in 41st IEEE Conf. Decision and Control, vol. 2, 2002, pp. 1301–1306 vol.2. [47] J. Gonzalez and A. Stentz, “Planning with uncertainty in position an optimal and efficient planner,” in IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2005, pp. 2435–2442. [48] L. Kavraki, P. Svestka, J.-C. Latombe, and M. Overmars, “Probabilistic roadmaps for path planning in high-dimensional configuration spaces,” IEEE Trans. Robotics and Automation, vol. 12, no. 4, pp. 566–580, 1996.
174
BIBLIOGRAPHY
[49] L. Yang and S. M. LaValle, “The sampling-based neighborhood graph: An approach to computing and executing feedback motion strategies,” IEEE Trans. Robotics and Automation, vol. 20, no. 3, pp. 419–432, 2004. [50] L. Graux, P. Millies, P. L. Kociemba, and B. Langlois, “Integration of a path generation algorithm into off-line programming of airbus panels,” in Aerospace Automated Fastening Conf. and Exp. SAE Tech., Oct. 1992. [51] L. Kavraki and J.-C. Latombe, “Randomized preprocessing of configuration for fast path planning,” in IEEE Int. Conf. Robotics and Automation, 1994, pp. 2138–2145 vol.3. [52] M. Overmars, “A random approach to path planning,” Utrecht University, Tech. Rep. RUU-CS-92-32, October, 1992. [53] S. Quinlan, “Efficient distance computation between non-convex objects,” in IEEE Int. Conf. Robotics and Automation, 1994, pp. 3324– 3329 vol.4. [54] L. Kavraki, M. Kolountzakis, and J.-C. Latombe, “Analysis of probabilistic roadmaps for path planning,” IEEE Trans. on Robotics and Automation, vol. 14, no. 1, pp. 166–171, 1998. [55] N. Amato and Y. Wu, “A randomized roadmap method for path and manipulation planning,” IEEE Int. Conf. Robotics and Automation, vol. 1, pp. 113–120, April 1996. [56] C. Nielsen and L. Kavraki, “A two level fuzzy PRM for manipulation planning,” in IEEE/RSJ Int. Conf. Intelligent Robots and Systems, vol. 3, 2000, pp. 1716–1721 vol.3. [57] R. Bohlin and L. E. Kavraki, “Path planning using lazy PRM,” in Proc. IEEE Int. Conf. Robotics and Automation, 2000, pp. 521–528. [58] L. Yang and S. LaValle, “A framework for planning feedback motion strategies based on a random neighborhood graph,” IEEE Int. Conf. Robotics and Automation, vol. 1, pp. 544–549, 2000.
BIBLIOGRAPHY
175
[59] V. Boor, M. H. Overmars, and A. F. van der Stappen, “The gaussian sampling strategy for probabilistic roadmap planners,” IEEE Int. Conf. Robotics and Automation, pp. 1018–1023, 1999. [60] J.-P. Laumond and T. Simeon, “Notes on visibility roadmaps and path planning,” Proc. Int. Workshop on Algorithmic Foundations of Robotics, 2000. [61] S. A. Wilmarth, N. M. Amato, and P. F. Stiller, “A probabilistic roadmap planner with sampling on the medial axis of the free space,” IEEE Int. Conf. Robotics and Automation, pp. 1024–1031, 1999. [62] P. Bessiere, J. M. Ahuactzin, E.-G. Talbi, and E. Mazer, “The ariadne’s clew algorithm: Global planning with local methods,” IEEE Int. Conf. Intel. Rob. Syst., vol. 2, pp. 1373–1380, 1993. [63] S. M. LaValle and J. J. Kuffner, “Randomized kinodynamic planning,” IEEE Int. Conf. Robotics and Automation, pp. 473–479, 1999. [64] D. Hsu, L. Kavraki, J.-C. Latombe, R. Motwani, and S. Sorkin, “On finding narrow passages with probabilistic roadmap planners,” Proc. Int. Workshop on Algorithmic Foundations of Robotics, 1998. [65] L. K. Dale, “Optimization techniques for probabilistic roadmaps,” Ph.D. Thesis, Texas A&M University, 2000. [66] O. B. Bayazit, G. Song, and N. M. Amato, “Enhancing randomized motion planners: exploring with haptic hints,” IEEE Int. Conf. Robotics and Automation, pp. 529–536, 2000. [67] S. J. Russell and P. Norvig, Artificial Intelligence: a modern approach, 2nd ed. Upper Saddle River, N.J.: Prentice Hall, 2003. [68] D. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models. N.J.: Prentice-Hall, 1976. [69] R. E. Bellman and L. A. Zadeh, “Decision-making in a fuzzy environment,” Management Science, vol. 17, no. 4, pp. B141–B164, 1970. [70] G. A. Klein, Decision Making in Action: Models and Methods. wood, N.J.: Abex, 1993.
Nor-
176
BIBLIOGRAPHY
[71] J. Kacpryzk, Mutistage Fuzzy Control. West Sussex, UK: John Wiley and Sons, 1997. [72] E. W. Dijkstra, “A note on two problems in connexion with graphs,” Numerische Mathematik, vol. 1, no. 1, pp. 269–271, 1959. [73] P. Hart, N. Nilsson, and B. Rafael, “A formal basis for the heuristic determination of minimum cost paths,” IEEE Trans. Systems Science and Cybernetics, vol. 4, pp. 100–107, 1968. [74] L. Harrison, P. Saunders, and J. Janowitz, “Artificial intelligence with applications for aircraft,” in Digital Systems Validation. Federal Aviation Administration, 1994, vol. 2. [75] M. Pivtoraiko and A. Kelly, “Generating near minimal spanning control sets for constrained motion planning in discrete state spaces,” in IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2005, pp. 3231– 3237. [76] J. van den Berg, D. Ferguson, and J. Kuffner, “Anytime path planning and replanning in dynamic environments,” in IEEE Int. Conf. Robotics and Automation, 2006, pp. 2366–2371. [77] A. Yahja, S. Singh, and A. Stentz, “An efficient on-line path planner for mobile robots operating in vast environments,” Robotics and Autonomous Systems, vol. 33, no. 2&3, 2000. [78] A. Stentz, “The focussed D* algorithm for real-time replanning,” in Int. Joint Conference on Artificial Intelligence, 1995, pp. 1652–1659. [79] M. Likhachev, “Search-based planning for large dynamic environments,” Ph.D. dissertation, Carnegie Mellon University, 2005. [80] A. Felner, S. Kraus, and R. E. Korf, “KBFS: K-best-first search,” Annals of Mathematics and Artificial Intelligence, vol. 39, no. 1, pp. 19–39, 2003. [81] R. E. Korf, W. Zhang, I. Thayer, and H. Hohwald, “Frontier search,” Journal ACM, vol. 52, no. 5, pp. 715–748, 2005.
BIBLIOGRAPHY
177
[82] R. E. Korf, “Space-efficient search algorithms,” ACM Computing Surveys, vol. 27, no. 3, pp. 337–339, 1995. [83] ——, “Linear-space best-first search,” Artificial Intelligence, vol. 62, no. 1, pp. 41–78, 1993. [84] R. Korf, “Depth-first iterative-deepening : An optimal admissible tree search,” Artificial Intelligence, vol. 27, no. 1, pp. 97–109, 1985. [85] R. E. Korf, M. Reid, and S. Edelkamp, “Time complexity of iterativedeepening-A*,” Artificial Intelligence, vol. 129, no. 1-2, pp. 199–218, 2001. [86] A. Saffiotti, “The uses of fuzzy logic in autonomous robot navigation,” Soft Computing - A Fusion of Foundations, Methodologies and Applications, vol. 1, no. 4, pp. 180–197, 1997. [87] S. M. LaValle and R. Sharma, “On motion planning in changing, partially predictable environments,” International Journal of Robotics Research (IJRR), vol. 16, no. 6, pp. 775–805, 1997. [88] B. Pfeiffer, R. Batta, K. Klamroth, and R. Nagi, “Probabilistic modeling for UAV path planning in the presence of threat zones,” Operations Research, 2005. [89] L. Bertuccelli and J. How, “Search for dynamic targets with uncertain probability maps,” in American Control Conference, 2006. [90] D. Ferguson and A. T. Stentz, “Planning with map uncertainty,” Robotics Institute, Carnegie Mellon University, Tech. Rep. CMU-RITR-04-09, February 2004. [91] B. Bakker, Z. Zivkovic, and B. Krose, “Hierarchical dynamic programming for robot path planning,” in IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2005, pp. 2756–2761. [92] N. Suzuki, D. Araki, A. Higashide, and T. Suzuki, “Geographical route planning based on uncertain knowledge,” in Proc. 7th Int. Conf. on Tools with Artificial Intelligence, 1995, pp. 434–441.
178
BIBLIOGRAPHY
[93] R. J. Adams, “How expert pilots think: Cognitive processes in expert decision making,” FAA, Tech. Rep. DOT/FAA/RD-93/9, 1993. [94] M. S. Cohen and J. T. Freeman, “Thinking naturally about uncertainty,” in 40th Annual Meeting of the Human and Ergonomics Society, vol. 1, Philadelphia, PA, 1996, pp. 179–183. [95] G. Coppin, F. Cadier, and P. Lenca, “Some considerations of cognitive modeling for collective decision support,” in 40th Annual Hawaii International Conference on System Sciences, 2007. [96] J. P. Barthelemy, R. Bisdorff, and G. Coppin, “Human centered processes and decision support systems,” European Journal of Operational Research, vol. 136, no. 2, pp. 233–252, 2002. [97] P. A. Simpson, “Naturalistic decision making in aviation environments,” Air Operations Division, Aeronautical and Maritime Research Laboratory, Tech. Rep. DSTO-GD-0279, 2001. [98] H. A. Simon, Models of Man. Wiley, 1957. [99] H. Graham, G. Coppin, and M. L. Cummings, “The PRIM: Extracting expert knowledge for aiding in C2 sense & decision making,” in 12th ICCRTS Adapting C2 to the 21st Century. Newport, Rhode Island: CCRP, 2007. [100] J. P. Barthelemy and E. Mullet, “A model of selection by aspects,” Acta Psychologica, vol. 79, pp. 1–19, 1992. [101] F. Cadier, G. Coppin, and P. Lenca, “Decision makers support system: Adapting the assistance to the decision maker’s behaviour,” Modelisation multiple, vol. 13, pp. 53–73, 2008. [102] R. Lipshitz, G. Klein, J. Orasanu, and E. Salas, “Taking stock of naturalistic decision making,” Journal of Behavioral Decision Making, vol. 14, no. 5, pp. 331–352, 2001. [103] R. Onken and A. Walsdorf, “Assistant systems for aircraft guidance: cognitive man-machine cooperation,” Aerospace Science and Technology, vol. 5, no. 8, pp. 511–520, 2001.
BIBLIOGRAPHY
179
[104] J. Rasmussen, “Skills, rules and knowledge, signals, signs and symbols, and other distinctions in human performance models,” IEEE Trans. Systems, Man and Cybernetics, vol. 13, pp. 257–266, 1983. [105] M. Cummings, C. Tsonis, and D. Cunha, “Complexity mitigation through airspace structure,” in 13th International Symposium on Aviation Psychology, Oklahoma City, OK, 2005. [106] G. F. List, P. B. Mirchandani, M. A. Turnquist, and K. G. Zografos, “Modeling and analysis for hazardous materials transportation - risk analysis, routing scheduling and facility location,” Transportation Science, vol. 25, no. 2, pp. 100–114, 1991. [107] H. Montgomery, “Decision rules and the search for a dominance structure: Toward a process model of decision making,” in Analyzing and aiding decision processes, P. Humphrey, O. Svenson, and A. Vari, Eds. Amsterdam: North-Holland, 1983. [108] H. Montgomery and H. Willen, “Decision making and action: The search for a good structure,” in Judgement and Decision Making, P. Juslin and H. Montgomery, Eds. Mahwah, New Jersey: Lawrence Erlbaum Associates, 1999. [109] R. Lipshitz, A. Sender, M. Omodei, J. McClellan, and A. Wearing, “What’s burning? the RAWFS heuristic on the fireground,” in Expertise out of context, R. Hoffman, Ed. Lawrence Erlbaum, 2006. [110] E. H. Ruspini, P. P. Bonissone, and W. Pedrycz, Handbook of fuzzy computation. New York: Wiley, 1996. [111] L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, pp. 338–353, 1965. [112] R. Jang, Fuzzy Logic Toolbox. The Mathworks, 2002. [113] C. D. C. Pereira, F. Garcia, J. Lang, and R. Martin-Clouaire, “Planning with graded nondeterministic actions: A possibilistic approach,” International Journal of Intelligent Systems, vol. 12, no. 11-12, pp. 935–962, 1997. [114] M. Patyra and D. Mlynek, Fuzzy logic : implementation and applications. New York: Wiley, 1996.
180
BIBLIOGRAPHY
[115] P. Wu, R. Clothier, D. Campbell, and R. Walker, “Fuzzy multiobjective mission flight planning in unmanned aerial systems,” in IEEE Symposium on Computational Intelligence in Multi-Criteria DecisionMaking, Honolulu, Hawaii, 2007. [116] G. Mouzouris and J. Mendel, “Nonsingleton fuzzy logic systems: Theory and application,” IEEE Trans. Fuzzy Systems, vol. 5, no. 1, pp. 56–71, 1997. [117] J. Mendel and R. John, “Type-2 fuzzy sets made simple,” IEEE Trans. Fuzzy Systems, vol. 10, no. 2, pp. 117–127, 2002. [118] P. Wu, P. Narayan, D. Campbell, M. Lees, and R. Walker, “A high performance fuzzy logic architecture for UAV decision making,” in IASTED International Conference on Computational Intelligence, San Francisco, 2006, pp. 260–265. [119] M. Grabisch and C. Labreuche, “Fuzzy measures and integrals in MCDA,” in Multiple Criteria Decision Analysis: State of the Art Surveys, J. Figueira, S. Greco, and M. Ehrgott, Eds. New York: Springer, 2005. [120] C. A. Bana e Costa and J. C. Vansnick, “The MACBETH approach: Basic ideas, software and an application,” in Advances in Decision Analysis, N. Meskens and M. Roubens, Eds. Dordrecht: Kluwer Academic Publishers, 1999. [121] M. Grabisch, I. Kojadinovic, and P. Meyer, “A review of methods for capacity identification in choquet integral based multi-attribute utility theory: Applications of the Kappalab R package,” European Journal of Operational Research, vol. 186, no. 2, pp. 766–785, 2008. [122] G. Choquet, “Theory of capacities,” Annales de I’Institut Fourier, vol. 5, pp. 131–295, 1953. [123] S. Bruni and C. M.L., “Human interaction with mission planning search algorithms,” in ASNE Human Systems Integration Conference, 2005.
BIBLIOGRAPHY
181
[124] C. Petres, Y. Pailhas, P. Patron, Y. Petillot, J. Evans, and D. Lane, “Path planning for autonomous underwater vehicles,” IEEE Trans. Robotics, vol. 23, no. 2, pp. 331–341, 2007. [125] K. Fujimura, “Path planning with multiple objectives,” IEEE Robotics & Automation Magazine, vol. 3, no. 1, pp. 33–38, 1996. [126] S. Bortoff, “Path planning for UAVs,” in American Control Conference, vol. 1, 2000, pp. 364–368 vol.1. [127] M. Jun and R. D’Andrea, “Path planning for unmanned aerial vehicles in uncertain and adversarial environments,” in Cooperative Control: Models, Applications and Algorithms, S. Butenko, R. Murphey, and P. Pardalos, Eds. Dordrecht: Kluwer Academic Publisher, 2002, pp. 95–111. [128] J. Reif and Z. Sun, “Movement planning in the presence of flows,” Algorithmica, vol. 39, no. 2, pp. 127–153, June 2004. [129] Z. Sun and J. Reif, “On robotic optimal path planning in polygonal regions with pseudo-euclidean metrics,” IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 37, no. 4, pp. 925–936, Aug. 2007. [130] N. A. Papadakis and A. N. Perakis, “Deterministic minimal time vessel routing,” Operations Research, vol. 38, no. 3, pp. 426–438, 1990. [131] J. Sellen, “Direction weighted shortest path planning,” IEEE Int. Conf. Robotics and Automation, vol. 2, pp. 1970–1975 vol.2, May 1995. [132] D. B. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence. San Francisco, CA: Morgan Kaufmann, 1995. [133] P. B. Mirchandani and H. Soroush, “Optimal paths in probabilistic networks: A case with temporary preferences,” Computers & Operations Research, vol. 12, no. 4, pp. 365–381, 1985. [134] P. Leonelli, S. Bonvicini, and G. Spadoni, “Hazardous materials transportation: a risk-analysis-based routing methodology,” Journal of Hazardous Materials, vol. 71, no. 1-3, pp. 283–300, 2000.
182
BIBLIOGRAPHY
[135] M. Zhang, Y. Ma, and K. Weng, “Location-routing model of hazardous materials distribution system based on risk bottleneck,” in Int. Conf. on Services Systems and Services Management, vol. 1, 2005, pp. 362– 368. [136] D. Nembhard and I. White, C.C., “Heuristic path selection in ORgraphs with application to HAZMAT routing,” in IEEE Int. Conf. Systems, Man, and Cybernetics ’Humans, Information and Technology’, vol. 2, 1994, pp. 1536–1540. [137] A. R. Soltani and T. Fernando, “A fuzzy based multi-objective path planning of construction sites,” Automation in Construction, vol. 13, no. 6, pp. 717–734, 2004. [138] B. S. Stewart and I. Chelsea C. White, “Multiobjective A*,” J. ACM, vol. 38, no. 4, pp. 775–814, 1991. [139] K. Erol, J. Hendler, and D. S. Nau, “HTN planning: Complexity and expressivity,” in AAAI National Conf. on Artificial Intelligence, 1995, p. 1123. [140] P. Doherty, J. Gustafsson, L. Karlsson, and J. Kvarnstrom, “(TAL) temporal action logics: Language specification and tutorial,” Electronic Transactions on Artificial Intelligence, vol. 2, no. 3-4, pp. 273–306, 1998. [141] T. Belker, M. Hammel, and J. Hertzberg, “Learning to optimize mobile robot navigation based on HTN plans,” in IEEE Int. Conf. Robotics and Automation, vol. 3, Sept. 2003, pp. 4136–4141. [142] M. Snorrason and J. Norris, “Vision based obstacle detection and path planning for planetary rovers,” in Unmanned Ground Vehicle Technology II (Aerosense), Orlando, Florida, April 1999. [143] P. Narayan, P. Wu, and D. Campbell, “Unmanning UAVs - addressing challenges in on-board planning and decision making,” in Humans Operating Unmanned Systems, Brest, France, 3-4 September 2008. [144] J. Bresenham, “Pixel-processing fundamentals,” IEEE Computer Graphics and Applications, vol. 16, no. 1, pp. 74–82, 1996.
BIBLIOGRAPHY
183
[145] W. F. Phillips, Mechanics of Flight. Hoboken, New Jersey, USA: John Wiley & Sons, 2004. [146] P. Wu, D. Campbell, and T. Merz, “On-board multi-objective mission planning for unmanned aerial vehicles,” in IEEE Aerospace Conference, Big Sky, Montana, 7-14 March 2009. [147] T. Merz, P. Rudol, and M. Wzorek, “Control system framework for autonomous robots based on extended state machines,” in Int. Conf. Autonomic and Autonomous Systems, July 2006. [148] Civil Aviation Safety Authority, “Visual flight rules operations,” 2001. [149] R. Clothier, R. Walker, N. Fulton, and D. Campbell, “A casualty risk analysis for unmanned aerial system (uas) operations over inhabited areas,” in Twelfth Australian International Aerospace Congress (AIAC12), Melbourne, Australia, 2007. [150] International Civil Aviation Organisation, “Procedures for air navigation services. air traffic management,” Tech. Rep. Doc 4444, 2007. [151] E. Williams, “ICAO ADS-B separation standards,” Australian Strategic Air Traffic Management Group (ABIT), Tech. Rep. AIBT09-IP008, 9 March 2006. [152] S. Thrun, W. Burgard, and D. Fox, Probabilistic robotics. Cambridge, MA: MIT Press, 2005. [153] K. D. McDonald, “The modernization of GPS: Plans, new capabilities and the future relationship to Galileo,” Journal of Global Positioning Systems, vol. 1, no. 1, pp. 1–17, 2002. [154] Bureau of Meteorology, “Aviation weather services,” 2008, http:// www.bom.gov.au/reguser/by prod/aviation/. [155] Airservices Australia, “Document and chart descriptions,” 24 Oct, 2008 2008, http://www.airservicesaustralia.com/publications/ docdescrip.asp. [156] Unmanned Dynamics LLC, “Aerosim blockset version 1.1 user’s guide,” 2003.
184
BIBLIOGRAPHY
[157] Cessna Aircraft Company, Model 172 and Skyhawk Owner’s Manual, 1985. [158] NOAA National Weather Service, “Structure and dynamics of supercell thunderstorms,” 2004, http://www.crh.noaa.gov/lmk/soo/docu/ supercell.php. [159] A. Stentz, “CD*: a real-time resolution optimal re-planner for globally constrained problems,” in AAAI 18th National Conference on Artificial Intelligence, Edmonton, Alberta, Canada, 2002, pp. 605–611.
Appendix A MACBETH Value Functions This appendix contains the remaining value functions for the MCDM cost function described in Section 6.2.1.
Figure A.1: Aircraft separation risk value function.
185
186
APPENDIX A. MACBETH VALUE FUNCTIONS
Figure A.2: Cruising levels rule value function.
Figure A.3: Value function for the time criterion. Note the use of cumulative time levels to ensure additive consistency.
187
Figure A.4: Normalised population risk value function.
Figure A.5: Storm cell risk value function.
Appendix B Example Planning Scenarios This appendix presents the data tables for the paths discussed in Chapter 6.3. In each data table, x, y, z and tl are the 4D waypoints, θ is the horizontal track angle, vc the airspeed (in m/s) and (wx , wy , wz ) the wind vector (in m/s). The aircraft separation risk probability is pa , ps the storm cell risk, NPD is the accumulated normalised population density as discussed in Section 7.4 and fuel is in L. Also included is the computation time, number of search iterations and total path cost. A movie clip of each of these scenarios can be found in the attached CD.
B.1
Scenario 1
The data tables corresponding to Fig. 6.9, 6.10, 6.11 and 6.12 are shown in the following tables for Vector A*, MSA*1, MSA*2 and MSA*3 respectively. Note that the first row in each table corresponds to the start node where the decision variables are all initialised to zero since decision variables are associated with trajectory segments.
188
B.1. SCENARIO 1
189
Table B.1: Scenario 1 - Vector A* (22.09s, 324571 iterations, path cost of 1.11) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 15 17 19 22 25 28 31 34 36 38 40 42 44 46 49
0 0 1 4 7 9 11 13 15 18 21 24 27 30 33 36 39
4 3 3 3 3 3 3 3 5 7 7 7 7 7 7 7 5
0 2 4 7 10 13 16 19 22 25 27 29 31 33 35 37 40
0 90 72 34 34 56 56 56 56 45 34 34 34 34 34 34 45
0 50 51.5 40 40 40.5 40.5 40 39.5 41 47 47.5 48 48 48 48 37
0.00 -4.07 -1.85 -2.22 -2.22 -2.22 -1.48 -0.37 0.74 1.85 5.93 5.19 4.82 4.82 4.44 4.44 4.44
0.00 -3.33 -3.33 -2.96 -2.96 -2.96 -3.33 -3.70 -3.70 -5.93 5.56 6.30 6.67 6.67 6.67 7.04 7.04
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.33 0.17 0.02 0.13 0.04 0.12 0.17 1.04 2.02 1.30 1.53 1.58 0.37 0.07 0.11 0.41
0.00 1.75 1.89 1.89 1.89 1.93 1.93 1.89 2.14 2.30 1.61 1.64 1.67 1.67 1.67 1.67 1.67
0 0.0524 0.057 0.0562 0.0563 0.0571 0.0572 0.0564 0.0652 0.0713 0.0498 0.0511 0.052 0.0501 0.0496 0.0497 0.0502
Table B.1: Scenario 1 - MSA*1 (10.88s, 185081 iterations, path cost of 1.11) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 15 18 21 24 27 30 31 34 36 38 40 42 44 46 49
0 0 1 3 6 8 11 13 15 18 21 24 27 30 33 36 39
4 3 3 3 3 3 3 5 5 7 7 7 7 7 7 7 5
0 2 4 7 10 13 16 19 21 24 26 28 30 32 34 36 39
0 90 72 56 45 56 45 56 27 45 34 34 34 34 34 34 45
0 50 51.5 40 47 40 47 39.5 39 41 47 47.5 48 48 48 48 37
0.00 -4.07 -1.85 -2.22 -2.22 -2.59 0.00 0.37 1.11 1.85 5.93 5.19 4.82 4.82 4.44 4.44 4.44
0.00 -3.33 -3.33 -2.96 -2.96 -2.59 -3.70 -3.70 -5.93 -5.93 5.56 6.30 6.67 6.67 6.67 7.04 7.04
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.33 0.17 0.22 0.42 0.10 0.28 0.32 0.54 2.02 1.30 1.53 1.58 0.37 0.07 0.11 0.41
0.00 1.75 1.89 1.89 2.41 1.89 2.41 2.14 1.23 2.30 1.61 1.64 1.67 1.67 1.67 1.67 1.67
0 0.0524 0.057 0.0566 0.0719 0.0563 0.0717 0.064 0.0372 0.0713 0.0498 0.0511 0.052 0.0501 0.0496 0.0497 0.0502
190
APPENDIX B. EXAMPLE PLANNING SCENARIOS Table B.1: Scenario 1 - MSA*2 (5.24s, 74155 iterations, path cost of 1.14) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 15 18 21 24 30 36 40 44 48 49
0 0 1 2 5 8 14 18 24 30 36 39
4 3 3 3 3 5 7 7 7 7 5 5
0 2 4 6 10 14 20 26 30 34 38 40
0 90 72 72 45 45 45 56 34 34 34 18
0 50 51.5 51 36 37 48 35.5 48 48 48.5 42
0.00 -4.07 -1.85 -2.22 -2.22 -2.59 -1.11 0.74 5.19 4.44 4.07 1.11
0.00 -3.33 -3.33 -2.96 -2.96 -2.59 -6.30 -5.93 6.30 7.04 7.04 5.93
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.33 0.17 0.21 0.58 0.37 1.36 2.10 1.79 1.33 0.54 0.20
0.00 1.75 1.89 1.86 2.23 2.56 5.25 3.39 3.34 3.34 3.32 1.35
0 0.0524 0.057 0.0554 0.0668 0.0762 0.1576 0.1036 0.1018 0.1011 0.0992 0.0404
Table B.1: Scenario 1 - MSA*3 (6.61s, 103063 iterations, path cost of 1.11) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 15 18 21 24 27 30 32 35 39 43 47 49
0 0 1 3 6 9 12 14 15 18 24 30 36 39
4 3 3 3 3 3 3 5 5 7 7 7 5 5
0 2 4 7 10 13 16 19 21 24 28 32 36 38
0 90 72 56 45 45 45 56 63 45 34 34 34 34
0 50 51.5 40 47 47 47 39.5 36 41 47.5 48 48.5 49
0.00 -4.07 -1.85 -2.22 -2.22 -2.59 -0.37 0.37 1.48 2.22 5.56 4.82 4.44 1.11
0.00 -3.33 -3.33 -2.96 -2.96 -2.59 -3.70 -3.70 -5.93 -5.56 5.93 6.67 7.04 5.93
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.33 0.17 0.22 0.42 0.34 0.20 0.60 0.47 2.48 2.13 1.71 0.47 0.24
0.00 1.75 1.89 1.89 2.41 2.41 2.41 2.14 1.12 2.30 3.28 3.34 3.32 1.73
0 0.0524 0.057 0.0566 0.0719 0.0718 0.0715 0.0644 0.0338 0.072 0.1006 0.1017 0.0991 0.0515
B.2. SCENARIO 2
B.2
191
Scenario 2
The data tables corresponding to Fig. 6.18 and 6.19 are shown in the following tables for Vector A*, MSA*1, MSA*2 and MSA*3 respectively. Table B.2: Scenario 2 - Vector A* (28.36s, 397587 iterations, path cost of 1.08) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
6 9 12 15 18 21 24 27 30 33 36 39 42
6 9 12 15 18 21 24 27 30 33 36 39 42
5 5 5 5 5 5 5 5 5 5 5 5 5
0 3 6 9 12 15 18 21 24 27 30 33 36
0 45 45 45 45 45 45 45 45 45 45 45 45
0 47.5 47 47 47 47 47 47.5 48 48 48 48 48
0.00 -3.70 0.00 -0.37 -0.74 -0.74 -1.11 -1.11 -1.11 -1.11 -1.11 -1.11 -1.11
0.00 -1.48 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70
0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.18 0.41 0.27 0.69 0.38 1.10 1.13 1.30 1.14 1.55 1.19 1.31
0.00 2.46 2.41 2.41 2.41 2.41 2.41 2.46 2.50 2.50 2.50 2.50 2.50
0 0.073 0.0721 0.0718 0.0725 0.072 0.075 0.0746 0.0761 0.0758 0.0765 0.0759 0.0761
Table B.2: Scenario 2 - MSA*1 (14.08s, 229240 iterations, path cost of 1.08) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
6 9 12 15 18 21 24 27 30 33 36 39 42
6 9 12 15 18 21 24 27 30 33 36 39 42
5 5 5 5 5 5 5 5 5 5 5 5 5
0 3 6 9 12 15 18 21 24 27 30 33 36
0 45 45 45 45 45 45 45 45 45 45 45 45
0 47.5 47 47 47 47 47 47.5 48 48 48 48 48
0.00 -3.70 0.00 -0.37 -0.74 -0.74 -1.11 -1.11 -1.11 -1.11 -1.11 -1.11 -1.11
0.00 -1.48 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70
0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.18 0.41 0.27 0.69 0.38 1.10 1.13 1.30 1.14 1.55 1.19 1.31
0.00 2.46 2.41 2.41 2.41 2.41 2.41 2.46 2.50 2.50 2.50 2.50 2.50
0 0.073 0.0721 0.0718 0.0725 0.072 0.075 0.0746 0.0761 0.0758 0.0765 0.0759 0.0761
192
APPENDIX B. EXAMPLE PLANNING SCENARIOS Table B.2: Scenario 2 - MSA*2 (7.25s, 90378 iterations, path cost of 1.10)
x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
6 9 12 14 20 26 32 38 42
6 9 11 12 18 24 30 36 42
5 5 5 5 7 9 9 7 5
0 4 6 8 14 20 26 32 38
0 45 56 63 45 45 45 45 34
0 36.5 58 37 47.5 47.5 47 47.5 42
0.00 -3.70 0.00 -0.37 -0.37 -0.37 -0.37 -0.74 -0.74
0.00 -1.48 -3.70 -3.70 -3.70 -6.67 -3.70 -3.70 -6.67
0 0 0 0 0 0 1.28E-11 1.15E-11 0
0 0 0 0 0 0 0 0 0
0.00 0.18 0.20 0.26 0.79 1.31 1.71 2.36 1.51
0.00 2.27 2.43 1.15 5.16 5.18 4.85 4.82 3.95
0 0.0675 0.0724 0.0345 0.1541 0.1555 0.1463 0.1463 0.1193
Table B.2: Scenario 2 - MSA*3 (8.72s, 123975 iterations, path cost of 1.08) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
6 9 12 15 18 21 24 27 30 33 36 39 42
6 9 12 15 18 21 24 27 30 33 36 39 42
5 5 5 5 5 5 5 5 5 5 5 5 5
0 3 6 9 12 15 18 21 24 27 30 33 36
0 45 45 45 45 45 45 45 45 45 45 45 45
0 47.5 47 47 47 47 47 47.5 48 48 48 48 48
0.00 -3.70 0.00 -0.37 -0.74 -0.74 -1.11 -1.11 -1.11 -1.11 -1.11 -1.11 -1.11
0.00 -1.48 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70 -3.70
0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.18 0.41 0.27 0.69 0.38 1.10 1.13 1.30 1.14 1.55 1.19 1.31
0.00 2.46 2.41 2.41 2.41 2.41 2.41 2.46 2.50 2.50 2.50 2.50 2.50
0 0.073 0.0721 0.0718 0.0725 0.072 0.0751 0.0746 0.0761 0.0758 0.0765 0.0759 0.0761
B.3. SCENARIO 3
B.3
193
Scenario 3
The data tables corresponding to Fig. 6.20, 6.21 and 6.22 are shown in the following tables for Vector A*, MSA*1, MSA*2 and MSA*3 respectively. Table B.3: Scenario 3 - Vector A* (24.77s, 386685 iterations, path cost of 2.57) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 0 0 0 0 2 5 8 11 14 17 20 23 26 29 32 35 38 41 44 42
3 0 3 6 9 12 13 14 17 18 21 24 27 30 33 36 38 41 44 45 48
3 4 4 4 3 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 4
0 2 4 6 8 11 13 15 18 20 23 26 29 32 35 38 41 44 47 49 52
0 180 0 0 0 34 72 72 45 72 45 45 45 45 45 45 56 45 45 72 326
0 47 46 46 47 34.5 43 43 39 43 39 39 39 39 39 38.5 31 38 38 44 42.5
0.00 5.19 4.44 4.44 4.44 5.19 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.56 5.56 5.56 4.07
0.00 -0.74 -0.37 -0.37 -0.37 -0.74 0.74 0.74 0.74 0.74 0.74 0.74 0.74 0.74 0.74 0.74 1.11 1.11 1.48 2.22 -4.44
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.35 0.27 0.17 0.11 0.23 0.37 0.18 0.60 0.14 0.54 0.64 1.34 0.67 0.50 0.30 0.21 0.13 0.13 0.11 0.39
0.00 1.73 1.55 1.55 1.57 1.88 1.40 1.40 1.84 1.40 1.84 1.84 1.84 1.84 1.84 1.81 1.45 1.78 1.78 1.45 2.00
0 0.0518 0.6331 0.633 0.0465 0.0559 0.042 0.0417 0.0554 0.0417 0.0553 0.0555 0.0566 0.0555 0.0553 0.0541 0.0432 0.053 0.053 0.0432 0.06
194
APPENDIX B. EXAMPLE PLANNING SCENARIOS Table B.3: Scenario 3 - MSA*1 (10.38s, 189489 iterations, path cost of 2.52) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 0 0 0 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 42
3 0 3 6 9 12 14 17 20 23 26 29 32 35 38 41 44 47 49 48
3 4 4 4 3 3 5 5 5 5 5 5 5 5 5 5 5 5 5 4
0 2 4 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 52
0 180 0 0 0 45 56 45 45 45 45 45 45 45 45 45 45 45 56 180
0 47 46 46 31 41 33 39 39 39 39 39 39 39 38.5 38 38 38 33.5 27
0.00 5.19 4.44 4.44 4.44 5.19 5.19 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.93 5.56 5.56 5.56 5.56 4.07
0.00 -0.74 -0.37 -0.37 -0.37 -0.74 -0.74 0.74 0.74 0.74 0.74 0.74 0.74 0.74 1.11 1.11 1.48 1.85 2.22 -4.44
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.35 0.27 0.17 0.11 0.62 0.42 0.50 0.72 1.02 1.80 2.14 0.93 0.31 0.06 0.38 0.20 0.28 0.24 0.10
0.00 1.73 1.55 1.55 1.38 1.96 1.81 1.84 1.84 1.84 1.84 1.84 1.84 1.84 1.81 1.78 1.78 1.78 1.56 0.41
0 0.0518 0.6331 0.633 0.0412 0.059 0.0542 0.0553 0.0556 0.0561 0.0574 0.0579 0.056 0.055 0.0537 0.0534 0.0531 0.0532 0.0466 0.0124
Table B.3: Scenario 3 - MSA*2 (4.11s, 61055 iterations, path cost of 2.54) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 0 0 0 0 3 6 12 18 24 30 36 42 44 42
3 0 3 6 9 11 14 18 22 26 32 38 44 48 48
3 4 4 4 3 3 5 7 9 11 11 9 7 5 4
0 2 4 6 8 10 14 20 24 28 34 40 46 50 52
0 180 0 0 0 56 45 56 56 56 45 45 45 27 270
0 47 46 46 47 52 29.5 33 49 43 30.5 32.5 36.5 34 34
0.00 5.19 4.44 4.44 4.44 5.19 5.19 5.93 4.82 12.22 13.70 13.33 11.85 4.44 4.07
0.00 -0.74 -0.37 -0.37 -0.37 -0.74 -0.74 0.74 0.74 0.00 5.19 5.56 1.48 1.85 -4.07
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.35 0.27 0.17 0.11 0.47 0.65 0.77 0.74 1.07 1.27 0.96 0.87 0.19 0.12
0.00 1.73 1.55 1.55 1.57 1.93 2.10 3.32 3.75 3.10 2.90 2.94 3.32 2.04 1.01
0 0.0518 0.6331 0.633 0.0465 0.0578 0.0633 0.0994 0.1122 0.0933 0.0878 0.0886 0.0997 0.0608 0.0302
B.4. SCENARIO 4
195
Table B.3: Scenario 3 - MSA*3 (5.92s, 93586 iterations, path cost of 2.50) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 0 0 0 0 2 3 6 12 18 24 30 36 40 42 43 42
3 0 3 6 9 12 14 17 21 23 27 33 39 42 46 48 48
3 4 4 4 3 5 5 7 9 11 11 11 9 7 5 5 4
0 2 4 6 8 11 13 16 22 26 30 36 42 46 50 52 53
0 180 0 0 0 34 27 45 56 72 56 45 45 53 27 27 270
0 47 46 46 47 34.5 32 40.5 31 35 41 31 32.5 31 30.5 33.5 35
0.00 5.19 4.44 4.44 4.44 5.19 5.93 5.93 4.82 12.22 13.70 13.70 13.33 11.85 4.82 5.56 4.07
0.00 -0.74 -0.37 -0.37 -0.37 -0.74 0.74 0.74 0.74 0.37 5.19 5.56 5.93 1.48 1.85 2.22 -4.07
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.35 0.27 0.17 0.11 0.23 0.30 0.48 1.00 1.29 1.04 1.29 0.90 0.09 0.36 0.12 0.10
0.00 1.73 1.55 1.55 1.57 1.88 0.99 2.21 3.18 2.45 2.65 2.94 2.94 1.88 1.83 1.05 0.52
0 0.0518 0.6331 0.633 0.0465 0.0559 0.0299 0.0663 0.0956 0.0746 0.0801 0.089 0.0885 0.0557 0.0548 0.0311 0.0156
B.4
Scenario 4
The data tables corresponding to Fig. 6.23 are shown in the following tables for Vector A*, MSA*1, MSA*2 and MSA*3 respectively.
196
APPENDIX B. EXAMPLE PLANNING SCENARIOS Table B.4: Scenario 4 - Vector A* (28.36s, 439654 iterations, path cost of 1.22) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 15 18 21 24 27 30 33 36 39 42 44 46 47 48 49 49 48
3 4 7 8 10 12 14 16 19 22 25 28 31 34 37 40 43 46 49
4 5 7 7 7 7 7 7 7 7 7 7 7 7 5 5 5 5 4
0 2 5 7 9 11 13 15 18 21 24 27 30 32 34 36 38 40 42
0 72 45 72 56 56 56 56 45 45 45 45 34 34 18 18 18 0 342
0 46 39.5 39 46 45 45 45 38 41.5 41 41 36 54 50.5 50 50 48 52
0.00 3.33 1.85 9.63 10.00 10.74 11.11 11.11 10.74 10.00 10.74 10.74 10.74 10.74 10.37 2.22 2.22 2.22 1.85
0.00 0.37 0.00 0.37 0.37 3.70 4.07 4.07 4.44 -6.67 -5.19 -4.82 -4.82 -5.19 -5.19 -2.22 -2.22 -2.22 -2.22
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.55 0.63 0.16 0.25 0.15 0.14 0.49 1.10 1.25 1.72 1.64 0.82 1.22 2.05 1.85 1.85 0.69 0.64
0.00 1.68 2.16 1.23 1.56 1.50 1.50 1.50 1.81 2.00 1.97 1.97 1.68 2.09 1.79 1.79 1.79 1.67 1.89
0 0.0506 0.0649 0.0367 0.0465 0.0447 0.0447 0.0453 0.0554 0.0613 0.0611 0.0609 0.0511 0.0637 0.0563 0.056 0.056 0.0505 0.057
Table B.4: Scenario 4 - MSA*1 (27.36s, 502097 iterations, path cost of 1.63) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 15 18 21 24 27 30 33 36 39 41 43 45 46 46 46 46 48
3 6 9 11 12 14 16 19 21 24 27 30 33 36 39 42 45 48 49
4 5 7 7 7 7 7 7 7 7 7 7 7 7 7 5 5 5 4
0 3 6 8 10 12 14 17 19 22 25 28 31 34 36 38 40 42 44
0 45 45 56 72 56 56 45 56 45 45 34 34 34 18 0 0 0 63
0 41.5 39.5 46 38 45 45 39.5 53.5 42 41.5 37 38 38 53 51.5 48 48 34
0.00 3.33 1.85 9.63 10.74 10.74 10.74 10.74 8.89 10.00 10.37 10.74 10.37 10.37 10.37 10.37 1.85 1.85 1.85
0.00 0.37 0.00 0.37 3.70 3.70 4.07 4.44 -7.78 -6.30 -5.56 -5.19 -5.19 -5.19 -5.56 -5.56 -2.22 -2.22 -2.22
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.70 0.57 0.25 0.18 0.11 0.02 0.69 0.94 2.31 2.42 1.21 1.28 1.68 2.37 2.11 1.15 0.80 0.67
0.00 2.12 2.16 1.56 1.19 1.50 1.50 1.92 2.05 2.04 2.00 1.74 1.79 1.79 2.01 1.87 1.67 1.67 1.01
0 0.0638 0.0648 0.0465 0.0356 0.0447 0.0445 0.0579 0.0622 0.064 0.0632 0.0533 0.055 0.0556 0.0633 0.0587 0.0512 0.0506 0.4222
B.4. SCENARIO 4
197
Table B.4: Scenario 4 - MSA*2 (12.47s, 185108 iterations, path cost of 1.65) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 18 24 30 32 34 36 38 39 41 41 42 44 45 46 48
3 5 11 15 21 24 27 30 33 35 36 39 41 42 45 48 49
4 5 7 7 7 5 5 5 5 5 5 5 5 5 5 5 4
0 2 8 12 18 20 22 24 26 28 30 32 34 36 38 40 42
0 56 45 56 45 34 34 34 34 27 63 0 27 63 18 18 63
0 53 38 45 39.5 58 56 56 56 36 34 48 36 34 50 50 34
0.00 3.33 1.85 10.74 10.74 9.26 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85
0.00 0.37 0.00 3.70 4.44 -7.41 -2.59 -2.59 -2.59 -2.59 -2.59 -2.22 -2.22 -2.22 -2.22 -2.22 -2.22
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.62 1.22 0.41 0.87 1.94 2.18 2.30 2.33 1.77 2.02 1.97 2.25 1.94 2.17 1.25 0.67
0.00 2.13 3.86 3.01 3.84 2.40 2.25 2.25 2.25 1.12 1.05 1.67 1.12 1.05 1.79 1.79 1.01
0 0.0641 0.1161 0.0897 0.1151 0.0741 0.0701 0.0703 0.0703 0.0359 0.0344 0.0525 0.0367 0.0342 0.0565 0.0551 0.4222
Table B.4: Scenario 4 - MSA*3 (16.52s, 266211 iterations, path cost of 1.64) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
9 12 18 24 30 32 34 36 38 39 41 42 42 42 44 46 48
3 5 11 15 21 24 27 30 33 35 36 38 39 42 45 48 49
4 5 7 7 7 5 5 5 5 5 5 5 5 5 5 5 4
0 2 8 12 18 20 23 26 29 31 33 35 36 38 41 44 46
0 56 45 56 45 34 34 34 34 27 63 27 0 0 34 34 63
0 53 38 45 39.5 58 38 38 38 36 34 36 33 48 38 38 34
0.00 3.33 1.85 10.74 10.74 9.26 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85 1.85
0.00 0.37 0.00 3.70 4.44 -7.41 -2.59 -2.59 -2.59 -2.59 -2.59 -2.22 -2.22 -2.22 -2.59 -2.22 -2.22
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.62 1.22 0.41 0.87 1.94 2.18 2.30 2.33 1.77 2.02 2.00 1.01 2.27 2.39 1.49 0.67
0.00 2.13 3.86 3.01 3.84 2.40 1.78 1.78 1.78 1.12 1.05 1.12 0.51 1.67 1.78 1.78 1.01
0 0.0641 0.1161 0.0897 0.1151 0.0741 0.0563 0.0565 0.0565 0.0359 0.0344 0.0363 0.0167 0.053 0.0566 0.0552 0.4222
198
APPENDIX B. EXAMPLE PLANNING SCENARIOS
B.5
Scenario 5
The data tables corresponding to Fig. 6.24 and 6.25 are shown in the following tables for Vector A*, MSA*1, MSA*2 and MSA*3 respectively. Table B.5: Scenario 5 - Vector A* (68.28s, 1050993 iterations, path cost of 2.10) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 2 5 7 10 13 16 19 22 25 28 31 34 37 40 43 45 47 49 49 49 49
3 6 9 12 15 16 17 18 19 20 21 22 23 25 27 30 33 36 39 42 45 48
5 5 5 5 7 7 7 7 7 7 7 7 7 7 7 7 5 5 5 5 5 4
0 3 6 9 12 14 16 18 20 22 24 26 28 31 34 37 40 43 46 48 50 52
0 34 45 34 45 72 72 72 72 72 72 72 72 56 56 45 34 34 34 0 0 0
0 40 45 40 45 45 45 45 45 45 45 45 45 36 35 43 39.5 40 40 51 51 49.5
0.00 1.11 1.11 1.48 1.48 7.41 7.41 7.41 7.41 7.78 7.78 7.78 8.15 8.15 8.52 8.89 9.26 2.96 2.96 2.96 2.96 2.96
0.00 -4.82 -4.82 -4.82 -4.82 -8.15 -8.15 -8.15 -8.15 -8.15 -7.78 -7.78 -7.78 -7.41 -7.04 -6.67 -5.93 -4.07 -4.07 -4.07 -4.07 -4.07
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.32 0.36 0.18 0.38 0.23 0.19 0.05 0.13 0.31 0.63 0.71 1.67 1.33 1.17 1.52 2.11 2.17 2.40 1.74 1.59 2.24
0.00 1.90 2.25 1.90 2.53 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.68 1.63 2.11 1.82 1.90 1.90 1.86 1.86 1.72
0 0.0568 0.0671 0.0566 0.0755 0.0449 0.0448 0.0446 0.0447 0.045 0.0455 0.0456 0.0472 0.052 0.0502 0.0648 0.0571 0.0597 0.0601 0.0579 0.0576 0.6414
B.5. SCENARIO 5
199
Table B.5: Scenario 5 - MSA*1 (29.91s, 533977 iterations, path cost of 1.92) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 3 6 9 12 15 18 21 24 27 30 33 36 39 41 43 45 46 46 46 46 48 49
3 6 9 12 15 16 17 18 19 20 21 22 24 27 30 33 34 36 39 42 45 46 48
5 5 5 5 7 7 7 7 7 7 7 7 7 7 7 7 7 5 5 5 5 5 4
0 3 6 9 12 14 16 18 20 22 24 26 29 32 35 38 40 42 44 46 48 50 52
0 45 45 45 45 72 72 72 72 72 72 72 56 45 34 34 63 27 0 0 0 63 27
0 45 45 45 45 45 45 45 45 45 45 45 36 43 39 39 30.5 38 51 51 51 35 37
0.00 1.11 1.11 1.48 1.48 7.41 7.41 7.41 7.41 7.78 7.78 7.78 8.15 8.52 8.89 9.26 9.26 9.63 2.96 2.96 2.96 2.96 2.96
0.00 -4.82 -4.82 -4.82 -4.82 -8.15 -8.15 -8.15 -8.15 -8.15 -7.78 -7.78 -7.41 -7.04 -6.67 -6.30 -5.93 -5.56 -4.07 -4.07 -4.07 -4.07 -4.07
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.47 0.37 0.34 0.30 0.37 0.08 0.20 0.14 0.48 0.61 1.10 1.50 2.20 1.20 2.22 1.87 1.90 1.89 2.17 1.72 1.76 1.84
0.00 2.25 2.25 2.25 2.53 1.50 1.50 1.50 1.50 1.50 1.50 1.50 1.68 2.11 1.85 1.85 0.96 1.15 1.86 1.86 1.86 1.08 1.11
0 0.0673 0.0672 0.0672 0.0753 0.0451 0.0446 0.0448 0.0447 0.0453 0.0455 0.0463 0.0523 0.0659 0.0566 0.0582 0.0313 0.0371 0.0581 0.0585 0.0578 0.0349 0.4271
200
APPENDIX B. EXAMPLE PLANNING SCENARIOS Table B.5: Scenario 5 - MSA*2 (14.03s, 188932 iterations, path cost of 1.94) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 6 12 18 24 30 36 40 44 45 46 46 46 46 48 49
3 9 13 15 17 19 21 24 30 33 36 39 42 45 46 48
5 7 7 7 7 7 7 7 5 5 5 5 5 5 5 4
0 6 12 16 20 24 28 32 38 40 42 44 46 48 50 52
0 45 56 72 72 72 72 53 34 18 18 0 0 0 63 27
0 45.5 38 46 45 45 45 38 39.5 52 52 51 51 51 35 37
0.00 1.11 7.04 7.41 7.41 7.41 7.78 8.15 8.89 2.59 2.96 2.96 2.96 2.96 2.96 2.96
0.00 -4.82 -8.52 -8.15 -8.15 -8.15 -7.78 -7.41 -6.67 -4.44 -4.07 -4.07 -4.07 -4.07 -4.07 -4.07
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 1.05 0.14 0.50 0.31 0.55 1.54 0.98 0.79 1.46 2.38 1.89 2.17 1.72 1.76 1.84
0.00 4.83 3.58 3.12 3.01 3.01 3.01 2.39 3.63 1.93 1.93 1.86 1.86 1.86 1.08 1.11
0 0.1446 0.1061 0.093 0.0895 0.0899 0.0915 0.0723 0.1087 0.0595 0.061 0.0581 0.0585 0.0578 0.0349 0.4271
Table B.5: Scenario 5 - MSA*3 (15.5s, 230305 iterations, path cost of 1.93) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
0 3 6 12 18 24 30 36 38 41 43 45 46 46 46 46 48 49
3 6 9 15 17 19 21 23 24 27 30 33 36 39 42 45 46 48
5 5 5 7 7 7 7 7 5 5 5 5 5 5 5 5 5 4
0 3 6 12 16 20 24 28 30 33 36 39 42 44 46 48 50 52
0 45 45 45 72 72 72 72 63 45 34 34 18 0 0 0 63 27
0 45 45 45 45 45 45 45 34 45 40 40 36 51 51 51 35 37
0.00 1.11 1.11 1.48 7.41 7.41 7.78 7.78 8.15 2.22 2.59 2.59 2.96 2.96 2.96 2.96 2.96 2.96
0.00 -4.82 -4.82 -4.82 -8.15 -8.15 -8.15 -7.78 -7.41 -4.44 -4.44 -4.44 -4.07 -4.07 -4.07 -4.07 -4.07 -4.07
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.47 0.37 0.78 0.39 0.27 1.03 2.15 0.74 1.37 0.86 1.60 2.38 1.89 2.17 1.72 1.76 1.84
0.00 2.25 2.25 4.75 3.01 3.01 3.01 3.01 1.02 2.25 1.90 1.90 1.68 1.86 1.86 1.86 1.08 1.11
0 0.0673 0.0672 0.1418 0.0896 0.0894 0.0907 0.0925 0.0313 0.0688 0.0576 0.0588 0.0534 0.0581 0.0585 0.0578 0.0349 0.4271
B.6. MOUNTAIN WIND SCENARIO
B.6
201
Mountain Wind Scenario
Below is the data table for MSA*2 and MSA*3 (which find exactly the same path) for the path shown in Fig. 7.3. Table B.6: Mountain wind scenario - MSA*2/MSA*3 (5.14s/7.89s, 120321/170123 iterations, path cost of 1.36/1.36) x
y
z
tl
θ
vc
wx
wy
wz
AGL
Fuel
Cost
0 3 5 6 6 12 18 24 30 36 42 48
24 23 24 22 24 24 24 24 24 24 24 24
5 5 7 9 11 13 11 9 9 9 7 5
0 2 4 6 8 14 18 22 26 32 36 40
0 108 63 153 0 90 90 90 90 90 90 90
0 59 44 41 33 41 57 57 57 41 57 57
0.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00 -10.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 -0.37 1.85 3.33 3.70 3.70 0.74 -4.82 -0.74 4.82 0.74 -3.70
0.00 5.50 5.50 7.50 9.50 11.50 10.93 1.62 1.62 8.29 7.50 5.50
0.00 2.52 1.58 1.39 1.10 3.99 4.79 4.78 4.69 3.84 4.71 4.70
0 0.0746 0.0466 0.0413 0.0327 0.1182 0.1417 0.1414 0.1388 0.1136 0.1394 0.1391
B.7
Replanning Scenario
Below are the MSA*3 data tables for the replanning scenario described in Fig. 6.26, 6.27 and 6.28 respectively. Table B.7: MSA*3 Replanning scenario - original plan (5.015s, 88458 iterations) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
3 6 9 12 15 18 21 24 27 29 31 34 37 40 39
0 3 6 9 12 15 17 19 21 24 27 30 33 36 39
5 5 5 5 5 5 5 5 3 5 5 5 5 5 6
0 3 6 9 12 15 18 21 24 27 30 33 36 39 41
0 45 45 45 45 45 56 56 56 34 34 45 45 45 342
0 45 45 45 45 45 38 38 40.5 41.5 39 45 45 45 52
0.00 -0.37 -0.37 -0.37 -0.37 -0.37 -0.37 -0.37 -0.37 -1.85 -0.37 -0.37 -0.37 -0.37 -0.37
0.00 -2.22 -2.22 -2.22 -2.22 -2.22 -2.22 -2.22 -2.22 -5.19 -2.22 -2.22 -2.22 -2.22 -2.22
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.00 0.16 0.15 0.36 1.41 1.33 0.65 1.02 1.73 1.93 0.92 0.99 0.30 0.30 0.25
0.00 2.25 2.25 2.25 2.25 2.25 1.78 1.78 1.88 2.28 1.84 2.25 2.25 2.25 2.06
0 0.0668 0.0668 0.0681 0.0688 0.0687 0.0538 0.0544 0.0584 0.0706 0.0559 0.0682 0.067 0.067 0.0614
202
APPENDIX B. EXAMPLE PLANNING SCENARIOS
Table B.7: MSA*3 Replanning scenario - near-optimal plan starting at t = 18min. (0.063s, 920 iterations) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
21 24 27 30 33 36 38 40 41 40 39
17 20 23 24 24 26 27 30 33 36 39
5 3 1 1 1 1 1 1 3 4 6
18 21 24 26 28 31 33 36 39 42 45
0 45 45 72 90 56 63 34 18 342 342
0 47.5 49 51 47 41.5 39 43 38 36 36.5
0.00 -0.37 -1.48 -0.74 -0.74 -1.11 -1.11 -1.11 -1.11 -1.85 -2.59
0.00 -2.22 -5.19 -6.30 -6.30 -6.30 -5.93 -5.93 -5.93 -5.19 -4.82
0 0 0 4.0E-04 2.0E-04 1.0E-04 0 0 0 0 1.0E-04
0 0 0 0 0 0 0 0 0 0 0
0.00 1.78 2.26 1.26 1.00 1.75 0.33 0.17 0.28 0.40 0.22
0.00 2.41 2.53 1.85 1.60 1.99 1.22 2.09 2.05 1.79 1.98
0 0.0741 0.0784 0.1061 0.0716 0.0708 0.0366 0.0621 0.0612 0.0537 0.0693
Table B.7: MSA*3 Replanning scenario - optimal plan starting at t = 21min. (1.844s, 29278 iterations) x
y
z
tl
θ
vc
wx
wy
pa
ps
NPD
Fuel
Cost
24 27 30 33 36 39 39 41 40 39
20 22 23 25 27 29 30 33 36 39
3 3 5 5 5 5 3 3 4 6
21 24 27 30 33 36 37 40 43 46
0 56 72 56 56 56 0 34 342 342
0 42 35 38 38 38 36 42 36 36.5
0.00 -1.48 -1.48 -0.74 -0.74 -0.74 -0.74 -1.85 -1.85 -2.59
0.00 -5.19 -5.19 -2.22 -2.22 -2.22 -2.22 -5.19 -5.19 -4.82
0 0 0 0 0 0 1.0E-04 0 0 0
0 0 0 0 0 0 0 0 0 0
0.00 1.22 2.01 1.63 1.65 0.33 0.00 0.13 0.40 0.22
0.00 2.02 1.90 1.78 1.78 1.78 0.54 2.02 1.79 1.98
0 0.0619 0.0596 0.0554 0.0554 0.0565 0.0225 0.0607 0.0537 0.0589
Appendix C Publications Publications stemming from this research are listed in chronological order below: P. Narayan, P. Wu, D. Campbell, and R. Walker, “An intelligent control architecture for unmanned aerial systems (UAS) in the national airspace system (NAS),” in 22nd International Unmanned Air Vehicle Systems Conference, Melbourne, Australia, 2007. P. Wu, R. Clothier, D. Campbell, and R. Walker, “Fuzzy multi-objective mission flight planning in unmanned aerial systems,” in IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making, Honolulu, Hawaii, 2007. P. Narayan, P. Wu, and D. Campbell, “Unmanning UAVs - addressing challenges in on-board planning and decision making,” in Humans Operating Unmanned Systems, Brest, France, 3-4 September 2008. P. Wu, D. Campbell, and T. Merz, “On-board multi-objective mission planning for unmanned aerial vehicles,” Big Sky, Montana, 7-14 March 2009. P. P.-Y. Wu, D. Campbell, and T. Merz, “Multi-objective 4D vehicle motion planning in large dynamic environments,” Submitted1 . IEEE Trans. Systems, Man and Cybernetics B. The second, fourth and fifth papers (where the author is first author) are included on the following pages. 1
To be re-submitted as per editor instructions - original paper was too long.
203
204
APPENDIX C. PUBLICATIONS
Fuzzy Multi-Objective Mission Flight Planning in Unmanned Aerial Systems Paul Wu, Reece Clothier, Duncan Campbell, Member, IEEE, Rodney Walker, Member, IEEE
Abstract—This paper discusses the development of a multiobjective mission flight planning algorithm for Unmanned Aerial System (UAS) operations within the National Airspace System (NAS). Existing methods for multi-objective planning are largely confined to two dimensional searches and/or acyclic graphs in deterministic environments; many are computationally infeasible for large state spaces. In this paper, a multi-objective fuzzy logic decision maker is used to augment the D* Lite graph search algorithm in finding a near optimal path. This not only enables evaluation and trade-off between multiple objectives when choosing a path in three dimensional space, but also allows for the modelling of data uncertainty. A case study scenario is developed to illustrate the performance of a number of different algorithms. It is shown that a fuzzy multiobjective mission flight planner provides a viable method for embedding human expert knowledge in a computationally feasible algorithm.
I. INTRODUCTION
U
NMANNED Aerial Systems (UAS), or robotic aircraft, will comprise a substantial component of future aviation. Already UAS, with their unique operational capabilities, have demonstrated successful applications in surveillance, communications, environmental monitoring, agriculture and defence. Ongoing advancements in enabling technologies, coupled with decreasing system and operational costs, will continue to strengthen the business case for UAS in a widening range of applications. As such, the expected growth in the Australian UAS market is estimated to increase by more than 198% between 2001 and 2010 [1]. However, there are a number of challenges which need to be addressed in order to realise the potential for this industry, one of which is gaining access to the National Airspace System (NAS) [2]. Manuscript received October 31, 2006. This work was supported by the Australian Research Centre for Aerospace Automation (ARCAA), Queensland University of Technology (QUT) and the Commonwealth Scientific and Industrial Research Organization (CSIRO), funded in part by the Queensland State Government, Smart State Research Fund. Mr Paul Wu, is a graduate student with ARCAA, QUT, GPO Box 2434, Brisbane, 4000, Australia. (phone: +61 7 31381400; e-mail:
[email protected]). Mr Reece A. Clothier, is a graduate student with ARCAA, QUT, Brisbane, 4000, Australia. (e-mail:
[email protected]). Associate Professor Duncan Campbell, is with the School of Engineering Systems and ARCAA, QUT, Brisbane, 4000 Australia. (email:
[email protected]). He is currently a Visiting Professor with the Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, MA 02139 USA. Associate Professor Rodney Walker is with the School of Engineering Systems and ARCAA, QUT, Brisbane, 4000, Australia. (e-mail:
[email protected]).
One of these challenges is the problem of mission flight (strategic) planning and in-flight (tactical) re-planning. To ensure the successful integration of UAS within the NAS, an Equivalent Level Of Safety (ELOS) to conventional aircraft operations must be demonstrated [3]. In addition, UAS must also operate under the existing rules and regulations governing the safe pilotage of conventional aircraft and must appear as ‘transparent’ users of the NAS [3]. In the absence of a technology able to provide an equivalent see-and-avoid capability for UAS (a requirement for the unsegregated operation of UAS alongside other aircraft, Civil Aviation Regulation 163A [4]); strategic flight planning plays an important role in the risk management, and subsequent approval, of UAS operations within the NAS. Mission flight planning must also ensure that the operation is conducted in accordance with the ‘rules of the air’ and that mission goals are achieved. A complex trade-off exists between mission goals, mission efficiency objectives and the rules of the air. To add to the complexity of the problem, the airspace environment is highly dynamic and uncertain. As a result, changes may need to be made to the flight plan during an operation as a result of (i) new information (e.g. detection of other aircraft or hazardous weather conditions), (ii) changes in aircraft performance (e.g. as a result of system failures) or (iii) changes in the mission goals. Tactical changes to the flight plan (performed whilst the UAS is airborne) place significant time pressures on the planning process. It is envisaged that the proposed mission flight planning process would help increase the level of autonomy within the UAS. It could conceivably allow UAS to operate at the sixth level in Parasuraman’s model of autonomy [5]. With this level of autonomy, the plan is executed automatically unless there is intervention by a human operator. This paper discusses the development and evaluation of an example mission flight planning system that addresses some of the requirements in operating UAS in the NAS. The first section provides a summary of existing approaches to the mission flight planning problem. The second section details the decision space, the costs and the rules governing the flight planning problem, and introduces a case study mission scenario. Following this, a number of potential multiobjective planning algorithms are presented and are implemented for the mission scenario. The final section discusses the results obtained with the different algorithms.
205
Fig. 1 Aerosonde Mark III (inset) and Operating Area
II. THE DECISION SPACE This section introduces the complex decision space for a case study flight planning scenario of a UAS operation within the NAS. This case study will help illustrate the mission flight planner design process and demonstrate how different input variable types (used to evaluate mission objectives) are handled in a multi-objective mission flight planner. The scenario decision space comprises the: A. Mission B. Physical Environment C. Decision Objectives A. Mission The mission flight planning task is specific to the UAS being operated, the decision objectives (such as flight rules and fuel rates) and the goals of the mission. A goal may be as simple as reaching a specific destination or can be more complex such as conducting a grid search. An example scenario is presented here to highlight the different aspects of the proposed planning algorithms. The prospective UAS in the case study scenario is an Aerosonde™ Mark III, inset in Fig. 1#. The mission is to be conducted over the Kingaroy region in Queensland, Australia. As can be seen from Fig. 1, the Aerosonde™ must fly a mission which comprises a starting waypoint and two goal waypoints which are to be completed sequentially. The straight line path between each waypoint is also indicated. For the sake of simplicity, three mission planning layers were defined at altitudes of 1500ft, 2500ft and 3500ft Above Mean Sea Level (AMSL). These layers correspond to the cruise altitudes defined by CAR173 [4]. Each layer comprises a 10 element by 10 element array of possible #
Illustration © Aerosonde Pty. Ltd. www.aerosonde.com.au
flight path nodes (not shown in Fig. 1). Thus the total three dimensional decision space comprises 300 possible flight path nodes. B. The Physical Environment The physical airspace environment comprises both static and dynamic aspects. Static elements include terrain, population areas, tall structures, location of aerodromes and designations of controlled or restricted airspace. Published aeronautical maps provide the a priori information necessary to generate mission flight plans ensuring the safe navigation of static aspects of the operating environment. Flight planning for dynamic aspects of the physical environment, such as weather, other aircraft or birds, is often done just prior to, or during, the flight. In-flight re-planning can partially address the problem of dynamic changes in the environment. For the case study scenario, the undulating terrain in the Kingaroy region is modelled using data obtained from the National Aeronautics and Space Administration (NASA) Shuttle Radar Topography Mission (SRTM) [6]. The resolution of the terrain data samples is three arc-seconds (approximately 90m) and is modelled as a surface using bilinear interpolation. C. Decision Objectives A UAS mission must be conducted in accordance with the rules and regulations governing the safe flight of conventional aviation. For the purposes of this scenario, the aircraft is assumed to operate under Visual Flight Rules (VFR) conditions. Only two rules are considered: namely Civil Aviation Regulation (CAR) 157 and 173. CAR157 stipulates that aircraft must maintain a minimum altitude of 500ft above terrain except when performing
206
APPENDIX C. PUBLICATIONS
specific tasks with the approval of the regulator. The clearance above terrain, measured as the altitude Above Ground Level (AGL) is a decision objective that addresses this regulatory requirement. This AGL variable is classified as an independent variable because the AGL value remains the same regardless of the path chosen. CAR173 relates to cruising levels [4]. Cruising altitudes are assigned based on the heading of the aircraft so as to reduce the likelihood of an aircraft encountering a head-on collision scenario. CAR173 states, that for headings from 0° to 179°, aircraft operating under VFR should cruise at altitudes of odd multiples of 1000ft plus 500ft AMSL (e.g. 1500ft, 3500ft, 5500ft AMSL). For headings between 180° and 359°, aircraft should cruise at even multiples of 1000ft AMSL plus 500ft (e.g. 2500, 4500, 6500ft AMSL) [4]. For operations below 5000ft AMSL, CAR173 is not a mandatory requirement but should be obeyed when terrain, weather and traffic conditions permit. This flight rule is encoded as another decision objective based on the aircraft’s altitude and heading angle. Note that the heading angle is a dependent variable as it changes depending on the chosen path. Risk is another independent variable that should be considered in the construction of a mission flight plan. The two primary hazards of unrestricted UAS operations within the NAS are that of a midair collision or the termination of flight in a populated area [7]-[9]. For the case study scenario, a single risk objective, the risk presented to people on the ground expressed as the number of ground casualties per flight hour of operation is considered. This was calculated using methods described in [10]. Fuel is another important consideration in the selection of a flight path. Optimum fuel performance is obtained by maintaining cruise or descending in altitude. Ascending comes at the expense of increased fuel consumption. For the case study scenario, an arbitrary fuel cost for each path segment is calculated based on the distance travelled and the pitch angle of the aircraft: qi = d ( 2sin θ i + cos θ i ) (1) where qi is the fuel consumed, d is the distance travelled and θi is the pitch angle for that path segment. This simple model is used purely for representative purposes as the case study does not consider aircraft velocity. However, fuel is a good example of a dependent, finite variable that is cumulative (accumulates value along the path). III. MULTI-OBJECTIVE MISSION FLIGHT PLANNER Finding the ‘best’ path whilst considering multiple objectives is a path planning problem that involves the aggregation of multiple objectives, sometimes referred to as a multi-objective search problem. The problem is further compounded by uncertainty in input data. An examination of existing work in this field is presented here.
A. Existing Work The focus of research in recent times in the field of path planning has been on techniques in computational geometry and graph theoretic planning [11]. It is common to employ a global planner (such as the mission flight planner described here) to generate an approximate plan which is then refined to obtain an exact trajectory using a more precise local planner [12]. Despite the existence of many path planning techniques (LaValle [13] provides a comprehensive study), there are relatively few methods that cater for multiple objectives. One field that has seen much use of multi-objective path planning is hazardous materials (HAZMAT) transportation. This stems from the need to make compromises between risk and transportation costs [14]. Many HAZMAT planners have employed a general optimal graph search algorithm such as Dijkstra’s or A* [13] and combined that with some form of weighted sum decision aggregation that computes an aggregated path cost [14]-[16]. However, the majority of these algorithms are confined to 2-dimensional searches in deterministic environments. [14]-[16] There are also multi-objective iterative graph search algorithms such as Fujimura’s algorithm [17] and MultiObjective A* [18]. Unfortunately, MOA* is limited to acyclic graphs and Fujimura’s implementation is computationally impractical for large state spaces. Fuzzy logic and fuzzy Membership Functions (MFs) have also been employed in multi-objective path planning. Soltani and Fernando [19] present a planner for deterministic environments that uses fuzzy MFs and Dijkstra’s search algorithm but does not employ fuzzy inferencing. Suzuki, Araki et al [20] describe a planner that uses fuzzy MFs to approximate the basic probabilities of input variables which are then aggregated using Dempster-Shafer theory. This aggregated probability is subsequently used as a cost value in a graph search algorithm. Again, fuzzy inferencing is not employed. Perhaps the most relevant work in the field has been done by McManus [21] and Tompkins [11]. However, McManus’ method does not make trade-offs between objectives. On the other hand, the Incremental Search Engine (ISE) developed by Tompkins, Stentz et al [11] does provide true multiobjective aggregation. It is a 2-dimensional, complete and optimal planner that, like the methods described above, combines a method for multi-objective aggregation with a graph search algorithm. It considers two types of variables, independent variables (such as spatial location) and dependent variables (such as energy and time). Tompkins, Stentz et al also consider four objectives, namely spatial location x, y, time and energy. To address the problem of dimensionality, they employ an aggregation function to collapse the energy dimension and perform a 3-dimensional search. However, their work differs from the work presented here in that they only plan for 2-dimensional Euclidean space and use a different method of aggregation that does not
207
consider uncertainty. It can be seen that the vast majority of multi-objective planners employ roadmap based planning by modifying the cost variable in a traditional graph search algorithm [11], [14]-[19]. This alleviates the computational complexity in searching a continuous state space with an infinite number of states. Therefore, this approach has been adopted.
Zadeh’s compositional rule of inference, the implicated consequent MF Yl of each rule l is found by taking the t-norm ( ∧ ) of the consequent MF µl ( y ) and the supremums of the t-norms of each input MF and its antecedent MF µ F l ( xk ) k
[27]. Yl = ∨
k =1.. p
B. Multi-objective Decision Making It was found that relatively few multi-objective path planning algorithms were capable of planning under uncertainty. Traditional Bayesian probability based approaches are hindered by high computational costs and the need for accurate a priori knowledge of probability distributions. Non-deterministic methods, on the other hand, do not require any a priori knowledge. However, these methods can return highly sub-optimal solutions as decisions are made based on worst-case scenarios. [13] A candidate method for planning with multiple objectives under uncertainty without a priori knowledge is fuzzy logic. Type 1 MFs can be used to represent uncertainty in input as a possibility distribution. Additionally, fuzzy inferencing can be used to embed expert knowledge when evaluating multiple competing objectives [22]. This is often also referred to as fuzzy Multi-Criteria Decision Making (MCDM) [23]. The ability to express expert knowledge is particularly advantageous as automated mission planning strives to replicate a human pilot’s cognitive abilities. Rasmussen [24] describes a process for human decision tasks that reflects cognitive levels of the decision making process. The rule driven aspects of this model can be represented using a rule based system which is suited for implementation using a fuzzy rule base [24]. Furthermore, the application of fuzzy MFs can help address Zadeh’s [25] principle of incompatibility, which observes that increased complexity corresponds to decreased precision in human cognition (hence a greater degree of approximate reasoning). Furthermore, fuzzy logic is completely deterministic, and when coupled with a deterministic path planner, produces deterministic solutions [22]. This is crucial to certification in the UAS environment. The use of fuzzy logic enables the multi-objective mission flight planner to handle uncertainty through approximation. Uncertainty stems from noisy input variables, uncertainty in the meaning of linguistic variables, in the rule consequents and in the tuning of the rules [26]. Non-singleton (NS) fuzzification, sometimes referred to as vector fuzzification, can be used to capture input uncertainty by modelling the input value as a fuzzy MF. This way, inferencing can be performed on actual possibility distributions (and thus capture input variable uncertainty) instead of crisp sample values (as is done with singleton fuzzification). For p input variables Xk on universes of discourse (UoD) xk, one constructs fuzzy MFs µ X k ( xk ) . By
{µ ( y) ∧ T (sup µ
p k =1
Note that T
p k =1
l
Xk
( xk ) ∧ µ F l ( xk ) k
)}
(2)
denotes a sequence of t-norm operations.
C. Path Planning The D* Lite graph search algorithm was chosen as it is an efficient, complete, optimal and deterministic planner which is suited to in-flight re-planning [28]. The algorithm performs an incremental, heuristic backwards search and has been shown to be more efficient than D* [29], which in turn has a worst case computational complexity of O(V 3/2) (where V is the number of nodes in the graph) [30]. A condensed pseudo-code representation of the D* Lite algorithm is shown in Fig. 2. For more detail, refer to [28]. Key ( x )
(
k = min g ( x), rhs( x) + h( x , x) 1 I UpdateState( x )
(
)
k = min ( g ( x), rhs( x) ) 2
)
if visited ( x) { g ( x) = ∞}
){
(
}
if x ≠ x rhs( x ) = min c( x, x ') + g ( x ')] G x ' ∈ Parents ( x) [ if ( x ∈ Queue ) {remove x from queue}
if ( g ( x ) ≠ rhs( x) ){insert x into queue with key ( x)} ComputeShortestPath()
(
(
))
while min key ( x) < key ( x ) ∨ rhs( x ) ≠ g ( x ) s ∈ Queue I I I Queue. pop( x ) if ( g ( x) > rhs( x ) ) g ( x) = rhs ( x) ∀x ' ∈ Neighbors( x ),UpdateState( x ) else g ( x) = ∞ ∀x ' ∈ Neighbors( x ) ∪ x, UpdateState( x) Fig. 2 Pseudo-code of the D* Lite Algorithm. Note that x, xI and xG are the current, initial and goal states respectively; g(xI), rhs(xI) and g(xG) are each initialised to ∞, rhs(sG) is initialized to 0. Each iteration of D* Lite corresponds to an execution of ComputeShortestPath [29].
A prioritised queue is employed so that the most promising states (defined here as a location in three dimensional space) are explored first. A state’s priority in the queue is determined by a key Key(x) which is calculated based on a heuristic estimate h(xI,x) of the cost to reach the start state. The estimate could be as simple as the Euclidean distance but must be admissible (guaranteed to underestimate the actual cost) to guarantee optimality. At each iteration, g(x) (the cost to reach state x from the goal xG) and the one step look-ahead cost rhs(x) is calculated and the queue updated. Additionally, the parent state of x is set; this is used to build up a path by tracing through the parent’s of each
208
APPENDIX C. PUBLICATIONS
state all the way to the goal. A parent of x is selected based on its g value and the cost c(x,x’) to reach x from that state. D* Lite finds the path that minimizes the total summed cost to produce a globally optimal path. When there are multiple objectives, it is possible to find a globally optimal path by changing the way in which the actual cost c(x,x’) is calculated as this influences the way in which parent states are chosen. The introduction of dependent variables does not affect the optimality of D* Lite as, like D*, it is a form of dynamic programming and therefore there is no assumption of prior knowledge of edge costs [13]. D. Integration of Path Planning and Decision Making In order to plan with multiple objectives using D* Lite, it is necessary to change: • the way that parents are chosen – which entails a change in the way costs are calculated and stored • the heuristic h(xI,x) It is important to note that an optimal path constitutes a series of segments which are themselves optimal [13]. Therefore, it is possible to retain optimality by replacing the calculation of rhs(x) in UpdateState (Fig. 2) with a multiobjective method that finds the optimal parent of x. This corresponds to changing the function c(x,x’) and the method for choosing the best parent state of x (refer Fig. 3). Even though the heuristic needs modification to reflect the multiobjective cost g(x) (which can be determined through analysis of the fuzzy inferencing engine), the most important aspect of integrating multi-objective decision making with D* Lite lies in modifying UpdateState with a suitable multiobjective decision making algorithm. D* Lite Compute Shortest Path
Aggregated costs Update State
Independent Input Variables Dependent Input Variables
Multi-objective Decision Making
Fig. 3 Integration of Multi-objective Decision Making with D* Lite – compare with Fig. 2.
It is important to note that the suitability of each of the candidate (or alternative) parent states is dependent on all 4 objectives and is not necessarily a monotonic function of input variables. Consequently, it is advantageous to employ a full fuzzy inferencing module (using (2)) as opposed to fuzzy number manipulation operations. The output of the fuzzy inferencing process is itself a fuzzy MF. In order to choose the optimal parent state, it is
necessary to calculate the total aggregated cost to reach the current state via each candidate parent state. These costs can then be ranked to determine the optimal parent state. The centre of gravity (CoG) method was selected for calculating aggregated costs as it not only takes into account the support of the MFs, but also the degree of membership at each point [31]. Therefore, the cost value for each alternative parent j at state xi is: ∫ y µB ' ( y )dy (3) rhs j ( xi ) = g j ( xi +1 ) + ∫ µB ' ( y)dy where y is the output UoD and µ B ' ( y ) is the aggregation of the implicated output MFs. E. Case Study Application A specific mission scenario is presented to illustrate the various components in the mission flight planning framework. The problem comprises the evaluation of four objectives, namely: altitude Above Ground Level (AGL), risk, heading angle and fuel. Uncertainty in these input variables is also taken into consideration. The state space is defined as 3 dimensional Euclidean space (latitude, longitude and altitude). A graph data structure was derived from a cubic cell representation of this state space where each node is located in the corner with smallest x, y, z coordinate values. 1) Input Variables For the purposes of this case study, uncertainty in the AGL variable was modeled (based on sensor and SRTM data error) as Gaussian with a 90% confidence interval of 100ft. The uncertainty in the calculated risk values was arbitrarily modeled as Gaussian with a spread of 0.05×10-6 casualties per hour of flight. The cruising levels heading rule was implemented by calculating the minimum angle θa (as measured from the longitude axis) of deviation from the boundaries of the acceptable heading angles at that cruising level. − min θ ,180 − θ θ rl ≤ θ < θ ru (4) θa = otherwise min θ ,180 − θ where θ is the heading angle and θrl and θru are the lower and upper acceptable heading angles respectively. This value is then modelled as a Gaussian fuzzy number with a standard deviation of 10 degrees. Fuel was also modelled as a Gaussian fuzzy number dependent on the amount of fuel “consumed” and the estimated fuel needed to reach the start state Qe. The fuel required to reach state xi (given by Qi) from xi+1 (since this is a backwards search) is calculated recursively given the incremental fuel cost qi (refer (1)) associated with transitioning from xi+1 to xi: (5) Qi = Qi +1 + qi . At the same time, a heuristic estimate of the fuel required to reach the start state is computed based on (1). Then, given the known crisp value of total fuel available Qt, the
209
remaining fuel as a ratio of total fuel can be computed: 1 (6) Ri = ( Qt − Qi − Qe ) . Qt There are advantages in using Ri instead of Qi. Consider a state xi which has two candidate parent states xi′+1 and xi′′+1 each with equal fuel costs Qi+1; additionally, the incremental fuel cost to reach xi is equal. Therefore, it is impossible to distinguish between these two states in terms of fuel costs when considering just Qi. However, by incorporating Qe, it is now possible to distinguish between these two distinct states as the Euclidean distance to the start state is almost certainly different. A direct consequence of this is that more promising states will be investigated first which can potentially decrease the number of states explored. 2) Inferencing As multiple fuzzy inferencing operations are performed at each iteration of D* Lite, it is desirable to minimise the number of rules and also the number of antecedent and consequent MFs. The implemented fuzzy logic system is a four input single output system with 13 rules. When designing the fuzzy rule antecedent MFs, it should be noted that there are regions on the UoD for AGL, risk and heading that do not influence the suitability of the outcome. For example, values of risk less than the lower decision threshold are all equally desirable and do not change the suitability of the path choice. Therefore, these antecedent MFs were modelled as trapezoidal MFs. Fuel on the other hand requires a different response for every change on the input UoD; hence, triangular MFs were used. In designing the fuzzy rule base, it should be noted that numerous trade-offs are required. For example, an altitude of 500ft should be maintained unless when approaching mission waypoints. The heading angle should adhere to the cruising levels rule but this is not mandatory below 5000ft, hence it can be traded off against other objectives (such as if running low on fuel) when ‘necessary’. Additionally, it is necessary to enforce hard limits of a minimum of 50ft AGL to avoid impact with terrain, a maximum permissible risk of 10-6 casualties per flight hour and a maximum fuel load. These “hard limits” are difficult to enforce in a fuzzy system especially if centre of gravity (CoG) defuzzification is employed. This is because multiple rules may be fired which results in non-zero truncation values for other consequent MFs – this changes the CoG result. Instead, the one tailed confidence interval for the input fuzzy number (which are Gaussian) is calculated and compared with the hard limit. A confidence interval of 90% was arbitrarily chosen. IV. EXPERIMENTS AND DISCUSSION In this investigation, three different methods for multiobjective decision making were evaluated with regards to planner performance when integrated with D* Lite (as described in section IIID). The first method employed weighted sum aggregation (a similar approach to [14]-[16])
under the Analytic Hierarchy Process (AHP) [32]. This was compared with the NS fuzzy planner described in section III and a variant of that which used singleton fuzzification (hence it does not model input uncertainty). Both fuzzy planners also use the same rule base and therefore are expected to produce similar results. For the mission scenario described in Fig. 1, the planner is executed twice to find a path from waypoint 1 to waypoint 2, and then again from waypoint 2 to waypoint 3. The solution given by the NS fuzzy planner is shown in Fig. 4 along with the costs for each of the four objectives when traversing from waypoint 2 to waypoint 3. The three layers in each subplot correspond to the three AMSL cruising layers. A number of different scenarios were also simulated to evaluate the performance of these planners. One method for measuring planner performance is the computational complexity. This provides an indication of the practical feasibility of the planner. Computational complexity can be measured in terms of time latency and also in terms of the number of iterations of D* Lite (which, along with the number of states explored, heavily influences the computation time). As well, the total costs and incremental costs for each objective can be examined to evaluate the decision trade-offs that were made at each step. Unsurprisingly, AHP required the least computation time, followed by singleton fuzzy and non-singleton fuzzy planners. Fuzzy calculations incur greater processing delays per iteration. Furthermore, computation time increases with the overall path distance due to the need to explore more states (refer Fig. 5). It was also observed that when performing re-planning (or in-flight re-planning), D* Lite tends to only expands states that change the optimality of the current path. This was found to significantly reduce computation time as reported by [28],[30]. Given that there are 300 nodes, all three planners required fewer than O(V 3/2) iterations to find an optimal path (refer Fig. 5); this empirically verifies the computational feasibility of augmenting D* Lite with multi-objective decision making. As well, it was observed that the fuzzy based planners tended to explore fewer states than the AHP planner and thus use fewer search iterations as shown in Fig. 5. This can be attributed to the fact that fuzzy rules provide the ability to create highly non-linear relationships which results in a cost that more accurately reflects the suitability of a certain path choice. With AHP, the cost is calculated as a linear weighted sum of the constituent objectives. However, the need to make complex trade-offs often results in nonlinearities. For example, a lower risk trajectory should be followed except when the fuel levels are low, in which case higher levels of risk are acceptable (within the upper bounds of course). The inflexibility of AHP was demonstrated in several scenarios where even with sufficient fuel, the planner still adopts a higher risk path instead of reverting to the longer, but safer route.
210
APPENDIX C. PUBLICATIONS
Fig. 4 Mission flight plan (path shown by gray line) given by the NS fuzzy planner for the case study with variable values shown for each objective. Note that the planned path avoids regions that exceed hard limits and is of minimal length. Furthermore, the planner chooses to climb in altitude when flying east to comply with the flight levels rule and to avoid low terrain as stipulated by AGL requirements.
usage, aircraft position and map accuracy in real UAS operations. The effects of these hard limits become more pronounced over longer paths and can be attributed to the hard limits and the activation of more rules in the nonsingleton planner [27]. An example of average path risk is shown in Table 1. TABLE 1: AVERAGE PATH RISK (×10-7) Path length AHP Fuzzy NS Fuzzy 8
0.946
0.703
0.582
11
2.29
1.11
1.19
15
2.00
1.76
1.68
16
2.40
1.00
0.953
26
9.61
8.07
8.07
Fig. 5 Number of search iterations needed versus path length
Both fuzzy planners chose similar paths with similar costs for each objective. This was as expected given that the rule bases were identical. However, changing the rule base drastically changes the planned path; this shows that the rules significantly influence the performance of the planner; therefore, the challenge remains in effectively eliciting and encoding, as rules, the knowledge from human experts. The primary difference between the results of the two fuzzy planners arose from the handling of hard limits. All the paths in each experiment adhered to hard limits, but the nonsingleton fuzzy planner avoided some cells not avoided by the fuzzy planner, even though these cells were within the bounds of the hard limits. The application of fuzzy numbers and confidence intervals creates an extra buffer due to uncertainty. This is a desirable trait given uncertainty in fuel
As D* Lite has been shown to find the least cost path [28], therefore, all three planners return a globally optimal path which is of the least aggregated cost. This is supported by the fact that in every simulation, the path that each method returns is always of the same length in terms of the number of state transitions. Both AHP and fuzzy based planners are completely deterministic and produce identical responses to identical scenarios with deterministic time delays every time. This provides an advantage over evolutionary and randomised search based methods (e.g. ant colony optimisation) from a certification standpoint which is an important aspect in all aviation software systems. It is conceivable that a panel of experts could create and verify the fuzzy rule base by examining the output costs for all possible combinations of input singletons.
211
Even though a specific mission scenario was presented here, the methodology is extensible to a wide variety of multi-objective path planning problems. V. CONCLUSION The evaluation of three methods of augmenting D* Lite in mission flight planning has shown that fuzzy rule based methods provide distinct advantages over conventional AHP MCDM methods. These include fewer search iterations and more flexibility in trading-off between competing objectives. It was also found that the computational efficiency of the D* Lite algorithm is preserved even when evaluating multiple objectives. By incorporating uncertainty in the input variables, a more conservative path is constructed. The results demonstrate that NS fuzzy planning provides a computationally feasible method for finding an optimal path, under input uncertainty, for multiple objectives in three dimensional space. This is currently not possible with existing multi-objective path planning algorithms especially when the graph can be cyclic. By incorporating hardware fuzzy processing, it is envisaged that real time re-planning is possible [33]. Furthermore, the algorithm is completely deterministic and as such could be certified under existing aviation software certification standards. It was demonstrated through the case study that a NS fuzzy multi-objective mission flight planning algorithm process is well-suited in efficiently calculating an optimal mission flight plan. Therefore, this framework can be applied to a wide variety of UAS missions with different objectives (such as time and dynamic obstacles). ACKNOWLEDGMENT The authors wish to thank Gilles Coppin, Visiting Professor, Humans and Automation Laboratory, MIT for his helpful suggestions in the preparation of this paper. REFERENCES [1] "Partnering in the global context: An Australian aerospace industry for the 21st century, report of the aerospace industry action agenda," Department of Industry, Tourism and Resources (DITR) 2003. [2] M. T. DeGarmo, "Issues concerning integration of Unmanned Aerial Vehicles in civil airspace," MITRE, Center for Advanced Aviation System Development, McLean, Virginia 2004. [3] "UAV task-force final report, a concept for European regulations for civil unmanned aerial vehicles (UAVs)," The Joint JAA/EUROCONTROL Initiative on UAVs, 2004. [4] "Civil aviation regulations 1988 (CAR 1988)," Civil Aviation Safety Authority (CASA), Australia 2003. [5] R. Parasuraman, T. B. Sheridan, and C. D. Wickens, "A model for types and levels of human interaction with automation," IEEE Trans. Systems, Man and Cybernetics, Part A, vol. 30, pp. 286-297, 2000. [6] NASA Jet Propulsion Laboratory, (2006). Shuttle radar topography mission [Online]. Available: http://www2.jpl.nasa.gov/srtm/ [7] R. Clothier and R. Walker, "Determination and evaluation of UAV safety objectives," in 21st International Unmanned Air Vehicle Systems (UAVS) Conf., Bristol, United Kingdom, 2006. [8] R. Weibel and R. Hansman, "An integrated approach to evaluating risk mitigation measures for UAV operational concepts in the NAS," in AIAA 4th Infotech@Aerospace Conf., Arlington, Virginia, U.S., 2005.
[9] G. Marsters and M. Sinclair, "Integrating UAVs with conventional air operations: Some regulatory issues," in 18th Bristol Int. UAV Systems’ Conf., Bristol, United Kingdom, 2003. [10] R. Clothier, R. Walker, N. Fulton, and D. Campbell, "A casualty risk analysis for unmanned aerial system (UAS) operations over inhabited areas," in Twelfth Australian Int. Aerospace Congress (AIAC12), Melbourne, Australia, 2007. [11] P. Tompkins, A. T. Stentz, and W. R. L. Whittaker, "Mission-level path planning for rover exploration," in 8th Conf. on Intelligent Autonomous Systems (IAS-8), March 2004. [12] D. Ferguson and A. Stentz, "Using interpolation to improve path planning: The field D* algorithm," Journal of Field Robotics, vol. 23, pp. 79-101, 2006. [13] S. M. LaValle, Planning algorithms. New York: Cambridge University Press, 2006. [14] G. F. List, P. B. Mirchandani, M. A. Turnquist, and K. G. Zografos, "Modeling and analysis for hazardous materials transportation - risk analysis, routing scheduling and facility location," Transportation Science, vol. 25, pp. 100-114, 1991. [15] P. Leonelli, S. Bonvicini, and G. Spadoni, "Hazardous materials transportation: A risk-analysis-based routing methodology," Journal of Hazardous Materials, vol. 71, pp. 283-300, 2000. [16] M. Zhang, Y. Ma, and K. Weng, "Location-routing model of hazardous materials distribution system based on risk bottleneck," in Int. Conf. on Services Systems and Services Management (ICSSSM), 2005, pp.362-368 Vol. 1. [17] K. Fujimura, "Path planning with multiple objectives," IEEE Robotics & Automation Magazine, vol. 3, pp. 33-38, 1996. [18] B. S. Stewart and I. Chelsea C. White, "Multiobjective A*," J. ACM, vol. 38, pp. 775-814, 1991. [19] A. R. Soltani and T. Fernando, "A fuzzy based multi-objective path planning of construction sites," Automation in Construction, vol. 13, pp. 717-734, 2004. [20] N. Suzuki, D. Araki, A. Higashide, and T. Suzuki, "Geographical route planning based on uncertain knowledge," in Proc. 7th Int. Conf. on Tools with Artificial Intelligence, 1995, pp.434-441. [21] I. McManus, "A multidisciplinary approach to highly autonomous UAV mission planning and piloting for civilian airspace," Ph.D. Thesis, School of Engineering Systems, QUT, 2004. [22] E. H. Ruspini, P. P. Bonissone, and W. Pedrycz, Handbook of fuzzy computation. New York: Wiley, 1996. [23] C. Carlsson and R. Fuller, "Fuzzy multiple criteria decision making: Recent developments," Fuzzy Sets and Systems, vol. 78, pp. 139-153, 1996. [24] J. Rasmussen, "Diagnostic reasoning in action," IEEE Trans. Systems, Man and Cybernetics, vol. 23, pp. 981-992, 1993. [25] L. A. Zadeh, "Outline of a new approach to the analysis of complex systems and decision processes," IEEE Trans. Systems, Man and Cybernetics, vol. 3, pp. 28-44, 1973. [26] J. M. Mendel and R. I. B. John, "Type-2 fuzzy sets made simple," IEEE Trans. Fuzzy Systems, vol. 10, pp. 117-127, 2002. [27] G. C. Mouzouris and J. M. Mendel, "Nonsingleton fuzzy logic systems: Theory and application," IEEE Trans. Fuzzy Systems, vol. 5, pp. 56-71, 1997. [28] D. Ferguson, M. Likhachev, and A. T. Stentz, "A guide to heuristicbased path planning," in Int. Conf. on Automated Planning and Scheduling (ICAPS), Monterey, CA, 2005. [29] S. Koenig and M. Likhachev, "Improved fast replanning for robot navigation in unknown terrain," in Proc. IEEE Int. Conf. on Robotics and Automation, Washington, D.C., 2002, pp.968-975 vol.1. [30] C. Tovey, S. Greenberg, and S. Koenig, "Improved analysis of D*," in Proc. IEEE ICRA, 2003, pp.3371-3378 vol.3. [31] D. Bailey, A. Goonetilleke, and D. Campbell, "A new fuzzy multicriteria evaluation method for group site selection in GIS," Journal of Multi-Criteria Decision Analysis, vol. 12, pp. 337-347, 2003. [32] T. L. Saaty, "How to make a decision The analytic hierarchy process," European Journal of Operational Research, vol. 48, pp. 9-26, 1990. [33] P. Wu, P. Narayan, D. Campbell, M. Lees, and R. Walker, "A high performance fuzzy logic architecture for UAV decision making," in IASTED Int. Conf. on Computational Intelligence, San Francisco, 2006.
212
APPENDIX C. PUBLICATIONS
On-Board Multi-Objective Mission Planning for Unmanned Aerial Vehicles Paul Pao-Yen Wu Queensland University of Technology 2 George Street Brisbane, QLD 4000 +61-7-3138-1362
[email protected]
Duncan Campbell Queensland University of Technology 2 George Street Brisbane, QLD 4000 +61-7-3138-2179
[email protected]
Abstract—A system for automated mission planning is presented with a view to operate Unmanned Aerial Vehicles (UAVs) in the National Airspace System (NAS). This paper describes methods for modelling decision variables, for enroute flight planning under Visual Flight Rules (VFR). For demonstration purposes, the task of delivering a medical package to a remote location was chosen. Decision variables include fuel consumption, flight time, wind and weather conditions, terrain elevation, airspace classification and the flight trajectories of other aircraft. The decision variables are transformed, using a Multi-Criteria Decision Making (MCDM) cost function, into a single cost value for a grid-based search algorithm (e.g. A*). It is shown that the proposed system provides a means for fast, autonomous generation of near-optimal flight plans, which in turn are a key enabler in the operation of UAVs in the NAS. 1 2
Torsten Merz CSIRO ICT Centre P.O. Box 883, Kenmore 4069, Queensland, Australia +61-7-3327-4123
[email protected]
UAV mission planning is a complex multi-objective decision problem that must consider not just the rules of the air, but also mission efficiency objectives and safety objectives. Pre-flight planning is necessary in the risk management and subsequent approval of flight operations. In-flight replanning, on the other hand, is required when changes to the operating environment, to the UAV or to mission goals, invalidate the strategic plan. Because the UAV operates in a dynamic, outdoor environment, it is impossible to predict with certainty true operating conditions. Replanning is used to mitigate this uncertainty. For fixed wing UAVs, there is significant time pressure on in-flight planning as the vehicle is in constant motion. The benefits of automating the mission planning process onboard the UAV are twofold. Firstly, onboard mission planning can help increase the level of autonomy of the UAV. Onboard replanning ensures continued compliance with the rules of the air despite changes to the operating environment, even in the event of a communications failure. This is crucial for operation in the NAS [1]. In order to realise this capability, a level of autonomy is required whereby the UAV executes decisions made autonomously unless the human operator intervenes [3]. Given the size of the search space and the complexity of the decision problem, an automated mission planner can also serve as a decision support tool to aid the human operator. This can be especially beneficial in the scenario where the human operator is controlling multiple UAVs.
TABLE OF CONTENTS 1. INTRODUCTION .................................................................1 2. BACKGROUND...................................................................2 3. DECISION CRITERIA .........................................................2 4. MULTI-OBJECTIVE PATH PLANNING ..............................5 5. SIMULATION AND DISCUSSION .........................................6 6. CONCLUSION ....................................................................8 ACKNOWLEDGEMENTS ........................................................8 REFERENCES ........................................................................8 BIOGRAPHY ........................................................................10
1. INTRODUCTION Mission planning is an integral component in the integration of Unmanned Aerial Vehicles (UAVs) within the National Airspace System (NAS). In order to gain access to the NAS, it is necessary to demonstrate an Equivalent Level of Safety (ELOS) to that of human piloted aircraft [1]. ELOS comprises three major requirements: (i) see and avoid capability, (ii) compliance with existing aviation rules and regulations, and (iii) transparency of operation [1]. Mission planning, which comprises pre-flight (strategic) planning and in-flight (tactical) replanning, ensures conformance with the rules of the air [2], and thus helps to address ELOS.
This paper presents a framework for UAV mission planning that adapts the human operator’s multi-objective decision rules in generating a flight plan. For the purposes of demonstration, this paper adopts an example mission scenario involving delivery of a medical package to a remote location, using a small UAV. Such a mission, due to the non-standard locale and low altitude ceiling of small UAVs [4] is performed under Visual Flight Rules (VFR) [5]. Examples of small UAVs include the RQ-2 Pioneer (100nm range), RQ-7 Shadow (68nm range), and Aerosonde (3000km range), all of which have a ceiling of 15000ft [4]. However, the proposed framework is applicable to en-route flight planning in general.
1
Mission planning, in the context of UAV en-route flight planning, is a path planning problem. It involves finding a
1 2
978-1-4244-2622-5/09/$25.00 ©2009 IEEE. IEEEAC paper #1140, Version 5, Updated December 19, 2008
1
213
sequence of waypoints (or nodes) in 3D along with the estimated time of arrival at each waypoint, that link the start waypoint to a specified goal waypoint. In an intelligent control architecture, these waypoints are passed to the aircraft’s autopilot controller through a sequencer [6]. The integration of planning and control is beyond the scope of this paper. In determining intermediary waypoints, it is necessary to evaluate multiple decision criteria. A mission objective is satisfied when that objective’s constituent decision criteria are satisfied. For example, satisfaction of fuel and flight time criteria leads to satisfaction of the mission efficiency objective.
function (typically a weighted sum) with a search algorithm (such as A* [20]). This framework is described in [21]. It is necessary to aggregate decision variables into a single cost value because of “exponential growth in planning time and memory usage with dimensionality” [22]. Existing HAZMAT route planning methods, however, are constrained to 2D environments and do not consider variables such as wind. Rubio [23] presents a 3D UAV path planner that incorporates wind conditions to find a path that minimises fuel consumption. However, the rules of the air were not incorporated into the planning process. Gu [24] proposes a bi-objective UAV flight planner that optimises for risk and fuel costs; but wind information is not used.
Existing work (as described in section 2) fails to address the simultaneous requirements for computational efficiency, and multi-criteria decision making, necessary to on-board mission planning. Section 3 discusses the decision criteria relevant to en-route flight planning. The proposed algorithm, which combines a candidate method for multiobjective aggregation with an efficient path planning algorithm, is presented in section 4. Finally, the simulation outcomes are discussed in section 5.
There are also generic multi-objective search algorithms such as MOA* [25] and Fujimura’s [26] algorithm. MOA* only works for acyclic graphs (graphs derived from grids are cyclic) and Fujimura’s algorithm is not scalable to large search spaces. It can be seen that existing work does not adequately address the requirements for UAV mission planning. Furthermore, it is necessary to minimise the dimensionality of the problem due to the intractability of path planning [8, 22]. A minimum of three spatial dimensions are required for UAV flight planning. In addition, because of the presence of dynamic obstacles (e.g. weather, other aircraft), variable wind conditions, and a need to optimise for flight time, it is desirable to include a time dimension. This guarantees resolution completeness [8] and path optimality when using an algorithm like A*. For the following section on decision criteria, it is assumed that the search space is four dimensional.
It is shown through simulation that the proposed multiobjective planning approach is fast enough to meet the requirements of on-board mission planning. This is a key enabler in the operation of UAVs in the NAS.
2. BACKGROUND Mission planning belongs to the class of planning problems referred to as the weighted region problem [7]. For these problems, the transition costs between nodes are nonbinary, i.e. regions of the search space can not be classified as purely untraversable or purely free space [8]. This is because it is necessary to distinguish between path segments which may lie in “free space” (as in free of obstacles), but have different degrees of attractiveness when evaluated against multiple decision criteria. Note that mission planning is a form of path planning [8], as it finds a sequence of waypoints that link the start to the goal (destination) waypoint. This differs from trajectory planning [8], where the solution path is expressed in terms of the degrees of freedom of the vehicle and velocity/angle rates.
3. DECISION CRITERIA A number of key decision criteria for UAV en-route planning under VFR [5] are presented. This section discusses, for each criterion, the potential data source, data storage structure, and impact on the medical package delivery mission. For the planning task at hand, it is desirable to find the optimal (or least cost) path, where the cost represents the combined degree of satisfaction of all the decision criteria.
Existing methods for UAV flight planning, have focused predominantly on finding paths that satisfy vehicle dynamics while avoiding obstacles (e.g. [8-16]). This is similarly the case for many generic path planning algorithms (refer [8] for a comprehensive survey).
Time Emergency delivery of a medical package to a remote location requires reaching the goal destination in minimal time. The time of arrival at each waypoint corresponds to a unique node in the 4D search space. This arrival time is conditional on the wind vector, selected cruise velocity, and the predicted flight trajectory between waypoints.
There are many examples of multi-objective path planners in the field of HAZardous MATerials (HAZMAT) transportation [17-19]. This is due to the need to make trade-offs between risk and transportation costs. Existing work in HAZMAT route planning almost exclusively adopts the approach of combining a multi-criteria cost
The predicted flight trajectory is contingent on the structure of the search space. Much of the recent work for vehicle 2
214
APPENDIX C. PUBLICATIONS
The estimated time of arrival at a node can be expressed as a simple recurrence relation:
planning has focused on techniques in computational geometry using a grid [9, 23, 24, 27-35]. However, for conventional grid based planning (using a 4/8-connected or 6/26-connected neighbourhood for 2D or 3D respectively), the resultant flight trajectory has limited track angle resolution. For example, in 2D planning, the resolution is limited to increments of 45°. This can lead to sub-optimal paths even after application of path smoothing [28].
t ( s ′ ) = t ( s ) + τ ( s, s ′ )
(1)
where s, s’ are parent and child nodes respectively, t is the time of arrival at a node, and τ is the transition time between nodes.
A number of methods have been proposed which increase the angular resolution of the search space [27, 28, 30, 32, 35]. However, [30] does not find the least cost path and [35] requires a priori cell decomposition; this is computationally infeasible given the presence of dynamic obstacles. As well, it is not possible to use the methods described in [27, 28] as the track angle is derived from a priori knowledge of path costs. This is not possible since, for en-route planning, the path cost is itself dependent on the track angle (due to wind effects). Consider the approach presented in [32] whereby predefined trajectory segments are used to connect nodes (which are placed in the centre of cells). Unlike the 26connected neighbourhood, successive nodes do not necessarily lie in adjacent cells (see Figure 1). This is sometimes referred to as a vector neighbour [8]. It is possible to have arbitrary angular resolution using vector neighbours. However, [32] focuses on trajectory planning for a 2D vehicle. For the purposes of UAV mission planning, it is sufficient to approximate the actual path costs by assuming linear trajectories (as in Figure 1). This is possible when the cell size is large compared to the aircraft’s tracking error and turn radius, and the time spent manoeuvring between tracks is small compared to time spent on track. Such an assumption forgoes the need for inclusion of horizontal and vertical track angle dimensions, as turning circles are not considered. In addition, computation of ground speed is simplified when using linear tracks. This assumption is necessary to avoid “the curse of dimensionality” [22].
Figure 2 – 3D Neighbourhood operator Using predefined, linear, flight trajectories, the transition time is thus:
τ ( s, s ′ ) =
d ( s, s ′ )
(2) vc + vw ! ! where d is the horizontal distance between the nodes, vc is the cruise velocity vector and vw the wind velocity. Wind and weather forecasts in Australia are obtainable from Airservices Australia [36]. For long range flight, wind forecasts are available in GRIdded Binary (GRIB) format [23] with 1×1.25° resolution. As small UAVs have limited engine power [4], the wind can drastically affect flight time and constrain potential paths. Therefore, vw can not be ignored. Fuel A decision criterion that is in direct contention with minimisation of travel time is minimisation of fuel consumption. Calculation of fuel usage is based on a specified cruise velocity, traversal time between nodes (equation (2)), flight altitude, climb/descent angle, aircraft parameters (e.g. fuel mixture, throttle, propeller pitch), and atmospheric temperature and pressure. Such fuel calculations are platform specific. For the purposes of this simulation, a simple look up table is used based on the Aerosim fuel model [37].
Figure 1 – A vector neighbour The adopted neighbourhood operator is shown in Figure 2. It provides an average horizontal angle increment of 15° and allows for flight path angles of up to ±6° in approximately 3° increments for cells of size 1nm×1nm×1000ft (equivalent to 1852m×1852m×304.8m). 3
215
where Hi is defined based on the cells, x, y, z in the grid (which represents the world space) W:
Altitude Above Ground Level (AGL) Australian civil air regulation CAR157 stipulates that flight must be conducted at altitudes of at least 1000ft Above Ground Level (AGL) over populous areas, and 500ft AGL otherwise [2]. For each cell in the search grid, AGL is calculated by subtracting terrain elevation from the altitude of that cell. Because of the relatively short range of small UAVs [4], the curvature of the earth can be neglected. Thus, it is possible to use the 3D grid discussed previously. The altitude levels (on the z axis) on this grid are expressed in feet Above Mean Sea Level (AMSL). Terrain elevation information can be obtained from a Digital Elevation Map (DEM), such as the National Aeronautics and Space Administration (NASA) Shuttle Radar Topography Mission (SRTM) map [38]. Elevation information is available at a resolution 90m for many regions in the world [38].
H i = {( x, y, z ) ∈ W , W | fi ( x, y, z ) ≤ 0}
(4)
It is beneficial to model half spaces using both flat (6) and curved (7) surfaces, due to the presence of cylindrical regions. ax + by + cz − d
( x − xc )
2
2
+ ( y − yc ) − r 2
(5) (6)
From (6) and (7), it can be seen that only the parameters a, b, c, d, r are stored – thus, this is significantly more memory efficient than a grid.
Airspace Classes
Aircraft Separation Risk
En-route path planning is constrained by the different classes of airspace and the requirements for operation in each class. For the purposes of simulation, it is assumed that the UAV operates under VFR and has access to the NAS. In Australia, there are five major categories of airspace, namely class A, C, D, E and G [5]. Regions of airspace, as shown in Figure 3, are defined using altitudes (e.g. en-route airspace, class A and E) or, in terms of altitude and proximity to an aerodrome (class C, and D). Class G airspace covers all regions not defined otherwise. Only in a small number of cases are there more complicated airspace designations (such as special use airspace or military airspace). Airspace charts can be obtained from Airservices Australia [36].
Another requirement in the design of a flight path is the avoidance of other aircraft. By incorporating a priori knowledge of aircraft movement, it is possible to strategically avoid collision scenarios without activating emergency collision avoidance systems. Potentially, this information can be obtained from flight plans lodged with the regulatory body. Alternatively, it is also possible to obtain position and velocity information of other aircraft from surveillance systems such as Automatic Dependent Surveillance Broadcast (ADS-B) [39]. Aircraft must maintain a minimal vertical separation of 1000ft as part of the Reduced Vertical Separation Minimum (RVSM) requirement [40]. However, lateral separation standards vary depending on aircraft flight vectors, and navigation systems used. For the purposes of simulation, it is assumed that the lateral separation is 5nm. This is the proposed separation for aircraft in conflict that use ADS-B [39]; it is identical to the standard used for operations under Route Surveillance Radar (RSR) [40]. It can be seen that other aircraft can be modeled as cylindrical obstacles (using (4), (5)) with a radius of 5nm, and height of 2000ft. Unlike regions of airspace, where the position and extents are static and known, there is uncertainty in the predicted position of a moving aircraft. This uncertainty grows with time [14]. Uncertainty can be modeled using probabilistic methods (refer [41]), or approximated probabilistic methods (such as [14]). Consider the case where initial position uncertainty is purely a result of Global Positioning System (GPS) uncertainty. GPS error is typically modeled using the Gaussian distribution [42]. Present day GPS systems have a horizontal accuracy of 5-10m (95% confidence) and vertical accuracy approximately 1.4 times the horizontal accuracy [43]. These errors are small when compared to the cell size (1852m×1852m×304.8m). However, the accumulated position uncertainty can be much greater given uncertainty in the predicted velocity vector. Where there is no further information regarding the performance, and operator
Figure 3 – Example airspace regions The airspace is suited to a polyhedron based representation, especially cylinders (for class C, D airspace), and rectangular prisms (for class A, E, G). For the mission at hand, the en-route flight path avoids restricted airspace, and classes A, D and C airspace. These obstacle regions O are represented as the conjunction of half-spaces H: O = H 0 ∧ H1 ∧ ... ∧ H n
(3) 4
216
APPENDIX C. PUBLICATIONS
the maximum allowable risk for a give flight path (i.e. sequence of cells), whereas the risk floor, pb is some minimum bound below which the risk is deemed negligible. For example, if the UAV never approaches within 2σ of the mean, then the maximum risk of encroachment pb is less than 0.028.
intentions of other aircraft, this uncertainty is assumed to be Gaussian. For the purposes of simulation, a simple bivariate Gaussian model was employed, extending the approximation technique described in [14, 44] to a 3D grid. Given independent specifications for GPS horizontal and vertical accuracy, the aircraft position density function p can be expressed as:
p ( r , z, σ r , σ z ) =
1 e 2πσ rσ z
! 2 ( z − z ( t ) )2 r c −# 2 + ## 2σ 2σ z2 % r
2
( x − x (t )) + ( y − y (t ))
where r =
c
c
2
" $ $$ &
Storm Cell Risk An important safety consideration for UAV operation is the avoidance of storm cells and their associated turbulence. Information about storm cells and their movements are provided by the Bureau of Meteorology [45]. It is possible to model storm cells in the same manner as for aircraft (described previously). The primary difference would be a higher degree of uncertainty, not just in the velocity vector, but the size (radius and height) would also vary with time. For simulation purposes, these were assumed to be piecewise linear in time.
(7)
, and σr, σz is the
standard deviation. The expected position, ( xc ( t ) , yc ( t ) , zc ( t ) ) is assumed to be a piecewise linear function of the form:
Cruising Levels xc ( t ) = vxi t + c i
(8)
In Australia, civil air regulation CAR173 [2] assigns cruising altitudes to aircraft operating under VFR based on their heading angle. This minimises the risk of head-on collisions. Permissible cruising altitudes for aircraft on headings from 0° to 179° are at odd multiples of 1000ft plus 500ft AMSL (e.g. 1500ft, 3500ft, 5500ft AMSL). For headings between 180° and 359°, aircraft should cruise at even multiples of 1000ft AMSL plus 500ft (e.g. 2500, 4500, 6500ft AMSL). CAR173 is not mandatory below 5000ft, but, for safety purposes, it is preferable to conduct flight in accordance with CAR173 where terrain, weather, and traffic conditions permit.
where vi is the predicted velocity vector for segment i. The probability density field for the aircraft separation zone (5nm by 2000ft cylinder), given independence between r and z, can derived from (7) using:
β ( r , z, σ r , σ z ) = z−H
r −R
z+H ∞
r+R ∞
' ' ' '
−∞ −∞
Π R ( r ′ ) Π H ( z ′ ) p ( r ′, z ′, σ r , σ z )dr ′dz ′ (9)
Π R ( r ′ ) Π H ( z ′ ) p ( r ′, z ′, σ r , σ z )dr ′dz ′
where Π is the rectangle (or gate) function, R is the radius (5nm), and H the half-height RVSM (1000ft). From an implementation perspective, (9) can be approximated using numerical means. For a given trajectory segment, the risk of encroachment (or aircraft separation risk) is the sum of the probabilities calculated using (9) for each cell on the trajectory. This is possible given that the cell size is small relative to 2R and 2H.
Population Risk The two primary safety concerns for operation of UAVs in the NAS are that of midair collision and flight termination in a populated area [46]. For the simulation studies, the risk presented to people on the ground as a result of flight termination is incorporated into the decision process. This risk value, which can be calculated using [46], is expressed as the number of ground casualties per flight hour.
In actuality, the estimated velocity v is also a random variable; however, recall that (10) has to be evaluated at every iteration in a search. To minimise computational complexity, the methodology presented in [14] is adopted by modelling σ as some function of time. For example, [14] proposes a model for σ based on the acceleration capability of the aircraft.
4. MULTI-OBJECTIVE PATH PLANNING The preceding section highlighted the numerous decision criteria and constraints involved in UAV mission planning. These extend beyond simply finding a shortest path that avoids obstacles. It can be seen that the decision space comprises 9 dimensions: x, y, z, t, fuel, aircraft separation risk, storm cell risk, heading angle, and population risk. Clearly, this highlights the intractability of path planning [8]. However, it can be noted that, based on the models presented in section 3, it is possible to derive all decision variables uniquely, given a waypoint x, y, z, t. An aggregated cost value can then be calculated based on the
By applying thresholds to (10), it can be seen that the separation zone is cylindrical (refer Figure 4 in section 5). Consider the scenario where two risk thresholds (pt, pb) are selected. Note that the (pt, pb) do not correspond to the likelihood of a fatal event (i.e. midair collision); they instead describe the likelihood of encroachment on the minimal separation requirement. The safety threshold, pt, is 5
217
previously described decision variables using a MultiCriteria Decision Making (MCDM) cost function.
The decision variables, also referred to as attributes, are incommensurate; for example, a fuel consumption of 0.2kg is not comparable with a storm cell risk of 0.03. One approach for calculating a cost from such incommensurate variable values is to use Multi-Attribute Utility Theory (MAUT) [49].
Planner Architecture The proposed planner adopts the approach of combining a heuristic search algorithm with a MCDM cost function [1719, 21]. Heuristic search algorithms are based on the dynamic programming equation [8]: g ( s ′ ) = g ( s ) + c ( s, s ′ )
MAUT provides a methodology for modelling decision maker preferences where preferences are represented as binary relations between objects (i.e. prefer A to B). The methodology comprises a two step process: (i) mapping of all decision variables onto a common scale using a set of value (or utility) functions, and (ii) computation of a single cost or utility value on the common scale. Typically, the output utility value is defined on (0,1). Methods for implementing step (ii) include the Choquet integral and weighted sum aggregation. The Choquet integral is a powerful, generic aggregation function that degenerates into the weighted sum when all decision variables are independent. However, weighted sum aggregation was chosen over the Choquet integral due to the computational complexity of evaluating a Choquet integral at every iteration, for every candidate child node. [49]
(10)
where s, s’ are parent and child nodes, g(s) is the total cost of the least cost path P from the start node to s, and c is the cost incurred by the trajectory segment between s and s’. Each node, s, is located in the centre of a grid cell in a four dimensional search space. Using the adopted planner architecture, c is a MCDM cost function. When using A*, c must be non-negative, and non-zero [20]. A* [20] has been selected as a suitable heuristic search algorithm as it finds the least cost path efficiently [8]. Despite the need for in-flight re-planning, a heuristic replanning algorithm (e.g. D* Lite [29]) was not selected because of the presence of dynamic elements. For example, changing wind conditions and storm cell or aircraft movements can invalidate prior search results for a large number of nodes in the search space. In these scenarios, replanning algorithms are less efficient than planning from scratch using A* [47].
5. SIMULATION AND DISCUSSION Preliminary analysis of the proposed planning framework for UAV mission planning was conducted in simulation. The planning algorithm was evaluated on a number of randomly generated UAV medical delivery scenarios to ascertain performance (Figure 4). This was measured in terms of computation time and path cost (when comparing a near-optimal to the optimal path). Each scenario has a search space size of 100nm×100nm×20000ft×250min. All simulations were run in Matlab on a 3.3GHz Intel Core 2
Decision Function Mission planning is a task that is performed proficiently at present by human experts [48]. Therefore, by capturing the decision strategies and preferences of a human expert, it is possible to find paths that best satisfy mission objectives.
Figure 4 – Simulated mission planning scenario showing moving aircraft separation zones 6
218
APPENDIX C. PUBLICATIONS
Duo CPU with 4GB of RAM running 32-bit Windows XP. An example of the decision criteria for a simulated scenario is depicted in Figure 4, Figure 5 and Figure 6. Note the complexity of the planning problem, which is compounded by an operating environment that contains dynamic elements, such as weather and other aircraft. Despite the presence of multiple path constraints and decision criteria, the planner finds always finds a path that meets these constraints (when one exists). The planned path tends to follow that of a straight line connecting the start and the goal waypoints. This is to be expected as A* finds the least cost path, and all trajectory transition costs between nodes are non-zero and non-negative.
Figure 6 – Illustration of wind and storm cells (gray cylinders) over green terrain An optimal solution, calculated using A* is compared with a near-optimal solution using an inflated heuristic [50]. A statistical box plot of computation time is shown in Figure 7.
Figure 5 – Illustration of restricted airspace The use of a multi-objective decision function enables the planner to find a path that addresses multiple, disparate, decision objectives. Consider for instance Figure 5 where the planner selects a path that does not overfly highly populated regions (darker region) whilst avoiding restricted airspace. Simultaneously, the path also meets the altitude requirements as per the cruising levels rule where the flight is operated at altitudes of 3500, 5500 and 7500ft given a north-easterly heading (Figure 4). Additionally, the flight path also takes into account wind and storm cells (Figure 6) in the minimisation of flight time and fuel consumption.
Figure 7 – Algorithm computation time It is clear that inflating the heuristic drastically reduces computation time. Over 184 different simulations, a nearoptimal solution is obtained with a mean computation time of 0.104s and an inter-quartile range of 3.3ms. This compares with a mean time of 32.1s and inter-quartile range of 17.8s for the optimal solution. The near-optimal solution is, on average, 30% more costly (in terms of the unit-less, aggregated cost) than the optimal. An increase in path cost is traded for a significant saving in computation time. The near optimal solution is obtained, by inflating (multiplying) the heuristic term in the optimal algorithm (e.g. A*), by a constant factor ε. It has been shown that total cost of the near-optimal solution is at most ε times the optimal [47].
Finding a path that satisfies multiple objectives is just one aspect of on-board UAV mission planning. The planner itself must be fast enough to meet the time constraints imposed on in-flight replanning. To analyse the computational efficiency of the algorithm, a number of complex mission scenarios were simulated; the simulation cases are complex in terms of the number of and movement of dynamic obstacles, terrain shape, and varying wind conditions.
The proposed algorithm not only meets the requirements of multi-objective en-route planning, but also meets the timeconstraints of in-flight re-planning. Selection of an optimal or near-optimal solution is dependent on the time available 7
219
for planning, which in turn depends on the current position, flight trajectory, and environmental conditions. Given the neighbourhood operator defined in section 3, all path waypoints have a minimum displacement of 3nm, which, at 50m/s, is traversed in 111.1s. Hence, it may be possible to find an optimal solution. There are no such time constraints on pre-flight planning.
REFERENCES [1]
[2] [3]
6. CONCLUSION This paper presented a system for automated, on-board mission planning for the purpose of operating UAVs in the NAS. In order to meet the rules of the air, safety objectives and mission efficiency objectives, it is shown that multiobjective planning is required. For the purposes of UAV enroute planning under VFR, the relevant planning criteria were found to be time, fuel, AGL altitude, airspace type, aircraft separation, the cruising levels rule, storm cells and population risk. To improve the modelling of time and fuel, a 3D vector neighbourhood operator was proposed to enable arbitrary angular resolution. Additionally, it was shown that airspace is suited to geometric modelling using cylinders and polyhedrons. This concept is extensible to the modelling of other aircraft and storm cells.
[4] [5] [6]
[7]
[8] [9]
Through simulation studies, it was found that the proposed planner, which combines a weighted sum MCDM cost function with A*, is efficient, and finds a path that satisfies multiple decision objectives. This algorithm finds a nearoptimal solution (with a cost that is on average 30% greater than the optimal) in a mean time of 0.104s, and an optimal one in 32.1s. Hence, the algorithm is suited to meeting the mission planning requirements for operation of UAVs in the NAS.
[10]
[11] Ongoing work includes improving existing probability density field approximations (equation (9)), and application of the algorithm to different planning scenarios. Of particular interest are missions conducted in windy conditions over mountainous terrain (updrafts and downdrafts). This is of interest due to the significant impact of wind on the safety and fuel efficiency of a small UAV.
[12]
[13]
[14]
ACKNOWLEDGEMENTS This work was supported by the Australian Research Centre for Aerospace Automation (ARCAA), Queensland University of Technology (QUT) and the Commonwealth Scientific and Industrial Research Organization (CSIRO), funded in part by the CSIRO ICT Centre top-up scholarship.
[15]
[16]
8
"UAV Task-Force Final Report, A Concept For European Regulations For Civil Unmanned Aerial Vehicles (UAVs)," The Joint JAA/EUROCONTROL Initiative on UAVs, 2004. "Civil Aviation Regulations 1988 (CAR 1988)," Civil Aviation Safety Authority (CASA), Australia, 2003. R. Parasuraman, T. B. Sheridan and C. D. Wickens, "A model for types and levels of human interaction with automation," IEEE Trans. Systems, Man and Cybernetics, Part A, vol. 30, pp. 286-297, 2000. Office of the Secretary of Defense, "Unmanned Aircraft Systems Roadmap: 2005-2030." Washington DC, 2005. Civil Aviation Safety Authority, "Visual Flight Rules Operations." Australia, 2001. P. Narayan, P. Wu, D. Campbell and R. Walker, "An Intelligent Control Architecture for Unmanned Aerial Systems (UAS) in the National Airspace System (NAS)," 22nd International Unmanned Air Vehicle Systems Conference, Melbourne, Australia 2007. S. Kambhampati and L. Davis, "Multiresolution path planning for mobile robots," IEEE Journal of Robotics and Automation, vol. 2, pp. 135-145, 1986. S. M. LaValle, Planning Algorithms. New York: Cambridge University Press, 2006. Y. Kim, D.-W. Gu and I. Postlethwaite, "Real-time path planning with limited information for autonomous unmanned air vehicles," Automatica, vol. 44, pp. 696712, 2008. I. K. Nikolos, K. P. Valavanis, N. C. Tsourveloudis and A. N. Kostaras, "Evolutionary algorithm based offline/online path planner for UAV navigation," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 33, pp. 898-912, 2003. P. O. Pettersson and P. Doherty, "Probabilistic Roadmap Based Path Planning for an Autonomous Unmanned Aerial Vehicle," ICAPS 2004. B. Pfeiffer, R. Batta, K. Klamroth and R. Nagi, "Probabilistic modeling for UAV path planning in the presence of threat zones," Operations Research, 2005. Y.-H. Qu, Q. Pan and J.-G. Yan, "Flight path planning of UAV based on heuristically search and genetic algorithms," IEEE Industrial Electronics Society 32nd Annual Conf. 2005, pp.5. D. Rathbun, S. Kragelund, A. Pongpunwattana and B. Capozzi, "An evolution based path planning algorithm for autonomous motion of a UAV through uncertain environments," Proceedings 21st Digital Avionics Systems Conference 2002, pp.8D2-1-8D2-12 vol.2. G. Yang and V. Kapila, "Optimal path planning for unmanned air vehicles with kinematic and tactical constraints," 41st IEEE Conf. Decision and Control 2002, pp.1301-1306 vol.2. I. McManus, "A multidisciplinary approach to highly autonomous UAV mission planning and piloting for civilian airspace," Ph.D. Thesis, School of Engineering Systems, QUT, 2004.
220
APPENDIX C. PUBLICATIONS
[17] G. F. List, P. B. Mirchandani, M. A. Turnquist and K. G. Zografos, "Modeling and analysis for hazardous materials transportation - risk analysis, routing scheduling and facility location," Transportation Science, vol. 25, pp. 100-114, 1991. [18] P. Leonelli, S. Bonvicini and G. Spadoni, "Hazardous materials transportation: a risk-analysis-based routing methodology," Journal of Hazardous Materials, vol. 71, pp. 283-300, 2000. [19] M. Zhang, Y. Ma and K. Weng, "Location-routing model of hazardous materials distribution system based on risk bottleneck," Int. Conf. on Services Systems and Services Management (ICSSSM) 2005, pp.362-368 Vol. 1. [20] P. Hart, N. Nilsson and B. Rafael, "A formal basis for the heuristic determination of minimum cost paths," IEEE Trans. Systems Science and Cybernetics, vol. 4, pp. 100-107, 1968. [21] P. Wu, R. Clothier, D. Campbell and R. Walker, "Fuzzy multi-objective mission flight planning in unmanned aerial systems," IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making, Honolulu, Hawaii 2007. [22] W. B. Powell, Approximate dynamic programming. New Jersey: Wiley-Interscience, 2008. [23] J. C. Rubio and S. Kragelund, "The trans-pacific crossing: long range adaptive path planning for UAVs through variable wind fields," 22nd Digital Avionics Systems Conference 2003, pp.8.B.4-81-12 vol.2. [24] D.-W. Gu, W. Kamal and I. Postlethwaite, "A UAV waypoint generator," AIAA 1st Intelligent Systems Conf. Chicago, IL, 2004. [25] B. S. Stewart and I. Chelsea C. White, "Multiobjective A*," J. ACM, vol. 38, pp. 775-814, 1991. [26] K. Fujimura, "Path Planning with Multiple Objectives," IEEE Robotics & Automation Magazine, vol. 3, pp. 3338, 1996. [27] J. Carsten, D. Ferguson and A. Stentz, "3D Field D*: Improved path planning and replanning in three dimensions," IEEE Int. Conf. Intelligent Robots and Systems 2006, pp.3381-3386. [28] D. Ferguson and A. Stentz, "Using interpolation to improve path planning: the Field D* algorithm," Journal of Field Robotics, vol. 23, pp. 79-101, 2006. [29] S. Koenig and M. Likhachev, "Improved fast replanning for robot navigation in unknown terrain," IEEE Int. Conf. Robotics and Automation 2002, pp.968-975 vol.1. [30] A. Nash, K. Daniel, S. Koenig and A. Felner, "Theta*: Any-angle path planning on grids," AAAI Conf. on Artificial Intelligence. Vancouver, BC, 2007. [31] C. Petres, Y. Pailhas, P. Patron, Y. Petillot, J. Evans and D. Lane, "Path planning for autonomous underwater vehicles," Robotics, IEEE Transactions on, vol. 23, pp. 331-341, 2007.
[32] M. Pivtoraiko and A. Kelly, "Generating near minimal spanning control sets for constrained motion planning in discrete state spaces," IEEE/RSJ Int. Conf. Intelligent Robots and Systems 2005, pp.3231-3237. [33] N. Suzuki, D. Araki, A. Higashide and T. Suzuki, "Geographical route planning based on uncertain knowledge," Proc. 7th Int. Conf. on Tools with Artificial Intelligence 1995, pp.434-441. [34] P. Tompkins, A. T. Stentz and W. R. L. Whittaker, "Mission-level path planning for rover exploration," 8th Conf. on Intelligent Autonomous Systems (IAS-8) March 2004. [35] A. Yahja, S. Singh and A. Stentz, "An efficient on-line path planner for mobile robots operating in vast environments," Robotics and Autonomous Systems, vol. 33, 2000. [36] Airservices Australia, (2008, 24 Oct, 2008). Document and chart descriptions, http://www.airservicesaustrali.com/publications/docdesc rip.asp [37] Unmanned Dynamics LLC, "Aerosim blockset version 1.1 user's guide." Hood River, OR, 2003. [38] NASA Jet Propulsion Laboratory, (2006). Shuttle radar topography mission, http://www2.jpl.nasa.gov/srtm/ [39] Civil Aviation Safety Authority, "Airworthiness approval of airborne automatic dependent surveillance broadcast equipment," 2007. [40] International Civil Aviation Organisation, "Procedures for air navigation services. Air traffic management," Fifteenth ed, 2007. [41] S. Thrun, W. Burgard and D. Fox, Probabilistic robotics. Cambridge, MA: MIT Press, 2005. [42] B. W. Parkinson, Global positioning system : theory and applications. Washington, DC: American Institute of Aeronautics and Astronautics, 1996. [43] K. D. McDonald, "The Modernization of GPS: Plans, New Capabilities and the Future Relationship to Galileo," Journal of Global Positioning Systems, vol. 1, pp. 1-17, 2002. [44] D. Rathbun and B. Capozzi, "Evolutionary approaches to path planning through uncertain environments," 1st AIAA Technical Conference and Workshop on Unmanned Aerospace Vehicles. Portsmouth, Virginia: AIAA, 2002. [45] Bureau of Meteorology, (2008). Aviation weather services, http://www.bom.gov.au/reguser/by_prod/aviation/ [46] R. Clothier, R. Walker, N. Fulton and D. Campbell, "A casualty risk analysis for Unmanned Aerial System (UAS) operations over inhabited areas," Twelfth Australian International Aerospace Congress (AIAC12), Melbourne, Australia 2007. [47] D. Ferguson, M. Likhachev and A. T. Stentz, "A guide to heuristic-based path planning," Int. Conf. on Automated Planning and Scheduling (ICAPS), Monterey, CA, June 2005.
9
221
[48] E. L. Deitch, "Learning to land: A qualitative examination of pre-rlight and in-flight decision-making processes in expert and novice aviators," Virginia Polytechnic Institute and State University, Falls Church, Virginia, Nov 21, 2001. [49] M. Grabisch, I. Kojadinovic and P. Meyer, "A review of methods for capacity identification in Choquet integral based multi-attribute utility theory: Applications of the Kappalab R package," European Journal of Operational Research, vol. 186, pp. 766-785, 2008. [50] M. Likhachev, "Search-based planning for large dynamic environments," School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 2005.
BIOGRAPHY Paul Wu is a PhD candidate at the Queensland University of Technology. He completed his degree in Electrical and Computer Engineering with honours at QUT in 2005 and obtained a masters degree in Computer and Communications in the same year. He has worked on a variety of projects, including a system for delivery of multimedia content to mobile phones, for which he was awarded the Engineers Australia Queensland Division J H Curtis Award, and performance optimised Field Programmable Gate Array (FPGA) fuzzy logic processing. He is currently undertaking research on algorithms and architectures for UAV path planning. Dr Duncan Campbell is an Associate Professor at the Queensland University of Technology (QUT), in Brisbane, Australia. He is the Alternate Head of the School of Engineering Systems. Dr Campbell has over 65 internationally peer reviewed papers and researches in the areas of robotics and automation, embedded systems, computational intelligence, intelligent control and decision support. He leads a group in the Australian Research Centre for Aerospace Automation and has collaborations with a number of universities around the world including Massachusetts Institute of Technology, USA and TELECOM-Bretagne, France. Dr Campbell is currently the IEEE Queensland Section chapter chair of the Control Systems / Robotics and Automation Society Joint Chapter.
10
Dr Torsten Merz is a senior research scientist with the CSIRO ICT Centre, Brisbane, Australia. He currently leads a project on the development of unmanned helicopters for civilian applications such as infrastructure inspection and search and rescue. He previously held a position as assistant professor at the Department of Computer and Information Science, Linköping University, Sweden. He was part of the WITAS UAV project team and involved in the development of flight control modes and mission planners for unmanned helicopters. He earned a Diploma in Informatics from the University of Bielefeld and received a Doctor of Engineering from the University of Erlangen-Nuremberg, Germany.
222
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
1
Multi-Objective 4D Vehicle Motion Planning in Large Dynamic Environments Paul P.-Y. Wu, Member, IEEE, Duncan Campbell, Member, IEEE, Torsten Merz, Member, IEEE
Abstract—This paper presents Multi-Step A* (MSA*), a variation of the A* search algorithm for multi-objective 4D vehicle motion planning (three spatial and one time dimension). The research is principally motivated by the need for offline and online motion planning for autonomous Unmanned Aerial Vehicles (UAVs). For UAVs operating in large, dynamic and uncertain 4D environments, the motion plan consists of a sequence of connected linear tracks (or trajectory segments). The track angle and velocity are important parameters that are often restricted by assumptions and grid geometry in conventional motion planners. Many existing planners also fail to incorporate multiple decision criteria and constraints such as wind, fuel, dynamic obstacles and the rules of the air. It is shown that MSA* finds a cost optimal solution using variable length, angle and velocity trajectory segments. These segments are approximated with a grid based cell sequence that provides an inherent tolerance to uncertainty. Computational efficiency is achieved by using variable successor operators to create a multi-resolution, memory efficient lattice sampling structure. Simulation studies on the UAV flight planning problem show that MSA* meets the time constraints of online replanning and finds paths of equivalent cost but in a quarter of the time (on average) of vector neighbourhood based A*. Index Terms—path planning, heuristic algorithms, multiobjective decision making, unmanned aerial vehicles.
A
I. I NTRODUCTION
N important component in the operation of vehicles in large, four dimensional (three spatial and one time dimension) dynamic environments is motion planning. This involves finding an optimal (least cost) or near-optimal sequence of 4D states that connect the initial vehicle state to a desired goal state [1]. For many applications, it is necessary to plan offline (e.g. prepare a mission plan for regulatory approval) and replan online when planning assumptions are invalidated by in-mission changes. The research is principally motivated by the operation of robotic vehicles, namely autonomous Unmanned Aerial Vehicles (UAVs) and equivalently, Autonomous Underwater Vehicles (AUVs). UAVs and AUVs are characterised by: (i) operation in large, outdoor environments, (ii) movement in three dimensions (x, y, z), (iii) uncertain and dynamic operating environment, The work was supported in part by the Australian Research Centre for Aerospace Automation (ARCAA) and the Commonwealth Scientific and Industrial Organisation (CSIRO) Information and Communication Technologies (ICT) Centre top-up scholarship Mr Paul Wu, is a graduate student with ARCAA, Queensland University of Technology (QUT), GPO Box 2434, Brisbane, 4000, Australia. (phone: +61 7 31381400; e-mail:
[email protected]) Associate Professor Duncan Campbell, is with the School of Engineering Systems and ARCAA, QUT, Brisbane, 4000 Australia. (e-mail:
[email protected]) Dr Torsten Merz is a senior research scientist with the CSIRO ICT Centre, Brisbane, Australia. (e-mail:
[email protected])
(iv) presence of environmental forces that affect motion (winds or currents), and (v) differential constraints on movement [2], [3]. Because of (ii) and (iii), the planning space must be four dimensional. Note that a dynamic environment refers not only to moving obstacles, but also to changing weather conditions. The proposed work mitigates the uncertainty inherent in a dynamic environment through online replanning and incorporation of tolerances in the planning process. There is significant time pressure on the in-flight replanning process. The motion plan is constrained by vehicle dynamics (such as maximum climb/ascent rate), environmental constraints (e.g. static and dynamic obstacles and wind/current) and rules of the air/sea. In addition, the planned path must satisfy multiple, possibly conflicting objectives such as fuel efficiency and flight time. Due to the “curse of dimensionality” [4], it is not computationally feasible to plan in a high dimensional search space consisting of all the aforementioned variables. It is common, instead, to plan the path in the world space (x, y, z, t) [1] by aggregating the decision variables into a single, non-binary cost term [4]. This planning problem is a type of weighted region path planning [5]. One of the unique UAV/AUV characteristics listed above is the presence of winds (or currents). These constrain vehicle movements and affect travel time and fuel consumption. In the presence of wind, it is especially important to have high track angle resolution as low track angle resolution can result in suboptimal paths that contain spurious turns [6], [7]. This is a shortcoming of conventional search grids as the track angle is in increments of 45◦ . Note that 4D motion planning as described here should not be confused with trajectory planning, which finds a path expressed in terms of the degrees of freedom of the vehicle and velocity/angle rates [1]. Instead, a 4D motion plan comprises a geo-referenced sequence of 3D waypoints and the desired track velocities between them. In this paper, such tracks are also equivalently referred to as trajectory segments. Typically, this form of planning resides in the topmost deliberative layer in conventional intelligent control architectures [8]. However, it is still necessary to incorporate an approximation of vehicle dynamics (which feature in the lower layers of an intelligent control architecture) to ensure that generated paths are physically realisable. This paper presents Multi-Step A* (MSA*), a method for 4D vehicle motion planning based on variable length, angle and velocity trajectory segments. Section II reviews existing path planning techniques. Based on A* [9], the proposed method is presented in Section III and shown to be cost optimal. To take advantage of variable trajectory segments,
223
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
a memory and time efficient multi-resolution lattice structure is proposed in section IV. A simulation study of MSA* for the UAV flight planning task is discussed in Section V. Analysis of the simulation results and a comparison of MSA* with existing work is discussed in section VI. II. E XISTING W ORK There exists a multitude of generic motion planning algorithms for finding the shortest path through a field of obstacles (for a comprehensive survey, see [1]). Much of the recent work for vehicle planning has focused on techniques in computational geometry using a grid [6], [7], [10]–[19]. However, a shortcoming of many grid-based approaches (e.g. [10], [12], [17], [19], [20]) is that the resultant path is confined to track angles that are multiples of 45◦ . As a result, the path can be sub-optimal and may contain spurious turns [6], [7]. A lack of regular high resolution track angles also affects methods based on Voronoi graphs (e.g. [14], [21]– [23]), methods that use probabilistic sampling (e.g. [24]–[26]), and generally methods where path angles are not considered (e.g. [27]–[29]). A review of motion planning algorithms is provided below for methods that address the requirements of vehicle motion planning vis a vis the track angle problem, wind/current effects and multi-objective optimisation. A. Methods with High Resolution Track Angles A number of grid based methods determine the track angle in continuous space (effectively infinite resolution) instead of sampling from predefined discrete track angles. Field D* [6] and 3D Field D* [16] for example interpolate the costs of adjacent cells to determine the track angle. This allows for arbitrary track angles but assumes that cell costs are known a priori. In a vehicle motion planning, this approach is not feasible given a 4D search space as the cost is dependent on the track angle. Theta* [7] works in a similar vein but unlike Field D*, a child or successor cell/node is not necessarily adjacent to the parent/source cell. Instead, the parent is chosen as either the start cell or a cell as close as possible to the start cell such that the resultant trajectory does not intersect any obstacles. Even though the method enables continuous angle resolution, it clearly does not find a path that optimises for the weighted region problem [5]. A similar method to Theta*, named A3D is presented by Kim et al. [18] for UAV trajectory planning. A3D also uses straight line projections from the current cell to the goal cell in constructing a path. A shortcoming of this methodology is the tendency to find paths that follow the boundaries of obstacles or threat zones. Like Theta*, A3D does not find the least cost path - rather it finds a path that does not exceed given risk thresholds. Pivtoraiko and Kelly [15] present an alternate method that provides regular, high resolution track angles by defining a successor operator (i.e. parent child cell relationships) that has a predetermined number of successors at selected track angles. Like Theta*, parent cells are not necessarily adjacent to child cells, hence the notion of a vector neighbourhood [1]. However, the method is formulated for 2D vehicle planning
2
with no consideration for winds/currents and is focused on satisfying vehicle dynamic constraints (i.e. trajectory planning [1] instead of path planning). Like [15], Yahja et al. [11] also present a method with predefined, discrete track angles. This method employs framed quadtrees and octrees for 2D and 3D planning respectively. Nodes are placed on the boundaries of each quad/octree decomposed cell without increasing the overall resolution of the search space (refer [11, Fig. 4]). This memory efficient sampling scheme enables high resolution track angles when traversing from one border of a cell to another. Unfortunately, transitions between cells are again limited to increments of 45◦ . Additionally, neither [11] nor [15] consider wind effects. A number of existing methods for incorporating wind (or current) effects use a constant weighting value for each polygonal (or polyhedral) shaped region to model a constant wind vector [30]–[33]. These methods exploit the geometric properties of such a representation to reduce the number of nodes in the search space [30], [31]. However, these methods do not consider multiple objectives and are not suited for planning in a dynamic environment. Another method for incorporating current effects in the planning process is presented by Petres et al. [17] for AUV motion planning. The algorithm is a heuristic directed method based on a fast marching algorithm (a search methodology that assumes constant traversal cost between nodes). This assumption does not hold for a multi-objective weighted region problem. In the UAV field, Rubio and Kragelund [13] incorporate wind effects to optimise fuel consumption for long distance flights. They combine linearly interpolated grid based wind forecasts with a planner based on artificial evolution. Uncertainty in the wind forecast is mitigated by online replanning. The presented method, however, does not consider other decision objectives such as flight time and the rules of the air. There are a number of planners based on artificial evolution (e.g. [13], [24], [25]) that find an obstacle free path (or in the probabilistic case, one that does not exceed risk thresholds) whilst incorporating vehicle dynamic constraints. With the exception of [13], these planners do not incorporate wind/current effects. Since the search space is non-discrete, the track angle resolution is infinite. The primary shortcoming for these evolutionary methods, however, is the inability to specify bounds on computation time or solution quality [34]. This can be problematic for online replanning (due to the time constraints imposed on planning) and for applications where determinism is a regulatory requirement (e.g. DO178-B [35] for aviation software). B. Multi-Objective Planning Algorithms None of the previously quoted methods explicitly address the requirements of multi-objective planning though many incorporate multiple motion constraints (e.g. water currents and vehicle dynamics in [17]). Examples of explicit multiobjective planning algorithms can be found in the study of HAZardous MATerials (HAZMAT) transportation due to the need for fuel efficient and low risk (presented to the population) paths [36]. These algorithms combine a a multi-objective
224
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
decision function (typically a weighted sum) with a graph search algorithm (such as A* or Dijkstra’s algorithm) on a grid [36]–[39] (refer [40] for a description). This methodology is also used by Gu [14] for a bi-objective (risk and fuel objectives) UAV motion planner based on Voronoi diagrams. Unfortunately, the plans generated by [14] tend to be globally sub-optimal as the path follows Voronoi lines. Additionally, the aforementioned HAZMAT planning methods are confined to 2-dimensional ground vehicles and none incorporate the effects of wind. These multi-objective planning algorithms (e.g. [14], [36]– [39]) almost universally adopt a global planning methodology where the track cost is calculated online much like a lazy probabilistic roadmap [1]. Furthermore, the roadmap itself (comprising nodes and the tracks/trajectories that link nodes) can be represented implicitly to obtain an even lazier algorithm [27]. This is necessary because it is computationally expensive to compute a priori all possible trajectory costs for all nodes in the search space especially when given a complex multiobjective decision function. An alternative approach to multi-objective path planning is to use a multi-objective search algorithm like [21], [41]. However, these algorithms are computationally expensive and in the case of [41], restricted to acyclic graphs; note that graphs derived from grids are cyclic. A similar, direct approach to multi-objective path planning is logic based planning. Two candidate approaches include the Hierarchical Task Network (HTN) [42] and Temporal Action Logic (TAL) [43]. An application of HTN to indoor robot navigation is described in [44]. However, logic based motion planning is generally computationally expensive and the resultant plan is typically non-optimal [1], [42]. C. Findings It can be seen that existing methods can not fully address the multi-objective vehicle motion planning problem. None of the work described previously explicitly consider dynamic obstacles or the effects of different vehicle velocities. However, there are a few points to note. Firstly, it was found that a ‘lazy PRM’ [1] style global planning approach is needed due to the intractability of computing trajectory costs a priori between all parent and child nodes. Secondly, all of the deterministic planning algorithms described above are based on a discrete search space. This is logical as wind data and obstacle maps are often formatted using a grid structure [13]. Thirdly, a vector neighbourhood was identified as a suitable method for overcoming the limited track angle resolution of a grid based search space [15]. It is desirable to have high track angle resolution to ensure path optimality [6], especially given unpredictable wind/current vectors. Fourthly, uncertainty is often mitigated with online replanning. Finally, given the complexity of planning [1], a multi-resolution search space can be used to increase memory efficiency and decrease computation time [6], [11], [16]. III. M ULTI -S TEP A* (MSA*) The planning task is defined as finding a path P through a roadmap S starting at node s0 and terminating at node sG .
3
Successor Cell Trajectory Segment
y3=ny y2
Cell Sequence
y1 y0=0 x0=0 x1=nx Source Cell
(a)
(b)
Fig. 1. Successor operator illustration, (a) a single 2D trajectory segment and the corresponding cell sequence, and (b) an example 2D successor operator showing individual trajectory segments. Note that the trajectory segment terminates in the centre of the successor cell, which is not necessarily adjacent to the source cell.
Each node s ∈ S is located at the centre of a 4D rectangloid cell defined in the world space W (x, y, z, t). Assuming a regular grid sampling of the search space, each node s maps uniquely to a cell in W . Thus s refers simultaneously to both the cell and the node located in the centre of each cell. The global planning approach used in multi-objective planning algorithms in section II is adopted whereby tracks are evaluated online and the initial roadmap is not explicitly represented. Instead, the roadmap is defined implicitly through a successor (or neighbourhood) operator Γ where for a given source (or parent) node s, Γ(s) denotes a set of cell sequences γs0 ∈ Γ(s) which begin at s and terminate at the successor (or child) node s0 ∈ S 0 . Consider the modeling of a vector neighbourhood like [15] where s0 does not necessarily lie adjacent to s. The successor operator is assumed to denote a linear trajectory/track connecting the centre of cell s to the centre of cell s0 (refer Fig. 1). It is assumed that turns (possibly required between tracks) have negligible impact on the overall path in terms of flight time and fuel consumption. For each successor s0 , the trajectory intersects a sequence of cells between s and s0 γs0 = {sj+m−1 = s0 ksj = s, . . . , sj+m−2 }
(1)
where sj , . . . , sj+m−1 are a sequence of m cells and k denotes a conditional dependence, i.e. sj+m−1 ksj , . . . , sj+m−2 is interpreted as cell sj+m−1 via a sequence of cells sj , . . . , sj+m−2 . In the ensuing sections, Γ is derived for a three dimensional world (Γ3D ∈ Γ, section III-A) and then extended to four dimensions (section III-B). This is possible as the search dimensions are orthogonal. A. Multi-Step 3D Successor Operator In Fig. 1b, it is possible to determine the horizontal track angle θ and slope from the endpoints of each trajectory segment. The vertical track angle φ and slope can be similarly determined for a 3D trajectory segment. In aviation, the horizontal and vertical track angles are referred to as the ground track angle and flight path angle respectively. Note that the vertical slope is the climb/descent rate. These angles can then be used to determine the cells that intersect the track using a variant of Bresenham’s pixel algorithm [45]. Note, because the vehicle controller has non-zero trajectory tracking
225
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
error, it is necessary to include cells whose edges/corners touch the trajectory line. This prevents the possibility of the vehicle squeezing through an infinitesimally small gap or “brushing past” an obstacle. Doing so provides an intrinsic tolerance to navigational and controller uncertainty as there is always a safety margin between the trajectory segment and the boundaries of a cell. Using Bresenham’s [45] line drawing concept, determination of a 3D cell sequence is based on the displacement (nx , ny , nz ) of the successor cell from the source in terms of the number of cells in the x, y, and z dimensions respectively. This sequence is invariant to the physical dimensions of each cell but assumes regular cells. The 3D line equations are ny nz y = nx x, z = nx x if |nx | ≥ |ny |, |nx | ≥ |nz | x = nx y, z = nz y if |n | < |n |, |n | ≥ |n | x y y z ny ny (2) ny nx x = nz z, y = nx x if |nx | ≥ |ny |, |nx | < |nz | y = ny z, x = nx y if |n | < |n |, |n | < |n | nz
ny
x
y
y
z
Note that, as is done in [45], line symmetry properties are exploited to avoid slopes greater than one. The cells in the sequence are determined by selecting and then applying the appropriate equation in (2) for each successor (nx , ny , nz ). The equation is evaluated at the midpoints between cells, i.e. 0.5, 1.5, . . . , n − 0.5 cell widths. If the midpoint lies on an edge, cells that share that edge are included in the cell sequence. If the midpoint intersects a corner point, all cells that share that corner point are included. This produces a cell sequence that has Manhattan stepping with non-zero spacing between the trajectory segment and cell boundaries. The horizontal and vertical track angles, θ and φ respectively can be calculated from the displacement as shown in (3) and (4) respectively. nx δx θ = arctan (3) ny δy nz δz (4) ϕ = arctan q 2 2 (nx δx) + (ny δy)
where δx, δy and δz correspond to the x, y and z dimensions of each cell respectively. Note that as Γ is specified a priori, there is no need to optimise the cell sequence generation algorithm. Consider the design of Γ3D . From (3) and (4), it can be seen that arbitrary track angles are possible but this can result in successors that are displaced by a great cell distance (nx , ny , nz ) from s. It is possible to reduce the physical track distance by increasing the cell resolution, but this also increases the computation time. Thus, the design of Γ3D is dependent on the available computation time, desired track length and angle resolution for a specific application. B. Extending the Successor Operator to 4D
Consider the extension of Γ3D to four dimensions where each cell has dimensions (δx, δy, δz, δt); δt specifies a duration of time spent inside a 3D cell. The vector neighbourhood concept extends to the time dimension in that s0 lies at a
4
discretised time level s0tl that is displaced from stl by ntl time levels; ntl corresponds to the track traversal time. The cost of traversal of a particular track (needed in many search algorithms [1]) in vehicle motion planning is dependent on the track velocity. Due to the presence of wind, it is not possible to predefine a set cruise velocity for each successor in the four dimensional successor operator Γ. However, given a 3D successor with displacement (nx , ny , nz ), it is possible to generate multiple 4D successors s0 with displacement (nx , ny , nz , ntl ) where ntl ∈ Ntl . The choice of successor time level displacements Ntl is application specific as it is dependent on track lengths, knowledge of expected wind magnitudes and the minimum and maximum cruise velocity of the vehicle. Note that for each successor, the cell sequence generated using (2) can be extended to 4D by simply calculating the time level displacement nstl for each cell s on the sequence s ns n = ntl nxx if |nx | ≥ |ny |, |nx | ≥ |nz | tl ns (5) nstl = ntl nyy if |nx | < |ny |, |ny | ≥ |nz | s ns ntl = ntl nzz otherwise where ns is the displacement of cell s from the source node. Given ntl , the vehicle cruise velocity can be derived from the track length via the track velocity v~t . q (s0x − sx )2 + (s0y − sy )2 + (s0z − sz )2 |~ vt | = (6) ntl δtl
This track velocity is itself a sum of the cruise and wind velocity vectors |~ vt | cos φ
sin θ cos θ
= |v~c | cos φ
sin α cos α
+
wx wy
(7)
where θ and φ are the horizontal and vertical track angles respectively, v~c is the cruise velocity, α is the vehicle heading angle and (wx , wy ) are the horizontal wind magnitudes. All angles are measured from true north in navigational tasks [46]. Note that (7) is formulated in two dimensions instead of three as the horizontal component of the track and track velocity is far greater than the vertical for both UAVs [46] and AUVs [17]. The vertical component of v~c and wind are treated as constraints instead. By separating the x and y components from (7) and then solving the resultant simultaneous equations, it is possible to get an expression for v~c given |vt | and (wx , wy ). |v~c | =
q 2 |~ vt | − 2 |~ vt | (wy cos θ + wx sin θ) + wx2 + wy2 (8)
Note that we are not interested in the negative root which corresponds to traversal in the opposite direction. The preceding section describes in effect a formulation of a vector neighbourhood (obtained using a successor operator) like the one in [15] but for (x, y, z, t). Unlike [15], the ensuing sections describe a multi-step variant of A* [9] (section III-C) that enables the use of a variable successor operator Γs for
226
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21:
for all s ∈ S do g(s) ← ∞ end for Queue ← ∅ s ← s0 Queue.Insert(s) g(s) ← 0 while s 6= sg do s = Queue.P op() if s = sg then return end if S 0 ← Succ (Γs (s)) for all s0 ∈ S 0 do gˆ(s0 ) = g(s) + c(s0 kγs0 ) ˆ 0 , sg ) fˆ(s0 ) = g(s0 ) + h(s if gˆ(s0 ) < g(s0 ) then g(s0 ) ← gˆ(s0 ) Queue.Insert(s0 ) end if end for end while
mapping (e.g. [40]). Using a weighted sum approach, each decision variable is mapped onto a commensurate scale on the interval (0, 1) using a value function ui (xi ). The final cost is c = w0 u0 (x0 ) + . . . + wn−1 un−1 (xn−1 ) + δc where n is the number of decision criteria and δc is a small positive value to ensure c > 0. A comprehensive evaluation of the decision objectives and decision variables for UAV flight planning is provided in [47]. D. Cost Optimality of MSA*
Fig. 2. MSA* Pseudocode. Note that at line 12, Succ (Γs (s)) extracts the set of successor nodes s0 ∈ S 0 where s0 is the last cell in each cell sequence γs0 ∈ Γs (s).
each node s. This dynamic operator enables the implementation of multi-resolution search and also enables one to impose a structure on the search space. These are further discussed in section IV. C. Search Algorithm The MSA* algorithm is listed in 2. Note that s0 and sg refer to the start and goal nodes respectively. Like A*, nodes are placed on a priority queue sorted according to the evaluation function f which is itself the sum of the cost to come g and ˆ In Fig. 2, Queue.Insert refers to the estimated cost to go h. addition of a node s0 to the queue such that f (s∗ ) ≤ f (s) ∀s ∈ Queue where s∗ is the topmost element in Queue. Queue.P op is the removal of this topmost element. The key distinction between MSA* and A* lies in the cost function c. A* computes the cost as a function of the cells s and s0 , whereas the MSA* cost is a function of multiple cells as defined in the cell sequence (equation (1)). g(s0 ) = g(s) + c(s0 = sj+m−1 ks = sj , . . . , sj+m−2 )
5
(9)
The cost c is calculated using a two step process. First, the decision variables xi (e.g. fuel, risk) are uniquely mapped or calculated from the cell sequence such that xi = ρi (s, . . . , s0 ) where ρi is the mapping function. For example, the total risk is the sum of the risk density for each cell on the cell sequence. Note that constraints can be imposed by setting c = ∞ if a particular decision variable exceeds a specified limit (e.g. maximum risk). Otherwise, a multi-criteria decision making (MCDM) cost function is used to transform the decision variables into a single cost term c where c is non-zero and monotonic (i.e. c > 0). This second step could be a weighted sum aggregation (like that used in [36]–[39]) or a fuzzy
It can be shown that MSA* will find the least cost path given a predefined set of successor operators Γs for each node s ∈ S and MCDM cost function c. As MSA* is derived from A* [9], cost optimality can be shown in a similar manner as well. Lemma 1: Consider any globally optimal path P ∗ = (s0 , . . . , sn ). It can be shown that P ∗ is itself composed of optimal paths. Proof: The optimal path for a given node sik is a path P ∗ = (s0 , . . . , sik ), such that for all possible paths P ∈ Π, g(sik kP ∗ ) < g(sik kP ). Recall that any path is made up of an integer number of trajectory segments K and each segment is represented by a cell sequence of length mj for segment j. Thus, the index to a node (or cell) in P ∗ at the k th trajectory segment is k−1 X ik = −k + mj (10) j=0
For the case where k = K, the lemma is trivially true by definition of Pi∗K as iK = n, Pn∗ = (s0 , . . . , sn ). Consider the case for k = K − 1 trajectory segments, whose cell sequence PiK−1 = (s0 , s1 , . . . , siK−1 ) is a subset of the optimal path P ∗ . If PiK−1 is not a least cost path, then there exists another path Pi0K−1 = s0 , s01 , . . . , s0iK−2 , siK−1 such that g(Pi0K−1 ) < g(PiK−1 ). But, as the K − 1th cell sequence siK−1 , . . . , siK is unchanged, then given (9), the cost term c is unchanged. This implies that there exists a path Pi0K = (s0 , s01 , . . . , s0iK−2 , siK−1 , . . . , siK ) such that g(PiK ) < g(Pi∗K ), contradicting the definition of Pi∗K . Therefore, PiK−1 must also be an optimal path. By mathematical induction, any optimal path P ∗ must itself be composed of optimal paths. Theorem 1: If the heuristic is admissible [9], then MSA* will find the optimal path if one exists. An admissible heuristic ˆ is an estimate of the cost to go that is always less than the h ˆ j , sg ) ≤ h(sj , sg ). actual cost to go, h(s Proof: Consider an optimal path P ∗ = (s0 , . . . , sg ) which contains K trajectory segments. On an optimal path, g(sj ) = g ∗ (sj ) for all j = (0, 1, . . . , iK ). From line 15, ˆ j , sg ) ≤ h(sj , sg ), therefore, f (sj ) ≤ f ∗ (sj ). because h(s In the trivial case where K = 0 (i.e. s0 = sg ), MSA* discovers the solution in one iteration. During initialisation (lines 1-7 in Fig. 2), s0 is placed on the queue with cost g(s0 ) = g ∗ (s0 ) = 0. Upon expansion of s0 , MSA* terminates. Consider the case where K = 1, i.e. sg ∈ S00 where S00 = Γs0 (s0 ). Let S ∗ denote the set of nodes on the queue that lie on an optimal path. After one iteration (i.e. expansion of s0 ), at least one of the successors s00 is a member of S ∗ . Consider the contrary where none of the nodes s00 ∈ S00 lie on an optimal
227
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
path and/or are not on the queue. There are two possibilities, one is that none of the successors are reachable (in which case no path exists) or at least one of them lies on a least cost path. Note that if a path exists, then an optimal path also exists. For this latter case, given a node s00 , an optimal path (s0 , . . . , s00 ) must exist that does not contain any nodes in S00 because by Lemma 1, an optimal path (s0 , . . . , sn ) comprises optimal paths (s0 , . . . , sn−1 ), (s0 , . . . , sn−2 ), . . . and none of the nodes in S00 lie on an optimal path (by assumption). This is not possible, hence, at least one node s00 must lie on an optimal path in which case, by line 16, s00 would be added to the queue at the expansion of s0 . Therefore, where a path exists, S ∗ 6= ∅ and sg ∈ S0∗ for the scenario K = 1. The preceding argument can be extended to show that up until algorithm termination, S ∗ 6= ∅. Let Sk0 denote the set of 0 0 nodes generated by Γsik−1 ∈Sk−1 (Γsik−2 ∈Sk−2 (. . . Γs0 (s0 ))) i.e. all nodes that can be reached in k trajectory segments. From above, all nodes s : s ∈ S ∗ , s ∈ Sk (k = 1) are placed on the queue after the first iteration. If optimal paths of length k + 1 exist, expansion of s : s ∈ S ∗ , s ∈ Sk must yield nodes s0 : s0 ∈ S ∗ , s0 ∈ Sk+1 . Otherwise, as before, there would exist nodes s0 ∈ / Sk that result in optimal paths of length k + 1 trajectory segments which is not possible. Hence, S ∗ 6= ∅. Because Γs is a finite set for all s ∈ S and because trajectory segments incur a non-zero and non-negative cost c, there are only a finite number of nodes such that f (s) ≤ f ∗ (sg ) = f ∗ (s0 ). Therefore, as S ∗ 6= ∅, the nodes on the optimal path P ∗ = (s0 , si1 , . . . , sik = sg ) are expanded (in sequence) in a finite number of iterations, terminating with the expansion of sg . It is not possible to terminate without finding the optimal path if one exists. Consider the scenario where MSA* terminates such that f (sg ) = g(sg ) > f ∗ (s0 ). But, by the analysis above, there exists a node s ∈ S ∗ just before termination such that f (s) ≤ f ∗ (s0 ) < f (sg ). Hence, s would be expanded instead of sg , contradicting the assumption that MSA* would have terminated. Therefore, MSA* will find an optimal path (s0 , . . . , sg ) in finite time where such a path exists.
IV. M ULTI -R ESOLUTION L ATTICE S TRUCTURE A class of variable successor operators is presented that can be used to implement a lattice based multi-resolution search space for the purposes of reducing computation time. The use of variable successor operators Γs is made possible by the MSA* search algorithm. As before, the three dimensional lattice structure is presented first followed by a conceptual extension to 4D. It is shown that Lattice MSA* reduces the size of the search space without sacrificing track angle resolution or soundness [48]. A. 3D Lattice A 3D illustration of this structure is presented in Fig. 3 with a 2D cut-away (x−y) view showing the source-successor trajectory segments in Fig. 4. The lattice comprises a series of planes parallel to the x − y, y − z and x − z Cartesian planes
6
x y
z
z y x Fig. 3.
General lattice structure, (x, y, z) dimensions shown
(a)
(b)
Fig. 4. Top (x − y) view of lattice showing trajectory segments for (a) lattice position (0, 0), (b) (0, 2)
at regular intervals of Λx , Λy and Λz (defined in terms of the number of cells) in the x, y and z planes respectively. The lattice design methodology is a three step process. Firstly, a series of base 3D successor operators Γ0 are chosen, one for each search space resolution. The choice of Γ0 takes into account the required track angle resolution and track length, both of which are related to the sampling density. Γ0 must be such that, in any plane x − y, x − z or y − z, all successors lie on the border of a rectangle centred at the source node (as in Fig. 4a). Using Γ0 , it is then possible to define the spacing between planes in the lattice, Λx , Λy , Λz using Λx = sup γ ∈ Γ0 : γnx ≥ γny , γnz γn x Λy = sup γ ∈ Γ0 : γny ≥ γnx , γnz (11) γny Λz = sup γ ∈ Γ0 : γnz ≥ γnx , γny γnz
where sup denotes the supremum operator (least upper bound). Note that if the start or goal nodes do not lie on the lattice, it is a simple matter to connect those nodes to one that is on the lattice using a local search technique (refer [1]). Using this lattice structure, it is then possible to define individual Γs operators for each node on the lattice. This is shown for a 2D lattice for the sake of clarity in Fig. 4. Due to the regularity of the lattice, nodes located at equivalent positions on the lattice share the same successor operator Γp~ for position p~. Two positions are equivalent if and only if they are separated by integer multiples of Λx , Λy and Λz cells. A
228
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
7
x y
z y x Fig. 5.
Constrained vertical angle lattice structure
lattice position p~ can be uniquely defined based on the modulus p~ = (mod(x, Λx ), mod(y, Λy ), mod(z, Λz ))
(12)
Referring to Fig. 3, the total number of unique lattice positions np is np = Λx Λy + Λx (Λz − 1) + (Λy − 1)(Λz − 1)
Fig. 6. Multi-resolution lattice with Λx = Λy = 3 on the left and Λx = Λy = 6 on the right. Selected trajectory segments are shown for the sake of clarity
(13)
Consider the case where the source node is at lattice position (0, 0, 0), i.e. at the intersection of lattice planes (Fig. 4a). The trajectory segments are chosen to terminate at successors that lie on the border of a rectangle centred on the source node with dimensions of (2Λx , 2Λy , 2Λz ) cells. Therefore, Γp~=(0,0,0) = Γ0 . For cells located at different positions on the lattice (refer Fig. 4b), the successors are chosen to maximise the number of identical successors to Γ(0,0,0) . Additionally, where possible, the successors are chosen to terminate at lattice position (0, 0, 0). This ensures that the same track angle can be maintained over consecutive trajectory segments to avert unnecessary turns. Consider the case where a maximum limit on the vertical track angle is imposed. For fixed wing UAVs, this angle is small as the horizontal cruise velocities (e.g. 36-57m/s for the RQ-7A Shadow UAV) are much greater than the maximum climb velocity (5m/s) [49]. Such a vehicle performance limit can be modeled by applying a constraint on all successors where if nz 6= 0, then either nx = Λx or ny = Λy . Given such a successor operator, Λz = −∞ (from (11)) which results in a simplified lattice structure as shown in Fig. 5. The maximum vertical track angle is (assuming δx = δy = δ) nz δ z φmax = arctan (14) δ max(nx , ny ) Additionally, the total number of unique lattice positions is greatly reduced: np = Λx + Λy − 1 (15)
sampling resolution is required at lower altitudes but a coarse sampling resolution can be used for high altitude en-route airspace. It is easy to implement multi-resolution search by using multiple base successor operators Γi0 . Consider the division of the search space into a series of N rectangular prism shaped regions each of which has a lattice resolution of (Λix , Λiy , Λiz ), i = 1, 2, . . . , N and a base successor operator Γi0 . Note that N is typically a small number as it is necessary to check the bounds of each region at every iteration. Each region must have dimensions that are a multiple of Λix , Λiy , Λiz . This ensures that all trajectories must terminate on and originate from a lattice plane separating the two regions (refer Fig. 6). Furthermore, assume that for any two adjacent regions i and j, Λix , Λiy , Λiz is an integer multiple of Λjx , Λjy , Λjz . This way, successors in Γi0 are displaced at integer multiples of those in Γj0 , which ensures that all horizontal track angles (calculated using (3)) in region j also exist in region i (see Fig. 6). Unfortunately, this is not the case for the vertical track angle even though angles in the x − z and y − z planes are replicated in region i. To avoid a large increase in the number of successors, it is also possible to filter out successors in Γi0 that are on track angles not represented in Γj0 . The aforementioned multi-resolution lattice provides a means for fine and coarse sampling corresponding to smaller and larger values of Λ respectively. There is no reduction in track angle resolution for coarsely sampled regions as larger values of Λ enable higher track angle resolutions. Additionally, each track is evaluated at the cell resolution using cell sequences defined in section III-A. This guarantees path soundness [48, Ch. 25.6] by avoiding the problem of mixed cells (which contains free space and obstacles [1]). C. 4D Multi-Resolution Lattice
B. 3D Multi-Resolution Lattice The purpose of multi-resolution sampling is to reduce the total number of nodes and thus reduce computation time. In many applications it is possible to divide the search space into regions of fine sampling resolution and regions of coarse sampling resolution. In UAV flight planning for example, fine
Extension of the previous 3D lattice to four dimensions involves selection of a suitable set of values ntl ∈ Ntl for each 3D successor in Γp~ using the methodology described in section III-B. The full 4D multi-resolution lattice structure is implicitly defined through the variable successor operators where, at each iteration of MMSA* search, Γp~ is selected based on (12) and the boundaries of each Γi0 region. This
229
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
approach enables consistent track angle resolution across fine and coarse resolution regions without sacrificing soundness. In addition, the lattice structure reduces memory usage. The underlying cell grid of MMSA* contains Nx Ny Nz Ntl nodes where Nx , Ny , Nz and Ntl are the total number of sample points in the x, y, z and t dimensions respectively. The total memory requirement (i.e. number of nodes) N for a lattice can easily be derived by counting the number of nodes in the x, y and z planes (refer Fig. 3) and then subtracting overlap regions. N = Ntl (Ny Nz αx + Nx Nz αy + Nx Ny αz − αx αy Nz
−αx αz Ny − αy αz Nx + αx αy αz ) (16) k j Nx −1 where αx = Λx + 1 (and similarly for y and z) and all division operations are integer divisions. Note that for the case where vertical track angle is constrained as in (14), the αz term in (16) goes to zero. V. E XPERIMENTAL A NALYSIS This section discusses some of the practical aspects of implementing the proposed algorithm including evaluation of MSA* against an existing vector neighbour based search algorithm (like [15]) in simulation. A. UAV Flight Planning Application The UAV flight planning problem was chosen to provide a practical context for evaluation of MSA*. This is an important application as onboard mission flight planning has been shown to be a key enabler in the operation of UAVs in the National Airspace System (NAS) [40]. The mission being undertaken is the delivery of a medical package to a remote location using a small UAV. This specific application is ideal for evaluating a 4D search algorithm due to the presence of multiple decision criteria, dynamic elements in the operating environment and the significant effect of wind on a small UAV. It is sometimes necessary to prepare a flight plan for regulatory approval prior to flight and, due to a dynamic and uncertain operating environment, may also be necessary to replan during flight. A number of safety, regulatory and mission efficiency objectives need to be considered in the design of a flight plan. The chosen mission is assumed to be operated under Visual Flight Rules (VFR) using Australian Civil Aviation Regulations (CAR) [50]. A brief summary of the relevant decision variables and decision criteria is provided here; more details can be found in [47]. Mission safety is given utmost consideration in the planning process. For simulation purposes, the aircraft separation requirement for en-route airspace is adopted. This stipulates that any two aircraft must be separated by 5NM (equivalent to 9260m) in the horizontal plane and 1000ft (304.8m) in the vertical [40]. The separation zone is cylindrical in shape, but because there is uncertainty in the projected position of other aircraft, the separation zone is actually a moving probabilistic zone [24]. This is similarly the case for storm cells, which must be avoided as per VFR requirements. Additionally, it is desirable (though not mandatory) to avoid overflying populous
8
regions to minimise the risk presented to people and property on the ground. For the purposes of simulation, this risk value is approximated with a Normalised Population Density (NPD) value. The flight plan must also conform with the rules of the air, many of which are safety oriented. In addition to the separation requirements stated earlier, the aircraft must also conform with the cruising levels rule, low flying restrictions and segregated airspace. For aircraft flying on headings from 0◦ to 179◦ , the permissible flight levels are at odd multiples of 1000ft plus 500ft Above Mean Sea Level (AMSL) (e.g. 1500ft, 3500ft, 5500ft AMSL). For headings between 180◦ and 359◦ , the cruise levels are at even multiples of 1000ft AMSL plus 500ft (e.g. 2500, 4500, 6500ft AMSL). The cruising levels rule is intended to minimise the risk of a head-on collision, and is mandatory above 5000ft. When flying at low altitudes, civil aviation regulations stipulate that the aircraft must maintain an altitude of 500ft Above Ground Level (AGL) over non-populous areas and 1000ft above population centres. Additionally, the aircraft must avoid certain classes of segregated airspace (e.g. controlled airspace) in the en-route portion of the flight. These can be modeled simply as no fly zones. The flight plan also needs to optimise for the objectives of the mission itself (i.e. the delivery task). These objectives include the delivery time (i.e. the time of arrival at the goal node) and fuel consumption. With a 4D search, it is possible to not only find a path that minimises the delivery time, but also to designate a specific delivery time or acceptable time window (like with [19]). The flight time, along with the cruise velocity, altitude and rate of climb are parameters used to optimise fuel consumption. These decision variables, in combination with the dynamic constraints of the aircraft are used to calculate the cost term c in (9). For more details, refer [47]. Note that as the dynamic constraints are closely linked with the chosen successor trajectory segments in Γ, these are discussed together in the following subsection. B. Experimental Setup The primary purpose of this experimental analysis is to compare the computational efficiency and solution path of the proposed algorithm with that of existing algorithms. In these comparisons, each test algorithm uses a different (set of) successor operator(s) but the same cost function c (a weighted sum of utility values) and heuristic function h. 1) Test Algorithms: Two different variants of MSA* are compared against a benchmark algorithm, Vector A* on 1000 randomly generated planning scenarios. Vector A* is a direct extension of A* using a vector neighbourhood (like that used in [15]). The successor operator is chosen to reflect the base successor operator used in the other test algorithms and is shown in Fig. 7. Using this successor operator, there are 360 successors for each source node. Vector A* is in effect a special case of MSA* where Γs is constant throughout the entire search space. Due to the popularity of A* and related algorithms in robotics [51], Vector A* serves as an ideal
230
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
9
map, no fly zones, other aircraft, storm cells, wind map and population density map (as a simple model of risk presented to people and property on the ground). For each world, a number of start and goal pairs were randomly chosen. The mission area for each world was arbitrarily chosen to be 50NM×50NM×15000ft×90min with a cell resolution of 1NM×1NM×1000ft×1min (1NM=1852m, 1ft=0.3048m). Note that the maximum distance of the search area approximately matches the maximum operating range of the RQ-7A Shadow UAV [49]. An artificial terrain map is randomly generated through summation of bivariate Gaussian functions with randomly chosen parameters (A, b, c, σ, n). Population density is also generated using this equation. Fig. 7. Chosen Γ0 successor operator, showing source cell (centre) and successor cells joined by trajectory segments. Λx = 3, Λy = 3, Λz = −∞, Nt = {2, 3, 4}min.
benchmark of calculation time and path cost for a deterministic 4D planning algorithm with selectable track angle resolution. Two variants of MSA*, MSA*1 and MSA*2 are tested against the benchmark. MSA*1 uses a single, fine resolution lattice based on Γ0 as shown in Fig. 7. MSA*2, on the other hand, uses a multi-resolution lattice where N = 2. A fine resolution lattice (based on Γ0 in Fig. 7) is used for altitudes less than 7000ft, and a coarse resolution lattice (Λx = 6, Λy = 6, Λz = −∞, Nt = {4, 6, 8}min) for altitudes above 7000ft. All experiments were performed on a 3.3GHz Intel Core 2 Duo QX6850 CPU with 4GB of physical RAM running 32-bit Windows XP. 2) Dynamic Constraints: The dynamic constraints of the aircraft were considered in the selection of Γ0 in Fig. 7. Oftentimes, these dynamic constraints are modeled with a minimum turn radius [1]. Assuming a maximum airspeed of 65m/s, a bank angle of 30◦ and a force of one g (9.8m/s2 ), the worst case turn radius is approximately 747m (refer [46, (3.9.10)]). As the turn radius is less than half the cell size, it is possible to execute a 180◦ turn within the bounds of a single cell. However, it is still desirable to minimise the turn angle as it is difficult for the flight controller to execute such a turn with accuracy under strong wind conditions. Turn angles and the impact on the track are further discussed in section V-C. For UAV operations, it is also necessary to incorporate climb/descent rate constraints and a maximum airspeed constraint (a constant value). The maximum climb rate, however, decreases with altitude and is zero at the aircraft ceiling [46]. At sea level, the maximum climb rate for a small UAV is limited to approximately 5m/s [49]. This matches the maximum climb rate achieved using Γ0 under no wind conditions: max (nz δz ) nz
min (nt δt )
nt ∈Nt
= 5.08m/s ≈ 5m/s
(17)
3) Simulation World: The simulation worlds were generated randomly to enable a Monte-Carlo evaluation of the test algorithms. Each simulation world comprises a terrain
z (x, y) =
n X
Ai e
−
(x−bi )2 +(y−ci )2 σ2 i
(18)
i=0
Airspace can be modeled with prisms and stacked cylinders representing the controlled airspace above aerodromes (refer [47, Fig. 3]). For prism shaped no fly zones, the parameters used to describe each region are the coordinates of each vertex. For controlled airspace, the only parameter required is the centre position of the cylinder stack as the radius and height of each cylinder in the stack is defined in aviation regulations. These parameters are randomly generated. The probabilistic model described in [47] is used to model other aircraft and storm cells. Parameters for this model, namely the position, velocity, standard deviations and volumetric description (radius and height) are randomly generated. For aircraft, the radius and height is defined by separation standards and the velocity is assumed to range between 50 knots and a speed limit (for flight below 10000ft) of 250 knots (128.6m/s) [50]. For storm cells, an average radius of 25km and height of 15km is assumed [52, Fig. 5]. The rate of movement of a storm cell is assumed to be between 5 to 20m/s for altitudes between 0 and 15000ft [52, Fig. 1]. Finally, a simple algorithm was used to generate wind maps that mimic real world winds. Firstly, a number of seed nodes are randomly generated at different positions (x, y) that contain a position ~vi , a direction φi and a vector of wind magnitudes m ~ z for each altitude level z. For each node s = (x, y, z) in the world space, a weighting vector w ~ is calculated wi = a0 d~i + a1 6 d~i − φi
(19)
where d~i = ~s − v~i and a0 , a1 are weights. The largest element in w ~ is then scaled by a∗ ; this gives the “winning seed” more weighting. For a given node s, the wind magnitude is fm (s) = w ~ ·m ~ z . The wind magnitude for each altitude level is randomly chosen based on average wind speeds from Bureau of Meteorology radiosonde measurements [53]. The direction ~ + σz where fd is calculated in a similar manner, fd (s) = w ~ ·φ σz is a small, random perturbation added to simulate wind shear.
231
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
10
Fig. 9. Example planning scenario showing a close up of no fly zones and other aircraft at t = 320s. Compare with Fig. 8.
C. Results A Monte-Carlo simulation of the three tests algorithms was performed on 1000 randomly generated planning scenarios. The results of these simulations are presented below and evaluated with respect to computation time and path cost. In addition, the algorithms were also evaluated on three special case tests scenarios. These were constructed to determine the effect of local minima and to test the adaptability of the planner to situations where the vertical wind velocity exceeds aircraft performance. An illustration of a typical multi-objective planning scenario is provided in Fig. 8 and Fig. 9 (showing other aircraft and no fly zones), Fig. 10 (risk map) and Fig. 11 (wind map). The solution path using Vector A*, MSA*1 and MSA*2 are also shown on each of these figures. In Fig. 8, all three planners select a path that avoids an aircraft on a converging course by descending and heading in an easterly direction (refer Fig. 9). Once a risk of collision is averted, the paths continue in a shortest path fashion towards the goal (marked by a red cross). There are deviations only to avoid terrain (where the paths hop over a mountain in Fig. 8) and route around high risk (population density) areas (Fig. 10). Note that, as shown in section III-D, each algorithm finds a path that satisfies all given constraints whilst minimising the overall path cost (which is a multi-objective cost function). 1) Computation time: The mean and standard deviation for the computation time (µt and σt respectively) and the loop count (µn and σn respectively) is presented in Table I for each test algorithm. From the results, it can be deduced that a lattice based successor operator can significantly reduce total computation time and that further time savings can be achieved with a multi-resolution lattice. This can be confirmed by examining a normalised histogram of the computation time ratio as shown in Fig. 12. The mean µr and standard deviation σr of the computation time ratio between algorithms is listed in Table II. For the test resolution level, Vector A*, MSA*1 and MSA*2 are all suitable for onboard replanning as the computation time is well within the minimum track traversal time of 2min. as specified in Nt . 2) Total path cost: The mean µc and standard deviation σc for the ratio of the path cost between MSA*1, MSA*2 and Vector A* is presented in Table III. A cumulative histogram of the ratios is illustrated in Fig. 13. As the successor operator
Fig. 10. Example planning scenario risk (represented by normalised population density) map.
Fig. 11. Example planning scenario wind map xy slice plane at z = 3500ft.
TABLE I C OMPUTATION T IME AND L OOP C OUNT
MSA*2 MSA*1 Vector A*
µt
σt
µn
σn
4.46s 9.23s 19.25s
2.36s 4.93s 10.41s
65501 161571 289015
34250 88386 160575
TABLE II R ATIO OF C OMPUTATION T IME
MSA*2 to Vector A* MSA*1 to Vector A* MSA*2 to MSA*1
µr
σr
0.239 0.484 0.496
0.042 0.050 0.084
232
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
11
Fig. 8. Example planning scenario showing no fly zones and other aircraft at t = 120s. Note for aircraft and weather, the inner cylinder represents the separation zone/storm cell extents (around the expected position) and the outer cylinder is the 2σ uncertainty boundary (which grows with time). Note also that a red X marks the goal position. TABLE III PATH C OST R ATIO
Norm. Freq. Plot of Comp. Time Ratio 16
MSA*1 to Vect A* MSA*2 to Vect A* MSA*2 to MSA*1
Normalised Frequency
14
µc
σc
0.9891 1.0334
0.0307 0.0376
12
MSA*1 to Vector A* MSA*2 to Vector A*
10 8 6
TABLE IV T URN A NGLE S TATISTICS
4 2 0 0
0.5
Ratio
1
1.5
Vector A* MSA*1 MSA*2
Fig. 12. Normalised frequency histogram of computation time ratio between Vector A*, MSA*1 and MSA*2.
µdθ
σdθ
11.72◦ 12.71◦ 16.42◦
19.01◦ 22.42◦ 21.93◦
Cumulative Frequency Plot of Cost Ratio Normalised Cumulative Frequency
1
0.8
0.6
0.4
0.2
0 0
Fig. 13.
MSA*1 to Vector A* MSA*2 to Vector A*
0.5
1 Cost Ratio
1.5
2
Cumulative histogram of relative path cost
Γp~ in MSA*1 is largely similar to Γ in Vector A*, it is unsurprising to find that both return paths of approximately equivalent cost. Of particular note however, is the fact that, on average, MSA*2 finds paths that are only 3.3% more costly than Vector A*. Therefore, it can be seen that MSA* finds paths of equivalent cost but with significantly less computation time. It is observed that each of the three test algorithms return a solution path that tends to follow the profile of a straight line
(shortest path). This is attributable to the minimisation process of A* and the fact that all trajectory segments have a nonzero and non-negative cost value. As a result, the turn angles are typically small. The mean turn angle µdθ and standard deviation in turn angles σdθ is listed in Table IV. The turn angles are small even without explicit optimisation of turn angles. Of the simulation scenarios where the maximum turn angle exceeds 90◦ , 90.6%, 96.1% and 81.3% of these cases for Vector A*, MSA*1 and MSA*2 respectively only contain a single turn of greater than 90◦ . This turn occurs on the transition to the last trajectory segment in the path like that shown in Fig. 14. A combined turn and altitude change occurs to ensure conformance with the cruising levels rule at the goal altitude. D. Special Cases It is widely acknowledged that A*, and best-first search algorithms in general require significantly more computation time in the presence of local minima [1]. This was tested for the single and double bug trap case as shown in Fig. 15 and Fig. 16 respectively. The computation time is recorded
233
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
Fig. 14. Planning scenario where there is a turn angle of greater than 90◦ at the last trajectory segment. Note that the allowable track angles at 3500 and 5500 ft are 0◦ to 179◦ , at 4500ft are 180◦ to 359◦ . Because the goal node is on a general North-Easterly heading from the start node, a turn of greater than 90◦ occurs in ensuing the track angle of the final track lies in the acceptable range.
Fig. 15.
Single bug trap case.
in Table V. It can be seen that even though the absolute computation time is approximately double to 2.5 times the mean obtained in the previous Monte Carlo simulations, the relative computation time between Vector A*, MSA*1 and MSA*2 remains approximately the same as before. A simulation scenario which mimics the presence of strong up/downdrafts in mountainous regions (where the vertical wind velocity can exceed the vehicle’s climb rate) is depicted in Fig. 17. Even though a variety of wind conditions were simulated in the previous Monte Carlo experiment, this experiment specifically studies the effect of wind by setting other decision variables (e.g. no fly zones, other aircraft, storm cells, risk) to TABLE V C OMPUTATION T IME
MSA*2 MSA*1 Vector A*
Single Bug Trap Time Loop Count
Double Bug Trap Time Loop Count
9.59s 23.45s 51.14s
9.17s 22.61s 50.65s
201197 515657 922544
194981 503871 926276
Fig. 16.
Double bug trap case.
Fig. 17.
Mountain wind simulation. The solution is found in 5.14s.
12
zero. As shown in Fig. 17, only MSA*2 successfully finds a traversable path that satisfies aircraft climb constraints. The chosen path climbs in a switchback pattern before ‘hopping over’ the mountain. Recall that as the aircraft’s maximum climb rate decreases with altitude, it is necessary to climb to 13500ft to accommodate loss of altitude in the downdrafts region. The reason that MSA*2 successfully finds a path while Vector A* and MSA*1 fail is because of the higher climb and descent rate required by the fine resolution successor operator. The same vertical distance is covered in a shorter time using the fine resolution operator (e.g. 1000ft in 2, 3 or 4min.) compared to the coarse resolution operator (e.g. 1000ft in 4, 6 or 8min.).
VI. D ISCUSSION This section reviews the proposed algorithm with respect to existing work in light of the practical simulation results and theoretical findings.
234
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
A. Online Replanning
B. Lattice Structure The computational efficiency of MSA* compared to Vector A* can be attributed to a smaller, lattice structure based search space compared to a full 4D grid. With the exception of the successor operator and cell sequence based sampling, Vector A* is virtually identical to MSA*. Using (16), a plot
6
3.5 x 10 3 Number of Nodes
The primary objective of the simulation experiments presented in section V is to determine whether the algorithm is fast enough for online replanning. Online replanning is needed to mitigate the uncertainty and unpredictably of an outdoor operating environment. An indicator of the available planning time can be derived from the minimum track traversal time as the motion plan is made up of discrete trajectory segments (i.e. tracks). For the successor operator selected in Fig. 7, the mean planning time for MSA*2, MSA*1 and Vector A* (of 4.46s, 9.23s and 19.25s respectively) was found to be much smaller than the minimum track traversal time (2min). All three tests algorithms were also shown to be capable of finding a path within the time constraints of online replanning for environments containing deep local minima and narrow escape passages (as per Fig. 15 and Fig. 16). In these experiments, it was found that the proposed algorithm offered an approximately two-fold reduction in computation time. Further reductions were obtained by using a multi-resolution lattice. This increased speed is significant when planning over a larger search space or on a more difficult search space. For a 100NM×100NM×15000ft×180min. search space, a similar Monte Carlo experiment found that MSA*2 is still able to meet the requirements for online replanning with a mean planning time of 48.25s and standard deviation of 20.62s (which is less than the minimum track traversal time of 2min.). However, Vector A* does not as it takes approximately four times the computation time. Similarly, when presented with local minima, which are known to be difficult to solve for best first search algorithms like A*, the computational efficiency of MSA* over Vector A* is significant in meeting online planning constraints [1]. The previous discussion of algorithm computation time assumes that in each case, we plan from scratch. This approach of always discarding previous planning information was adopted because in applications such as UAV package delivery, online changes can occur anywhere in the search map and affect large swathes of the search space. If a large number of nodes are changed and/or changes occur close to the goal node, replanning algorithms like D* and D* Lite are less efficient than one that plans from scratch [51]. The presence of fast moving aircraft and storm cells for example can affect large areas of the search space that are not necessarily localised around the vehicle’s current position. Hence, it is more efficient to plan from scratch each time. Due to the time critical nature of online replanning, it is preferable to use a fast and near-optimal planner rather than an optimal planner which may be too slow. Under these conditions, MSA*2 is the best candidate for online replanning out of the three test algorithms.
13
2.5 Λz =1 2
Λz =2 Λ =3 z
1.5
Λz =4 Λ =5
1
z
Λ = −∞ z
0.5 1
Grid 2
3
4
5 6 Λx = Λy
7
8
9
10
Fig. 18. Number of nodes in the search space for different values of Λ. Note that Λz = −∞ corresponds to the constrained vertical track angle case described in section IV-A
of the number of nodes for a search space of dimensions 50×50×15×90 given different values of Λ assuming Λx = Λy is shown in Fig. 18. Note that the memory required in a full grid corresponds to the case where Λx = 1, or Λy = 1, or Λz = 1. From Fig. 18, it is evident that larger values of Λ produce a lattice structure with fewer sample nodes. However, the corresponding successor operator Γs has potentially more successors per node with greater track angle resolution and greater track length. As a result, fewer search iterations are required but each iteration incurs more computation time. For example, it is possible to evaluate 17400 nodes per second (on average) using MSA*1 whereas only 14750 nodes per second are possible (on average) using MSA*2. It can be seen that the variable successor operator Γs is a crucial, application-specific design parameter that influences the path cost, the traversability of the path and computation time. For the demonstration UAV application, it has been shown that the selected fine resolution and especially the multi-resolution successor operators are effective at delivering a solution of comparable cost with significant savings in computation time. The use of a coarse resolution successor operator Γs for high altitudes in MSA*2 is especially suited to UAV planning because of the scarcity of obstacles and reduced climb rate at high altitudes (refer section V-D). Note that even though the proposed lattice sampling structure enables significantly reduced computation time, it is impossible to escape the “curse of dimensionality” [4]. Regardless of the sampling scheme employed (probabilistic or deterministic), if a constant (or minimal) dispersion is required, then there is always exponential growth in memory and time with dimensionality [27]. Hence the selection of a 4D search space for vehicle motion planning instead of a higher dimensional search space that incorporates the degrees of freedom of the vehicle and decision variables (e.g. heading angle, fuel, risk probability). The lattice structure presented here is similar to the framed quad/octree presented by [11]. A key improvement in terms of 3D sampling is that the proposed lattice comprises sample planes that are one cell wide (Fig. 4) whereas that used in framed octree is two cells wide ( [11, Fig. 4]). This results in
235
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
14
fewer nodes in the search space and hence reduced memory and computation time requirements. Additionally, the track angle in a framed octree is constrained to intervals of 45◦ when transitioning between quadtree nodes. Finally, the proposed method guarantees path soundness by sampling each trajectory segment at the same high resolution cell size thus avoiding the problem of mixed cells when using cell decomposition based methods (such as quad/octree based methods like [11]). C. Path Cost With an admissible heuristic, it has been shown that all three test algorithms find a traversable path that is of least cost with respect to the successor operator being used. Through Monte Carlo simulations and special case demonstrations, it was further shown that careful selection of Γs can produce paths of equivalent cost to that found with a fine resolution successor operator. The cost function c needs to be additively consistent because of a variable successor operator Γs and variable track lengths for each successor in Γs . An additively consistent cost function c is defined as follows. Given any two nodes s, s0 and 0 intermediary nodes s00i such that Γp~ (s) = {s, . . . , s } = Γp~0 (s000 = s), . . . , Γp~k−1 (s00k−1 ) , c(s0 ks, . . . , s0 ) = c(s001 ks000 = s, . . . , s001 ) + . . . + c(s00k ks00k−1 , . . . , s00k = s0 )
(20)
Without additive consistency, the search algorithm is unfairly biased to select longer (or shorter) trajectory segments due to the equation for calculating g (9). This effect is especially evident in multi-resolution planning. For the previous Monte Carlo simulation, MSA*2 finds paths with an average cost ratio compared to Vector A* of 1.033 with an additively consistent cost function. Without additive consistency, the cost ratio rises to 1.139 for the case in (20) where LHS < RHS. Inspection of simulation results confirmed the presence of bias as the planner tends to select paths containing longer trajectory segments in the coarse resolution region. Without additive consistency, the ratio of coarse to fine resolution trajectory segments over all simulation scenarios is 0.854, but is only 0.258 with additive consistency. Therefore, additive consistency is paramount to any practical implementation of a planning algorithm where there is significant variation in length between successor trajectory segments. Consider the multi-objective cost function used for the previous UAV flight planning application. The decision variables used to evaluate the cost can be classified as either cumulative or non-cumulative. Cumulative variables are those like fuel and risk where the total value for a trajectory segment is found by integrating the fuel rate over the length of the track or integrating the risk density over the cell sequence. Thus, the total value of the decision variable is always a summation of the value at each cell or increment in the trajectory segment. If the cost function is also a linear function of cumulative variables, it can be seen that (20) is always satisfied. In UAV flight planning, the cumulative variables are fuel, aircraft separation risk, storm cells risk and population risk (modeled with population density).
Fig. 19. Illustration of a cell in the cell sequence for a given trajectory segment AB
On the other hand, it is much more difficult to ensure additive consistency for non-cumulative variables. An example of a non-cumulative variable is the cruising levels variable ccl . At altitudes below 5000ft, if the track does not conform to the cruising levels rule, then a constant value is assigned to ccl ; otherwise, ccl = 0. Given two nodes (s, s0 ), the LHS in (20) will be less than the RHS as the effect of ccl is added multiple times on the RHS. A simple method for enforcing additive consistency is to transform all non-cumulative variables into cumulative variables. The cruising levels rule can thus be transformed by multiplying ccl by the length of the track. Another non-cumulative decision variable is the time of arrival at a successor node. Even though both the time of arrival and the transition time can be used to address the decision objective of minimising flight time, the time of arrival has the added benefit of increasing in value as time passes. Therefore, given a constant weighting (again assuming a weighted sum cost function), it is possible to obtain increasing weighting for the time decision criterion with flight time. This can be used to implicitly enforce maximum flight time and fuel consumption. The time of arrival decision variable can be transformed into a cumulative variable by taking the sum of the time of arrival at each cell for all cells on the cell sequence. D. Uncertainty MSA* returns a path comprising a sequence of cells which form a corridor in 4D space around the planned trajectory. This differs from existing vector neighbour based methods like [15] which do not explicitly associate cells or a volume of space with each trajectory segment. Such a cell sequence provides an inherent tolerance to uncertainty. This approach avoids the intractability of directly incorporating uncertainty into the search space (using methods such as Markov Decision Processes) for a large, high dimensional search space [1]. The level of tolerance can be determined by finding the minimum perpendicular distance d between the track and the cell boundaries for each cell on the trajectory segment. This −−→ is shown in Fig. 19 where AB is the trajectory segment and −−→ DC is the perpendicular distance to an exterior corner of a cell on the sequence γs0 . An exterior corner is one that is not completely enclosed by adjacent cells. The angle θ can be
236
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
determined by the dot product of vector AB and AC, −−→ −→ −−→ −→ AB · AC = AB AC cosθ
(21)
In 4ADC, as 6 ADC = 90 (by definition), then AD = AC cos θ. The position of D can then be determined using a simple vector line equation and AD −−→ → ~ + AB − (22) D=A −−→ AC cos θ AB ◦
Because the cell sequence generated using (2) only includes cells that intersect the track, if D does not lie within the cell, then that particular corner is ignored. Otherwise, the perpendicular distance DC is −−→ −→ (23) DC = AC sin θ
The value of d is determined by taking the minimum DC value for all corners of all cells in γs0 . Note that θ = 0 implies that the trajectory intersects a cell edge/corner in which case all adjacent cells were already included in the cell sequence. Hence, this implies a non-exterior cell corner. For the successor operator depicted in Fig. 7 and a 3D cell size of 1852m×1852m×304.8m, the minimum 3D tolerance is 50.6m, and the minimum horizontal (xy) tolerance is 256.8m. Note that all transition manoeuvres (i.e. turns) needed to transition between tracks are assumed to be of negligible cost compared to the tracks themselves. These manoeuvres are assumed to stay well within the boundaries of the cell sequence. It is possible to modify the cell sequence returned by the method described in section III-A to enforce a minimum tolerance constraint (3D or horizontal only). For each exterior corner of each cell that does not satisfy the distance constraint, it is a simple matter to include all cells adjacent to that corner to increase the minimum tolerance. This procedure is repeated until all exterior corner points satisfy the minimum tolerance. The time level of each added cell can be determined in the same manner as in (5). VII. C ONCLUSION The paper presented MSA*, a method for motion planning using a variable successor operator that finds least cost paths. A variable successor operator enables variable track length, angle and velocity trajectory segments that are modeled using a computer graphics inspired cell sequence. The ability to define arbitrary track angle resolution and define arbitrary cruise velocities is important for 4D vehicle motion planning due to the presence of wind. Furthermore, the modeling of trajectory segments with a cell sequence provides an inherent tolerance to uncertainty. The tolerance is measured using the minimum distance between the track and cell sequence boundaries. Additionally, a variable successor operator enables the imposition of a multi-resolution lattice structure on the search space which drastically reduces the number of search nodes and search time. Extensive simulations for a UAV flight planning task reveal that multi-resolution MSA* is approximately
15
four times faster than vector neighbourhood based A* (Vector A*) but returns paths of approximately the same cost (average path cost ratio of 1.033). Even with a uniform, fine resolution lattice, MSA* is still twice as fast as Vector A* with an average path cost ratio of 0.99. It is shown that MSA* is suited to online replanning with an average computation time (4.46s for multi-resolution MSA*) that is a fraction of the minimum track traversal time (2min.). Simulation studies also revealed the need for additive consistency to avoid bias in the selection of trajectory segments, especially for multi-resolution planning. This property is based on the premise that, given a particular cell sequence, the cost of traversal using a single trajectory segment is equal to that of using two or more shorter trajectory segments. It can be seen that MSA* addresses the requirements of vehicle motion planning as it efficiently finds a path that considers multiple planning objectives under varying wind conditions. As well, the algorithm mitigates uncertainty through tolerance and its suitability to online replanning. Future work primarily revolves around the implementation and real world testing of the proposed algorithm in a closed loop intelligent control system. Additional avenues for study include the use of heuristic inflation (such as with anytime replanning A* [51]) and multi-objective heuristics to further reduce computation time. ACKNOWLEDGMENT The authors would like to express their sincere gratitude to Dr Oliver Obst for his insightful feedback on the paper. R EFERENCES [1] S. M. LaValle, Planning Algorithms. New York: Cambridge University Press, 2006. [2] C. C. Eriksen, T. J. Osse, R. D. Light, T. Wen, T. W. Lehman, P. L. Sabin, J. W. Ballard, and A. M. Chiodi, “Seaglider: a longrange autonomous underwater vehicle for oceanographic research,” IEEE Journal of Oceanic Engineering, vol. 26, no. 4, pp. 424–436, 2001. [3] S. S. Wegener, S. S. Schoenung, J. Totah, D. Sullivan, J. Frank, F. Enomoto, C. Frost, and C. Theodore, “UAV autonomous operations for airborne science missions,” in AIAA 3rd “Unmanned Unlimited” Technical Conference, Workshop and Exhibit, Chicago, Illinois, 2004. [4] W. B. Powell, Approximate dynamic programming. New Jersey: WileyInterscience, 2008. [5] J. S. Mitchell and C. H. Papadimitriou, “Weighted region problem. finding shortest paths through a weighted planar subdivision,” Journal of the Association for Computing Machinery, vol. 38, no. 1, pp. 18–73, 1991. [6] D. Ferguson and A. Stentz, “Using interpolation to improve path planning: the Field D* algorithm,” Journal of Field Robotics, vol. 23, no. 2, pp. 79–101, 2006. [7] A. Nash, K. Daniel, S. Koenig, and A. Felner, “Theta*: Any-angle path planning on grids,” in AAAI Conf. on Artificial Intelligence, 22-26 July 2007. [8] R. Bonasso, D. Kortenkamp, D. P. Miller, and M. G. Slack, “Experiences with an architecture for intelligent, reactive agents,” in International Joint Conference on Artificial Intelligence, 1995. [9] P. Hart, N. Nilsson, and B. Rafael, “A formal basis for the heuristic determination of minimum cost paths,” IEEE Trans. Systems Science and Cybernetics, vol. 4, pp. 100–107, 1968. [10] N. Suzuki, D. Araki, A. Higashide, and T. Suzuki, “Geographical route planning based on uncertain knowledge,” in Proc. 7th Int. Conf. on Tools with Artificial Intelligence, 1995, pp. 434–441. [11] A. Yahja, S. Singh, and A. Stentz, “An efficient on-line path planner for mobile robots operating in vast environments,” Robotics and Autonomous Systems, vol. 33, no. 2&3, 2000.
237
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
[12] S. Koenig and M. Likhachev, “Improved fast replanning for robot navigation in unknown terrain,” in IEEE Int. Conf. Robotics and Automation, vol. 1, 2002, pp. 968–975 vol.1. [13] J. Rubio and S. Kragelund, “The trans-pacific crossing: long range adaptive path planning for UAVs through variable wind fields,” in 22nd Digital Avionics Systems Conference, vol. 2, 2003, pp. 8.B.4–81–12 vol.2. [14] D.-W. Gu, W. Kamal, and I. Postlethwaite, “A UAV waypoint generator,” 20-22 Sep 2004. [15] M. Pivtoraiko and A. Kelly, “Generating near minimal spanning control sets for constrained motion planning in discrete state spaces,” in IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2005, pp. 3231– 3237. [16] J. Carsten, D. Ferguson, and A. Stentz, “3D Field D*: Improved path planning and replanning in three dimensions,” in IEEE Int. Conf. Intelligent Robots and Systems, 2006, pp. 3381–3386. [17] C. Petres, Y. Pailhas, P. Patron, Y. Petillot, J. Evans, and D. Lane, “Path planning for autonomous underwater vehicles,” IEEE Trans. Robotics, vol. 23, no. 2, pp. 331–341, 2007. [18] Y. Kim, D.-W. Gu, and I. Postlethwaite, “Real-time path planning with limited information for autonomous unmanned air vehicles,” Automatica, vol. 44, no. 3, pp. 696–712, 2008. [19] P. Tompkins, A. T. Stentz, and W. R. L. Whittaker, “Mission-level path planning for rover exploration,” in 8th Conf. on Intelligent Autonomous Systems (IAS-8), March 2004. [20] J. Gonzalez and A. Stentz, “Planning with uncertainty in position an optimal and efficient planner,” in IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2005, pp. 2435–2442. [21] K. Fujimura, “Path planning with multiple objectives,” IEEE Robotics & Automation Magazine, vol. 3, no. 1, pp. 33–38, 1996. [22] S. Bortoff, “Path planning for UAVs,” in American Control Conference, vol. 1, 2000, pp. 364–368 vol.1. [23] Y.-H. Qu, Q. Pan, and J.-G. Yan, “Flight path planning of UAV based on heuristically search and genetic algorithms,” in IEEE Industrial Electronics Society 32nd Annual Conf., 2005, p. 5. [24] D. Rathbun, S. Kragelund, A. Pongpunwattana, and B. Capozzi, “An evolution based path planning algorithm for autonomous motion of a UAV through uncertain environments,” in Proceedings 21st Digital Avionics Systems Conference, vol. 2, 2002, pp. 8D2–1–8D2–12 vol.2. [25] I. K. Nikolos, K. P. Valavanis, N. C. Tsourveloudis, and A. N. Kostaras, “Evolutionary algorithm based offline/online path planner for UAV navigation,” IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 33, no. 6, pp. 898–912, 2003. [26] P. O. Pettersson and P. Doherty, “Probabilistic roadmap based path planning for an autonomous unmanned aerial vehicle,” in ICAPS, 2004. [27] M. S. Branicky, S. M. LaValle, K. Olson, and Y. Libo, “Quasirandomized path planning,” in IEEE Int. Conf. Robotics and Automation (ICRA), vol. 2, 2001, pp. 1481–1487 vol.2. [28] E. B. Smith and R. Langari, “Fuzzy multiobjective decision making for navigation of mobile robots in dynamic, unstructured environments,” Journal of Intelligent & Fuzzy Systems, vol. 14, no. 2, pp. 95–108, 2003. [29] B. Pfeiffer, R. Batta, K. Klamroth, and R. Nagi, “Probabilistic modeling for UAV path planning in the presence of threat zones,” Operations Research, 2005. [30] J. Reif and Z. Sun, “Movement planning in the presence of flows,” Algorithmica, vol. 39, no. 2, pp. 127–153, June 2004. [31] Z. Sun and J. Reif, “On robotic optimal path planning in polygonal regions with pseudo-euclidean metrics,” IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 37, no. 4, pp. 925–936, Aug. 2007. [32] N. A. Papadakis and A. N. Perakis, “Deterministic minimal time vessel routing,” Operations Research, vol. 38, no. 3, pp. 426–438, 1990. [33] J. Sellen, “Direction weighted shortest path planning,” IEEE Int. Conf. Robotics and Automation, vol. 2, pp. 1970–1975 vol.2, May 1995. [34] D. B. Fogel, Evolutionary Computation: Toward a New Philosophy of Machine Intelligence. San Francisco, CA: Morgan Kaufmann, 1995. [35] RTCA, “Software considerations in airborne systems and equipment certification,” 1992.
16
[36] G. F. List, P. B. Mirchandani, M. A. Turnquist, and K. G. Zografos, “Modeling and analysis for hazardous materials transportation - risk analysis, routing scheduling and facility location,” Transportation Science, vol. 25, no. 2, pp. 100–114, 1991. [37] P. B. Mirchandani and H. Soroush, “Optimal paths in probabilistic networks: A case with temporary preferences,” Computers & Operations Research, vol. 12, no. 4, pp. 365–381, 1985. [38] P. Leonelli, S. Bonvicini, and G. Spadoni, “Hazardous materials transportation: a risk-analysis-based routing methodology,” Journal of Hazardous Materials, vol. 71, no. 1-3, pp. 283–300, 2000. [39] M. Zhang, Y. Ma, and K. Weng, “Location-routing model of hazardous materials distribution system based on risk bottleneck,” in Int. Conf. on Services Systems and Services Management, vol. 1, 2005, pp. 362–368. [40] P. Wu, R. Clothier, D. Campbell, and R. Walker, “Fuzzy multi-objective mission flight planning in unmanned aerial systems,” in IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making, Honolulu, Hawaii, 2007. [41] B. S. Stewart and I. Chelsea C. White, “Multiobjective A*,” J. ACM, vol. 38, no. 4, pp. 775–814, 1991. [42] K. Erol, J. Hendler, and D. S. Nau, “Htn planning: Complexity and expressivity,” in AAAI National Conf. on Artificial Intelligence, 1995, p. 1123. [43] P. Doherty, J. Gustafsson, L. Karlsson, and J. Kvarnstrom, “(TAL) Temporal action logics: Language specification and tutorial,” Electronic Transactions on Artificial Intelligence, vol. 2, no. 3-4, pp. 273–306, 1998. [44] T. Belker, M. Hammel, and J. Hertzberg, “Learning to optimize mobile robot navigation based on htn plans,” vol. 3, Sept. 2003, pp. 4136–4141. [45] J. Bresenham, “Pixel-processing fundamentals,” IEEE Computer Graphics and Applications, vol. 16, no. 1, pp. 74–82, 1996. [46] W. F. Phillips, Mechanics of Flight. Hoboken, New Jersey, USA: John Wiley & Sons, 2004. [47] P. Wu, D. Campbell, and T. Merz, “On-board multi-objective mission planning for unmanned aerial vehicles,” in IEEE Aerospace Conference, Big Sky, Montana, 7-14 March 2009. [48] S. J. Russell and P. Norvig, Artificial Intelligence: a modern approach, 2nd ed. Upper Saddle River, N.J.: Prentice Hall, 2003. [49] Office of the Secretary of Defense, “Unmanned aircraft systems roadmap: 2005-2030,” Tech. Rep., 2005. [50] Civil Aviation Safety Authority (CASA), “Civil aviation regulations 1988 (CAR 1988),” August 2003. [51] D. Ferguson, M. Likhachev, and A. T. Stentz, “A guide to heuristic-based path planning,” in Int. Conf. on Automated Planning and Scheduling (ICAPS), Monterey, CA, 2005. [52] NOAA National Weather Service, “Structure and dynamics of supercell thunderstorms,” 2004, http://www.crh.noaa.gov/lmk/soo/docu/supercell. php. [53] Bureau of Meteorology, “Aviation weather services,” 2008, http://www. bom.gov.au/reguser/by prod/aviation/.
Paul Wu is a PhD candidate at the Queensland University of Technology. He completed his degree in Electrical and Computer Engineering with honours at QUT in 2005 and obtained a masters degree in Computer and Communications in the same year. He has worked on a variety of projects, including a system for delivery of multimedia content to mobile phones, for which he was awarded the Engineers Australia Queensland Division J H Curtis Award, and performance optimised Field Programmable Gate Array (FPGA) fuzzy logic processing. He is currently undertaking research on algorithms and architectures for UAV path planning.
238
APPENDIX C. PUBLICATIONS
IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART B: CYBERNETICS, VOL. , NO. , MONTH YEAR
Dr Duncan Campbell is an Associate Professor at the Queensland University of Technology (QUT), in Brisbane, Australia. He is the Alternate Head of the School of Engineering Systems. Dr Campbell has over 65 internationally peer reviewed papers and researches in the areas of robotics and automation, embedded systems, computational intelligence, intelligent control and decision support. He leads a group in the Australian Research Centre for Aerospace Automation and has collaborations with a number of universities around the world including Massachusetts Institute of Technology, USA and TELECOM-Bretagne, France. Dr Campbell is currently the IEEE Queensland Section chapter chair of the Control Systems / Robotics and Automation Society Joint Chapter.
Dr Torsten Merz is a senior research scientist at CSIRO’s Autonomous Systems Laboratory where he is leading the development of civil aerial robots. Previously, he held a position as assistant professor at the Department of Computer and Information Science at Link¨oping University, Sweden. He was part of the WITAS UAV project team and involved in the development of flight control modes and mission planners for unmanned helicopters. He earned a Diploma in Informatics from the University of Bielefeld and received a Doctor of Engineering from the University of Erlangen-Nuremberg, Germany.
17
