Classical Planning: Exploring the Boundaries

[email protected] – 2016

Modeling in PDDL vs. Modeling in the State Transition System

What can be modeled at all vs. What can be modeled conveniently, efficiently



What is a "statement"? Integer An actual number, a member of ℤ

3 "Integer Statements" A way of describing a number using a specific language "10" chars "1" "0" "X" "0x0A" "p#c" [in some made-up language] …

jonkv@ida

Preliminaries



What is a "statement"? Planning Problem

The actual structure: A state transition system, an initial state, a set of goal states

s0 s2

s1 s3 s5

s4 s6

s7

s8

4 Problem Statement

A way of describing the problem Many problem statements correspond to the same problem – Could list all states, actions, transitions – Could use PDDL: (:action MOVE :parameters (?disk ?source ?dest) …) … – General Problem Solver language –…

jonkv@ida

Preliminaries (2)

5

Real World

The "real domain"

A way of writing it down Abstraction Approximation

State Trans Syst Σ = (S,A,γ)

Equivalence

First-order language L defined by predicates, objects Operators O

States need no internal structure

States are sets of atoms, induced by the predicates and objects in L

Actions are unstructured symbols

Operators are structured, have preconditions and effects

State transitions are unstructured (γ specified by state / action symbols)

Each operator specifies part of γ, through its preconds and effects

jonkv@ida

Problems vs. Problem Statements 1

6

Real World + current problem

The "real problem"

A way of writing it down Abstraction Approximation

Planning Problem P = (Σ, s0, Sg)

Equivalence

Language L defined by predicates, objects Problem Statement in L P=(O,s0,g)

Specifies the ID of the initial state: s0

Specifies the true atoms in the init state: { attached(p1,loc1), in(c1,p1), … }

Specifies a set of possible goal states: Sg = { s0, s1, s2, s20, s21, s22, s4912, … }

Specifies a set of literals that must hold: g = { in(c1,p2), in(c2,p2) , … }

jonkv@ida

Problems vs. Problem Statements 2

7

Real World + current problem Language L defined by predicates, objects Planning Problem P = (Σ, s0, Sg)

Problem Statement P=(O,s0,g)

Limits what can be modeled

Limits what is easy to express

Some extensions require changing the STS

Many extensions can easily be translated into the same STS structure Can still be called classical planning, since no STS changes are required

Planners use the language, not the STS: An extension changing the STS may be easily handled (action costs) or hard (probabilities), one that leaves the STS unchanged may be easy or hard…

jonkv@ida

Boundaries: Of what?



Let's model a "drive" operator for a truck  ”Natural” parameters: The truck and the destination ▪ (:action drive :parameters (?t – truck ?dest – location) :precondition … :effect … )  ”Natural” precondition: ▪ There must exist a path between the current location and the destination ▪ Should use the predicate (path-between ?loc1 ?loc2 – location)  How? ▪ (:precondition (path-between …something… ?dest)) ??? ▪ In a first-order predicate representation, we can only test whether a truck is at some specific location: (at ?t ?location)

9

jonkv@ida

Properties of Objects

10

jonkv@ida

Alternative Representations Three wide classes of logic-based representations (general classes, containing many languages!) Propositional (boolean propositions)

First-order (boolean predicates)

State-variable-based (non-boolean functions)

atHome, atWork

at(truck, location)

loc(truck) = location

PDDL :strips (if you avoid objects)

PDDL :strips, …

Read chapter 2 of the book for another perspective on representations…



11

Classical planning with classical representation  A state defines the values of logical atoms (boolean) ▪ adjacent(location, location) – can you go directly from one loc to another? ▪ loaded(robot, container) – is the robot loaded with the given container?

May be wasteful: Can represent a container being on many robots, which never happens

c3 c1 Can be convenient, space-efficient  often used internally!



p1

loc2

c2 p2

loc1

r1 Seems more powerful, but is equivalent!

Alternative: Classical with state-variable representation  A state defines the values of arbitrary state variables ▪ boolean adjacent(location, location) ;; still boolean! ▪ container carriedby(robot) ;; which container is on the robot?

jonkv@ida

Classical and State-Var Representation



Back to the "drive" operator…  ”Natural” parameters: The truck and the destination ▪ (:action drive :parameters (?t – truck ?dest – location) :precondition … :effect … )  ”Natural” precondition: ▪ There must exist a path between the current location and the destination ▪ Should use the predicate (path-between ?loc1 ?loc2 – location) ▪ State variable representation  can express the location of the truck: (:precondition (path-between (location-of ?t) ?dest)) ▪ No STS extensions are required!

12

jonkv@ida

Property Values 1



To avoid using state variables:  Add a parameter to the operator ▪ (:action drive :parameters (?t – truck ?from – location ?dest – location) :precondition … :effect … )  Constrain that variable in the precondition ▪ :precondition (and (at ?t ?from) (path-between ?from ?dest)) ▪ Can only apply those instances of the operator where ?from is the current location of the truck

13

jonkv@ida

Property Values 2



Example:  Initially: ▪ (at truck5 home)

14 These parameters are "extraneous" in the sense that they do not add choice: We can choose truck and dest (given some constraints): from is uniquely determined by state + other params!

 Action: ▪ (:action drive :parameters (?t – truck ?from – location ?dest – location) :precondition (and (at ?t ?from) (path-between ?from ?dest)) :effect … )  Which actions are executable? ▪ (drive truck5 work home) – no, precond false ▪ (drive truck5 work work) – no, precond false ▪ (drive truck5 work store) – no, precond false ▪ (drive truck5 home store) – precond true, can be applied!

With quantification, we could have changed the precondition: (exists (?from – location) (and (at ?t ?from) (path-between ?from ?dest)) No need for a new parameter – in this case…

jonkv@ida

Property Values 3



15

What about effects?  Same ”natural” parameters: The truck and the destination ▪ (:action drive :parameters (?t – truck ?dest – location) :precondition … :effect … )  ”Natural” effects: ▪ The truck ends up at the destination: ▪ The truck is no longer where it started:

(at ?t ?dest) (not (at ?t …???... ))

 How do you find out where the truck was before the action? ▪ Using an additional parameter still works: (not (at ?t ?from)) ▪ The value of ?from is constrained in the precondition – before ▪ The value is used in the effect state

jonkv@ida

Property Values 4



17

We often need at least some ”primitive” support for numbers  Elevator domain: ▪ Which floor is an elevator at? ▪ Which is the next floor? ▪ Which is the previous floor?

jonkv@ida

Numbers 1



18

jonkv@ida

Numbers 2 PDDL 2.1 supports numeric fluents  (define (domain jug-pouring)

(:requirements :typing :fluents) (:types jug) (:functions (amount ?j - jug) (capacity ?j - jug)) (:action pour :parameters (?jug1 ?jug2 - jug) :precondition (>= (- (capacity ?jug2) (amount ?jug2)) (amount ?jug1)) :effect (and Change the value: (assign (amount ?jug1) 0) Absolute or (increase (amount ?jug2) (amount ?jug1))) relative to old value ) Easily supported by the formal STS, if you have a finite set of numbers (even 64-bit floating point is finite!) Unsupported in many planners due to constrained heuristics etc.



19

If there is no explicit support:  Create a type of ”pseudo-numbers” ▪ (:types … num …)  Define a set of value objects ▪ (:objects … n0 n1 n2 n3 n4 n5 n6 n7 – num)  Define the operations you need – for example, find the next number ▪ (:predicates … (next ?numA ?numB – num) ▪ (:init … (next n0 n1) (next n1 n2) (next n2 n3) … (next n6 n7))  Use the value objects as if they were numbers These are also ▪ (:action move-up "extraneous" parameters… :parameters (?e – elevator ?from ?to – num) :precondition (and (at ?elevator ?from) ;; Where is the elevator? (next ?from ?to) …) ;; Now ”to” is the next number :effects (and (not (at ?elevator ?from)) (at ?elevator ?to))) There is no ”next” for n7  Won’t be able to move up from the top floor (an "implicitly modeled" precondition)

jonkv@ida

Numbers 3



21

Suppose we have a number of ground robots

r1

 Can drive between ?from and ?to if there is a road,

or the robot has all-wheel-drive

r2 r3

 Disjunctive representation: ▪ (:requirements :disjunctive-preconditions …) (:action drive :parameters (?r - robot ?from ?to - location) :precondition (and (or (road-between ?from ?to) (all-wheel-drive ?r)) (at ?r ?from)) :effect (and (at ?r ?to) (not (at ?r ?from)) ))  The precondition no longer corresponds to a set of literals that must hold! ▪ Outside the book's classical representation… ▪ But can be modeled as an STS – only affects where edges go

jonkv@ida

Disjunctive Preconditions



Disjunctive preconditions:  Convenient  Easily supported by the formal model ▪ Simply an easier way of specifying the state transition function  Not always supported by planners ▪ Some algorithms are very efficient, but cannot handle disjunctions ▪ Some heuristics are very informative, but cannot handle disjunctions ▪ … ▪ Tradeoff between convenience and efficiency!

22

jonkv@ida

Disjunctive Preconditions (2)



23

Workaround 1: Rewrite the disjunction using two distinct operators

r1 r2

r3 ▪ (:action drive-on-road :parameters (?r - robot ?from ?to - location) :precondition (and (road-between ?from ?to) (at ?r ?from)) :effect (and (at ?r ?to) (not (at ?r ?from)) )) ▪ (:action drive-all-wheel-drive :parameters (?r - robot ?from ?to - location) :precondition (and (all-wheel-drive ?r) (not (road-between ?from ?to) (at ?r ?from)) :effect (and (at ?r ?to) (not (at ?r ?from)) )) Why should we have this?  Any problems?

What about the condition (a ∨ b ∨ c ∨ d) ∧ (e ∨ f ∨ g ∨ h)?

jonkv@ida

Disjunctive Preconditions (3)



Workaround 2: use a different domain formulation  Add a predicate: (can-drive-between ?robot ?from ?to) ▪ Specify its value explicitly in the initial state ▪ Redundant – but planners can use it efficiently!



Planners could:  Directly and efficiently support disjunctions ▪ Possible for some algorithms, some heuristics  Automatically rewrite into multiple operators ▪ Could lead to inefficient planning, without any indication of which constructs are inefficient  Disallow disjunctions ▪ Encourages writing another domain model – might be more efficient ▪ Can still use external rewriting tools

24

jonkv@ida

Disjunctive Preconditions (4)



Quantifiers in preconditions can be convenient  To drive a car, all doors must be closed ▪ (:requirements :universal-preconditions) (:action drive (:parameters ?car – car ?from ?to – location) (:precondition (forall (?door – door) (implies (belongs ?door ?car) (closed ?door)))))  Can be transformed to a conjunction by expanding the quantifier ▪ Suppose we have 4 doors: { d1, d2, d3, d4 } ▪ (:precondition (and (implies (belongs d1 ?car) (closed d1)) (implies (belongs d2 ?car) (closed d2)) (implies (belongs d3 ?car) (closed d3)) (implies (belongs d4 ?car) (closed d4)))) ▪ Must know which doors we have (instance-specific!) ▪ Suppose we have 100 cars, 400 doors…

26

jonkv@ida

Quantified Preconditions



Existential quantifiers are also convenient  To drive a car, I must have some matching key ▪ (:requirements :existential-preconditions) (:action drive (:parameters ?c – car ?from ?to – location) (:precondition (exists (?k – key) (and (have ?k) (matches ?k ?c)))))  Can be transformed to a disjunction by expanding the quantifier ▪ Suppose we have 4 keys: { k1, k2, k3, k4 } ▪ (:precondition (or (and (have k1) (matches k1 ?c)) (and (have k2) (matches k2 ?c)) (and (have k3) (matches k3 ?c)) (and (have k4) (matches k4 ?c)))) ▪ Could then transform this disjunction into multiple operators… ▪ Again, the domain can be modeled differently: (have-key-matching ?c)

27

jonkv@ida

Quantified Preconditions (2)



Alternative workarounds exist

28 c2

loc2

c3 p2  Introduce redundant predicates c1 r1 loc1 p1 ▪ Dock Worker Robots: (:predicates (at ?r - robot ?l - location) ; robot ?r is at location ?l ?l - location) ; there is a robot at location ?l (occupied …) ▪ Where (occupied ?loc) is the same as (exists (?r – robot) (at ?r ?loc))! Corresponds to (not (exists (?r2 – robot)) (at ?r2 ?to))

 Update redundant predicates when necessary ▪ (:action move :parameters (?r - robot ?from ?to - location) :precondition (and (adjacent ?from ?to) (at ?r ?from) (not (occupied ?to)) ) :effect (and (not (at ?r ?from)) (at ?r ?to) (not (occupied ?from)) (occupied ?to) ))

jonkv@ida

Quantified Preconditions (3)



Wumpus World:  Can move between two squares  Move into a pit  you die

▪ (:action move :parameters (?who – agent ?from – square ?to – square) :precondition (and (alive ?who) (at ?who ?from) (adjacent ?from ?to)) :effect (and (not (at ?who ?from)) (at ?who ?to) (when (or (pit ?to) (has-living-wumpus ?to)) (not (alive ?who))) ) Again, fits the same STS structure: Simply a more convenient way of specifying the transition function

30

jonkv@ida

Quantified Effects (1)



31

To remove conditional effects:  Convert to multiple actions ▪ (:action move-stay-alive :parameters (?who – agent ?from – square ?to – square) :precondition (and (alive ?who) (at ?who ?from) (adjacent ?from ?to) (not (pit ?to)) (not (has-living-wumpus ?to))) :effect (and (not (at ?who ?from)) (at ?who ?to) )) ▪ (:action move-and-die :parameters (?who – agent ?from – square ?to – square) :precondition (and (alive ?who) (at ?who ?from) (adjacent ?from ?to) (or (pit ?to) (has-living-wumpus ?to)) :effect (and Once more, a potential combinatorial (not (at ?who ?from)) explosion in the number of actions (at ?who ?to) (not (alive ?who)) If the planner can handle conditional )) effects: Probably more efficient!

jonkv@ida

Quantified Effects (2)

▪

33 (and (at truck1 locA) (in pkg1 truck1) … (in pkg4 truck1) (at pkg1 locA) (at pkg2 locA) (at pkg3 locA) (at pkg4 locA))

▪ (:action drive-truck :parameters (?truck - truck ?loc-from ?loc-to – location ?city - city) :precondition (and (at ?truck ?loc-from) (in-city ?loc-from ?city) (in-city ?loc-to ?city)) :effect (and (at ?truck ?loc-to) (not (at ?truck ?loc-from)))) All packages in the truck should move Unknown how many  can't be parameters!

jonkv@ida

Motivation



34

If you drive a truck, all items in the truck should follow it  Example: ▪ (:requirements :universal-preconditions :conditional-effects …) (:action drive-truck :parameters (?truck - truck ?loc-from ?loc-to – location ?city - city) :precondition (and (at ?truck ?loc-from) (in-city ?loc-from ?city) (in-city ?loc-to ?city)) :effect (and (at ?truck ?loc-to) (not (at ?truck ?loc-from)) (forall (?x - obj) (when (in ?x ?truck) (and (not (at ?x ?loc-from)) (at ?x ?loc-to))))))  In this model, if an object is initially at locationA: ▪ (at ?obj locationA) remains true when the object is loaded into the truck ▪ (at ?obj locationA) becomes false only when the truck drives away

jonkv@ida

Universal and Conditional Effects



35

If a planner does not support this:  Quantifiers can be expanded for a specific problem instance, as before ▪ (forall (?x – obj) …)  (and (when (in packageA ?truck) (…)) (when (in packageB ?truck) (…)) … (when (in packageX ?truck) (…))

 Conditional effects can be expanded into multiple operators ▪ One with precond (and … (in packageA truck) (in packageB truck) …) ▪ One with precond (and … (not (in packageA truck)) (in packageB truck) …), and so on

Works – but can be inefficient!

jonkv@ida

Universal and Conditional Effects (2)



36

Sometimes you can use different domain models  Alternative: A package in a truck is not at any location at all! ▪ (at ?obj ?location) removed by load-package action, before driving ▪ (in ?obj ?truck) added instead  Driving a truck only moves the truck ▪ Packages are still in the same truck, at no location at all ▪ No need for quantified conditional effects here  Unloading a package: ▪ (in ?obj ?truck) removed ▪ (at ?obj ?new-location) added

Works – as long as no operator or goal cares about "at" for a package that is in a truck!

jonkv@ida

Universal and Conditional Effects (3)



Quantified goals:  Universal goals (all crates should be at their destinations) are simple ▪ Expand into a conjunction  Existential goals seem more difficult ▪ We defined a goal as a set of literals, all of which must be true



Can we indirectly implement existential goals even if only conjunctive goals are explicitly supported?  Yes, through new actions and predicates! ▪ Suppose we have a goal: (or a b c d) ▪ Add a new predicate “goal-achieved”, which replaces the goal ▪ Make the predicate false in the initial state ▪ Add an operator: (:action finish (:precondition (or a b c d)) (:effect goal-achieved)))

38

jonkv@ida

Quantified Goals