Dipartimento di Elettronica Informazione e Bioingegneria

Cognitive Robotics Planning agents: from STRIPS to non linear planning @ G. Gini 2015

3 blocks problem in STRIPS start: ON(C, A) ∧ ONTABLE(A) ∧ ONTABLE(B) ∧ ARMEMPTY

goal: ON(A,B) ∧ ON(B, C) ∧ ONTABLE(C)

Which one is the greatest difference

? A

C

A

B B

C

G. Gini

2

Reach subgoals in order

3

1. on(A,B) UNSTACK(C,A), PUTDOWN(C), PICKUP(A), STACK(A,B)

2. on(B,C): UNSTACK(A,B), PUTDOWN(A), PICKUP(B), STACK(B,C)

3. ontable(C) is true in the initial state The plan is the concatenation 1. UNSTACK(C,A), 2. PUTDOWN(C), 3. PICKUP(A), 4. STACK(A,B), 5. UNSTACK(A,B), 6. PUTDOWN(A), 7. PICKUP(B), 8. STACK(B,C)

… is wrong

B A

B

C

Sussman anomaly G. Gini

C

A

Solution: put goals on a stack • • • •

•

on(A,B) is deleted by step 5 Add it on the stack and reach it again The plan: UNSTACK(C,A) PUTDOWN(C) PICKUP(A) UNSTACK(A,B) PUTDOWN(A) PICKUP(B) PICKUP(A) STACK(A,B) Optimize the plan UNSTACK(C,A) PUTDOWN(C) PICKUP(B) C STACK(B,C) A B PICKUP(A) STACK(A,B)

No guarantee to succeed G. Gini

4

STACK(A,B) STACK(B,C)

A B C

After STRIPS and after Shakey • two critical points: ƒ Poor planning language • without time, conditions, changes, … ƒ

Sussman anomaly: for multiple goals the stacking strategies cannot guarantee to find a correct plan • A plan is not a sequence of complete sub-plans

• Non linear planning: choose the operators, then order them Stanford Cart, 1970s G. Gini

5

Planning language

6

• What is a good language? ƒ

Expressive enough to describe a wide variety of problems. ƒ Restrictive enough to allow efficient algorithms to operate on it. ƒ Planning algorithms should be able to take advantage of the logical structure of the problem.

• STRIPS, ADL, PDDL

G. Gini

ADL Action DescriptionLanguage-Pednault 1989 •

7

Generalizes STRIPS to allow for ƒ Arbitrary first-order preconditions (quantifiers) ƒ Conditional effects ƒ Functions

Init (On(A, Table) ∧ On(B,Table) ∧ On(C,Table) ∧ Block(A) ∧ Block(B) ∧ Block(C) ∧ Clear(A) ∧ Clear(B) ∧ Clear(C)) Goal(On(A,B) ∧ On(B,C)) Action(Move(b,x,y) PRECOND: On(b,x) ∧ Clear(b) ∧ Clear(y) ∧ Block(b) ∧ (b≠ x) ∧ (b≠ y) ∧ (x≠ y) EFFECT: On(b,y) ∧ Clear(x) ∧ ¬ On(b,x) ∧ ¬ Clear(y)) Action(MoveToTable(b,x) PRECOND: On(b,x) ∧ Clear(b) ∧ Block(b) ∧ (b≠ x) EFFECT: On(b,Table) ∧ Clear(x) ∧ ¬ On(b,x)) G. Gini

universal quantifier • Example: move a container and all its content

G. Gini

8

PDDL (McDermott) • • • •

Defined for planning competitions - 1998 Different versions Defines types Indicates the extension used: ƒ

STRIPS is the basic

G. Gini

9

Ex- Hanoi tower (define (problem hanoi4) //3 pegs (:domain hanoi) (:objects peg1 peg2 peg3 d1 d2 d3 d4) (:init (smaller d1 peg1) (smaller d2 peg1) (smaller d3 peg1) (smaller d4 peg1) (smaller d1 peg2) (smaller d2 peg2) (smaller d3 peg2) (smaller d4 peg2) (smaller peg3 d1) (smaller peg3 d2) (smaller peg3 d3) (smaller peg3 d4) (smaller d1 d2) (smaller d1 d3) (smaller d2 d3) (smaller d1 d4) (smaller d2 d4) (smaller d3 d4) (clear peg2) (clear peg3) (clear d1) (on d4 peg1) (on d3 d4) (on d2 d3) (on d1 d2)) (:goal (and (on d4 peg3) (on d3 d4) (on d2 d3) (on d1 d2)))) • •

smaller is defined between disks and between peg and disk. clear is a property of disks and pegs. G. Gini

10

Operator move

11

(:action move :parameters (?disc ?from ?to) :precondition (and (smaller ?disc ?to) (on ?disc ?from) (clear ?disc) (clear ?to)) :effect (and (clear ?from) (on ?disc ?to) (not (on ?disc ?from)) (not (clear ?to)))) )

G. Gini

Non linear nature of plans • The actions can be performed in different orders ƒ It is enough to fullfill some constraints

• Planning is built adding actions, not simulating states • Plan in the planning space, not in the state space G. Gini

12

Non linear plans TWEAK (MIT 1987) Actions are described by STRIPS operators • Partial plan: incrementally built through ƒ

Add a link from action to an open precondition ƒ Add a step to satisfy an open condition ƒ Order the steps

• Least commitment ƒ

Partial order ƒ Partial instantiation G. Gini

13

Operators to construct the plan

14

• Step Addition: add a step. • Promotion: constrain 2 operators. • Declobbering: insert a step S2 between S1 -S3 to ri-assert preconditions for S3 • Simple Establishment: assign a value • Separation: avoid to call a given operator on a given variable

G. Gini

3 blocks problem Initial state: ontable (C), on (C, A), ontable (B), clear (B), clear (C) Goal: ON(A,B)∧ON(B,C)

A C

B A

B

C

finish

start

G. Gini

15

Plan construction

16

Start ontable (C), on (C, A), ontable (B), clear (B), clear (C)

on(A,B), on(B,C)

Finish G. Gini

Step addition to get ON(A,B), ON(B,C) • step addition: STACK(A,B) e STACK(B,C) CLEAR(B) *HOLDING(A) _____________ STACK(A,B) ____________ ARMEMPTY ON(A,B) ¬CLEAR(B) ¬HOLDING(A)

CLEAR(C) *HOLDING(B) ______________ STACK(B,C) ______________ ARMEMPTY ON(B,C) ¬CLEAR(C) ¬HOLDING(B)

Partial plan: STACK(A, B), STACK (B,C)

G. Gini

17

18

Start ontable (C), on (C, A), ontable (B), clear (B), clear (C)

clear(B), holding (A)

clear(C), holding (B)

Stack(A, B)

Stack (B, C)

on(A,B), on(B,C)

Finish G. Gini

19

Start ontable (C), on (C, A), Ontable (B), clear (B), clear (C)

Clear(B), holding (A)

Clear(C), holding (B)

Stack(A, B)

Stack (B, C)

on(A,B), on(B,C)

Finish G. Gini

20

* indicates preconditions not satisfied in the initial state;

• Step addition to obtain Holding for PICKUP: *CLEAR(A) ONTABLE(A) *ARMEMPTY _____________ PICKUP(A) ____________ ¬ONTABLE(A) ¬ARMEMPTY HOLDING(A)

*CLEAR(B) ONTABLE(B) *ARMEMPTY ______________ PICKUP(B) ______________ ¬ONTABLE(B) ¬ARMEMPTY HOLDING(B)

Partial plan: STACK(A, B), STACK (B,C), PICKUP(A), PICKUP(B)

G. Gini

21

Start Ontable (C), ON (C, A), Ontable (B), clear (B), clear (C)

Pickup(A)

Promotion: PICKUP before the relative STACK PICKUP(A)←STACK(A,B) PICKUP(B)←STACK(B,C)

Holding (A)

Clear (B), armempty

Pickup(B) Holding (B)

Stack (B, C)

Stack(A, B) ON(A,B), ON(B,C)

Finish G. Gini

22

4 preconditions to verify: 2 CLEAR and 2 armempty *CLEAR(A) is not true in the initial state; *CLEAR(B) is true in the initial state, not true after STACK(A,B).

• •

• •

Work on B: To verify CLEAR(B), use promotion PICKUP(A)←STACK(A,B), PICKUP(B)←STACK(B,C), PICKUP(B)←STACK(A,B) armempty is true in start, not true after the pickup(B), use promotion PICKUP(B)← PICKUP(A)

Partial plan PICKUP(A)←STACK(A,B) PICKUP(B)←STACK(B,C) PICKUP(B)←STACK(A,B) PICKUP(B)← PICKUP(A) ƒ *ARMEMPTY and *CLEAR(A) for PICKUP(A) are to be verified G. Gini

23

• To verify *ARMEMPTY for PICKUP(A) • use declobbering: Insert a step between PICKUP(B) and PICKU(A) • PICKUP(B)← STACK(B,C) ← PICKUP(A) • To verify *CLEAR(A) for PICKUP(A) • Step addition : add unstack to remove what is over A: *ON(X,A) *CLEAR(X) *ARMEMPTY _____________ UNSTACK(X,A) ______________ ¬ARMEMPTY CLEAR(A) HOLDING(A) ¬ON(X,A) X is not instantiated until necessary We satisfy ON(X,A) with simple establishment: instantiate X to C UNSTACK(C,A) G. Gini

24

• again promotion: UNSTACK(X,A)←STACK(B,C) UNSTACK(X,A)←PICKUP(A) UNSTACK(X,A)←PICKUP(B) • PICKUP(B) needs HARMEMPTY • HARMEMPTY is negated by UNSTACK(X,A); • declobbering • Add a new step between: HOLDING(C ) _____________ PUTDOWN(C ) _____________ ¬HOLDING(C ) ONTABLE(X) ARMEMPTY promotion UNSTACK(X,A)← PUTDOWN(C ) ←PICKUP(B) G. Gini

Final plan

25

Total ordering UNSTACK(X,A)←STACK(B,C) UNSTACK(X,A)←PICKUP(A) UNSTACK(X,A)←PICKUP(B) UNSTACK(X,A)← PUTDOWN(C ) ←PICKUP(B) 1. 2. 3. 4. 5. 6.

UNSTACK(C,A) PUTDOWN(C ) PICKUP(B) STACK(B,C) PICKUP(A) STACK(A,B)

(it is the same found in stack STRIPS) G. Gini

POP (partial order planning) • •

• •

26

Partial order planning Any planning algorithm that can place two actions into a plan without defining which comes first is a POP derived from TWEAK Simplify the plan construction using only causal links A p B the operator A adds the precondition p for operator B

- then order constraint G. Gini

Shoe example

27

Goal(RightShoeOn ∧ LeftShoeOn) Init() Action(RightShoe, Action(RightSock, Action(LeftShoe, Action(LeftSock,

PRECOND: RightSockOn EFFECT: RightShoeOn) PRECOND: EFFECT: RightSockOn) PRECOND: LeftSockOn EFFECT: LeftShoeOn) PRECOND: EFFECT: LeftSockOn)

Planner: combine two action sequences (1)leftsock, leftshoe (2)rightsock, rightshoe

G. Gini

solution

28

G. Gini

State space vs plan space • •

29

in state-space each node is a state of the world in plan-space each node is a partial plan

Def – open condition is a precondition not yet reached The planning algorithm moves from an incomplete (with open conditions) to a completely defined plan (no open conditions) 1. 2. 3.

Add a link from an action to an open condition add a step to satisfy an open condition Order the steps

•

least commitment: ƒ ƒ

Partial order instead of total order Partial instantiation

G. Gini

Definitions for POP • A solution is a consistent and complete plan • A precondition is satisfied iff it is the effect of a previous step and no steps can interfere to vanify it • A plan is complete iff all the preconditions are satisfied • A plan is consistent iff the order constraints and the substitutions are coherent

G. Gini

30

Threat: promotion and demotion

31

A threat is a step that can destroy the conditions created by a causal link S3 S1 S3 c

S1

not c

S1

not c

c c

S2

S2

S2 S3 not c

e.g. pickup(B) is a threat for handempty G. Gini

POP algorithm

32

function POP(initial, goal, operators) returns plan Plan Sneed to Links( plan) add Sadd < Sneed to Orderings( plan) if Sadd is a newly added step from operators then add Sadd to Steps( plan) add StartSj in Links( plan) do choose either Demotion: Add Sthreat