Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike.

Apresentação em tema: "Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike."— Transcrição da apresentação:

1 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Capítulo 9 Heurísticas em Planejamento Leliane Nunes de Barros MAC5788 IME-USP 2005 Planejamento em Inteligência Artificial

2 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Planejamento como Busca Não-determinística: uma visão conceitual O nó u representa um conjunto de planos soluções  u : conjunto de todas as soluções alcançáveis de u.

3 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Planejadores como instâncias de Abstract-search (I) l Planejamento no espaço de estados: u é uma seqüência de ações. Toda solução alcançável apartir de u contém essa seqüência como prefixo ou sufixo. l Planejamento no espaço de planos: u é um conjunto de ações com vínculos causais, restrições de ordenação e restrições de unificações. Toda solução alcançável apartir de u contém todas as ações e satisfazem as restrições l Algoritmo Graphplan: u é um sub-grafo de um grafo de planejamento, isto é, uma seqüência de conjuntos de ações junto com restrições de pré-condições, efeitos e exclusões mútuas. Toda solução alcançável apartir de u contém as ações em u correspondente aos níveis já resolvidos (da busca no grafo) e pelo menos uma ação de cada nível qua ainda não foi resolvido em u.

4 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Planejadores como instâncias de Abstract-search (II) l Planejamento basead em SAT: u é um conjunto de literais valorados e o restante das cláusulas, cada uma sendo uma disjunção de literais que descrevem ações e estados. Toda solução alcançável apartir de u corresponde a uma atribuição de valores verdade ou falso aos literais não valorados tal que todas as cláusulas restantes sejam satisfeitas. l Planejamento baseado em CSP: u é um conjunto de variáveis CSP e restrições com algumas variáveis com valores já atribuidos. Toda solução alcançável apartir de u inclue essas variáveis valoradas e atribuições para as outras variáveis CSP que satisfazerm as restrições.

5 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Planejadores como instâncias de Abstract-search (III) l Planejamento no espaço de estados: u é um plano parcial l Planejamento no espaço de planos: u é um plano parcial l Algoritmo Graphplan: nem todas as ações em u farão parte do plano solução l Planejamento basead em SAT: nem todas as ações em u farão parte do plano solução l Planejamento baseado em CSP: nem todas as ações em u farão parte do plano solução

6 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Planejadores como instâncias de Abstract-search (III) l Planejamento no espaço de estados: u é um plano parcial l Planejamento no espaço de planos: u é um plano parcial l Algoritmo Graphplan: nem todas as ações em u farão parte do plano solução l Planejamento basead em SAT: nem todas as ações em u farão parte do plano solução l Planejamento baseado em CSP: nem todas as ações em u farão parte do plano solução Disjunctive refinement approaches:

7 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Planejadores como instâncias de Abstract-search (IV) l Refine: modifica a coleção de ações e/ou restrições associadas com um nó u l Branch: gera um ou mais filhos de u que serão os nós candidatos para ser o próximo nó visitado. Cada filho v representa um sub-conjunto de soluções  v   u l Prune: remove do conjunto de nós candidatos {u 1, u 2,..., u k } alguns nós que parecem ser não promissores para a busca (por exemplo, um nó já visitado que não levou a solução) l Planejadores podem: »Variar a ordem em que esses 3 procedimentos são executados »Usar diferentes mecanismos de controle, como IDS ou A* l Como os planejadores que estudamos realizam esses procedimentos (páginas 195 à 197)

8 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Tornando Abstract-search Determinística

9 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Heurística de seleção de nós: resolve o problema da escolha não-determinística l Suponha que para cada estado s, temos uma estimativa h(s) da distância de s para uma solução u h(s) é usada para decidir qual será o próximo nó a ser expandido u quanto mais informativa for h(s) menor será o número de escolhas erradas u h(s) deve ainda ser facilmente computável »Heurísticas são geralmente baseadas mo princípio de relaxação de problemas: para avaliar o quão desejável é um nó consideramos um problema mais simples obtido apartir do original fazendo suposições de simplicação e relaxando restrições

10 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Heurística de seleção de nós l Suppose we’re searching a tree in which each edge (s,s') has a cost c(s,s') u If p is a path, let c(p) = sum of the edge costs u For classical planning, this is the length of p l For every state s, let u g(s) = cost of the path from s 0 to s u h*(s) = least cost of all paths from s to goal nodes u f*(s) = g(s) + h*(s) = least cost of all paths u from s0 to goal nodes that go through s l Suppose h(s) is an estimate of h*(s) u Let f(s) = g(s) + h(s) » f(s) is an estimate of f*(s) u h is admissible if for every state s, 0 ≤ h(s) ≤ h*(s) u If h is admissible then f is a lower bound on f*

11 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ The A* Algorithm l A* on trees: loop choose the leaf node s such that f(s) is smallest if s is a solution then return it and exit expand it (generate its children) l On graphs, A* is more complicated: additional machinery to deal with multiple paths to the same node l If a solution exists (and certain other conditions are satisfied), then: u If h(s) is admissible, then A* is guaranteed to find an optimal solution u The more “informative” the heuristic is (i.e., the closer it is to h*), the smaller the number of nodes A* expands u If h(s) is within c of being admissible, then A* is guaranteed to find a solution that’s within c of optimal

12 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Heuristic Functions for Planning l  *(s,p): minimum distance from state s to a state containing p l  *(s,s’): minimum distance from state s to a state containing every p in s’ l  i (s,p) and  i (s,s’), where i = 0, 1, 2, … u estimates of  *(s,p) and  *(s,s’) and p  s, l h(s) =  0 (s,g), where g is the goal  0 ignora os efeitos negativos das ações

13 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Heurística da relaxação de independência (polinomial no número de proposições e ações) l Given s, can compute  0 (s,p), for every proposition p: l From this, can compute h(s) =  0 (s,g) =  p  g  0 (s,p)

14 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Heuristic Forward Search l Note: this is depth-first search; thus admissibility is irrelevant

15 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Heuristic Backward Search l Same comment as on the previous slide

16 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ and p  s, l h(s) =  0 (s,g) is an inadmissible heuristic l  1 : like  0 except that  1 (s,g) = max p  g  0 (s,p) u This heuristic is admissible; thus it could be used with A* u It is not very informative

17 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ A More Informed Heuristic l Instead of computing the maximum distance to each p in g, compute the maximum distance to each pair {p,q} in g:

18 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ More Generally, …

19 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Complexity of Computing the Heuristic l Takes time  (n k ). l If k = max(|g|, max{|precond(a)| : a is an action}) then computing  (s,g) is as hard as solving the entire planning problem

20 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Get Heuristic Values from a Planning Graph l The planning graph provides not only the distance estimates from s 0 to every reachable proposition p but also the mutex relations  P i u The Solution extraction procedure selects a set of propositions g in a layer only if no pair of elements in g is mutex. Theorem: if a pair of elements in not mutex in a layer, then it remains nonmutex in the following layers  this is very close to  2 (s,g)

21 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Get Heuristic Values from a Planning Graph l Recall how GraphPlan works: loop Graph expansion: extend a “planning graph” forward from the initial state until we have achieved a necessary (but insufficient) condition for plan existence Solution extraction: search backward from the goal, looking for a correct plan if we find one, then return it repeat this takes polynomial time this takes exponential time

22 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Using the Planning Graph to Compute h(s) l In the graph, there are alternating layers of ground literals and actions l The number of “action” layers is a lower bound on the number of actions in the plan l Construct a planning graph, starting at s l  g (s,p) = level of the first layer that “possibly achieves” p l  g (s,g) is very close to  2 (s,g) u  2 (s,g) counts each action individually u  g (s,g) groups together the independent actions in a layer

23 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ The FastForward Planner l Use a heuristic function similar to h(s) =  g (s,g) u Some ways to improve it (I’ll skip the details) l Don’t want an A*-style search (takes too much memory) l Instead, use a greedy procedure: until we have a solution, do expand the current state s s := the child of s for which h(s) is smallest (i.e., the child we think is closest to a solution) l There are some ways to improve this (I’ll skip the details) l Can’t guarantee how fast it will find a solution, or how good a solution it will find u However, it works pretty well on many problems

24 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ FastForward in the AIPS-2000 Planning Competition FastForward did quite well l In the AIPS-2000 competition, all of the planning problems were classical problems l Two tracks: u Fully automated planners  FastForward participated in the fully automated track »It got an “outstanding performance” award u Large variance in how close its plans were to optimal »However, it found them very fast compared with the other automated classical planners

25 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ FastForward in the AIPS-2002 Planning Competition Among the automated planners, FastForward was roughly in the middle LPG (graphplan + local search) did much better, and got a “distinguished performance of the first order” award One of thing that caused difficulty for FastForward u The problems in the AIPS-2002 went beyond classical planning »Numbers, optimization, time durations l Example: u A domain inspired by the Hubble space telescope (a lot simpler than the real domain, of course) »A satellite needs to take observations of stars »Gather as much data as possible before running out of fuel u Any amount of data gathered is a solution »Thus, FastForward always returned the null plan

26 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Heuristics for Plan-Space Planning l How to select the next flaw to work on?

27 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ One Possible Heuristic l Fewest Alternatives First (FAF)

28 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Do Others Work Better? l Sometimes yes, sometimes no l Limits to how good any flaw-selection heuristic can do

29 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ … l The search space is an AND/OR tree l Deciding what flaw to work on next = serializing this tree (turning it into a state-space tree) u at each AND branch, choose a child to expand next, and delay expanding the other children Partial plan p 1 Partial plan p n Partial plan p Goal g 2 Goal g 1 Operator o n Operator o 1 Variable constraint Goal gVariable constraint Order tasks Goal gOrder tasks Variable constraint Goal ordering Serializing and AND/OR Tree …… … …… 2 …… 2 Goal g 1 Operator o n Operator o 1

30 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ One Serialization

31 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Another Serialization

32 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Why Does This Matter? l Different refinement strategies produce different serializations u the search spaces have different numbers of nodes l In the worst case, the planner will search the entire serialized search space l The smaller the serialization, the more likely that the planner will be efficient l One pretty good heuristic: fewest alternatives first (FAF heuristic) => choose the flaw having the smallest number of resolvers. This will limit the cost of eventual backtracks. Takes O(n) where n is the number of flaws in a partial plan.

33 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ How Much Difference Can the Refinement Strategy Make? l Case study: build an AND/OR graph from repeated occurrences of this pattern: l Example: u number of levels k = 3 u branching factor b = 2 l Analysis:  Total number of nodes in the AND/OR graph is n =  (b k ) u How many nodes in the best and worst serializations?

34 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Case Study, Continued The best serialization contains  (b 2 k ) nodes The worst serialization contains  (2 k b 2 k ) nodes u The size differs by an exponential factor, but the best serialization still is exponentially large u To do better, need good node selection, branching, pruning

35 Leliane Nunes de Barro. Slides extraidos de Lecture slides for Automated Planning by Dana Nau. Licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License: http://creativecommons.org/licenses/by-nc-sa/2.0/ Resolver selection heuristics l Let  be the set of all plans generated with all the possible reasolvers. Select a plan  that has the smallest set of sug-goal whithout causal links (not very informed heuristic). l A more informative heuristic: construct an AND/OR graph along regression steps defined by  -1 down to some fixed level k l Select the plan  that minimizes the wheighted sum of: 1. The number of actions in this graph that are not in  and 2. The number of subgoals remaining in its leaves that are not in the initial state