Top-Down Parsing Teoría de Autómatas y Lenguajes Formales

Apresentação em tema: "Top-Down Parsing Teoría de Autómatas y Lenguajes Formales"— Transcrição da apresentação:

1 Top-Down Parsing Teoría de Autómatas y Lenguajes Formales
M. Luisa González Díaz Universidad de Valladolid, 2006

2 Task Parsing (of course); but do it: Top-Down Easy and algorithmic
Efficiently Knowing (input) as little as possible Marking errors as soon as possible

3 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type ejemplo type array [ num ptpt num ] of char $ Lexical Analyzer

4 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type array array [ num ptpt num ] of char $ Lexical Analyzer

5 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type array [ [ num ptpt num ] of char $ Lexical Analyzer

6 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type num ptpt num array [ num num ptpt num ] of char $ Lexical Analyzer

7 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type num ptpt num array [ num ptpt ptpt num ] of char $ Lexical Analyzer

8 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type num ptpt num array [ num ptpt num Lexical Analyzer num ] of char $

9 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type num ptpt num array [ num ptpt num ] Lexical Analyzer ] of char $

10 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type num ptpt num array [ num ptpt num ] of Lexical Analyzer of char $

11 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type simple num ptpt num char array [ num ptpt num ] of char Lexical Analyzer char $

12 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type array [ simple ] of type num ptpt num simple char array [ num ptpt num ] of char Lexical Analyzer $

13 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type $ (no more) array [ simple ] of type num ptpt num simple char array [ num ptpt num ] of char $ Lex. An.

14 Code procedure type; if … then match(array); match (‘[‘); simple;
match(‘]‘); match(of); type else if … then match(‘^’); simple else if ... then simple else error

15 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type Bet guess 2 ^ ^ integer Lexical Analyzer

16 Example type → simple | ^ simple | array [simple] of type
simple → integer | char | num ptpt num type ^ simple ^ integer integer $ Lexical Analyzer

17 match procedure match (t: token); begin if lookahead = t then
looakahead := nexttoken (lexical analyzer) else error end

18 Solving ifs type → simple | ^ simple | array [simple] of type simple → integer | char | num ptpt num We said “that rule” because we know next input symbol (lookahead): type: array : ^ : rule 1: any other : error will be detected later integer, char, num : error detected now rule 3 rule 2

19 Predictive Parsing Table
PPT … terminal Non terminal rule PPT integer char num array ^ [ of ] $ type rule 1 rule 3 rule 2 simple rule 4 rule 5 rule 6

20 FIRST (α) := { a Є ΣT / α ═>* a β }
Generalizing type → simple | ^ simple | array [simple] of type simple → integer | char | num ptpt num Choose rule A → α when a (lookahead) can appear as first symbol derived from α FIRST (α) := { a Є ΣT / α ═>* a β } Still not complete! α β a A B γ a A α = δ β a A α =

21 Predictive Parsing Table
PPT … terminal Non terminal rule PPT … a A A → α A → α with α≠ε to PPT [ A, FIRST(α) ]

22 FIRST (α) := { a Є ΣT / α ═>* a β }
Still not complete! FIRST (α) := { a Є ΣT / α ═>* a β } FIRST (a) = { a } FIRST (A) = FIRST (α) FIRST (A α) = FIRST (A) A→α Є P U

23 E → T E’ FIRST E E’ T T’ F E’ → + T E’ | ab + a ( n * a ( n ( n T → F T’ T’ → * F T’ | a F → ( E ) | n First (F) = { (, n } First (T) = First (FT’) = First (F) First (E) = First (TE’) = First (T) First(E’) = First ( +TE’) U First (ab) = {+, a } First (T’) = First ( *FT’) U First (a) = { *, a }

24 ( n a * + ) $ E E’ T T’ F TE’ TE’ ab +TE’ FT’ FT’ a *FT’ (E) n
FIRST E E’ T T’ F E’ → + T E’ | ab ( n + a ( n * a ( n T → F T’ T’ → * F T’ | a F → ( E ) | n ( n a * + ) $ E E’ T T’ F TE’ TE’ A → α with α ≠ ε to T [ A, First (α) - {ε} ] ab +TE’ FT’ FT’ a *FT’ E’ → + T E’ T’ → * F T’ F → ( E ) E → T E’ E’ → ab T → FT’ F → n T’ → a (E) n

25 A bad example program → program id ; | program id ( par-list ) ;
Lexical Analyzer

26 A bad example: backtracking
program → program id ; | program id ( par-list ) ; program $ (no more) program id ; Ups! program id ( Lexical Analyzer

27 A bad example: backtracking
program → program id ; | program id ( par-list ) ; program $ (no more) program id ; program id ( Lexical Analyzer

28 A bad example: backtracking
program → program id ; | program id ( par-list ) ; program $ (no more) program id ( param-list ) ; Lexical Analyzer

29 Not so bad: factorising
program → program id R ; R → ( par-list ) | ε program $ (no more) program id R ; ε program id ; $ Lexical Analyzer

30 Another bad example E → E + n | n E E + n E + n E + n n n + n + n + n

31 Another bad example E → E + n | n E E E Lexical Analyzer $ + n + n n n

32 Not so bad either: eliminating left recursion
E → n E’ E’ → + n E’ | ε E → E + n | n E $ n E’ + n E’ + n E’ ε Lexical Analyzer n + n + n $

33 FIRST (α) := { a Є ΣT / α ═>* a β } { ε }
∩ A=>*ε

34 E → T E’ FIRST E E’ T T’ F E’ → + T E’ | ε + ε ( n * ε ( n ( n T → F T’ T’ → * F T’ | ε F → ( E ) | n First (F) = { (, n } First (T) = First (FT’) First (E) = First (TE’) First(E’) = {+, ε } First (T’) = { *, ε }

35 E → T E’ FIRST E E’ T T’ F E’ → + T E’ | ε ( n + ε ( n * ε ( n T → F T’ T’ → * F T’ | ε F → ( E ) | n First ( E’F ) = { + , (, n } First ( E’T’F ) = { + , * , (, n } First ( E’T’T’ ) = { + , * , ε }

36 FIRST (α) := { a Є ΣT / α ═>* a β } { ε }
∩ A=>*ε FIRST (a) = { a } FIRST (A) = FIRST (α) FIRST (A α) = (FIRST (A) – { ε }) FIRST (α) ε Є FIRST (X1X2 … Xp) iff ε Є FIRST (Xi) i A→α Є P U ∩ A=>*ε A

37 Remember program → program id R ; R → ( par-list ) | ε
$ (no more) program id R ; ε program id ; $ Lexical Analyzer

38 Remember program → program id R ; R → ( par-list ) | ε
$ (no more) program id R ; ERROR ε program id id Lexical Analyzer

39 “Marking errors as soon as possible”
Remember program → program id R ; R → ( par-list ) | ε “Marking errors as soon as possible” program $ (no more) program id R ; ERROR program id id Lexical Analyzer

40 ε - rules Choose rule A → ε when a (lookahead)
can appear following A in a sentential form FOLLOW (A) := { a Є ΣT / S ═>* α Aa β } { $ } ∩ S=>*αA … a A C α β S B A C α β S a … … B A C α a S ε a A C α β S ... … B A C α S $ …

41 Follow Righthand sides: α A β with β ≠ ε
add First(β)-{ε} to Follow (A) For every rule B → α A add Follow (B) to Follow (A) For every rule B → α A β with β ═> * ε Add $ to Follow (Start Symbol)

42 Follow: algorithm 0. Add $ to Follow (Start symbol)
Righthand sides: α A β with β ≠ ε add First(β)-{ε} to Follow (A) For every rule B → α A or B → α A β with β ═> * ε add Follow (B) to Follow (A)

43 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F 0) $ 0) Add $ to Follow (Start Symbol)

44 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε + T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ 0) +TE’ 1) T E’ α A β T E’ As before Righthand sides: α A β with β ≠ ε Add First(β)-{ε} to Follow (A)

45 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε * T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ + 0) * F T’ 1) F T’ α A β F T’ As before Righthand sides: α A β with β ≠ ε Add First(β)-{ε} to Follow (A)

46 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ + * 0) ) 1) ( E ) α A β ( E ) Righthand sides: α A β with β ≠ ε Add First(β)-{ε} to Follow (A)

47 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ + * 0) ) ) 1) 2) For every rule like B → α A or B → α A β with β ═> * ε Add Follow (B) to Follow (A) 2) E → T E’ E → T E’ B → α A E → T E’ B → α A β

48 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ + * 0) ) ) 1) 2) For every rule like B → α A or B → α A β with β ═> * ε Add Follow (B) to Follow (A) 2) E’ → + T E’ E’ → + T E’ B → α A E’ → + T E’ B → α A β

49 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ + * 0) ) ) 1) 2) Every rule B → α A or B → α A β with β ═> * ε Add Follow (B) to Follow (A) 2) T → F T’ T → F T’ B → α A T → F T’ B → α A β

50 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ + * 0) ) ) 1) 2) Every rule B → α A or B → α A β with β ═> * ε Add Follow (B) to Follow (A) 2) T’ → * F T’ T’ → * F T’ B → α A T’ → * F T’ B → α A β

51 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ ) + * $ ) 0) $ ) 1) 2) Every rule B → α A or B → α A β with β ═> * ε Add Follow (B) to Follow (A) 2)

52 E → T E’ FIRST E E’ T T’ F + ε * ( n E’ → + T E’ | ε T → F T’ T’ → * F T’ | ε F → ( E ) | n FOLLOW E E’ T T’ F $ ) $ ) + $ ) + $ ) * 0) + $ ) 1) 2) Every rule B → α A or B → α A β with β ═> * ε Add Follow (B) to Follow (A) 2)

53 Predictive Parsing Table Construction

54 ( n * + ) $ E E’ T T’ F TE’ TE’ FIRST E E’ T T’ F + ε * ( n FOLLOW E
A → α with α ≠ ε to T [ A, First (α) - {ε} ] ( n * + ) $ E E’ T T’ F TE’ TE’ E → T E’

55 ( n * + ) $ E TE’ E’ T T’ F +TE’ FIRST E E’ T T’ F + ε * ( n FOLLOW E
A → α with α ≠ ε to T [ A, First (α) - {ε} ] ( n * + ) $ E TE’ E’ T T’ F +TE’ E’ → + T E’

56 ( n * + ) $ E TE’ E’ +TE’ T T’ F FT’ FT’ FIRST E E’ T T’ F + ε * ( n
FOLLOW E E’ T T’ F $ ) + * A → α with α ≠ ε to T [ A, First (α) - {ε} ] ( n * + ) $ E TE’ E’ +TE’ T T’ F FT’ FT’ T → F T’

57 ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ F *FT’ FIRST E E’ T T’ F + ε * ( n
FOLLOW E E’ T T’ F $ ) + * A → α with α ≠ ε to T [ A, First (α) - {ε} ] ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ F *FT’ T’ → * F T’

58 ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ *FT’ F (E) FIRST E E’ T T’ F + ε *
FOLLOW E E’ T T’ F $ ) + * A → α with α ≠ ε to T [ A, First (α) - {ε} ] ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ *FT’ F (E) F → ( E )

59 ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ *FT’ F (E) n FIRST E E’ T T’ F + ε
FOLLOW E E’ T T’ F $ ) + * A → α with α ≠ ε to T [ A, First (α) - {ε} ] ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ *FT’ F (E) n F → n

60 ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ *FT’ F (E) ε ε FIRST E E’ T T’ F +
FOLLOW E E’ T T’ F $ ) + * A → α with α ≠ ε to T [ A, First (α) - {ε} ] 2) A → ε or A → α with α ═> * ε to T [ A, Follow (A) ] ( n * + ) $ E TE’ E’ +TE’ T FT’ T’ *FT’ F (E) ε ε E → ε

61 ( n * + ) $ E TE’ E’ +TE’ ε T FT’ T’ *FT’ F (E) ε ε ε FIRST E E’ T T’
FOLLOW E E’ T T’ F $ ) + * A → α with α ≠ ε to T [ A, First (α) - {ε} ] 2) A → ε or A → α with α ═> * ε to T [ A, Follow (A) ] ( n * + ) $ E TE’ E’ +TE’ ε T FT’ T’ *FT’ F (E) ε ε ε T’ → ε

62 ( n * + ) $ E TE’ E’ +TE’ ε T FT’ T’ *FT’ F (E) Procedure E;
If lookahead in {‘(‘, n} then T ; E’ else error Procedure E’; If lookahead =‘+’ then match (‘+’); T; E’ else if lookahead in {‘)’, $} then nothing else error ( n * + ) $ E TE’ E’ +TE’ ε T FT’ T’ *FT’ F (E) Procedure F; If lookahead = ‘(‘ then match (‘(‘) ; E; match (‘)’) else if lookahead = n then match (n) else error

63 Recursive Descendent Predictive Parsing Example
$ Recursive Descendent Predictive Parsing Example T E’ F T’ + T E’ n ε ( n * + ) $ E TE’ E’ +TE’ ε T FT’ T’ *FT’ F (E) F T’ ε n * F T’ n ε n + n * n $ L A

64 Predictive Parsing Example non recursive
stack input output $E n+n*n$ E → TE’ Predictive Parsing Example non recursive $E’T n+n*n$ T → FT’ $E’T’F n+n*n$ F → n $E’T’n n+n*n$ $E’T’ +n*n$ T’ → ε ( n * + ) $ E TE’ E’ +TE’ ε T FT’ T’ *FT’ F (E) $E’ +n*n$ E’ → +TE’ $E’T+ +n*n$ $E’T n*n$ T → FT’ $E’T’F n*n$ F → n $E’T’n n*n$ $E’T’ *n$ T → *FT’ $E’T’F* *n$ $E’T’F n$ F → n $E’T’n n$ $E’T’ $ T’ → ε $E’ $ E’ → ε $ $ OK

65 Predictive Parsing Example non recursive
output E → TE’ T → FT’ F → n T’ → ε E’ → +TE’ T → *FT’ E’ → ε OK Predictive Parsing Example non recursive

66 L L (1) Predictive Parsing Example non recursive E $ T E’ F T’ + T E’
output Predictive Parsing Example non recursive E $ E → TE’ T E’ T → FT’ F → n F T’ + T E’ T’ → ε L L (1) n ε E’ → +TE’ Input from the left Leftmost derivation 1 lookahead input symbol F T’ ε T → FT’ F → n n * F T’ T → *FT’ n ε F → n T’ → ε E’ → ε OK

67 A definitely bad example
S → if c then S | if c then S else S | o S S if c then S if c then S else S if c then S else S if c then S o o o o if c then if c then o else o

68 A definitely bad example
S → if c then S | if c then S else S | o if c then if c then o else o No Yes headache Yes pill study have it No ( ) if c then else o

69 A definitely bad example
S → if c then S | if c then S else S | o if c then if c then o else o No Yes headache No Yes pill study have it ( ) if c then else o

70 A definitely bad example: ambiguous grammar
1:S → if c then S R 2:S → o 3:R → else S 4:R → ε if o else $ S R 1 2 FIRST S R else ε if o 3 4 4 Not LL(1) FOLLOW S R $ else $ else

71 Another definitely bad example
S → P d | Q e P → a P c | b Q → a Q d | b S P d b c a S Q e b d a an b cn d an b dn e

72 Another definitely bad example
S → P d | Q e P → a P c | b Q → a Q d | b S P d b c a S Q e b d a a a … b Lexical Analyzer

73 Another definitely bad example
S → P d | Q e P → a P c | b Q → a Q d | b First (P) = { a, b } PPT a b c d e $ S P aPc Q aQd Pd Pd First (Q) = { a, b } Qe Qe First (Pd) = { a, b } First (Qe) = { a, b } Not LL(1)

74 LL(1) Grammars G ambiguous => not LL(1)
G left recursive => not LL(1) G not left factorised => not LL(1) Grammars not left fact. ambiguous left recursive LL(1)

75 Predictive parsing The good case Only one symbol to decide :
LL(1) grammars The good case No left recursion No left factor problems No ambiguity And more

76 A complete example S → AB | bDBE A → aSe E → e B → bBd | UV U → u | ε
V → k | ε D → aD | k

77 a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd
FIRST S A E U V B D a b a e b u ε k ε a k u k ε S → AB | bDBE A → aSe E → e B → bBd | UV U → u | ε V → k | ε D → aD | k a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd UV UV E → e V → k D → aD D → k U → u S → bDBE S → AB A → aSe B → UV B → bBd u k aD k

78 a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd
FIRST S A E U V B D FOLLOW S A E B U V D a b a e b u ε k ε a k $ b u k e k b u k e u e d k ε S → AB | bDBE A → aSe E → e B → bBd | UV U → u | ε V → k | ε D → aD | k a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd UV UV B → bBd S → AB E → e D → k A → aSe D → aD B → UV U → u V → k S → bDBE u k aD k

79 a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd
FIRST S A E U V B D FOLLOW S A E B U V D a b a e b u ε k ε a k $ b u k e k b u k e u e d k ε S → AB | bDBE A → aSe E → e B → bBd | UV U → u | ε V → k | ε D → aD | k a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd UV UV B → bBd S → AB E → e D → k A → aSe D → aD B → UV U → u V → k S → bDBE u k aD k

80 a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd
FIRST S A E U V B D FOLLOW S A E B U V D a b a e b u ε k ε a k $ $ $ e $ e b u k e d $ e d $ e k b u k e u e d k ε S → AB | bDBE A → aSe E → e B → bBd | UV U → u | ε V → k | ε D → aD | k a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd UV UV u k aD k

81 LL(1) a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe
FIRST S A E U V B D FOLLOW S A E B U V D a b a e b u ε k ε a k $ e b u k $ e $ e e d $ k e d $ e d $ b u k e u k ε S → AB | bDBE A → aSe E → e B → bBd | UV U → u | ε V → k | ε D → aD | k a b d e u k $ S AB bDBE A aSe E B bBd UV U ε V D aD AB bDBE aSe e bBd UV UV UV UV UV U → ε B → UV V → ε ε ε u ε ε LL(1) ε ε ε k aD k