CS504-Theory of Computation Lecture 4: Context-Free Languages
Waheed Noor Computer Science and Information Technology, University of Balochistan, Quetta, Pakistan
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
1 / 37
Outline
1
Language Generators
2
Context-free Grammar
3
Ambiguity
4
Pushdown Automata
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
2 / 37
Outline
1
Language Generators
2
Context-free Grammar
3
Ambiguity
4
Pushdown Automata
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
3 / 37
Language Generators In the previous few sessions, we have seen regular languages and their properties, specifically from the perspective of regular expressions. Now, we will learn about more important and powerful class of languages, called context-free languages. In both cases, important for us is the theory of generators of formal (artificial) languages. This theory has practical value in the specification and analysis of computer languages. This theory complements the study of automata, which are language recognizers.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
4 / 37
Language Generators Consider the regular expression a(a∗ |b∗ )b, can you describe? Basically, a grammar consists of collection of rules that are used to generate strings.
Example Every string in this language starts with symbol a, followed by a middle part generated by (a∗ |b∗ ) and ends with symbol b. Now, to be formal, let S be a new symbol representing strings generated by this language, and M representing the middle part, we can write a rule: S → aMb, where the symbol → can be read as “can be”. Now consider M, which is a string of a’s or a string of b’s; M → A and M → B, where A and B are two new symbols. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
5 / 37
Language Generators Example (Cont.) Now, what can be a string of a’s; it can be an empty string; A → Or it can be a string with leading a, followed by a string of a’s; A → aA Similarly, B → and B → bB. Now see if both language generators can generate the language: aaab. For the set of rules, we just produced, we start with the symbol S, which is actually our starting symbol. The set of rules and operation of generating strings, we just saw above is called context-free grammar. The context-free grammar is a language generator and it is context free since when replacement takes place of a symbol with the right side of the rule, the context of the strings are ignored. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
6 / 37
Outline
1
Language Generators
2
Context-free Grammar
3
Ambiguity
4
Pushdown Automata
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
7 / 37
Context-free Grammar I Informally
A context-free grammar consists of terminals, nonterminals, a start symbol, and finite set of productions (rules) for specifying the syntax of language. Terminals: These are basic symbols forming strings and they . In the above example, a and b are terminals. Nonterminals: These are grammar variables denoting set of strings. Nonterminals are helpful to define language generated by the grammar. Start symbol: One of the nonterminals is designated as start symbol, and the set of strings it denotes is the language generated by the grammar. Normally, productions/rules for the start symbol are listed first.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
8 / 37
Context-free Grammar II Informally
Productions: Specify the way in which we can combine terminals and nonterminals to form strings, where each production consist of 1 A nonterminal on the left side of the production called head or left side. 2 The symbol →, sometimes ::=. 3 The body or right side consisting of zero or more terminals and nonterminals, which describe a way of constructing strings from the non terminal at the head.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
9 / 37
Context-free Grammar III Informally
Given these definitions, we will actually have finite set of symbols of a grammar at the first place, which consists of terminals and nonterminals. We may also have an epsilon production (null rule) in the grammar, which will take the form A → .
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
10 / 37
Context-free Grammar (CFG) I Formally
Definition A context-free grammar G is a quadruple (V , Σ, R, S), where V is an alphabet of the grammar or set of grammar symbols, Σ ⊆ V is the set of terminal symbols, S ∈ (V − Σ) is the start symbol, and R ⊆ (V − Σ) × V ∗ is finite set of rules (productions). The members of (V − Σ) are nonterminals of G.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
11 / 37
Context-free Grammar (CFG) II Formally
For any A ∈ V − Σ and u ∈ V ∗ , we write A →G u whenever (A, u) ∈ R.
Definition (Derivation) Let A → u be a production rule in G, and let αAβ is some sequence of grammar symbols, where obviously A is a nonterminal. Then we can write a derivation step αAβ ⇒G αuβ When we have sequence of the form α1 ⇒G α2 ⇒G · · · ⇒G αn that rewrites α1 to αn then it is called derivation in G of αn from α1 . In other words, we can say that α1 derives αn in G, in some steps. Often we want to say that something derives something in zero or ∗ more steps, which can be denoted as ⇒G .
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
12 / 37
Context-free Grammar (CFG) III Formally
∗
That is, α ⇒G α for any string α, ∗
∗
and if α ⇒G β and β → γ, then α ⇒G γ. ∗
Thus, the relation ⇒G is reflexive and transitive closure of ⇒G . ∗
The language L(G) generated by G is {w ∈ Σ∗ : S ⇒G w}. The language L, if L = L(G), is called context-free language.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
13 / 37
Example I
Let the CFG G = (V , Σ, R, S), where V = {S, a, b}, Σ = {a, b}, and R consists of the rules S → aSb S→ Can we derive, aabb and aaabbbb from the above CFG, G? What language G generates? Hence, some context-free languages are not regular. In fact, every regular language is context-free, however every context-free is not regular.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
14 / 37
Example II
Let the CFG G = (V , Σ, R, S), where V = {S, (, )}, Σ = {(, )}, and R consists of the rules S→ S → SS S → (S) Can we derive, ()(()) and (()()) from the above CFG, G? What language G generates?
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
15 / 37
Example III One of the most important utility of CFG is its usage in defining syntax of programming languages, where it is important to identify syntactically correct computer programs. Let the CFG G = (V , Σ, R, E), where V = {(, ), +, −, ∗, id, T , F , E}, Σ = {(, ), +, −, ∗, id}, and R consists of the rules E T F F
→ E + T | E − T | T, → T ∗ F | F, → (E), → id Can we derive, id + id − id, (id + id) ∗ (id + id).
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
16 / 37
Sentential Form and Sentences ∗
Let S be a start symbol of a grammar G, and if S ⇒ α.
Definition (Sentential Form) If α contains terminals, nonterminals or empty, then we say it is a sentential form of G.
Definition (Sentence) If a sentential form α of G contains only terminals then we say it is a sentence of G and set of sentences generated by a grammar G is called language, denoted by L(G).
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
17 / 37
Regular Languages and Context-free Languages Theorem All regular languages are context-free. We have already seen an example of constructing context-free grammar from regular expression, here we show it formally again by construction.
Proof. Let all regular languages are accepted by the deterministic finite automaton M = (Q, Σ, δ, s, F ), The same language is generated by a context-free grammar G, i.e., G(M) = (V , Σ, R, S), where V = Q ∪ Σ, S = s and R = {q → ap : δ(q, a) = p} ∪ {q → : q ∈ F } Furthermore, simple languages ∅ and {a} are definitely context-free with grammar with no rules and only rule S → a, respectively. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
18 / 37
Regular Languages and Context-free Languages Example
Now we consider following DFA M and apply the construction of the proof in order to construct the CFG G
Class Activity Now, you shall construct CFG for the language accepted by following NFA.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
19 / 37
Types of Derivations
Definition (Leftmost Derivations) The leftmost nonterminal in each sentential form is always selected for L replacement. We denote leftmost derivation by ⇒
Definition (Rightmost Derivations) The rightmost nonterminal in each sentential form is always selected R
for replacement. We denote leftmost derivation by ⇒ Now, repeat the class activity of previous slide using both types of derivations.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
20 / 37
Parse Trees and Derivations Definition Parse tree is a graphical representation of a derivation that shows the order in which productions have been applied. Each time a production is applied to replace a nonterminal; children of that node are created from the body of the production from left to right. The leaves of a parse tree contains terminals. The string a parse tree can derive (called yield) can be obtained by reading the leaves from left to right. There is many to one relationship between derivations and parse trees, i.e., we get the same parse tree even if we apply leftmost or rightmost derivations as far as the choice of production for a nonterminal is same. Therefore, we can use either leftmost or rightmost derivation to get a unique parse tree. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
21 / 37
Summary What we have learn about parse trees and derivations can be summarized in the following theorem.
Theorem Let G = (V , Σ, R, S) be a context-free grammar, and let A ∈ V − Σ, and w ∈ Σ∗ , then following statement are equivalent: ∗
1
A ⇒ w.
2
There is a parse tree with root A and yield w.
3
There is a leftmost derivation A ⇒ w.
4
There is a rightmost derivation A ⇒ w.
L∗
Waheed Noor (CS&IT, UoB, Quetta)
R∗
CS504-Theory of Computation
July 2014
22 / 37
Class Activity
Consider the grammar: S → SS + |SS ∗ |SS − |id and consider the string idid − id + id∗ Give a leftmost derivation Give a rightmost derivation Give a parse tree
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
23 / 37
Outline
1
Language Generators
2
Context-free Grammar
3
Ambiguity
4
Pushdown Automata
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
24 / 37
Ambiguity Definition (Ambiguous Grammar) A grammar is ambiguous, if it produces multiple parse trees for the same sentence. -ORA grammar that produces more than one leftmost or rightmost derivations for the same sentence. Consider: E → E + E | E ∗ E | (E) | id | num Looks similar to the one for arithmetic expression we have seen earlier, still there are important differences. Consider the string id + id ∗ id.
Example Following grammar for conditional statement is also ambiguous. stmt → if expr then stmt | if expr then stmt else stmt | other Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
25 / 37
Outline
1
Language Generators
2
Context-free Grammar
3
Ambiguity
4
Pushdown Automata
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
26 / 37
Pushdown Automata I
Question Can we simulate the rule A → aB by a finite automaton? If yes, how about the rule A → aBb? We now know that not every context-free language is regular; some are not regular. Therefore, every context-free language can not be recognized by a finite automaton. Definitely, we need a more powerful machine than finite automata to recognize context-free languages.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
27 / 37
Pushdown Automata II Can you think of features that we need to add to the finite automata such that they can accept context-free languages?
Example Consider the context-free language L = {ww R : w ∈ {a, b}∗ }, with rules S → aSa, S → bSb and S → .
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
28 / 37
Pushdown Automata III
We need a machine that can remember the input it has read so far. To achieve this, we need a storage where the machine can append the input, one symbol at a time, when it is read. Then it may deterministically or nondeterministically check when the center of the input has arrived, and remove one symbol at a time from the storage as it reads from the input. The machine accepts the string if it consumes the string and the storage is empty.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
29 / 37
Pushdown Automata IV
The storage for this purpose can be a sort of stack or a pushdown storage.
Example What features would be needed to recognize the language of balanced parenthesis?
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
30 / 37
Pushdown Autotmata: Definition Definition A pushdown automaton is a six-tuple M = (Q, Σ, Γ, ∆, s, F ), where Q is a finite set of states, Σ is an alphabet, the input symbols, Γ is an alphabet, the stack symbols, s ∈ Q is the start state, F ⊆ Q is the set of final states and ∆ ⊆ {(Q × (Σ ∪ {}) × Γ∗ ) × (Q × Γ∗ )} is the transition relation. A pair ((p, a, β), (q, γ)) ∈ ∆ is a transition, where M is in state p with β at the top of the stack and read from input the symbol a, then M replaces β by γ on the top of the stack, and enters state q. If a = then input is not read. If only β = then top of the stack is not checked and γ is pushed to the stack. If β = and γ = then stack is not consulted. If only γ = then β is popped from the stack. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
31 / 37
Pushdown Automata: Configurations I As in finite automata, the input already read have no effect on the computation of pushdown automaton.
Definition A configuration of a pushdown automaton is the member of Q × Σ∗ × Γ∗ , where first component is the state of the machine, second is the portion of the input yet to be read and third is the contents of stack (pushdown store). For example, a configuration (q, w, abc), means the machine is state q, w is the remaining input, and “abc” is the contents of the stack with “a” on the top and “c” at the bottom. (p, x, α) and (q, y , ζ) are configurations of M and if there is a transition ((p, a, β), (q, γ)) ∈ ∆ such that x = ay , α = βη, and ζ = γη or η ∈ Γ∗ . Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
32 / 37
Pushdown Automata: Configurations II
Denoted by (p, x, α) `M (q, y , ζ), i.e ,(p, x, α) yields in one step (q, y , ζ). Similarly, we denote reflexive and transitive closure of `M by `∗M . A string w ∈ Σ∗ is accepted by M if and only (s, w, e) `∗M (p, e, e) for p ∈ F . Finally, the language accepted by M is denoted by L(M).
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
33 / 37
Pushdown Automata: Example Example Design a pushdown automaton to accept the language L = {wcw R : w ∈ {a, b}∗ } Let M = (Q, Σ, Γ, ∆, s, F ), where Q = {s, f }, Σ = {a, b, c}, Γ = {a, b}, F = {f } and ∆ contains following rules 1
((s, a, ), (s, a))
2
((s, b, ), (s, b))
3
((s, c, ), (f , ))
4
((f , a, a), (f , ))
5
((f , b, b), (f , ))
Let’s simulate M on the input “abbcbba”. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
34 / 37
Pushdown Automata: Example Example Design a pushdown automaton to accept the language L = {w ∈ {a, b}∗ : number of a’s are equal to the number of b’s} Let M = (Q, Σ, Γ, ∆, s, F ), where Q = {s, q, f }, Σ = {a, b}, Γ = {a, b, c}, F = {f } and ∆ contains following rules 1 ((s, , ), (q, c)) 2 ((q, a, c), (q, ac)) 3 ((q, a, a), (q, aa)) 4 ((q, a, b), (q, )) 5 ((q, b, c), (q, bc)) 6 ((q, b, b), (q, bb)) 7 ((q, b, a), (q, )) 8 ((q, , c), (f , )) Let’s simulate M on the input “abaababb”. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
35 / 37
Pushdown Automata: Example
Class Activity Design a pushdown automaton M to accept the language L = {ww R : w ∈ {a, b}∗ }
Class Activity Design a pushdown automaton M to accept the language L = {an bn : n ≥ 0}
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
36 / 37
Pushdown Automata and CFG Theorem The class of languages accepted by pushdown automata is exactly the class of context-free languages.
Theorem The context-free languages are closed under union, concatenation, and Kleene star.
Theorem The intersection of a context-free language with a regular language is a context-free language.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
37 / 37
References I
Harry Lewis and Christos Papadimitriou. Elements of the Theory of Computation. Prentice Hall, 2nd edition, 1998. Michael Sipser. Introduction to the Theory of Computation. Thomson Course Technology, 2nd edition, 2006.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
July 2014
38 / 37
