Review •
Languages and Grammars – Alphabets, strings, languages
•
CS 301 - Lecture 23 Recursive and Recursively Enumerable Languages
Regular Languages – – – – –
•
Context Free Languages – – – – – – – – –
Fall 2008 •
Deterministic Finite and Nondeterministic Automata Equivalence of NFA and DFA Regular Expressions Regular Grammars Properties of Regular Languages Languages that are not regular and the pumping lemma Context Free Grammars Derivations: leftmost, rightmost and derivation trees Parsing and ambiguity Simplifications and Normal Forms Nondeterministic Pushdown Automata Pushdown Automata and Context Free Grammars Deterministic Pushdown Automata Pumping Lemma for context free grammars Properties of Context Free Grammars
Turing Machines – – – – –
Definition, Accepting Languages, and Computing Functions Combining Turing Machines and Turing’s Thesis Turing Machine Variations Universal Turing Machine and Linear Bounded Automata Today: Recursive and Recursively Enumerable Languages
Definition:
Recursively Enumerable and Recursive Languages
A language is recursively enumerable if some Turing machine accepts it
1
Let
L be a recursively enumerable language
and
M the Turing Machine that accepts it
For string
w:
if
w∈ L
then
M halts in a final state
if
w∉ L
then
M halts in a non-final state or loops forever
Let
L be a recursive language
and
M the Turing Machine that accepts it
For string
Definition: A language is recursive if some Turing machine accepts it and halts on any input string
In other words: A language is recursive if there is a membership algorithm for it
Turing acceptable languages and Enumeration Procedures
w:
if
w∈ L
then
M halts in a final state
if
w∉ L
then
M halts in a non-final state
2
We will prove:
Theorem:
(weak result)
• If a language is recursive then there is an enumeration procedure for it
if a language L is recursive then there is an enumeration procedure for it
(strong result)
• A language is recursively enumerable if and only if there is an enumeration procedure for it
Proof:
If the alphabet is {a, b} then ~ M can enumerate strings as follows:
Enumeration Machine
~ M
Enumerates all strings of input alphabet
M
Accepts
L
a b aa ab ba bb aaa aab ......
3
Enumeration procedure
Example:
~ M
Repeat:
~ M generates a string w M checks if w∈ L YES: print NO:
ignore
w to output w
End of Proof
Theorem:
L = {b, ab, bb, aaa,....} L(M )
a b aa ab ba bb aaa aab ......
Enumeration Output
b
b
ab
ab
bb aaa
bb aaa
......
......
Proof:
if language L is recursively enumerable then there is an enumeration procedure for it
Enumeration Machine
~ M
Enumerates all strings of input alphabet
M
Accepts L
4
If the alphabet is {a, b} then ~ M can enumerate strings as follows:
a b aa ab ba bb aaa aab
NAIVE APPROACH Enumeration procedure Repeat:
~ M generates a string w M checks if w∈ L YES: print NO:
Problem: If w∉ L machine M
w to output
ignore
w
may loop forever
BETTER APPROACH
~ M Generates third string w3
~ M Generates first string w1
M executes first step on w1
M executes first step on w3
~ M Generates second string w2
M executes first step on w2 second step on
w1
second step on
w2
third step on
w1
And so on............
5
Step in string
w1
w2
w3
w4
1
1
1
1
2
2
2
2
3
3
3
3
If for any string wi machine M halts in a final state then it prints wi on the output
End of Proof
Theorem: If for language L there is an enumeration procedure then L is recursively enumerable
Proof:
Input Tape
w Machine that accepts L Enumerator for L
Compare
6
Turing machine that accepts For input string
L
We have proven:
w
A language is recursively enumerable if and only if there is an enumeration procedure for it
Repeat: • Using the enumerator, generate the next string of
L
• Compare generated string with
w
If same, accept and exit loop End of Proof
We will prove: 1. There is a specific language which is not recursively enumerable (not accepted by any Turing Machine)
2. There is a specific language which is recursively enumerable but not recursive
Non Recursively Enumerable
Recursively Enumerable
Recursive
7
Some Languages Are Not Recursively Enumerable
Definition:
A set is uncountable if it is not countable
Proof Using Uncountable Sets
Theorem: Let
Proof:
S be an infinite countable set
The powerset
2 S of S is uncountable
Since
S is countable, we can write S = {s1, s2 , s3 ,…} Elements of
S
8
Elements of the powerset have the form:
{s1, s3} {s5 , s7 , s9 , s10 } ……
Let’s assume (for contradiction) that the powerset is countable.
Then:
we can enumerate the elements of the powerset
We encode each element of the power set with a binary string of 0’s and 1’s Encoding
Powerset element
s1
s2
s3
s4
{s1}
1
0
0
0
{s2 , s 3 }
0
1
1
0
{s1, s 3 , s4 }
1
0
1
1
Powerset element
Encoding
t1
1
0
0
0
0
t2
1
1
0
0
0
t3
1
1
0
1
0
t4
1
1
0
0
1
9
Take the powerset element whose bits are the complements in the diagonal
t1
1
0
0
0
0
t2
1
1
0
0
0
t3
1
1
0
1
0
t4
1
1
0
0
1
New element:
0011…
(birary complement of diagonal)
The new element must be some of the powerset
ti
Since we have a contradiction: The powerset
2 S of S is uncountable
However, that’s impossible: from definition of
ti
the i-th bit of ti must be the complement of itself Contradiction!!!
10
An Application: Languages Example Alphabet :
{a, b}
Example Alphabet :
The set of all Strings:
{a, b}
The set of all Strings:
S = {a, b}* = {λ , a, b, aa, ab, ba, bb, aaa, aab,…}
S = {a, b}* = {λ , a, b, aa, ab, ba, bb, aaa, aab,…}
infinite and countable
infinite and countable A language is a subset of
S:
L = {aa, ab, aab}
Languages: uncountable
Example Alphabet : {a, b} The set of all Strings:
S = {a, b}* = {λ , a, b, aa, ab, ba, bb, aaa, aab,…} infinite and countable The powerset of
L1
L2
L3
M1
M2
M3
Lk
?
Turing machines: countable
S contains all languages:
S
2 = {{λ},{a},{a, b}{aa, ab, aab},…} L1 L2
L3
L4
There are more languages than Turing Machines
uncountable
11
Conclusion: There are some languages not accepted by Turing Machines
(These languages cannot be described by algorithms)
Languages not accepted by Turing Machines
Lk Languages Accepted by Turing Machines
We want to find a language that is not Recursively Enumerable
A Language which is not Recursively Enumerable
This language is not accepted by any Turing Machine
12
Consider alphabet
{a}
Consider Turing Machines that accept languages over alphabet
a, aa, aaa, aaaa, …
Strings:
a1 a 2
a3
a4
{a}
They are countable:
M1, M 2 , M 3 , M 4 , …
Example language accepted by M i
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
L( M 3 )
0
1
1
1
L( M 4 )
0
0
0
1
L( M i ) = {aa, aaaa, aaaaaa}
L( M i ) = {a 2 , a 4 , a 6 } Alternative representation 1
a
a
L( M i ) 0
1
2
a
3
0
4
a
1
0
a
5
a
6
1
a
7
0
13
Consider the language
L = {a i : a i ∈ L( M i )}
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
L( M 3 )
0
1
1
1
L( M 4 )
0
0
0
1
L consists from the 1’s in the diagonal
Consider the language
L
L = {a i : a i ∈ L( M i )} L = {a i : a i ∉ L( M i )} L
consists of the 0’s in the diagonal
L = {a 3 , a 4 ,…}
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
L( M 3 )
0
1
1
1
L( M 4 )
0
0
0
1
L = {a1, a 2 ,…}
14
Theorem: Language
Proof:
L is not recursively enumerable
Assume for contradiction that
L is recursively enumerable
There must exist some machine that accepts L
Mk
L( M k ) = L
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
L( M 3 )
0
1
1
L( M 4 )
0
0
0
Question: M k = M1 ?
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
1
L( M 3 )
0
1
1
1
1
L( M 4 )
0
0
0
1
Answer:
M k ≠ M1
a1 ∈ L( M k ) a1 ∉ L( M1)
15
a1
a2
a3
a4
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
L( M 2 )
1
0
0
1
L( M 3 )
0
1
1
1
L( M 3 )
0
1
1
1
L( M 4 )
0
0
0
1
L( M 4 )
0
0
0
1
Answer:
Question: M k = M 2 ?
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
L( M 3 )
0
1
1
L( M 4 )
0
0
0
Question: M k = M 3 ?
a 2 ∈ L( M k )
Mk ≠ M2
a 2 ∉ L( M 2 )
a1
a2
a3
a4
L ( M1 )
0
1
0
1
L( M 2 )
1
0
0
1
1
L( M 3 )
0
1
1
1
1
L( M 4 )
0
0
0
1
Answer: M k ≠ M 3
a3 ∉ L( M k ) a3 ∈ L( M 3 )
16
Similarly:
Mk ≠ Mi
for any
Therefore, the machine
i
Because either:
a i ∈ L( M k ) a i ∉ L( M i )
or
M k cannot exist
Therefore, the language L is not recursively enumerable
a i ∉ L( M k ) a i ∈ L( M i )
End of Proof
Non Recursively Enumerable
Observation: There is no algorithm that describes
(otherwise L would be accepted by some Turing Machine)
L
L Recursively Enumerable
Recursive
17
What’s Next •
Read – Linz Chapter 1,2.1, 2.2, 2.3, (skip 2.4), 3, 4, 5, 6.1, 6.2, (skip 6.3), 7.1, 7.2, 7.3, (skip 7.4), 8, 9, 10, 11 – JFLAP Chapter 1, 2.1, (skip 2.2), 3, 4, 5, 6, 7, (skip 8), 9, (skip 10), 11
•
Next Lecture Topics From 11.1, 11.2, 11.3, and 11.4
•
Quiz 3 in Recitation on Wednesday 11/12
– More Recursive Languages, Unrestricted Grammars, Context Sensitive Grammars, and the Chomsky Hierarchy – Covers Linz 7.1, 7.2, 7.3, (skip 7.4), 8, and JFLAP 5,6,7 – Closed book, but you may bring one sheet of 8.5 x 11 inch paper with any notes you like. – Quiz will take the full hour •
Homework – Homework Due Thursday
18
