CS504-Theory of Computation Lecture 1: Introduction Waheed Noor Computer Science and Information Technology, University of Balochistan, Quetta, Pakistan
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
1 / 57
Outline
1
Course Description
2
Overview
3
Preliminaries
4
Proof Techniques
5
Quiz 1
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
2 / 57
Outline
1
Course Description
2
Overview
3
Preliminaries
4
Proof Techniques
5
Quiz 1
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
3 / 57
Theory of Computation (3-0) Objective This course is mainly about abstractions of what we can compute. The treatment is mathematical, but the point of view is that of Computer Science. Roughly speaking, the theory of computation can be divided into three overlapping subareas such as formal languages and automata theory, computability theory and complexity theory. In this course: Fundamental capabilities and limitations of computers Mathematical properties of computer hardware and software Relevance to practice, for example, in the design of new programming languages, compilers, string searching, pattern matching, computer security, artificial intelligence, etc. Learn about problem solving through argue, prove, express and abstract Assignments & problem sets after each unit Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
4 / 57
Books & Resources
Books Michael Sipser, Introduction to Theory of Computation, 2nd Edition, Thomson Course Technology, 2006 [3] Harry Lewis and Christos Papadimitriou, Elements of the Theory of Computation, 2nd Edition, Prentice Hall, 1998 [1]
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
5 / 57
Origin
Origin Started in 1930’s by mathematicians and logicians in an attempt to understand computation Mainly, to find the answer that all mathematical problems can be solved in a systematic way These research led the computers we know and use today
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
6 / 57
Outline
1
Course Description
2
Overview
3
Preliminaries
4
Proof Techniques
5
Quiz 1
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
7 / 57
Overview
What can be computed? Consider a very simple model of a problem. A problem is a set of strings (often binary strings) that we wish to distinguish.
Example One problem might be the set of all binary strings divisible by three. To solve this problem we need to be able to say yes or no to each candidate binary string as to whether it is in the set or not. If we have an algorithm to do this, then we have solved the problem. These kinds of problems are called decidable.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
8 / 57
Main Topics
Finite Automata and Regular Languages Context-Free Languages Turing Machines Decidable and Undecidable Languages Complexity Theory
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
9 / 57
Subareas of Theory of Computation I Definition (Automata Theory) Automata theory deals with the definition, properties and power of different computation models such as finite automata, context-free grammars and Turing machines.
Example Finite Automata used for text processing, compiler design and hardware design. Context-free Grammer used in defining programming languages and in AI. Turing Machines used for abstraction of real computers.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
10 / 57
Subareas of Theory of Computation II Definition (Computability Theory) Deals with the theoretical models that classify problems as being solvable or unsolvable. These models played fundamental role in the development of real computers. In 1930, Godel, Turing and Church found that some of the mathematical problems are not solvable through computation.
Example An arbitrary mathematical statement is true or false Definitely we need to formally what is computer, algorithm and computation.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
11 / 57
Subareas of Theory of Computation III
Definition (Complexity Theory) Deals in classifying problems based on their level of difficulty with rigorous proofs. The level of difficulty corresponds to the computational efficiency of the solution.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
12 / 57
Subareas of Theory of Computation IV
Example Easy Problems (Efficiently solvable) Sorting a list of 1 million numbers Computing the fastest path from Quetta to Karachi Hard Problems Time table scheduling of all courses in a large university Factoring a large integer number (say 250 digits) into its prime factors
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
13 / 57
Outline
1
Course Description
2
Overview
3
Preliminaries
4
Proof Techniques
5
Quiz 1
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
14 / 57
Preliminaries: Mathematical Concepts I
Set: A collection of well defined objects Natural Numbers: A set N = {1, 2, 3, . . .} Integer: A set Z = {. . . , −2, −1, 0, 1, 2, . . .} Rational Numbers: A set Q = { m n : m ∈ Z, n ∈ Z, n 6= 0} Real Numbers: A set R = {x:x is a real number} Empty Set: Denote by φ = {} Subset: If A and B are two sets then A is a subset of B, A ⊆ B if every element of A is also an element of B. φ is the subset of every set.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
15 / 57
Preliminaries: Mathematical Concepts II Power set: if B is a set then power set of B is the set of all subsets of B, P(B) = {A : A ⊆ B}
Note φ ∈ P(B) and B ∈ P(B) Union: The union of two sets A and B is A ∪ B = {x : x ∈ A or x ∈ B} Intersection: The intersection of two sets A and B is A ∩ B = {x : x ∈ A and x ∈ B} Difference: The difference of two sets A and B is A − B or A\B = {x : x ∈ A and x ∈ / B} Cartesian Product: The product of two sets A and B is A × B = {(x, y ) : x ∈ A and y ∈ B} Complement: The complement of a set A is A = {x : x ∈ / A}
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
16 / 57
Preliminaries: Mathematical Concepts III Binary Relation: The binary relation of two sets A and B is R such that R ⊆A×B Function: A function f from set A to B is f : A 7→ B is a binary relation, R, that contains an ordered pair for each element a ∈ A as first element of the pair.
Note Function f is also referred as f (a) = b or f maps a to b or image of a is b under f . The set A is called the domain of f . The set {b ∈ B : there is an a ∈ A with f (a) = b} is called the range of f.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
17 / 57
Preliminaries: Mathematical Concepts IV
Injective Function: or one-to-one, if for any distinct element a ∈ A and a 0 ∈ A, we have f (a) 6= f (a 0 ). Surjective Function: or onto, if for each element b ∈ B there exist a ∈ A such that f (a) = b, i.e., the range of f is the set B.
Bijective Function If f is both injective and surjective.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
18 / 57
Preliminaries: Mathematical Concepts V
Equivalence Relation: A binary relation R ⊆ A × A is equivalence relation if and only if R is reflexive, symmetric, and transitive.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
19 / 57
Preliminaries: Mathematical Concepts VI
Conditions Reflexive: ∀a ∈ A, we have (a, a) ∈ R. Symmetric: ∀a, b ∈ A, if (a, b) ∈ R then (b, a) ∈ R. Transitive: ∀a, b, cinA, if (a, b) ∈ R) and (b, c) ∈ R then (a, c) ∈ R
Example Let A = {a, b, c}, a possible equivalence relation?
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
20 / 57
Graph I
A graph G consist of pair of a set V and a set E, i.e., G = (V , E), called vertices and edges, respectively. The elements of set of edges E are distinct pairs of vertices connected to each other. For example, in the following figure K5 is the graph of five vertices and K3,3 also called complete bipartite graph on 3 + 3 = 6 vertices, and 3 = 9 edges.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
21 / 57
Graph II
Degree of a vertix v denoted by deg(v ) of a graph is the number of edges incident to the vertex, where we count the loop twice. Path in a graph is a sequence of connected vertices by edges.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
22 / 57
Alphabets, Strings & Languages
Definition (Alphabet) An alphabet is a finite set of symbols. For example, letters, digits, and punctuations. The alphabet is usally denoted by Σ
Example The set {0, 1} is the binary alphabet. ASCII is an example of alphabet Unicode approximately consist of 100,000 characters from alphabets around the world
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
23 / 57
Alphabets, Strings & Languages Definition (Strings) A string is a finite sequence of symbols drawn from the alphabet and that string is called the string over that alphabet. String is also sometimes called “word” or “sentence”. The length of a string s is usually written as |s| and the empty string is denoted by whose length is zero.
Example university is a string of length ten.
Concatenation Concatenation is a string operation that simply join/append a string with another string. For example, if s1 = Balochistan and s2 = University are two strings, then concatenation denoted by s1 s2 = BalcohstanUniversity. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
24 / 57
Terms for Parts of Strings
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
25 / 57
Alphabets, Strings & Languages
Definition (Language) Broadly, a language is any countable set of strings over some fixed alphabet. Here it is not related to the meanings associated to the strings.
Example Set of all syntactically well-formed C programs is a language. Set of all grammatically correct English sentences is also a language.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
26 / 57
Boolean Values Definition Boolean values are denoted as 1 and 0 representing true and false respectively that are used in logic, specifically connected to boolean algebra, boolean functions, and proposition calculus. The basic boolean operations and the truth table is given bellow
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
27 / 57
Outline
1
Course Description
2
Overview
3
Preliminaries
4
Proof Techniques
5
Quiz 1
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
28 / 57
Theorem Definition In mathematics, a theorem is a statement that has been proved true on the basis of some previously well established statements. The proof of a theorem is the sequence of mathematical statements that justifies the truth of the theorem. A proof contains axioms (assumptions, premise, considered to be true without proof, fundamental to the subject), hypotheses, and previously proven theorems. A proof of a theorem is a logical argument that deduces the conclusion from the hypothesis.
Theorem (Example) If x is odd then x 2 is odd.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
29 / 57
Some Terminologies
Axiom: A starting point of reasoning, self-evident principle or a statement accepted without proof, e.g., a + b = b + a. They are also considered as assumptions that characterize the area of study. Proposition: A theorem of no particular importance and whose proof is very easy. Lemma: A short theorem that is used as stepping stone for proving some large theorems. Corollary: A theorem that clearly follows from the truth of some other theorem and often requires little or no proof.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
30 / 57
How to Prove a Theorem? I It is related to question “ How to solve a problem?”. There is not specific way of proofing something but following tips may be helpful[2]. Read and completely understand the statement of the theorem to be proved. Most often this is the hardest part. Sometimes, theorem contains theorems inside them. For example, “Property A is and only if property B”, we may need to show two statements. 1 2
If property A is true then property B is true, i.e., A ⇒ B If property B is true then property A is true, i.e., B ⇒ A
Question Set A is equal to set B
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
31 / 57
How to Prove a Theorem? II
Try to work out a few simple cases of the theorem just to get a grip on it, For example, “Sum of cubes of success natural numbers (positive integers) is a square”. Try to write down the proof once you have it. This is to ensure the correctness of your proof. Often, mistakes are found at the time of writing. Finding proofs takes time, we do not come prewired to produce proofs. Be patient, think, express and write clearly and try to be precise as much as possible.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
32 / 57
Proofing Techniques
Some generic proofing strategies that we will study in this course are as followed: Direct Proof Constructive Proof Nonconstructive Proof Proof by Contradiction Proof by Induction
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
33 / 57
Direct Proof Definition In direct proof, we often approach the theorem directly by showing its truth or falsehood by directly combining well established facts. This strategy is often applicable to prove propositions, since the proposition are often so obviously true. In direct proof Take the original statement (hypothesis), say p. Assume it to be true. Show directly that another statement (conclusion), say q, is true. I.e., p ⇒ q.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
34 / 57
Direct Proof: Examples Theorem If n is an odd positive integer then n2 is odd as well.
Proof. Assume that n is an odd positive integer that can be written as, n = 2k + 1, where k is any positive integer such that k ≥ 0. Then n2 = (2k + 1)2 = 4k 2 + 4k + 1 since 2 is a common factor, we will have n2 = 2(2k 2 + 2k ) + 1 since the term 2(2k 2 + 2k ) is even, and even plus one is odd, therefore we conclude that n2 is odd.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
35 / 57
Direct Proof: Examples Theorem Let a, b and c be integers and if a divides b and a divides c then a also divides b + c.
Proof. Let a, b and c be integers and assume that a divides b and a divides c, then by definition b = ak and c = al, where k and l are also integers, and then b + c = ak + al, since, a is a common factor, then we can write b + c = a(k + l), and since k + l is an integer, hence a divides b + c Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
36 / 57
Direct Proof: Examples
Theorem (Class Activity) If m and n are odd integers then mn is also an odd integer.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
37 / 57
Direct Proof: Examples Theorem Let G = (V X, E) be a graph, then the sum of the degrees of all vertices, deg(v ), is an even integer. ∀v ∈V
Tip: First try on some graphs.
Proof. Assume G be a graph with the set of vertices V and set of edges E, then since, each edge is incident on two distinct vertices, and since, by definition, degree of a vertex is the number edges incident on that vertex, therefore, each edge will contribute 2 to the summation, hence, the sum of the degrees of all vertices is an even integer. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
38 / 57
Direct Proof: Class Work Theorem Let G = (V , E) be a graph, then the sum ofX the degrees of all vertices is equal to twice the number of edges, i.e, deg(v ) = 2 |E|. ∀v ∈V
Theorem if n is an even integer then 7n + 4 is an even integer.
Theorem if m is an even integer and n is an odd integer then m + n is an odd integer.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
39 / 57
Outline
1
Course Description
2
Overview
3
Preliminaries
4
Proof Techniques
5
Quiz 1
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
40 / 57
Quiz 1
Q1 : Let m and n are integers and if m and n are perfect squares then mn is also perfect square. Q2 : If m is an even integer and n is an odd integer then mn is an even integer. Q3 : If m and n are odd integers then mn is also an odd integer.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
41 / 57
Constructive Proofs Definition A technique of proofing that shows the existence of a mathematical object by creating or providing a way for creating the object. It is also called “constructive proof of existence”. The structure for such theorems and proofs are as followed:
Theorem There exists x, i.e., ∃x ∈ Ux and x have P(x)
Proof. Let x = a (where a is a specific element in Ux ) Verify that the property P(x) is true.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
42 / 57
Constructive Proofs: Example
Theorem Prove that there exists an integer n such that
m−7 2m+4
= 5.
Comment: Often time we need to do the real work before writing the proof of a theorem. Here, we must first derive some integer value for m that satisfy the property/equation.
Proof. Set m = −3, then m−7 2m+4
=
−3−7 2(−3)+4
=
−10 −2
Waheed Noor (CS&IT, UoB, Quetta)
=5
CS504-Theory of Computation
March 2014
43 / 57
Constructive Proof: Example
Theorem There exists are real number x such that
x 2 +3x−3 2x+3
= 1.
Proof. Set x = 2, then x 2 +3x−3 2x+3
=
4+6−3 4+3
=
7 7
= 1.
You can also set x = −3 and prove.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
44 / 57
Constructive Proof: Class Activity
Theorem There exists distinct positive integers m, n, and r such that each is perfect square and m = n + r .
Theorem If f (x) = x 3 + x − 5, then there exists a positive real number k such that f ‘(k ) = 7.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
45 / 57
Non-Constructive Proof
Definition A proof that indirectly shows a mathematical object exists with out providing an example or way of creating it. Sometime, in this type of proof, we may use contradiction, which we will study shortly. Constructive, and non-constructive proof are also called proof of existence. In calculus, there are very popular example of non-constructive proof of existence such as “intermediate value theorem” and “mean value theorem”.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
46 / 57
Non-constructive Proof: Example Theorem Let f (x) = x 3 − 3x 2 + 2x − 4, there exists a real number r such that 2 < r < 3 and f (r ) = 0.
Proof. Note that f (2) = 23 − 3(22 ) + 2(2) − 4 = −4 and f (3) = 33 − 3(32 ) + 2(3) − 4 = 2. Thus f (2) < 0 < f (3), since f is continuous function and by intermediate value theorem, there is a real number r such that 2 < r < 3 and f (r ) = 0.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
47 / 57
For All Proof: Example Theorem For all positive real number x, x +
4 x
≥4
Comment: Some time doing proof backward is helpful but definitely you should present it in the forward manner, i.e., assuming the hypothesis and then concluding P(x).
Proof. Let x be a positive real number, then clearly, (x − 2)2 ≥ 0, since the square of a real number is always positive and expanding will give us x 2 − 4x + 4 ≥ 0, and by assumption x is positive, so, dividing by x will preserve the inequality and we will get x −4+
4 x
≥ 0,
finally, x +
4 x
≥ 4.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
48 / 57
Proof by Cases: Example Sometime, the hypothesis is broken into simpler cases that are investigated separately.
Theorem If x is real number such that −2 > x < −1.
Waheed Noor (CS&IT, UoB, Quetta)
x 2 −1 x+2
> 0, then either x > 1 or
CS504-Theory of Computation
March 2014
49 / 57
Proof by Cases: Example Theorem For every real number a, lim
x→a
Waheed Noor (CS&IT, UoB, Quetta)
(x − 1) sin x exists. x2 − x
CS504-Theory of Computation
March 2014
50 / 57
Proof by Contradiction
Definition In this technique we assume the hypotheses are true while the conclusion is false and try to reach at the contradiction. If theorem states that a statement S is true, then by contradiction we assume that the statement S is false. That is, we need to show ¬S ⇒ false is true. And if we show this false then we are at the contradiction.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
51 / 57
Proof by Contradiction: Example Theorem Let x and y be real numbers. if 5x + 25y = 1723, then x or y are not an integer.
Proof. Assume x and y be real numbers such that 5x + 25y = 1723, Now, assume that both x and y are integers. Using the distributive law the expression will become 5(x + 5y ) = 1723. Since, x and y are integers this implies that 1723 is divisible by 5. However, 1723 is clearly not divisible by 5. Hence, this contradiction establishes the result that x or y is not integer.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
52 / 57
Proof by Contradiction: Example Theorem For all √ positive real numbers a, b, and c, if ab = c, then a ≤ b ≤ c.
√
c or
Proof. Suppose, a, b, and c are positive real numbers such that ab = c, √ √ and suppose a > c and b > c, √ note that the c is √ the positive square root and by order property of √ real numbers, b > c ⇔ ab > a c, √ √ √ √ since a > 0, and a > c ⇔ a c > c. c = c, √ √ √ √ since c > 0. Thus ab > a c > c. c = c implies ab > c. But since ab = c, hence ab is not greater than c, a contradiction. √ √ This proves that our assumption a > c and b > c can not be true when a, b, and √ c are positive √ real numbers such that ab = c. Therefore, a ≤ c or b ≤ c. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
53 / 57
Proof by Contradiction: Class Activity
Theorem Let n be a positive integer. If n2 is even, then n is even.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
54 / 57
Proof by Induction Principle of Mathematical Induction
Definition It is a method of proof that is used to show a given statement P(n) is true for all natural numbers n. It normally consists of two steps, base case or basis and inductive step, where the base case is to prove the given statement for first natural number, while the inductive step shows the statement is true for the next natural number. We infer, from these two steps, that the statement is true for all natural numbers. This kind of proof takes the form: Basis: Show that P(1) is true. Induction Hypothesis: Assume that for some fixed but arbitrary n ≥ 1, P(n) is true. Induction Inductive Step: Using the induction hypothesis, we show that the P(n + 1). Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
55 / 57
Proof by Induction: Example Theorem For any n ≥ 0, 1 + · · · + n =
n2 +n 2
Proof. Basis Step, Let n = 0, then the sum on left is zero. Induction Hypothesis, Assume that for some n > 0, 2 1 + · · · + m = m 2+m , where m ≤ n. Induction Step, 1 + 2 + · · · + n + (n + 1) = (1 + 2 + · · · + n) + (n + 1) = = =
n2 +n 2 + (n + 2 n +n+2n+2 2 (n+1)2 +(n+1) 2
1)
Hence, proved. Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
56 / 57
Homework Reading Read
the contrapositive proof, show the proof of the following theorem. Hint: You can first try by direct proof, however it will be difficult therefore if you fail try the contrapositive proof.
Theorem For all integers m and n, if the product mn is even, then m is even or n is even. Prove the following theorem by contradiction and direct proof.
Theorem For all positive real numbers a, b, and c, if a2 + b2 = c 2 , then a + b ≥ c.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
57 / 57
References I
Harry Lewis and Christos Papadimitriou. Elements of the Theory of Computation. Prentice Hall, 2nd edition, 1998. G. Polya. How to Solve It: A New Aspect of Mathematical Methods. Princeton University Press, 2nd edition, 1957. Michael Sipser. Introduction to the Theory of Computation. Thomson Course Technology, 2nd edition, 2006.
Waheed Noor (CS&IT, UoB, Quetta)
CS504-Theory of Computation
March 2014
58 / 57
