CS 561: Artificial Intelligence „

„ „

Instructor: Prof. Prof Hadi Moradi, Moradi [email protected] Lectures: M-Th 09:00-10:40, OHE136 Office hours: MW 2:30 – 4:00 pm, SAL310, Or by O b appointment i TAs: Jeong-Yoon Lee „

„

„ „ „

SAL 112 Office hours: TTH 1:00-2:30 Email: [email protected]

CS 561: Artificial Intelligence „

Course web page: „ „ „ „

„ „

http://www-scf.usc.edu/~csci561a Up to date information, lecture notes Relevant dates, links, etc. Also you may check http://den.usc.edu

Class format: two sections of 45 minutes Course material: „

[AIMA] Artificial Intelligence: A Modern Approach, by Stuart Russell and Peter Norvig. 2nd edition

1

CS 561: Artificial Intelligence „

„

Course overview: foundations of symbolic intelligent systems. Agents, search, problem solving, logic, representation, reasoning, symbolic programming, probabilistic reasoning, and robotics. Prerequisites: CS 455x, i.e., „

programming principles, discrete mathematics for computing, software design and software engineering concepts. Some knowledge of C/C++ for some programming assignments.

CS 561: Artificial Intelligence „ „ „ „ „

Grading: 25% for midterm 25% for final 40% for homeworks and projects 10% for f Quizzes Q i

2

Practical issues „

Class list: use learn.usc.edu learn usc edu „

„

„

Login with your USC username and password If CSCI561A is not listed as your courses, notify ot y the t e TA.

Submissions: See class web page under Assignments submit -user csci561 -tag HW3 HW3.tar.gz

Administrative Issues „

Midterm 1: 7/26/10 9:00 - 10:40pm

„

Midterm 2: 8/10/10 9:00 - 10:40pm See also the class web page: http://den usc edu/ http://den.usc.edu/

3

Why study AI?

Search engines Science Medicine/ Diagnosis Labor Appliances

What else?

Humanoid Robots: From Honda to Sony

Walk

Turn http://world.honda.com/robot/

Stairs

4

Sony AIBO

movie1

http://www.aibo.com

Natural Language Question Answering

http://aimovie.warnerbros.com

http://www.ai.mit.edu/projects/infolab/

5

Robot Teams

USC robotics Lab

Modular robots self re-assembly.

What is AI? The exciting new effort to make “The The study of mental faculties computers thinks … machine through the use of computational with minds, in the full and models” (Charniak et al. 1985) literal sense” (Haugeland 1985) “The art of creating machines that perform functions that require intelligence when performed by people” (Kurzweil, 1990)

A field of study that seeks to explain and emulate intelligent behavior in terms of computational processes” (Schalkol, 1990)

6

AI – The Bigger Picture ?

Computer Science

Philosophy p y

Artificial Intelligence Cognitive Science (Psychology)

Robotics (Engineering)

Neuroscience (Biology)

?

Acting Humanly: The Turing Test „

Alan Turing Turing'ss 1950 article Computing Machinery and Intelligence discussed conditions for considering a machine to be intelligent

7

Acting Humanly: The Turing Test

What tasks require AI? „

„

“AI AI is the science and engineering of making intelligent machines which can perform tasks that require intelligence when performed by humans …”

What tasks require AI?

8

What tasks require AI? „

Tasks that require q AI: „ „ „ „ „ „ „ „ „ „

Solving a differential equation Brain surgery Inventing stuff Playing Jeopardy Playing Wheel of Fortune What about walking? What about grabbing stuff? What about pulling your hand away from fire? What about watching TV? What about day dreaming?

Acting Humanly: The Full Turing Test

• Problem:

9

What would a computer need to pass the Turing test? „ „ „ „

Communication: Memory: Reasoning: Learning:

What would a computer need to pass the Turing test? „

Sensing:

„

M t control Motor t l (total (t t l ttest): t)

10

Thinking Humanly: Cognitive Science „

1960 “Cognitive Cognitive Revolution Revolution”:: information-processing psychology replaced behaviorism

Thinking Humanly: Cognitive Science „

Cognitive science and modeling the activities of the brain „

„

What level of abstraction? “Knowledge” or “Circuits”? How to validate models?

11

Thinking Rationally: Laws of Thought „

Aristotle (~ ( 450 B.C.) attempted to codify “right thinking” „

What are correct arguments/thought processes?

Thinking Rationally: Laws of Thought „

Problems:

12

Acting Rationally: The Rational Agent „ „

Rational behavior: Doing the right thing! Provides the most general view of AI because it includes:

Acting Rationally: The Rational Agent „

Advantages:

13

How to achieve AI? „

How is AI research done? „ „

Theoretical Experimental

How to achieve AI? „

There are two main lines of research: „

„

„

Biological, study humans and imitate their psychology or physiology. phenomenal, study and formalize common sense facts about the world and the problems that the world presents to the achievement of goals.

The two approaches interact to some extent, and both should eventually succeed. It is a race, but both racers seem to be walking. [John McCarthy]

14

Branches of AI „ „ „ „ „ „ „ „ „

Logical AI Search Natural language processing pattern recognition Knowledge representation Inference From some facts,, others can be inferred. Automated reasoning Learning from experience Planning To generate a strategy for achieving some goal

AI Prehistory

15

Brief History of AI Thinking Rattionally:Laws of T Thought

Ancient Times

M iddle Age

384 B.C.

1200

- Aristotle - Logic: The science of knowing. Ramon Lull Ars Magnus: a rule-based device to model man's behavior and nature - Empiricism Explanation of processes - Gottfried Leibniz - 1st system of formal logic -

Renaissance 17 th Century

18 th Century

19 th Century

Next time implement links

Rene Descartes Dualism 1845

- Charles Babbage - Analytical Engine - George Boole - Formalization of the Laws of Logic -

1879-1903

Early 20th Century

1910-1912

-

Gottlob Frege First-order predicate calculus Russel-Whitehead Principia Mathematica Bertrand Russel

1931

- Kurt Godel - Incompleteness Theorem of Logic -

Roots of AI in Science: „ „

„

„

„

„

„

Aristotle(b.384-): syllogism – formal reasoning Ramon Lull (b.1235): Ars Magna – a machine capable of answering all questions Rene Descartes (1596): mind / body separation (dualism); "cogito ergo sum“ Wilhelm Liebniz (1646-1716): a mechanical concept generator;; "materialism" g Charles Babbage(1792-1871), Ada Lovelace (1815-1860): Analytical Engine – a general-purpose calculator George Boole(1815-1864): logic algebras - logical encoding and calculation of thoughts Gottlob Frege(1848-1925): predicate calculus

16

Birth of Artificial Intelligence 1940-1956 1942

1943

1945

Greaat Expectations

1949

-

ENIAC :First digital computer

Mc Culloch and Pitts Artificial neural network J. Von Neumman Modern computer architecture

Claude Shannon Use of heuristics to solve complex problems

1950

- Alan M.Turing - Computing Machinery and - Intelligence: Turing Test

1955

- Herbert Simon,Alan Newell - 1st AI program:Logic Theorist -

1956

- Dartmouth Conference -

Herbert Simon

The Beginning of AI „

„

„

„

McCulloch & Pitts „ developed theory of artificial neurons (precursor to ANN's) – 1943 Alan Turing – "Can Machines Think?" „ the turing test (1950) „ the turing machine Marvin Minsky & Dean Edmonds „ first ANN constructed, 1951 John McCarthy „ convened the Dartmouth conference that coined the term artificial intelligence (AI) (1956) and set the research agenda „ symbolic AI „ connectionism st AI language „ LISP (list processing) 1958 1

17

The Rise of AI 1957- 1960’s 1958

1960

1961

Growingg Disenchantment

1962

1965

- John McCarthy. - LISP -

Marvin Minsky Theory of Frames Herbert Simon,Alan Newell GPS:General Problem Solver Herbert Simon

Frank Rosenblatt Perceptron: Learning in Neural Networks

- L Lotfi tfi A. A Zadeh Zd h Fuzyy Logic Fuzzy Sets -

1968

Joseph Weizenbaum ELIZA: simulates diagnosis by a psychiatrist.

1969

- Marvin Minsky,Seymour Papert - Limitations of Perceptrons S. Papert

An Optimistic Start „

In the 50's, 60's and early 70's, much exciting progress was being made in AI: Chess „

„

The Logic Theorist „

„

Feigenbaum, Buchanan, Lederberg, 1969

SHRDLU – NLP (Blocks World) „

„

Joseph Weizenbaum, 1966

DENDRAL – Knowledge-Based System „

„

Arthur Samuels, 1959

Eliza - NLP „

„

Alan Newell, Cliff Shaw, Herb Simon, 1957

Checkers (Machine Learning) „

„

Claude Shannon, 1950

Terry Winnograd, 1972

GPS (General Problem Solver) „

Alan Newell & Herb Simon, 1972

18

The 70’s Bi th and Birth d Rise Ri of fE Expertt S Systems t 1970-mid 1980’s 1973

1974

-

1975

Alain Colmerauer PROLOG Paul Werbos Neural Networks Back Propagation Law E. Feigenbaum, R. Lindsay. Dendral E.FeigenBaum

Edward Shortliffe MYCIN 19761980

R. Duda, P.Hart, P. Barnett PROSPECTOR: The first commercial Expert System

1982

John McDermott XCON – "Expert Configurer

P.Hart

The Plateau

In the 70's, AI researchers began to discover that the problem wasn't as easy as it looked! „

The Frame Problem

„

L k of Lack f Common C Sense S Reasoning R i

„

Combinatorial Explosion

„

The Gap – "Toy" vs. "Real" worlds

„

„

Perceptrons, by Minsky & Papert (1969) – proved limitations of perceptron networks and acted to limit significant research in the 70 70'ss Lighthill Report – 1973: curtailed research funding in British Universities AI developed a reputation as "over-hyped" and unrealistic

19

1982

Rebirth of Arttificial Neural Netw works

Commercializa ation of Expert Systtems

The 80’s

- John Hopfield - Hopfield Networks -

1982

1986

Teuvo Kohonen self-organising feature maps for speech recognitizion

T Sejnowski S j ki - Terrence - NETTalk Rumerhalt,McMelland

Neural Networks Rediscovering of Back-Propagation Learning 1987

- Marvin Minsky - The Society of Minds -

Fuzzy Appliances

1989

- Dean Pomerleau - ALVINN -

Commercial Success Despite it's it s reputation as "over over-hyped hyped", certain AI applications became very successful during the 70's – 80's: •

Expert Systems

•

Industrial Robotics

•

Planning & Scheduling Applications

AI became a $2,000,000,000 industry by 1988

20

Nowadays… - Major advances in all areas of AI, with - significant demonstrations -

Early 90’s

Late 90’s

1995

Birth of Intelligent Systems

1997

The Deep Blue chess program beats Garry Kasparov

- Web crawlers - AI-based information extraction - programs Intelligent Room and Emotional Agents at MIT's AI Lab

2000-

Interactive robot pets The Nomad robot

The Gartner Hype Curve „

Interest in AI followed this pattern pattern,

typical of the hype surrounding new technologies

21

AI State of the art „

Have the following been achieved by AI? „ „ „ „ „ „ „ „ „

World-class chess playing Playing table tennis Cross-country driving Solving mathematical problems Discover and prove mathematical theories Engage in a meaningful conversation Understand spoken language Observe and understand human emotions …

Types of expertise

(with examples)

Deep cognitive skills

Judgmental High-level skills social skills

Highly creative

Musician

Senior manager

Analytical

Mathemati i ician

Economist, Social programmer scientist i ti t

Typist Strictly procedural

Driver

Author, poet

Social worker

22

A driving example: Grand Challenge

„

Goal:

Artificial Intelligence Applications Artificial Intelligence

Cognitive Science Applications •Expert Systems •Fuzzy Logic •Genetic Algorithms •Neural Networks

Robotics Applications

•Visual Perceptions •Locomotion •Navigation •Tactility

Natural Interface Applications •Natural Language •Speech Recognition •Multisensory Interface •Virtual Reality

23

AI Application Areas in Business Neural Networks Fuzzy Logic Systems Genetic Algorithms Virtual Reality y AI Application Areas in Business

Intelligent Agents Expert Systems

Components of Expert Systems The Expert System Expert Advice

User

User IInterface t f Programs

Inference E i Engine Program

Knowledge K l d Base

Workstation

Expert System Development Knowledge Engineering

Knowledge Acquisition Program Workstation

Expert and/or Knowledge Engineer

24

Expert System Applications Decision Management Diagnostic/Troubleshooting

Maintenance/Scheduling

Design/Configuration

Major Application Categories of Expert Systems

Selection/Classification

Process Monitoring/Control

Course Overview General Introduction „

Introduction. [AIMA Ch 1] Course Schedule. Homeworks, exams and grading. Course material, TAs and office hours. Why study AI? What is AI? The Turing test. Rationality. Branches of AI. Research disciplines connected to and at the foundation of AI. Brief history of AI. Challenges for the future. Overview of class syllabus.

25

Agent

effectors

sensors

Course Overview

General Introduction „

Intelligent Agents. [AIMA Ch 2] What is

an intelligent agent? Examples. Doing the right thing (rational action). Performance measure. Autonomy. Environment and agent design. Structure of agents agents. Agent types types. Reflex agents agents. Reactive agents. Reflex agents with state. Goal-based agents. Utility-based agents. Mobile agents. Information agents.

Course Overview (cont.) „

Problem solving and search. [AIMA Ch 3] „ „ „ „ „ „

„

measuring problem. Types of problems. More examples. Basic idea behind search algorithms. Complexity. Combinatorial explosion and NP completeness. Polynomial hierarchy.

3l

5l

9l

Using these 3 buckets, measure 7 liters of water.

Traveling salesperson problem

26

Course Overview (cont.) How can we solve complex problems? „

Uninformed search. [AIMA Ch 3] „ „ „ „ „ „

Depth-first. Breadth-first. Uniform-cost. Depth-limited. Iterative deepening. Examples. Properties.

3l

5l

9l

Using these 3 buckets, measure 7 liters of water.

Traveling salesperson problem

Course Overview (cont.) How can we solve complex p p problems? „

Informed search. [AIMA Ch 4] „ „ „ „ „ „

Best-first. A* search. Heuristics. Hill climbing. Problem of local extrema. Simulated annealing.

Traveling salesperson problem

27

Course Overview (cont.) Practical applications of search „

Constraint Satisfaction

[AIMA Ch 5] „ „

Backtracking g Local search

Course Overview (cont.) Practical applications of search „

Game playing

[AIMA Ch 6] „ „ „ „

The minimax algorithm. g Resource limitations. Aplha-beta pruning. Elements of chance and non-deterministic games.

tic-tac-toe

28

Course Overview (cont.) Towards intelligent agents „

Agents that reason logically 1

[AIMA Ch 7] „

„

„

Knowledge-based agents. Logic and representation. Propositional (boolean) logic.

wumpus world

Course Overview (cont.) Towards intelligent agents „

Agents that reason logically 2.

[AIMA Ch 7] „

„ „ „

Inference in propositional ii l logic. l i Syntax. Semantics. Examples.

wumpus world

29

Course Overview (cont.) Building u d g knowledge-based o edge ased agents: 1st Order Logic „

First-order logic 1. [AIMA Ch 8] „ „ „ „ „ „ „

Syntax. Semantics. Atomic sentences. sentences Complex sentences. Quantifiers. FOL knowledge base. Situation calculus.

Course Overview (cont.) Building knowledge knowledgebased agents: 1st Order Logic „

First-order logic 2.

[AIMA Ch 9] „ „ „

Describing actions. Planning. Action sequences.

30

Course Overview (cont.) Reasoning Logically „

Inference in first-order logic.

[AIMA Ch 9] „ „ „ „

Proofs. Unification Unification. Generalized modus ponens. Forward and backward chaining. Example of backward chaining

Course Overview (cont.) Representing and Organizing Knowledge „

Building a knowledge base.

[AIMA Ch 10] „ „ „ „

Knowledge bases. Vocabulary and rules. Ontologies Organizing knowledge. An ontology for the sports domain

31

Course Overview (cont.) Systems y that can Plan Future Behavior „

Planning.[AIMA Ch 11] „ „

„ „

Definition and goals. Basic representations for planning. l i Situation space and plan space. Examples.

Course Overview (cont.) Learning g from Observation „

Decision Trees [AIMA 18] „ „ „ „

Introduction to decision trees. Information theory. Constructing DT. Examples.

32

Course Overview (cont.) Expert p Systems y „

Probabilities + Bayesian Networks [AIMA 13 + 14] „ „ „ „ „

Basics of probability theory Bayesian rule. Conditional d l reasoning. Bayesian Networks. Reasoning under uncertainty

Course Overview (cont.) Statistical Learning g Methods „

Neural Networks. [AIMA 20] „ „ „ „

Human brain structure Neuron and activation function. Forward and backward propagations. Examples.

33

Course Overview (cont.) Logical g Reasoning g in the Presence of Uncertainty „

Fuzzy logic [Handout] „ „ „ „

Center of gravity

Introduction to fuzzyy logic. g Linguistic Hedges. Fuzzy inference. Examples.

Center of largest area

Course Overview (cont.) Machine Learning g „

Genetic Algorithms [Handout + AIMA 4] „ „ „

Genetic algorithm approach. Mutation, Crossover, Fitness function. Examples.

34

Course Overview (cont.) What challenges g remain? „

Towards intelligent machines. [AIMA Ch 25] „

The challenge of robots: „ „ „ „ „ „

with what we have learned, what hard problems remain to be solved? Different types of robots. Tasks that robots are for. Parts of robots. Architectures. Configuration spaces.

robotics@USC

Course Overview (cont.) What challenges remain? „

Overview and summary. [all of the above] „ „

What have we learned. learned Where do we go from here?

robotics@USC

35

Outlook „ „

AI is a very exciting area right now. now This course will teach you the foundations.

36