Artificial Intelligence 15381

1
Artificial Intelligence 15-381

• 15-381 Artificial Intelligence
• Today Introduction to AI Search Methods
• (and Aministrivia)
• Jaime Carbonell
• jgc_at_cs.cmu.edu
• 14-January-2003

2
Todays Topics

• What exactly is AI, anyway?
• AI searchknowledgelearning
• AI application areas
• Course Outline
• Administration and grading ?
• PROJECT Free-text Q/A
• Basic search methods (Most of this lecture)

3
What is AI Some Quick Answers

• From the Media AI is
• What socially-inept superhackers do
• The opposite of natural stupidity
• Building useful idiot-savant programs
• Deep Blue (IBMs chess program)
• Robots with feelings (Spielberg)

4
What is AI Some Quick Answers (cont.)

• From Academia AI is
• modeling aspects of human cognition by computer
• the study of ill-formed problems
• "nothing more" than advanced algorithms research
• cool stuff! Machine learning, data mining,
  speech, language, vision, web agentsand you can
  actually get paid a lot for having fun!
• what other CS folks dont yet know how to do,
  and we AIers arent always too sure either

5
Operationally Speaking, AI is

• Applied Cognitive Science
• Computational models of human reasoning
• Problem solving
• Scientific thinking
• Models of non-introspective mental processes
• Language comprehension, language learning
• Human memory organization (STM, LTM)

6
Operationally Speaking, AI is

• Knowledge Engineering
• Codify human knowledge for specific tasks
• E.g. Medical diagnosis, Machine Translation
• Central in 1970s 80s ? just one lecture here
• Problem-Solving Methods
• How to encode and use knowledge to find answer
• E.g. HS, MEA, A, Logic resolution
• Always at the very core of AI ? many lectures

7
Operationally Speaking, AI is

• Machine Learning
• Learning as the hallmark of intelligencebut it
  is already practical in multiple applications
• E.g. D-trees, rule-induction, reinforcement,
  NNets
• Discredited in 1960s ? Vibrant core 90s 00s
• Applications data text mining, speech,
  robotics
• Most active research area in AI ? many lectures

8
AI Application Areas

• Rule-Based Expert Systems
• Medical Diagnosis MYCIN, INTERNIST, PUFF
• CSP Scheduling ISIS, Airline scheduling
• Data Mining
• Financial Fraud detection, credit scoring
• Sales Customer preferences, inventory
• Science NASA galaxy DB, genome analysis

9
AI Application Areas (cont.)

• Language Processing
• Speech dictation, HCI
• Language Machine Translation
• ML NLP Fact Extraction
• ML words Information Retrieval
• Robotics
• Machine Vision
• Mobile Robots agents
• Manipulation

10

• 15-381 AMINISTRIVA
• Lectures on Web ( Occasional Recitations)
• Text Artificial Intelligence A Modern
  Approach by Russell Norvig, Prentice Hall,
  Second Edition
• Tasks Grading
• Problem sets 1 2 10 each
• Midterm 15 (closed book)
• Final 25 (open book)
• Project 40 (2 or 3
  person teams)

11

• TAs
• Andy Yang ayang_at_andrew
• Maxim Likhachev likhachev_at_cs
• Jing Xiao xiaoj_at_cs
• Joao Pedro Suosa jpsousa_at_cs
• Course Admin
• Eleanor Cambridge eleanorc_at_cs
• 4517 NSH x8-4788
• Late Policy 10 lower grade if up to a week late

12
AI-Based Problem Solving

• BASIC SEARCH METHODS
• State-Space ltS, S0, SGj, Oigt
• S0 Initial State
• SG Goal State(s) to be reached
• Oi Operators O S gt S

13
AI-Based Problem Solving (cont.)

• State-Space Navigation
• Forward Search BFS, DFS, HS,
• Backward Search BFS-1, Backchaining,
• Bi-Directional Search BFS2,
• Goal Reduction Island-S, MEA
• Transformation S ? S
• Abstraction S ? SA MEA (SA)
• Analogy If Sim(P,P) then Sol(P)? Sol(P)

14
More on the State Space

• Useful Functions
• Succ(si) sk oj(si) sk
• Reachable(si) Usk Succ (si)
• Succ-1(si) sk oj(sk) si)
• Reachable-1(si) Usk (Succ-1)(si)
• s-Path(sa0, san) (sa0, sa1,, san)
• such that for all sa1 exists oj(sai) sai1
• o-Path(sa0, san) (oj0, oj1,, ojn-1)
• such that for all sa1 exists oj(sai) sai1

15
More on the State Space (cont.)

• Useful Concepts
• Solution o-Path(s0, sG) or s-Path
• Cost(Solution) ? cost(oj) often cost(oj) 1
• P is solvable if at least one o-Path(s0, sG)
  exists
• Solutions may be constructed forward, backward or
  any which way
• State spaces may be finite, infinite, implicit or
  explicit

16
Zero-Knowledge Search

• Simple Depth-First Search
• DFS(Scurr, Sgoal, S-queue)
• IF Scurr Sgoal, SUCCESS
• ELSE Append(Succ(Scurr), S-queue)
• IF Null(S-queue), FAILURE
• ELSE DFS(First(S-queue), Sgoal, Trail(S-queue))

17
Depth First Search
1
SI
2
7
3
8
5
4

SG
6
18
DFS (cont.)

• Problems with DFS
• Deep (possibly infinite) rat holes
• ? depth-bounded DFS, D max depth
• Loops Succ(Succ(..Succ(S))) S
• ? Keep s-Path and always check Scurr
• Non-Optimality Other paths may be less costly
• ? No fix here for DFS
• Worst-case time complexity (O(bmax(D,d))

19
DFS (cont.)

• When is DFS useful?
• Very-high solution density
• Satisficing vs. optimizing
• Memory-limited search O(d) space
• Solution at Known-depth (then Dd)

20
Zero Knowledge Search (cont.)

• Simple Breadth-First Search
• BFS(Scurr, Sgoal, S-queue)
• IF Scurr Sgoal, SUCCESS
• ELSE Append(Succ(Scurr), S-queue)
• IF Null(S-queue), FAILURE
• ELSE BFS(Last(S-queue), Sgoal,
• All-But-Last(S-queue))

21
Breadth-First Search
1
4
3
2
11
10
9
8
7
6
5
12

SG
22
Simple BFS cont.

• Problems with BFS
• Loops Succ(Succ(Succ(S)))S
• Pseudo-loops Revisiting old states off-path
• ? Keep full visited prefix tree
• Worst case time complexity O(bd)
• Worst case space complexity O(bd)
• When is BFS Useful?
• Guarantee shortest path
• Very sparse solution space
• (better if some solution is close to SI)

23
Zero Knowledge Search (cont.)

• Backwards Breadth-First Search
• BFS(Scurr, Sinit, S-queue)
• IF Scurr Sinit, SUCCESS
• ELSE Append(Succ-1(Scurr), S-queue)
• IF Null(S-queue), FAILURE
• ELSE BFS(Last(S-queue), Sinit,
• All-But-Last(S-queue))

24
Backwards Breadth-First Search
9
SI

8
7
6
5
4
3
2
1
SG
25
Backward-BFS (cont.)

• Problems with Backward-BFS
• All the ones for BFS
• Succ(Scurr) must be invertible Succ-1(Scurr)
• When is Backward-BFS useful?
• In general, same as BFS
• If backward branching lt forward branching

26
Bi-Directional Search

• Algorithm
• Initialize Fboundary Sinit
• Initialize Bboundary Sgoal
• Initialize treef Sinit
• Initialize treeb Sgoal
• For every Sf in Fboundary
• IF Succ(Sf) intersects Bboundary
• THEN return APPEND(Path(treef), Path-1(treeb))
• ELSE Replace Sf by Succ(Sf) UPDATE (treef)
• 6. For every Sb in Bboundary
• IF Succ(Sb) intersects Fboundary
• THEN return APPEND(Path(treef),
  Path-1(treeb))
• ELSE Replace Sb by Succ-1(Sb) UPDATE
  (treeb)
• 7. Go to 5.
• Note wheres the bug?

27
Bi-Directional Breadth-First Search
1
SI
3
4
8
9
10
11
12
13

7
5
6
2
SG
28
Bi-Directional Search (cont.)

• Problems with Bi-BFS
• Loops Succ(Succ(Succ(S))) S
• Loops Succ-1(Succ-1( Succ-1(S)))) S
• Pseudo-loops Revisiting old states off-path
• ? Keep full visited prefix treef, trees
• Succ(Scurr)must be invertible Succ-1(Scurr)
• When is Bi-BFS useful?
• Space and time complexity
• O(bfd/2) O(bbd/2) O(bd/2) if bf bb

29
Island-Driven BFS

• Definition
• An island is a state known a-priori to be on the
  solution path between Sinit and Sgoal.
• If there are k sequential islands
• BFS(Sinit, S-(goal)
• APPEND(BFS(Sinit, Sk1), BFS(Sk1, Sk2),BFS(SIk,
  Sgoal))
• Upper bound complexity O(kmaxi0kbdki,ki1)
• Complexity if islands are evenly spaced
• O((k1)bd/(k1))

30
Island-Driven Search
1
SI

SIsland
SG