Integration of Search and Learning Algorithms

1
Integration of Search and Learning Algorithms

• Eugene Fink

2
Basic intuition
An intelligent system should select or invent an
effective approach to a given problem.
3
Challenges

• Select among available search algorithms (or
  invent a new algorithm)
• Restate the problem, to improve the efficiency of
  the selected algorithm

4
Search algorithm
search
problem
5
Search algorithm
search
problem
solution ?
6
Search algorithm
search
problem
solution ?
A search module inputs a problem and runs until
finding a solution, failing, or reaching a time
limit.
7
Speed-up learning algorithm
speed-up learning
problem
8
Speed-up learning algorithm
speed-up learning
problem and new knowledge
problem
9
Speed-up learning algorithm
speed-up learning
search
problem and new knowledge
problem
10
Speed-up learning algorithm
speed-up learning
search
problem and new knowledge
problem
An algorithm inputs a problem and generates
additional knowledge, which improves search
efficiency.
11
or static-analysis algorithm
static analysis
search
problem and new knowledge
problem
An algorithm inputs a problem and generates
additional knowledge, which improves search
efficiency.
12
Goals

• Construct an architecture for integration of
  multiple search and learning modules

• Centralized architecture
• Integration tools

13
Goals

• Construct an architecture for integration of
  multiple search and learning modules

• Centralized architecture
• Integration tools

• Develop a control mechanism that chooses
  appropriate modules for each problem

• Top-level control decisions
• Reuse of accumulated knowledge

14
Related work

• AI systems with multiple learning and search
  modules Soar, Prodigy,
• Integration tools Multiagent Planning
  Architecture (Wilkins and Myers), Plan (Yang et
  al.)
• Selection among available algorithms Horvitz,
  Breese, Russell, Minton, ...

15
Shaper system

• Three main parts
• Library of search modules
• Library of learning modules
• Top-level control mechanism

16
System outline

top-level control
search modules
learning modules
17
System outline

top-level control
database of learned knowledge
search modules
learning modules
18
System outline

top-level control
Control center
database of learned knowledge
search modules
learning modules
19
System outline
top-level control
Manual control
Automatic control
Control center
database of learned knowledge
search modules
learning modules
20
System outline
top-level control
Manual control
Automatic control
Control center
database of learned knowledge
search modules
learning modules

• Given a new problem
• Select appropriate modules
• Apply them to solve the problem
• Repeat if necessary

21
Features of the top-level control

• Fast Less than a second per problem
• General Independent of specific modules
• Allows participation of a human expert Initial
  knowledge and interactive advice

22
Features of the automatic control

• Several learning mechanisms
• Symbolic and statistical techniques
• Combining exploitation and exploration

23
Automatic control
Solve the problem or learn additional knowledge?
24
Automatic control
Solve the problem or learn additional knowledge?
learn
Which learner to apply? Which past results to use?
25
Automatic control
Solve the problem or learn additional knowledge?
learn
Which learner to apply? Which past results to use?
Invoke the selected learning module
26
Automatic control
Solve the problem or learn additional knowledge?
learn
solve
Which learner to apply? Which past results to use?
Solve or skip the problem? With which search
module? Which learned data to use?
Invoke the selected learning module
27
Automatic control
Solve the problem or learn additional knowledge?
learn
solve
Which learner to apply? Which past results to use?
Solve or skip the problem? With which search
module? Which learned data to use?
solve
skip
Invoke the selected learning module
Invoke the selected search module
Wait for the next problem
28
Automatic control
Solve the problem or learn additional knowledge?
learn
solve
Which learner to apply? Which past results to use?
Solve or skip the problem? With which search
module? Which learned data to use?
solve
failure
skip
Invoke the selected learning module
Invoke the selected search module
success
Wait for the next problem
29
Performance example
Solving a series of 50 problems, and learning
which modules are the best.
gain on each problem
order of solving problems
30
Performance example
Solving a series of 50 problems, and learning
which modules are the best.
accumulated average gain
order of solving problems
31
Performance example
Solving a series of 500 problems, and learning
which modules are the best.
accumulated average gain
order of solving problems
32
Applications

• Past AI planning (Prodigy)
• Present Hardware design (power estimates)
• Future Biomedical data, vision, ...