Advanced Artificial Intelligence Lecture 1

1
Advanced Artificial IntelligenceLecture 1

• Bob McKay
• School of Computer Science and Engineering
• College of Engineering
• Seoul National University

2
Outline

• Course Formalities
• Brief outline of course content
• Basics of Machine Learning
• Learning Relationships
• Relational Methods
• Inductive Logic Programming
• Statistical Relational Learning
• Evolutionary and Similar Methods
• Genetic Programming
• Estimation of Distribution and Ant Methods
• Summary

3
Detailed Course Aims

• To understand what relational learning problems
  are, and why they are important
• To understand the approaches which have been
  tried so far
• Logic-based methods
• Evolutionary and related methods
• To understand the limitations of existing
  approaches

4
What is Machine Learning?

• Commonest form
• Summarisation of data as a model
• Issues
• Modelling Languages
• Search Algorithms
• Measuring Correctness of Models
• Generalisation of Models
• (will it fit new data)
• Noise in Data
• Missing Data
• (well look at more careful definitions later)

• Facts

• Bob wears glasses

• Model

• Australian prime minister Howard wears glasses

• All Australians wear glasses

5
Why study Machine Learning?

• Explicit representation of knowledge allows
  systems to generate new data
• Prediction

• Model

• All Australians wear glasses

• New Fact

• Russell Crowe is Australian

• Russell Crowe wears glasses

• Note It might be wrong!

• Explicit representation of knowledge may be more
  readily understood than the data
• Explanation

6
What is Relational Learning?

• What it isnt!!
• You will often hear people talk of machine
  learning as discovering the hidden relationships
  in the data
• For example, a learning system might discover a
  link between a persons blood pressure,
  cholesterol level, and likelihood of a heart
  attack
• This is not what we mean by relational learning!
• Relational learning discovers relationships
  between data instances
• For example, that if your father had a heart
  attack, you have an increased probability of a
  heart attack

7
So Why Care?

• Today, learning systems for learning
  non-relational problems are highly effective
• There is still interesting research to be done on
  improving reliability with noise, missing data
  etc, and speeding up algorithms
• But improvements are largely incremental
• For non-relational problems, machines generally
  (hugely) outperform humans
• By contrast, learning systems for relational
  problems arent very good
• Humans are far better at tough relational
  learning problems than machines
• Theres heaps of space for new research!

8
Learning Relational Problems

• Two major strands of research
• Based in Logic and Statistics
• Largely Deterministic Algorithms
• Usually exploring search spaces defined by logic
  representation
• Based in Evolutionary Computation (Genetic
  Algorithms)
• Stochastic Algorithms
• Usually exploring search spaces defined by some
  form of function representation
• Only a small number of methods cross these
  boundaries

9
Typical Logic-Based Approaches

• Use a formal logic representation
• Often prolog
• Start with some data
• sort(,).
• sort(1,1).
• sort(2,2).
• sort(2,1,1,2).
• Sort(2,1,3,1,2,3).
• Etc

10
.Logic-Based Approaches

• Maybe use some background knowledge
• E.g. how to merge lists
• merge(,Ys,Ys).
• merge(Xs,,Xs).
• merge(XXs,YYsXZs) -
• X gt Y,
• merge(Xs, YYs, Zs).
• merge(XXs, YYs, YZs) -
• X lt Y,
• merge(XXs, Ys, Zs).

11
.Logic-Based Approaches

• Define relatively complex operators and
  heuristics
• Use deterministic backtracking algorithms

12
Logic-Based Approaches

• Learn a new program
• sort(,).
• sort(Unsorted,Sorted) -
• merge(U1, U2, Unsorted),
• sort(U1, S1),
• sort(U2, S2),
• merge(S1, S2, Sorted).

13
Typical Evolutionary Approaches

• Use a function-based representation
• Often close to lisp
• Start with some data
• (site 1
• (colour brown) (Gld 6) (landsys kln))
• (site 2
• (colour black) (Gld 3) ..)
• Etc
• (next-to 1 2)
• (next-to 2 3)
• Etc

14
Evolutionary Approaches

• Use simple operators (crossover, mutation)
• Use relatively simple heuristics
• Fitness function
• Use highly stochastic search
• Evolutionary / Genetic Algorithms
• Estimation of Distribution Algorithms
• Ant-based Algorithms

15
Evolutionary Approaches

• Learn a new program
• (if (and ( (Color here) brown)
• (and (gt (Gld here) 6) ( (Landsys here)
  KLN)))
• 3
• (if (and ( (Assoc here) eucobl)
• (and (next-to here loc) (gt (Gld loc) 6)))
• 2
• (if (and (gt (Xanscore here) 1)
• (and (lt (N_fire here) 5) (gt (Gld here) 6)))
• 1
• 0)))

16
Issues in Relational Learning

• Improving computational feasibility
• Especially, avoiding infinite loops (logic-based)
• Scalability
• Incorporating statistical models and handling
  noise
• Handling missing data
• Controlling genome complexity (evolutionary)

17
?????