Learning

1
Learning

• Leslie Pack Kaelbling
• Computer Science and Artificial Intelligence
  Laboratory
• Massachusetts Institute of Technology

2
What is learning?

• Learning denotes changes in the system that are
  adaptive in the sense that they enable the system
  to do the task or tasks drawn from the same
  population more efficiently and more effectively
  the next time. -- Herb Simon

3
Learning is crucial

• Any system that is connected to a real external
  environment needs to learn
• Humans, even if expert, are inarticulate
• Single system should be adaptable to multiple
  environments
• Environments change over time

4
Some machine learning successes

• assessing loan credit risk
• detecting cell phone fraud
• improved manufacturing electric pumps, steel
  rolling, rotogravure, separation of gas and oil
• cataloging astronomical images
• helping NBA coaches analyze performance
• personalizing news and web searches
• steering an autonomous car across the US

5
Supervised learning

• Given data (training set)
• Goal find a hypothesis h in hypothesis class H
  that does a good job of mapping x to y

input
output
Classification discrete Y Regression continuous
Y
6
Spam filtering

• Represent document as a bag of words
• How many times has each word occurred in the
  document?
• x lt 0, 0, 0, 1, 0, 0, 2, 0, 0, 4, 0, 0, 0,
  1, 0, 1, gt
• Label is it spam or not?
• Classifier given a new document, predict whether
  its spam

7
Attribute vectors

• Is this an image of a face?
• ltpixel1, pixel2, pixel3, gt
• Will John be late to the meeting?
• lt time, location, topic, preceding appt, gt
• Will my proposal be funded?
• x lt dollar-amount, num-pis, topic, length,
  font, gt

8
Memory
yes
no
No
9
Statistics
yes
no
Yes
10
Generalization
?
?
11
Hypothesis
yes
blue?
oval?
yes
no
big?
no
no
yes
12
Decision Tree
blue?
oval?
yes
big?
no
?
no
yes
13
Decision Tree
blue?
oval?
yes
big?
no
?
no
yes
14
Whats the right hypothesis?
15
Whats the right hypothesis?
16
Now, whats the right hypothesis?
17
Now, whats the right hypothesis?
18
How about now?
19
How about now? Answer 1
20
How about now? Answer 2
21
How about now?
22
How about now?
23
No Free Lunch

• Unless you know something about the distribution
  of problems your learning algorithm will
  encounter, any hypothesis that agrees with all
  your data is as good as any other.
• You cant learn anything unless you already know
  something.

24
Supervised learning is reasonably well solved

• Learning methods differ in terms of
• the form of the hypothesis
• the way the computer finds a hypothesis given the
  data
• Variety of methods
• nearest neighbor
• decision trees
• neural networks
• support vector machines

25
Support vector machines

• Based on two important technical ideas
• some linear separators are better than others
• complex separators in the original space are
  linear separators in a complex space
• Similar performance to multi-layer neural nets,
  but no local optima (much easier to apply)
• Computationally expensive, but manageable

26
SVMs Maximize the margin
27
SVMs Maximize the margin
28
SVMs Maximize the margin
29
SVMs Maximize the margin
30
SVMs Kernel space

• Data are not separable in one dimension

x
31
SVMs Kernel space

• Data are separable in ltx, x2gt space

x2
x
32
SVMs Kernel space

• Data are separable in ltx, x2gt space

x2
x
33
SVMs Kernel space

• Clever mathematical kernel trick allows us to
  find
• maximum margin separator
• in many high or infinite-dimensional spaces
• in time (empirically) polynomial in the number of
  data points
• no local optima!

34
Framing a Problem

• Clever human needs to
• decide what information about the examples is
  important to the prediction problem
• choose an encoding of that information
• choose a class of hypotheses that is likely to
  contain a reasonably good solution
• gather training data
• set parameters in the learning algorithm
• finally, run the learning algorithm on the data
  to get a prediction rule

35
So is all of learning well solved?
36
Personal assistant that learns

• You have a meeting with Bob at 2 today. Are you
  going to talk about the budget or the system
  architecture?
• Architecture. Could you get me the architecture
  slide Alice showed at last weeks design meeting?
  
• Here it is. By the way, you might want to
  consider taking the metro to the meeting it
  looks like traffic is badly backed up downtown.
• I never take the metro. But maybe you can
  suggest the best driving directions given the
  current traffic.
• It looks like going around the beltway is the
  best option

37
PAL has to learn

• Your world meetings, projects, participants,
  rooms, locations, topics, and how they are all
  related
• How to perceptually recognize people, locations,
  activities people are engaging in
• How to extract information from documents about
  your world
• Your practices and preferences how you commute,
  what the standard office procedures are

38
What makes learning in PAL hard?

• Huge sets of labeled training examples not
  available
• Large variety of possible tasks
• Learning problems not framed explicitly in
  advance by humans

39
Four technological problems

• learning with much less labeled data
• using richer representations of situations and
  hypotheses
• learning to behave in complex environments
• life-long and life-sized learning

40
Four technological problems

• learning with much less labeled data
• using richer representations of situations and
  hypotheses
• learning to behave in complex environments
• life-long and life-sized learning

41
Using unlabeled data

• Most of generalization depends on a notion of
  distance similar objects should have similar
  properties
• Labeled data is expensive unlabeled data is
  cheap
• Use unlabeled data to learn underlying properties
  of the data space

42
Unlabeled data
43
Underlying manifold
44
A few labeled examples
45
Generalization
46
Four technological problems

• learning with much less labeled data
• using richer representations of situations and
  hypotheses
• learning to behave in complex environments
• life-long and life-sized learning

47
Learning richer knowledge

• Standard learning methods do not represent or
  learn relational knowledge
• Susan is Joes boss
• Susan works on the CALO project
• A purchase order must be signed by the
  purchasers boss
• Its okay to schedule a meeting of a project if
  all but one of its members can attend

48
Learning logical representations

• Vector-space representation is effective but
  limiting
• Inductive logic programming learns logical rules
• alpha0(A,B) -
• position(A,E,O), not_aromatic(O),
• small_or_polar(O), position(A,B,C),
  very_hydrophobic(C),
• can_sign_proposal(A, B) -
• higher_in_org_chart(A, B),
• no_conflict_of_interest(A, B)

49
Learning probabilistic representations

• Bayesian networks represent probabilistic
  relations among state variables
• gene regulatory networks
• observations, locations in mapping
• diseases and symptoms, computer diagnosis,

50
Probabilistic relational models

• Learn uncertain relations between properties of
  individuals, independent of particular
  individuals

51
Instantiating the model

• boss(John) Jane
• boss(Mary) Jane
• boss(Jane) Pat

John is rich
John is happy
Jane is rich
Jane is happy
Mary is happy
Pat is happy
Mary is rich
52
Broad range of applications

• disease transmission
• paper citations
• vehicle tracking
• gene expression

53
Dynamics of complex worlds

• Agents actions change relations among objects
• call(Person, Message)
• urgent(Message),inMeeting(Person) ?

0.7 knows(Person, Message), happy(Person) 0.2
knows(Person, Message), angry(Person) 0.05 angry(P
erson) 0.05 nothing changes
54
Four technological problems

• learning with much less labeled data
• using richer representations of situations and
  hypotheses
• learning to behave in complex environments
• life-long and life-sized learning

55
Reinforcement learning

• given a connection to the environment
• find a behavior that maximizes long-run
  reinforcement

State
Reinforcement
Observation
Action
56
Why reinforcement learning?

• Supervision signal is rarely available to agents
• Reward is easier than behavior for humans to
  specify
• for transmitting message
• - for emailing
• - - for phoning in office
• - - - for phoning during meeting

57
Reinforcement learning is hard

• less information per training instance
• requires active exploration
• actions have long-term consequences
• on-line performance important

58
RL successes

• backgammon player ties human world chamption
• elevator scheduling
• cell-phone channel allocation
• network routing
• All in simulation
• need too much data for online use in real domains

59
Learning by watching

• Initial exploration phases intolerably long
• Get help from humans
• built-in reflexes
• demonstration
• leading the robot by the hand
• declarative advice

60
Learning a world model

• Let your hypotheses die in your stead. Popper
• learn a model
• mentally simulateexperience
• no bad consequences
• transfer to other tasks

61
Four technological problems

• learning with much less labeled data
• using richer representations of situations and
  hypotheses
• learning to behave in complex environments
• life-long and life-sized learning

62
Lifelong learning

• Knowledge learned today provides inductive
  leverage for learning tomorrow
• Once agent can recognize people, it can analyze
  social structure of meetings
• Once agent learns the boss relation, it can
  generalize rule for who can sign travel claims

63
Learning tasks interconnect

• Natural language techniques to extract relational
  information from text
• Prior information about relations helps
  disambiguate text
• Perceptual recognition should be primed by
  expectations from relational knowledge (relative
  positions of objects, attendees of meetings)

64
Learning to live

• You are simultaneously trying to
• remain nourished
• retain your job
• have a good time
• not fall asleep in this talk
• You know about
• dancing
• differential equations
• donuts
• dinosaurs

65
Learning to live

• You are simultaneously trying to
• remain nourished
• retain your job
• have a good time
• not fall asleep in this talk
• You know about
• dancing
• differential equations
• donuts
• dinosaurs

How can you possibly decide what to do next?
66
Performance curves
computers
performance
humans
life
domain complexity
67
Dynamic problem reformulation
tractable sub-problem
perception
action
68
Multiple-resolution plans
Fine view of near-term high-probability
events Coarse view of distant low-probability
events
69
Learning applications
70
Learning to perceive

• Virtually every perceptual system built today
  relies on learning
• vision person and face recognition,
  segmentation, activity recognition
• speech recognition of phonemes, words, sentence
  structures
• language grammatical models, word-sense
  disambiguation, named-entity extraction
• None of these systems could have been built by
  hand.

71
Learning in computer systems

• active document retrieval
• repetitive editing by example
• scheduling machine instructions to optimize
  program execution time
• adaptive routing
• system policies
• when to spin hard disk up /down
• when to turn wireless transmitter on/off
• where to cache data in distributed system

72
Learning is the future

• Learning techniques will be a basis for every
  application that involves a connection to a real
  world
• Basic learning algorithms are ready for use in
  limited applications today
• Prospects for broader future application make for
  exciting fundamental research and development
  opportunities

73
(No Transcript)