CSE 574: Artificial Intelligence II Statistical Relational Learning

1
CSE 574 Artificial Intelligence IIStatistical
Relational Learning

• Instructor
• Pedro Domingos

2
Logistics

• Instructor Pedro Domingos
• Email pedrod_at_cs.washington.edu
• Office hours Wednesdays 300-350, CSE 648
• TA Aniruddh Nath
• Email nath_at_cs.washington.edu
• Office hours Mondays 300-350, CSE 216
• Web http//www.cs.washington.edu/574
• Mailing list cse574a_au11_at_uw.edu

3
Source Materials

• TextbookP. Domingos D. Lowd,Markov Logic An
  Interface Layer for AI, Morgan Claypool, 2009
• Papers
• SoftwareAlchemy, etc.
• Models, datasets, etc.Alchemy Web site
  (alchemy.cs.washington.edu)

4
Evaluation

• Seminar (Pass/Fail)
• Project (100 of grade)
• Proposals due October 19
• Progress report due November 16
• Presentation in class
• Final report due December 7
• Conference submission Winter 2012 (!)

5
Possible Projects

• Apply SRL to problem youre interested in
• Develop new SRL algorithm
• Other

6
What Is StatisticalRelational Learning?

• A unified approach to AI/ML
• Combines first-order logic and probabilistic
  models
• Example Markov logic
• Syntax Weighted first-order formulas
• Semantics Templates for Markov nets
• Inference Logical and probabilistic
• Learning Statistical and ILP

7
Why Take this Class?

• Powerful set of conceptual tools
• New way to look at AI/ML
• Powerful set of software tools
• Increase your productivity
• Attempt more ambitious applications
• Powerful platform for developing new learning and
  inference algorithms
• Many fascinating research problems

Caveat Not mature!
8
Sample Applications

• Information extraction
• Entity resolution
• Link prediction
• Collective classification
• Web mining
• Natural language processing

• Computational biology
• Social network analysis
• Robot mapping
• Activity recognition
• Personal assistants
• Probabilistic KBs
• Etc.

9
Overview of the Class

• Background
• Representation
• Inference
• Learning
• Extensions
• Applications
• Your projects

10
Background

• Markov networks
• Representation
• Inference
• Learning
• First-order logic
• Representation
• Inference
• Learning (a.k.a. inductive logic programming)

11
Representation

• Alphabet soup
• Markov logic
• Properties
• Relation to first-order logicand statistical
  models

12
Inference

• Basic MAP and conditional inference
• The MC-SAT algorithm
• Knowledge-based model construction
• Lazy inference
• Lifted belief propagation
• Probabilistic theorem proving

13
Learning

• Weight learning
• Generative
• Discriminative
• Incomplete data
• Structure learning and theory revision
• Statistical predicate invention
• Transfer learning

14
Extensions

• Continuous domains
• Infinite domains
• Recursive MLNs
• Relational decision theory

15
Applications

• (Sampled according to your interests)

16
Your Projects

• (TBA)

17
Class begins here.
18
AI The First 100 Years
IQ
Human Intelligence
Artificial Intelligence
1956
2056
2006
19
AI The First 100 Years
IQ
Human Intelligence
Artificial Intelligence
1956
2056
2006
20
AI The First 100 Years
Artificial Intelligence
IQ
Human Intelligence
1956
2056
2006
21
The Interface Layer
Applications
Interface Layer
Infrastructure
22
Networking
WWW
Email
Applications
Internet
Interface Layer
Protocols
Infrastructure
Routers
23
Databases
ERP
CRM
Applications
OLTP
Interface Layer
Relational Model
Transaction Management
Infrastructure
Query Optimization
24
Programming Systems
Programming
Applications
Interface Layer
High-Level Languages
Compilers
Code Optimizers
Infrastructure
25
Hardware
Computer-Aided Chip Design
Applications
Interface Layer
VLSI Design
Infrastructure
VLSI modules
26
Architecture
Operating Systems
Applications
Compilers
Interface Layer
Microprocessors
ALUs
Infrastructure
Buses
27
Operating Systems
Applications
Software
Interface Layer
Virtual Machines
Infrastructure
Hardware
28
Human-Computer Interaction
Applications
Productivity Suites
Interface Layer
Graphical User Interfaces
Infrastructure
Widget Toolkits
29
Artificial Intelligence
Planning
Robotics
Applications
NLP
Multi-Agent Systems
Vision
Interface Layer
Representation
Inference
Infrastructure
Learning
30
Artificial Intelligence
Planning
Robotics
Applications
NLP
Multi-Agent Systems
Vision
Interface Layer
First-Order Logic?
Representation
Inference
Infrastructure
Learning
31
Artificial Intelligence
Planning
Robotics
Applications
NLP
Multi-Agent Systems
Vision
Interface Layer
Graphical Models?
Representation
Inference
Infrastructure
Learning
32
Logical and Statistical AI
Field Logical approach Statistical approach
Knowledge representation First-order logic Graphical models
Automated reasoning Satisfiability testing Markov chain Monte Carlo
Machine learning Inductive logic programming Neural networks
Planning Classical planning Markov decision processes
Natural language processing Definite clause grammars Prob. context-free grammars
33
We Need to Unify the Two

• The real world is complex and uncertain
• Logic handles complexity
• Probability handles uncertainty

34
Artificial Intelligence
Planning
Robotics
Applications
NLP
Multi-Agent Systems
Vision
Interface Layer
Statistical Relational Learning
Representation
Inference
Infrastructure
Learning