Prof' Dr' Luc De Raedt,

1
Probabilistic Logic Learning
The Combination of Probability, Logic, and
Learning

• Prof. Dr. Luc De Raedt,
• Dipl.- Inf. Kristian Kersting, Dipl-Inf. Albrecht
  Zimmermann

2
Overview

• Objectives
• Traditional approaches
• Real world
• Example 1 Linked Bibliographic Data
• Example 2 Protein Fold Classification
• Example 3 Metabolic Pathways
• Probabilistic Logical Models
• Why using? Why learning?
• Organization

3
Objectives
One of the key open questions of artificial
intelligence concerns "probabilistic logic
learning",
i.e. the integration of probabilistic reasoning
with
first order logic representations and
machine learning.
4
Overview

• Objectives
• Traditional approaches
• Real world
• Example 1 Linked Bibliographic Data
• Example 2 Protein Fold Classification
• Example 3 Metabolic Pathways
• Probabilistic Logical Models
• Why using? Why learning?
• Organization

5
Bayesian Networks
Pearl 91
Compact representation of joint probability
distributions

• Qualitative part
• Directed acyclic graph (DAG)
• Nodes - random vars.
• Edges - direct influence

Traditional Approaches
6
Hidden Markov Models
Rabiner 89

• Finite number of states

coin1
coin2
Traditional Approaches
7
Logic (Programming)
connected(bond_street,oxford_circus,central). conn
ected(oxford_circus,tottenham_court_road,central).
connected(bond_street,green_park,jubilee). connec
ted(green_park,charing_cross,jubilee). connected(g
reen_park,piccadilly_circus,piccadilly). connected
(piccadilly_circus,leicester_square,piccadilly). c
onnected(green_park,oxford_circus,victoria). conne
cted(oxford_circus,piccadilly_circus,bakerloo). co
nnected(piccadilly_circus,charing_cross,bakerloo).
connected(tottenham_court_road,leicester_square,n
orthern). connected(leicester_square,charing_cross
,northern).
Two stations are nearby if they are on the same
line with at most one other station in between
nearby(bond_street,oxford_circus).nearby(oxford_
circus,tottenham_court_road). or
better nearby(X,Y)-connected(X,Y,L).nearby(X,Y)
-connected(X,Z,L),connected(Z,Y,L).
Traditional Approaches
8
Overview

• Objectives
• Traditional approaches
• Real world
• Example 1 Linked Bibliographic Data
• Example 2 Protein Fold Classification
• Example 3 Metabolic Pathways
• Probabilistic Logical Models
• Why using? Why learning?
• Organization

9
Linked Bibliographic Data
Random samples of homogeneous objects from
single relation
Logistic Regression, SVMs,
Naive Bayes,
Real World
10
Linked Bibliographic Data
P2
paper
P1
author
institution
P3
I1
citation
co-citation
A1
author-of
P4
author-affiliation
Attributes/classes

• Multi-relational, heterogeneous
• and semi-structured

Real World
11
Linked Bibliographic Data
Authors
Text/attributes
Citations
Real World
12
Protein Secondary Structure

• Fold classification /prediction
• Fold comparison
• Homology modeling

strand(s1,null,2).helix(h(right,alpha),6).strand(
s1,plus,2). helix(h(right,alpha),4).strand(s1,p
lus,2).helix(h(right,alpha),4).
strand(s1,plus,3). helix(h(right,alpha),4).strand(
s1,plus,1). helix(h(right,alpha),6).
Real World
13
Metabolic Pathways
reaction(Enz,Co,Env,X,X-h(N)) -
reductase(Enz), cofactor(Enz,Co),
not(inhibitor(Enz,Inh),in(inh,Env)),prop1(X).
reaction(Enz,Co,Env,X-p-o(3),X-h) -
pjosphotranspherase(Enz), cofactor(Enz,Co),
not(inhibitor(Enz,Inh),in(inh,Env)),prop2(X).
Real World
14
Overview

• Objectives
• Traditional approaches
• Real world
• Example 1 Linked Bibliographic Data
• Example 2 Protein Fold Classification
• Example 3 Metabolic Pathways
• Probabilistic Logical Models
• Why using? Why learning?
• Organization

15
Probabilistic-Logical Models (PLMs)

• Combine (at least) relational logic and some
  probabilistic model
• Unique probability distribution
• Learning approaches

bt(X) father(F,X), bt(F),mother(M,X), bt(M).
PLMs
16
Why use PLMs?

• General regularities
• Explicit management of uncertainty
• Compression in number of parameters
• Prior background knowledge
• Comprehensible models
• Can be used to answer arbitrary queries

PLMs
17
Why use PLMs?

• Diagnosis
• Prediction
• Classification
• Decision-making
• Data mining

Web Mining
Computational Biology
Medicine
Speech recognition
Robotics
PLMs
18
Why Learning PLMs?

• Knowledge acquisition bottleneck

PLMs
19
Why Learning PLMs?

• Protein Fold Classification
• 140 parameters
• instead of 60000

PLMs
20
Why Learning PLMs?

• Knowledge acquisition bottleneck / data is cheap

• Combining domain expert knowledge with data
• Logical structure provides insight into domain
• Effective representation and inference
• Handling missing data author_of(p1)?

PLMs
21
Overview

• Objectives
• Traditional approaches
• Real world
• Example 1 Linked Bibliographic Data
• Example 2 Protein Fold Classification
• Example 3 Metabolic Pathways
• Probabilistic Logical Models
• Why using? Why learning?
• Organization

22
Seminar

• 3 credit points
• Two talks
• basic and (before christmas)
• advanced techniques (after christmas)
• Technical report covering both topics
  (around 10 pages)
• Collaborative talks possible
• Grade 2x25 talk, 25 report, 25 activity

Organization
23
Practical Course

• 6 credit points
• One project
• 1-2 page(s) problem specification
  (latest after 4 weeks)
• Presentation/Demonstration before and after
  Christmas
• Technical report (around 10 pages)
• Grade 25 demo, 25 Coding, 25 report, 25
  activity

Organization
24
Seminar Possible Topics
Organization
25
Practical Course Possible Topics

• PRMs nutshell in Prolog
• (Advanced) HMMs for information extraction
• Learning stochastic CFGs
• SLPs inference, parameter estimation
• PRISMs inference, parameter estimation
• Logical Markov Decision Programms (parameter
  estimation)
• ...

Organization