Discriminative Probabilistic Models for Relational Data

1
Discriminative Probabilistic Models for
Relational Data

• Ben Taskar, Pieter Abbeel, Daphne Koller

2
Tradition statistic classification Methods

• Dealing with only flat data IID
• In many supervised learning tasks, entities to be
  labeled are related to each other in complex way
  and their labels are not independent
• This dependence is an important source of
  information to achieve better classification

3
Collective Classification

• Rather than classify each entity separately
• Simultaneously decide on the class label of all
  the entities together
• Explicitly take advantage of the correlation
  between the labels of related entitiies

4
Undirected vs. directed graphical models

• Undirected graphical models do not impose the
  acyclicity constraint, but directed ones need
  acyclicity to define a coherent generative model
• Undirected graphical models are well suited for
  discriminative training, achieving better
  classification accuracy over generative training

5
Our Hypertext Relational Domain
Label
Label
...
...
HasWord1
HasWordk
HasWord1
HasWordk
Doc
Doc
From
To
Link
6
Schema

• A set of entity types
• Attribute of each entity type
• Content attribute E.X
• Label attribute E.Y
• Reference attribute E.R

7
Instantiation

• Provide a set of entities I (E) for each entity
  type E
• Specify the values of all the attribute of the
  entities, I.x, I.y, I.r
• I.r is the instantiation graph, which is call
  relational skeleton in PRM

8
Markov Network

• Qualitative component Cliques
• Quantitative component Potentials

9
Cliques

• A set of nodes in the graph G such that
• for each are connected by an
• edge in G

10
Potentials

• The potential for the clique c defines the
  compatibility between values of variables in the
  clique
• Log-linearly combination of a set of features

11
Probability in Markov Network

• Given the values of all nodes in the Markov
  Network

12
Conditional Markov Network

• Specify the probability of a set of target
  variables Y given a set of conditioning variables
  X

13
Relational Markov Network (RMN)

• Specifies the conditional probability over all
  the labels of all the entities in the
  instantiation given the relational structure and
  the content attributes
• Extension of the Conditional Markov Networks with
  a compact definition on a relational data set

14
Relational clique template

• F --- a set of entity variables (From)
• W--- the condition about the attributes of the
  entity variables (Where)
• S --- subset of attributes (content and label
  attribute) of the entity variables (Select)

15
Relationship to SQL query

• SELECT doc1.Category,doc2.Category
• FROM doc1,doc2,Link link
• WHERE link.Fromdoc1.key and link.Todoc2.key

Doc1
Doc2
Doc1
Link
16
Potentials

• Potentials are defined at the level of relational
  clique template
• The cliques of the same relational clique
  template have the same potential functions

17
Unrolling the RMN

• Given an instantiation of a relational schema,
  unroll the RMN as follows
• Find all the cliques in the unrolled the
  relational schema where the relational clique
  templates are applicable
• The potential of a clique is the same as that of
  the relational clique template which this clique
  belongs to

18
link1
Doc1
Doc2
link2
Doc3
19
Probability in RMN
20
(No Transcript)
21
Learning RMN

• Given a set of relational clique templates
• Estimate feature weight w using conjugate
  gradient
• Objective function--Product of likelihood of
  instantiation and parameter prior
• Assume a shrinkage prior over feature weights

22
Learning RMN (Contd)

• The conjugate gradient of the objective function
• where

23
Inference in RMN

• Exact inference
• Intractable due to the network is very large and
  densely connected
• Approximate inference
• Belief propagation

24
Experiments

• WebKB dataset
• Four CS department websites
• Five categories (faculty,student,project,course,ot
  her)
• Bag of words on each page
• Links between pages
• Experimental setup
• Trained on three universities
• Tested on fourth

25
Flat Models

• Based only on the text content on the WebPages
• Incorporate meta-data

26
Relational model

• introduce relational clique template over the
  labels of two pages that are linked

Doc2
Doc1
Link
27
Relational model (Contd)

• relational clique template over the label of
  section and the label of the pages it is on
• Relational clique template over the label of the
  section containing the link and the label of the
  target page

28
(No Transcript)
29
Discriminative vs. Generative

• ExitNaïve Bayes a complete generative model
  proposed by Getoor et al
• Exitlogistic using logistic regression for the
  conditional probability distribution of page
  label given words
• Link a fully discriminative training model

30
Thank You!