GraphBased Concept Learning

1
Graph-Based Concept Learning

• Jesus Gonzalez, Lawrence Holder and Diane Cook
• Department of Computer Science and Engineering
• The University of Texas at Arlington

2
Outline

• Relational concept learning
• Graph-based concept learning
• Conceptual graphs and Galois lattice
• Graph-based discovery in Subdue
• SubdueCL
• Empirical results
• Conclusions

3
Relational Concept Learning

• Inductive Logic Programming (ILP)
• FOIL
• Progol
• First-order logic vs. graphs
• Expressiveness
• Interpretability
• Conceptual graphs

4
Conceptual Graphs

• Logic-based knowledge representation

shape(X,triangle) shape(Y,square) on(X,Y)
5
Conceptual Graphs

• Graph Logic
• PAC-learning CGs Jappy Nock
• Size of CG class and generalization (projection)
  operator polynomial in
• Number of relations
• Number of concepts
• Number of labels

6
Galois Lattice

• Each node consists of a description graph and set
  of subsumed examples
• Begins with positive examples
• Generalization operator
• Most specific generalization
• Union of example sets

7
Galois Lattice
, 3, 4 2, 3, 4 1, 2, 3,
4
1, 2
3, 4 1, 2, 3 1, 2, 4
1
1, 2
1, 2
triangle
triangle
triangle
triangle
triangle
triangle
triangle
triangle
circle
circle
triangle
triangle
triangle
on
on
triangle
triangle
on
on
on
on
on
on
on
on
on
on
on
on
on
rectangle
rectangle
on
on
square
square
rectangle
rectangle
triangle
triangle
on
on
8
Galois Lattice

• Galois lattice creation O(n3p)
• n examples
• p nodes in lattice
• Tractable for poly-time generalization
• GRAAL system

9
Graph-Based Discovery

• Finding interesting and repetitive
  substructures (connected subgraphs) in data
  represented as a graph

10
Graph-Based Discovery

• Interesting defined according to the Minimum
  Description Length principle
• min DL(S) DL(GS)
• General-to-specific beam search through
  substructure space
• Poly-time inexact graph match
• Subdue system

S
11
Subdue System

• Graph-based
• Discovery
• Concept learning
• Hierarchical conceptual clustering
• Background knowledge
• Parallel/distributed capability
• http//cygnus.uta.edu/subdue

12
Graph-Based Concept Learning
13
Graph-Based Concept Learning

• Extension to graph-based discovery
• Input now a set of positive graphs and a set of
  negative graphs
• Set-covering approach
• Iterate until all positive graphs and no negative
  graphs covered
• Result is a substructure DNF

14
Graph-Based Concept Learning

• Solution 1
• Find substructure compressing positive graphs,
  but not negative graphs
• Compress graphs and iterate until no further
  compression
• Problem
• Compressing, instead of removing,
  partially-covered positive graphs leads to
  overly-specific hypotheses

15
Graph-Based Concept Learning

• Solution 2
• Find substructure covering positive graphs, but
  not negative graphs
• Remove covered positive graphs and iterate until
  all covered
• Substructure value 1 - Error

16
Empirical Results

• Comparison with ILP systems
• Non-relational domains from UCI repository

17
Empirical Results

• Comparison with ILP systems
• Relational domains Chess endgame

lt
BKR
lt
WKC
adj
pos
pos
eq
WK
BKC
BK
WKR
adj
adj
lt
WR
adj
lt
WRR
WRC
pos
18
Empirical Results Chess

• FOIL 11 rules, 99.34
• Progol 5 rules, 99.74
• SubdueCL 7 rules, 99.74

lt
lt
adj
adj
WKC
WKC
BKC
BKC
WKC
WKC
BKC
BKC
pos
pos
pos
pos
adj
adj
adj
adj
WKR
WKR
BKR
BKR
WKR
WKR
BKR
BKR
19
Empirical Results

• Relational domain Cancer
• SubdueCL 62 Progol 64 72

20
Empirical Results

• Relational domain Web
• Professor () vs. student (-) websites
• Hyperlink structure and page content

21
Empirical Results

• Relational domain Web
• Computer store () vs. professor (-) websites
• Hyperlink structure only

22
Conclusions

• Theoretical analysis of graph-based concept
  learning
• PAC-learning conceptual graphs
• Galois lattice
• Next step Relax graph constraints
• Empirical analysis
• Competitive with other relational concept
  learners (ILP)
• Next step More relational domains
• SubdueCL (http//cygnus.uta.edu/subdue)