RDM Chapter 4: Representations

1
RDM Chapter 4 Representations

• prepared for COMP422/522-2008, Bernhard Pfahringer

2
Overview

• right representation is essential
• hierarchy of representations
• integrated framework including
• propositionalization
• aggregates

3
Learning from entailment

• Le and Lh are logical formulas and c(H,e) true
  if and only if H e
• e.g.
• e flies ? black,bird,hasFeathers,
• hasWings,normal,laysEggs
• H flies ? bird,normal
• flies ? insect,hasWings,normal
• then H e, as H covers e

4
Learning from interpretation

• Lh set of logical formulas
• Le set of (Herbrand) interpretations (aka
  possible worlds)
• c(H,e) true if e is a model of H
• e black,bird,hasFeathers, hasWings,normal,laysEg
  gs

5
Subtle differences

• Interpretations everything not listed is assumed
  false, but all predicates are equal
• clustering. association rule mining
• Entailment not such assumption, but one
  predicate is special (flies ? )
• concept learning

6
Attribute-value learning

• single-table single-tupel assumption
• r(v1,,vN)
• class(vN) ? r(v1,,vN-1)
• assume exactly one value each (4.1/4.2)
• see playTennis example p.75
• boolean and av representation called
  propositional
• exercise 4.7

7
Multi-instance learning

• important position between propositional and
  (multi-)relational
• single-table multi-tuple assumption
• asymmetric bag is positive if at least one bag
  member is positive
• e.g. conformations of compounds
• exercise 4.9

8
Relational learning

• multiple tables
• related to ER model in databases
• Pno ? participant(N,J,C,P),
• sub(N,C1), course(C1,L,advanced)
• attends(adams) ?
• participant(adams,researcher,scuf),
  sub(adams,erm), sub(adams,so2), ..
• see examples 4.10,4.11, exercise 4.12

9
Logic Programs

• most general, automatic programming
• generate programs from IO description
• hard because of need for subroutines
• allows for functors and recursion
• huge space, undecidable
• limit for pragmatic reasons

10
Specialised representations (skip)

• 4.6 Sequences, lists, grammars
• 4.7 Trees, terms
• 4.8 Graphs

11
Background knowledge

• c(H,B,e) true iff H ? B e
• examples 4.26, 4.27
• global info for all examples
• - cover computation can be much more expensive
• intensional part of the db (views)

12
Background knowledge and learning from
interpretation

• covers(H,B,e) true if minimal Herbrand model
  M(e ? B) is also a model of H
• examples 4.29, 4.30, 4.31
• again more expressive, but also more expensive to
  compute
• skip 4.10 DIY

13
Hierarchy of representations

• what is the relationship between BL, AV, MI, RL,
  and LP?
• uniform h ? l1, , lN with appropriate
  constraints for each
• representable correct and complete hypothesis
  exists
• reducible X ? Y, if two functions exists for
  mapping h and e, and if h covers e then also
  fh(h) covers fe(e)
• if bijective, then can also map back hypothesis

14
easy direction

• BL ? AV ? MI ? RL ? LP
• f is simple identity function
• more of theoretical interest, would be
  inefficient in practise
• other direction more interesting
• propositionalization incomplete, lossy
  reduction

15
AV ? BL ?

• AV can be reduced to BL for nominal values, but
  constraints are lost
• turn outlook sunny
• into outlookIsSunny true
• but loose knowledge that exactly one of
• outlookIsSunny,
• outlookIsRainy,
• outlookIsOvercast
• must be true
• What about numbers?

16
MI ? AV ?

• naïve reduction concatenate examples/tupels
• o(red,triangle), o(blue,circle) into
• o2(red,triangle,blue,circle), but also
• o2(blue,circle,red,triangle)
• same for rules, I.e. unorderedness leads to
  exponential explosion must also assume max
  number of tupels and pad concatenation of
  smaller bags

17
MI ? AV ?

• alternative enumerate ALL possible rules, use
  each as a boolean feature
• o(X,Y),o(red,X),o(blue,X),o(Y,circle),
  o(red,triangle), o(blue,circle)
• again, not very practical large number plus
  loose more general relationship (but )

18
RL ? MI ?

• must turn a collection of tables into a single
  table?
• Database theory universal relation/join
• see example 4.39
• again not practical in general, but see
  propositionalization

19
LP ? RL ?

• impossible LP is turing-complete, relational
  algebra (RL, SQL) is not
• but flattening can transform single LP clauses
  into RL
• member(X,cons(X,Y)) ?
• member(X,cons(Y,Z)) ? member(X,Z)
• member(X,V) ? pcons(X,Y,V)
• member(X,V) ? pcons(Y,Z,V), member(X,Z)
• pcons(X,Y,cons(X,Y)) ?

20
LP ? RL ?

• unflattening simply use resolution to undo
  flattening
• ground variant of flattening is often used for LP
  learning (ex.4.46)
• but flattening is NOT a proper reduction (see
  also ex.4.49)

21
Propositionalization

• proper reductions either do not exist (LP -gt RL)
  or are infeasible (RL -gt MI, MI -gt AV)
• P can be seen as approximate reduction
• will loose some information, but may still be
  sufficient, can then use any AV algorithm

22
Table-based approach

• start with some clause,
• simply collect all the substitutions which make
  the clause true into one table of variable
  bindings (cf ex.4.50)
• can result in either
• AV (only 11 and 1n relationships)
• MI (general case)

23
Query-based approach

• (cf MI -gt AV)
• must filter, often heuristically, e.g. must cover
  at least N examples, etc.
• other distinction static ltgt dynamic
• static up-front, one-shot
• dynamic on-demand generate more features (only
  very few papers so far )

24
Aggregation

• again from database theory
• sometimes summary info more interesting than
  details
• count, sum, avg, min, max,
• (see Figure 4.12 / 4.13)
• again, large search space, plus problem with
  generality (Chapter 5)
• but very useful for MI -gt AV (even Weka supplies
  Relaggs filter)