BLOG: Probabilistic Models with Unknown Objects

1
BLOG Probabilistic Models with Unknown Objects

• Brian Milch, Bhaskara Marthi, Stuart
  Russell,David Sontag, Daniel L. Ong, Andrey
  Kolobov
• University of California at Berkeley

2
Basic Task

• Given observations, make inferences about
  underlying objects
• Difficulties
• Dont know list of objects in advance
• Dont know when same object observed twice

(identity uncertainty / data association / record
linkage)
3
Handling Unknown Objects

• Standard practice special-purpose algorithms to
  resolve identity uncertainty
• Goal Resolve identity uncertainty by inference
  in probabilistic model
• Bayesian LOGic (BLOG) representation language
  for models with
• Unknown set of objects
• Unknown map from observations to objects

4
Outline

• Motivating applications
• Bayesian Logic (BLOG)
• Syntax
• Semantics
• Proof-of-concept experimental results

5
Example 1 Aircraft Tracking
DetectionFailure
6
Example 1 Aircraft Tracking
UnobservedObject
FalseDetection
7
Example 2 Bibliographies
8
Simple Example Balls in an Urn
P(n balls in urn)
P(n balls in urn draws)
Draws
(with replacement)
1
2
3
4
9
Possible Worlds
3.00 x 10-3
7.61 x 10-4
1.19 x 10-5


2.86 x 10-4
1.14 x 10-12


10
Distributions over First-Order Structures

• Idea goes back to Gaifman 1964
• Halpern 1990 defines language for stating
  constraints on such distributions
• But not specifying a distribution uniquely
• Logic programming approaches Poole 1993 Sato
  Kameya 2001 Kersting De Raedt 2001 define
  unique distributions, but assume unique names and
  domain closure
• PRMs Koller Pfeffer 1998 have special
  constructs for number uncertainty, existence
  uncertainty
• BLOG Unified syntax for distributions over
  worlds with
• Varying sets of objects
• Varying mappings from observations to objects
• See also MEBN Laskey and da Costa, UAI 2005

11
Generative Process for Possible Worlds
Draws
(with replacement)
1
2
3
4
12
BLOG Model for Urn and Balls

• type Color type Ball type Draw
• random Color TrueColor(Ball)random Ball
  BallDrawn(Draw)random Color ObsColor(Draw)
• guaranteed Color Blue, Greenguaranteed Draw
  Draw1, Draw2, Draw3, Draw4
• Ball Poisson6()
• TrueColor(b) TabularCPD0.5, 0.5()
• BallDrawn(d) UniformChoice(Ball b)
• ObsColor(d) if (BallDrawn(d) ! null) then
  NoisyCopy(TrueColor(BallDrawn(d)))

13
BLOG Model for Urn and Balls

• type Color type Ball type Draw
• random Color TrueColor(Ball)random Ball
  BallDrawn(Draw)random Color ObsColor(Draw)
• guaranteed Color Blue, Greenguaranteed Draw
  Draw1, Draw2, Draw3, Draw4
• Ball Poisson6()
• TrueColor(b) TabularCPD0.5, 0.5()
• BallDrawn(d) UniformChoice(Ball b)
• ObsColor(d) if (BallDrawn(d) ! null) then
  NoisyCopy(TrueColor(BallDrawn(d)))

header
number statement
dependencystatements
14
BLOG Model for Urn and Balls

• type Color type Ball type Draw
• random Color TrueColor(Ball)random Ball
  BallDrawn(Draw)random Color ObsColor(Draw)
• guaranteed Color Blue, Greenguaranteed Draw
  Draw1, Draw2, Draw3, Draw4
• Ball Poisson6()
• TrueColor(b) TabularCPD0.5, 0.5()
• BallDrawn(d) UniformChoice(Ball b)
• ObsColor(d) if (BallDrawn(d) ! null) then
  NoisyCopy(TrueColor(BallDrawn(d)))

?
Identity uncertainty BallDrawn(Draw1)
BallDrawn(Draw2)
15
BLOG Model for Urn and Balls

• type Color type Ball type Draw
• random Color TrueColor(Ball)random Ball
  BallDrawn(Draw)random Color ObsColor(Draw)
• guaranteed Color Blue, Greenguaranteed Draw
  Draw1, Draw2, Draw3, Draw4
• Ball Poisson6()
• TrueColor(b) TabularCPD0.5, 0.5()
• BallDrawn(d) UniformChoice(Ball b)
• ObsColor(d) if (BallDrawn(d) ! null) then
  NoisyCopy(TrueColor(BallDrawn(d)))

Arbitrary conditionalprobability distributions
CPD arguments
16
BLOG Model for Urn and Balls

• type Color type Ball type Draw
• random Color TrueColor(Ball)random Ball
  BallDrawn(Draw)random Color ObsColor(Draw)
• guaranteed Color Blue, Greenguaranteed Draw
  Draw1, Draw2, Draw3, Draw4
• Ball Poisson6()
• TrueColor(b) TabularCPD0.5, 0.5()
• BallDrawn(d) UniformChoice(Ball b)
• ObsColor(d) if (BallDrawn(d) ! null) then
  NoisyCopy(TrueColor(BallDrawn(d)))

Context-specific dependence
17
BLOG Model for Urn and Balls

• type Color type Ball type Draw
• random Color TrueColor(Ball)random Ball
  BallDrawn(Draw)random Color ObsColor(Draw)
• guaranteed Color Blue, Greenguaranteed Draw
  Draw1, Draw2, Draw3, Draw4
• Ball Poisson6()
• TrueColor(b) TabularCPD0.5, 0.5()
• BallDrawn(d) UniformChoice(Ball b)
• ObsColor(d) if (BallDrawn(d) ! null) then
  NoisyCopy(TrueColor(BallDrawn(d)))

18
Generative Process for Aircraft Tracking
Existence of radar blips depends on existence
and locations of aircraft
Sky
Radar
19
BLOG Model for Aircraft Tracking
Source
a

• Aircraft NumAircraftDistrib()
• State(a, t) if t 0 then InitState() else
  StateTransition(State(a, Pred(t)))
• Blip (Source, Time) -gt (a, t)
  NumDetectionsDistrib(State(a, t))
• Blip (Time) -gt (t) NumFalseAlarmsDistrib()
  
• ApparentPos(r)if (Source(r) null) then
  FalseAlarmDistrib()else ObsDistrib(State(Source
  (r), Time(r)))

Blips
2
t
Time
Blips
2
t
Time
20
Declarative Semantics

• What is the set of possible worlds?
• What is the probability distribution over worlds?

21
What Exactly Are the Objects?

• Objects are tuples that encode generation history
• Aircraft (Aircraft, 1), (Aircraft, 2),
• Blip from (Aircraft, 2) at time 8 (Blip,
  (Source, (Aircraft, 2)), (Time, 8), 1)

22
Basic Random Variables (RVs)

• For each number statement and tuple of generating
  objects, have RV for number of objects generated
• For each function symbol and tuple of arguments,
  have RV for function value
• Lemma Full instantiation of these RVs uniquely
  identifies a possible world

23
Another Look at a BLOG Model

• Ball Poisson6()
• TrueColor(b) TabularCPD0.5, 0.5()
• BallDrawn(d) UniformChoice(Ball b)
• ObsColor(d) if !(BallDrawn(d) null) then
  NoisyCopy(TrueColor(BallDrawn(d)))

Dependency and number statements define CPDs for
basic RVs
24
Just a Bayes Net?
Ball
TrueColor(B1)
TrueColor(B2)
TrueColor(B3)

ObsColor(D1)
Infinite parent set
BallDrawn(D1)
Standard BN results no longer apply
25
Probability Distribution

• BLOG model specifies
• Conditional distributions for basic RVs
• Factorization properties for certain finite
  instantiations of basic RVs
• Theorem Under certain conditions (analogous to
  BN acyclicity), every BLOG model defines unique
  distribution over possible worlds

26
Inference

• Does infinite set of basic RVs prevent inference?
• No Sampling algorithm only needs to instantiate
  finite set of relevant variables
• Algorithms
• Rejection sampling this paper
• Guided likelihood weighting Milch et al.,
  AI/Stats 2005
• Theorem For large class of BLOG models, sampling
  algorithms converge to correct probability for
  any query, using finite time per sampling step

27
Proof-Of-Concept Experiment

• Given 10 draws, all appearing blue
• 5 runs of 100,000 samples each

prior
posterior
28
Conclusions

• Bayesian logic (BLOG) models define unique
  distributions over first-order model structures
  with
• Varying sets of objects
• Varying mappings from terms to objects
• Future work
• Practical inference algorithms
• Applications to text understanding
• Applications to situation awareness (DBLOG)