Relational Preference for Control

1
Relational Preference for Control

• Ronen Brafman
• Department of Computer Science Ben-Gurion
  University

2
Decision Making Contexts Two Extremes

• Online applications
• Lay users
• Preference elicitation interface must be based on
  very simple natural language expressions
• Users will spend at most a few minutes, usually
  less
• Decision analysis context
• Facilitated by a human expert
• Complex modeling that requires intelligent
  framing
• Quantitative model
• Considerable effort

3
A Third Context System Control

• Goal Design a system that behaves well in a
  large and diverse number of settings that cannot
  all be foreseen offline.
• A generic preference model is used by the system
  to select among alternative choices
• The designer is willing to spend time
  constructing the model
• Yet, the language must be intuitive so that
• The specification process is convenient and
  accessible
• The specification is easy to modify

4
Decision-Theoretic System Design

• Not a new idea
• See Russell and Norvigs AI A Modern Approach
• Recent successful autonomous vehicles
• A purist approach
• Maintain a probability distribution, utility
  function
• Select actions that maximize expected utility

5
Decision-Theoretic System Design Utilities Get
the Short End

• Work on probabilistic reasoning is far ahead
• Relational and object-oriented probabilistic
  models
• Inference algorithms
• Learning methods
• Good progress on the utility/preference
  specification side, but not on par with the
  probabilistic models

6
Some Possible Reasons

• Historic lag
• Machine learning
• Probably the most important application area of
  AI
• A lot of data
• Great need for modeling it

7
What About Preference Models?

• Modeling preferences is harder
• It is harder to introspect about preferences
• It is harder to learn preferences little data
  if any
• Can we learn a preference model for a new
  application?
• Still - as system become more complicated, the
  need for tools that guide them is more pressing
• We need to provide designers with convenient
  tools for describing complex control preferences

8
Rule-Based Systems

• Rule-based systems are intuitive
• Are often used in industry to describe complex
  control rules
• Are very rigid
• If the body is true, so is the head
• Certain contexts (constraints) can render them
  inconsistent
• Context sensitivity must be built in carefully
• Limited modularity

9
Utility Functions

• Immediate context awareness
• If some constraint holds, seek best feasible
  solution
• Additive forms (GAI) provide modularity
• Defined over a rigid set of propositions
• ? work with a rigid, predefined set of objects

10
Relational/OO Preference Rules

• Simple and intuitive like rule-based systems
• Induce a utility/value function for any given set
  of objects
• Simple additive semantics provides modularity

11
A Concrete Application

• Command and Control Monitoring
• High ranking officials monitoring masses of data
  in a real-time command control center
• Our goal decide which data to show at each point
  in time
• Video streams
• Sensor data
• Results of relevant queries
• Results of data analysis (e.g., simulations, risk
  assessment)

12
Example Fire Department

• Cameras on fireman helmet, fixed surveillance
  cameras
• Heat sensors, smoke detectors, co2-levels
• Area maps, building plans, driving distance,
  number of residents
• Simulation of structure strength, time to contain
  a fire as function of wind and other weather
  conditions, etc.

13
Requirements Object Oriented and Generic
Specification

• Fire departments in different cities have
  different personal, equipment
• Objects within a single department change
• New firemen
• New equipment
• Fire events naturally modeled as an object
• One preference specification should cover all

14
Modeling a Fire Department - I

• Object classes fireman, fire-engine, fire,
  camera, heat-sensor, co2-sensor
• Fireman attributes
• Name, Location, Rank, Role,
• Camera, heat-sensor,
• Camera attributes
• On
• Display
• URL

15
Modeling a Fire Department - II

• Rules
• Fireman(x) ? fire(y) ? x.location y.location
• ? x.camera.display on 4, off 0
• Fireman(x) ? x.co2-levelhigh
• ? x.camera.display on 8, off 0

16
More Formally

• Model includes
• Set of object classes
• Each class specifies object attributes and their
  type (and possibly class attributes)
• Set of preference rules of the form
• rule-body ? rule-head v1, w1 v2 w2vm wm

17
Preference Rules

• rule-body ? rule-head v1, w1 v2 w2vm wm
• Rule-body class1(x1) ? ? classk(xk) ? a1 (xi1)
  ? ? as (xis)
• al (xl) xi.path REL xj.path OR xi.path REL
  value
• REL lt, gt, , ?, etc.
• Path an attribute chain, such as
  x.mother.profession
• Rule-head xi.path
• vi a possible value of xi.path
• wi a real number

18
Controllable and Uncontrollable Attributes

• We distinguish between controllable attributes,
  to which we need to assign a value
• Example camera.display
• And uncontrollable, or context attributes
• Example. Fire.location
• The body may refer to both controllable and
  uncontrollable attributes
• The head contains a single controllable attribute

19
Informal Semantics

• Intuitive semantics the value of value vi for
  xi.path is wi when a1 ? ? al are satisfied
• A form of conditional, valued preference
• If you want, a relational UCP-net
• Preferences rules r set of object instances o
  assignment to uncontrollable (context) attributes
• ?
• utility function over all possible assignments
  to the controllable attributes of o.

20
Obtaining the Value Function

• Step 1 Given a set of objects, and a set of
  rules, generate all ground instances of the rules
• Step 2 Filter rule bodies based on the value of
  context attributes
• Remove satisfied conjuncts
• Eliminate rules with false conjuncts
• Step 3 We now have a set of ground rules
  containing only controllable attributes
• Step 4 The value of assignment p to the
  uncontrollable attributes is the sum of all
  weights associated with heads of unassigned rules
  whose body p satisfies

21
Example

• Consider the rules we saw before
• fireman(x) ? fire(y) ? x.location y.location
• ? x.camera.display on 4, off 0
• Fireman(x) ? x.co2-levelhigh
• ? x.camera.display on 8, off 0
• Objects fireman(Alice)
• Fireman(Alice) ? Alice.co2-levelhigh
• ? Alice.camera.display on 8, off 0
• Objects fireman(Alice),fire(fire1)
• fireman(Alice) ? fire(fire1) ? Alice.location
  WTC.location
• ? Alice.camera.display on 4, off 0
• Fireman(Alice) ? Alice.co2-levelhigh
• ? Alice.camera.display on 8, off 0

22
Example - Filtering

• Fireman(Alice) ? Alice.co2-levelhigh
• ? Alice.camera.display on 8, off 0
• Alice.co2-levelhigh
• True ? Alice.camera.display on 8, off 0
• Alice.co2-levellow
• ??
• fireman(Alice) ? fire(fire1) ? Alice.location
  fire1.location
• ? Alice.camera.display on 4, off 0
• Alice.location wtc fire1.location wtc
• True ? Alice.camera.display on 4, off 0
• Alice.location ? WTC. Location
• ? ?

23
Example

• Objects fireman(Alice), Alice.co2-levelhigh
• True ? Alice.camera.display on 8, off 0
• Value(Alice.camera.display on) 8
• Value(Alice.camera.display off) 0
• Objects fireman(Alice),fire(fire1),
  Alice.co2-levelhigh, Alice.location
  fire1.location wtc
• True ? Alice.camera.display on 8, off 0
• True ? Alice.camera.display on 4, off 0
• Value(Alice.camera.display on) 12
• Value(Alice.camera.display off) 0

24
(a bit) More Formally

• A variable may appear only if it also appears
  within a unary class variable.
• Constrains the number of ground instances to be
  finite
• Finite universe also implies finite number of
  attributes
• Filtered ground rules induce value function over
  the set of assignments to controllable attributes
• vR,O (a) value of assignment a given rules r
  objects o
• r ground instances of rules in r
• W(r,a) the weight associated with the rule
  head of r and assignment a

25
Some Modeling Issues

• Compositionality
• Should we allow multiple rules with identical
  heads?
• Our answer yes
• Fits naturally with the idea of additive
  representation
• Example
• fireman(x) ? fire(y) ? x.location y.location
• ? x.camera.display on 4, off 0
• fireman(x) ? fireman(y) ? fire(z) ? x.location
  y.location ? x.location z.location ? x ? y ?
  x.camera.display on ? y.camera.display on
  -4, off 0

26
More Modeling Issues

• Indirect preferences
• I want to say low-ranking drivers are preferred
  for fire-engines
• I could write
• Fire-engine(x) ? x.driver.rank low 4, high
  0
• But rank is not a controllable attribute driver
  is!
• Violates our restriction on rule-heads
• Maybe fire-engine(x) ? fireman(y) ? y.rank low
  ? x.drivery T 4, F 0?
• Somehow, more flexibility is desirable.

27
Computing Optimal Assignment

• Branch and bound
• Local search
• May be good for real-time applications with many
  computations with similar sets of objects
• Reduction to relational probabilistic models

28
Related Work

• Relational probabilistic models
• Close links between probabilities/utilities/values
  
• All are forms of orderings
• Multiplicative decomposition of joint
  distribution similar to additive decomposition of
  value function
• Just take the logarithm
• There are various prm formalisms
• Syntax is similar
• Semantics is similar in the sense that prm
  objects induces a distribution over assignment to
  Herbrand Base
• Compositionality more of an issue
• Inference is also mostly based on grounding first

29
Related Work

• Relational Models translation
• Given rule-body ? rule-head v1, w1 v2 w2vm
  wm
• Markov Logic rule-body ? rule-head(vi) wi
• Relational Bayesian Logic
• Pr(rule-head vi rule-body) exp(wi)
• Renormalize weights to ensure exp(wi) in 0,1
• Use MAP queries to find optimum

30
Related Work

• Soft Constraint Programs more general
• Preference rules attempt to provide a minimalist
  solution that generalizes GAI value function

31
Ongoing Work

• CC system implementation on-going
• Prototype completion expect in July
• Will test BB vs. local search
• Future plans try to use Alchemy (engine for
  Markov Logic)
• Integrate with uncertainty
• May not be hard given the relation with PRMs
• May be viewed as some form of relational
  influence diagram where decisions are not ordered
• Cyclic dependencies!
• Try to learn from observation
• Based on user interaction with the system, try to
  learn/modify rules

32
Summary

• Very simple formalism
• Attempt to minimally extend rule-based systems to
  define value functions
• Much similarity to existing approaches for PRM
  and soft constrain programming
• Can utilize algorithms from PRM