1 Department of Computer Science and Engineering, University of South Carolina

1
Bayesian Network Development

• Kevin B. Korb and Ann E. Nicholson
• Bayesian Artificial Intelligence, Chapman and
  Hall, 2004, esp. part III
• Kathryn B. Laskey and Suzanne M. Mahoney
• Knowledge Engineering for Agile Belief Network
  Models, IEEE Transactions on Data and Knowledge
  Engineering, 12, 4 (July/August 2000), 487-498
• Paul Sticha, Dennis Buede, and Richard L. Rees
• APOLLO An Analytical Tool for Predicting a
  Subjects Decision Making, Proceedings of IA-05
• (https//analysis.mitre.org/proceedings/Final_Pap
  ers_Files/143_Camera_Ready_Paper.pdf)

2
APOLLO

• Models built to date (May 2005)
• Invasion
• National strike
• Domestic threat
• Missile testing
• Support for the Global War on Terrorism
• Dispute over contested territory
• Peace/cease-fire negotiations
• Use of WMD
• Monetary devaluation
• Establishment of a new caliphate
• Operational planning in a terror cell

3
Use of the Models

• The analysts want to model the decisions of
  foreign leaders
• Software alerts the analyst when certain
  thresholds are met within the model, indicating
  that evidence suggests a change in what is to be
  believed
• Models are continuously updated
• Models provide an auditable record of the
  assumptions and of the supporting evidence
• Neutralize various analytic biases such as
• Recency, halo, proximity, hindsight,
  personalization
• Neutralize various social biases such as
• Senior expert, party line, published record,
  biggest fistful of cables, personality

4
Process

• Two-day facilitated meetings per model
• Participants
• Analysts, subject-matter experts, facilitator,
  and model developer
• Analysts provide information about the questions
  to be addressed by the model
• Both analysts and external experts provide the
  information and assessments
• Facilitator keeps the group framing the questions
  properly, e.g. in terms of conditional
  probability assessments, and keeps a healthy
  debate going
• Model developer also acts as notetaker
• Model is projected on screen as it is developed

5
Steps

• Defining the question, e.g. what will a national
  leader do in case of a national strike
• Leave country, make concessions, hold a
  referendum, let a regional organization
  arbitrate, wait out, repress violently
• Identify situational variables that may affect
  the leaders decision, e.g.
• What are the leaders objectives
• Add situational variables subject to the time
  constraints for the model development process
• One-and-a-half day of the two-day session are
  typically completed by this point.
• A personality module is added to each model,
  linked through intervening variables.
• What-if analyses
• Sensitivity

6
General Strike
Figures are from Sticha et al.s paper
7
Conditional Probability Assessment
Table is from Sticha et al.s paper
8
Monitoring the Situation over Time
Figure is from Sticha et al.s paper
9
Linkage Variables for Personality Model
Table is from Sticha et al.s paper
10
A Complete Model
Figure is from Sticha et al.s paper
11
Laskey and Mahoney

• Probability elicitation worksheets (figure 6 in
  paper)
• Exploit partitions in the state space of parent
  variables
• Compare distributions
• Focus on order-of-magnitude differences in small
  probabilities

12
Korb and Nicholson

• Types of variables
• Target or query
• Evidence or observation
• Context
• Sensing conditions, setting factors, background
  causal conditions
• Controllable
• May be set, rather than observed
• Values
• How to discretize

13
Graphical Structure

• Causal Relationships
• Cause, effect, prevention, interference,
  moderation, invalidation, enabling, explanation
• Dependence and Independence Relationships
• D-separation
• Relevance
• Association relationship
• Temporal relationship

14
Parameters (Probabilities)

• Sources
• Data
• Domain experts
• The literature
• Elicitation
• Verbal maps
• Odds
• Pie charts, histograms
• Lotteries
• Local structure
• causal interaction or lack thereof addition,
  prevention, XOR, synergy
• Partitioning
• Divorcing
• Preference structures (not in Korb and Nicholson)

15
Nodes

• Nodes in a Bayesian network are in one-to-one
  correspondence with (random) variables.
• Variables map states (also known as values) to
  subsets of the event space
• The probability of a variable having a certain
  value is the probability of all the events
  consistent with that variable having that value
• Variables represent propositions about which the
  system reasons they are therefore sometimes
  called propositional variables, even though they
  may take values other than true and false.

16
Attributes

• Each variable has a set of identifying attributes
• Attributes play the role of variables in a logic
  programming language Laskey and Mahoney,
  UAI-97
• Attributes identify a particular instance of a
  random variable
• Attributes are used to combine fragments
• Fragments can be combined only if their
  attributes unify

17
Fragments As Templates

• Fragments are template models
• A template model is appropriate for problem
  domains in which the relevant variables, their
  state spaces, and their probabilistic
  relationships do not vary from problem instance
  to problem instance LM, UAI-97
• A scenario is a combination of instantiated
  template models
• The attributes are used to identify and combine
  fragment instances but the probabilistic
  relationships do not change from instance to
  instance
• The probability distribution described in the
  Bayesian network is a joint distribution on the
  nodes only, not on the nodes and the attributes

18
Medical Illustration

• A medical diagnosis template network would
  contain variables representing background
  information about a patient, possible medical
  conditions the patient might be experiencing, and
  clinical findings that might be observed.
• The network encodes probabilistic relationships
  among these variables. To perform diagnosis on a
  particular patient, background information and
  findings for the patient are entered as evidence
  and the posterior probabilities of the possible
  medical conditions are reported.
• Although values of the evidence variables vary
  from patient to patient, the relevant variables
  and their probabilistic relationships are assumed
  to be the same for all patients. It is this
  assumption that justifies the use of template
  models.
• Direct quote from Laskey and Mahoney, UAI-97

19
Guidance for Selection of Nodes and Attributes

• Nodes represent the variables on which the
  assessment of a situation depends. For example
• State and hypothesis variables
• Observation and test variables
• Intermediate and theoretical variables
• Setting factors
• Attributes identify a particular situation.
  E.g.
• Location
• Time
• Name
• Case ID

20
MEBNs As a System Integrating First-Order Logic
and Probability

• Paulo C.G. da Costa and Kathryn B. Laskey.
  Multi-Entity Bayesian Networks without
  Multi-Tears. Available at http//ite.gmu.edu/kla
  skey/publications.html Costa, 2005
• Kathryn B. Laskey. First-order Bayesian Logic.
  Available at http//ite.gmu.edu/klaskey/publicati
  ons.html Laskey, 2005

21
Sample BN Fragments
Laskey, 2005
22
Using MEBNs

• Bayesian Network Fragment (BNF)
• It is the basic unit. Each network fragment
  consists of a set of related variables together
  with knowledge about the probabilistic
  relationships among the variables.
• Multi Entity Bayesian Network (MEBN)
• Collection of BNFs specifying probability
  distribution over attributes of and relationships
  among a collection of interrelated entities
• Situation-Specific Network(SSN)
• Ordinary finite Bayesian Network constructed
  from an MEBN knowledge base, to reason about
  specific target hypothesis, with a particular
  evidence.

Laskey, 2005
23
Formal Specifications

• First-Order Bayesian Logic
• A logical foundation that fully integrates
  classical first-order logic with probability
  theory
• Because first-order Bayesian logic contains
  classical first-order logic as a deterministic
  subset, it is a natural candidate as a universal
  representation for integrating domain ontologies
  expressed in languages based on classical
  first-order logic or subsets thereof.

Laskey, 2005
24
Logic in BN Fragments
Laskey, 2005
25
A Simple Bayesian Network
Laskey, 2005
26
A Conditional Proabability Table
Laskey, 2005
27
Multiple Instances
Laskey, 2005
28
Temporal Repetition
Laskey, 2005
29
A Fragment (MFrag)
Laskey, 2005
30
An Instance of an MFrag
Laskey, 2005
31
A Temporal MFrag
Laskey, 2005
32
Temporal Situation-Specific BN
Laskey, 2005
33
Other Issues in Laskey, 2005

• Generative Theories
• Composition Algorithm
• Related Research
• HMMs
• DBNs
• Plates
• Object-Oriented BNs
• Probabilistic Relational Models
• Learning
• Decision Making
• Multiple-entity decision graphs (MEDGs) are to
  influence diagrams what MEBNs are to Bayesian
  networks
• OWL-P
• A planned MEBN-based extension to OWL

34
Korb and Nicholson

• Types of variables
• Target or query
• Evidence or observation
• Context
• Sensing conditions, setting factors, background
  causal conditions
• Controllable
• May be set, rather than observed
• Values
• How to discretize

35
Graphical Structure

• Causal Relationships
• Cause, effect, prevention, interference,
  moderation, invalidation, enabling, explanation
• Dependence and Independence Relationships
• D-separation
• Relevance
• Association relationship
• Temporal relationship

36
Parameters (Probabilities)

• Sources
• Data
• Domain experts
• The literature
• Elicitation
• Verbal maps
• Odds
• Pie charts, histograms
• Lotteries
• Local structure
• causal interaction or lack thereof addition,
  prevention, XOR, synergy
• Partitioning
• Divorcing
• Preference structures (not in Korb and Nicholson)

37
Modeling Exercise Natural Disaster

• What will be the economic effect on a country due
  to a natural disaster disrupting the availability
  of a commodity?
• Answers Uprising, War to acquire by force,
  Recession, None
• Situational variables
• Extent of disaster
• Importance of commodity
• Alternate commodity
• Alternate supply
• Evidence or observation variables
• Projected need for commodity
• Import amount
• Export amount

38
Basic Scenario for Terrorism Network

• Leader communicates goals and ideology to
  planners and operatives
• Planners arrange funds choose target specify
  logistics
• Logistics types are weapons, transportation
  resources
• Funds support operatives pay for logistics
• Operatives acquire logistics use Intel use
  logistics perform terrorist act
• Intel identify logistics identify target
  support planners
• Targets types are facilities, resources,
  people, relationships
• Terrorist act requires operatives requires
  logistics

(Not all relationships are shown)
39
Basic Scenario for Economic Interdependency

• Catastrophic Event either a natural disaster,
  such as an earthquake, or a terrorist action,
  such as an oil pipeline disrupted
• Supply Chains the key resources and industries,
  from raw materials to finished goods
• Industries types
• Countries locations of resources and factories
• Consequences the possibilities for political or
  military actions, and the ramifications

Catastrophic Event
40
Economic Disruption Scenario

• An analyst receives the following message
• Intelligence Report 11/11/04 A conversation
  recorded by a wiretap on a suspected terrorist
  cell in Beirut had a discussion about crippling
  the Iranian economy by destroying oil production
  facilities.
• The original intelligence message is passed to
  the Bayesian reasoning system for further
  analysis.
• The message is parsed, marked-up semantically,
  and matched with prior analytical knowledge in
  the form of Bayesian network fragments.
• The fragments are assembled into a number of
  plausible scenarios that explain the input
  information. The most plausible and complete
  scenario is shown to the analyst.
• The scenario shows that the most likely and
  highest value target would be a pipeline, if it
  were known that the pipeline crossed an
  international border and that the nations on each
  side of the border had a history of distrust.
• The analyst is interested in pipelines as a means
  of oil transportation for all known refineries
  and fields in Iran and asks for that information
  from the CBR for KD system.
• The CBR for KD system locates a URL concerning
  the locations of cross border pipelines in the
  region http//www.eia.doe.gov/emeu/cabs/iran.html
  The URL is passed to the Bayesian reasoning
  system.
• The Bayesian reasoning system augments the
  scenario it constructed earlier with the
  additional information and determines that there
  is a pipeline that would be at risk.
• The analyst is alerted to the risk and is
  presented with the evidence and the scenario
  showing the reasoning behind the alert.

41
Basic Scenario for Capabilities to Produce Weapon
X

• Weapon X Production gt
• Raw Materials
• Personnel
• Expertise
• Funding
• Manufacturing Facilities
• Motivation and Intent

Motivation and Intent
Personnel
Expertise
Manufacturing Facilities
Funding
Weapon XProduction
Raw Materials