Bayesian Advisor Project

1
Bayesian Advisor Project

• Probability Elicitation Tool
• Group
• David Cornett
• Shih-Che Huang
• Derek Williams

2
Graphic from 2
3
Whats the point? Our Objectives

• Upgrade the Probability Elicitation Tool
• Implement some form of consistency checking
• Add full tutorial and help system
• Fix the exponential growth problem

4
Consistency Checking

• Inconsistencies in subjective data are inevitable
  when there are multiple users involved
• Consistency Checking ensures that the users are
  at least notified that data within a domain is
  inconsistent

5
Consistency Checking (contd)

• PET implements two forms of Consistency Checking
  within the PET tool
• 1) Internal Consistency Checking
• 2) External Consistency Checking

6
Internal Consistency

• Internal Consistency (or Self Consistency)
  focuses on maintaining a similar distribution of
  values within a set of data produced by one user
• The main issue in maintaining Internal
  Consistency was that the data from a user
  remained monotonic

7
Internal Consistency (contd)

• The PET tool implements Internal Consistency
  Checking by testing whether the current row
  entered into a CPT maintains a similar
  distribution with the three previous rows
• This method considers only a subset of the CPT
  instead of the entire table!

8
Internal Consistency (contd)

• When a users current data is found to be
  inconsistent with their previous data they are
  prompted by the PET tool

9
External Consistency

• External Consistency is concerned with
  maintaining a similar distribution of values
  within sets of data from different users within
  one domain
• Each domain will have a summary file containing
  information on which distributions users adhere
  to

10
External Consistency (contd)

• As the user enters values into the CPT the file
  for External Consistency Checking is updated
• Currently the file holds the user names plus
  which distribution most likely to be used by them
  in this domain

11
External Consistency (contd)

• When the user closes out the current work space
  the PET tool calculates External Consistency and
  prompts the user

12
User Control

• The PET tool also allows the user to turn off
  both Internal and External Consistency Checking
  (just in case they start feeling bad from all the
  criticism)

13
Implementation

• Both Internal and External Consistency Checking
  were implemented using Java (because PET is
  written in Java)
• External Consistency file in XML format (and uses
  DOM parser to evaluate)
• Estimated lines of code 1000
• Actual lines of code 400

14
Snags along the Way

• Computation involved with Internal Consistency
  Checking
• Dr. Goldsmith provided an algorithm to decrease
  the computational cost
• Integrating code into original PET (NO useful
  internal documentation)
• Deciding what information to hold within the
  External Consistency file and how to represent it

15
Possible Future Work

• Allow the user different choices of how Internal
  and External Consistency will be checked
• Give PET tool the functionality to automatically
  correct inconsistencies within the data ( hard)

16
Why Use JAVA and HTML?

• PET is JAVA based
• HTML is easy to use and familiar
• HTML can be used on the website and is faster
  than a JAVA Applet
• For integrating proposes

17
Data Flow Diagram
18
Interactive Tutorial
19
Interactive Tutorial
20
Interactive Tutorial
21
Interactive Tutorial
22
Keyword Index (Option 1)
23
Keyword Index (Option 2)
24
Keyword Index (Option 2)
25
Code estimates Real Numbers

• Estimates Numbers 500 lines of JAVA code
• 50 HTML pages
  
• Current Numbers 367 lines of JAVA code
• 40 HTML Pages

26
Problems Solutions

• Customer was not satisfied with the interface
• Put the extra buttons inside the HTML pages
• Confusing to the user
• Solution
• Research commercial software
• Redesign user interface
• Discuss with the customer again

27
Problems Solutions

• Keyword list did not automatically scroll to
  ensure closest matched keyword was visible
• Solution
• Use predefined Java functions to ensure that the
  matching list index is visible

28
Problems Solutions

• The keyword input did not automatically highlight
  the closest matched keyword correctly
• Solution
• Used predefined text listeners to dynamically
  check user input against the list items

29
Lessons Learned

• Use predefined JAVA functions whenever applicable
• Human Computer Interaction increases usability
  and aids in intuitive layout
• Maintain communication with customer for all
  aspects of design

30
Future Work

• Animated Tutorial System
• Update help files as new functionality becomes
  available

31
Exponential Growth Problem

• Number of cells grows exponentially with the
  number of parents and outcomes

32
Exponential Growth Problem

• Number of cells grows exponentially with the
  number of parents and outcomes

33
Noisy-OR
34
Noisy-OR

• Consider dependency nodes independent of each
  other

35
(No Transcript)
36
Noisy-OR

• Consider dependency nodes independent of each
  other
• Calculate the distribution by combining nodes
  from all independent tables

37
Noisy-OR

• Consider dependency nodes independent of each
  other
• Calculate the distribution by combining nodes
  from all independent tables
• Provides a logarithmic reduction in the number of
  cells

38
Noisy-OR Algorithm
39
Noisy-OR Algorithm

• The probability of a variable node is given as

40
Noisy-OR Algorithm

• The probability of a variable node is given as
• Noisy-OR is based on the extent to which each
  condition inhibits A from occurring 1

41
Noisy-OR Algorithm

• The probability of a variable node is given as
• Noisy-OR is based on the extent to which each
  condition inhibits A from occurring 1

42
Noisy-OR Algorithm

• The probability of a variable node is given as
• Noisy-OR is based on the extent to which each
  condition inhibits A from occurring 1
• The inhibitor for a parent is one minus the
  probability that A occurs given the parent is true

43
Noisy-OR Algorithm

• The probability of a variable node is given as
• Noisy-OR is based on the extent to which each
  condition inhibits A from occurring 1
• The inhibitor for a parent is one minus the
  probability that A occurs given the parent is
  true
• False parents are not considered

44
Noisy-OR Algorithm

• The probability of a variable node is given as
• Noisy-OR is based on the extent to which each
  condition inhibits A from occurring 1
• The inhibitor for a parent is one minus the
  probability that A occurs given the parent is
  true
• False parents are not considered
• Easily expanded to non-Boolean Noisy-MAX

45
Data Flow Diagram
Active PET Node
PET
Active PET Table
PET Layer
Noisy Max Layer
Init NMAX
Data Mapping
Data Parsing
Rounding
Set Precision
Close
PET Frame
Noisy-MAX calculation
Menu Bar
Button Bar
Tabbed Panes
Tables
Table Model
46
(No Transcript)
47
Problems Solutions
48
Problems Solutions

• PET layout is cramped
• Where do you put noisy-MAX
• No room for another panel
• Use PET tabbed table layout?
• Interferes with PET data
• PET tries to write NMAX tables to file on save

49
Problems Solutions

• PET layout is cramped
• Where do you put noisy-MAX
• No room for another panel
• Use PET tabbed table layout?
• Interferes with PET data
• PET tries to write NMAX tables to file on save
• Solution
• Render NMAX GUI in un-anchored window
• Use same tabbed pane table layout for efficiency
  and familiarity

50
Problems Solutions

• PET code lacks useful in-line comments
• Data structures are difficult to understand
• Dependency resolution is difficult

51
Problems Solutions

• PET code lacks useful in-line comments
• Data structures are difficult to understand
• Dependency resolution is difficult
• Solution
• Use a very object oriented approach
• Develop independent packages rather than in-line
  integration
• Touch PET data structures as little as possible

52
Problems Solutions

• JTables are very complex
• Difficult to modify layout
• Default tools lack specific functionality
• All cells are editable

53
Problems Solutions

• JTables are very complex
• Difficult to modify layout
• Default tools lack specific functionality
• All cells are editable
• Solution
• Create abstract table model to override defaults
• Greater control, but a lot of extra work

54
Problems Solutions

• Java shortcomings
• No simple rounding or precision for decimals
• No pointer declaration, no explicit pass by value
• No explicit way to free memory

55
Problems Solutions

• Java shortcomings
• No simple rounding or precision for decimals
• No pointer declaration, no explicit pass by value
• No explicit way to free memory
• Solution
• Implement own rounding and precision functions
• Write passed arguments to new structures
• Rely on JAVA for garbage collection

56
Lessons Learned
57
Lessons Learned

• Comment your code!

58
Lessons Learned

• Comment your code!
• Know pros/cons of programming language of choice

59
Lessons Learned

• Comment your code!
• Know pros/cons of programming language of choice
• Easier to add/change features during development

60
Lessons Learned

• Comment your code!
• Know pros/cons of programming language of choice
• Easier to add/change features during development
• HCI (Human Computer Interaction) layout issues
  greatly impact usability

61
Code Estimates and Totals
62
Code Estimates and Totals

• Estimate (for NMAX) 1000 lines of code

63
Code Estimates and Totals

• Estimate (for NMAX) 1000 lines of code
• Implemented 1000 (and growing)
• - Also created Java style API documents for the
  NMAX package

64
Conclusions and Delusions
65
Conclusions and Delusions

• Assumptions
• Integration would be difficult
• Noisy-MAX not a toy program to implement
• Time would be tight

66
Conclusions and Delusions

• Assumptions
• Integration would be difficult
• Noisy-MAX not a toy program to implement
• Time would be tight
• Now
• Integration was very difficult in some parts
• About what was expected

67
Future Work
68
Future Work

• Local file load/save for Noisy-MAX

69
Future Work

• Local file load/save for Noisy-MAX
• Currently implementing LEAK option

70
Future Work

• Local file load/save for Noisy-MAX
• Currently implementing LEAK option
• User specified ignorable outcomes
• Example Grade outcomes A E, and not taken
• not taken need not be considered for Noisy-MAX

71
Future Work

• Local file load/save for Noisy-MAX
• Currently implementing LEAK option
• User specified ignorable outcomes
• Example Grade outcomes A E, and not taken
• not taken need not be considered for Noisy-MAX
• Add PET slider and normalization tools to NMAX

72
Future Work

• Local file load/save for Noisy-MAX
• Currently implementing LEAK option
• User specified ignorable outcomes
• Example Grade outcomes A E, and not taken
• not taken need not be considered for Noisy-MAX
• Add PET slider and normalization tools to NMAX
• Force extremely large tables to NMAX
• 10 parents w/5 outcomes 48,828,125 cells in
  PET!
• NMAX represents this w/10 tables, 25 cells each
  (250 total)

73
References

• 1 Adam Zagorecki, Marek Druzdzel. An Empirical
  Study of Probability Elicitation under Noisy-Or
  Assumption. In American Association For
  Artificial Intelligence. 2004.
• 2 Alex Dekhtyar, Judy Goldsmith. The Bayesian
  Advisor Project. March 15, 2002.