ARLHRED

1
ARL-HRED

• FAMU-FSU Simulation Group
• Output Analysis Overview
• 7 Dec 2005

2
Who we are

• Dr. James Simpson, Principal Investigator
• Associate Professor of Industrial Engineering
• Florida State University
• Graduate Assistants
• Lisa Hughes
• Nicholas Done
• Wayne Wesley
• Michelle Zeisset

3
What we do
Improve IMPRINT as a decision-making tool

• Bring operations research expertise
• Not directly involved in product development so
  come from perspective of a potential user
• Military background adds realism

4
Our history with IMPRINT
5
Areas of concentration
6
Study approach

• Use small simple models
• Compare observed results to expected results
• Interpret results in context of a question a
  potential user may want to answer
• Validate and demonstrate suggested improvements
  using realistic models

7
Areas of concentration
Output Variable Assessment

• Multiple runs

• Useful metrics

Data Compilation

• Efficient data reporting
• Graphical and tabular reports

User Support

• GUI
• Analysis tools

8
Need for multiple runs

• of Time Overloaded (C/G)

N 1

• Mean 27.9
• Std dev 3.7
• Min 10.8
• Max 33.8

frequency
25
N 100
20
15
frequency
10
5
0
Combat model (Advanced operations module)
9
How many runs?
Stryker model (Maintenance model)
10
Multiple run outputFrequency histogram
Number of Times Overloaded
30
25
20
15
Frequency
10
5
0
100
179
258
36.8
52.6
68.4
84.2
115.8
131.6
147.4
163.2
194.8
210.6
226.4
242.2
BFV (VACP operations module)
11
Which events led to results?

• 3 Function-level probability nodes act as
  switches

Function Network of BFV
12
Histogram broken down by function
Histogram of Number of Times Overloaded,
Color-Coded by Trace
Histogram of Number of Times Overloaded
30
25
20
15
F11
Frequency
F10
10
F8
F4
5
0
100
179
258
36.8
52.6
68.4
84.2
115.8
131.6
147.4
163.2
194.8
210.6
226.4
242.2
BFV (VACP operations module)


13
Tabular report by function
BFV (VACP operations module)
14
Areas of concentration
Output Variable Assessment

• Multiple runs
• Useful metrics

Data Compilation

• Efficient data reporting
• Graphical and tabular reports

User Support

• GUI
• Analysis tools

15
Compiled per run report
Operator Overload Report
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
? ? ?
2 vs. 3 study (Goal orientation operations module)
16
Maximum workload peak report
Workload Peak Summary Measures
Tasks contributing to max workload peaks
Task
i
times
times
167
30
83.33
168
3
8.33
Emax workload peaks
67.81
209
3
8.33
E of max workload peaks
1.47
210
21
58.33
E of ongoing tasks
8.67
211
12
33.33
212
2
5.56
Varmax workload peak
19.23
219
36
100.00
Var of max workload peaks
0.68
222
36
100.00
Var of ongoing tasks
0.43
223
27
75.00
227
36
100.00
228
29
80.56
229
36
100.00
232
36
100.00
multiple runs, 36 max workload peaks
through 100 runs
2 vs. 3 study (Goal orientation operations module)
17
Contingency table
n 5 replications
Simple study (Maintenance model)
18
Areas of concentration
Output Variable Assessment

• Multiple runs
• Useful metrics

Data Compilation

• Efficient data reporting
• Graphical and tabular reports

User Support

• GUI
• Analysis tools

19
User support output analysis

• Study objective
• Preliminary output assessment
• System analysis modified and tailored to the
  needs of user
• Summary performance
• Comparative model study
• Model characterization
• Sensitivity analysis/Model validation
• Model optimization or enhancement

20
Analysis guidelines
21
Example multiple factor study

• Factor values may vary depending upon
  characteristics such as manufacturer or material
  type.
• EXAMPLE A certain component is made by two
  different companies. Component A has a MOUBF of
  1 hour. Component B is a higher quality product
  and therefore has a higher MOUBF of 3 hours.

Simple study (Maintenance model)
22
Example multiple factor study

• Significant Factors
• MOUBF
• MTTR
• Statistical Prediction Model
• Total Direct Man hours 68.7 - 1.4(MOUBF)
  2.2(MTTR)
• 2.3( of Maintainers) 3.16( of
  Systems)
• 3.24(Length of Run)
• Prediction

• of Maintainers
• of Systems

• Length of Run

MOUBF 2 hr MTTR 20 min maint 3 sys
7 run length 30
Estimated total direct man hours 236.12
Simple study (Maintenance model)
23

• Collaborative
• Discussion

24
Workload level histogram
of Time at Workload Levels for Driver
25.00
100 runs

20.00
21.05
15.00
of Time
78.95
10.00
5.00
0.00
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
100
105
110
115
120
125
130
135
140
145
150
155
160
165
170
175
180
185
190
195
200
gt200
Workload Level
2 vs. 3 study (Goal orientation operations module)