Slide 1 of 88

1
Detailed Modeling and Terminating Statistical
Analysis

• Chapter 5

2
What Well Do ...

• Explore lower-level modeling constructs
• Model 5-1 A generic call-center system
• Nonstationary arrival process
• Balking, three-way decisions, sets, variables,
  expressions, submodels, and costing
• Debugging
• Model 5-2 Animating the call center model
• Plots, global pictures, and storages
• Model 5-3 The model with overall performance
  measures
• Run conditions, model size and speed, overall
  performance measures

3
What Well Do ... (contd.)

• Statistical analysis of simulation output
  (terminating systems)
• Time frame of simulations
• Strategy for data collection and analysis
• Confidence intervals
• Comparing two alternatives
• Comparing many alternatives via the Arena Process
  Analyzer (PAN)
• Searching for and optimal alternative with
  OptQuest

4
Generic Call Center(Model 5-1)

• Single telephone number, 26 trunk lines
• If all 26 lines busy, caller gets busy signal and
  goes away
• Answered call gets recording asking
• Technical support? (76 of callers choose this)
• Sales information? (16)
• Order-status inquiry? (8)
• Time for caller to choose UNIF (0.1, 0.6)
• All times are in minutes in this model

5
Technical Support Calls

• Get second recording asking
• Product type 1? (25 of tech support callers
  choose this)
• Product type 2? (34)
• Product type 3? (41)
• Recording and choosing takes UNIF(0.1, 0.5)
• If a qualified tech-support person is available
  for chosen product, call routed for immediate
  service
• If not, call placed in (electronic) queue,
  subjected to annoying rock music
• All tech support conversations TRIA (3, 6, 18)
• When call done, customer exits system

6
Technical Support Calls (contd.)

• 4 of tech support calls need further assistance
  after completion of their call
• Questions forwarded to another tech group that
  prepares a response time to prepare this
  response EXPO (60)
• Response sent back to the same tech-support
  person who took the original call
• This person calls the customer back and talks,
  which lasts TRIA (2, 4, 9)
• These calls require one of the 26 trunk lines and
  take priority over incoming calls
• If return call not completed on same day, its
  carried over to the next day

7
Sales Calls

• Call automatically routed to sales staff
• Sales staff is separate from tech-support staff
• If a sales-staff person is available, call gets
  immediate service
• If not, call placed in (electronic) queue,
  treated to soothing new-age space music
• All sales conversations TRIA (4, 15, 45)
• When call done, customer exits system

8
Order-Status Calls

• Automatically handled by phone system no people
• No limit on number handled at a time (but still
  limited by the 26 trunk lines)
• Time for conversation TRIA (2, 3, 4)
• After call, 15 take option to talk to a real
  person (the rest exit the system)
• These calls are routed to sales staff
• Have same priority as incoming sales calls
• Conversation durations TRIA (3, 5, 10)
• Then exit the system

9
Call Arrivals

• Calls accepted from 8 AM until 6 PM
• Some staff available until 7 PM
• Incoming calls shut out after 6 PM
• But all calls that entered before 6 PM are
  answered
• Call arrival rate varies substantially over the
  day
• Data on rate (calls per hour) for each half-hour
  period

10
Staffing

• Sales staff 7 people with staggered schedules
• 3 on duty for first 90 minutes (notation 3_at_90)
• Then 7_at_90, 6_at_90, 7_at_60, 6_at_120, 7_at_120, 4_at_90
• Tech support staff 11 people variously
  qualified for the three different product lines
• Work eight-hour days plus 30 minutes off for
  lunch
• Some people only qualified on one line
• Some people qualified on two or maybe all three
  lines
• Detailed staffing description and schedule ...

11
Technical-Support Staff Schedules
12
New Modeling Issues

• This is a service (not manufacturing) system
• But can use same modeling capabilities
• Nonstationary arrival process
• Arrivals occur one at a time and are independent
  of one another
• Average rate varies over time (would be constant
  for a stationary Poisson process)
• Built into Create module (beware of
  popular-but-wrong methods details in book)
• Balking
• Required because there are only 26 trunk lines
• Entity arrives at queue, which is full (capacity
  is 0 here)
• Entity departs from system count these

13
New Modeling Issues (contd.)

• Three-way decisions
• Entity or call can go to one of three places in
  model based on call type
• Similarly, tech-support calls can go to one of
  three places based on product type
• Capability available in Decide module
• Sets
• Groups of similar objects
• Can be referenced by a common set name and index
  (1, 2, 3, ) into the set
• Can also be referenced by original name,
  independent of set
• Technical-support staff requires sets
• An object can be a member of more than one set
• Sets data module

14
New Modeling Issues (contd.)

• Variables and Expressions
• Can be referenced in model by name
• Can be one- or two-dimensional arrays, indexed by
  one or two integers
• User-defined Variables
• Store some numerical value (not a formula)
• Can be initialized in Variable data module
• Can be used, reassigned during the simulation run
  by any entity
• User-defined Expressions
• A name defined by a mathematical expression
• This name can be references anywhere in the model
• Can use constants, Variables, Attributes, system
  state variables, values from distribution
  connected via mathematical operations
• Can use Expression Builder to help define
• Defined in Expression data module (Advanced
  Process panel)

15
New Modeling Issues (contd.)

• Submodels
• Partition simulation model into several smaller
  submodels
• Can link them together, more manageable pieces
• Just like a normal model view within a submodel
• Submodels can also contain further submodels,
  etc. hierarchical structure
• Submodels can be externally connected to other
  modules or submodels
• Navigate panel in Project Bar shows submodels,
  under Top-Level Model
• Costing
• Automatic time and cost information for entities
• Wait, value-added, non-value-added, transfer,
  other
• You must enter cost information Entity and
  Resource data modules

16
Key Output Performance Measures

• Count balks no. of attempted incoming calls
  sent away due to all 26 trunk lines being busy
• Will not model reneging customers in queue
  leaving the system if they get sick of waiting
• Total time in system, by customer type
• Time waiting for a real person, by customer type
• Contact time, by customer type
• Number of calls waiting, by customer type
• Personnel utilization

17
Animation Requirements

• No entity movement to animate here
• Can still display queues
• To see how well staffing matches up with load,
  craft appropriate plots vs. time
• Number of calls balked
• Lengths of queues
• Number of idle staff
• Strategy to improve performance alter the
  staffing schedule, see if it produces a better
  matchup of the plots

18
System or Simulation Type

• Terminating
• Known starting and stopping conditions part of
  model
• Time frame is known (and finite)
• Steady-State
• Initial conditions are not always well defined
• No defined stopping condition (theoretically
  infinite)
• Interested in system response over the long run
• Call-center model
• Start at 8 AM and end at 7 PM
• Some Technical support calls are held over, but
  not many well ignore this aspect (sort of
  fixed below)
• Treat the system as terminating (sort of see
  below)

19
Arena Modeling Panels

• Basic Process panel
• Highest level of modeling
• Advanced Process panel
• More detailed (and different) modeling
  capabilities
• Advanced Transfer panel
• Material-handling, entity-movement capabilities
• Blocks, Elements panels
• Lowest level of modeling capabilities the
  underlying SIMAN simulation language itself
• Other panels are created using modules from these
  panels
• Occasionally needed, but not very often

20
Building the Model

• Defining the Data
• Submodel Creation
• Divide model in sections or submodels
• Increment the Time Period
• Create Arrivals and Direct to Service
• Technical Support Calls
• Technical Support Returned Calls
• Sales Calls
• Order-Status calls
• Well discuss each of these in turn

21
Simulation Replication Data

• Project replication parameters
• Run/Setup dialog Replication Parameters tab
• 10 Replications of 11 hours each
• Four options for Initialization Between
  Replications
• Initialize system (yes), initialize statistics
  (yes)
• 10 independent and identical replications no
  calls carried over
• Reports for each day separately
• Initialize system (yes), initialize statistics
  (no)
• 10 independent and identical replications no
  calls carried over
• Cumulative summary reports (day 1, days 1-2, days
  1-3, , days 1-10)
• Initialize system (no), initialize statistics
  (yes) Selected
• 10 continuous days calls carried over
• Reports are by replication (day)
• Initialize system (no), initialize statistics
  (no)
• 10 continuous days calls carried over
• Cumulative summary reports

22
Schedule Data

• Schedules
• Enter into Schedule data module
• 13 schedules required
• One for each of the 11 technical-support people
• One for the sales staff overall
• The arrival process (Type Arrival, not
  Capacity)
• Use Graphical schedule editor (initially)
• Use Edit via Dialog (or Edit via Spreadsheet) if
  you need trailing zeros in the capacity to fill
  out the cycling time window
• We need this in this model due to not
  Initializing System between replications see
  book for details

23
Resource Data

• Define resources
• Use Resource data module
• 13 resources
• Sales staff
• 11 technical support staff
• Trunk Line (single resource with 26 units)
• Enter Schedule Name for all but Trunk Line
• For resources on a Schedule, use Ignore option
  for Schedule Rule to ensure correct cross-day
  modeling details in book
• Add hourly wage under costing data

24
Sets Data

• Use Set data module (Basic Process panel)
• Develop three Resource sets for technical support
  staff
• Product 1
• Charity, Noah, Molly, Anna, Sammy
• Product 2
• Tierney, Sean, Emma, Anna, Sammy
• Product 3
• Shelley, Jenny, Christie, Molly, Anna, Sammy
• Note that Anna and Sammy are in all three sets
• Consistently listed the more versatile staff at
  the end of the list in each set save them
  discussed later

25
Sets Data (contd.)

• Develop two Tally sets
• Tech Calls
• Product 1 Call, Product 2 Call, Product 3 Call
• Returned Time
• Return 1 Call, Return 2 Call, Return 3 Call
• Sets used to collect statistics by product type
• Develop a Counter set
• Keep track of number of balks per half-hour
  period
• 22 counters one for each half-hour period
• First defined 22 counters in Statistic data
  module (Advanced Process panel)

26
Variables and Expressions Data

• Variables
• Use Variable data module to define thee variables
• Period (the current time period)
• Busy Per Period (busy signals in current time
  period)
• Per Period Balk (total balks for last completed
  time period)
• Note explicit use of Variables module is
  required only if you want a Variable to have a
  non-zero initial value
• Expressions
• Use Expression data module to define three
  expressions
• Returned Tech Time, for duration of returned
  tech-support calls TRIA(2, 4, 9)
• Tech Time, for duration of tech-support calls
  TRIA(3, 6, 18)
• Available 1, Available 2, and Available 3
• Sum of currently available, but idle, resources
  by product type, for staffing plots
• Use Expression Builder details in book, model

27
Submodel Creation

• Object/Submodel/Add Submodel menu option to
  create a submodel well use six submodels
• Define (right-click, then Properties)
• Name
• Number of entry, exit points (could be 0 if
  theres no flow interaction)
• Move between submodels Navigate panel, Named
  Views, or mouse
• Double-click on a submodel to open it
• When in a submodel, right-click in an empty
  place, then Close Submodel, to go up

28
Time Period Counter Submodel

• Increments the time period counter
• No entry or exit points interacts via
  Variables, not flow
• Create Counter Entity Create module
• Time Between Arrivals 660 minutes (constant)
• Assign Period Assign module
• Initialize Period variable value to zero

29
Time Period Counter Submodel (contd.)

• Assign Variables Assign module
• Increment Period variable for the next half-hour
  period
• Assign Per Period Balk to Busy Per Period
  variable value (number of calls balked during
  previous half hour)
• Set Busy Per Period variable to zero to start
  balk counting during the half hour starting now
• Check Period Decide module
• 2-Way by Condition
• Determine if there are still more periods in this
  day (i.e., if Period lt 22)
• Yes Delay for a half hour Delay module, then
  loop back
• No Dispose of entity Dispose module

30
Create and Direct Arrivals Submodel

• Creates arrivals, checks for available trunk
  line, and directs to appropriate type of service
• No entry points
• Three exit points
• Tech Call, Sales Call, Order Status Call

31
Submodel Logic

• Create arriving calls
• If a trunk line is available seize one
• Assign Arrival Time attribute (for use
  downstream)
• Delay to listen to recording
• Determine call type
• Direct call and assign entity type
• Else (all trunk lines are busy)
• Count balked call
• Increment Busy Per Period counter
• Dispose of call

32
Develop Submodel

• Create arriving calls Create module
• Time Between Arrivals
• Type Schedule
• Schedule Name Arrival Schedule
• Was defined when we defined the data for the
  model
• Check for available trunk line
• Queue/Seize module combination (Blocks panel)
• Set queue capacity to zero
• If trunk line available, resource seized in
  following Seize module
• If no truck line available, entity will
  automatically balk

33
Develop Submodel (contd.)

• Assign arrival time Assign module
• Use Arena variable TNOW current simulation
  clock
• Delay for Recording Delay module
• Used Delay module from Blocks panel
• Be careful of units no choice here (uses Base
  Time Units)
• Direct call Decide module
• Use N-way by Chance option
• Enter probabilities as percents (0 100)
• Assign call type Assign module
• Assign entity type to call type
• Three modules

34
Develop Submodel (contd.)

• Balking entities
• Count balked call Record module
• Record into counter set Busy Lines
• Set index is the variable Period
• Increment Busy Per Period variable Assign
  module
• Dispose of balked call call Dispose module

35
Technical Support Calls Submodel

• Logic for servicing technical support calls
• One entry point
• One exit point follow-up calls

36
Submodel Logic

• Delay to listen to recording Delay module
• UNIF(0.1, 0.5) minutes
• Determine product type Decide module
• N-way by Chance
• Seize technical support person
• Seize module Advanced Process panel
• Request from appropriate set for product type
• Preferred order within the set
• Save more versatile employees for other things
• Save set index (particular tech-support person)
  in attribute Tech Agent Index
• In case returned tech call is needed get same
  tech-support person to call customer back

37
Submodel Logic (contd.)

• Save product type and call start time Assign
  module
• Save type (1, 2, or 3) in attribute Product Type
• Assign value from TNOW to attribute Call Start
• Delay for call Delay module
• Use value from expression Tech Time
• Release tech-support person and trunk line
• Release module Advanced Process panel
• Use set index in attribute Tech Agent Index to
  release the particular tech-support person
  assigned from set
• Release the seized unit of Trunk Line resource

38
Submodel Logic (contd.)

• Record call and line time Record module
• Time Interval type
• Tally set Tech Calls
• Set index Product Type
• Records only the time spent during the
  tech-support conversation (necessary?)
• Record tech line time Record module
• Time Interval type
• Tally Tech Support Line Time (not a Tally set)
• Use Arrival Time attribute set when call first
  arrived, so this records the total time in the
  system so far
• Direct to exit point

39
Returned Tech Calls Submodel

• Logic for returned tech calls
• One entry point, no exit points

40
Submodel Logic

• Check for returned call Decide module
• If no returned call is needed
• Dispose of entity
• If a returned call is needed
• Entity Type set to Returned Call Assign module
• Delay for response time Delay module
• Direct by product type
• N-way by Condition based on attribute Product
  Type
• Seize tech-support person and trunk line
• Seize module Seize specific member of
  appropriate Resource set
• Use Set Index Tech Agent Index attribute
• Seize Trunk Line

Note use of to check for equality.
41
Submodel Logic (contd.)

• Delay for call time Delay module
• Expression Returned Tech Time
• Release tech person and trunk line Release
  module
• Record returned time Record module
• Use beginning-time attribute Arrival Time,
  defined when call first arrived, so this records
  total time in system
• Use Tally Set Return Time indexed by Product Type
• Dispose of entity Dispose module

42
Sales Calls Submodel

• Logic for sales calls
• One entry point, no exit points
• Uses a Shared Queue
• Single queue
• Shared by two or more seize activities
• In this case, the real incoming sales calls, as
  well as those order-status calls requiring more
  than just the automated response

43
Submodel Logic

• Seize sales person Seize module
• Shared queue declared in Queue data module
• Delay for call Delay module
• Release sales person and Trunk Line Release
  module
• Record call time Record module
• Records elapsed time from calls original arrival
  until now
• Dispose of entity - Dispose module

44
Order-Status Calls Submodel

• Logic for order-status calls
• One entry point, no exit point
• Shared queue used when sales person required

45
Submodel Logic

• Delay for call Delay module
• Decide if Sales person required Decide module
• If sales person is required
• Seize sales person Seize module, shared queue
• Follow-up delay Delay module
• Release sales person Release module
• Record call Record module
• Elapsed time from calls arrival to system up to
  now
• Release trunk line Release Module
• Dispose of entity Dispose module

46
Finding and Fixing Model Errors

• Arena picks up simple errors in Check phase,
  and leads you to them via Find and Edit buttons
  in Errors/Warnings windows
• Undefined variables, attributes, resources
• Unconnected modules
• Duplicate module names
• Typos
• Other kinds of errors are more complex, cant be
  detected without trying to run options on Run
  Interaction toolbar or on Run menu
• Only mention capabilities here see text for
  details

47
Finding and Fixing Model Errors (contd.)

• Run Controller Command-driven window to
  control, display details about model operation
  and underlying SIMAN code
• Trace Follow active modules, selected variables
• Highlight active module highlights the active
  module during the simulation run
• Layers gives control over what you see during
  the simulation run

48
Finding and Fixing Model Errors (contd.)

• Break on Module Break stop run when entity
  hits a selected module, at a specific time, or
  when a selected entity is about to become active
• Watch select expressions to display in a window
  as model runs
• Look at reports when model is running or paused
• Remember to close reports windows

49
Model 5-2 Animating the Model

• No normal entity animation just plots,
  queues, a few other data animations
• Plots (all vs. time on horizontal axis)
• Queue lengths (as in earlier models)
• Balks per period reason for variable Per Period
  Balk
• Number of tech support people available for each
  product type reason for the Available
  expressions defined in Expressions module
• With multiple plots, configure first one, then
  copy/edit for others to get consistent look and
  feel snap to grid to align
• Variable animations for Period and Day

50
Animating the Model (contd.)

• Created digital clock by hand (details in text)
• Why not ready-made animated clocks? We didnt
  reset the system state between replications, so
  internal clock just keeps increasing
• Resource and queue animations
• Just for realism doesnt add any analysis value
• Resource button from Animate toolbar
• Take pictures from libraries (.plb files),
  different states
• Queue button from Animate toolbar
• Add various text annotations, boxes, etc.

51
Animating the Model (contd.)

• Storages for further animation
• Requires using Delay modules from Blocks panel,
  not Advanced Process panel
• We did this only for the delay listening to the
  first recorded message
• Enter a Storage name in the Blocks Delay module
  (Message 1)
• Storage button from Animate Transfer toolbar
• Animated variables
• Number of available trunk lines
• Number of available salespeople
• Number of sales calls in progress

52
Model 5-3 Model for Analysis with Overall
Performance Measure

• Modify the call-center model for intensive study
• Different run conditions to allow valid
  statistical analysis
• Smaller size to continue to fit in academic
  version and make room for other enhancements
• Faster to allow for extensive analysis
• New overall performance measures to consider both
  resource costs and customer-oriented performance
• Base on Model 5-1 rather than Model 5-2 since the
  latter adds only animation and were now
  crunching numbers

53
Run Conditions

• Want valid terminating statistical analysis
• New replications must start independently with no
  model-state carry-over
• Run/Setup/Replication Parameters
• Check Initialize System Between Replications
• Still check Initialize Statistics Between
  Replications
• Get truly independent and identically distributed
  replications
• Unreturned tech-support follow-up calls lost
  unrealistic
• Compromise redefine a replication to be a
  five-day work week Monday-Thursday returned
  tech-support calls carried over, Friday-night
  ones lost
• Run/Setup/Replication Parameters Replication
  Length 5 Days, specify 11 Hours Per Day
• Run/Run Control Batch Run (No Animation)

54
Slimming Down and Speeding Up

• Academic version places several different kinds
  of limits on model size
• Max of 150 concurrent entities though millions
  could pass through this limit is not a problem
  here
• Max of 150 module instances includes
  flowchart modules, and each entry (line) in data
  modules this limit is a problem
• Also need room to add new output performance
  measures
• Reduce number of module instances
• Eliminate many statistical accumulators
  included getting rid of lines in data modules,
  entire flowchart modules, and unchecking
  stat-collection boxes details in book
• This also increased speed by factor of 3 to 4
• Important since this model will be exercised
  intensively

55
Overall Performance Measures

• Form an overall cost measure reduce, minimize
• What controllable input parameters affect cost?
  How?
• Two components to cost
• Due to staffing and resources tangible,
  measurable
• Due to poor customer service intangible, hard
  to measure
• Staffing and resource costs
• Hourly salaries 18 for sales, 16 to 20 for
  tech-support depending on training (see Resource
  data module)
• Salaries paid whenever person is on duty, whether
  busy or not
• Get current weekly payroll of 13,110 Staff
  Cost variable

56
Generalize for More Staff

• Try to improve service via more staff
• Will certainly increase staff cost
• Try to improve customer service to make it
  worthwhile
• Base-model results worst staffing shortfalls
  are between 1130AM and 330PM
• Add sales and tech-support staff for that
  four-hour period (half-hours 8 through 15)

57
Additional Sales Staff

• Add variable New Sales to be the number of
  additional salespersons to add in periods 8-15
• Define in Variable data module
• Use in Schedule data module (under Sales
  Schedule) add to number of sales staff in base
  model in periods 8-15
• Must use Edit via Dialog or Spreadsheet since
  Graphical Schedule Editor cannot handle Variables
• Cost for each new salesperson 15/hour
• Each will work 20 hours/week, so cost 300/week
• Variable New Sales Cost set to 300
• Additional cost is (New Sales) (New Sales
  Cost), used in Expression for for new resource
  cost (New Res Cost)

58
Additional Tech-Support Staff

• Possibly add new tech-support staff for products
  1, 2, 3 (only), and for all products
• Variables New Tech 1, New Tech 2, New Tech 3
• Named Larrys, Moes, and Curlys, respectively
• Each paid 14/hr 20 hrs/week 280/week
  (variable LMC cost)
• Variable New Tech All
• Named Hermanns
• Each paid 17/hr 20 hrs/week 340/week
  (variable Her cost)
• Resource data module define resources Larry,
  Moe, Curly, Hermann and hourly costs for them
  (not used)
• Set data module add Larry, Moe, Curly, Hermann
  to appropriate resource sets
• Schedule data module add Larry, Moe, Curly,
  Hermann schedules cant use Graphical Schedule
  Editor

59
Changing the Number of Trunk Lines

• Each trunk line costs a flat 89/week, including
  all calls (even long-distance)
• Is 26 the right number?
• To change it, just edit the Capacity entry in the
  Resource data module
• Add variable Line Cost to be 89, multiply by
  number of trunk lines (MR(Trunk Line)) to get
  weekly cost of trunk lines

60
Total New Resource Cost

• Define an Expression called New Res Cost
• New Sales New Sales Cost
• (New Tech 1 New Tech 2 New Tech 3) LMC
  Cost
• New Tech All Her Cost
• Line Cost MR(Trunk Line)
• Does not depend on what happens during simulation
  used only at end in Statistic module
• Does not include cost of the base-model human
  staff (sales, tech-support) viewed as sunk, and
  constant for all variants of staffing changes

61
Costs for Putting Customers on Hold

• Impute a cost for making customers wait on hold
• Trade off against resource costs
• Use model to understand, improve, optimize this
  tradeoff
• Such customer-dissatisfaction costs are hard to
  quantify
• People have a tolerance for holding
• Tech-support calls 3 minutes (variable Tolerance
  Tech)
• Sales calls 1 min. (Tolerance Sales)
• Order-status calls 2 min. (Tolerance Order
  Status)
• Beyond the tolerance point, system incurs cost of
• Tech-support calls 1.67/min. (variable TWT
  Cost)
• Sales calls 3.72/min. (SWT Cost)
• Order-status calls 1.58/min. (OSWT Cost)

62
Costs for Putting Customers on Hold (contd.)

• Accumulate excess waiting time (time past
  tolerance) for each call type
• Assign module when call is done
• Use built-in Arena attribute ENTITY.WAITTIME
• Accumulates total of times in queues as entity
  goes along, and other Wait-allocated times
  but there are none upstream in this model so this
  attribute will have the waiting time on hold
• Requires that Costing box be checked in
  Run/Setup/Project Parameters
• Variable Excess Tech Wait Time accumulates via
  adding in for each tech-support call
• MAX( ENTITY.WAITTIME - Tolerance Tech, 0 )
• At end of run, multiply Tech Wait Time by TWT
  Cost
• Similarly for Sales, Order-Status calls

63
Total Cost

• Adding together all the costs, get the overall
  economic (cost) performance measure
• Total Cost New Res Cost
• Excess Sales Wait Time SWT Cost
• Excess Status Wait Time OSWT
  Cost
• Excess Tech Wait Time TWT Cost
• Staff Cost
• This is defined in Statistic data module
• Type Output already being computed, just
  report it
• In Category Overview Report, get via User
  Specified ? Other ? Output

64
Percent-Busy Requirement

• Above cost performance measure ignores calls
  balked away due to no trunk line busy signal
• Clearly, undesirable very hard to put a cost on
  it
• Instead, have a strong goal to limit this to no
  more than 5 of incoming calls a model
  configuration not satisfying this will be deemed
  unacceptable no matter how attractive (low) the
  cost may be
• Like a constraint except its on an output, not
  an input call it a requirement
• Compute via two Record modules in arrival
  submodel to count incoming and balked calls and
  Percent Busy line in Statistic module

65
Aside Generality of Models

• We could have done a lot of things in very
  different ways in this model
• Using Arenas costing functions more and doing
  fewer of our own external calculations
• Reparameterize using only primitive parameters
  (e.g., hourly wage rates) and programming Arena
  to do the calculations
• How much of this you do depends on models
  intended use and users
• Tradeoff between generality (elegance?) vs. time
  spent building the model

66
Results

• Base case (no more people, 26 trunk lines)
• Total cost (for the week) 34K
• Percent busy signals 11 (unacceptable)
• Added one resource unit for each type
• New Sales, New Tech 1, New Tech 2, New Tech 3,
  New Tech All, and go to 27 trunk lines
• Total cost (for the week) 28K
• Added resources reduced customer waiting time by
  more than enough to cover their cost
• Percent busy signals 3 (acceptable)
• Extra trunk line, plus added resources to move
  calls through
• Is the modification truly and reliably better???

67
Statistical Analysis of Output from Terminating
Simulations

• Random input leads to random output (RIRO)
• Run a simulation (once) what does it mean?
• Was this run typical or not?
• Variability from run to run (of the same model)?
• Need statistical analysis of output data
• From a single model configuration
• Compare two or more different configurations
• Search for an optimal configuration
• Statistical analysis of output is often ignored
• This is a big mistake no idea of precision of
  results
• Not hard or time-consuming to do this it just
  takes a little planning and thought, then some
  (cheap) computer time

68
Time Frame of Simulations

• Terminating Specific starting, stopping
  conditions
• Run length will be well-defined (and finite)
• Steady-state Long-run (technically forever)
• Theoretically, initial conditions dont matter
  (but practically they usually do)
• Not clear how to terminate a simulation run
• This is really a question of intent of the study
• Has major impact on how output analysis is done
• Sometimes its not clear which is appropriate
• Here Terminating (steady-state in Section 6.3)

69
Strategy for Data Collection and Analysis

• For terminating case, make IID replications
• Run/Setup/Replication Parameters Number of
  Replications field
• Check both boxes for Initialize Between
  Replications
• Separate results for each replication Category
  by Replication report
• Model 5-3, base case, 10 replications

70
Strategy for Data Collection and Analysis
(contd.)

• Category Overview report will have some
  statistical-analysis results of the output across
  the replications
• How many replications?
• Trial and error (now)
• Approximate number for acceptable precision
  (below)
• Sequential sampling (Chapter 11)
• Turn off animation altogether for max speed
• Run/Run Control/Batch Run (No Animation)

71
Confidence Intervals forTerminating Systems

• Using formulas in Chapter 2, viewing the
  cross-replication summary outputs as the basic
  data
• Possibly most useful part 95 confidence
  interval on expected values
• This information (except standard deviation) is
  in Category Overview report
• If gt 1 replication specified, Arena uses
  cross-replication data as above
• Other confidence levels, graphics Output
  Analyzer

72
Half Width and Number of Replications

• Prefer smaller confidence intervals precision
• Notation
• Confidence interval
• Half-width
• Cant control t or s
• Must increase n how much?

Want this to be small, say lt h where h is
prespecified
73
Half Width and Number of Replications (contd.)

• Set half-width h, solve for
• Not really solved for n (t, s depend on n)
• Approximation
• Replace t by z, corresponding normal critical
  value
• Pretend that current s will hold for larger
  samples
• Get
• Easier but different approximation

s sample standard deviation from
initial number n0 of replications
n grows quadratically as h decreases.
h0 half width from initial number n0 of
replications
74
Interpretation of Confidence Intervals

• Interval with random (data-dependent) endpoints
  thats supposed to have stated probability of
  containing, or covering, the expected valued
• Target expected value is a fixed, but unknown,
  number
• Expected value average of infinite number of
  replications
• Not an interval that contains, say, 95 of the
  data
• Thats a prediction interval useful too, but
  different
• Usual formulas assume normally-distributed data
• Never true in simulation
• Might be approximately true if output is an
  average, rather than an extreme
• Central limit theorem
• Issues of robustness, coverage, precision
  details in book

75
Comparing Two Alternatives

• Usually, want to compare alternative system
  configurations, layouts, scenarios, sensitivity
  analysis here just two alternatives
• Base case of Model 5-3, vs. adding one resource
  unit for each type
• New Sales, New Tech 1, New Tech 2, New Tech 3,
  New Tech All, and go to 27 trunk lines
• Earlier, one run of each suggested big
  differences real?
• Reasonable but not-quite-right idea Make
  confidence intervals on expected outputs from
  each alternative, see if they overlap
• Doesnt allow for a precise, efficient
  statistical conclusion

76
Compare Means via the Output Analyzer

• Output Analyzer is a separate application that
  operates on .dat files produced by Arena
• Not installed by default from book CD need
  custom install
• Launch separately from Windows, not from Arena
• To save output values (Expressions) of entries in
  Statistic data module (Type Output) enter
  filename.dat in Output File column
• Just did for Total Cost, not Percent Busy
• Will overwrite this file name next time either
  change the name here or out in Windows before the
  next run
• .dat files are binary can only be read by
  Output Analyzer

77
Compare Means via the Output Analyzer (contd.)

• Start Output Analyzer, open a new data group
• Basically, a list of .dat files of current
  interest
• Can save data group for later use .dgr file
  extension
• Add button to select (Open) .dat files for the
  data group
• Analyze/Compare Means menu option
• Add data files A and B for the two
  alternative
• Select Lumped for Replications field
• Title, confidence level, accept Pared-t Test,
  Scale Display

78
Compare Means via the Output Analyzer (contd.)

• Results
• Confidence interval on difference misses 0, so
  conclude that there is a (statistically)
  significant difference

79
Evaluating Many Alternatives with the Process
Analyzer (PAN)

• With (many) more than two alternatives to
  compare, two problems are
• Simple mechanics of making the possibly many
  parameter changes, making the runs, keeping track
  of the many output files
• Statistical methods for drawing reliable and
  useful conclusions
• Process Analyzer (PAN) addresses these
• PAN operates on program (.p) files produced
  when .doe file is run (or just checked)
• Start PAN from Arena (Tools/Process Analyzer) or
  via Windows
• PAN runs on its own, separate from Arena

80
PAN Scenarios

• A scenario in PAN is a combination of
• A program (.p) file
• Set of input controls that you choose
• Chosen from Variables and Resource capacities
  think ahead
• You fill in specific numerical values
• Set of output responses that you choose
• Chosen from automatic Arena outputs or your own
  Variables
• Values initially empty to be filled in after
  run(s)
• To create a new scenario in PAN, double-click
  where indicated, get Scenario Properties dialog
• Specify Name, Tool Tip Text, .p file, controls,
  responses
• Values of controls initially as in the model, but
  you can change them in PAN this is the real
  utility of PAN
• Can duplicate (right-click, Duplicate) scenarios,
  then edit for a new one
• Think of a scenario as a row

81
PAN Projects and Runs

• A project in PAN is a collection of scenarios
• Program files can be the same .p file, or .p
  files from different model .doe files
• Controls, responses can be the same or differ
  across scenarios in a project usually will be
  mostly the same
• Think of a project as a collection of scenario
  rows a table
• Can save as a PAN (.pan extension) file
• Select scenarios in project to run (maybe all)
• PAN runs selected models with specified controls
• PAN fills in output-response values in table
• Equivalent to setting up, running them all by
  hand but much easier, faster, less error-prone

82
Running Model 5-3 with PAN

• Scenarios
• Base case (no additional resources)
• Imagine 1200/week to spend on each additional
  resource type, one at a time (no mixed
  enhancements)
• 7 scenarios in all (details in book)
• Select all to run (click on left of row,
  Ctrl-Click or Shift-Click for more)
• To execute, or Run/Go or F5

83
Statistical Comparisons with PAN

• Model 5-3 alternatives were made with 10
  replications each
• Better than one replication, but what about
  statistical validity of comparisons, selection of
  the best?
• Select Total Cost column, Insert/Chart (or
  or right-click on column, then Insert Chart)
• Chart Type Box and Whisker
• Next, Total Cost Next defaults
• Next, Identify Best Scenarios
• Smaller is Better, Error Tolerance 0 (not the
  default)
• Show Best Scenarios Finish

84
Statistical Comparisons with PAN (contd.)

• Vertical boxes 95 confidence intervals
• Red scenarios statistically significantly better
  than blues
• More precisely, red scenarios are 95 sure to
  contain the best one
• Narrow down red set more replications, or Error
  Tolerance gt 0
• More details in book

85
Searching for an Optimal Alternative with OptQuest

• The scenarios just considered with PAN are just 7
  of many, many possibilities
• Try to find input-control values that minimize
  Total Cost while keeping Percent Busy lt 5
• Formulate as an optimization problem
• Minimize Total Cost
• Subject to 26 ? Trunk Lines ? 50
• New Sales, New Tech 1, New Tech 2, New Tech
  3, New Tech All ? 0
• New Sales New Tech 1 New Tech 2 New
  Tech 3 New Tech All ? 15
• Percent Busy lt 5
• Reasonable starting place best acceptable
  scenario so far Add 3 New Tech All
• Where to go from here? Explore all of feasible
  six-dimensional space exhaustively? No.

86
OptQuest

• OptQuest searches intelligently for an optimum
• Like PAN, OptQuest
• Runs as a separate application can be launched
  from Arena
• Takes over the running of your model
• Asks that you identify the input controls and the
  output (just one) response objective
• Unlike PAN, OptQuest
• Asks that you specify constraints on the input
  controls
• Asks that you specify requirements on outputs
• Decides itself what input-control-value
  combinations to try
• Uses internal heuristic algorithms to decide how
  to change the input controls to move toward an
  optimum configuration
• You specify stopping criterion for the search

87
Using OptQuest

• Tools/OptQuest for Arena
• New session (File/New or CtrlN or )
• Make sure the desired model window is active
• Select controls Variables, Resource levels
• Trunk Line, New Tech 1, 2, 3, and New Tech All
• Bounds 26 ? Trunk Line ? 50, others between 0
  and 15
• Type is Discrete for all, Input Step Size 1
• Constraints enter
• New Sales New Tech 1 New Tech 2 New Tech 3
  New Tech All lt 15
• Objective and Requirement
• Total Cost Response Select Minimize Objective
• Percent Busy Response Select Requirement, enter
  5 for Upper Bound
• Reorder

88
Using OptQuest (contd.)

• Options window computational limits, procedures
• Time tab accept Run for 10 minutes default
• Precision tab vary number of replications from
  3 to 10
• Preferences tab various settings (accept
  defaults)
• Can revisit Controls, Constraints, Objective and
  Requirements, or Options windows via
• Run via wizard (first time through a new
  project), or Run/Start or
• Open View/Status and Solutions and
  View/Performance Graph to watch progress
• Cant absolutely guarantee a true optimum
• Usually finds far better configuration than
  possible by hand