Process Equivalence: Comparing Two Process Models Based on Observed Behavior

1
Process EquivalenceComparing Two Process Models
Based on Observed Behavior

• Wil van de Aalst Ana Karla Medeiros Ton Weijters
• Eindhoven University of Technology
• Department of Information Systems
• a.k.medeiros_at_tm.tue.nl

2
Outline

• Process Equivalence
• Process Mining
• Metrics for Equivalence Quantification
• Applications
• Conclusions and Future Work

3
Process Equivalence Objective
How similar are the behaviors of two process
models?
100? 80? 20? 0?
Our aim Quantify equivalence!
4
Process Equivalence Current Approaches
1) Structural comparison
Do the processes have the same structure?
Different structures do not necessarily imply
different behavior!
5
Process Equivalence Current Approaches
2) Trace Equivalence
Do the processes generate the same set of traces?
Set of traces must be finite! Does not consider
frequency of traces!
6
Process Equivalence Current Approaches
3) Other approaches bisimilarity, branching
bisimilarity, etc.
Yes/no answer!
7
Process Equivalence Our Approach
Compare two process models based on example
behavior!
Log
8
Process Mining Objective
Log
Mining Techniques
9
Process Mining Motivation

• Before deployment
• Objective picture of how the process has been
  executed
• After deployment
• Feedback mechanism

10
Process Mining Evaluation
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Original Process
Logs
Genetic Miner
Mined Process
How much of the behavior in the log can be
generated by the mined process? How close is the
behavior of the mined process to the original one?
11
Process Mining Evaluation
aaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaa

• How much of the behavior in the log can be
  generated by the mined process?
• Genetic Miner How fit is the mined model?
• Fitness percentage of the log that can be
  correctly parsed by the mined model
• Continuous semantics parsing
• Problems are registered, but tasks are replayed
  even when disabled

12
Process Mining Fitness
13
Process Mining Fitness
14
Process Mining Fitness
15
Process Mining Fitness
16
Process Mining Fitness
17
Process Mining Fitness
?
18
Process Mining Fitness (F)
?
?
?
?
F 1.0
19
Process Mining Fitness
?
F (40 ) / 160
20
Process Mining Fitness
?
?
F (40 85 ) / 160
21
Process Mining Fitness
?
?
?
F (40 85 ) / 160
22
Process Mining Fitness
?
?
F (40 85 ) / 160
23
Process Mining Fitness
?
?
F (40 85 ) / 160
24
Process Mining Fitness
?
?
3/4
F (40 85 (153/4) ) / 160
25
Process Mining Fitness
?
?
3/4
3/4
F (40 85 (153/4) (203/4)) / 160 0.945
26
Process Mining Fitness (F)
F
1.0
0.945
0.828
1.0
27
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)
• How much behavior two models (M1 and M2) have in
  common with respect to a log (L)?
• Main idea
• Compare the set of enabled tasks in both models
  while replaying the traces in the log
• Consider the frequency of log traces

28
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)

M1
M2
29
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)

M1
M2
A
A
Intersection Enabled(M1,M2,pos)
A
30
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)

M1
M2
B, C, D
B, C, D
Intersection Enabled(M1,M2,pos)
B,C,D
31
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)

M1
M2
D
D
Intersection Enabled(M1,M2,pos)
D
32
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)

M1
M2
E
E, F
Intersection Enabled(M1,M2,pos)
E
33
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)

M1
M2
Intersection Enabled(M1,M2,pos)

34
Metrics for Equivalence Quantification
M1

• Behavioral Precision
• M2 with respective to M1, based on the log L
• Behavioral Recall
• M1 with respective to M2, based on the log L
• Frequency of the traces are taken into account!
• Example
• Bp(M1,M2,L) 0.75 and Br(M1,M2,L)1.0

L
M2
35
Metrics for Equivalence Quantification

• Behavioral precision (Bp) and recall (Br)

36
Applications

• Process Mining
• Assess the quality of the mined models
• Do they really portrait the most frequent
  behavior in the log?
• Log may be incomplete or contain noise
• Detect over-general models
• Models that can parse
• any log with the tasks
• in the input log

37
Applications

• Metrics are implemented in the ProM framework,
  available at processmining.org

38
Conclusions

• Behavioral precision and recall are useful
  metrics to quantify how similar two process
  models are with respect to a log
• Main advantages
• Frequencies of the log traces are considered
• Log is not required to be exhaustive
• Models do not need to be able to parse all the
  behavior in the log
• Metrics are implemented in the open source tool
  ProM

39
Future Work

• Apply the metrics to other scenarios beyond the
  process mining scope
• If available, extend the metrics to consider
  semantic information

www.ip-super.org
40
http//www.processmining.org
Thank you! Questions? Comments?