Efficient Ticket Routing by Resolution Sequence Mining

1
Efficient Ticket Routing by Resolution Sequence
Mining

• Qihong Shao, Yi Chen Shu Tao, Xifeng Yan, Nikos
  Anerousis
• Arizona State University IBM T.J.Watson Research
  Center

2
Roadmap

• Motivation
• Problem Definition
• Resolution sequence mining
• Routing algorithms
• Experiments
• Summary

3
Motivation

• Managing problem tickets is a key issue in IT
  service industry.
• The key to problem solving is to find the right
  expert who can solve the problem.
• The efficiency of problem ticket management
  highly depends on ticket routing
• transferring tickets among expert groups in
  search of the right resolver.
• Currently, ticket routing is based on human
  decision.

4
Problem Definition

• Can we improve the overall efficiency of ticket
  routing ( MSTR Mean of Steps To Resolve ) ?
• the set of
  all expert groups.
• 
  the routing sequence of a ticket
• a ticket is first issued to g(1), transferred in
  the order of g(2), g(3), g(k),
• Gi ( i1,2,,m) m ticket routing sequences
• More steps involved, the longer delay
• Goal minimize MSTR

5
Research Challenges

• The great diversity of reported problems ( e.g.
  500 problem categories in our testing data ).
• Problem category (e.g., DB2, AIX) is specified by
  a set of predefined parameters
• The large number of expert groups to choose from
  ( e.g. 50 groups involved in each category in
  our testing data ).

6
Roadmap

• Motivation
• Problem Definition
• Resolution sequence mining
• Routing algorithms
• Experiments
• Summary

7
Is Resolution Sequence Mining Enough?

• Ticket data that can be utilized
• Resolution sequence (sequence of groups)
• Ticket content
• Can we reduce MSTR by mining only ticket
  resolution sequence and identifying ticket
  transfer patterns?
• The answer is YES!!!!!
• If a new ticket share similar resolution sequence
  with a closed ticket
• They are likely to have similar root cause.
• As a result, they might share similar transfer
  route toward resolution
• Past ticket transfer decisions reflect functional
  relationships between expert groups.

8
Resolution Sequence Mining based on Markov
Modeling

• Markov model to capture past ticket transfer
  decisions embedded in resolution sequences
• A naturally fit for our problem.
• Each state an expert group holding the ticket
• Transition probabilities given the previous
  groups, the probability of the ticket being
  transferred to group gi, as the next step

9
How to Train the Markov Model?

• Shall we train the model using all intermediate
  transfer steps or only the last steps (to
  resolver group)?
• Approach(1) only use the last step
• Approach(2) all transfer steps, including the
  intermediate steps
• For example, a resolution sequence ltA, B, Cgt
• Approach(1) use ltA,Cgt, ltB, Cgt, ltA, B, Cgt for
  training
• Approach(2) use ltA,Cgt, ltB, Cgt, ltA, B, Cgt, ltA,
  Bgt for training
• We choose approach (2), the intuition
• Majority of local ticket transfer decisions were
  logically correct
• Long resolution sequences were typically results
  of few local misrouting decisions.

10
How to Determine the Order?

• How many past states are considered to predict
  the future state?
• Use conditional entropy to determine Markov
  order
• The higher the order, the better prediction, the
  more complex the system
• Find a right tradeoff for order k
• Beyond threshold, the improvement of predict
  ability becomes small

11
System diagram
Markov model
closed tickets
Sequence Extractor
Ticket DB

AIX
DB2
Windows
Resolution Sequence Miner
Routing model 1
Routing model 2
Routing model 3
Routing Engine
open ticket
Customers
Routing recommendations
12
Roadmap

• Motivation
• Problem Definition
• Resolution sequence mining
• Routing algorithms
• Experiments
• Summary

13
First-order Memoryless (FM)

• Similar to depth-first search
• Choose the next node with the highest transition
  probability.
• Stop until find the resolver or reach a node
  without any unvisited neighbor nodes.
• Same group should not be visited twice.

• Drawback
• only relies on the current state to make
  transfer decisions
• May not lead to the best direction toward the
  resolver

14
First-order Multiple active State(FMS)

• Ticket transfer decision is based on any one of
  the past states (instead of the current state,
  which may be result of a wrong decision).
• Lv the groups visited so far.
• Lc unvisited neighbors of all visited groups.
• check all candidate groups g of visited group Lc.
• select the next group that maximizes the
  first-order transition probability.

15
Variable order Multiple active State (VMS)

• With multiple active states, use higher-order
  whenever it can make more confident prediction
• Lv visited group set
• Lc candidate group set
• check all available transfer probabilities for
  all of the groups that have been visited in the
  past.
• select the next group g that maximizes the
  transfer probability from S(k).

• Why not ALWAYS higher order?
• Higher order may be not available due to the
  limited size of training dataset
• Lower order rule has a stronger indicator than
  the higher order one

16
Roadmap

• Motivation
• Problem Definition
• Resolution sequence mining
• Routing algorithms
• Experiments
• Summary

17
Experiments

• Experiment settings
• 1.4 million problem tickets from IBM in 1 year
  span (Jan 1, 2006 to Dec 31, 2006)
• 553 problem categories
• On average, 50 - 900 groups (both resolvers and
  non-resolvers) were involved in resolving the
  tickets in each problem category
• Divide data into training set and testing set,
  compared the resolution sequences with and w/o
  using our approach
• Our experiments demonstrate
• Effectiveness
• Robustness
• Case study

18
Effectiveness of search algorithms

• Performance VMS(J) gt VMS(I) gt FMS gt FM

VMS(J) considers all intermediate steps in
training VMS(I) consider only the resolver steps
19
MSTR improvement

• MSTR improvement for different problem categories

Human decision
Improvement
20
Robustness
Slide the window
Training set size
Different problem categories
The effectiveness of our approach is CONSISTANT!
21
Why our model works A case study
GUI is failing with Unable to logon.. . .. .
.
Stopped transition on g2 and g4, recycle WAS on
e8/e9/ec/ed Then restart transitions. But still
does not work
Steps that can be bypassed
Resolution DB2B was recycled
22
Summary

• We develop an approach to improve ticket routing
  by mining historical resolution sequences.
• Developed Markov model with variable-order to
  statistically capture the ticket transfer
  decisions made in the past
• Based on the model, we designed an algorithm (
  VMS ) to generate routing recommendations
• implemented a prototype ticket routing
  recommendation system EasyTicket (demo in
  VLDB08)
• 2. Future work
• extend our approach with text mining capabilities
  for better performance.

23
Thank You!
Questions? Welcome to visit EasyTicket demo in
VLDB 08 and Todays(Tue, 26th) poster session!