Approaching Predictive Modeling

1
Approaching Predictive Modeling
From a Different Perspective

• Steven S. Eisenberg, MD
• Chief Science Officer
• United HealthCare

4th Annual DM Colloquium June 23, 2005
2
Overview

• Types of predictive modeling
• What predictive modeling can do
• The usual
• The less usual (focus)
• Summary
• QA

3
My Favorite Philosopher on Predictive Models
4
What Are We Hoping to Learn?

• From an actuarial perspective
• more accurately predict utilization and cost of
  populations
• adjunct to better and more accurate pricing
  decisions
• Perhaps flattening the actuarial cycle
• From a medical management perspective
• identify individuals at very high risk for high
  utilization
• open the door to managing those individuals
• case management/disease management
• prior to the high utilization
• mitigating some of the impact
• healthier population
• lower costs

Leading to Improvements in Disease
Management Patient Care Quality Cost
Management
5
The Tools

• Model is a mathematical representation of reality
• Relevant, consistent input data are needed
• The outcome must be measurable
• A way to relate the two mathematically must exist
  
• Currently well over 100 different models
• For our (healthcare) purposes these really break
  down into three groups
• Artificial Intelligence
• Statistical Models
• Rules Based Algorithms

6
(No Transcript)
7
One Size Fits All ?
8
What Predictive Modeling Can Do

• Stratify members
• Primary or secondary
• Enhance impact of interventions
• Identification of high utilizers
• Assign risk scores
• Describe comparative severity of illness
• Identify members not receiving proper
  care/requiring special care
• Case Management/Disease Management
• Highlight inconsistency/inefficiency of care
• Prospectively identify adverse events
• Allow focused interventions
• Maximize benefits of disease management
• Allow intervention earlier in disease cycle
• Financial forecasting Actuarial risk

9
Application of Predictive Models

• Identifying/managing complexly ill members
  (hospitalization avoidance)
• Refining disease management strategies
• Managing pharmacy services (integrated with
  medical management)
• Underwriting more precisely
• Reimbursement based on illness burden
• Assessing physician management strategies

10
Additional Uses of Modeling

• Influence adoption of best practices
• Track effectiveness of interventions
• Establish pay for performance
• Set more accurate premiums
• Develop contracts with providers
• Actuarial
• Help plan network composition
• Based on member needs
• Develop specific, targeted interventions
• Probabilities for certain outcomes
• Practice guidelines
• Practice standardization
• Decrease variation

11
Choosing the Right Model

• There is no one model that does everything the
  best
• What are you trying to do?
• Is there a model that fits the problem (or the
  data) better?
• What is available to you?
• Can you use whatever model/data you
  have available?
• What can you afford?
• What are you willing to compromise on?

12
The Usual Suspects

• Most DM programs / Healthplans use grouper rules
  based algorithms prospectively
• ERGs/ETGs
• DCGs
• ACGs
• Most Fraud Abuse programs use
• Decision trees
• Rules based algorithms
• Neural nets
• Pattern analysis

13
Some Less Usual Suspects

• There are some inexpensive and less cumbersome
  ways to do some predictive modeling
• Trend lines
• Time series
• Markov Models
• Pharmacy only models

Statistical Models
14
Trend Lines Regression

• Definition the technique of fitting a simple
  equation to real data points
• Linear regression is the most common type
• e.g. yabxe
• Other Types
• Multilinear regression
• Logistic Regression

• It is a mathematical way of
  assessing the impact and
  contribution of diverse/disparate variables on a
  process or outcome
• Linear regression is used for continuous
  variables
• Logistic regression is used for binomial
  variables

15
Trend Lines

• Poor mans Predictive model
• Built into Excel

16
Example
17
Doing a Prediction
Double click on the trendline
18
Project the Trendline 6 Months Forward
The Prediction
By doing so you are making the assumption
that all the variables are and will
remain constant
19
Time Series

• Time series analysis accounts for the fact that
  data points taken over time may have an internal
  structure reflecting a pattern or more than one
  pattern
• Trend
• Seasonal variation (seasonality)
• General aspects
• Trend
• systematic linear or (most often) nonlinear
  component that changes over time and does not
  repeat or at least does not repeat within the
  time range captured by our data
• Seasonality
• May have a relationship similar to trend
  but tends to repeat
  itself in sytematic
  intervals over time

20
Common Uses of Time Series

• Economic Forecasting
• Sales Forecasting
• Budgetary Analysis
• Stock Market Analysis
• Yield Projections
• Process and Quality Control
• Inventory Studies
• Workload Projections
• Utility Studies
• Census Analysis

Example Pharmacy Utilization
21
Example

• Pharmacy Utilization over time Excel w Trend
  Line

Trend Line
22
Example When Analyzed via Time Series
23
Example
24
The Prediction Using Time Series
25
Markov Models

• A probabilistic process over a finite set of
  possibilities, S1, ..., Sk, usually called its
  states
• The model is capable of showing the probability
  of any given state coming up next, pr(xtSi), and
  this may depend on the prior history (to t-1).
• originally introduced in the late 1960s and
  early 1970s
• used for a variety of applications in science and
  technology
• Markov disease state simulations portray the
  progression of disease over time
• It does this by dividing the disease into
  discrete states,
• specifying the risks of progression per unit time
  between those states,
• assigning utilities and costs to each state,
• and conducting a simulation with a defined
  end-point.

26
Markov Models Are Often Represented Graphically

• Transition Probability Matrix

• State Transition Figure

27
The Midwest Healthplan Project

• A Real World Example of Using
• Markov Models

28
Project Overview

• A large Midwest Healthplan wants to understand
  the movement of members by segments over time and
  be able to identify future high cost utilizers
• Leverage their investment in medical management
  programs/techniques
• Help control runaway medical inflation
• Make points with their large employers by showing
  proactive management
• Wanted to do it without having to buy and support
  another technology

29
Project Overview, cont.

• Focusing on certain higher risk parts of their
  book of business we developed the Markov Model
  for them to better understand their historical
  movement of members from one disease state level
  (severity) to another over time
• at a population level
• The model is then being run to predict what is
  predicted to occur over the ensuing 6-12 months
• The Healthplan can then de-encrypt and identify
  these members and reach out to them with
  case/disease management
• at an individual level
• Outcomes and costs can be monitored over time
• Pre- Post Analysis
• Matched Cohort Analysis

30
Diabetes Example Predicting Member Counts (By
Age Band and Gender)
(all diabetes)
31
The Distribution of Disease States
Baseline Population Level
32
The Markov Model Prediction
33
Individual Member Transition Prediction
Member A012556
34
Final Analysis
with thanks to Ken Kubisty, Bearing Point
Solutions
35
Pharmacy Risk Groups

• Rules based, member centric
• Uses only pharmacy, demographic, and eligibility
  data as the inputs
• Developed by Symmetry Health Data Systems
• Assigns weighted risk score individuals based on
• distribution of drugs a member is taking, age,
  and sex
• weights differ by
• Threshold assumption -- 250K, 100K, 50K, 25K
• Stop-loss amount is typically used as the cut-off
  point
• Combines PRG profile and weights
• represents relative health risk for a member for
  future period

36
Advantages

• Data
• Availability
• Cleanliness and accuracy
• Timeliness
• Cost effective IT and administration
• Supports more frequent risk assessment
• Predictive accuracy
• R squared and other predictive measures close
• to those of claims based systems

37
Disadvantages

• Pharmacy plus medical claims can improve accuracy
  e.g.
• Members w/ medical use, w/o pharmacy use
• Conditions where drugs not integral component of
  treatment
• Further stratification within a disease
• Incentives
• linking risk to specific drug treatments may not
  provide best incentives for efficient and quality
  care
• Linking risk to disease prevalence
• harder to do without disease categorization

38
Example
39
The DHS Pilot Project

• A Real World Example of Using
• PRGs

40
State of MN, Department of Human Services

• Desire to extend disease management to FFS
  Medicaid population
• 100,000
• High risk population
• High morbidity/Chronic Illness
• Very low income
• Distrust of managed care
• Need to demonstrate to legislature that concepts
  work for this population
• Establish the opportunity for a formalized DM
  approach to this population
• Collect a series of success stories
• Provide the data and the stories to the
  legislature

41
The Approach

• Use the tool as the first pass to provide the
  basic output file
• Rank order of patients by prospective risk
• Analyze the medical history of the highest risk
  members
• Create a clinical vignette of their medical
  history
• ? Focus on those conditions and diseases that
  have a track record of success in disease or case
  management
• Focus only on the top few percent of highest risk
  members
• About 250 for the pilot project

42
The Pilot

• Members included
• Medicaid FFS only
• Continuously enrolled for at least 18 months
• Members excluded
• Primary serious mental health diagnoses
• Members in skilled or unskilled nursing homes
• Primary concern is cognition
• Need for a short time frame
• Program began mid January 04
• Program ended mid June 04
• Final Dataset 14,443 members
• members with highest prospective risk score had a
  complete claims dump for the prior 18 months
• Highest 2 underwent detailed claims analysis

43
Results Top 24 Patients

• 11 females, 13 males
• Range 20-63 y.o.
• average 45.4 y.o.
• Costs for 18 months
• 5,432 - 491,331
• Average 117,945
• Total of Claims/Pt
• 195 2,531
• Average 1,129

• Diagnoses
• Diabetes - 11
• Chronic Renal Failure/ESRD - 9
• Post kidney transplant - 5
• HIV positive 5
• AIDS - 2
• Cystic Fibrosis - 3
• Active malignancy - 2
• Smokers - 5

44
Clinical Vignettes

• Member 1 is a 30 year old woman with long
  standing Cystic Fibrosis. She has problems with
  malaise, fatigue, skin disease, and hair loss as
  well as multiple dislocated vertebrae in her
  neck. She had a very rocky 18 month course with
  multiple recurring episodes of pneumonia
  requiring hospitalization as well as multiple
  episodes of dehydration and bouts of painful
  Herpes Simplex.
• Her prospective risk score was over 27 and she
  had the highest total expenditure in the dataset
  of 491,000 for the 18 month time period.

45
Summary and Conclusions

• Predictive Modeling is a tool
• It is a method, not an answer in itself
• Modeling is only an arrow to add to the quiverit
  is not the whole quiver
• Consider the use of multiple models
• just as multiple forms of assessment are done for
  diagnosis
• May increase reliability and accuracy
• Predictive modeling is also a way to better
  understand your data accuracy
• and conversely where you have problems with
  your data

46
Challenges of Predictive Modeling

• All of the models are more accurate at the
  aggregate (population) level than at the
  individual level
• Most results published are at the population
  level
• Population level may work well for actuarial
• Medical Mgmt is typically focused on the
  individual
• You can adjust (improve) the results by changing
  the threshold, the specificity, sensitivity, etc.
• Models demonstrate better R squared values when
  outliers are excluded
• e.g. Stop-loss amounts
• But the outliers may be exactly the members that
  you are trying to find to have the impact you are
  looking for

47
Summary Conclusions

• There is no one clearly superior
  predictive model
• Certain approaches may be more valuable for
  underwriting
• Other approaches may be more valuable for
  managing care
• The actionability quotient must also be
  considered
• If you cannot act on the results, the study is
  merely interesting
• Linking models with interventions can help you
  improve quality and efficiency of care

48
Summary and Conclusions

• All predictive models tend to overpredict low
  utilizers and under predict very high utilizers
• Some of this may be mitigated by using a
  threshold and excluding costs beyond a certain
  point (typically at a stop-loss amount)
• But this can exclude exactly those folks you may
  want to identify
• None of the models can predict random events
• Trauma
• Pregnancy
• Catastrophic Claims
• Measurement of success is very difficult
• How do you unmanage a case to determine
  savings?
• But the tools are very valuable, getting better,
  and can be made to work
• You will see increasing success over the next
  several years

49
You can either take action or you can hang back
and hope for a miracle. Miracles are great, but
they are so unpredictable. Peter Drucker
Are There Any Questions?