Spar Nord Banks application score

1
Spar Nord Banks application score

• a rating system for new retail customers
• Rasmus Waagepetersen
• Spar Nord Bank
• DK-9100 Aalborg

2
Basel II background

• Basel II regulations for calculation of capital
  requirements (solvency).
• Capital requirement depends on the banks risk
  profile.
• Three types of risk
• credit risk risk that customer does not pay back
  his/her loan (i.e. default)
• market risk e.g. the risk that stock holdings
  loose value
• operational risk e.g. break down of computer
  systems or fraud
• Capital requirement related to credit risk
  own capital at least 8 of risk weighted assets.
• Risk weighted assets each asset (loan) is
  multiplied with a risk weight depending on
  probality of default (PD) and loss given
• default (LGD).

3
Risk weight formula

• Based on binomial mixed model
• latent variable Z state of economy
• given Z, indicators of default X_1,,X_n
  conditionally independent Bernouilli variables
  with conditional PD P(X_i1Z)F(a_ibZ)
• where F() standard normal distribution
  function (binomial GLM with probit link).
• a_i controls size of PD for ith exposure, b
  controls correlation between defaults.
• Portfolio loss LL_1X_1L_nX_n (L_i loss
  given default for ith costumer).
• Risk weight formula based on asymptotic formula
  for 99 quantile of L (value of risk) (n tends to
  infinity, L_i tends to zero)

4
IRB internal rating based approach

• Risk weight
• value of b supplied by Basel II regulations
  (correlation depending on loan type)
• value of a_i obtained from PD_iP(X_i1)
  estimated internally (IRB).
• IRB PD_i and LGD_i estimated from banks internal
  assessment of risk/banks own historical data.
• Central ingredient rating system places
  loans/customers in rating classes which are
  differentiated with respect to risk (PD and LGD).

5
Rating system for new retail customers/application
score

• Rating of new customers based on variables such
  as age, type of housing, income, assets, debts,
• NB for existing customers additional information
  is available transaction behaviour, overdrafts,
  cash flow, (behavioural score)

6
Empirical model

• Rating system may be based on direct estimation
  of probability of default (logistic regression).
• Problem low quality of historical data. Missing
  variables or incorrect records.
• Common problem use of quantitative methods for
  credit risk management still quite new in
  conventional danish banks.
• Problem frequency of default quite low (1
  within a one year timespan) in historical data.
  Hence large data sets needed in order to fit a
  differentiated model.
• (binary observations provided limited
  information)

7
Expert model

• Aim construct a model which based on customer
  variables gives a rating which an experienced
  Spar Nord Bank credit officer would give based on
  the same variables.
• Advantage historical data obtained in a period
  of favorable economic conditions. Ratings from
  experienced bank people may reflect knowledge of
  difficult times (1990s).
• Accept among users model reflects best practice.
  

8
Basic model for rating systems
Customer variables
Weights

• Problem obtain weights so proper balance
• between variables contributing to the score
• Problem convert score into rating consistent
  with
• rating of an experienced bank person
• Consultants in PWC or the like will suggest
• various ad hoc solutions
• Credit people trained to assess customers not
• to assign weights
• Better solution let credit people rate
  customers
• and leave computation of weights to
  statistician

age, capital, income,
w1, w2, w3,
score
Red, yellow, green rating (traffic light)
9
Statistical model based on expert ratings
Weights w1, w2, parameters to be estimated in
regression model for expert ratings given
customer variables.
Data
Ratings from panel of 19 experts
Population of around 3000 customer cases
case1 age, capital, case2 age,
capital, case3 age, capital,
red green, green,
10
Rating scale and design of experiment
6 step scale

• 2 rating workshops one week between
• 15 experts each rated 105 cases (25 cases common
  to all experts)
• 17 experts each rated 102 cases (13 also took
  part in first workshop)

In total 3321 cases rated. The 25 common cases
enables direct comparison of experts.
Stratified sample of customer cases for each
expert capital, debt factor (i.e. debt/income)
and good/bad status (manual classification)
11
Rating af common cases 1-4
25 cases rated by all experts
G/V indicates good/bad status
Considerable variation for average customers
12
Threshold-model for rating data
Score weighted sum of customer variables sx1w1x
2w2
Expert assessment (latent variable) VsU where
E(U)0
Thresholds V below T1 yields red, between T1 og
T2 yellow below average etc.
Greater probability for red rating with score
S1 compared with score S2
13
Interpretation of VsU

• experts only see customer variables and not score
  (score mathematical construction).
• U reflects rating variation an expert may assign
  different ratings to customers with same score
  (measurement error)
• - moreover
• variation between experts.
• variation between workshops.
• Obvious variance component model (later)
• Logistic distribution for U yields cumulative
  logistic regression/proportional odds model

14
Variables in model

• Basic variables age, size of household, type of
  housing, type of loan (fixed or variable interest
  rate, with or without amortization), income,
  assets, debts
• Derived variables single parent, capital, debt
  factor, income per person in household,
  solvency ratio (capital/assets)
• Interactions capital/age, capital/debt factor,
  type of housing/debt factor,
• age/debt factor

In total 76 parameters (grouped quantitative
variables) estimated from 2900 customer cases
(omitted two extreme experts)
15
Evaluation of model

• does model fit expert ratings ?
• is it useful for identifying weak customers
  (sensitivity)?
• is models assessment of risk concordant with
  empirical risk ?
• does model classify too many good customers as
  weak (specificity)

16
Deviations between model ratings and expert
ratings
Model rating rating with highest probability
according to model
For 90 of expert ratings at most one step
deviation from model.
17
Comparison of model ratings and expert ratings
of common customer cases
model-plot shows model probabilities for each
rating
We can provide both the most probable rating but
also the precision of the rating
18
Standardized residuals
(consider ratings in 1,,6 as quantitative
variables)
Boxplot for each expert
19
Validity of proportional odds assumption
Plot empirical estimates of log odds after
grouping according to estimated score
Note very small odds when r1 and groups with
small scores sensitive to outliers
20
Sensitivity model ratings of weak customer cases
2006-2007 and 2008
(new customers identified as weak by internal
credit surveillance team)
2006-2007 72 rated red or yellow
2008 78 rated red or yellow
21
Specificity model ratings of strong customers
Tricky issue definition of a strong customer ?
Model ratings of customer cases with
behavioural score 1-3 2 years after first loan
7 red 19 yellow
22
Basel II

• Strict Basel II definition of default
  devaluation of loan or loss
• 0nly 30 defaults in data set with 3500 customer
  cases
• Dimension reduction expert model reduces large
  number of variables to just one number (score)
• Estimate PD using logistic regression with score
  as covariate

23
Variance components
Decomposition of latent expert assessment
24
Potential advantages of variance component model

• In model without expert effects, opinion of
  experts who rated two data sets count more than
  experts who only rated one data set.
• More appropriate quantification of variation in
  data.
• Need numerical integration to compute likelihood
  and predictive probabilities

25
Results for variance component model (GLIMMIX)

• variance for logistic distribution 3.29
• estimated variance for expert effects 0.55
• with 2 extreme experts omitted 0.31
• largest variance component mea-surement error
  (i.e. logistic distribution)

Predictions of expert effects
26
Representation of rating
Bar represents probabilities of red, green and
yellow representation of model certainty