Benchmarking the Performance of US Banks

1
Benchmarking the Performance of US Banks

• R. Barr, SMU
• T. Siems, Federal Reserve Bank of Dallas
• S. Zimmel, SMU
• Financial Industry Studies, Dec. 1998
  www.dallasfed.org

2
Motivations and Goals

• Motivations
• Safety and soundness of banking system
• Protection of FDIC insurance fund
• Best allocation of examiner resources
• Goals
• Prioritization of on-site examinations
• Early-warning indicators of troubled banks

3
Objectives of the Research

• Benchmark the U.S. banking system over the last
  decade
• Assess performance with DEA-based model
• Isolate best- and worst-practice banks
• Support bank auditors by predicting trouble
• Evaluate DEA in large-scale benchmarking role

4
Previous Work

• Measuring bank management quality with DEA
• Barr, Seiford, Siems, 1993
• Bank Failure Prediction Model
• DEA score as input to logit forecasting model
• Barr and Siems, 1996
• Technical report versions available at
• www.smu.edu/barr

5
Data Envelopment Analysis

• A methodology for integrating and analyzing
  benchmarking data that
• Performs a multi-dimensional gap analysis
• Considers interactions, tradeoffs, substitutions
• Integrates all performance measures
• Gives an overall performance rating
• Suggests credible organizational goals,
  benchmarking partners, .

6
Bank Performance Model
Inputs (Resources, Xs)
Outputs (Desired outcomes, Ys)

• Earning assets
• Interest income
• Noninterest income

• Salary expense
• Premises fixed assets
• Other noninterest expense
• Interest expense
• Purchased funds

7
Defining Efficiency

• Efficiency ratio of weighted sums of the inputs
  and outputs (gt0)
• Defines best practice in a DEA model

8
How DEA Works

• Instead of using fixed weights for all units
  under evaluation,
• DEA computes a separate set of weights for each
  bank
• Weights optimized to make that banks score the
  best possible
• Constraints no banks efficiency exceeds 1 when
  using the same weights

9
Formulating a DEA Model

• There are many DEA models
• The basic idea in each is to choose a set of
  weights for DMU k that

10
Measuring Distance
Efficient frontier of best practice
f1
z
f
Inefficient bank
11
Introducing Expert Judgment

• Classic models may result in unreasonable weight
  assignments for inputs outputs
• e 0 weights on unflattering dimensions
• Can overemphasize secondary factors
• We added weight multipliers to the DEA
• Based on survey of 12 FRB bank examiners
• Used response ranges to set UB/LBs on weights

12
Survey-Derived Constraints
Analytic Hierarchy
Survey range
Survey average
process weights
Inputs
Salary Expense
15.8 - 35.9
23.10
25.20
Premises/Fixed Assets
3.1 - 15.7
9.60
11.40
Other Noninterest Expense
15.8 - 35.9
22.70
19.80
Interest Expense
17.2 - 42.8
25.90
23.50
Purchased Funds
12.1 - 34.0
18.80
20.20
Outputs
Earning Assets
40.9 - 69.5
51.30
52.50
Interest Income
25.7 - 46.9
34.30
33.80
Noninterest Income
10.2 - 20.2
14.40
13.70
13
Banking Industry Test Data

• End of year data for
• 1991 11,397 banks
• 1994 10,224 banks
• 1997 8,628 banks
• Used constrained CCR-I model
• Run with large-scale specialized DEA software

14
1991 Profiles by DEA E-Quartile
1991 data
DEA Efficiency Quartile
most to
1
2
3
4
least efficient
most efficient
least efficient
difference
INPUTS
Salary Expense / Total Assets
1.43
1.54
1.65
1.83
-0.40

Premises and Fixed Assets / Total Assets
1.00
1.48
1.76
2.22
-1.22

-0.87

Other Noninterest Expense / Total Assets
1.53
1.62
1.84
2.41
0.08

Interest Expense / Total Assets
4.71
4.70
4.66
4.62
Purchased Funds / Total Assets
6.29
8.17
11.12
16.07
-9.78

OUTPUTS
Earning Assets / Total Assets
92.68
91.67
90.59
88.24
4.44

Interest Income / Total Assets
8.68
8.71
8.67
8.55
0.13

Noninterest Income / Total Assets
0.95
0.79
0.89
1.00
-0.05
N
2,850
2,848
2,849
2,850
0.2728

average efficiency score
0.7340
0.5982
0.5387
0.4611
lower boundary
0.6334
0.5665
0.5092
0.0000
upper boundary
1.0000
0.6334
0.5664
0.5091
Significant
at 0.01
(Values expressed as a percent of total bank
assets)
15
1997 Profiles by DEA E-Quartile
1997 data
DEA Efficiency Quartile
most to
1
2
3
4
least efficient
most efficient
least efficient
difference
INPUTS
Salary Expense / Total Assets
1.67
1.60
1.64
1.75
-0.08
Premises and Fixed Assets / Total Assets
0.98
1.55
1.94
2.44
-1.45

Other Noninterest Expense / Total Assets
1.85
1.31
1.50
1.92
-0.07
Interest Expense / Total Assets
3.29
3.30
3.27
3.15
0.14

-4.85

Purchased Funds / Total Assets
10.46
12.33
13.63
15.32
OUTPUTS
Earning Assets / Total Assets
92.99
92.60
91.83
90.65
2.33

Interest Income / Total Assets
7.45
7.41
7.37
7.33
0.13

Noninterest Income / Total Assets
1.80
0.77
0.84
0.90
0.90

N
2,157
2,157
2,157
2,157
average efficiency score
0.6685
0.4313
0.3717
0.3067
0.3617

lower boundary
0.4722
0.3982
0.3451
0.0000
upper boundary
1.0000
0.4721
0.3981
0.3450
16
Analysis of Results

• 1991 significant differences, Q1-Q4
• All inputs, and most outputs
• DEA scores
• Changed by 1997
• Inputs Salary, other non-interest (not sig.)
• Outputs non-interest income now signif.

• Noninterest income a new focus for banks
• Fee income
• Off-balance sheet activities

17
Other Bank Performance Metrics
18
Relationship with Other Metrics

• Efficient banks
• Greater return on assets
• Higher equity capital
• Fewer risky assets

• 1991 vs. 1997
• Not comparable scores
• But underlying trends of variables importance
  help explain banking industry changes

19
FRB Bank Examination Criteria

• Capital adequacy
• Asset quality
• Management quality
• Earnings
• Liquidity

20
Bank Examiner Ratings

• Confidential scores from on-site visits
• On each CAMEL factor and overall
• Values from 1 to 5

• 1 sound in every respect
• 2 sound, modest weaknesses
• 3 weaknesses that give cause for concern
• 4 serious weaknesses
• 5 critical weaknesses, failure probable

21
CAMEL Ratings DEA Scores

• Compared CAMEL ratings and DEA efficiency scores
• Included banks examined recently
• 1991 7,487 banks
• 1994 7,679 banks
• 1997 4,494 banks

• CAMEL rating groups
• Strong 1 or 2 rating
• Weak 3-5 rating
• DEA-score groups
• Quintile, by efficiency
• If no relationship, each group should contain 20
  of each of the other metrics groups

22
Efficiency vs. CAMEL Ratings
23
Strong vs. Weak CAMELs
24
In Summary

• DEA useful in benchmarking in service industry
• Can provide information for examiners, but not
  perfect predictor
• Large-scale efficiency analyses can give insight
  into industry dynamics and structure changes