1
Mathematical Foundations of Robustness and
Qualitative Robustness in Bootstrapping
Mohammed Nasser and Nor AishahHamzah Institute
of Mathematical Sciences University of Malaya
2
Contents

• Interplay between Statistics, Mathematics and
  Computer Science
• Mathematical Foundations of Robustness
• Qualitative Robustness in Estmation
• Qualitative Robustness in Bootstrapping

3
Interplay Among Statistics Mathematics and
Computer Science

• During the first three decades of the twentieth
  century three great revolutions swept across
  mathematics and science---
• the development of Quantum Mechanics
  (Heisenberg, 1930) after invention of quanta by
  Planck in 1900 and that of Relativity Theory (
  Einstein, 1905 and 1916) in physics,
• ii) axiomatization of Analysis and Algebra (,
  1906 Hausdorff, 1914 etc ) following the
  success of Set Theory of Cantor (1874 and 1892),
  and of Non-Euclidean Geometries, and
• iii) the beginning of Parametric Statistics (K.
  Pearson, 1900 Fisher, 1922 and 1925 etc).

4
Interplay Among Statistics Mathematics and
Computer Science
1.
They used Classical Analysis, Euclidean Geometry
and Algebra. Their use of mathematics is not
rigorous.They used both sample space X and
parametric space W as subset of Rk, k, finite.
Computation power was limited.
K.Pearson (1900), Student (1908), R.A. Fisher
(1922, 1925), J. Neyman and E.S. Pearson (1933)
etc founded modern parametric inference.
Cont.
5
Interplay Among Statistics Mathematics and
Computer Science
2.
Kolmogorov (1933), Cramer (1946), Wald (1947,
1950), Halmos and Savage(1949), von-Mises (1937
and 1947), Lehmann (1959) etc generalized the
previous results and made theoretical statistics
rigorous.
They not only used Modern Analysis, Measure
Theory, Point-set Topology and Functional
Analysis but also developed them. X becomes
topological spaces (mainly topological vector
spaces other than Rk or their some subsets)
Cont.
6
Interplay Among Statistics Mathematics and
Computer Science
3.
Both Non-parametric Statistics (Wilcoxon, 1945
Hoeffding, 1948 Hajek, 1969 etc) and Robust
Statistics (Huber ,1964 Hampel,1968) are
reactions to strict assumptions in Classical
Parametric Inference and its inability in
considering departures from the set of
assumptions.
Mathematical tools are both heuristic and
rigorous. Computer intensive-techniques as well
as rigorous functional analytic tools become
indispensable for statistics. W no longer remains
within domain of Rk.
Cont.
7
Interplay Among Statistics Mathematics and
Computer Science
4.
Though Rao (1945) and Jeffrey (1946) first
introduced Classical DG (Differential Geometry)
in Parametric Statistics to show relation between
Riemannian Metric and Fisher Information,
interest in Modern DG flared after the work of
Efron (1975) have been steadily rising since
eighties and nineties.
Theoretical statistics used all the important
concepts MDG. Also New geometrical concepts were
introduced (Amari,1985). MDG is yet to be
applied in Non-parametric and Robust Statistics.
Meaningful works have been undertaken with
semiparametric models (Bickel et al., 1993).
Cont.
8
Interplay Among Statistics Mathematics and
Computer Science

• Recently Computer plays an important role in
  statistics.
• Calculating fast classical statistical
  procedures and representing their summaries
• Simulating Sampling Distributions specially when
  it is not analytically calculable or, valid
  approximation is not possible.
• Bootstrapping (Weak simulation), Bagging
  (Bootstrap Aggregating), Boosting, MCMC and Data
  mining.

9
Mathematical Foundations of Robust Statistics
10
Mathematical Foundations of Robust Statistics
11
Mathematical Foundations of Robust Statistics
12
Mathematical Foundations of Robust Statistics
13
Mathematical Foundations of Robust Statistics
14
Mathematical Foundations of Robust Statistics
15
Mathematical Foundations of Robust Statistics
16
Mathematical Foundations of Robust Statistics
17

• Sample QRI, SQRI(?)
• 1/(1n-1?(?i-?))
• Its maximum value is 1
• Its minimum value is zero
• The more SQRI is the more qualitative robust the
  estimator is
• We can have plots to demonstrate sample QRI

18
Mathematical Foundations of Robust Statistics
19
Hampel in His Ph.D. Thesis (1968) Developed Three
Concepts

• Qualitative robustness ( also ?-robustness)
• Breakdown point
• Influence function
• To assess robustness in estimation and thus
  raised rigorousness in robust estimation to a
  satisfactory level

20
Hampel (1968) Developed

• Qualitative robustness to uphold qualitative
  side of robustness gauging distributional
  robustness.
• ?-robustness a form of qualitative robustness
  suitable for dependent observations.
• Breakdown point to quantify global side of
  robustness.
• Influence function to quantify infinitesimal
  side of robustness.

Cont.
21

• Hampel discussed and elaborated this concept at
  the outset of his thesis, breakdown point and
  influence function in the latter part.
• His seminal article on qualitative robustness
  (1971) was published before his mostly quoted
  article on influence function (1974).
• Breakdown point attracted wide range of
  researchers only after the development of finite
  version of breakdown point by Donoho (1982) and
  Donoho and Huber (1983).

Cont.
22

• Qualitative robustness has gained less
  popularity than other two concepts influence
  function and breakdown point.
• Why??
• It is no less important than the other two from
  the viewpoint of robustness.
• Most probably its mathematical complic- ations
  and absence of finite versions have acted
  behind this present unpopularity.

23
Development After Hampel
The concepts of qualitative robustness and ?
robustness (more restrictive concept than
qualitative robustness) were extended in
different directions in last eighties.

• Huber(1977,1981) modified Hampel's definition
  suggesting asymptotic equicontinuity of sampling
  distribution of the estimators with respect to
  n on the ground that nonrobustness gets worse for
  large n.
• Rieder (1982) and Lambert (1982) introduced
  qualitative robustness in hypothesis testing,

Cont.
24
Development After Hampel

• Boente et al. (1987) following Papatoni-Kazakos
  and Gray (1979) and Cox (1981) generalized
  qualitative robustness for stochastic processes
• Cuevas (1987 and 1988) adjusted some results of
  Hampel (1971) and Huber (1981) in the context of
  abstract inference.He showed incompatibility of
  consistency and qualitative robustness in the
  case of kernel density estimators

Cont.
25
After Eighties We Came Across Few Works in These
Area

• Cuevas and Romo (1993) and Nasser (2000) applied
  this concept in nonparametric bootstrapping and
  Basu et al. (1998) in Bayesian inference.
• Daouia and Ruiz-Gazen (2004) etc studied
  qualitative robustness of nonparametric frontier
  estimator.

26
Hampel's Definition
Cont.
27
SP(?)
Sp(X)
LF(Tn) LG(Tn)
hn
F G
d2
d1
lte
ltd
According to Hampel hn is equicontinuous at F
for all n. ? Tn is robust at F.
Huber argued that hnshould be asymptotically
equicontinuous at F.
Cont.
28
Meaningful Topology for Equicontinuity

• Both argued that Prokhorov metric meet the
  demand of robustness.
• It catches both gross error and rounding error,
  and also weak convergence.
• Huber argued that when X or ? is R or subset of
  R, any metric like Levy that generates weak
  topology could be used for our purpose.

29
Hampels Results
He deduced two main theorems, three lemmas and
two corollaries to show the relation between
concept of qualitative robustness and continuity
of Tn in two cases

• The general case TnTn(Fn) and
• 2) Particular case TnT(Fn)

Cont.
30
(No Transcript)
31
Comment. Let Tn T( Fn) be robust at F and
consistent at all G in a nbd of F (with
T8(G)T(G)). Then T is continuous at F.
32
Mathematical Foundations of Robust Statistics
33
Hampels Main Result Vs Hubers
Hubers main result (Proposition 6.2 in his
book) Assume that Tn (T(Fn)) is consistent in a
nbd of F. Then T is continuous at F? Tn is
robust at F.
Hampels FinalResult For TnT(Fn), T is
continuous at all F? Tn is robust and
consistent,tending to T(F) at all F
34
Consistency vs. Qualitative Robstness
35
Generalizations
36
All the results of Hampel and Huber can be
generalized to the case of generalized
statistics (statistics which take values in the
general complete separable metric spaces"). It
requires only two modifications 1) using the
metric d(x,y) of parametric space in place of
x-y and adjusting the definitions with the
metric. 2) applying Cantors Intersection Theorem
for general complete metric space in proving
lemma 2 in Hampel (1971)
37
Thank You