Quantile Regression

1
Quantile Regression
2
Quantile Regression

• The Problem
• The Estimator
• Computation
• Properties of the Regression
• Properties of the Estimator
• Hypothesis Testing
• Bibliography
• Software

3
Quantile Regression

• The Problem

4
Quantile Regression

• Problem
• The distribution of Y, the dependent variable,
  conditional on the covariate X, may have thick
  tails.
• The conditional distribution of Y may be
  asymmetric.
• The conditional distribution of Y may not be
  unimodal.
• Neither regression nor ANOVA will give us robust
  results. Outliers are problematic, the mean is
  pulled toward the skewed tail, multiple modes
  will not be revealed.

5
Quantile Regression

• Problem
• ANOVA and regression provide information only
  about the conditional mean.
• More knowledge about the distribution of the
  statistic may be important.
• The covariates may shift not only the location or
  scale of the distribution, they may affect the
  shape as well.

6
Quantile Regression
7
Quantile Regression
8
Quantile Regression
9
Quantile Regression

• The Estimator

10
Quantile Regression

• Ordinarily we specify a quadratic loss function.
  That is, L(u) u2
• Under quadratic loss we use the conditional mean,
  via regression or ANOVA, as our predictor of Y
  for a given Xx.

11
Quantile Regression

• Definition Given p ? 0, 1. A pth quantile of a
  random variable Z is any number ?p such that
  Pr(Zlt ? p ) p Pr(Z ? p ). The solution
  always exists, but need not be unique.Ex
  Suppose Z3, 4, 7, 9, 9, 11, 17, 21 and p0.5
  then Pr(Zlt9) 3/8 1/2 Pr(Z 9) 5/8

12
Quantile Regression

• Quantiles can be used to characterize a
  distribution
• Median
• Interquartile Range
• Interdecile Range
• Symmetry (?.75- ?.5)/(?.5- ?.25)
• Tail Weight (?.90- ?.10)/(?.75- ?.25)

13
Quantile Regression

• Suppose Z is a continuous r.v. with cdf F(.),
  then Pr(Zltz) Pr(Zz)F(z) for every z in the
  support and a pth quantile is any number ? p such
  that F(? p) p
• If F is continuous and strictly increasing then
  the inverse exists and ? p F-1(p)

14
Quantile Regression

• Definition The asymmetric absolute loss function
  is
• Where u is the prediction error we have made and
  I(u) is an indicator function of the sort

15
Quantile RegressionAbsolute Loss vs. Quadratic
Loss
16
Quantile Regression

• Proposition Under the asymmetric absolute loss
  function Lp a best predictor of Y given Xx is a
  pth conditional quantile. For example, if p.5
  then the best predictor is the median.

17
Quantile Regression

• Definition A parametric quantile regression
  model is correctly specified if, for
  example,That is, is a particular
  linear combination of the independent variable(s)
  such thatwhere F( ) is some univariate
  distribution.

18
Quantile Regression
.25
19
Quantile Regression

• Definition A quantile regression model is
  identifiable ifhas a unique solution.

20
Quantile Regression

• A family of conditional quantiles of Y given Xx.
• Let Ya ßx u with a ß 1

21
Quantile Regression
22
Quantile Regression
23
Quantile Regression
24
Quantile Regression

• Quantiles at .9, .75, .5, .25, and .10. Todays
  temperature fitted to a quartic on yesterdays
  temperature.

25
Quantile Regression

• Computation of the Estimate

26
Quantile RegressionEstimation

• The quantile regression coefficients are the
  solution to
• The k first order conditions are

27
Quantile Regression Estimation

• The fitted line will go through k data points.
• The of negative residuals np of neg
  residuals of zero residuals
• The computational algorithm is to set up the
  objective function as a linear programming
  problem
• The solution to (1) (2) previous slide need
  not be unique.

28
Quantile Regression

• Properties of the Regression

29
Quantile RegressionProperties of the regression

• Transformation equivarianceFor any monotone
  function, h(.),
• since P(Tlttx) P(h(T)lth(t)x). This is
  especially important where the response variable
  has been censored, I.e. top coded.

30
Properties

• The mean does not have transformation
  equivariance since Eh(Y) ? h(E(Y))

31
Quantile RegressionEquivariance

• Practical implications

32
Equivariance

• (i) and (ii) imply scale equivariance
• (iii) is a shift or regression equivariance
• (iv) is equivariance to reparameterization of
  design

33
Quantile RegressionProperties

• Robust to outliers. As long as the sign of the
  residual does not change, any Yi may be changed
  without shifting the conditional quantile line.
• The regression quantiles are correlated.

34
Quantile Regression

• Properties of the Estimator

35
Quantile RegressionProperties of the Estimator

• Asymptotic Distribution
• The covariance depends on the unknown f(.) and
  the value of the vector x at which the covariance
  is being evaluated.

36
Quantile RegressionProperties of the Estimator

• When the error is independent of x then the
  coefficient covariance reduces to where

37
Quantile RegressionProperties of the Estimator

• In general the quantile regression estimator is
  more efficient than OLS
• The efficient estimator requires knowledge of the
  true error distribution.

38
Quantile RegressionCoefficient Interpretation

• The marginal change in the Tth conditional
  quantile due to a marginal change in the jth
  element of x. There is no guarantee that the ith
  person will remain in the same quantile after her
  x is changed.

39
Quantile Regression

• Hypothesis Testing

40
Quantile RegressionHypothesis Testing

• Given asymptotic normality, one can construct
  asymptotic t-statistics for the coefficients
• The error term may be heteroscedastic. The test
  statistic is, in construction, similar to the
  Wald Test.
• A test for symmetry, also resembling a Wald Test,
  can be built relying on the invariance properties
  referred to above.

41
Heteroscedasticity

• Model yi ßoß1xiui , with iid errors.
• The quantiles are a vertical shift of one
  another.
• Model yi ßoß1xis(xi)ui , errors are now
  heteroscedastic.
• The quantiles now exhibit a location shift as
  well as a scale shift.
• Khmaladze-Koenker Test Statistic

42
Quantile RegressionBibliography

• Koenker and Hullock (2001), Quantile
  Regression, Journal of Economic Perspectives,
  Vol. 15, Pps. 143-156.
• Buchinsky (1998), Recent Advances in Quantile
  Regression Models, Journal of Human Resources,
  Vo. 33, Pps. 88-126.

43
Quantile Regression

• S Programs - Lib.stat.cmu.edu/s
• www.econ.uiuc.edu/roger
• http//Lib.stat.cmu.edu/R/CRAN
• TSP
• Limdep