Functional Data Analysis T-61.6030

1
Functional Data Analysis T-61.6030

• Chapters 10,11,12
• Markus Kuusisto

2
Topics

• 10 PCA of mixed data
• 11 Canonical Correlation Analysis
• 12 Functional linear models

3
PCA of mixed data

• Both functional part and vector part (xi ,yi)
• Canadian temperature Registeration process finds
  suitable shift.
• - Vector part is size of shift
• - Functional part is shifted curve

4
Canadian temperature
5
Canadian temperature (shifted)
6
Using PCA, vector part yi

• yi are nuisance parameters -gt we ignore
• yi are marginal importance -gt we ignore them when
  calculating PCA, but afterwards we investigate
  connections between PCA scores and yi
• yi are primary importance with functions xi -gt
  we treat them as a hybrid data (xi ,yi)

7
The PCA of hybrid data

• PCA weight function (?,v)
• PCA score of particular observation
• ?i xi (s) ?(s) ds yi v
• inner product zi (xi , yi )
• ?z1 , z2 ? x1 x2 y1 y2
• To find leading principal component maximize
  sample variance of the ?(?,v) , zi ? when
  (?,v) 1

8
Balance between functional and vector variation

• Measure units between functional and vector parts
  usually are not comparable
• ?z1 , z2 ? x1 x2 C2 y1 y2
• Choice of C2
• C2 T , where T is interval of function xi
• C2 T / M, where M is length of y
• C2 Var(x) / Var(y)

9
Incorporating smoothing

• Roughness of z (x , y )
• - D2z (D2x, 0)
• - D2z 2 D2x 2
• Calculating like in chapter 9

10
Canonical Correlation Analysis

• CCA is a way of measuring the linear relationship
  between two multidimensional variables
• Ordinary correlation analysis is dependent on the
  coordinate system in wich variables are described
• CCA finds the coordinate system where the
  correlation is maximized

11
Definition of CCA

• Consider the linear combination
• x xT wx y yT wy
• Function to be maximized is
• The maximum of ? with respect to wx and wy is
  maximum canonical correlation
• The number of solutions are limited to the
  smallest dimensionality of x and y

12
Car marks example
13
CCA of car marks

• Correlation r1 0.9792 r2 0.8851
• wx1 wx2
• Price -0.4935 0.6887
• Value 0.8697 0.7251
• wy1 wy2
• Economy -0.5471 0.4693
• Service 0.2418 0.4496
• Design 0.0060 -0.0097
• Sport 0.5800 -0.0790
• Safety 0.2817 -0.0117
• Easy h. 0.4758 0.7558

14
(xTwx2 , yTwy2 )
15
Predicting by CCA
16
Learning

• wx corresponds output (x)
• wy corresponds 52 previous datapoints (y)
• Learning
• - Finding maximum canonical correlation and its
  weights wx , wy
• - Linear line fitting
• Predicting output x is done by projecting y yT
  wy
• to fitted line.

17
Predicting recursively next 50 data points
18
Functional canonical correlation analysis

• Function to be maximized
• subject to constraints
• It is possible allways to find perfect
  correlation
• Maximization does not produce a meaningfult
  result

19
Unsmoothed canonical variate weight function that
attain perfect correlation.

• A standard condition for classical CCA
• n gt p q 1,
• - n is number of samples
• - p is length of xi and q is lenght of yi
• In functional case p and q are infinite, no
  unique solution
• Overfitting

20
Smoothing

• Smoothing is essential
• Choice of ? can be done
• subjectively
• by leave one out cross validation, maximazing
  squared correlation. (11.3.3)
• ccorsq? calculated as above but with the
  observation (Xi ,Yi) omited

21

• Smoothed canonical weight functions

22
Functional linear models

• Previous we have been exploring the variability
  of a functional variables
• Now we explore how much of variation is explained
  by other variables
• In calssical statistics we do that by linear
  regression and the general linear models.
• Now functional linear models

23
Precipitation example

• Preciptitation ( total rainfall) of particular
  area
• where i indexes the 35 weather stations
• Does the precipitation depend on temperature of
  that area
• Overfitting without smoothing

24
A Functional response and a functional
independent variable

• How does a precipitation profile depend on the
  associated temperature profile ?
• Concurrent Precipitation now depends only on the
  temperature now
• Annual Precipitation now depend on the
  temperature of the whole year

25

• Short-term feed-forward For reasons of
  parsimony, precipitation now depends on the
  temperature over an interval back in time.
• Local influence Precipitation now depends on the
  temperature over an interval back in time and the
  season (is it summer or winter ?)

26
Predicting derivatives

• Dynamic model Model is designed to explain a
  derivative of some order
• homogenous first order linear differential
  equation
• -nonhomogenous
• temperature in the equation is called forcing
  function

27
References

• Book Functional Data Analysis, J.O.Ramsay,
  B.W.Silverman
• http//www.functionaldata.org/
• matlab toolbox for FDA
• http//www.imt.liu.se/magnus/cca/
• Classical Canonical Correlation Analysis
• Method about solving blind source separation
  problem based on CCA
• Matlab functions cca.m and ccabss.m
• http//www.quantlet.com/mdstat/scripts/mva/htmlboo
  k/mvahtmlnode95.html
• Car marks example
• You may get confused because results presented
  here differs from the site above. Reason is that
  in that site the first and second canonical
  correlations are changed places.
• http//www.estsp2007.org/
• Data of example Predicting by CCA