Examining Biases in Measures of Firm Innovation

1
Examining Biases in Measures of Firm Innovation
DRUID Summer Conference Copenhagen Business
School, 27th-29th June, 2005 Paul H. Jensen and
Beth Webster Melbourne Institute and IPRIA
2
Outline of the Presentation

• Defining innovation
• Conceptual issues in measuring innovation
• Hypotheses regarding biases in commonly-used
  proxies such as RD expenditure, patent
  applications.
• Research Question Are those firms that report
  (in surveys) that they are innovative the same as
  those that
• Have large intellectual property (IP) portfolios?
• Report high levels of RD expenditure?
• Conclusions

3
Defining Innovation

• Innovation is typically defined as the creation
  of new and improved products and processes
• What does new imply?
• New to the world (i.e. invention)?
• New to the firm (which may simply be imitation)?
• Even with a broad definition, innovation is
  difficult to measure

4
Measurement Problems

• There are 4 basic problems in measuring
  innovation
• Innovation is a dynamic process, not a point in
  time
• Reliant on static indicators to measure a dynamic
  process
• Counts of products, IP measure successful
  innovation
• Inputs and outputs are heterogeneous
• Quality of RD effort varies
• Most patents have zero economic value
• Prices of inputs and outputs change over time
• Hedonics cannot be used for true inventions
• GDP deflators under-estimate innovative firms
  output
• Unobservability of much innovative activity
• Most process innovations are not marketed
• Evidence suggests trade secrets are more
  important than patents

5
Hypotheses

• RD expenditure data biased towards
• Large firms and publicly-listed firms
• Patent application data biased towards
• Large firms, manufacturing firms and more
  profitable firms
• Trade marks, designs and patent applications are
  biased towards
• Product rather than process innovations
• Surveys of managers suffer fewer biases
• Accordingly, we use a survey measure of
  innovation as the benchmark for comparison here

6
Survey Data and Methodology

• Data on firm-level innovation collected from MI
  Business Surveys, 2001-03 (641 obs.)
• Sample IBISWorld top 1000 Australian firms
• Not a panel data set, but 266 obs. are repeat
• Innovation measure constructed using factor
  analysis
• Firm-level survey data then matched to firm-level
  RD data (IBISWorld) and IP data (IP Australia)
• Computed pair-wise correlations between the
  survey measure and 1) RD and 2) IP stocks

7
Empirical Results (1)
Notes , , significant at the 1, 5 and 10
per cent levels respectively.
8
Empirical Results (2)
Notes , , significant at the 1, 5 and 10
per cent levels respectively.
9
Analysis of the Results

• Main result relationships between survey measure
  of innovation and RD, IP is weaker than
  expected
• Correlation between RD and survey is 0.06
• Correlation between patents and survey is 0.08,
  but is much higher (0.17) for IP stocks
• Other results
• RD is a better measure for medium-sized and
  private firms
• Patents are a better measure for medium-sized
  firms
• Patents are better for manufacturing firms, but
  TMs are not
• Patents are not biased towards more profitable
  firms
• All IP proxies biased towards product innovation

10
Conclusions

• Innovation is a complex concept whose measurement
  is often trivialised
• Patents and RD expenditure are quite poor
  proxies for firm-level analysis of innovation
• However, as long as their weaknesses are
  understood, they are still be useful for
• Estimating industry-level trends in innovative
  activity over time
• International benchmarking (assuming that levels
  of unobservable innovation dont vary by country)