P1253037239iGaoc

1
Quality and Inequality in Academic Labor Markets
by
James Moody The Ohio State University
2

• Quality Inequality in Academic Labor Markets
• Introduction Background
• Academic Caste Systems
• Suggestive Findings from a sociology market
• A reasonable null?
• Simulation Setup
• Market Elements
• An example run
• Simulation Results
• Market Clearing
• Size Quality
• Position Stability
• Academic Castes?
• Tentative Conclusions
• Some mechanisms
• Potential implications
• Future changes / extensions

3
Introduction Background Academic Caste Systems

• Merton (1942,1968)
• Two key features that shape the academic market
• Universalistic criteria to evaluate quality
• Mathew effect the cumulative advantage of
  prestige
• Burris (2004239) states as fact that prestige is
  ascribed rather than achieved, arguing that
• Moreover, through a process of cumulative
  advantage, academic scientists and scholars who
  secure employment in the more prestigious
  departments gain differential access to resources
  and rewards that enhance their prospects. This
  cycle results in a stratified system of
  departments and universities, ranked in terms of
  prestige, that is highly resistant to change.
  (p.239)
• Burris attributes much of this stability to
  Social Capital in the PhD hiring market.

4
Introduction Background Academic Caste Systems

• Two types of evidence are used to demonstrate
  non-universalistic effects
• A less-than-perfect association between measures
  of faculty productivity and department prestige
  (Long, Hargens, Jacobs, Baldi, Burris)
• Burris shows that between 30 and 50 of the
  variance in NRC rankings can be accounted for
  with standard productivity measures, leaving the
  remainder for non-meritocratic factors.
• A strong correlation between simple number of
  faculty and prestige (r 0.63 in sociology).
• Probability / prestige of first job due to origin
  of PhD rather than publication record (but see
  Cognard-Black, 2004 and below).

5
Introduction Background Academic Caste Systems
Two types of evidence are used to demonstrate
non-universalistic effects 2. An extreme
stability of department rankings over time
Burris, ASR 2004
The correlation in NRC faculty quality scores in
Sociology from 1982 to 1993 is 0.92
6
Introduction Background Academic Caste Systems
The combined effect becomes clear in the PhD
exchange network
Hanneman (2001), overlapping PhD exchange
networks, Sociology
7
Introduction Background Academic Caste Systems
The combined effect becomes clear in the PhD
exchange network
Hanneman (2001), overlapping PhD exchange
networks, Sociology
8
Introduction Background Academic Caste Systems
Han, S-K. Social Networks 2003251-280. Figure 1
9
Introduction Background Academic Caste Systems
The resulting status-based network has a strong
correlation between centrality in the hiring
network quality ranking
Social Capital Bonacich Centrality on
symmetric version of the PhD exchange Network
10
Introduction Background Academic Caste Systems
Can we square this stability centrality with
universalistic scientific norms?
First, research on markets and cultural
consumption suggests that quality is accurately
perceived particularly when external measures
show small differences (White 2002, J. Blau,
Bourdieu). Quality exists, whether
it's defined or not. - Robert Pirsig
(1972) That is, we know quality even if our
systematic measures of quality are poor, which is
reflected (in part) through market convergence on
particular candidates (see below).
11
Introduction Background Academic Caste Systems
Can we square this stability centrality with
universalistic scientific norms?
Second, most data on the market structure
systematically selects on the dependent variable,
as only those who are eventually hired are
observed. This has the effect of a) limiting
variation on observed quality measures b) makes
it impossible to disentangle PhD volume from
placement patterns Recent dissertation work by
Cognard-Black, for example, shows that the
independent effect of PhD institution on
placement is often lower than publication quality
measures, once you expand the sample of PhDs
beyond those hired to major research
universities.
12
Introduction Background Suggestive Findings
from Sociology
Further evidence a sample based on all
applicants for an open position

• Data from the OSU jr. recruitment committee last
  year
• Systematically code productivity (a function of
  (1) number of publications, (2) weighted by
  impact factor of the outlet, (3) type of
  publication (book gt article gt chapter gt review),
  and authorship order.
• Followed all applicants through the process to
  see where people take jobs.
• Data are limited to OSU applicants (but to an
  open position, and we have people from all ranks
  who take jobs at all ranks) and only have 1-side
  of the matching process (i.e. we dont know
  where people applied).

13
Introduction Background Suggestive Findings
from Sociology
Further evidence a sample based on all
applicants for an open position
14
Introduction Background Suggestive Findings
from Sociology
Further evidence a sample based on all
applicants for an open position
Regression line
15
Introduction Background Academic Caste Systems
Further evidence a sample based on all
applicants for an open position
Job Placement
Treat this distribution as a ranked outcome, and
model by productivity prestige
Hired at Non-PhD Granting Institution
No Job
Bottom (51) Dept
50 - 21 Dept
20 11 Dept
Top 10 Dept
Based on 116 new PhDs applying to the OSU open
search in 2004
16
Introduction Background Academic Caste Systems
Further evidence a sample based on all
applicants for an open position
Based on coefficients from an ordered logistic
regression model for job placement rank, using
116 new PhDs applying to the OSU open search in
2004 (model also controls for minority status
gender)
17
Introduction Background Identifying a
Reasonable Null
What should the PhD production system look
like? In systems with open markets, merit-based
hiring rational actors 1) How stable will
quality rankings be? 2) Will size and quality be
correlated? 3) Will network exchange centrality
predict quality? Each has been used as evidence
for non-meritocratic prestige systems, but we
dont know how the observed cases match the
expected cases, because we have no reasonable
null distribution. A key advantage of using a
simulation is to identify a range of reasonable
null distributions.
18
Introduction Background Structure from Action
A key question in sociology is where structure
comes from. A long line of simulation studies
have show how very simple individual rules can
generate complex global patterns Schelling on
racial segregation Axlerod on systems of
cooperation Epstein Axtell's Sugarscape for
inequality
In all of these cases, we can often generate a
macro-system with all of the relevant
characteristics (spatial segregation, high gini
coefficients) from very clear local behavior that
is indifferent to the global features.
19
Introduction Background Two Sided Matching
Markets

• A long-line of work (building on Roth), focuses
  on the incentive structure and performance of
  markets where two sets of actors rank each other.
• Non-academic examples include
• Artists and galleries
• Medical interns and hospitals
• Rushees and Greek houses
• Law graduates and Federal Clerkships
• These markets are distinguished from commodity
  markets in that goods are not easily
  substitutable, there is usually a constrained
  time-frame for action in the market, and each
  side of the market plays an active role in
  completing the market transaction.
• The market is characterized by two dirty sorts
  ? where each side ranks the other to make an
  exchange.

20
Introduction Background Two Sided Matching
Markets

• Two-sided matching markets are famous for their
  dramatic failures
• Market unraveling where timing is pushed ever
  earlier to jump the competition (rushing
  high-school students, appointing 1st year
  students to clerkships, etc.)
• Exchanges that do not please all/most actors
• Holding places / offers to trade up
• Opportunistic contract arrangements
• Many of these failures have two things in common
• They rob actors of information necessary to make
  good choices
• They result in placements that do not maximize
  preferences

21
Introduction Background Two Sided Matching
Markets
The Sociology market, for example, is clearly
inching toward unraveling Typical application
dates are moving up, and variance is becoming
smaller.
Nov 1
Oct 22
Oct 23
Oct 15
Oct 7
Dec. 31
Jul 19
Aug 3
Aug 18
Sep 18
Sep 17
Oct 2
Oct 17
Nov 1
Nov 16
Dec 1
Dec 16
22
Introduction Background Two Sided Matching
Markets
While economists have focused on identifying the
conditions necessary to solve such
inefficiencies, they have paid much less interest
to how choice-relevant factors in these markets
affect organizational structures and
performance. By systematically varying the
market features that shape the dirty sorts
driving such markets, we can generate null
hypotheses for questions about market prestige
stability, exchange hierarchy and overall
quality.
23
Simulation Setup Purpose Questions

• The purpose of this simulation is to examine the
  effect of market-relevant behavior under
  ideal-typical conditions. This involves
  simplifying the real world as much as possible,
  to isolate how particular factors affect outcomes
  of interest.
• Key real-world properties of interest
• Stable prestige / quality rankings
• Strong correlation between size and quality
• Centralized hiring networks
• Strong correlation between centrality, prestige,
  size
• Currently all actors in the simulation follow the
  same strategies, which vary across simulation
  runs. A future goal is to vary department
  strategies within runs to see what features lead
  to competitive advantage.

24
Simulation Setup Market elements

• Actors
• Departments Collections of faculty who hire
  applicants produce new students. (N100).
• Initial department size is drawn from a normal
  distribution with mean 25, std12, but I
  re-draw if size is less than 10, so the actual
  distribution is a truncated normal.
• Applicants Students from (other) departments who
  apply for jobs.
• Departments seek to hire the best students,
  students want to work at the best departments.
• Both actors are rational, honest, and
  risk-averse. But all actors have individual
  preferences / errors in vision.

25
Simulation Setup Market elements

• Attributes
• Quality. Each faculty member and student has an
  overall quality score.
• Initial faculty quality is distributed as random
  normal(0,1).
• Student quality is a (specifiable) random
  function of faculty quality.
• Departments are rated based on the mean of
  faculty quality.
• While each person has a set real quality score,
  actor choices are made based on an evaluation of
  quality that varies across actors.
• This variation reflects jointly differences in
  preferences and ability to discern quality from
  production.

26
Simulation Setup Market elements

• Action Departments
• Departments hire produce students.
• For each of 100 years
• Every department produces students (conditional
  on size).
• A (random) subset of departments have job
  openings based on retirements in the prior year
  current size relative to their resource-based
  target size.
• Departments rank applicants by their evaluation
  of applicant quality, and make offers to their
  top choices.
• If a departments 1st choice goes elsewhere, they
  go to next for a specifiable number of rounds to
  a specifiable depth into the pool.
• Jobs can go unfilled, which means that
  departments can both grow and shrink

27
Simulation Setup Market elements
Action Departments The probability a job opening
in any given year is a function of size
retirements (1-year replacement)
28
Simulation Setup Market elements
Action Departments Faculty size decreases
through retirement
29
Simulation Setup Market elements

• Action Students / Applicants
• Students rank departments that make them an offer
  by their evaluation of department quality, and
  take the best job they are offered.
• If an applicant does not receive a job offer in a
  given year, they move out of the system
• Lots of applicants dont get jobs (at PhD
  granting universities)
• Applicants are not strategic they do not hold a
  good offer while waiting for a better one (though
  this could be added)

30
Simulation Setup Variable Market Parameters
Parameter Description Specification
Hiring probability Likelihood of a job opening beyond retirement replacement. Cubic function of department size. 3 levels
Student production Probability that each faculty member putts a student on the market in a given year. Binomial (0,1), p (0.06 to or 0.08). 2 levels. X1 165 X2 220
Faulty - Student Quality Correlation The correlation between student and faculty quality. Specify as a correlation from 0.37 to 0.91 3 levels
Applicant Quality Evaluation Used by departments to rank applicants. Each department assigns applicants an observed quality score based on this function. Observed (Student quality) b(N(0,1)). b 0.3 to 0.9. 3 levels
Department Quality Evaluation Used by applicants to rank job offers. Each student assigns departments an observed quality score based on this function. Observed (Department quality) b(N(0,1)). B 0.1 to 0.25. 2 levels
Hiring Rounds Number of offer rounds made. Approximates time by limiting opportunity to make alternative offers. Specify as number. 3 or 4 2 levels
Depth of Search
How deeply into the pool of candidates
departments are willing to go.
Specify as max depth. 10 to 30 3 levels
There are 3233223 648 parameter sets 30
draws from each set ? 19,440 observations
31
Simulation Setup A single simulation run

• Initial Conditions
• 100 departments
• Size distributed normally with mean of 25 std of
  12 and an initial floor of 10. This is the
  resource-based target size for departments.
• Faculty quality is distributed normally (N(0,1))
• Age is initially distributed uniformly from 0 to
  40 (starting with a distribution means that
  retirements dont go in waves)
• Parameter Settings
• Hiring curve Medium
• Student Production 0.06 (150 applicants per
  year)
• Student-Faculty Quality Correlation 0.67
• Disagreement on applicant quality 0.60
• Disagreement on department quality 0.1
• Hiring Rounds 4
• Depth of Search 20

32
Simulation Setup A single simulation run
Market Size

• Over the first 10 years
• 66 to 104 positions advertised
• 147 to 169 students on the market
• 59 to 72 people were hired each year

33
Simulation Setup A single simulation run
Student-Faculty Quality Correlation
Student Quality
r0.65
r0.49
Faculty Quality
Department Quality
34
Simulation Setup A single simulation run
Distribution of size over time
35
Simulation Setup A single simulation run
Correlation between final size and target size
Quality gt Mean 1std
Quality lt Mean 1std
Target Equality
Final Size
Target Size
36
Simulation Setup A single simulation run
Distribution of quality over time
37
Simulation Setup A single simulation run
Correlation of Size and Quality over time
Burris reports the correlation between size and
prestige as 0.63
38
Simulation Setup A single simulation run
Correlation of Quality 10 years prior
39
Results All results are presented around the
competitive field
Disagreement on Candidate Quality
0.3
0.6
0.9
10
20
Depth of Search
30
40
Results Market Clearing proportion of jobs that
are filled
Calculated for year y100
41
Results Size Quality Department Size
Calculated for year y100
42
Results Size Quality Average Department Quality
Calculated at final year ( y100)
43
Results Size Quality Correlation of Size and
Quality
Calculated at final year ( y100)
44
Results Size Quality Correlation of Size and
Quality
Calculated at final year ( y100)
45
Results Quality Stability 10 Year Correlation of
Quality
Calculated at final year ( y100)
46
Results Department Features Summary
Calculated at final year ( y100)
47
Results Academic Castes?
The production and hiring of PhDs generates an
exchange network, connecting the sending
department to the hiring department. I record
this network for all hires in the last 10 years
of the simulation history, and construct two
measures a) The network centralization
score b) The correlation between network
centrality quality size. 10 years is
conservative ? all of the centralization effects
I describe below are stronger if you limit the
network to the last 5 years (which is closer to
what people have done in the literature).
48
Results Academic Castes Process Expectations
A basic feedback process seems to be operating,
and that should lead to a highly centralized and
stable system.
49
Results Academic Castes?
For what follows, working within one region of
the parameter space
Disagreement on Candidate Quality
Depth of Search
A preliminary regression over the entire space
shows that hiring rates quality correlation
matter most for centralization
50
Results Academic Castes Network Centralization
by Quality Correlation Job Openings
Bonacich Centrality
51
Results Academic Castes Network Centralization
by Quality Correlation Job Openings
Degree Centralization
52
Results Academic Castes Correlation of
Centrality Department Size
Bonacich Centrality
53
Results Academic Castes Correlation of
Centrality Department Size
Degree Centrality
54
Results Academic Castes Correlation of
Centrality Department Quality
Bonacich Centrality
55
Results Academic Castes Correlation of
Centrality Department Quality
Out-Degree Centrality
56
Results Academic Castes Parameter Centrality
Summary

57
Tentative Conclusions Observations Summary
Market Effects

• The very simple market model proposed here can
  account for many of the features we see in real
  PhD exchange markets
• Stable quality rankings
• Strong Correlation between Size Quality
• Highly Centralized Networks
• Correlation between Quality ranking and
  Centralization
• Though to be fair, this correlation is not as
  strong as reported empirically.
• Qualitatively, it is appears that you can order
  most of these networks with a pretty clear
  distinction between top or core departments
  and a periphery, characterized by asymmetric flow
  of students.

58
Tentative Conclusions Summary Some Potential
Mechanisms

• There are two broad features that shape these
  networks.
• Market competition
• Market competition factors (mainly agreement on
  quality depth of search, but also simple
  student production hiring rates) have a huge
  effect on the mean levels of department
  characteristics seen across the simulation
  settings.
• When the competition for students is high, offers
  converge on small numbers of market stars. This
  generates a sellers market, where a small
  number of market stars dominate hiring patters,
  take jobs at the most prestigious institutions,
  leaving many departments with failed searches,
  and ultimately lowering the quality for the
  discipline as a whole.
• This mechanism can account for much of the
  observed stability, growth and quality outcomes
  observed over the simulation runs

59
Tentative Conclusions Summary Some Potential
Mechanisms

• There are two broad features that shape these
  networks.
• The development of a hierarchical network
  exchange structure depends on a correlation
  between faculty and students, though the effect
  appears not to be linear.
• For the most part, a quality correlation
  reinforces quality rankings due to the main
  reinforcement mechanism sketched below

60
Tentative Conclusions Summary Some Potential
Mechanisms

• There are two broad features that shape these
  networks.
• The development of a hierarchical network
  exchange structure depends on a correlation
  between faculty and students, though the effect
  appears not to be linear.
• But when the correlation is too high, the
  inequality in student production starts to
  dominate. This has the result of
• (a) flooding the market with relatively
  low-quality students, that
• (b) has the effect of mirroring tight-market
  competition factors.
• Since the hiring practices in this simulation
  were tied to quality ranks instead of cardinal
  values (or values relative to self), this means
  departments are forced by retirements to dig too
  deep in the pool, resulting in a lowering of
  overall relative quality, which then gets
  translated into lower centralization in the
  networks.

61
Tentative Conclusions Observations Summary
Market Effects

• There is still some room for non-market effects
  here, however, since the resulting hierarchies
  are not perfect
• Self-selection Effects
• Students avoiding applying out of their league
• Adjusting depth of search to be linked to current
  quality
• Social Network Effects
• Burris Social Capital effect Give a positive
  weight to students who come from departments
  where current faculty originated
• Reputation effects
• Add a positive intercept shift in the perception
  of students who come from highly ranked
  departments

62
(No Transcript)