CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

1
CHAPTER 5

• CONCEPTUALIZATION, OPERATIONALIZATION, AND
  MEASUREMENT

2
Conceptualization - the process of coming to an
agreement about how to use a social scientific
term (e.g., alienation, poverty, social class,
romantic love, democracy).Conceptualization
results in definitions of concepts mental
images of what we mean by a term.
3
Can concepts be defined objectively?

• If not, how can we conduct research using
  concepts?
• Intersubjectivity agreements among subjective
  beings (humans) about the definitions of terms
  used in research.

4
Types of definitions.  Which would be useful to a
researcher?

• Real definition
• Nominal definition
• Operational definition

5

• Real definition a statement of the essential
  nature or essential attributes of a term
• Nominal definition a definition that is
  assigned to a term for purposes of communication
  or inquiry a working definition

6

• Operational definition a specification of how a
  defined term will be measured for purposes of
  research development of specific research
  procedures that will result in empirical
  observations representing the concept in the real
  world

7
Linking conceptualization and data collection -
C-M Diagram

• Conceptualization

Nominal Definition
Operational Definition
Measurements in the Real World
8
Conceptualization practice

• The concept might be defined in an ordinary
  dictionary and might also have been used
  previously in social scientific theory and/or
  research.
• Web of Online Dictionaries
• Sociology Dictionary

9

• Consider possible indicators of the concept -
  What would indicate the presence of the concept
  in empirical reality, and what would indicate its
  absence?
• What indicators could you use to determine
  whether people are experiencing the emotion
  called "romantic love."

10

• Consider possible dimensions of the concept -
  different facets or aspects of the concept. 
  Concepts can be unidimensional or
  multidimensional.

11
They make it look easy, but its not!

• Definitions of concepts in the social scientific
  literature
• Romantic love (according to Rubin)
• Workplace alienation (according to Blauner)

12
Operationalization choices

• Level of measurement
• Range of variation of attributes (nature of
  extremes)
• Precision of variations between extremes
• Use of Single or Multiple Indicators

13
Level of measurement

14
GSS variables and LOM

• MARITAL
• AGE
• EDUC
• DEGREE
• ZODIAC
• RELIG
• TRAUMA1

• CONLEGIS
• MILQUAL
• PRES96
• WRKSTAT
• INCOME98
• SIBORDER

15
Implications of levels of measurement for data
analysis

• Restrictions on quantitative data analysis
  techniques
• Higher level may be converted to lower level
  (recoded), but not vice versa

16
Range of variation of attributes (nature of
extremes)

• To what extent are we measuring the extremes of a
  phenomenon?
• GSS examples - POLVIEWS, CONLEGIS, COMMUN

17
Precision of variations between extremes

• How precisely defined are the attributes of the
  variable between the extremes?
• Examples from 1996 General Social Survey 1996
  GSS data

18
Use of Single or Multiple Indicators

• GSS - Attitudes toward abortion
• One item ABLEGAL
• Multiple items - ABNOMORE, ABDEFECT, ABRAPE,
  ABPOOR, ABHLTH, ABANY, ABSINGLE

19
Index construction

• Combining responses to two or more items into one
  ordinal item (index)
• GSS Constructing an index (ABINDX) combining the
  responses to ABNOMORE, ABDEFECT, ABRAPE, ABPOOR,
  ABHLTH, ABANY, and ABSINGLE
• Please tell me whether or not you think it
  should be possible for a pregnant woman to obtain
  a legal abortion if (1Yes, 2No)

20
(1) ABNOMORE (2) ABDEFECT (3) ABRAPE (4)
ABPOOR (5) ABHLTH (6) ABANY (7) ABSINGLE


21
Criteria for measurement quality

• 1. Precision
• 2. Accuracy
• 3. Reliability
• 4. Validity

22
1. Precision

• Definition - the fineness of distinctions made
  between attributes of a variable
• Which is more precise?
• EDUC or DEGREE?
• RINCOM98 or Gross income reported to the IRS last
  year?
• Necessity for precision depends on purpose of the
  research

23
2. Accuracy

• Definition - the extent to which the measure is a
  reflection of the real world degree of
  conformity of a measure to a standard or a true
  value.
• Survey example - Which one is likely to be more
  accurate?
• What is your age?
• In what year were you born?

24
3. Reliability

• Definition - whether a measurement technique,
  applied repeatedly to the same phenomenon would
  yield the same result
• Types of reliability
• a. Test-retest
• b. Split half
• c. Use of established measures
• d. Reliability of research workers

25
3. Reliability - a. Test-retest

• Definition - testing for reliability using two
  tests, separated by a time interval, in a
  situation in which the researcher does not expect
  the variable of interest to change in value.
• Examples knowledge of research methods,
  attitudes toward abortion (ABDEFECT), level of
  formal education (EDUC)

26
3. Reliability - b. Split half

• Definition - testing for reliability of a
  multiple-item measure by splitting it randomly
  into two halves and seeing if the two halves
  correspond in value.
• Examples 150-item test of knowledge of research
  methods, 50-item romance attitudes scale

27
3. Reliability - c. Use of established measures

• Many measures of social scientific concepts have
  already been tested and shown to have adequate
  reliability by previous researchers.
• Examples
• Rubin's definition and operationalization of
  romantic love
• Sroles definition and operationalization of
  anomia

28
3. Reliability - d. Reliability of research
workers

• Research workers may make mistakes which produce
  unreliability (e.g., interviewers reading
  questions incorrectly coders entering
  information incorrectly into data files).
• Careful training and supervision of research
  workers is essential to maximizing reliability.
• If the research workers are the data collectors,
  this is also called Inter-Rater or Inter-Observer
  Reliability

29
4. Validity

• Definition - whether a measurement technique
  adequately reflects the nominal definition of the
  concept

30
Validity on the C-M Diagram

• Conceptualization

Nominal Definition
Does the measurement technique reflect the
nominal definition?
Operational Definition
Measurements in the Real World
31
Approaches to Validity

• 1. Face Validity
• 2. Content Validity
• 3. Criterion-related Validity
• 4. Construct Validity

32
1. Face Validity - Definition

• Definition - does the measurement technique fit
  common agreements and mental images concerning a
  particular concept does it seem reasonable?

33
1. Face Validity - Example

• Nominal Definition Religiosity - extent to which
  a respondent adheres to a set of cultural ideas,
  symbols, and practices that focus on the meaning
  of life and the nature of the unknown.
• Measurement technique - ask respondents what
  their shoe size is

34
2. Content Validity

• Definition - the degree to which a measure
  covers the range of meanings included within a
  concept
• Examples
• Does an exam assess all aspects of course
  content?
• Do questions asking respondents opinion of
  abortion in case of rape and in case the mothers
  health is endangered assess all aspects of
  abortion issues?

35
3. Criterion-related Validity

• Definition - the degree to which a measure
  relates with some external criterion measuring
  the same concept
• Example
• Does SAT score predict performance in college?
  Some research data

36
4. Construct Validity

• Definition - the degree to which a measure is
  related to other variables within a system of
  theoretical relationships as the researcher
  hypothesizes

37
4. Construct Validity - Example

• Hypotheses
• Women should support abortion more frequently
  than men do.
• Catholics should oppose abortion more frequently
  than those of other or no religious affiliation.
• Research results If hypotheses are supported by
  research, construct validity of measurement
  technique is supported. If not, poor
  measurement, poor hypotheses, and/or poor
  research.

38
Does this figure make sense (Figure 5-2, p. 140)?
39
A Caution!

• Validity has to do with assessing whether the
  measures of our concepts reflect the nominal
  definition we have assigned, NOT the "real"
  definition!