More Research Basics

1
More Research Basics

• Descriptive Research
• Sampling
• Measurement

2
Descriptive Research

• Naturalistic Observation
• Observation and recording of naturally occurring
  behavior
• Often must limit the behaviors of interest
• Interobserver reliability
• Degree to which multiple observers agree about
  the behavior they observed

3
Descriptive Research

• Case Studies
• Intensive observation of one person or a small
  group of people for an extended period of time

4
Descriptive Research

• Meta-Analysis
• Study of existing research
• Method of statistically combining the results of
  many studies on the same topic

5
Descriptive Research

• Survey Research
• Want to know about attitudes, beliefs, behaviors,
  thoughts? Just ask
• Must have adequate sample to answer the questions
  youre asking
• Representative of the population you want to
  generalize to

6
Sampling

• Generalization
• The degree to which the conclusions in your study
  would hold for other persons in other places and
  at other times

7
Sampling
8
Sampling

• Random Sampling Techniques
• Simple random sampling
• Stratified random sampling
• Systematic random sampling
• Cluster sampling
• Multistage sampling

9
Sampling

• Nonrandom Sampling Techniques
• Accidental Sampling
• Convenience Sampling
• Expert Sampling
• Quota Sampling
• Heterogeneity Sampling
• Snowball Sampling

10
Measurement

• The process of observing and recording the
  observations that are collected as part of a
  research effort

11
Measurement

• Construct Validity The approximate truth of the
  conclusion that your operationalization
  accurately reflects its constructs.
• Central questions to ask are
• Is your operationalization an accurate
  translation of the construct?
• Does your program/treatment accurately reflect
  what you intended?
• Does your sample accurately represent your idea
  of the population of interest?
• Are you measuring what you intended to measure?

12
The Idea of Construct Validity
13
Measurement Validity Types

• Construct Validity
• Translation Validity
• Face Validity
• Content Validity
• Criterion-related Validity
• Predictive Validity
• Concurrent Validity
• Convergent Validity
• Discriminant Validity

14
Translation Validity

• Focuses on whether the operationalization (i.e.,
  measure) is a good translation of the construct
• Face Validity On its face, does the
  operationalization look like a good translation
  of the construct?
• Weakest way to demonstrate construct validity
• Improved with expert judgment

15
Translation Validity

• Focuses on whether the operationalization (i.e.,
  measure) is a good translation of the construct
• Content Validity Operationalization is checked
  against the relevant content domain for the
  construct
• May be difficult to determine all of the
  characteristics that constitute the domain
• A systematic check of the operationalization
  against the content domain increases rigor

16
Criterion-related Validity

• The performance of your operationalization (i.e.,
  measure) is checked against some criterion
• A prediction of how the operationalization will
  perform on some other measure based on your
  theory or construct
• Validating a measure based on its relationship to
  another independent measure

17
Criterion-related Validity

• Predictive Validity Operationalizations
  ability to predict something it should
  theoretically be able to predict
• Demonstrated by a high correlation between your
  measure and the criterion measure
• Is SAT score able to predict success in college
  (i.e., college GPA)?

18
Criterion-related Validity

• Concurrent Validity Operationalizations ability
  to distinguish between groups it should
  theoretically be able to distinguish between
• Results are more powerful if your measure is able
  to distinguish between two similar groups
• Can a measure of empowerment distinguish between
  supervisors and employees?

19
Criterion-related Validity

• Convergent Validity Degree to which the
  operationalization is similar to (converges on)
  other operationalizations to which it
  theoretically should be similar
• Demonstrated by high correlation of your measure
  with another measure of the same construct
• Does your measure of depression correlate highly
  with another measure of depression?

20
Criterion-related Validity

• Discriminant Validity Degree to which the
  operationalization is not similar to (diverges
  from) other operationalizations to which it
  theoretically should not be similar
• Demonstrated by low correlation of your measure
  with a measure of a theoretically opposite (or
  dissimilar) construct
• Is there a low correlation between your measure
  of depression and a measure of well-being?

21
Convergent and Discriminant Validity Correlations
22
Threats to Construct Validity

• Inadequate preoperational explication of
  constructs
• Mono-operation bias
• Mono-method bias
• Interaction of different treatments
• Interaction of testing and treatment
• Restricted generalizability across constructs
• Confounding constructs and levels of constructs
• Social threats to construct validity

23
Levels of Measurement

• The relationship among the values that are
  assigned to the attributes for a variable
• Determines what statistical analyses are
  appropriate
• Levels
• Nominal
• Ordinal
• Interval
• Ratio

24
Levels of Measurement

• Nominal Numerical values simply name the
  attribute
• No ordering of values is implied
• Numerical values are simply short codes for
  longer names
• Color of your car
• 1 red
• 2 yellow
• 3 black

25
Levels of Measurement

• Ordinal Attributes can be rank ordered
• Distances/intervals between attributes have no
  meaning
• Degree of satisfaction
• 1 dissatisfied
• 2 neutral
• 3 satisfied

26
Levels of Measurement

• Interval Distances/intervals between attributes
  do have meaning
• Temperature on a Fahrenheit scale
• Ratios do not make sense
• 80 degrees is not twice as hot as 40 degrees
• Not meaningful, absolute zero
• Zero degrees (F) is not the absence of temperature

27
Levels of Measurement

• Ratio Always a meaningful, absolute zero
• Allows construction of meaningful ratios
• Weight
• Zero pounds is the absence of weight
• 80 pounds is twice as much weight as 40 pounds

28
Levels of Measurement