Reliability Presentation Test-Retest

1
ReliabilityPresentationTest-Retest

• James Blackwood AED 615
• Fall Semester 2006

2
Test-Retest Reliability

• Test-Retest method of determining reliability is
  accomplished by administering a test to a group
• After a period of time has passed, the same test
  is re-administered to the same group.
• It is also known as stability reliability
• It is used in both qualitative and quantitative
  research (qualitative requires a different
  technique of analysis)

3
Reliability Coefficient

1. After two tests have been administered, a
   reliability coefficient is calculated to
   determine the relationship between the two scores
   obtained.
2. If the same results are obtained from the two
   tests then the coefficient 1.0
3. The coefficient is influenced by the amount of
   time that has passed between the administration
   of the two tests.

4
Reliability Coefficient Measurement

• The reliability coefficient is expected to be
  lower the longer the time interval between the
  tests due to the possibility of changes in the
  population taking the test.
• The shorter the time gap, the higher the
  correlation the longer the time gap, the lower
  the correlation.

5
Test-Retest Issues

• There cannot be any measurable change in the
  construct being measured between the two tests.
• This method will not work when measuring a
  variable that is not stable in an individual.
• Unless the instrument is reliable, relationships
  with other variables in the study will not be
  identified.

6
Testing..

• You can obtain considerably different estimates
  of reliability depending on the interval between
  tests.
• For educational research, examination of scores
  over a two to three month period is sufficient
  for test-retest reliability verification.
• The time interval between the two tests should
  always be reported when using test-retest as a
  measure of reliability.

7
Test-Retest Equation
8
Test-Retest Issues

• Requires twice the data collection
• Population for the test would need to be willing
  to repeat the test (impractical)
• Higher costs due to multiple tests being
  administered
• Only works well when practical (better for
  smaller population rather than a large one)
• Researcher may not be able to retest the
  population

9
Examples of Test-Retest

• Educational assessment
• Drug testing
• Testing Measurement Equipment
• Medical Evaluations

10
Research Literature Example

• American Journal of Agricultural Economics
• Volume 71 - Number 1 (Feb 1989), pp 76-84
• Test-Retest Reliability of the Contingent
  Valuation Method A Comparison of General
  Population and Visitor Responses
• John B. Loomis
• Abstract
• The reliability of the contingent valuation
  method is evaluated by resurveying the same
  general households and visitors nine months after
  their original survey. Test-retest correlations
  on willingness to pay are statistically
  significant and ranged from .422 for the general
  population sample to .782 for the visitor sample.
  Using a paired T-test, there was no statistical
  difference between an individual's first and
  second reported willingness to pay. Chow tests
  comparing the original and resurvey
  willingness-to-pay functions showed no
  statistical difference at the .01 level. Reported
  willingness to pay is reasonably stable over the
  time period surveyed.

11
References

• Fraenkel, J. R., Wallen, N. E. (2006). How to
  design and evaluate research in
• education (6th ed.). New York Mc-Graw-Hill.
• Guttman, L. (1946). The test-retest reliability
  of qualitative data. Psychometrika, 11(2),
• 81-95. Abstract retrieved October 15, 2006 from
  Springer Link database.
• Loomis, J. B. (1989) Test-retest reliability of
  the contingent valuation method A
• comparison of general population and visitor
  responses. American Journal of Agricultural
• Economics, 71(1), 78-84.
• Trochim, W. M., (2006). Types of reliability.
  Research Methods Knowledge Base. Retrieved from