Class III Asking Clinical Questions, Sampling, and Reliability and Validity

1
Class IIIAsking Clinical Questions, Sampling,
and Reliability and Validity

• OT 667

2
Components of Clinical Questions

• Foreground and background questions
• Background questions refer to questions about the
  diagnosis itself
• Foreground questions are questions about
  treatment

3
Question Components as Defined by Law

• Client/population - the specific group or
  diagnosis being treated or under study
• Intervention or exposure - the kind of
  intervention or strategy you are considering
  using in treatment
• Outcome - the results of treatment or
  intervention
• Comparison - the analysis of outcomes between two
  or more treatments

4
Judging Assessments
5
Reliability - estimate of the extent to which a
test score is free from error
6
Statistical Tests Used to Calculate Reliability

• Correlation - a measure of the degree of
  association or relationship between two variables
  or sets of data.
• The research question when using correlation is
  What is the relationship between.?

7
Correlation Coefficients

• Are scored in a range of 1 to -1
• Correlations can be found significant and still
  be low (under .50), so look for degree of
  correlation rather than significance
• Several forms of correlations for various levels
  of data (Spearmans for ordinal data, Pearsons
  for interval or ratio data, Phi for nominal data,
  point biserial correlation between one interval
  data set and one ordinal data set)

8
Cronbachs Alpha

• Procedure that is used for dichotomous data or
  are answered in multiple choice format
• Calculated as correlations, -1 to 1

9
Intraclass Correlation Coefficients

• Preferred over other measures of reliability
  because
• it can analyze more than 2 data sets
  simultaneously
• desirable since many researchers include more
  than one measure of reliability in studies

10
About ICCs

• Results in a score in the 1 to -1 range
• Reflects both correlation and degree of agreement
• Can be used to calculate ratings between 2 or
  more ratings
• Does not require the same number of ratings for
  each subject
• Can be used with ordinal, interval and ratio data
• 6 different versions for use with various kinds
  of comparisons

11
Reliability Procedures

• rater agreement
• interclass correlation coefficients
• test-retest
• correlation coefficients, inter-class
  correlations
• alternate forms
• correlation coefficients
• internal consistency
• Cronbachs coefficient alpha, item to item
  correlation (how scores on individual items
  relate to scores on the instrument as a whole)

12
Validity - concerns the degree to which a tool
assesses what it is designed to measure
13
Kinds of Validity

• Face Validity
• Content Validity
• Criterion-related Validity
• Concurrent Validity
• Predictive Validity
• Prescriptive Validity
• Construct Validity

14
Prescriptive Validity - the situation where
predictor variables are used to choose an
appropriate intervention
15
Sensitivity and Specificity

• Sensitivity - the ability of a test to generate
  a positive result when the target condition is
  really present
• Specificity - the ability of a test to generate
  a negative result when the target condition is
  really absent

16
Sensitivity and Specificity

• DX DX- Total
• Positive 15 13 28
• _______________________
• Negative 1 5 6
• _______________________
  
• 16 18 34

17
Sensitivity and Specificity

• a 15
• Sensitivity _________ ______
  94
• a c 16
  
• d 5
• Specificity ________ ______
  28
• b d
  18

18
SAMPLING
19
Target Population
Accessible Population
Sample
20
Study Criteria

• Inclusion Criteria
• primary characteristics of target and primary
  populations used to qualify someone for the study
• Exclusion Criteria
• characteristics or factors that will preclude the
  inclusion of a subject into the sample

21
BOTOX Study

• Inclusion criteria - Ss with stroke who are a
  year or more post-insult and who have intractable
  spasticity and are between the ages of 18-75
  years
• Exclusion criteria - Ss whose medications for
  spasticity management have been altered in the
  last 3 months who receive a score of 13 or more
  on the Beck Depression Scale Ss who are unable
  to follow 3 step commands Ss who are unable to
  get to SRC X2 weekly for outpatient therapy

22
Probability Sampling

• simple random sampling - a sample drawn randomly
  from the accessible population
• systematic sampling - the sample is divided by
  the number of elements to be selected subjects
  are then selected using the resulting number as
  the interval between Ss selection
• stratified random sampling - sample is divided
  into homogeneous groups based on specified traits
• disproportional sampling - proportions of the
  sample are pre-weighted to match the proportion
  of the population
• cluster sampling - random sampling of a series
  of units in the population

23
Nonprobability Sample

• Convenience Sampling - subjects chosen on the
  basis of availability
• Quota Sampling - stratification of a convenience
  sample
• Purposive Sampling - subjects are chosen for
  specific traits
• Snowball Sampling - asking subjects with
  specified traits to identify others with those
  traits

24
RECRUITMENT
25
Other Aspects of Test Assessment

• Cost
• Time required to administer and to score
• Viability with mission of clinical setting
• Time required to learn to give the test
• Compatibility with outcomes measured in a
  specific clinic