Data Collection Strategies An Overview

1
Data Collection StrategiesAn Overview
2
How Do We Assess ?
3
Data Collection Strategies

• BEFORE collecting data, your first question
  should be . . . .

Has someone already collected the information
from my sample of interest?
4
Data Collection Strategies
Direct Contact
Indirect Contact

• Paper/pencil survey
• Web survey
• Database/Warehouse
• Existing Reports
• Naturalistic observation

• Personal interview
• Focus group
• Telephone poll
• Telephone interview
• Classroom assessment

5
Assessment Instruments
When selecting assessment tools you have a
choice

• Standardized
• Build Your Own

6
Existing Standardized Instruments
Pros
Cons

• Cost
• Processing Delay
• Item pool
• Inappropriate norms

• Proven item pool
• Reliability / Validity
• Clarity of directions
• Scoring
• Benchmark comparisons
• Data analysis
• Reporting formats

7
Build Your Own Instruments
Pros
Cons

• Relevant item pool
• Stakeholder involvement
• Theory/Model based
• Lower Cost
• Processing control
• Data analysis control
• Report format

• Long development time
• Local technical expertise
• Unknown reliability/validity
• Length
• Poorly worded items

8
Evaluating Existing Instruments

• Does the title convey the content to student?
• Review content of EVERY item by team.
• Are items relevant to ALL students?

9
Evaluating Existing Instruments

• Does manual report reliability ?
• Does manual report validity evidence?
• How appropriate is normative/benchmark data?

10
Evaluating Existing Instruments

• Are results timely?
• Is the cost worth the benefit?

11
Sources for Assessment Instruments

• Instrument Sources