Studying the Impact of Online Professional Development on Educators

1
Studying the Impact of Online Professional
Development on Educators Technology Integration
Skills

• Tom Clark, PhDTA Consulting

Elizabeth Oyer, PhDEvaluation Solutions
In 63.030 Questions, Methods and Findings from
Program Evaluations and Research Examining
Digital Online Learning Projects. Le Centre
Sheraton Montreal, Salon 4, 405-535 pm, April
14, 2005
2
Study Context

• STAR Project TechKnowledgy Initiative
• PURPOSE Increase tech integration preparedness
  of K-12 teachers via online PD
• Created in 1997, became centerpiece of 5-year
  STAR Project in 2000
• Participants follow a reflective PD model using a
  Star-Online VTLC www.star-online.org/tk

3
Study Context

• Leaders In Technology Enhanced Schools (LITES)
  Kit Project
• PURPOSE Increase tech integration in inquiry
  units of K-12 teachers
• Used packaged inquiry curriculum integrated with
  technology in a portable format
• Common evaluation question
• Does project participation increase the quality
  and level of teachers technology integration?
• Methodological issue
• Is there a valid and reliable way to measure
  changes in quality and level of tech integration?

4
Methods

• Participants
• 30 teachers (60 online and video cases total)
• 27 teachers for face-to-face ratings
• Data Sources
• Video of STAR units implemented in classroom
• Online artifacts for STAR portfolios
• FTF observation of LITES teachers

5
Methods

• Instrument
• MTimms Technology Observation Instrument
  originally developed for FTF observation
• Addresses style of teaching, types of tech use,
  tech integration
• Design
• Initial study subscales combined for composite
  score correlations provide reliability estimates
• Follow-up study subscales interpreted in
  original scale inter-rater consensus computed

6
Results

• Correlations between raters in initial study
• rVideo(36).735, plt.01
• rOnline(36).684, plt.01

7
Results

• Consensus Across Methods

8
Results

• Consensus for Video Ratings

9
Results

• Consensus for Online Ratings

10
Discussion

• Reliability of tool not yet fully acceptable
• Positive results for face-to-face ratings
  highlight potential
• Post-analysis
• Individual rater bias
• Shorter rating period

11
Implications

• If reliability and validity established, several
  research questions of interest. For STAR
• What is the correspondence between teacher plans
  for tech integration (through portfolios/artifacts
  ) and actual integration?
• How do tech proficiency and level of experience
  influence this relationship?

12
Thank You!

• Tom Clark, PhD
• TA Consulting
• taconsulting_at_yahoo.com
• [phone]

Elizabeth Oyer, PhD Evaluation
Solutions eoyer_at_evalsolutions.com [phone]