ePortfolio Scaffolding Study

1
ePortfolio Scaffolding Study

• Sandra A. Lathem, Joyce L. Morris, Jing Qi
• The University of Vermont
• Kirk Vandersall
• Arroyo Research Services
• SITE 2007
• San Antonio, Texas
• March 27, 2007

2
Electronic Portfolio Connection

• PT3 grant awarded 2003
• University of Vermont
• Lyndon, Johnson Castleton State
• Champlain College
• IBM Reinventing Education
• Apple Computer
• Vermont Dept of Education
• Arroyo Research Services
• www.uvm.edu/pt3

3
UVM Scaffolding Study

• Conducted spring semester 2006
• Examines three programs
• Elementary education
• Middle level education
• Physical education
• Relationship between instructional scaffolding
  and
• portfolio development
• technology proficiency
• artifact and reflection quality

4
Research Question 1

• What is the evidence that instructional
  scaffolding in portfolio development and
  technology skills occurs at the micro (faculty
  member/student) and macro (department /program)
  level?

5
Research Question 2

• What is the relationship between the
  instructional scaffolding that pre-service
  teachers receive and the quality of artifacts and
  reflections found in the licensure portfolio?

6
Research Question 3

• What are UVM student attitudes about the
  licensure portfolio process in their programs?

7
Research Question 4

• Will the quality in artifact development and
  reflection differ between paper and electronic
  portfolios?

8
Licensure in Vermont

• Governed by Vermont Standards Board for
  Professional Educators (VSBPE)
• VSBPE oversees accreditation process through ROPA
  (Results Oriented Program Approval)

9
Five Standards for Teachers A Vision for
Schooling

• Learning
• (Expertise in Endorsement Area)
• Professional Knowledge
• (Methodology and Pedagogy)
• Colleagueship
• Advocacy
• Accountability

10
ROPA Scoring Rubric

• Common statewide assessment for all teacher
  education programs
• Six entries
• Teaching Episodes (two episodes)
• Understanding Student Learning
• Accommodating Students Identified
• Teaching Over Time
• Colleagueship and Advocacy
• Self-reflection and Vision

11
Scaffolding

• Intentional planning of curricula to organize
  tasks and activities to build understanding
• Two levels --
• Micro (teacher/student)
• Macro (program or organization)
• Consistent with contemporary theory (Bransford)
  on How People Learn
• Prior knowledge
• Metacognition

12
PT3 Professional Development

• Portfolio support to faculty and students
• Creation of html templates structured to meet
  ROPA standards

13
Methodology

• Exploratory
• Mixed Methods
• Student Surveys (n100)
• Faculty Interviews (n10)
• Faculty Student Observations (n3)
• Student Focus Groups (n3 students 38)
• Course Syllabi (n34)
• Licensure Portfolio Review (n35)

14
Findings-Question 1What is the evidence that
instructional scaffolding in portfolio
development and technology skills occurs at the
micro (faculty member/student) and macro
(department/program) levels?

• Micro Level
• Many experiences in artifact development
• Assignments geared toward licensure portfolio
  artifacts
• Reliance on student teaching for artifact
  development
• Technology integration less evident
• Macro Level
• Middle level program most intentional in
  portfolio construction for undergraduates
• Courses identified for portfolio artifact
  development
• Three undergraduate portfolios developed
• Elementary education program intentional artifact
  development in specific courses
• Artifact development driven by the ROPA scoring
  rubric

15
Findings- Question 2What is the relationship
between the instructional scaffolding that
pre-service teachers receive and the quality and
reflections found in the licensure portfolio?

• 35 licensure portfolios scored by Arroyo Research
  Services
• Elementary program
• 12 paper
• 15 electronic (using template)
• Middle level
• 8 electronic (using template)
• Physical education
• Not scored - unable to collect
• Small sample size is a limitation to findings.

16
Question 2 - Continued

• Arroyo Research Scoring Rubric (not ROPA)
• Artifact Selection
• Reflection
• Collaboration
• Assessment
• Technology
• Scale
• 1 no evidence
• 2 limited or negative evidence
• 3 some evidence (but needs development)
• 4 ample evidence

17
Findings 2 - continued

• Weak scores in reflection, collaboration,
  assessment.
• Statistically significant differences found
  between elementary middle level programs in
  several areas. Further research needed.
• Evidence of technology-supported teaching
  strategies was very low (mean 1.3143, std. .6761)

18
Findings - Question 3What are UVM student
attitudes about the licensure portfolio process
in their programs? Do they feel adequately
prepared by their course experiences?

• Student surveys
• 100 students (79 female, 19 male, 2 missing)
• Overall, a 50-50 split in agreement about
  purpose of portfolio process -- concerning
  introduction to portfolios process, its value as
  a tool to demonstrate student learning,
  usefulness of collaboration, reflection,
  feedback.
• Focus Group Sessions
• Portfolios extremely stressful exercise
• Like taking a test
• Lack of technology skills very stressful for some
  students, especially middle level graduate
  students

19
Middle Level Focus Group
20
Findings - Question 4Will the quality in
artifact development and reflection differ
between paper and electronic portfolios?

• Licensure portfolios (35 scored)
• 27 elementary education
• 12 paper
• 15 electronic using html template
• 8 middle level
• 8 electronic using html template
• Results
• Paper portfolios receiving scores of 3 or 4
  predominate in every scored criterion, even
  evidence concerning technology supported teaching
  strategies.
• Statistically significant difference found.
• Small sample size is a major limitation for these
  findings

21
Conclusion - 1

• Scaffolding at both macro micro levels is
  occurring.
• Middle level - most developmental approach at
  undergraduate level but post baccalaureate
  students dont benefit from this approach.
• ROPA scoring rubric drives artifact development
• Students are ambivalent about the process many
  view it as a test rather than a portrayal of
  professional growth. Portfolio construction is a
  stressful process.

22
Conclusion - 2

• Reflection is weak - lack of time to develop
  reflections, given heavy reliance on student
  teaching experience.
• Technology proficiencies needed to produce
  electronic portfolios using html editors may be
  hindering quality integrity of portfolios.
• Quality of paper portfolios reviewed were
  overall higher than those produced with a
  template and html editor.
• Continued research is needed.

23
www.uvm.edu/pt3

• This paper, student survey, ROPA rubric,
  portfolio scoring rubric, and The Five Standards
  documents are available on our web site
• Click Research
• Click Papers and Presentations

24
Questions?

• For additional information, please contact the
  authors -
• Joyce Morris, Sandy Lathem, Jing Qi
• University of Vermont
• 802-656-4140
• http//www.uvm.edu/pt3
• Email jmorris_at_uvm.edu, slathem_at_uvm.edu,
  jqi_at_uvm.edu
• Kirk Vandersall, Arroyo Research Services
• Email kirk_at_arroyoresearchservices.com
• http//www.arroyoresearchservices.com
• 213-291-1556