Developing and Sustaining Electronic Portfolios for Program and Institutional Assessment

1
Developing and Sustaining Electronic Portfolios
for Program and Institutional Assessment

• A Bakers Dozen Lessons Learned

Victor Borden and Timothy Thomas Indiana
University Purdue University Indianapolis AAHE
Assessment Conference Denver, CO June 26, 2001
2
Objectives

• Describe the types of resources required to
  create and sustain electronic portfolios
• Analytical
• Technical
• Organizational
• Design

3
Context

• Functional Needs Assessment conducted as part
  of the Urban University Portfolio Project
• Six urban, public universities developing
  web-based portfolios for accountability and
  improvement
• 3-year project funded by Pew Charitable Trusts
  and co-sponsored by AAHE

4
Relevancy to Other Types of Portfolios

• Some aspects directly relevant and some not
• Especially relevant when considering how to
  summarize results from student/course
  portfolios as measures of institutional
  effectiveness

5
Defining the Need

• Projects staffed to promote campus leadership
  commitment and ties to assessment and
  institutional research
• Technical components started as secondary but
  quickly move to forefront
• Technical needs assessment recast as functional
  needs assessment to integrate form and function

6
Method

• RFP developed and distributed to technology,
  assessment, and organizational development
  consultants
• Awarded to EduTech, Incorporated
• Site visits include UUPP Tech. Dev. Assoc.
• Products individual campus reports and
  project-wide report

7
Method (cont.)

• Visit protocol evolves through RFP and bid
  specifications
• Common and campus-specific questions
• Development goes on as visits proceed
• Findings derive as much from further project
  development as from needs assessment analysis

8
Lesson 1

• Commonality emerges more from experience than
  from planning
• Difficult to impose commonality as pre-condition
• Best practices identified and adapted
• Project meeting shares and critical friend
  comments and suggestions facilitate emerging
  commonalities
• BUT There must be some initial vision and
  framework

9
Lesson 2

• Many different uses and approaches to electronic
  portfolio development
• Identifying primary uses and strategies early is
  critical
• BUT it is important to accommodate significant
  changes in context (e.g., new leadership, sudden
  budget crises), which is highly possible, if not
  inevitable.

10
Lesson 3

• High level leadership commitment is critical
• Especially given novelty of concept and
  requirements for significant cooperation from a
  broad array of campus constituents
• BUT there are some advantages to stealth
  development, i. e., periods of low visibility and
  low expectations

11
Lesson 4

• EP development proceeds best if it fits well
  within existing structures, processes, and
  functions
• Difficult to develop if there is not already a
  significant culture of evidence
• BUT since it is also developmental, need to
  accommodate new thinking and possible new
  organizational functions and relationships

12
Lesson 5

• Broad-based participation helps promote
  visibility and usefulness
• Helps to created needed fit between
  organizational processes and EP functions
• BUT broad participation can create burdensome
  expectations and scope creep

13
Lesson 6

• There is a particular period in the development
  of assessment capacity when EP projects are most
  helpful
• Requires enough capacity to develop fit and
  sustain development
• Requires enough room for development to benefit
  from what EP has to offer, primarily a clear
  focus on the audience perspective

14
Lesson 7

• It doesnt take much hardware to develop
  relatively few, rich portfolios
• BUT it does take a lot to support relatively
  many portfolios, even if each is small

15
Lesson 8

• EP projects benefit greatly from linkages to
  institutions information systems
• BUT This comes at a great cost (i.e., be
  careful near the event horizon of the IT black
  hole)

16
Lesson 9

• There is no wrong answer in choosing
  authoring/development tools
• Guidelines
• Locally supported
• Official or unofficial standards
• Strong support community
• Somewhat interchangeable
• BUT There is no right answer

17
Lesson 10

• Stay safely away from the event horizon of the IT
  black hole
• It will cost much more, take more time, and then
  be immediately obsolete
• EP projects are not likely to stay on the IT
  radar screen for too long given operational
  issues
• BUT seek advice from appropriate folks in your
  IT shop

18
Lesson 11

• Draw clear lines between EP projects and other
  somewhat related electronic initiatives (e.g.,
  campus web site datawarehouse info systems
  migration)
• BUT Stay in touch and keep an eye toward a
  convergent future

19
Lesson 12

• EP projects cannot be approached as marginal,
  add-on work
• They require significant resources, especially
  including the time and attention of those who may
  already be stretched rather thinly
• BUT

20
Lesson 13

• Electronic portfolio development is like a gas
  it will occupy any volume it is provided

21
www.imir.iupui.edu/portfolio