Paradise Valley Community College

1
Paradise Valley Community College

• 2004 Summer
• Assessment Institute

2
Our Goals

• Select one or two PVCC Gen. Ed. L.O.s as an
  Institute focus.
• Produce plans and products to implement in Fall
  2004.
• Plan to report results at a January 2005 Learning
  Week session.

3
Overall Institute Agenda

• M Plenary Session Introduction/Project teams
• T Project teams work
• W Plenary Progress Reports/Project teams
• Th Project teams work
• F Project Teams work/ Plenary Session Planning

4
Mondays Agenda

• Plenary Session, 900-1200 P.M.
• Orientation/Introductions
• Review of College Assessment Initiative
• Course Competencies/Course Outcomes/Gen Ed L.O.s
• Answer the Q Which Gen Ed L.O.s do our
  projects most clearly support?
• How can we accomplish both our project goals and
  the promotion of a college level Gen Ed L.O.?
• Assessment Methods
• Project teams, 100-300 P.M.
• Begin working on plans

5
Institute Participants
6
What characterizes a Learning Centered College?

•  Learning outcomes have been identified and made
  explicit.
• Learning outcomes serve as the centerpiece for
  program and curriculum development.
• Learning outcomes are measured for the purpose of
  intervention, remediation and continuous
  improvement.

7
What is Assessment?

• Assessment is an ongoing process aimed at
  understanding and improving student learning. It
  involves making our expectations explicit and
  public setting appropriate criteria and high
  standards for learning quality systematically
  gathering, analyzing, and interpreting evidence
  to determine how well performance matches those
  expectations and standards and using the
  resulting information to document, explain, and
  improve performance.
• --Cal State Long Beach
• http//www.csulb.edu/7Esenate/assessment/assessme
  nt_glossary.htmlprocess

8
Evaluation vs. Assessment

• Purpose primarily Formative.

• Purpose primarily Summative.

• Focus on group performance.

• Focus on individual performance.

• Helps instructors/ dept/ school know how they and
  students performed.

• Helps individuals know how they performed.

• Results in a plan for improvement.

• Results in a grade for the activity or course.

• Always used for improvement of instruction/
  learning.

• May or may not be used for improvement of
  instruction/ learning.

9
Students completing the MCCCD General Education
Curriculum should be able to

• Access, evaluate, analyze, synthesize, and use
  information wisely.
• Communicate effectively personally, socially, and
  professionally.
• Think critically, make informed decisions, solve
  problems, and implement decisions.
• Use technological resources appropriately, and
  productively.
• http//www.dist.maricopa.edu/academic/curric/cphb/
  gs/gsinfo.html

10
Where are we with Assessment?

• Review of 5-year plan
• Review of course-mapping and its relationship to
  PVCC Gen L.O.s
• Whos been doing what, related to what L.O.s
• Where we need to go (Institutional level
  assessment)

11
PVCC Gen. Ed. Learning Outcomes
http//www.pvc.maricopa.edu/AI/outcomes.html
(Core Outcome Critical Thinking)

• Communication
• Listening
• Reading
• Speaking
• Writing

• Information Literacy
• Problem Solving
• Technology

12
MCCCD The student will be able to Communicate
effectively personally, socially, and
professionally.
PVCC The student will be able to listen with
critical comprehension.

• receive the speakers purpose and organization of
  ideas and information
• discriminate between statements of fact and
  statements of opinion
• distinguish between emotional and logical
  statements
• detect bias and prejudice

(Whos measuring these and where should they be
taught/measured?)
13
Whos teaching listening?

• 208 Gen Ed courses analyzed
• 110 have listening skill as a potentially
  significant part
• 21 actually list some type of listening
  competency.

14
Whos teaching listening?
15
How are learning outcomes different from course
competencies?

• Learning outcomes are statements that specify
  what learners will know or be able to do as a
  result of a learning activity. Outcomes are
  usually expressed as knowledge, skills, or
  attitudes.
• Phillips, Louis. The Continuing Education Guide
  the CEU and Other Professional Development
  Criteria. /Hunt Publishing Co., 1994
• http//www.allnet.org/prodev/outcomes

16
How are Maricopas learning outcomes different
from its course competencies ?

• Competencies are statements which describe
  skills or knowledge students are expected to
  possess as a result of completing the course
  successfully.
• --MCCCD Curriculum Procedures Handbook

• MCCCD competency for ENG 061 Basic Writing
  Skills 3
• Generate writing that demonstrates one main
  focus with specific details by following the
  steps in the writing process.

17
MCCCD competencies suggest general outcomes but
are not specific measures

• ECE103  Engineering Problem Solving and Design
• Describe the role of engineers and the role of
  the engineering profession in today's society.
• Demonstrate critical, sympathetic, and creative
  listening skills.

18
MCCCD competencies suggest general outcomes but
are not specific measures

• CIS102   Interpersonal and Customer Service
  Skills for IT Professionals
• Identify and describe the key components of a
  customer focused organization.
• Develop listening and oral communication skills.

19
Learning Outcomes get specific

• Learning outcomes have three distinguishing
  characteristics the specified action by the
  learner must be
• observable,
• measurable, and
• done by the learner.

20
How do you write a measurable learning outcome
statement?

• Use an action verb to specify what the student
  will have to do to demonstrate the competency.
• State the conditions under which s/he will
  demonstrate competency.
• State the criteria for competency.

• Two websites that can offer help
• http//www.stedwards.edu/cte/resources/learningout
  .htm
• http//www.catl.uwa.edu.au/online/outcomes.html

21
From General to SpecificThe student will be
able to

• General PVCC Critical Thinking Outcome
  evaluate information, materials, and numerical
  and/or graphical data by drawing inferences
• Math Department L.O. . ..analyze information
  and solve problems in complex, real-world
  situations.
• Related Course Competency for MAT 102
  ...summarize, interpret, and use graphs to
  display statistical data.
• Specific measurable activity describe a
  trend indicated in a chart or graph and make
  predictions based on that trend.

22
From General to SpecificThe student will be
able to

• General PVCC Writing Outcome The student
  will be able to communicate in writing his / her
  ideas in an organized manner using an appropriate
  rhetorical strategy.
• Related Course Competency for ENG 061
  Generate writing that demonstrates one main
  focus with specific details by following the
  steps in the writing process.
• Specific measurable activity When
  provided with a general topic, the student will
  be able to write an introductory paragraph with a
  clearly focused specific topic sentence and three
  supporting detail sentences.

23
The outcome assessment
A lesson-specific PTA that relates back to the
measurable course outcome and back to the PVCC
Gen Ed outcome.

• 2 Meets standard
• 1 Needs improvement
• 0 Does not meet minimum standard
• _____ Thesis Main Idea is clearly presented.
• _____ Supporting Details Main idea is fully
  developed with appropriate support (facts,
  examples, analysis, etc).
• _____Focus Paper contains a single focus/major
  point of discussion and does not wander or give
  unnecessary or extraneous details.
• _____Coherence Logical progression of ideas
  using transitions and key words.
• _____Mechanics Correct use of spelling, grammar,
  punctuation, capitalization, word choice, and
  sentence structure.
• _____ Style Awareness of audience, appropriate
  language, originality of ideas.

24
Strategic Question

• Though all of the PVCC Gen Ed L.O.s are
  important, which one, or two, strategically,
  should we focus on?

25
Strategic Questions

• What seems critical to us as an institution?
• Is there a way to prioritize our Gen Ed L.O.s?
• Which L.O.s most obviously cut across the most
  courses?
• Which FTSE leaders have which L.O.s at their
  core?
• Have we neglected a particular L.O.?
• Which two or three, in which order, should we
  pursue?

26
One Strategic Answer--Strategic Agreement

• Our PVCC Gen Ed Learning Outcomes reflect what is
  truly important for us to accomplish with our
  students.
• We all share responsibility in helping our
  students accomplish those outcomes.
• We share responsibility for assessing those
  outcomes in meaningful ways, both at the course
  and institutional levels.

27
Project Team Questions

• How can our project/plan advance an overall
  institutional initiative?
• How can what we do with our project be
  transferred to other courses/disciplines?

28
How can we advance the goal of Institutional
Assessment?

• Establish measurable L.O.s that generalize across
  disciplines.
• Develop assessment instruments that can be used
  with little or no adaptation in several
  disciplines.
• Track individual students across those diverse
  disciplines and measure at key points/times.

29
Writing Across the CurriculumIndividual and
Institutional

• Assess/survey writing practices across campus.
• Analyze writing assignments.
• Develop generic PTAs and streamlined style
  guides for disciplines.
• Pilot in several disciplines.
• Expand the conversation.

30
The PTA a prime assessment tool.

• Versatile can be created in virtually any
  discipline to measure performance.
• Easy to develop and understand.
• Useful to both teacher and student
• Makes L.O. explicit to student shows where he
  was strong/weak basis for assignment grade.
• Can be used for grading individuals as well as
  assessing whole class.

31
The PTA a prime assessment tool.

• Can be applied at all levels institutional,
  division, department, course, class, specific
  assignment.
• Can reduce between-instructor variance across
  multiple instructors and course sections.
• PTA Example from PVCC Math
• http//www2.pvc.maricopa.edu/ms/math/rubric.html

32
More PTA Examples

• A really good explanation of PTAs and their
  value
• http//www.assessment.gatech.edu/eReports/slide20
  shows/Using20the20Grading20Process20for20Asse
  ssment.pdf
• Scientific paper http//www.siue.edu/deder/assess
  /cats/rubex.html
• Dental Hygiene
• http//www.rwc.uc.edu/phillips/ExamplesForms/ptaex
  amples/dentalhygiene.pdf
• Article Four PTAs tried and explained Math,
  Literature, German, Chemistry
• http//www.rwc.uc.edu/phillips/Faculty/jge.pdf

33
Assessment Methodologies

• Primary Trait Analysis Embedded Questions
• Student Portfolios
  Electronic Portfolios
• Institutional Portfolio
  Standardized/National Exams
• Pre/Post Tests Locally
  Developed One-Shot Items
• Capstone Experiences/Courses
  Classroom Based Assessment
• Surveys
  Other Indirect Methods
• These ideas are explained in detail at
• http//www.pvc.maricopa.edu/AI/methods.html

34
The Math Model of Embedded Questions (see
handout)

• A common L.O. is selected to measure across
  multiple sections of a course.
• A PTA is constructed to measure students
  performance on one or more embedded exam
  questions.
• Instructors are free to teach the PTA as they
  wish, and to create their exams as they wish,
  with the exception that they agree to embed
  common questions in their exams to measure
  performance on that L.O.
• Instructors compile data and analyze it together
  to determine if different skills/content need to
  be approached differently.

35
An excellent resource

• Angelo, T.A. Cross, P.K. Classroom Assessment
  Techniques A Handbook for College Teachers (3rd
  Ed.). Jossey-Bass 2003
• (We have a copy in E OL)

36
Another resource

• Stiehl, R. and L. Lewchuck. The Outcomes Primer
  Reconstructing the College Curriculum (2nd Ed.).
  Corvallis, OR The Learning Organization, 2002.
• (We have a copy)