Undergraduate Psychology Major Learning Goals and Outcomes

1
Undergraduate Psychology Major Learning Goals and
Outcomes

• Jerry Rudmann
• Coastline College
• California Assessment Institute
• September 30, 2002

2
What?

• In April 2001, the APA appointed a taskforce to
  identify what a graduate (BA/BS) in psychology
  should know and be able to do.

3
Why Develop Learning Outcomes for Psychology?

• In response to increasing pressures for
  accountability
• To clarify psychologys identity
• Students and prospective employers will benefit
  from clear expectations for graduates
• To demonstrate educational leadership
• Facilitate development of articulation agreements
  between 2-year and 4-year colleges and
  universities
• Aid in the evaluation of online programs
• To help identify best practices in higher
  education
• To facilitate review of psychology programs
  (program review)

4
Why is this of interest to the Student Learning
Outcomes assessment community?

• The Taskforce includes some nationally recognized
  assessment experts
• The Taskforce is now developing an assessment
  component for the learning outcomes

5
Who is on the APA Task Force?

• Jane Halonen (Chair) James Madison University
• Drew Appleby Indiana University Purdue
• Barney Beins
• APA Staff Liaison
• Martha Braswell
• APA Staff Liaison
• Charles Brewer Furman University
• William Buskist Auburn University

• Diane Halpern,
• Claremont McKenna College
• Margaret A. Lloyd Georgia Southern University
• G. William Hill Kennesaw State University
• Jerry Rudmann
• Coastline Community College
• Valjean M. Whitlow
• Belmont University

6
The Ten Student Learning Outcomes (Psychology)
Each of the 10 major SLOs incorporate a subset of
several more specific SLOs for the psychology
major. The complete set of SLOs can be seen by
obtaining a copy of the Psychology Learning
Outcomes Document at www.apa.org/ed/pcue/taskforc
ereport2.pdf
7
SLO OneTheory and Content of Psychology
Category 1 of the 10 SLO Areas in Psychology
Knowledge, Skills and Values Consistent with the
Science and Application of Psychology

• Students will demonstrate familiarity with the
  major concepts, theoretical perspectives,
  empirical findings, and historical trends in
  psychology.

8
SLO TwoResearch Methods in Psychology

• Students will understand and apply basic research
  methods in psychology, including research design,
  data analysis, and interpretation.

9
SLO ThreeCritical Thinking Skills in Psychology

• Students respect and use critical thinking,
  skeptical inquiry, and, when possible, the
  scientific approach to solve problems related to
  behavior and mental processes.

10
SLO FourApplication of Psychology

• Students will understand and apply psychological
  principles to personal, social, and
  organizational issues.

11
SLO FiveValues in Psychology

• Students will be able to weigh evidence, tolerate
  ambiguity, act ethically, and reflect other
  values that are the underpinnings of psychology
  as a discipline.

12
SLO SixInformation and Technological Literacy
Category 2 of the 10 Outcome Areas Knowledge,
Skills and Values Consistent with a Liberal Arts
Education that are Further Developed in Psychology

• Students should demonstrate information
  competence and the ability to use computers and
  other technology for many purposes.

13
SLO SevenCommunication Skills

• Students will be able to communicate effectively
  in a variety of formats.

14
SLO EightSociocultural and International
Awareness

• Students will recognize, understand, and respect
  the complexity of socio-cultural and
  international diversity.

15
SLO NinePersonal Development

• Students will develop insight into their own and
  others behavior and mental processes and apply
  effective strategies for self-management and
  self-improvement.

16
SLO TenCareer Planning and Development

• Students will emerge from the major with
  realistic ideas about how to implement their
  psychological knowledge, skills, and values in
  occupational pursuits in a variety of settings.

17
Psychology Taskforces Guidelines for Getting
Started in Assessment

• To begin, decide on a limited number of goals
  (SLOs) to assess
• Select one of the easier assessment methods
• Testing
• Surveys
• Archival Data
• Seek exemplars
• Evaluate each possible assessment method
• ( advantages, disadvantages, considerations)
• A matrix approach will help

18
Some Sources of Assessment

• STUDENT CENTERED
• Exit interview
• Focus group
• Case study
• Portfolio (paper or electronic)
• Self-rating of progress
• Capstone course
• Senior exit essay
• ARCHIVAL
• Library use, on-line activity
• Transcript analysis
• Transfer records
• STAKEHOLDERS
• Alumni, employee, employer surveys

• TESTS
• Standardized test
• Homegrown test
• In class objective test
• In class essay
• Course embedded
• PERFORMANCES
• Critical situation scenario
• Debate
• Demonstration
• Oral presentation
• Phone call
• Online monitoring

19
(No Transcript)
20
Considerations when developing measurements

• Availability
• Efficiency
• Cost (money and labor)
• Rubric development
• Inter-rater reliability
• Faculty buy in

21
Standardized Tests
Consider the advantages and disadvantages of
each Assessment method. For example

• Advantages
• Convenient
• Easy to score
• Provide comparisons over time

• Disadvantages
• Expensive
• No gains over time
• Motivation hassle
• Not student centered
• Fit - may not match department goals

22
In Class Objective Testing
Consider the advantages and disadvantages of
each Assessment method. For example

• Advantages
• Ease of scoring
• Quick feedback
• Item analysis possible
• Accommodates embedded assessment

• Disadvantages
• Difficult to write
• Usually test at shallow level of learning

23
Paper and Electronic Portfolios some examples

• One university offers a capstone seminar for
  sophomores. Students are taught how to create a
  Professional Planning Portfolio in which they
  collect evidence of progress toward their
  baccalaureate goals.
• One university is currently creating a portfolio
  in an electronic format. Students present this
  proof of competence to potential employers and
  graduate school admissions committees.
• One college requires freshmen to take Portfolio
  100 seminar, and Portfolio 400 as seniors.

24
Senior Exit Essays

• The School of Science requires graduating
  seniors to write essays that describe how
  successfully they have accomplished the
  Universitys Principles of Undergraduate
  Learning.

25
Alumni Surveys

• One universitys research office administers
  surveys to alumni designed to collect their
  perceptions of a wide variety of curricular and
  extracurricular topics.

26
Draft
27
Where do the 2-year colleges fit into the
psychology learning outcome work? Some
Considerations

• Many psychology majors come from 2-year colleges.
• Community college psychology departments need to
  be proactive rather than reactive to the learning
  outcomes movement within education and
  psychology.
• This year the ICAS Project IMPAC will focus on
  the social science disciplines. This is an
  opportunity for CC and University faculty to
  dialog on learning outcomes and assessment
  methods.

28
Where do the 2-year colleges fit into the
psychology learning outcome work?

• If the psychology learning outcomes are made
  explicit at 2-year colleges, they will
• Contribute to campus efforts to assess general
  education outcomes
• Help help align the 2-year college psychology
  curriculum with university lower division
  psychology curriculum
• Provide psychology departments with valuable
  program assessment methodology
• Provide psychology faculty with the assessment
  processes and tools to evaluate different
  instructional strategies
• Consequentially will improve student learning
  outcomes

29
One Community College Psychology Departments
Reflections on the SLOs

• We now realize that many out-of-class activities
  make very important contributions to student
  learning outcomes (and we need to measure and
  document these outcomes)
• Psi Beta club
• Annual CC psychology conference
• Student participation at professional psychology
  conferences

• The dialog has begun
• Need to identify majors
• Need to link with 4-year psychology faculty
• They can help us prepare transfer students for
  the types of assessments they will face
• We (2-year faculty) can learn from the SLO
  assessment experiences of 4-year faculty

30
One Community College Psychology Departments
Reflections on the SLOs (continued)

• We need SLO data on our transfer students
• E.g., grades in upper division courses
• Transfer GPA
• Time to degree
• Need to examine the need for prerequisites for
  some psyc courses
• E.g. writing prereq for research course

• Need to offer a Careers in Psychology course
• Need to identify new outcome measures where
  needed
• students presenting a research paper at a
  conference
• students entering and winning student
  competitions for research papers and community
  services projects
• Expand classroom use of Cooperative Learning
  Groups

31
One Community College Psychology Departments
Reflections on the SLOs (continued)

• The psychology department intends to publish
  the student learning outcomes widely (department
  web page, program brochure, syllabi, etc.)
• The psychology department is considering
  establishment of a program advisory committee
  composed of
• Students
• Alumni
• Faculty representatives from nearby 4-year
  college psychology departments
• In house psychology faculty (including adjuncts)

32
Sequence the Psychology Department will Follow to
Implement the SLOs in Psychology

• We will use the SLO outcomes
• To Evaluate of the Psychology Program (Program
  Review)
• To Support the Colleges Accreditation Work by

33
Identify SLOs Appropriate for Lower Division
Psychology Students

• The psychology department will review then
  incorporate the American Psychological
  Association's ten standards for undergraduate
  education as its student learning outcomes
  (SLOs).

Note the following example was adapted from a
slide presentation by Dr. Drew Appleby at the
UIPUI.
34
2. Identify Assessment Strategies for the
Respective SLOs

• Next, the department will identify assessment
  strategies that have been or will be used to
  assess students abilities to accomplish these
  SLOs.

35
3. Organize

• The next step will be to organize these SLOs and
  Assessment Strategies into a matrix displaying
  outcomes by assessment methods and. . .

36
(No Transcript)
37
. . . faculty will place an X in each of the
cells that represents an existing assessment
strategy.
38
(No Transcript)
39
4. Evaluate, Adopt an Assessment Plan

• The comprehensiveness of the assessment plan will
  be evaluated by identifying how many of the SLOs
  in the matrix have actually been assessed and how
  many of the methods have actually been used to
  assess the SLOs (see next slide).
• A second aspect of this evaluation will be to
  determine if the results from these assessments
  have actually been used to produce data-informed
  curricular revisions.

40
5. Create Electronic Matrix to Facilitate
Assessment Work

• A website will be created that displays this
  matrix.
• Faculty who have performed assessment strategies
  will be asked to submit reports of their
  strategies via an electronic form.
• These reports will then be linked to the website
  via hot-link buttons, which will replace the Xs
  in the matrix.
• Each report will consist of the following
  sections

41

• 1. A description of the assessment strategy
• 2. The results of the strategy
• 3. The changes produced by these results
• This will allow all departmental assessment
  activities to be centralized in one,
  easy-to-access location.

42
6. Maintain

• The website and all of its contents (i.e., the
  matrix, its linked reports, and copies of all
  previous and current annual departmental
  assessment reports) will be maintained as a
  permanent part of the departments website.

43
7. Revise

• The website will allow our department to revise
  its assessment strategies easily.
• SLOs and Assessment strategies can be added,
  deleted, or modified.
• New curricular changes can be described.
• Annual assessment reports can be added.

44
8. Utilize

• The methods described in the website can be
  utilized by members of the department to assess
  and improve teaching, advising, and learning.
• A tour of the matrix will provide website
  visitors with all the assessment information that
  exists in the department.

45
9. Communicate

• Although its initial purpose will be to inform
  the colleges accreditation team, the website
  will also communicate the departments assessment
  activities to
• prospective students and their parents
• other psychology departments
• individuals, academic departments, or
  administrative offices at the college

46
The structure of the website will be . . .

• a 10x8 matrix composed of
• the departments Student Learning Outcomes
• the Assessment Strategies that have or will be
  used to evaluate the accomplishment of these SLOs

47
A First Effort Assessment Attempt Using the New
SLOs in Psychology
48
SLO TwoResearch Methods in Psychology

• Students will understand and apply basic research
  methods in psychology, including research design,
  data analysis, and interpretation.

49
SLO SixInformation and Technological Literacy

• Students should demonstrate information
  competence and the ability to use computers and
  other technology for many purposes.

50
A survey was designed and administered to assess

• GOAL 2 Research Methods in Psychology
• Interpret basic statistical results
• Collect, analyze, interpret, and report data
  using appropriate statistical strategies to
  address different types of research questions and
  hypotheses
• GOAL 6 Information and Technological Literacy
• Use appropriate software to producestatistical
  and qualitative analyses, including graphical
  representation of data
• Use basic word processing, database, email,
  spreadsheet, and data analysis programs

51
Student Survey 40 questions on skills in using
computer tools by undergraduates after completion
of Psychology 10 (statistics) and Psychology 2
(research methods)
Please respond to each of the following
statements (1-21) by answering the extent to
which you agree with each. A Strongly Agree B
Agree C Neutral D Disagree E Strongly
Disagree DO THIS ANONYMOUSLY!  DO NOT PUT YOUR
NAME OR IDENTFYING MARKS ON THE ANSWER SHEET. 
1. I can activate Excel on a PC running
Windows. 2. I can cut, paste, and edit cells in
Excel. 3. I can generate formulas for mean and
standard deviation using the function
option in Excel. 4. I can sort data sets and
insert rows and columns into spreadsheets.
52
5. I know which axes of a graph should be used to
plot the independent and dependent
variables, respectively. 6. I know how to save
files to a designated disk in any Office
application. 7. I have heard of SPSS or know
what its purpose is. 8. I know how to enter data
into SPSS. 9. I know how to import data from
Excel files into SPSS. 10. I know how to define
variables in SPSS. 11. I know the key
differences between paired sample and independent
sample t-tests. 12. I know how to perform a
paired sample and an independent sample t-test in
SPSS. 13. I know how to read and interpret the
output of a t-test from SPSS.
53

• To my colleagues, from JR
• My 20 years in teaching P2 and P10 has opened my
  eyes to a few issues about what we expect vs.
  what is reality in our training and competence of
  undergraduates here. I assumed all students
  finishing these two courses (statistics and
  research methods) would know some basics about
  use of computer tools for quantitative
  applications (i.e., Excel and SPSS). I was
  wrong. Based upon our recent survey (spring, 2001
  final exam week) of students who had completed
  these courses here or at other colleges 
• 71 of the class does not know how to generate a
  mean or sd with the fx option in Excel
• 25 of the class does not know how to generate
  graphs using a program such as Excel or Word
• 35 does not know which axis of the graph to use
  for
• independent and dependent variables
• 20 of the class has never heard of SPSS
• 62 does not know the difference between
  independent and
• paired-sample t-tests
• 71 of the class believes they should learn Excel
  and SPSS
• before entering Psychology 2
• 53

54
Proposed Change

• Form a teaching circle to discuss findings and
  to propose changes in teaching and/or curriculum.
  (Teaching circles are one way in which faculty at
  the college can complete their professional
  development/flex hour obligation.)

55
The goal of this teaching circle will be to
bring about curricular changes that will produce
a standardization of the teaching of SPSS and
Excel in our methods course. This change will
have three important results. 1st ? The faculty
who teach the course will have an opportunity to
sit down and discuss their concerns and
expectations with each other. 2nd ? Instructors
in the course can be assured their students will
leave their course with a specific level of SPSS
and Excel competence. 3rd ? All students who
complete our methods sequence will exit with the
same level of SPSS and Excel skill.
56
Closing Observations The new emphasis on
assessment of Student Learning Outcomes will

• Facilitate the paradigm shift from
  teacher-centered to student-centered instruction
• Benefit students
• Help psychology departments to conduct more
  meaningful program reviews
• Facilitate communication and agreement among
  faculty regarding
• SLOs for psychology transfer students
• Where these SLOs should be taught
• How these SLOs should be taught and the
  effectiveness of different teaching strategies
• The means by which SLOs can be assessed

57
The End of a New Beginning

• Thank You!