College Portraits: Holding Ourselves Accountable

1
College PortraitsHolding Ourselves Accountable

• Presented by Richard H. Wells, Chancellor, UW
  Oshkosh and Chair, VSA Student Engagement Task
  Force
• September 2007

2

• Voluntary System of Accountability (VSA) Goals
• Redefine what is important when choosing a
  college
• Make it easy to find this information
• Respond to concerns raised by the Spellings
  Commission
• Standardize presentation to ease criticisms
  about comparable information availability
• Add process and outcome measures to what is
  reported

3
2 Associations 83 Participants 110 Monitors 4
Meeting Rounds High Transparency Funded by a
Lumina Foundation Education grant
4
VSA Web Page

• VSA will include transparent information about
  each participating university and its programs
  utilizing common definitions and format.  
• Each universitys VSA web page will include
• descriptive data about the university, the
  programs offered and characteristics of its
  students,
•  a mechanism for the students to calculate their
  estimated net cost of attendance,  
• various success measures such as graduation rates
  and continued enrollment of students who transfer
  into other universities,
• a measures of post graduation plans,
• direct learning outcome measurement of the
  value-added by the university to undergraduates
  in the areas of critical thinking, analytic
  reasoning and written communications ability,
• indicators of the engagement exhibited by the
  campus students in several strategic areas.

5
Student Engagement Task Force

• A large body of research clearly documents the
  types of student engagement activities and
  programs that are known to contribute to student
  learning and development
• Active learning experiences
• Group learning experiences
• Institutional commitment for student learning and
  success
• Educationally purposeful student interaction with
  campus faculty and staff
• Experiences with diverse groups of people and
  ideas
• Student satisfaction with overall undergraduate
  experience

6
Model of Factors that Impact Student Learning and
Growth
7
Student Engagement

• Student engagement occurs at the intersection of
  student behaviors and institutional conditions.
• Assessment of student engagement experiences will
  require periodic (at least once every 3 years)
  administration of
• NSSE
• CIRP
• CSEQ or
• U-CUES.
• Instrument choice is up to the institution.
  Administration is for new freshmen and
  about-to-graduate seniors.

8
Student Behaviors

• Interaction with Faculty
• Student-faculty interaction is important because
  it encourages students to devote effort to other
  educationally purposeful activities. Both the
  nature and the frequency of the contact matter.
• The evidence suggests that student-faculty
  interactions outside the classroom that reinforce
  or extends the formal academic experience or that
  focus on issues of student development can have
  positive effects on dimensions of general
  cognitive development - for example, working on a
  professors research project.
• Student contact with faculty members outside the
  classroom also promotes student persistence,
  educational aspiration, and degree completion
  even when other factors are taken into account.
• Informal, social interactions between faculty and
  students appear to have less of an impact on
  student learning.
• The material presented in this and the next 10
  slides is taken directly from Christine Kellers
  white paper, Student Behaviors and Institutional
  Conditions that contribute to Student Learning,
  which provides a broad overview of the literature.

9
Student Behaviors

• Peer Interaction
• Peers are the single more potent source of
  influence on virtually every aspect of
  development cognitive, affective,
  psychological, and behavioral (Astin 1992, p.
  398).
• Peer influence is a statistically significant and
  positive force in students persistence
  decisions.
• Interaction with peers that extend and reinforce
  broad ideas introduced in the academic experience
  and that confront the individual with diverse
  interests, values, political beliefs, and
  cultural norms appear the most salient for
  positive impact on critical thinking, analytical
  skills, and post-formal reasoning.
• Specific peer interactions that foster learning
  include the following (Kuh et al. 2007).
• Discussing core content with other students
• Working on group projects with other students
• Teaching or tutoring other students
• Participating in intramural sports
• Discussing racial or ethnic issues
• Socializing with someone from a different racial
  or ethnic group
• Being elected to a student office

10
Student Behaviors

• Experience with Diversity
• Student involvement in diversity experiences has
  a positive effect on dimensions of general
  cognitive development such as critical thinking,
  analytical competencies and thinking complexity
  as well as academic subject matter for students
  of all racial groups. However, there is evidence
  that White students benefit more from diversity
  experiences than African American or Hispanic
  students.
• Salient diversity experiences include informal
  interaction with racially and culturally diverse
  peers as well as involvement with more formal
  programs such racial cultural awareness workshops
  and coursework focusing on social cultural
  diversity and interracial relations. Research
  suggests that formal programs and coursework have
  a more significant effect on White students than
  informal interactions.

11
Student Behaviors

• Service Learning
• Involvement in academically integrated service
  learning has a unique positive impact on
  dimensions of general cognitive development such
  as critical thinking, analytical competencies and
  thinking complexity as well as academic subject
  matter knowledge
• The most effective service learning experiences
  appear to be those that integrate service with
  course content, provide for reflection about the
  service experience and permit the student to
  apply subject matter learning to the service
  experience and vice versa.
• Not only does participation in academically
  integrated service learning increase general and
  subject matter knowledge, it has also been shown
  to enhance a students grade point average and
  aspirations for advanced degrees. It is also
  associated with increased time devoted to
  homework and studying and increased contact with
  faculty.

12
Student Behaviors

• Academic Effort and Involvement
• Non-classroom interactions with peers and faculty
  that extend and reinforce what is happening in
  ones academic experience appear to have the most
  consistent positive impact.
• A students level of academic effort and
  involvement has an important net influence on
  growth in general cognitive skills and
  intellectual development. The level of individual
  effort or engagement in areas such as hours
  studied per week, the number of non-assigned
  books read, writing experience, library use, and
  course learning activities appear to have a
  unique positive impact on standard measures of
  critical thinking as well as on self reported
  gains in critical thinking and intellectual
  development.

13
Student Behaviors

• Co-curricular Activities and Involvement
• The nature of students social and cocurricular
  involvement has a unique impact on learning - in
  addition to more formal classroom instruction.
  Co-curricular involvement is also positively
  related to persistence.
• Co-curricular activities such as involvement in
  clubs and organizations may foster critical
  thinking.
• Participation in intercollegiate athletics,
  particularly mens revenue producing sports,
  appears to have an inhibiting impact on the
  development of critical thinking skills as well
  the acquisition of subject matter knowledge.
• Greek affiliation has a negative impact on
  subject matter knowledge and cognitive
  development during the first year. However, the
  negative effect diminishes in future years.
• On or off campus work during college,
  particularly part time, has a trivial influence
  on student learning and cognitive development.
• There is no consistent evidence to suggest that
  living off campus and commuting directly inhibits
  acquisition of subject matter knowledge or
  cognitive development. However, living on campus
  appears to exert an indirect positive influence
  on learning by enhancing academic and social
  engagement. Students who live on campus are more
  likely to persist and graduate than students that
  commute as living in a residence hall facilitates
  students social and academic involvement with
  other students, with faculty members, and with
  their institution.

14
Student Behaviors

• Differences in Engagement among Student Groups
• First generation students tend to be less engaged
  than other students, perhaps in part because they
  have less tacit knowledge of and fewer
  experiences with college campuses and related
  activities, behaviors, and roles models (Kuh et
  al. 2007).
• NSSE results demonstrate that transfer students
  share many characteristics with both older
  students and commuters, but differ in marked ways
  from students who start and graduate from the
  same college (Kuh et al. 2007).
• Interact less with faculty
• Participate in fewer educationally enriching
  activities
• Did more active and collaborative learning
• Viewed the campus as less supportive
• Were less satisfied overall with college
• Academic-social engagement in college may have
  stronger positive effects on general cognitive
  development for student with relatively low
  tested academic ability than for their
  counterparts with relatively higher academic
  ability. Further, grades of lower ability
  students are positively affected by engagement in
  educationally effective activities to a greater
  degree as compared with higher ability students.

15
Institutional Conditions

• Faculty-Student Interaction
• Replicated evidence that suggests that student
  critical thinking, analytical competencies, and
  general intellectual development are enhanced by
  an institutional environment that stresses close
  relationship and frequent interaction between
  faculty and students and faculty concern about
  student growth and development.
• Several studies suggest that students perception
  of faculty members availability and interest in
  them may be enough to promote persistence.
• The relationship between faculty and students
  appear to facilitate two processes. One is the
  socialization of students to the normative values
  and attitudes of the academy. The other is the
  development and strengthening of the bond between
  student and institution.

16
Institutional Conditions

• Academic Programs and Services
• Academic programs and experiences that actively
  engage students and foster academic and social
  integration are positively linked to student
  learning first year seminars, effective
  academic advising, peer mentoring , advising and
  counseling, summer bridge programs, learning
  communities, living-learning centers, and
  undergraduate research programs.
• Studies uniformly demonstrate that first year
  seminars, effective academic advising, and
  comprehensive support and retention programs
  (e.g., the TRIO programs) have positive effects
  on student persistence and graduation.
• The combination of faculty-student contact and
  active learning within undergraduate research
  programs appears to be particularly potent in
  increasing persistence and degree completion
  rates.

17
Institutional Conditions

• Classroom Learning Environments
• Environments that stress high standards and
  expectations for student performance are linked
  to increased student learning.
• Pedagogical approaches that appear to improve
  subject matter learning over traditional
  approaches include learning for mastery,
  computer assisted instruction, active learning,
  collaborative learning, cooperative learning, and
  small group learning.
• Problem-based learning and learning communities
  appear to foster improved subject matter learning
  and the development of general cognitive skills
  such as critical thinking.
• Certain teaching behaviors have a positive effect
  on the acquisition of subject matter knowledge
  e.g., teacher preparation-organization, quality
  and frequency of feedback, teacher availability.
  Moreover, teacher preparation and organization
  also appear to have positive impacts on more
  general measures of learning not tied to specific
  courses
• Little consistent evidence to suggest that a
  students major field of study in and of itself
  leads to different effects on general measures of
  critical thinking.
• Intellectual training in different fields of
  study leads to the development of different
  reasoning skills.
• Evidence to suggest that coursework in natural
  science courses may positively influence growth
  in critical thinking skills.
• Learning a computer programming language provides
  advantages in general cognitive skills such as
  planning, reasoning, and metacognition.
• Coursework requiring students to learn to use
  computers to analyze data, make visual displays,
  and search the internet for course material also
  contributes to greater growth in general
  cognitive skills.
• Critical thinking is enhanced by curricular
  experiences that require the integration of ideas
  and themes across courses and disciplines.
  Interdisciplinary or core curriculum that
  emphasize making explicit connections across
  courses and among ideas and disciplines
  positively influence students ability to solve
  ill-structured problems.

18
Institutional Conditions

• Institutional Structures and Environments
• Institutional structure is less important to
  positive educational outcomes than the kinds of
  peer groups and faculty environments that emerge
  across different types of institutions. For
  example, the underlying importance of peer groups
  is that students in similar circumstances and
  with common needs and interests have been
  afforded an opportunity to interact and learn
  together (Astin 1992, p. 415). Such peer groups
  may happen more naturally at a liberal arts
  residential campus, while other types of campuses
  may have to be more deliberate in creating such
  opportunities and environments.
• Institutional resources and reputation (e.g., as
  reported in U.S. News) are largely irrelevant to
  high quality educational experiences as measured
  by student engagement in educationally enriching
  activities (Kuh et al. 2007).
• College quality has less important implications
  for intellectual and personal growth during
  college than for career and socioeconomic
  achievement. Net of confounding influences (pre
  college abilities or cognitive level) attending
  an academically selective institution has a
  negligible effect on knowledge acquisition or
  general cognitive development (Pascarella
  Terenzini 2005).
• Studies consistently indicate that students
  attending private institutions have an advantage
  over their counterparts at public colleges in
  terms of persistence and degree completion.
  Although with controls in place for precollege
  characteristics, the private institution
  advantage is muted or disappears. The effects of
  attending a private college are likely indirect
  and influenced by other institutional traits such
  as size, selectivity, emphasis on undergraduate
  education, and faculty and peer relations.

19

• Learning Outcomes
• Reports of Learning Outcomes will be based on
  administration of one of three VSA approved
  instruments
• College Learning Assessment (CLA)
• Motivational Appraisal of Personal Potential
  (MAPP)
• Collegiate Assessment of Academic Proficiency
  (CAAP)
• The same instrument must be administered to
  students during their first semester of
  attendance and also during their last semester of
  attendance.

20
Learning Outcomes

• Analysis will be cross-sectional and the results
  will be displayed as gain scores (senior scores
  freshman scores).

All of the Test Providers have committed to
providing the data in the same above fashion.
This is a made-up CLA example. The National
Comparison comment is based on distance from
expected score for the institution. A result of
/- 1 standard error is reported As Expected.
21
Next Steps

• UW Oshkosh Timeline for Achieving VSA Early
  Adopter Status
• September and October 2007
• Presentations to all governance groups
• Campus-wide Open forums
• November 1, 2007
• UW Oshkosh makes decision about becoming an
• Early Adopter of VSA

Proposed VSA Rollout Schedule