Proving and Improving: Purposeful Themed Learning Communities Assessment

1
Proving and Improving Purposeful Themed Learning
Communities Assessment

• International Conference on The First-Year
  Experience

Lauren Chism, Associate Director, Center on
Integrating Learning and Coordinator of Themed
Learning Communities Michele J. Hansen, Ph.D.,
Director of Assessment, University College
July 11, 2007 - Hawaiis Big Island, Hawaii
2
IUPUI Context

• Large urban, commuter, public university.
• 76 commute to campus.
• Large number of under-prepared fist-year students
  (33 are admitted conditionally).

• 59 of fall first-year students are
  first-generation college students (neither parent
  completed a four year college degree).
• 30 of first-time, full-time students report that
  they plan to work more than 20 hours per week
  while attending school

3
Learning Communities at Your Institutions

• How many of you are coming from institutions with
  learning communities?
• What is a learning community at your
  institution?
• Some types of learning communities
• Residential programs
• Freshman seminar courses
• Linked courses or course clusters
• Team-taught coordinated study programs
• Any combination of these and more!

4
What is a Themed Learning Community (TLC)?

• 3 or more linked courses including an integrative
  first year seminar connected through an
  interdisciplinary theme.
• First year seminar is taught by an instructional
  team including a faculty member, academic
  advisor, librarian and student mentor
• Faculty and instructional team members work
  together to integrate the curricula
• Involve exciting opportunities for experiential
  learning (co-curricular/service learning
  experiences)

5
Examples of TLCs

• Examples of Themed Learning Communities
• For Love AND Money
• (English, Psychology, Math Career
  Exploration Seminar)
• Urban Community Past and Present
• (Education, History, English First Year
  Seminar)
• Cant We All Just Get Along?
• (Anthropology, Psychology, English and First Year
  Seminar)
• Complete listings and descriptions
• http//opd.iupui.edu/Units/COIL/tlcoverview.asp
• Examples of Experiential Learning
• Athletic TLC students visit NCAA headquarters
• Education TLC students tutor children in the
  community
• Multicultural Perspectives TLC students attend a
  Ramadan celebration

6
Evolution of Learning Communities

• 1995 
• Learning communities begin at IUPUI
• LCs are first year seminar courses often paired
  with another introductory course using an
  instructional team model
• 2001 
• After an external team of reviewers encouraged
  IUPUI to extend learning communities beyond
  linking two courses, block scheduling piloted in
  2001.
• 2002-2003
• The University College Schedule Block
  Committee/National Learning Communities institute
  team strongly recommended the move to Themed
  Learning Communities in 2002.
• 2003-Present
• 7 TLCs in 2003 27 TLCs scheduled for 2007

7
Goals

• TLC Goals for IUPUI
• To increase retention rate for first-term
  students
• To improve TLC participants GPA
• To improve TLC participants satisfaction with
  IUPUI
• To improve TLC participants graduation rates
• TLC Goals for Students
• To form learning support networks among students
  in their community
• To enhance student contact with a network of
  faculty and staff
• To promote collaborative and active learning
• To understand the value of diversity by exposure
  to multiple points of view
• To apply classroom learning to the real world
• To understand the relationship between academic
  learning and co-curricular activities
• To provide opportunities to integrate learning
  across academic and professional disciplines that
  will enable students to understand their learning
  in coherent, comprehensive ways

8
The Plenary

• Interdisciplinary, integrated core curriculum
  emphasizing links across courses and ideas
• P. Terenzini
• International FYE Conference
• Big Island, Hawaii
• 7/9/07

9
Why Assess Learning Communities?

• Demonstrate Worth and Value
• Learn about Impacts and Goal Achievement
• Course Development and Improvement
• Obtain Student Feedback

10
Characteristics of Fall 2005 TLC Cohort

• 391 Students Participated
• 74 Women
• 12 African American
• 63 First-Generation College Student
• 35 Admitted Conditionally
• 967 Average SAT Score
• 62 Average High School Percentile Rank
• 25 Campus Housing
• 25 Summer Bridge Participants
• 18-19 Primary Ages. Average18.63

11
Characteristics of Fall 2006 TLC Cohort

• 377 Students Participated
• 74 Women
• 8 African American
• 68 First-Generation College Student
• 26 Admitted Conditionally
• 990 Average SAT Score
• 67 Average High School Percentile Rank
• 27 Campus Housing
• 28 Summer Bridge Participants
• 18-19 Primary Ages. Average18.96

12
Assessment Methods

• Comparative G.P.A and Retention
• Comparative NSSE Data
• Student Feedback Questionnaire
• Satisfaction
• Self-Reported Learning Gains
• Quantitative Qualitative Items
• Other Assessment Methods
• Faculty Student Focus Groups
• Assessment conducted within TLCs

13
How Can We Produce Appropriate Comparisons?

• Barriers
• Random assignment is simply not possible or
  ethical.
• The vast majority of incoming students
  participate in a freshman seminar with a history
  of increased retention.
• There are potential disparities in students
  previous academic achievement and other
  predictors of success.
• It is important to control for background
  characteristics, academic preparation, other
  program participation.

14
First Semester G.P.A.

• Comparison group students who participated in
  a freshman seminar or learning community.
• G.P.A. adjusted to control for significant
  covariates including course load, age, gender,
  ethnicity, SAT scores, high school percentile
  ranks, units of high school math, and summer
  bridge participation.
• plt.01

15
Retention

• Fall 2003 Adjusted One Year Retention
•  
• of Students Adjusted Retention Rate
• TLC Participants 121 76
• Non-TLC Participants
  1164 69
•  
•  
• Fall 2004 Adjusted One Year Retention
•  
• of Students Adjusted Retention Rate
• TLC Participants 287 69
• Non-TLC Participants 1351
  68
• Fall 2005 Adjusted One Year Retention
•  
• of Students Adjusted Retention Rate
• TLC Participants 338 70
• Non-TLC Participants 1211
  65

16
A Combination of Interventions
Summer Bridge Academy

• About ½ of TLC students participated in Service
  Learning
• 54 in 2005 44 in 2006
• Approximately ¼ of TLC students participated in
  the Summer Bridge Academy
• 25 in 2005 28 in 2006

First Year Seminar
TLCs
Service Learning
In 2005, participation in the Summer Bridge
Academy and/or Service Learning appeared to
significantly increase TLC students GPA. Bridge
appeared to have a significant impact on
retention. N Adjusted GPA Adjusted
First Year Retention Bridge TLC 97
3.00 83 TLC 294 2.60 67 Service
Learning TLC 212 2.82 73 TLC 179
2.57 69 In 2006, there was no significant
difference for either.
17
2006 Student Feedback Questionnaire (N320) Top
ranked by mean rating ( Very Much and Much)

• Formed friendships that will maintain after the
  semester (79).
• Exchanged ideas with students with different
  views (61).
• Became more effective in communicating my
  thoughts in writing (57).
• Understood connections between different
  disciplines/courses (61).
• Developed a better understanding of complex real
  world social problems and issues (60).
• Actively discussed complex issues and ideas
  (59).
• Discussed ideas with peers outside of class
  (52).
• Overall how satisfied were you with your TLC
  experience?
• Mean 4.03
• 81 satisfied/very satisfied 5 dissatisfied,
  very dissatisfied)

18
Qualitative Survey Items

• Please describe how the theme of your learning
  community contributed to your learning
• Knowledge of Career/Major/Career Exploration
  Process (25)
• Cohort/Support Network (18)
• Connections/Integrative Curricula (15)
• Please describe what you liked most about your
  Themed Learning Community experience
• Cohort/Support Network (45)
• Professors (14)
• Interdisciplinary Connections (10)
• Please describe what you liked least about your
  Themed Learning Community experience
• Nothing (15)
• Lack of Interdisciplinary Connections/Coordination
  (12)
• Assignments (12)

19
National Survey of Student Engagement

• Significant differences between TLC students (80)
  in comparison to others (280)
• Students participating in TLCs more often
• Worked on a paper or project that required
  integrating ideas or information from various
  sourcesmade more class presentations
• Included diverse perspectives (different races,
  religions, genders, political beliefs, etc.) in
  class discussions or writing assignments
• Put together ideas or concepts from different
  courses when completing assignments or during
  class discussions
• Worked harder than thought they could to meet an
  instructor's standards or expectations
• Tried to better understand someone else's views
  by imagining how an issue looks from his or her
  perspective
• Learned something that changed the way you
  understand an issue or concept
• TLC students also reported more instituional
  emphasis on
• Providing the support needed to help students
  succeed academically
• Encouraging contact among students from different
  economic, social, and racial or ethnic
  backgrounds
• Helping students cope with non-academic
  responsibilities (work, family, etc.)
• Working effectively with others
• Understanding people of other racial and ethnic
  backgrounds

20
Assessment Conducted Within TLCs

• TLC entitled For Love and Money
• First-Year Seminar, Psychology, English, and Math
• TLC designed to help students learn more about
  who they are and what majors and careers would
  help them reach meaningful life goals and pursue
  their passions.
• Career Decision Self-Efficacy Scale - Short Form
  (Betz and Taylor, 2001)
• Composed of Sub-Scales
• Employed a Pre-Post Design with a Control Group
  (a non-TLC First-Year Seminar Course)

21
TLC Career Decision Making Self-Efficacy (n17)
22
Non-TLC Career Decision Making Self-Efficacy
(n19)
23
Dissemination of Data

• Discussed at annual retreats and faculty
  development workshops
• Included in TLC brochures and faculty handbooks
• Presented annually to academic advisors
• Presented at New Faculty Orientation
• Disseminated to any deans, administrators,
  faculty, staff or students interested in the
  program.

24
Lessons Learned from Assessment

• Themed Learning Communities seem to provide
  opportunities for student connections.
• Students establish friendships by participating.
• Students able to integrate learning between
  courses and disciplines.
• Important to create structures and procedures
  that allow faculty to collaborate and engage in
  interdisciplinary pedagogies.
• Early findings suggest positive impacts on
  academic success and retention. Will to continue
  monitor impacts

25
Work Undone and Remaining Challenges in Themed
Learning Community Assessment

• What are the impacts of discrete components or
  intervening variables that impact LC success
  (e.g., co-curricular activities, service
  learning, integration of learning, writing across
  the curriculum, establishing connections with
  other students, civic engagement, etc.)
• What structures can implemented to facilitate
  faculty interaction and collaboration?
• How can faculty effectively integrate themes and
  still cover discipline course content?

26

• Questions Sharing of Assessment Practices
• Contact information
• lchism_at_iupui.edu
• mjhansen_at_iupui.edu