Dale R. Tampke

1
Developing and Implementing a Web-Based Early
Alert System

• Dale R. Tampke Dean, Undergraduate Studies,
  University of North Texas
• dale.tampke_at_unt.edu

2
Where were headed today

• Our Context - UNT
• Early Alert as a Concept
• Project Scope (the tech-y part)
• Building Advocacy
• Functionality
• End-user
• Responder
• Data from 2011-12 (and what weve learned so
  far)
• System improvements

3
University of North Texas - UNT

• Main campus Denton, TX
• Enrollment
• 35,754 total headcount
• 23,756 undergraduates
• Moderately selective
• SAT 1105
• ACT 23.4

• 11 Colleges/Schools
• Degrees
• 97 Bachelors
• 101 Masters
• 48 Doctoral
• Faculty
• 988 FT
• 519 PT
• Median Class Size - 28

4
A bit more about UNT

• Gender
• Female (56.0)
• Ethnicity
• White (62.2)
• African American (13.2)
• Latino (12.8)
• Asian (5.5)
• Native American (0.7)
• Non-resident Alien (4.7)
• Over 80 from lt100 mi

• 25 Pell eligible
• 49 first-generation
• Students admitted into colleges and schools
• Mandatory two-day summer orientation
• FTIC retention rate 75.6 (2011 cohort)
• Six-year graduation rate 49.4 (2005 cohort)

5
Please respond to the following

• Describe your institution
• Public or Private
• Two-year or Four-year
• Small (999 and below), Medium (1,000 4,999),
  Large (5,000 24,999), Mega
  (25,000 and up)
• Residential or commuter
• Urban or rural

6
The Early Alert concept

• Grounded in literature on undergraduate retention
• Student behavior can predict attrition
• Early intervention can change outcomes
• First efforts were course-centered
• Poor performance
• Excessive absences
• (Think mid-term grades)

7
Early Alert progresses

• Expansion to campus-wide availability
• Include psycho-social concerns
• Web front end
• E-mail back end
• Authentication varies
• Integration varies
• A common issue
• How many faculty use the system?

8
Our idea

• Integrate with student information system
• We could build it ourselves
• Start with a focus on faculty (make it easy for
  them)
• Designate a central receiver of the data
• Expand beyond academic issues
• Have a ready referral
• Begin a personal, caring conversation

9
Heres a question

• What stakeholders would you need to include to
  implement an Early Alert system on your campus?

10
Building Advocacy

• Include stakeholders
• Students (8 from office staffs)
• Faculty (12 from Arts and Sciences)
• Academic Advisors (10 from all colleges)
• Student Services (15 areas)
• IT
• Get feedback at the conceptual stage
• Be ready to adopt a good idea
• Create a faculty test group

11
Things to ask (examples)

• Issues that affect student performance
• User access to the system
• Information a user would need to know about a
  student
• How and whether to inform the student of the
  alert
• Security and permissions
• Real time or batch processing
• Reporting (programmed, ad hoc, or both?)

12
Aspects of the system

• Secure authentication required
• Campus wide access
• Easy for faculty to use
• Menu-driven
• Minimal information about the student needed
• Ability to inform referred student via e-mail
• Timely
• Real-time ad hoc query capability
• Nightly reporting
• Completed in six weeks by one programmer

13
A question

• What student issues would be included in a
  drop-down menu on an Early Alert system at your
  campus?

14
Reasons for Referral(whats on the drop down
menu)

• Poor class attendance
• Poor performance on quizzes/exams
• Poor performance on writing assignments
• Does not participate in class
• Difficulty completing assignments
• Difficulty with reading
• Difficulty with math
• Sudden decline in academic performance
• Concerns about their major

• College adjustment issues
• Financial problems
• Physical health concerns
• Mental health concerns
• Alcohol or substance use concerns
• Roommate difficulty
• Disruptive behavior
• Absent from work
• Student needs veterans assistance
• Other concerns (text box)

15
How Early Alert works

• EARS 1.0 (early alert referral system) is
  available from the on-line class roll
• Instructors of record receive an e-mail reminding
  them of EARS at the beginning of the term
• Accessed through the faculty portal (The Faculty
  Center)
• Nightly report delivered to a central office
  (Student Academic Readiness Team START)
• Follow up within one day of receiving

16
Other features

• Relationship to student
• Professor, instructor
• Teaching assistant, teaching fellow
• Academic Advisor
• Mentor
• Department administrator
• Campus Employer
• Club, organization advisor
• I have had a conversation with the student
• Send a copy of the referral to the student (via
  e-mail)

17
Another question

• How would access to alert records be determined
  on your campus? Consider academic advisors,
  student services staff, faculty, clerical staff,
  others?

18
Accessing Early Alert
From the Faculty Center in the Student
Information System
19
To the class roster
20
From the class roster
21
To the Early Alert form
22
After the referral is made

• Review report every morning
• Real-time e-mail prompt to sender
• Morning report
• Includes following information
• Demographics
• Student ID
• Faculty members name
• Course
• Reason(s) for referral

23
Follow-up Routing alerts

• First responders Routine referrals
• Residence hall staff
• Course Achievement Assistants (peers)
• More serious issues
• Academic Readiness Advisors
• Academic Advisors
• CARE team
• Counseling, Health Center
• EARS is not designed for urgent situations

24
More follow-up The student experience

• Caring conversation (no scolding)
• Emphasize mattering
• Resources
• Self-efficacy
• Focus on academic success
• Follow-up2 (we need to get better at this)

25
EARS Data from UNT

• Descriptive data from academic year 2010-11

26
Alert frequency during the term
27
Alert frequency during the term
28
First reasons for alerts
29
Demographic data
30
Gender
31
Annual Totals
32
Outcomes data

• Analysis from Fall 2008 (pilot year)

33
Outcomes

• Literature suggests early intervention impacts
• Student success
• Student persistence/progression
• Fall GPA
• Spring re-enrollment
• Use a within-group comparison
• No useful control group

34
Findings

• Course Grade Distribution
• As 3.4
• Bs 5.9
• Cs 11.9
• Ds 12.3
• Fs 43.0
• Is 1.3
• Drops 21.7

• Success and Persistence
• Fall GPA 1.39
• Cumulative GPA 1.94
• Persistence 70.2

35
Contact types (frequencies)

• Faculty
• E-mail notice only 42.0
• Personal 8.2
• Both 3.5
• None 46.3
• Academic Readiness
• E-mail notice only 65.9
• Personal (phone, response from student, meeting)
  34.1

36
Outcomes by contact type
Fall GPA Persistence ( re-enrolling)
Faculty
E-mail only 1.19 62.6
Personal 2.17 85.7
Both 2.07 77.8
None 1.39 73.7
START
E-mail only 1.26 67.9
Personal 1.64 74.7
37
Some statistics
Personal Contact Mean Term GPA Significance
Faculty
Yes (n25) 2.15
No (n213) 1.30 F11.894, plt.001
START
Yes (n60) 1.63
No (n158) 1.26 F 5.436, plt.021
38
Outcomes by Contact Type by Reason(Attendance)
Attendance (n144) Fall GPA Persistence ( re-enrolling)
Faculty
E-mail only 0.83 53.1
Personal 1.96 100.0
Both 1.77 80.0
None 1.34 71.2
START
E-mail only 1.06 62.3
Personal 1.48 73.7
39
Outcomes by Contact Type by Reason(Performance)
Performance (n74) Fall GPA Persistence ( re-enrolling)
Faculty
E-mail only 1.90 83.3
Personal 1.88 100.0
Both 2.48 100.0
None 1.52 80.0
START
E-mail only 1.58 82.4
Personal 1.88 85.0
40
System Improvements

• EARS 2.0

41
Making the system better EARS 2.0

• Available to all staff via web portal
• Immediate e-mail communication
• To referrers
• To service providers
• To students
• Real-time referral based on alert type
• Improved outcome tracking using workflow
• Batch uploads (at-risk students)

42
From the staff portal
43
New responder screen
44
Responder notes
Responders can add an infinite number of Alert
Notes to track conversations / referrals they
have made for each student. Each note will be
time / date stamped and include Advisors EUID
and name.
45
Assessment data
46
COTS Early Alert Offerings

• SunGard Course Signals (Purdue) -
  http//www.sungardhe.com/signals/
• Hobsons Early Alert system - http//www.hobsons.c
  om/products/earlyAlert.php
• Starfish Early Alert - http//www.starfishsolution
  s.com/sf/solutions/earlyalert.html
• Datatel Retention Alert - http//www.datatel.com/p
  roducts/products_a-z/student-retention-software.cf
  m
• EducationDynamics Early Alert -
  http//www.educationdynamics.com/Retain-Students/E
  arly-Alert-Systems.aspx
• EBI MAPWorks - http//www.map-works.com/
• Sinclair Community College -http//www.sinclair.ed
  u/support/success/ea/

47
What weve learned

• Including faculty in the design was critical
• Linking to class roll, self-populating made it
  easier for faculty to use
• Faculty generally focus on course-related issues
• Personal faculty contact is the most effective
  follow-up
• E-mail contact by itself is not effective
• Some positive effect on success and persistence
  based on type of contact
• Timing of alert has no apparent effect on success
  or persistence
• Tracking confirmed contacts needs improvement
• EARS is not a large class solution

48
Resources

• Bowen, E., Price, T., Lloyd, S., Thomas, S.
  (2005). Improving the quantity and quality of
  attendance data to enhance student retention.
  Journal of Further and Higher Education, Vol. 29
  (4), 375-385.
• Eimers, M. (2000). Assessing the impact of the
  early alert program. AIR 2000 Annual Forum Paper.
  (ERIC Document Reproduction Service No. ED446511)
  Retrieved February 28, 2009, from ERIC database.
• Fischman, J. (2007, October 29). Purdue uses data
  to identify and help struggling students.
  Chronicle of Higher Education Online, Retrieved
  May 15, 2009 from http//chronicle.com/daily/2007/
  10/530n.htm.
• Geltner, P., Santa Monica Coll., CA. (2001).
  The characteristics of early alert students, Fall
  2000. (ERIC Document Reproduction Service No.
  ED463013) Retrieved February 28, 2009, from ERIC
  database.
• Hudson, W. (2006). Can an early alert excessive
  absenteeism warning system be Effective in
  retaining freshman students? Journal of College
  Student Retention, Vol. 7(3-4), 217- 226.

49
More references

• Kelly, J. Anandam, K. (1979). Computer enhanced
  academic alert and advisement system. (ERIC
  Document Reproduction Service No. ED216722)
  Retrieved February 23, 2009, from ERIC database.
• Richie, S. Hargrove, D. (2005). An analysis of
  the effectiveness of telephone intervention in
  reducing absences and improving grades of college
  freshmen. Journal of College Student Retention,
  Vol. 6(4), 395-412.
• Tampke, D. (2013). Developing, implementing, and
  assessing an early alert system, Journal of
  College Student Retention, 15 (1), in press.
• The Hanover Research Council. (May 2008).
  Intrusive advising and large class intervention
  strategies A review of practices. Washington,
  DC Author.
• Wasley, P. (2007, February 9). A secret support
  network. Chronicle of Higher Education, 53(23),
  A27.

50
Thank you for your participation!

• Dale R. Tampke
• Dean, Undergraduate Studies
• University of North Texas
• dale.tampke_at_unt.edu