Georgias eCore Project Course Evaluations

1
(No Transcript)
2
University System of Georgia Faculty Usage of
Course Management Systems (CMS) Survey
Dr. H. Hilliard Gastfriend Dr. Catherine
Finnegan Advanced Learning Technologies Assessment
and Public Information Board of Regents,
University System of Georgia
3
Technology Use in CoursesAdapted from Campus
Computing, 2002 - 2004Kenneth Green, October
2002 - 2004
4
Course Management Systems

• E-mail
• Bulletin Boards
• Chat
• Content Delivery
• Internet Links
• Online Testing
• Student Tracking
• File Exchange

5
Course Management Tools Percentage of courses
using course mgmt tools, by sector,
2000-2004Adapted from Kenneth Green,
www.campuscomputing.net
2000
2001
2002
2003
2004
6
University System of Georgia (USG)

• 34 public institutions
• Including 4-year and 2-year programs
• Student headcount enrollment
• 250,659 (Spring 2005)
• Total faculty
• 9,553 (Fall 2004)

7
USG WebCT Usage Overview

• 1996 Individual early adopter usage
• 1999 USG WebCT System Server
• Hosted over 2,000 courses for 20 institutions
• 2002 System-wide Vista License
• 2003 First institutions migrate to WebCT Vista
• 2004 USG server hosts more than 3000 courses
• 2006 All institutions complete migration to
  WebCT Vista

8
Historical Review of the CMS Survey

• Educause Center for Applied Research (ECAR)
  completed a course management systems (CMS)
  survey for the University of Wisconsin System in
  May 2003
• Wisconsins System comprises 15 institutions
• 13 universities 13, 2-yr colleges statewide
  extension service
• 133,701students (FTE, AY 2001-2002)
• 740 faculty responded (6718 FTE, 2002-2003)
• 140 interviews held

9
USG Adaptation

• Adapted Wisconsin study to meet needs of the
  Georgia System
• 33 institutions participated in the survey
• Georgia State had previously surveyed their
  faculty on similar topics and declined to
  participate
• 3,228 faculty have completed the survey
• Approximately 37 percent response rate

10
Survey Participation by Institutional Sector

• Sector Type Number Surveyed
• State Universities 13
• 2-year Colleges 13
• State Colleges 2
• Research Institutions 3
• Regional Universities 2

11
Survey Sections

• Overall Usage
• Familiarity with CMS
• Using CMS for the First Time
• Using CMS Tools
• CMS Advantages and Disadvantages
• Pedagogy and CMS
• CMS at the USG
• Affiliation and Status

12
Survey Format

• Total of 45 questions possible
• Branched survey formatnot everyone sees all
  questions depending upon their answers to key
  items
• Major branch point Do you currently use a CMS?
• If answer no, respondent by-passes approx. 25
  questions on CMS usage
• If answer yes, respondent sees majority of the 45
  questions

13
Key Study Questions

• What is the extent of the faculty use of CMS on
  each campus and system wide?
• What factors encourage faculty to start using a
  CMS
• What factors determine whether faculty increase
  or decrease their use?
• For what purposes are CMS used?
• What are the pedagogical advantages of using a
  CMS?
• What barriers prevent faculty from using a CMS?

14
Faculty Demographics and Extent of CMS Usage

• 46.3 of respondents were currently using a CMS
• 58.0 of respondents were tenured/tenure track
  faculty
• 36.1 have been using a CMS for more than three
  years
• Length of experience among CMS users
• One year or less (26.9 )
• Two years (21.2 )
• 61.1 consider themselves intermediate users

15
Sample Analysis of Survey Data

• Q.2 (all) Do you currently use a course
  management system (CMS), such as WebCT,
  Blackboard, LearningSpace, etc. for teaching and
  interacting with students?
• Choice Count Answering Asked of Total
• Yes 1486 46.3 46.2 46.0
• No 1722 53.7 53.5 53.3

16
Sample Analysis of Survey Data

• Q.3 (non users) What is the primary reason you do
  not currently use a CMS? (n1722 sample1743,
  N3228)
• Original table from Perseus
• Choice Count Answering Asked Total
• 1. Other (please specify) 431 25.0 24.7 13.4
• 2. I have no pedagogical reason
  427 24.8 24.5 13.2
• 3. I have not received training 415 24.1 23.8 1
  2.9
• 4. CMS are too time consuming to
  use 168 9.8 9.6 5.2
• 5. I did not know they were available 131 7.6 7.
  5 4.1
• 6. CMS are too difficult to use 49 2.8 2.8 1.5
  
• 7. CMS are not powerful/good enough
  47 2.7 2.7 1.5
• 8. CMS are unreliable 39 2.3 2.2 1.2
• 9. It is too difficult to obtain access
  12 0.7 0.7 0.4
• 10. They are too expensive to use 3 0.2 0.2 0.1
  

17
Sample Analysis of Survey Data

• Table after analysis
• Choice Count Answering Asked Total
• No pedagogical reason to use 491 28.5 28.2 15.2
  
• Not received adequate training 426 24.7 24.4 1
  3.2
• CMS are too time consuming 234 13.6 13.4 7.2
• Did not know available 131 7.6 7.5 4.1
• Uses other web-based tool 90 5.2 5.2 2.8
• CMS are not powerful enough 66 3.8 3.8 2.0
• CMS are too difficult to use 50 2.9 2.9 1.5
• CMS are unreliable 41 2.4 2.4 1.3
• Might/will use in future 32 1.9 1.8 1.0
• Unique response 26 1.5 1.5 0.8
• Not currently teaching 25 1.5 1.4 0.8

18
Sample Analysis of Survey Data

• Choice Count Answering Asked Total
• Ignorant of CMS 24 1.4 1.4 0.7
• Multiple responses 15 0.9 0.9 0.5
• Student concerns 15 0.9 0.9 0.5
• Difficult access 18 1.0 1.0 0.6
• New faculty 12 0.7 0.7 0.4
• Not specified 12 0.7 0.7 0.4
• Dislike WebCT 6 0.3 0.3 0.2
• Too expensive 3 0.2 0.2 0.1
• Misclassified 3 0.2 0.2 0.1
• Lack of support 2 0.1 0.1 0.1
• New category added after analysis

19
Most Important Factor Leading Faculty to CMS Use

• Training opportunities (24.9)
• Administrative requirement (22.9)
• Because of a specific teaching or pedagogical
  challenge (19.8)
• Recommendation from peers or others (18.4)

20
Purposes For Using a CMS

• Main reasons for using CMS
• To supplement or provide access to course
  materials
• To provide more interactivity between students
  and the teaching materials
• To provide more or more prompt feedback to
  students
• To increase instructor-student contact
• How instructors use CMS (multiple responses
  allowed)
• 90.6 to enhance their face-to-face instruction
• 43.8 for fully on-line instruction
• 43.8 for hybrid course application

21
Barriers to CMS Use

• Among non-users, the three most common reasons
  for not using a CMS were
• No current pedagogical need (28.5)
• Lack of adequate training in their use (24.7)
• Belief that CMS is too time consuming (13.6)

22
Change in CMS Usage

• 65.2 of respondents reported increasing their
  use of CMS over time
• Factors that contributed to increased usage
• Began to see increased uses for it in teaching
• Level of comfort with the technology increased
  and began to use it in new ways
• Received additional training
• Student requests or notice in student improvement
• 6.1 reported decreasing their use of CMS over
  time
• Factors that contributed to decreased usage
• CMS was too time consuming
• CMS was too inflexible
• CMS too difficult to use
• CMS unsuited to the teaching goals trying to
  achieve

23
Advantages and Disadvantages

• Major advantages of using a CMS
• It provides access to course materials to
  students (29.4)
• It organizes course materials (25.4)
• It provides convenient online testing environment
  (11.6)
• It provides course security (e.g., only enrolled
  students can view course materials) (11.5)
• Major disadvantages of using a CMS
• CMS are too difficult or time consuming to use
  (26.3)
• CMS are too inflexible or clunky to use (21.4)
• CMS are too limited in their functionality
  (12.6)
• CMS lack reliability (9.9)

24
CMS and Student Engagement

• 55.6 of user respondents felt that a CMS
  increased amount of contact with their students
• 63.5 of user respondents felt that use of CMS
  increased student engagement with the course
  materials
• 54.2 of respondents felt that use of CMS allowed
  for inclusion of more interactive activities in
  their class
• 67.6 of respondents felt that CMS allowed them
  to accommodate more diverse learning styles

25
Conclusions and Important Issues

• The USG expects an 80 increase in student
  enrollment during the next decade
• Technology increasingly will play a role in
  instruction in higher education
• Nearly half (46.3) of all USG faculty currently
  use a CMS in their instruction
• Of those using a CMS, nearly two-thirds have
  increased their usage over time

26
Conclusions and Important Issues

• Surveys such as this CMS survey can form a solid
  baseline metric for CMS use and implementation.
• Modified surveys will be deployed approximately
  every 18-24 months for longitudinal study
• Future surveys will increase focus on pedagogical
  aspects of using CMS for distance/hybrid courses
• Surveys of this type may also help institutions
  comply with SACS requirements to assess faculty
  use and proficiency with technology

27
Questions?
hilliard.gastfriend_at_usg.edu catherine.finnegan_at_usg
.edu www.alt.usg.edu