Enhancing the Classroom Learning Experience with Web Lectures

1
Enhancing the Classroom Learning Experience with
Web Lectures

• Jason Day Jim Foley
• dayja, foley_at_cc.gatech.edu
• Past Contributor Remco Groeneweg

2
Introduction

• By taking the lecture out of the class onto the
  web, time in class can be spent in more engaging
  ways

3
Motivation

• Use class time for more learning by doing, less
  learning by listening
• A lot of lecture material to be
  coveredlearning-by-doing activities take a lot
  of time

4
Research Areas

• Educational
• Educational framework for web lectures
  meaningful in-class learning activities
• Ways to motivate students to watch web lectures
• Which in-class activities work well with web
  lectures?
• Technological Affordances
• Ways to make web lecture watching more engaging
  (delivery mechanism, interactive elements, etc.)

5
Technology

• Web lecture studio-recorded, condensed lecture

6
Web Lecture Production Workflow
PowerPoint slides
Streamed video HTML
web lecture
Microsoft PowerPoint
Microsoft Producer
Streaming Server
Web Browser
Publish
Import
Narrate slides with audio/video
View lecture
7
Web Lecture Studio Design
Total equipment cost 3000
8
Pedagogy

• Accepted educational approaches as inspiration
  for the design of in-class learning activities
• Learning activities stimulate the social process
  of articulation and reflection on shared public
  artifacts
• Learning activities grounded in real-world
  examples or anchored in group-project activities

9
Related Work

• Tutored Video Instruction _at_ Stanford (Gibbons et
  al. 1977)
• eTeach _at_ UWisconsin (Moses, 2002)
• iCampus _at_ MIT
• Internet Learning Environments _at_ Edith Cowan
  University (Oliver, 2001)
• HWebs _at_ GT (Collard et al. 2002)
• eClass _at_ GT (Abowd et al. 1999)

10
Research Context

• CS/Psych 4750 Senior elective course on Human
  Computer Interaction
• Typically 25-35 students
• Semester-long UI design project
• 3 to 5-person teams
• Gather requirements, prototype, build, test
• Midterm, final, homeworks

11
Formative Evaluation, Fall 2003

• Student feedback guided us to optimal level of
  web lecture production quality
• Notable improvements
• Audio quality
• Animate slides to help viewer focus
• Include body and hands in video
• Teleprompter and slide advancement

12
Pilot Study, Spring 2004

• 35 students enrolled
• 27 class meetings (down from 30)
• 17 lectures (down from 25)
• 13 web lectures, totaling 277 minutes (about 3.5
  80-minute class meetings)
• 7 new constructivist-inspired learning activities

13
Example Class Activities

• Project anchored
• Requirements Gathering presentations
• Prototype poster session (old)
• Evaluation Plan presentations
• Cognitive Walkthrough presentations
• Critiquing
• Professor critiquing
• Small group critiques
• UI Hall of Fame/Shame
• Use and critique of physical interfaces

14
Evaluation Methods

• Four surveys
• Survey 1 Second week of class
• Survey 2 Mid-Semester
• Survey 3 Group vs. Individual
• Survey 4 Final
• Three focus groups
• Focus group 1 Second week, 5 students
• Focus group 2 Sixth week, 5 students
• Focus group 3 Week before finals, 4 students
• Three observations

15
Pilot Study Results

• Satisfactory web lectures can be created with
  modest faculty time and simple, inexpensive
  equipment
• Students desire some form of explicit motivation
  to watch web lectures
• Students prefer watching web lectures
  individually over group watching
• Students find class activities educational and
  enjoyable

16
Pilot Study Results (2)

• Slight preference for this course format over
  traditional lecture format (avg. 3.36 out of 5)
• Web lectures not as useful for exam review as
  students predicted on first and second surveys
• Covered same amount of material and added 7
  learning activitieswith 3 fewer class meetings
• Informal judgment that educational outcomes were
  same as in Fall 03

17
Quasi-experimental Study, Spring 2005

• Two sections of 4750
• One taught traditionally (n18)
• One using our web lecture / class activity format
  (n29)
• Hypothesis
• Web lecture intervention as good or better based
  on educational outcomes and subjective attitudes

18
Study Design

• Matched two sections on
• Instructor teaching the course
• Topics covered
• Lecture slides used in class or integrated into
  web lecture
• Assigned reading
• LHWs, homeworks, and semester project
• Mid-term and final exams
• Time on task
• Control section time spent in class
• Experimental section time spent in class plus
  total running time of assigned web lectures
• Counterbalanced and blind grading
• Exception Projects were graded together with
  same grading criteria

19
Lecture Homeworks (LHWs)

• Short homeworks associated with each web lecture
  / lecture
• Blind to instructor
• Served as
• explicit motivation (15 LHWs worth 1 each)
• discussion guide for next class meeting

20
Study Details Experimental Section

• 21 class meetings (compared to 28)
• 3 lectures (compared to 25)
• 27 web lectures, totaling 537 minutes (about 7
  80-minute class meetings)
• 13 QA / discussion / learning activity classes

21
Preliminary Results Educational Outcomes

• So far, experimental section average grades have
  been higher for every graded assignment and exam

22
Preliminary Results Educational Outcomes (2)

• Experimental section average LHW grades across
  all LHWs is significantly (plt0.01) higher than
  control section.

23
Preliminary Results Educational Outcomes (3)

• Experimental section average project grades are
  significantly higher for Phase 1 (plt1.8E-7) and
  Phase 3 (plt0.03) Phase 2 grades were higher, but
  not statistically significant (p0.06).

24
Preliminary Results Subjective Attitudes

• If you compare the new course format of web
  lectures and in-class activities to the
  traditional in-class lecturing format, how would
  you rate the new course format?
• Average Responses
• Interim Survey after 11 web lectures 3.11
• Midway Survey after 25 web lectures 3.61
• Statistically significant increase (plt0.04)

25
Preliminary Results Subjective Attitudes (2)

• Both sections reported LHWs helped them focus on
  and learn the material presented
• Both sections came in with very positive
  attitudes towards the relevance of HCI to
  education and career
• Experimental section has slightly increased
• Control section has slightly decreased

26
Future Work

• Look for correlation between survey responses and
  digital library use
• Implement and evaluate technological affordances
  of web lectures
• Question submission mechanism, built-in
  interactive learning activities, discussion
  forum, FAQs, variable playback speed, etc.
• Development new in-class activities, current
  activities will be further improved
• Use of this class format in other courses

27
The End

• Web lectures available at http//hcc.cc.gatech.ed
  u/videolectures
• Questions/comments
• Stop by the recording studio TSRB 377
• Email dayja_at_cc

28
Web Lecture Host Infrastructure
Streaming Server Windows Media Services 9 Series
Web Server Apache/1.3.29
TCP/IP Internet
HTTP
MMS
Client Machine Web browser Media Player
29
Production Quality

• How would you rate the production quality (i.e.
  audio/video quality) of the web lectures?

30
Perceived Usefulness of In-class Activities
31
Perceived Usefulness of Other Possible
Technological Affordances
32
Human-Centered Computing Education Digital Library

• Ed Clarkson, Jason Day, Jim Foley, Aarjav Trivedi

33
IntroductionHuman-Centered Computing (HCC)
Education Digital Library

• Support the new GT College of Computing Ph.D.
  program in Human-Centered Computing
• Expose Georgia Tech as a leader for HCI/HCC
  educational content
• Leverages one of the largest HCI faculty in the
  world!
• Enhance the overall learning experience via rich
  set of instructional material

34
A Question

• What resources are there for people who want to
  learn or teach HCI/HCC?

35
Existing Answers

• HCI/HCC-specific
• HCI Bibliography (Gary Perlman)
• SIGHCI Teaching Resources
• Colleagues
• General
• Google
• Educational Digital Libraries (e.g.,
  )

36
Common Issues

• HCI-specific
• Materials not specifically for education
• Not active
• Not universal
• General
• Wide target audiences
• Problems with linking
• Browsing?
• Quality control

37
Requirements Gathering

• Focus groups
• Online Surveys
• HCI Education Workshop (CHI 05)
• http//hcc.cc.gatech.edu/chi2005workshop.htm

38
Derived Requirements

• High-quality, locally-stored content
• Syllabi, lectures, tests exams, homework and
  project assignments, videos
• Quality assurance
• Filtering over reviewing
• Controlled reviews over free-for-all

39
Derived Requirements (contd)

• Browse vs. search
• I would like to see the detailed level of
  granularity because many instructors of HCI are
  thrown into the task rather then being trained in
  it.
• Scalability/sustainability

40
Prototype Implementation

• Demo the live site

41
Future Work, Research Goals and Questions

• Version 2.0 in progress
• Dynamic, database-backed
• What can aggregate user behavior over time tell
  us?
• Web lecture
• Automated hierarchy reorganization

42
Future Work, Research Goals and Questions

• Scalability/sustainability
• What mechanisms (both organizational and
  technological) are needed for a sustainable
  educational knowledge repository?
• How well does Wikipedia model transfer?
• How does the taxonomic organizational scheme
  scale? What are alternatives?

43
Questions?