http://www.free-clip-art.com. Taxonomy o

1
Asynchronous Online Tutoring Using
Multimedia to Teach Difficult Concepts

• Presented at the Seventh Sloan-C
• International Conference on Online Learning,
• November 16-18, 2001
• Prof. Richard Larson and Laura Koller
• Center for Advanced Educational Services
• Massachusetts Institute of Technology
• 77 Massachusetts Ave.
• Cambridge, MA 02139

2
Why PIVoT?

• Start with a brilliant educator
• Broadcast video Help Sessions over MIT cable TV
• Convert TV help to asynchronous, web-based
  virtual office hours

3
Motivation

• Newtonian Physics most difficult subject for MIT
  freshmen
• In a lecture hall with 100s of other students,
  how much real interaction between student and
  professor?
• Wouldnt each student like to have the professor
  as a tutor, 24/7, anywhere?

4
Revising our Thinking about Teaching and Learning

• Teaching Learning
• Teacher Mentor or Coach
• Student Learner
• Synchronous Asynchronous
• Passive Active
• Linear Nonlinear
• Scheduled On-demand
• Teaching material Accomplishing a Goal

5
An Intelligent Matching of Pedagogy and
Technology to Content and Learners
Specific Knowledge Domain
Pedagogy
College 18-26 yr.olds
Technology
Post- graduate
K-12
6
Fundamental Hypothesis

• PIVoT can increase the equivalent face-to-face
  contact between learner and mentor by an order of
  magnitude or more.

7
Related Literature
http//www.free-clip-art.com
8
Taxonomy of Scientifically-Oriented Educational
Websites Nachmias and Turi, 2001

• Descriptive dimension (e.g., target population,
  site developers, language)
• Pedagogical dimension
• (instructional model, instructional means,
  cognitive demands)
• Representational dimension (representational
  structure and means, navigation tools)
• Communication dimension (links configuration,
  distant learning modes)
• Scientific content dimension (content level,
  visualization means, historical background)

9
Descriptive dimension (target population, site
developers, language)College freshmen taking
Newtonian physics, CECI, web-based languages with
database
10
Pedagogical dimension (instructional model,
instructional means, cognitive demands) -
Nonlinear student-directed inquiry
into various media - Lecture, demo, problem
solutions, concept discussions,
simulations, tests - Reflection and
integration of physical world and math world
depictions
11
Representational dimension (representational
structure and means, navigation tools) -
Streaming video of lectures and problem
sessions - Textbook - Tests -
Animations - Various search options (key word,
syllabus)
12
Communication dimension (links configuration,
distant learning modes) - Links to animations
and various key words - No difference between
on-campus asynchronous learning and the
distance learning option
13
Development of PIVoT

• 1M in grants
• Start April 1998
• Design features acquire content build
• Beta release, Fall 1999
• Evaluate beta, Spring 2000
• Final deployment Fall 2000
• Final evaluation, Spring 2001

14
Spring 2000 PIVoT Evaluation Results (Prof.
Lewin taught the course)
According to a student survey (45 of registered
users responded)

• 98 registered to use it
• 90 had a positive experience
• 87 accessed it from living group
• 52 used it 1-10 times 48, 11-2 times

15
Spring 2000 Evaluation Results (contd.)
Student survey says

• 84 watched help session videos
• 83 worked on problem sets
• 78 watched lectures
• 64 used it to study for exams
• 63 used it to learn concepts
• 52 watched exam reviews

16
Some student comments...

• A lot of people think it is the only reason they
  are passing 8.01, because its a tutor available
  24 hours a day.
• PIVoT is what makes MIT, MIT.
• Ive never spoken to a fellow freshman who
  hasnt used and benefited from PIVoT.
• Students are very thankful for the existence of
  PIVoT.
• ...even more helpful than lectures because it
  simulates a one-on-one learning environment.

17
Spring 2001 Evaluation (Prof. Lewin did not
teach the course)

• PIVoT was used in entry-level physics classes
  during the Fall 2000 semester at
• MIT
• RPI
• Wellesley

18
MIT
PIVoT was used in different ways

• Traditional large lecture/recitation format
• Voluntary educational supplement

RPI

• Studio physics format
• Required for problem set homework

Wellesley

• Small lectures
• Required for problem set homework

19
The Study had Two Main Goals

• To learn about PIVoT functionality, student
  usage patterns, and student attitudes toward
  PIVoT in different educational contexts
• To see if the use of PIVoT enhanced conceptual
  learning

20
Methodologies

• A student survey was administered to registered
  PIVoT users at all three campuses.
• Data were collected and analyzed at RPI, where a
  controlled educational experiment was conducted
  
• four out of nine entry-level physics
  classes used PIVoT, while five did not.

21
Evaluation Instruments

• Two pre- and post- diagnostic tests were
  administered to assess students improvement in
  conceptual learning
• Force Concept Inventory (FCI)
• assesses understanding of basic Newtonian
  physics concepts
• most widely used physics instrument
• Force Motion Conceptual Examination (FMCE)
• assesses conceptual learning gains

22
Findings

• On the FCI, PIVoT users had significantly higher
  gain scores than non-users (data on following 2
  slides).
• In regard to the FMCE gain scores, there were few
  differences between the two groups. This may be
  because the FCI tests topics that PIVoT covers,
  but the FMCE does not.

23
Table 1. RPI Class Scores by PIVoT Use
PIVoT Non-
Users Users Total Means FCI Pre-Test 53
50 52 FMCE Pre-Test 42 43 42 FCI
Post-Test 68 56 63 FMCE Post-Test
71 67 70 FCI Post-Pre Difference
14 7 11 FMCE Post-Pre Difference
28 25 28 FCI Class Gain 31 13
23 FMCE Class Gain 50 43 48 FCI
Indiv, Student Gain 33 -1 19 FMCE
Indiv. Student Gain 49 42 46 (T-test
level of statistical significance plt.001,
plt.01, plt.05)
24
Table 2. RPI Class Scores by PIVoT Use
PIVoT Non- Users Users
Total Exam 1 80 79 79 Exam 2 75 74
75 Final Exam 76 73 75 Homework 84 84
84 In-Class Activities 96 97 96 Quizzes
95 92 93 Final Numerical Grade Score 83
82 83 Percent Distribution A (91-100) 28
33 31 B (81-90) 42 36 39 C (71-80)
21 17 19 D (61-70) 6 9
8 F (60 or lower) 3 4 4
25
PIVoT Evaluation Report

• Available online at
• http//caes.mit.edu/research/pivot/
• PIVOT_report1.pdf

26
PIVoT Architecture
Web server
Client
Video server
Database server
information flow
27
PIVoT Navigation
Multiple nonlinear navigation modes

• Free text search
• Keyword index
• Topic tree
• Lectures
• Textbook

28
PIVoT Features

• 35 Lectures
• 20 hrs. Help Sessions
• 600-page Textbook
• FAQs Answers
• Practice Problems
• Java Simulations
• Discussion Board

29
Helping Signposts for Students
30
Signposts PIVoTs Personal Tutor

• Follows learner as she acts as spelunker through
  the web site
• Makes suggestions based on key words and topics
  being studied

31
PIVoT Web Accessibility

• Research focus
• Making complex scientific web sites accessible to
  deaf and blind users
• Methods
• video captions
• audio descriptions
• overall web site design guidelines
• Collaboration with WGBH-NCAM
• Funded by NSF and Mitsubishi

32
More information on the Access to PIVoT Project

• MIT
• http//caes.mit.edu/research/
• access_pivot/index.html
• WGBH National Center for Accessible Media
• http//ncam.wgbh.org/webaccess/pivot/

33
Selected References

• Integrating Multiple Teaching Methods into a
  General Chemistry Classroom, J. S. Francisco, G.
  Nicoll and M. Trautmann, J. Chem. Education, 75
  (2) 1998 210-213.
• Taxonomy of Scientifically Oriented Educational
  Websites, R. Nachmias, I. Tuvi, J. Science
  Education and Technology, 10 (1) 2001 93-104.
• Using the Internet to Enhance Student
  Understanding of Science The Knowledge
  Integration Environment, M. C. Linn, P. Bell, S.
  Hsi, Interactive Learning Environments, 6 (1)
  1998 4-38.
• Lifelong Science Learning A Longitudinal Case
  Study, M. C. Linn and B. S. Eylon, Proc. Of
  CogSci96 (pp. 597-602), 1996 Lawrence Erlbaum
  Associates.
• Creating Lifelong Science Learners What Models
  Form a Firm Foundation? M. C. Linn and L.
  Muilenburg, Educational Researcher 25 (5) 1996
  18-24.
• "A Three-School Comparative Analysis of Student
  Usage Patterns and Attitudes toward PIVoT" by Dr.
  Alberta Lipson, MIT Teaching and Learning
  Laboratory, July 2001.

34
Selected References, contd

• Devlin, Maureen, Richard Larson and Joel Meyerson
  (eds.), The Internet and the University 2000
  Forum, EDUCAUSE, 2001 Boulder, CO.
• Larson, Richard C. and Joel W. Meyerson, Weve
  Got the Steam Locomotive Now Lets Design the
  Railroad Cars! Preface in The Internet and the
  University 2000 Forum. 2001
• Larson, Richard C. and Glenn P. Strehle,
  Edu-Tech Whats a President to Do? An
  invited book chapter in Technology Enhanced
  Learning Opportunities for Change, edited by
  Paul Goodman of Carnegie Mellon University and
  produced by Lawrence Erlbaum publishers, 2001.

35
URLs and Contact Info
To request a guest account, go to the PIVoT web
site http//curricula2.mit.edu/pivot/ MIT
Center for Advanced Educational Services
http//caes.mit.edu/ Prof. Richard Larson,
Principal Investigator rclarson_at_mit.edu Lau
ra Koller, Project Manager
lkoller_at_ceci.mit.edu