Steven Goldfarb for Jeremy Herr

1
WLCD A Web Lecture Capture System with Robotic
Speaker Tracking

• Steven Goldfarb for Jeremy Herr
• CHEP
• Mumbai, India - 15 Feb 2006

2
The Web Lecture Archive Project

• Partnership
• University of Michigan
• UM ATLAS Collaboratory Project
• Department of Physics
• Media Union
• CERN
• Technical Acedemic Training
• Summer Student Program
• IT
• American Physical Society
• Primary Goals
• implement an electronic archival system for
  slide-based presentations on the Internet
• assess the ability of the system to improve upon
  and complement existing archival methods
• determine the resources required to install and
  maintain the system as a standard archival
  service.

3
Current Recording Activities

• Web Lecture Recording
• ATLAS LHC In General
• Physics Software Tutorials
• ATLAS Week plenary sessions
• CERN Training Seminars, Tutorials
• American Physical Society
• Opportunities for Physics in Biology 2002, 2004
• April Meeting 2004
• Fermilab
• Software Tutorials
• University of Michigan
• Saturday Morning Physics
• Medical School Symposium on Clinical Research
  2004
• Harvard
• International Conference for Systems Biology 2005

4
Development of Automated Recording Technology

• Motivations Driving Automation
• Simultaneous Recording of Multiple Lectures
• Limited Manpower
• Need for Instant Publication
• Components of Lecture Archiving Requiring
  Automation
• Video Capture and Compression
• Audio Recording
• Slide Capture and Synchronization
• Construction, Publication of Electronic Lecture

5
Development of Automated Recording Technology (2)

• Brief History of Lecture Recording Automation
• 1999 Used Sync-O-Matic 3000 Software to Record
  CERN Summer School Lectures
• Sync-O-Matic completely automatic if
• Installed on presentation computer
• Slides in Powerpoint
• But
• Everybody wrote plastic transparencies back then!
• And access to presentation computers not always
  possible
• So
• Had to record timing by hand, process lectures
  after the talk.
• 2000 Extra video camera was pointed at the
  screen
• timing information was collected manually
  afterward
• 2001 Giosue Vitaglione wrote CarpePPT
• Same idea as Sync-O-Matic, but producing Lecture
  Objects (non-proorietary)
• 2002 Jake Bourjaily wrote CarpeLecture
• Extension, allowing PDF and integrated with CDS
  Agenda
• Now Web Lecture Capture Device
• VGA splitter connected to signal coming from
  presentation computer
• high-resolution capture card is used with
  change-detection software

6
Recall the Lecture Object (Talk before Coffee)

• Lecture Object
• Standardize Storage of Web Lecture Ingredients,
  Recipe
• Archive media in standard formats (mpeg-4, jpeg,
  etc.)
• XML extension to SMIL (Synchronized Multimedia
  Integration Language)
• Describes slide timing, media location, other
  information
• SMIL W3C Standard
• Add metadata with RDF (Resource Description
  Framework)
• RDF W3C Standard
• Standard Procedure for WLAP

Transformations
meta data
XML Description
mpeg-4
jpeg
Lecture Object
7
Lecture Object - Definition

• Organized Directory Structure and Naming System

• Contents
• Audio/Video media MPEG-4 video file
• Slides JPEG images of everything displayed on
  lecturers screen
• Supporting Documents original PowerPoint file,
  animations, etc.
• Log Files describing technical history of and
  changes to media files
• XML file
• a simple format similar to SMIL, following
  commonly used international standards and
  customized to effectively describe a typical
  lecture.
• contains metadata fields using subsets of Dublin
  Core and IEEE Learning Object Metadata,
  describing media data
• timing and synchronization information

• Future Directions
• Allow more media streams (animations, data
  displays, chalkboards)
• Allow annotations, notes, links to other
  resources
• Transformations to more viewing objects
  Quicktime, Media Player, iPod, etc.

8
Lecture Object - Example

• Example of Lecture Object using current standard

9
Robotic Camera Tracking

• System Requirements
• Portable
• Robust
• Affordable
• Runs without expert intervention
• Little setup or calibration
• Survey of Existing Technologies

10
Robotic Camera Tracking Current System

• Current solution active IR system using
  dual-cameras and infrared necklace
• Target wears necklace made of IR LEDs
• Invisible to human eye, but very bright in IR
• Two cameras mounted together
• IR camera tracks target
• visible-light camera provides color video
• This system satisfies our criteria
• Portable sits on a cart
• Robust simple design makes it very robust
• Affordable currently under 4,000 USD
• No expert intervention start it and it just
  works
• Little setup almost no calibration required
• Accurate to within centimeters

11
Robotic Camera Tracking Future Development

• Remaining Challenges
• Rooms with windows on bright days, excessive
  sunlight can increase background noise and
  illuminate reflective objects
• Other IR sources
• Tablet PCs use bright IR pens
• incandescent light fixtures give off a lot of IR
• transparency projectors give off a lot of IR
• Improvements - Next system machine-vision camera
• Stationary, wide-angle camera
• Frame rate, exposure time, timing are all
  controllable
• The necklace strobes
• identifies necklace uniquely to the machine
  vision camera
• saves battery life
• We are building and testing this system now
• This should greatly improve reliability in
  adverse conditions

12
Technology Transfer

• March 2005
• Provisional patent filed, describing the current
  dual-camera with active IR necklace.
• October 2005
• University of Michigan researchers filed 287
  invention disclosures in 2005
• Eight of these were selected to be featured at
  the 2005 Tech Transfer Inventors Reception, as
  the most commercially promising and
  world-changing.
• Our tracking system was one of these eight.
• March 2006
• Permanent patent will be filed.

13
Applications

• Plans for Using the WLCD
• Enable automated large-scale recording, streaming
  and archiving of university courses
• Record and disseminate multiple parallel sessions
  at large professional society meetings
• Facilitate the preparation of materials for
  distance learning
• Generate supplemental instructional content for
  outside-of-class review
• Provide services for recording of sample lectures
  given by faculty candidates
• Recording of ATLAS Plenary Sessions
• Many more ...

14
Hot Item U-M Campus-Wide Classroom Recording

• Now - Physics 140 test
• Big impact over 500 students each semester,
  including pre-medical, engineers, science majors.
• Recording with current tracking system will begin
  March 2005
• Two 1-hour lectures per week, each available
  through U-M CourseTools within hours, and also
  podcast to students iPods.
• Chalkboard work will be captured.
• Entire system fits on one cart.
• Fall 2006 Pilot Project begins
• Project has support of the Provost and many
  academic units
• Three different classes will be recorded
• Student usage will be analyzed through surveys
  and log monitoring by education experts in the
  Center for Research on Learning and Teaching
  (CRLT).
• Project will continue through Summer 2006

15
Example Videos

• Demo video made with tracking system in May 2005
• Tracking camera video of Jen McCormick made in
  December 2005
• Web Lecture for iPod - David Gerdes, UM Physics
  Professor