Analysis and Knowledge Extraction from Video

1
Analysis and Knowledge Extraction from Video
Audio
Rick Parent Jim Davis Raghu Machiraju Deleon Wang
Department of Computer and Information
Science Ohio State University
2
Overview
Streaming data from video audio
Motivation
Human operators large data sets
Problem
Focus on human behavior
Solution
Extract important events
Use multimodal approach
Security (real-time processing) Annotating
recorded video Processing archival material
Applications
3
Objectives
Detect and track people to extract audio-visual
events
Present graphical summaries to human operator via
secure web-based interface
Build prototype system
3 level system Person/action detection Sequential
long-term tracking Multi-modal identification
Incrementally constructs event model to focus
attention and resources to track and recognize
people across sequences
4
Person Detection and Activity Recognition(Jim
Davis)
Thermal-based image analysis and person detection
Framework for recognizing basic human activities
5
Sequential-frame tracking(Raghu Machiraju, Rick
Parent)
Monitor across sequences
Tack human figure poses
Capture appearance
Characterize motions
6
Robust Speaker Recognition(Deleon Wang)
Usable speech extraction from multiple speaker
audio
By tracking pitch and extracting voiced segments
7
Deliverables

• Demonstration subsystems
• Person detection
• Long-term tracking
• Speech recognition

6 mos review of basic work 12 months demo of
capabilities, summary report
8
Expenditures
6 Student-quarters of support over 12 months
2 Qtrs Person detection (Davis) 3 Qtrs Tracking
(Machiraju Parent) 1 Qtr Speech (Wang)