Dennis Moellman,

1
Video Analysis and Content Extraction (VACE) RD
Program Overview of Phase 2

• Dennis Moellman,
• VACE Program Manager

2
ARDAS Charter and Mission
Charter An Advanced Research and Development
Activity focused on Information Technology that
provides a world-class, multi-disciplinary
capability available to the Intelligence
Community and, subject to the concurrence of the
Secretary of Defense, the Information Security
Community. This activity is the Intelligence
Community's center for conducting advanced
research and development related to extracting
intelligence from, and providing security for,
information transmitted or manipulated by
electronic means. Mission Identify important
information technology problems of special or
unique interest, focus multi-disciplinary
expertise against these problems, support high
risk/high payoff research, coordinate with other
government entities, industry and academe and
assist the transfer of solutions to the applied
technology centers of other IC organizations.
3
What ARDA Does

• We originate and manage RD programs
• With fundamental impact on future operational
  needs and strategies
• That demand substantial, long-term venture
  investment to spur risk-taking
• That progress measurably toward mid-term and
  final goals
• That take many forms and employ many delivery
  vehicles

4
VACE Video Analysis and Content Extraction

• GOALS
• Robust person, vehicle, and text detection and
  recognition.
• Cross media content analysis and extraction
• Fully automatic video indexing based in image,
  text, and audio content.
• Efficient methods for representing video content.
• Event Detection, Recognition and Understanding.
• Low cost video corpus marking and preparation

• HIGHLIGHTS - Phase 1
• Primarily Focused on Core-Level Research
  Objectives some early investigations into Video
  Events
• Devised novel techniques for training up general
  object detectors with minimal annotated video
  data Applied technique to faces, vehicles and
  other common objects found in office environment
• Significantly improved tracking of multiple
  objects within video streams robust to the
  merging separating of objects, to varying rates
  of motion, to temporarily stationary objects and
  to occlusions
• Very encouraging, preliminary success in
  detecting and understanding single-thread video
  events as well as simple, multi-thread video
  events

• CHALLENGE PROBLEMS - Phase 2
• Significantly enhanced detection, recognition and
  tracking of people, faces and broad classes of
  other selected objects
• Object modeling, measurement and mensuration
• Multi-modal fusion
• Added emphasis on UAV Motion Imagery Problems
• Video Event Challenge Problems
• Foreign Broadcast News
• Formal and Informal Meetings
• Abstraction Inferencing from Surveillance Video
• UAV Motion Imagery
• Ground Reconnaissance Video

As of Nov 03
5
VACE Video Analysis and Content Extraction

• VACE Program Research Goals
• Robust person, vehicle, and text detection and
  recognition.
• Cross media content analysis and extraction
• Fully automatic video indexing based in image,
  text, and audio content.
• Efficient methods for representing video content.
• Event Detection, Recognition and Understanding.
• Low cost video corpus marking and preparation

6
VACE Video Analysis and Content Extraction

• Phase 1 Research Objectives
• Primary
• Object Detection -- Object Recognition
• Object Tracking -- Event Understanding
• Video Summary -- Multi-Modal Fusion
• Video Query by Example
• Secondary
• Multi-modal Video Mining -- Kinematics Analysis
• Motion Analysis -- Video Mensuration
• Integrity Analysis -- Model Reconstruction

7
VACE Program Contractors
8
VACE Video Analysis Content Extraction

• HIGHLIGHTS Phase 1
• Primarily Focused on Core-Level Research
  Objectives some early investigations into Video
  Events
• Devised novel techniques for training up general
  object detectors with minimal annotated video
  data Applied technique to faces, vehicles and
  other common objects found in office environment
• Significantly improved tracking of multiple
  objects within video streams robust to the
  merging separating of objects, to varying rates
  of motion, to temporarily stationary objects and
  to occlusions
• Very encouraging, preliminary success in
  detecting and understanding single-thread video
  events as well as simple, multi-thread video
  events

9
VACE Phase II Video Research Environment
10
VACE Phase II Challenge Problems

• Continue Investigation of Core Technology Issues
• Significantly enhanced detection, recognition and
  tracking of a broad classes of people, faces and
  objects
• Detailed object modeling, measurement and
  mensuration
• Multi-modal fusion
• NEW Video Event Challenge Problems
• Foreign Broadcast News
• Formal and Informal Meetings
• Abstraction Inferencing from Surveillance Video
• UAV Motion Imagery
• Ground Reconnaissance Video
• Military Exercises / Rocket Launches
• NEW UAV Video Specific Technical Challenges

11
VACE Phase 2 ARDAs Plan of Attack

• Envisioned as a high risk, long term RD Program
• Phase 1 Fall 2000 - Fall 2002
• Phase 2 Winter 2003 - Winter 2005
• Phase 3 Spring 2006 - Spring 2008
• VACE Program Executive Committee
• -- CIA -- NSA -- DIA -- NIMA -- DHS
  -- NIST -- ARDA
• Continue Investigation of Core Technology Issues
• Significantly enhanced detection, recognition and
  tracking of a broad classes of people, faces and
  objects
• Detailed object modeling, measurement and
  mensuration
• Multi-modal fusion
• NEW Video Event Challenge Problems
• Challenge 1 Foreign Broadcast News
• Challenge 2 Formal and Informal Meetings
• Challenge 3 Abstraction Inferencing from
  Surveillance Video
• Challenge 4 UAV Motion Imagery
• Challenge 5 Ground Reconnaissance Video
• NEW UAV Video Specific Technical Challenges

12
VACE Program Phase 2 Contractors
13
VACE Program Phase 2 ContractorsLate Fall 2003
Late Fall 2005
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
VACE Program Phase 2 ContractorsLate Fall 2003
Late Fall 2005
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
VACE Program Phase 2 ContractorsLate Fall 2003
Late Fall 2005
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)