Video Information Retrieval

1
Video Information Retrieval

• Mark Ruzomberka
• IST 497
• 11/07/02

2
Joke
3
Outline

• What is Video Information Retrieval (VIR) ?
• Reasons VIR is necessary
• Theoretical
• Where we are today
• Examples
• Problems
• Future Work
• Conclusion

4
What is Video Information Retrieval (VIR) ?

• Recognition technologies
• Image
• Voice
• Text transcripts
• Document retrieval technologies
• Topic segmentation
• Topic matching
• Text summarization
• Presentation Technologies
• Combine Recognition and retrieval technologies
• Result is an integrated application

5
VIR-Need, or Why do I care?

• Consider the task of trying to find a five
  minute video clip of interest in a library of
  1000 hour long tapes.
• Consider the go to the part where problem

6
What do people want from IR

• D-Lib Magazines asks
• What do People want from Information Retrieval?
• 8 Multimedia

7
Specificly, Reasons for Video IR

• Reading is slow compared to your potential for
  understanding information
• Humans think in pictures not words
• Reading is particularly slow on a computer screen
• Example Daydreaming while some one is talking
• Reading a page in a book and not remembering what
  it was about

8
VIR makes for quicker human understanding.

• Palm/Grafitti 25
• Hand Writing 35-40
• Typing 50-70
• Speaking 135-175
• Reading 200
• Listening 400 - 500
• Thinking 500

• Video IR allows for faster access to information

9
Theoretical

• Think of the Jetsons mail system
• You talk to the computer,
• Computer intelligently talks back to you

10
Where we are today

• Two of Video Information Retrieval System are
  currently available
• Type One- keyword/text based
• Type Two- Content based

11
Type One- keyword/text based

• DVR- basic expansion of image IR,
• not as interesting

12
Type Two- Content based
MSR Video Skimmer
Video Mail
Informedia
13
Example Video Mail

• University of Cambridge
• 1994-1996
• ATT
• 1999
• 2000-project ended

14
Video Mail Medusa network

• Medusa multimedia environment at Olivetti
  Research Ltd. In Cambridge
• It takes a modular approach unlike that of a pc
  or workstation
• Unified by a common interface to ATM network
• Devices plug directly into network and include
• Cameras
• Audio devices
• Networked frame buffers
• Processor farms
• Disk drives

15
Video Mail Medusa Network

• The network is the computer metaphor is used
• Solves storage and network speed problems
• Complicates expense problem

16
How it works-Overview
17
The Integrated Application

• narrow by sender,date, time

18
Video Mail Video Browser

• Content is now being viewed
• Keywords are flagged

19
Video Mail Video Browser

• In the latest version thumb-nailed pictures of
  key frames replace color coded line of the search
  keyword

20
Informedia

• The Informedia Digital Video Library Project
  automatically combines speech, image and natural
  language understanding to create a full-content
  searchable digital video library.

21
Informedia
22
Informedia human factor issues

• Interaction
• Motivation
• Effective usage modes
• Commercial compression
• VHS quality playback.
• Terabyte (1,000 gigabytes) of storage
• 1000 hours of video.

23
Problems

• Human understanding
• Spoken document retrieval
• Poor video browsers
• Expensive
• Slow access to data
• Large amounts of data

24
Microsoft Research (MSR) Video Skimmer
25
Microsoft Research (MSR) Video Skimmer

• Enhanced Browser Controls
• Time Compression
• Pause Removal
• Textual Indices
• TOC, Notes
• Visual Indices
• Shot Boundary Frames
• Timeline Markers
• Jump Control (Back/Next)

26
Problem Poor Content Based Video Browsers

• Current VCR model allows for poor navigation
• go the the part where they say problem

27
Problem Expensive

• Hard drive space expensive
• Video adds to problem
• High bandwidth needs are also expensive

• http//www.littletechshoppe.com/ns1625/winchest.ht
  ml

28
Problem Slow Access to Data

• Broadband still not available everywhere
• Availability doesnt mean acceptance
• Especially after dot com crash 2000

29
Problem Large Amounts of Data

• Current Systems use MPEG2
• Newer compression technologies
• MPEG 4-DIVX -DVD Quality
• Video consumes orders of magnitude more storage
  than text
• MPEG 7 is on horizon

30
Future Work ?

• Sky the limit ?
• Sci-Fi the limit ?
• Hard Drive Space, Bandwidth are current
  limitations.

31
Conclusion

• Not yet ready for prime time
• Storage and Network Costs decreasing
• Success is in day to day usage
• Slowly Becoming Mainstream E.x.Tivo
• Problems of real world tests
• Idiot proof
• ATM and Medusa arent mainstream

32
Papers

• Video Mail Retrieval Using Voice Report on
  Keyword.. - Jones, Foote, Jones.. (1994)
• What do people want from Information Retrieval?.
  Croft, Bruce W. D-Lib Magazine. (1995)
• Video Skimming for Quick Browsing based on Audio
  and Image.. - Smith, Kanade (1995)
• The VISION digital video library (context) -
  Gauch, Li et al. (1997)
• Informedia News-on-Demand Multimedia
  Information.. - Hauptmann, Witbrock (1997)
• M.G. Christel and D.J. Martin, "Information
  Visualization within a Digital Video Library", J.
  Intelligent Info. Systems 11(3), (1998), pp.
  235-257
• Browsing Digital Video. Li, Gupta, Sanocki et.
  Al.

33
Questions?
34
Joke?

• "There are 10 types of people in the world...
• those who understand binary and those who don't."