Java Media Framework

1
Java Media Framework

• Matthew Mastracci

2
Overview

• What is JMF?
• Players
• Capturing
• Plugins
• Codecs/Effects
• Mux/Demux

3
What Is JMF?

• A method for working with time-based data in Java
• Acquisition, processing and delivery of media
• Handles real-time or stored audio and video

4
Raw Video Capture
5
Video Stream Playback
6
Audio Stream Playback
7
JMF Data Model
Controls
Duration
Manages
Datasource
SourceStream
Control
8
Player

• Player object allows easy embedding of content
• Create widget and catch realization events
• Heavy-weight widget

9
Player contd

• public void init()
• setLayout( new BorderLayout() )
• String mediaFile getParameter( "FILE" )
• try
• URL mediaURL new URL( getDocumentBase(),
  mediaFile )
• player Manager.createPlayer( mediaURL )
• player.addControllerListener( this )
• catch (Exception e)
• System.err.println( "Got exception " e )

10
Player contd

• public synchronized void controllerUpdate(
  ControllerEvent event )
• if ( event instanceof RealizeCompleteEvent )
• Component comp
• if ( (comp player.getVisualComponent()) !
  null )
• add ( "Center", comp )
• if ( (comp player.getControlPanelComponent())
  ! null )
• add ( "South", comp )
• validate()

Add the control panel to the bottom of the dialog
Add the content to the center of the dialog
11
Player contd

• ltAPPLET CODEPlayerApplet WIDTH320 HEIGHT300gt
• ltPARAM NAMEFILE VALUE"sprk.mpeg"gt
• lt/APPLETgt

12
Player contd

• ltAPPLET CODEPlayerApplet WIDTH320 HEIGHT20gt
• ltPARAM NAMEFILE VALUE"etr.mp3"gt
• lt/APPLETgt

13
Capturing

• 1. Query the CaptureDeviceManager
• 2. Get a CaptureDeviceInfo object for the device
• 3. Get a MediaLocator to create a DataSource
• 4. Create a Player or Processor using the
  DataSource
• 5. Start the Player or Processor

14
Capturing for Display

• One step use a player and a content specifier
  URL
• Example vfw//0, rtp//ip-address/type/ttl

15
Plugins

• Plugins can be dynamically loaded in system
• Add functionality to the system
• Codecs, Effects, Mux, Demux, Renderers
• Media handlers

16
Plugins Interface Hierarchy
Plugin
Codec
Demux
Mux
Renderer
VideoRenderer
Effect
17
Plugin Manager

• Use addPlugin() to add a new plug-in to the list
• commit() updates changes to the plug-in list on
  disk
• Register with the manager at install-time

18
Codecs

• Codec interface extends Plugin
• Codecs have input and output, modifying the
  content

Codec A
Codec B
Data 1 Format A
Data 2 Format B
Data 3 Format C
19
Effects

• Effect interface extends Codec
• Input format of Effect is same as output

Effect A
Effect B
Data 1 Format A
Data 2 Format A
Data 3 Format A
20
Multiplexer

• Merges multiple streams to a single stream

Data 1A Audio
Data 1 Compound
Mux
Data 1B Video
21
Demultiplexer

• Takes a single buffer and outputs to many buffers

Data 1A Audio
Demux
Data 1 Compound
Data 1B Video
22
MediaHandler Demux

• Outputs multiple stream formats

Audio stream
Video stream
23
Applications of JMF

• Teleconferencing
• Multimedia editing/sequencing
• Streaming multimedia web content

24
Problems with JMF

• JMF 2.0 is still beta
• Crashes occasionally
• Can lock up video capture devices
• API well documented, but usage and examples not

25
Conclusions

• JMF is highly flexible
• Shows a lot of promise
• Major stability and documentation issues
• Video For Windows (VFW) support adds great value
  (no need for native drivers)
• Currently difficult to use to create a complex
  program

26
Issues for Discussion

• Can JMF help propel Java into the field of
  multimedia display and editing?
• Will Sun develop a Linux-native "performance
  pack" for JMF, even though Linux competes with
  Solaris?
• What changes need to be made to the current beta
  version of JMF to make it more usable/stable when
  it becomes a release?