JMF Multimedia Networking for the rest of us

1
JMF Multimedia Networking for the rest of us

• Anwar M. Haneef
• Networks and Infrastructure Research Laboratory
• Motorola Labs

2
Agenda

• JMF - The UN folk who lend money ?
• JMF - Divine Inspiration
• JMF Architecture
• RTP Time Matters
• So why JMF _at_MotLabs ?
• Cerebration

3
JMF The UN folk who lend Money ?

• Nope, thats the IMF !
• JMF The Java Media Framework
• Multimedia Enabling Java
• Making the dream of a platform independent
  multimedia application a reality
• Standard Java API for display, capture and
  Synchronization of Time-Based Media

4
Java Media and Communication APIs

• Java 2D
• Java 3D
• Java Advanced Imaging API
• Java Speech
• Java Telephony
• Java Shared Data
• Java Sound
• Java Media Framework

5
JMF Types

• Pure Java Version
• Linux
• UNIX variants
• Performance Pack
• Windows 9x/Me/NT/2000
• Solaris

6
Brief History of Time

• JMF 1.0 API defined by Sun, Intel and SGI
• Finalized 9/97
• Supports playback only (from file or network)
• Implementations from Intel, Sun, RealNetworks
• JMF 2.0 API defined by Sun and IBM Haifa
• Supports capture, plug-in components
• Released for public review 12/98
• Finalized 2Q99

7
JMF 1.0 Features

• Developed by Sun
• Supported Media Types
• Audio WAV, AIFF, AU, GSM, MIDI, RMF
• Video AVI, MPEG, QuickTime, Vivo
• Supported Protocols
• File, FTP, HTTP, RTP (Receive only)
• High Performance
• Native codecs
• Native Video Renderers (DirectX, GDI, Xlib)
• Native Audio Renderer (JavaSound Engine)

8
JMF 1.1

• Jointly developed by Sun and IBM Haifa
• All-Java version enables cross-platform JMF
  support, fully compatible with JMF 1.0 API
• Supported Media Types
• File Formats WAV, AU, AIFF, MOV, VIV, AVI, RTP
• Audio Codecs PCM, ADPCM, Mu-Law, GSM, G.723
• Video Codecs Cinepak, H.263
• Released 12/98

9
JMF Implementations
10
JMF 2.0

• Jointly Developed by Sun and IBM Haifa
• API released for public review 12/98
• Implementation released 2Q99
• Features
• Media Capture
• Programmable Media Playback
• Plug-In Components

11
Agenda

• JMF - The UN folk who lend money ?
• JMF - Divine Inspiration
• JMF Architecture
• RTP Time Matters
• So why JMF _at_MotLabs ?
• Cerebration

12
JMF Divine Inspiration ?

• Not quite.
• Models real-world multimedia tools

13
Agenda

• JMF - The UN folk who lend money ?
• JMF - Divine Inspiration
• JMF Architecture
• RTP Time Matters
• So why JMF _at_MotLabs ?
• Cerebration

14
JMF - Architecture

• Understanding JMF Requires Time
• Virtually everything revolves around the Clock
  interface
• Types of Time
• TimeBase Time
• Media Time
• GMT is a TimeBase Time

15
The Controller

• Controller extends the Clock interface
• Controllers enhance the clock interface by
  dividing the stopped state into five stages

16
The Controller

• Some transitions are done asynchronously and some
  synchronously
• State transitions release Events

17
Sample Event

• // the EndOfMediaEvent indicates we've run out
  of dataif (event instanceof EndOfMediaEvent)
  // rewind (i.e. set media time to
  0 player.setMediaTime(new Time(0))

18
The 3 Musketeers

• The fun guys who love to Play
• DataSource
• Player
• Manager

19
DataSource

• Retrieve data, place it in buffers and stream
  these buffers to client applications. Their main
  responsibility is to convert dedicated audio and
  video file formats into industry standard formats
  
• Types of DataSources
• PushDataSource TV and Radio broadcast
• PullDataSource HTTP Requests

20
Player

• MediaHandlers retrieve buffers from a DataSource,
  massage the data and transport the resultant
  buffers to a multimedia device
• The most common type of MediaHandler is the
  Player, a MediaHandler that also implements the
  controller interface
• Player API

21
Using the Player

• public void start()
• // when the Web page is loaded, immediately
• // start playbackif (player !
  null) player.start()
• //More control can be built in by listening to
• //Events transmitted by the player

22
Processor

• A type of Player and has similar presentation
  controls
• Besides playing the media, it can also output
  media through a DataSource
• Has two additional states
• Configuration demux input stream and access
  information about the media format
• Configured once media stream determined

23
Player and Processor

• Player API

• Processor API
• Capture
• Programmable Playback
• Supplies output through DataSource

24
Managers who wants them anyway ? -)

• Manager not only finds the appropriate DataSource
  for your content, but it also constructs a
  Player, connects the Player to the DataSource and
  returns a Player reference to your application
• Only information needed by a Manager is the
  location of the media, represented by a
  MediaLocator

25
Managers - Types

• Good
• Bad
• Ugly

Just kidding ! ?
26
Managers - Types

• Manager used to create Players, Processors,
  DataSources and DataSinks
• PacakgeManager maintains registry of packages
  with JMF and other customized classes such as
  custom Players
• CaptureDeviceManager maintains a registry of
  available capture devices
• PlugInManager - maintains a registry of available
  JMF plug-in processing components

27
Sample Usage

• MediaLocator mrl new MediaLocator(rtp//224.144
  .251.10449150/audio/1)
• // create a Player with a Media Locator
• player Manager.createPlayer(mrl)

28
JMF 2.0 Architecture

• Programmable at two levels
• High Level - Player, Processor
• Low Level - Plug-Ins

29
JMF 2.0 Architecture

• Plug-In Components API
• Parsers, Codecs, Effects, Renderers, Multiplexers
• Implemented in software or hardware

30
Agenda

• JMF - The UN folk who lend money ?
• JMF - Divine Inspiration
• JMF Architecture
• RTP Time Matters
• So why JMF _at_MotLabs ?
• Cerebration

31
RTP Time Matters

• Real-time Transport Protocol
• Internet Standard for transporting real-time data
• Used by streaming audio applications such as
  RealPlayer
• Uses unreliable UDP to transmit packets

32
RTP - Features

• Supports unicast and multicast delivery
• Source and payload type identification
• Sequencing, Timing, and Synchronization
• Source merging Multiple contributing sources for
  a combined stream produced by an RTP mixer.
  32-bit Synchronizing source (SSRC) id.
• Stream translation High-speed to low speed

33
RTP Does NOT Provide

• Reliable data delivery
• Quality of Service guarantees
• Resource reservations
• Delivery of encryption key to participant

34
RTP and associate RTCP

• RTCP Real-time Transport Control Protocol
• Convey information about participants
• Convey information about relationships among
  sessions
• Monitor application performanceÞ Feedback on
  quality of data
• Automatically adjusts overhead
• (Report frequency based on participant count)

35
JMF and RTP

• JMF provides APIs to develop applications that
  support RTP and RTCP
• Seamless integration with the standard JMF APIs
  mentioned earlier

36
Agenda

• JMF - The UN folk who lend money ?
• JMF - Divine Inspiration
• JMF Architecture
• RTP Time Matters
• So why JMF _at_MotLabs ?
• Cerebration

37
So why JMF_at_labs.mot.com ?

• Obvious advantages of a complete Java solution
• Greater Flexibility
• Codecs can be plugged in as needed and could be
  provided as Codec Services served off the network
• More control and tighter integration with the
  COLLAGE solution

38
Agenda

• JMF - The UN folk who lend money ?
• JMF - Divine Inspiration
• JMF Architecture
• RTP Time Matters
• So why JMF _at_MotLabs ?
• Cerebration

39
Cerebration

• Lets brainstorm
• Let those creative juices flow after all, its
  more Liquid Media !

40
Thank You !!

• Anwar M. Haneef
• a.m.haneef_at_ieee.org
• http//www-unix.ecs.umass.edu/ahaneef