Embedded Streaming Media with GStreamer and BeagleBoard

1
Embedded Streaming Media with GStreamerand
BeagleBoard

• ESC-321
• Presented by Todd Fischer
• todd.fischer_at_ridgerun.com

2
Agenda

• Introduction to BeagleBoard multimedia
  architecture features.
• GStreamer concepts
• OMAP GStreamer hands on exercises
• DMAI and GStreamer
• OpenMAX and GStreamer
• Questions

3
Introduction

• Primary objective for this class is to introduce
  audience to GStreamer on OMAP3 with hands on
  exercises
• This doesnt class won't cover extensively the
  APIs available and assumes basic knowledge on C
  programming, Linux and computer graphics.

4
BeagleBoard and OMAP3 architecture

• Provides architecture with several multimedia
  features
• Cortex A8 with Neon
• C64x DSP
• Video accelerators
• How to utilize the hardware features with the
  software stack?

5
BeagleBoard and OMAP3 architecture

• Provides architecture with several multimedia
  features
• Cortex A8 with Neon
• C64x DSP
• Video accelerators
• How to utilize the hardware features with the
  software stack?

6
BeagleBoard and OMAP3 architecture

• Provides architecture with several multimedia
  features
• Cortex A8 with Neon
• C64x DSP
• Video accelerators
• How to utilize the hardware features with the
  software stack?

7
GStreamer

• Streaming media framework audio and video
• Close to 200 plug-ins available
• Higher level than just input / filters / output
• Networking, audio/video mixed streams, auto data
  handling
• Various options utilizing hardware accelerators

8
GStreamer Overview

• Elements
• Sources, filters, sinks
• Bins and Pipelines
• Containers, pipeline is the overall bin
• Pads
• Element source / sink connection points
• Caps
• Capabilities organized by stream type with a set
  of properties

9
GStreamer Overview

• Elements
• Sources, filters, sinks
• Bins and Pipelines
• Containers, pipeline is the overall bin
• Pads
• Element source / sink connection points
• Caps
• Capabilities organized by stream type with a set
  of properties

• Plugin
• Collection of
• elements

10
Simple MP3 Player

• Create dynamically using gst-launch
• Source element reads from a file
• Filter element converts MP3 to PWM
• Sink element passes to ALSA output

gst-launch filesrc locationa.mp3 ! mad ! alsasink
11
Hands On Exercise 0

• Start up Windowing environment with two terminals
• cd gst
• . src

Period character to source the filee
12
Hands On Exercise 0

• Play audio file
• a2
• Actual command
• gst-launch filesrc
• locationbbb.flac !
• flacdec ! alsasink

13
Simple Audio PlayerSource Code

• Create pipeline, source, filter, sink
• Set element properties
• Build into pipeline
• Connect src and sink pads
• Setup pipeline event handler
• End of stream
• Set pipeline state to play
• Run

14
Simple PA System

• Create dynamically using gst-launch
• Source element ALSS audio in
• No filters
• Sink element passes to ALSA output

gst-launch alsasrc num-buffers100 ! alsasink
15
Keeping Plug-ins Organized

• Each known plug-in is added to registry
• Most aspects of plug-in are tracked in the
  registry
• Registry support run-in pipeline creation and
  dynamic filter selection
• Use gst-inspect to list plug-ins

gst-inspect lessgst-inspect filesrc
16
Hands On Exercise 1

• Using gst-inspect, list
• All plug-ins
• All video plug-ins
• Element properties for filesrc plug-in

17
Hands On Exercise 2

• GStreamer video pipelines
• gst-launch videotestsrc ! ffmpegcolorspace !
  fbdevsink

18
PerformanceData Passing

• Stream held in buffers with data, timestamp,
  other info
• When possible, buffer memory allocated by sink
  pad
• Use hardware when data copy is necessary

19
PerformanceData Transformation

• Cortex A8 compiler optimization
• NEON
• Single Instruction Multiple Data
• C64
• Video accelerator
• DMA and other data movers

20
Davinci MultimediaApplication Interface
21
DMAI and GStreamer

• Davinci Multimedia Application Interface
• Exposes OMAP/Davinci hardware using high level of
  abstraction
• Stream audio / video
• Graphics display
• Hardware optimized frame/data copy

22
OMAP3 Codec Engine

• Isolates users for audio/video codecs from those
  implementing the codecs
• Codec can run in several places without calling
  application being aware
• Cortex A8, NEON, C64, hardware accelerator
• Uses DSPLink and DSPBios conventions to support
  DSP based algorithms dynamically

23
Video Phone

• Vision video phone application can create audio
  and video GStreamer pipelines for capture /
  rendering
• DMAI, Codec engine, DSPLink, C64 algorithms
  perform heavy lifting
• OMAP3 hardware capabilities utilized
• Video phone application focused on call
  management, not streaming data manipulation

24
OpenMAX and GStreamer

• Addresses codecs using hardware accelerators
• GStreamer integration via gst-openmax
• All OpenMax codecs available as GStreamer
  plug-ins
• Initial focus on high performance codecs

25
OpenMAX and OMAP3

• ARM Inc has Cortex-A8 / NEON DL implementation
• Texas Instruments has AAC and MPEG4 IL / DL
  support

26
GStreamer in3 Layers