MPEG4 XMT SMILSVG - PowerPoint PPT Presentation

1 / 39

About This Presentation

Title:

MPEG4 XMT SMILSVG

Description:

SMIL/SVG/Flash can be compiled to BIFS. Smaller profiles. Too low-level ... Flash to XMT / SVG. Graphics and animations, no scripts. More complex that XSLT ... – PowerPoint PPT presentation

Number of Views:79

Avg rating:3.0/5.0

Slides: 40

Provided by: jeanclaud6

Category:

more less

Transcript and Presenter's Notes

Title: MPEG4 XMT SMILSVG

1
MPEG-4 XMT ? SMIL/SVG

Jean-Claude Dufourd
Ecole Nationale Supérieure des Télécommunications

2
Overview

MPEG
MPEG-4
MPEG-4 Systems
XMT
XMT wrt SMIL/SVG
Authoring with XMT and XSL
Conversions
Demos

3
MPEG

ISO/IEC JTC1/SC29/WG11
Based on National Bodies (ANSI, DIN, AFNOR)
MPEG-1 CD-I
MPEG-2 DVD, digital TV
MPEG-4 the ultimate multimedia standard ?
MPEG-7 meta-data for searching multimedia
MPEG-21 everything you need for multimedia

4
MPEG-4

Video Simple and Adv. Simple profiles
Audio AAC 2 x MP3
Systems
SNHC synthetic content
Conformance
Reference software
Advanced Video Coding (a.k.a. JVT, H264, H26L)

5
Structure of Systems

MPEG-4 more objects and more types
More Streams ES Management
OD Framework
Sync Layer
Systems Decoder Model
DMIF
Present Objects Scene Description

6
Structure of Systems (2)

MPEG-J Java APIs to terminal components
XMT textual exchange format

7
ES Management

MPEG-4 system architecture
Object-based compression and delivery
How to link elementary streams to the scene
Object descriptors
How to synchronize elementary streams
System decoder model
Sync layer
How to multiplex and deliver elementary streams
MPEG-4 stream delivery

8
Linking streams into the scene
An object descriptor containsES descriptors
pointing to

Scalably coded content streams
Alternate quality content streams
Object content information
IPMP information

ES descriptors have subdescriptors for
Decoder configuration

Sync layer configuration

Quality of service information

Future / private extensions

9
MPEG-4 stream delivery
C o m p o s i t o r
D e l I v e r y I n t e r f a c e (DAI)
decom- press
Scope of MPEG-4 Systems
10
Synchronization of multiple ES

Based on two well known concepts
Clock references
Convey the speed of the encoder clock
Time stamps
Convey the time at which an event should happen
Time stamps and clock references are
defined in the system decoder model
conveyed on the sync layer

11
MPEG-4 content access procedure
OD

Locate an MPEG-4 content item (e.g. by URL)and
connect to I

How? - Via the DMIF Application Interface (DAI)

Reminder This abstraction is being mapped to
actual signaling protocols (RTSP, MPEG-2 PSI,
etc.)

Retrieve a first Object Descriptor

This Object Descriptor points to BIFS OD stream

Open these streams via DAI

The Scene Description points to other streams
through Object Descriptors

Open the required streams via DAI

Start playing!

12
Scene Description BIFS

BIFS is based on VRML97
Scene Graph
Structure
Grouping, Spatial composition
Leaves
AV objects
Behavior and Animation
Binary Encoding

13
BIFS defines

2D
2D/3D integration
Advanced Audio
Better Timing Model
Scene Update/Streaming
Scene Animation
Parameterized compression
New objects Face, Body, Mesh

14
BIFS examples
15
(No Transcript)
16
(No Transcript)
17
Improved Time Model

VRML timing can only work in download and play
BIFS Access Units and time inherited from SL
and OD
Media Control/Sensor/Buffer work on a media
clock
FlexTime manage indeterminate times

18
Streamability

Scene updates
Replace scene
Insert/Delete an object
Modify properties
Insert/Delete behaviors

19
Binary encoding

More efficient than ASN.1, not flexible
Bit-level encoding
Nodes and Fields
ID
Suite de valeurs de champs
Masque binaire
Liste index-valeur
Contextual encoding of node types

20
Quantization

Compact representation of numbers int, float,
double and aggregates
14 quantization categories / semantics
position/size 2D/3D
color, angle, scale, normal, rotation axis
Each field has static quantization category
defining Q() and Q-1()

21
Advanced Text and Graphics

New extension
Bring BIFS on par with other formats
Better text
Downloadable font
Missing graphic features

Viewport
Curve types
Filters
Holes

Gradients
Line Properties
TransformMatrix

22
BIFS Pros and Cons

Built-in interface with AV
Tight synchronisation
Streamability
Compression (for streaming graphics)
SMIL/SVG/Flash can be compiled to BIFS
Smaller profiles
Too low-level
Codec complexity

23
XMT Design Constraints

Exchange between authors/tools
Interoperability with X3D and SMIL
Allow manual editing
XMT-A all of BIFS, nothing but BIFS
XMT-O as much of SMIL/SVG as possible, staying
translatable to BIFS

24
XMT-A

XMLization of BIFS strict equivalence
ltpargt for timing
We started from X3D Schema
We added scene updates, anim and OD
ltStreamSourcegt
ltEncodingHintsgt

25
XMT-A Design Choices

XML Schema
Strong type validation
Nodes are elements
Simple fields are attributes
Complex fields are sub-elements with wrappers

26
Exemple XMT-A

ltAppearancegt
ltmaterialgt
ltMaterial2D
emissiveColor"0.8 0.5 0.3"
filled"true"/gt
lt/materialgt
lt/Appearancegt

27
XMT-O ?? SMIL

Animation yes but final value
Content Control partial, needs to be mapped
statically to ODs
Layout partial, layout consistant with the BIFS
scene tree
Linking partial, need to be mapped onto Anchor
Media yes, extended
Metainformation yes, extended to MPEG-7
Timing and Synchronization almost complete
Time Manipulations yes
Transitions partial

28
Example XMT-O header

lt?xml version"1.0" encoding"UTF-8"?gt
ltXMT-O xmlns"urnmpegmpeg4xmtoschema2002"
xsischemaLocation"urnmpegmpeg4xmtoschema20
02 xmt-o.xsd"
xmlnsxsi"http//www.w3.org/2001/XMLSchema-insta
nce"
xmlnsxlink"http//www.w3.org/1999/xlink"gt
ltheadgt
ltlayout type"xmt/xmt-basic-layout"
metrics"pixel"gt
lttopLayout height"400" width"600"/gt
lt/layoutgt
lt/headgt

29
Example XMT-O body 1

ltbodygt
ltpar begin"0ms" dur"5000ms"gt
ltrectangle begin"0ms" dur"5000ms
id"Rectangle" size"100 100"gt
lttransformation order"2" translation"-5
10"/gt
ltmaterial color"0 0.3 0.8"
filled"true"/gt
lt/rectanglegt
lt/pargt
lt/bodygt
lt/XMT-Ogt

30
Example XMT-O body 2

ltbodygt
ltpar dur"60s"gt
ltbackdrop dur"indefinite" color"white"/gt
ltrectangle dur"60s" size"300 200"gt
ltmaterial color"blue" filled"true"gt
ltanimateColor dur"2s" repeatDur"60s"
calcMode"linear"
attributeName"color" from"blue"
to"aqua"/gt
ltoutline id"out" color"red"
width"20" style"solid"gt
ltanimateColor id"anim" dur"5s"
repeatDur"60s"
calcMode"linear"
attributeName"color
from"red" to"yellow"/gt
lt/outlinegt
lt/materialgt
lt/rectanglegt
lt/pargt
lt/bodygt
lt/XMT-Ogt

31
Existing Tools

IBM
XMT-A ? BIFS
XMT-O ? XMT-A
XMT-O Reference Software
GMD/Optibase XSLT XMT-A ? VRML
XSLT then BifsEnc then MP4Enc
XMT-A Reference Software
Mp4tool BIFS ? XMT-A
Various academic tools (nothing free)

32
Authoring with XMT and XSLT

Required
XMT to MP4 mp4tool
XSLT Xalan
MP4 Player Osmose or EnvivioTV
High-Level Objects
SVG
Application Specific ECS
Ad-Hoc B4

33
SVG to MP4

Most of SVG Tiny, no glyph/text
Some of SVG Basic
99 XSL, 1 scripting

34
Application Specific Cartoons

XML output of cartoon CAD
Dump of tool internal data shapes, graphic
properties, insertion/deletion, depth, textures
Selection of Translatable Features
Slight shuffling
Generate XMT syntax
Easy, extensible

35
Ad-Hoc B4