Chapter 2 Enabling Technologies

1
Chapter 2 Enabling Technologies
Multimedia Systems
2
Key Points

• All types of media are represented digitally as
  patterns of bits.
• Analogue signals must be digitized, which
  comprises sampling and quantization.
• The Sampling Theorem states that, if the highest
  frequency component of a signal is at fh, the
  signal can be properly reconstructed if it has
  been sampled at a frequency greater than 2fh.
• Contemporary personal computers are capable of
  playing back most types of media.
• Multimedia production places much greater demands
  on computer systems.

3
Key Points

• Multimedia production requires the use of many
  different software applications, including
  authoring systems.
• Networks, and the Internet in particular, offer
  valuable opportunities for distributing
  multimedia, but they also present formidable
  technical difficulties.
• The most popular form of online multimedia
  delivery is the World Wide Web, which uses the
  Hypertext Transfer Protocol (HTTP) to organize
  its communications.
• Various formal and informal standards govern
  multimedia data, especially in the context of
  networks.

4

• Production and consumption of digital multimedia
  depends on ability of digital computer.
• Media data in digital form
• Program can be used to change, combine, store and
  display media of all types.
• Distribution
• Networks
• Removable storage CDROM, DVD

5

• Computers are not only devices that can
  manipulate digital data.
• DVD, set-top box, CD player
• Computer
• Full interactivity
• Future multimedia player
• Cheaper than computer

6
Digital Representations

• Bits, bytes, words.
• Data structures
• Programs
• Digitization
• Analogue and digital representations
• Analogue to digital converters
• Sampling and quantization
• Digital signals are much more robust than
  analogue ones, and do not suffer degradation
  when they are copied, or transmitted over noisy
  media.

7
Sampling and Quantization

• Sampling
• Measure the signals value at discrete intervals
• Sampling rate
• Quantization
• Restrict the value to a fixed set of levels
• Quantization levels
• Analogue to digital converter (ADC)

8
DAC

• Sample and holdFig. 2.4
• Not a good approximation
• Such a signal is passed to an output device such
  CRT or speaker, these discontinuities can be
  smoothed out.

9
Frequency

• Cycles per second (cps)
• Hertz (Hz) KHz, MHz
• Velocity frequency wavelength
• FFTAny periodic waveform can be decomposed into
  a collection of different frequency components.
• Frequency
• Temporal, spatial
• Fig. 2.6, vary periodically in space
• Fig. 2.7, Frequency components of a square wave
• DC component, f0
• Higher frequency abrupt transitions
• Fig. 2.8, square wave in frequency domain

10
Filter

• Remove certain frequencies
• High pass filter
• Low pass filter

11
Sampling Theorem

• If the highest frequency component of a signal is
  at f, the signal can be properly reconstructed
  at the sampling rate 2f.This limiting value is
  known as the Nyquist rate.
• Fig. 2.9
• The hearing range of the average person is
  approximately 20 Hz to 17kHz.(Sounds above this
  range is called ultrasonic sounds)
• Audio cards use 44.1kHz with sample sizes 8 and
  16 bits

12
Undersampling and Quantization noise

• Undersampling
• Sampling rate is less than the Nyquist rate.
• Aliasing
• In images, it is usually seen in the form of
  jagged edges.
• Quantization noise
• Number of quantization levels
• Image grey levels
• Sound

13
Hardware Requirements

• Multimedia PC (MPC) Level 3

14
Hardware

• PDA
• Less powerful devices
• Difficult to achieve the same performance of PC
• Limitation on the amount of storage
• Often based on cellular telephone network, less
  bandwidth
• Challenge of web designers
• Accessible to powerful PC and to PDA
• Multimedia architecture must be scaleable
• Transparently to devices with different
  capabilities

15

• High quality multimedia production are demanding
  than those for its consumption
• Special-purpose graphics acceleration hardware
  and input devices for video and audio
• High-speed disks, video capture card, specialized
  peripherals
• High-powered, SGI 3D animation and video
  effects, common in film and TV
• Special-purpose array processing in Pentium and
  PowerPC (MMX, AltiVec)
• Blur the distinction between high-end PC and
  workstations

16

• Producing multimedia content
• Processing power
• High-speed data buses
• Large memories
• Powerful graphics boards
• Content preparation
• Substantial more storage than finished product
• E.g. image layers up to 100
• High quality audio
• Compression only for final delivery

17

• Speed of data transfer to and from disks is a
  serious limiting factor
• Digital video 5Mbytes to 30 Mbytes
• IDE, SCSI-1 cannot cope with
• SCSI-2 supports 40Mbytes
• Ultra SCSI-2 80Mbytes
• Firewire, IEEE 1934 50 Mbytes, DV cameras
• Current disk speed for a dedicated computer
  adequate performance
• Higher performance of disk for central server
• RAID array Redundant array of inexpensive disks
• Improved performance in parallel
• AV disk

18
RAID

• 8 levels for different degrees of performance and
  fault-tolerance
• RAID 0 no protection against disk failure
• If one disk fails, whole array fails
• RAID 1 mirrors disk

19
Specialized Peripherals

• Graphics tablet with a pressure-sensitive pen
• Large high-resolution monitor
• Two monitors preparing images and display images
• Scanners
• Digital cameras
• Video camera and sound recording
• The equipment may itself be digital and produce
  digital output that can be sent to computer over
  Firewire.
• Video and audio capture from analogue equipment

20
Software

• The essence of multimedia is the combination of
  different media elements through computer
  programs.
• Different applications for different media
• Graphics Image editing, painting, drawing
• Text editors, layout
• Video capture, editing, postproduction
• Motion graphics and animation
• Sound recording, editing, effects
• Music synthesizer and sequencers
• Writing program for combination advanced
  programming skills
• An authoring systema software that permits a
  user to combine and organize these media elements
  (example, Director)

21
Authoring Systems

• May be based on
• Layout model with markup language
• Timeline
• Scripting language

22
Multimedia Production

• A host of software tools and skills
• Require a team
• Low end of multimedia software, web page design
• Non-specialists
• Hide technical details such as HTML tags and
  JavaScript code
• Wizards, assistants guide the production process

23
Multimedia Presentation

• A unified interface and manipulated a single
  program
• Production different software tools and team
  input
• WWW
• Define a format (in markup language, HTML, XML)
  that can accommodate different media and view it
  using a dedicated browser
• Define an architecture (format) and API provides
  a rich set of functions to manipulate data in
  that format
• QuickTime
• Deliver in a stand alone form, needs no
  additional software
• Director
• Three approaches are not necessarily separate or
  incompatible
• QuickTime movies in WWW pages
• Director movies in Shockwave and embedded in WWW
  pages
• Fig. 2.11

24
(No Transcript)
25
Networks

• TCP/IP
• ISP, Internet Service Providers
• Dial-up connection, 56 kbps
• V90 standard downstream56k, 33.6k in opposite
  direction
• 34-48 kbps are more realistic
• Maximum bandwidth using analogue connections
• Digital telephone, ISDN
• A digital channel64 kbps
• Basic rate two connections128kbps
• Mid-1980

26

• ADSL
• Downstream up to 6.1 Mbps
• Upstream up to 640 kbps
• Cable modem
• 500 kbps to 10 Mbps
• T11.544 Mbps, T344.736 Mbps
• LAN
• 10 base T Ethernet, 10 Mbps
• 100 base T, 100Mbps
• Table 2.1

27
Networks
28
Clients and Servers

• Server listen on a communication channel for
  requests from clients.
• Server receives a request, it sends a response.
• The requests and responses conform to a protocol.
• HTTP HyperText Transfer Protocol
• HTML markup language
• Browser Netscape, IE
• FTP
• Real-time data streaming for audio and video

29
Dynamically Generate Web Pages

• Dynamically generate Web pages from database
• CGI, common gateway interface
• Microsoft, Active Server Pages, ASP
• Apple, WebObjects

30
MIME Types

• OS
• Extension of files name
• Multipurpose Internet Mail Extension
• Content-type type/subtype
• Type broad indication of the sort of data
• text, image, audio, video
• model for 3D model data such as VRML
• message email
• application binary data, e.g. application/postscr
  ipt
• Subtype precise format
• HTML, GIF, AIFF
• x-, experimental, video/x-msvideo

31
Standards

• Define interfaces, file formats, markup
  languages, network protocols, and so on
• Three organization
• ISO
• International Electrotechnical Commission (IEC)
• International Telecommunication Union (ITU)
• ISO, IEC commercial companies
• Countries
• ANSI USA
• BSI UK
• DIN Germany

32

• Rapidly changing environment of computers,
  networks, multimedia
• Standard often obsolete before they have passed
  through all the stages.
• Internet standard
• Semi-formal
• Arpanet NSFNET TCP/IP
• Company production
• Postscript
• Quicktime
• SVG (W3C) lt Flash