Multimedia Technology SMD151 Lecture 1

1
Multimedia TechnologySMD151Lecture 1

• Peter Parnes
• Peter.Parnes_at_ltu.se

2
Today

• Course information
• Overview of the multimedia area

3
Who am I?

• Dr Peter Parnes Assistant Professor
• Peter.Parnes_at_ltu.se
• http//media.csee.ltu.se/peppar/
• 0920/492421
• 070/6614567
• A3307
• E-meeting in the CDT session
• ICQ (4208035)
• Microsoft Instant Messenger (peppar_at_cdt.luth.se)

4
Course Goal

• Understanding the area of distributed multimedia
• Media coding
• Real-time communication
• Architectures for distributed multimedia
  applications
• And much much more -)

5
Course Information

• Course web page http//media.csee.ltu.se/peppar/k
  urs/smd151/
• Fronter www.fronter.com/ltu/ All handins
  (reports etc) are to be done via Fronter!
• Announcements will be sent via Fronter so make
  sure you are on the course.

6
Course Overview

• Three major parts
• Lectures
• 8-9 in total
• Covers material from the book
• Also much additional material will be presented
• Slides available on-line
• Seminars
• ....
• Labs
• ....

7
Seminars

• Groups of 3
• Find an interesting subject
• Do a literature study
• Write a scientific report
• Verbal presentation (9 minutes long)
• Subject deadline 040906 12.00
• Fronter
• More info on the course page(http//media.csee.lt
  u.se/peppar/kurs/smd151/)

8
Labs

• Groups of 3
• More information from Johan now
• Presented in written reports (on the web) and as
  running applications when applicable
• More information on the web

9
Exam

• Home-exam
• Released mid course
• A major assignment
• Individual work
• Ca 2 questions with a word limit on each.
• Handin via Fronter
• Graded.

10
Grade

• Final grade
• 25 home exam
• 25 seminars
• 50 labs

11
Deadlines!!!

• All parts in the course will have deadlines.
• Honor them and get the grade you deserve.
• Be on time for lectures!!!
• PLEASE!

12
The Book

• Loads of text...
• Reading instructions on the course page in a few
  days...
• Book information, examples etc
  http//www.cs.sfu.ca/mmbook/

13
Questions?

• Questions now?
• Please ask question at any time in the course!!!!
  
• Protest!! Interact!!
• Send suggestions via email (to the list or
  directly to me).
• Use the discussion forum in Fronter.
• Do not be afraid to critique me, I can take it
  -)

14
Multimedia ???

• What is multimedia???

15
What is Multimedia?

• Multimedia is a very broad area
• Mainly undefined with many different views.
• Multi many
• Media plural of medium middle, center,
  intermediary
• Multiple intermediaries

16
The Information Field

• The word multimedia is used in various fields in
  information handling
• Storing and processing in computing
• Production in publishing
• Distribution in mass media
• Transmission in telecommunications
• Presentation in interaction between people and
  systems
• Perception in peoples interaction with outside
  world

17
Digital Multimedia

• What is digital multimedia?
• Digital multimedia is the field concerned with
  the computer-controlled integration of text,
  graphics, still and moving images, animation,
  sounds, and any other medium where every type of
  information can be presented, stored and
  processed digitally.
• F. Fluckiger

18
Multisensory Systems

• Humans are multisensory.
• Sight, hearing, smell, touch and taste.
• Visual media
• Audio media

19
Multisensory Systems

• Advantages of multisensory systems
• Appear more natural and friendly
• Redundant information more possible
• Complementary information improves memorization
• Emotional information easier to convey

20
Classification of Media
Time/space nature
Continuous (time-based)
Animation
Moving Images
Sound
Discrete (space-based)
Still Images
Text
Graphics
Origin
Captured from real-world
Synthesized by computers
21
Networked Multimedia

• This course is really about networked
  multimedia.
• Two main categories
• Genuinely networked applications
• Client-Server model based applications

22
Examples

• Video teleconferencing.
• Distributed lectures for higher education.
• Tele-medicine.
• Co-operative work environments.
• Searching in (very) large video and image
  databases for target visual objects.
• Augmented reality placing real-appearing
  computer graphics and video objects into scenes.

23
More examples

• Including audio cues for where video-conference
  participants are located.
• Building searchable features into new video, and
  enabling very high- to very low-bit-rate use of
  new, scalable multimedia products.
• Making multimedia components editable.
• Building inverse-Hollywood" applications that
  can recreate the process by which a video was
  made.
• Using voice-recognition to build an interactive
  environment, say a kitchen-wall web browser.

24
Example Desktop audio/video
25
Example Wearable
26
Multimedia Research Topics

• Multimedia processing and coding
• Multimedia system support and networking
• Multimedia tools, end-systems and applications
• Multi-modal interaction and integration

27
Interesting projects

• Camera-based object tracking technology
• 3D motion capture
• Multiple views
• 3D synthesis technology
• Specific multimedia applications
• Digital fashion
• Electronic house call system
• Augmented Interaction applications

28
Tangible Interfaces
29
I/O Brush
30
History

• 1945 - Vannevar Bush wrote a landmark article
  describing what amounts to a hypermedia system
  called Memex.
• 1960 - Ted Nelson coined the term hypertext.
• 1967 - Nicholas Negroponte formed the
  Architecture Machine Group.
• 1968 - Douglas Engelbart demonstrated the On-Line
  System (NLS), another very early hypertext
  program.
• 1969 - Nelson and van Dam at Brown University
  created an early hypertext editor called FRESS.
• 1976 - The MIT Architecture Machine Group
  proposed a project entitled Multiple Media
  resulted in the Aspen Movie Map, the first
  hypermedia videodisk, in 1978.

31
More history

• 1985 - Negroponte and Wiesner co-founded the MIT
  Media Lab.
• 1989 - Tim Berners-Lee proposed the World Wide
  Web
• 1990 - Kristina Hooper Woolsey headed the Apple
  Multimedia Lab.
• 1991 - MPEG-1 was approved as an international
  standard for digital video led to the newer
  standards, MPEG-2, MPEG-4, and further MPEGs in
  the 1990s.
• 1991 - The introduction of PDAs in 1991 began a
  new period in the use of computers in multimedia.
• 1992 - JPEG was accepted as the international
  standard for digital image compression led to the
  new JPEG2000 standard.
• 1992 - The first MBone audio multicast on the Net
  was made.
• 1993 - The University of Illinois National Center
  for Supercomputing Applications produced NCSA
  Mosaic, the first full-fledged browser.

32
Even more history

• 1994 - Jim Clark and Marc Andreessen created the
  Netscape program.
• 1995 - The JAVA language was created for
  platform-independent application development.
• 1996 - DVD video was introduced high quality
  full-length movies were distributed on a single
  disk.
• 1998 - XML 1.0 was announced as a W3C
  Recommendation.
• 1998 - Hand-held MP3 devices rst made inroads
  into consumerist tastes in the fall of 1998, with
  the introduction of devices holding 32MB of flash
  memory.
• 2000 - WWW size was estimated at over 1 billion
  pages.

33

• Integrating Digital Information

34
Multimedia Systems

• Essential characteristics of multimedia systems
• Computer controlled
• Integrated
• Digitally represented information
• Optionally offer interactivity

35
Computer Controlled

• Rather obvious
• One or several computers have to be involved in
  the presentation of the information.
• E.g. a CD-player is not a multimedia system.

36
Integration

• A multimedia system should handle producing,
  storing, carrying and presenting information in
  an integrated way.
• E.g. Computer Integration
• E.g. Network Integration

37
Integration
Presentation integration
Storage integration
CD-ROM
video window
Magnetic Disk
text window
text
Network integration
Capture integration
video camera
Single Network
integrated microphone
38
Digital Representation

• What is digitization?
• The transformation from an analog signal to a
  digital signal
• An analog signal varies continuously and is said
  to be analogues to the measured value

39
The Digitization Process
sampled signal
analog signal
Sampler
Quantizer Coder
digitized signal
40
Conversion
digital signal
analog signal
A/D converter
(011001101 ...)
D/A converter
analog signal
41
Why Digital Representation?

• All types of information can be represented as
  bits
• Can be moved over digital networks
• No loss in copy
• Easy to maintain the information error free

42
Drawbacks of Digital Rep.

• Distortion
• Might require high bit-rates to transfer
• Fast networks...
• Large storage needed...

43
Digital Representation
Memorandum This is a note to inform you that ...
one page formed of digital characters
4 000 bytes
50 000 bytes
Memorandum This is a note to inform you that ...
same printed page scanned and digitized
44
Two Modes of Presentation

• Two basic modes of presentation
• Passive or Linear
• TV, movies, radio
• Interactive or Non-linear
• The receiver can modify the presentation
• Time
• Order 4 degrees of customization
• Speed
• Form
• Newspaper, book, computer game

45
Examples

• PocketPro
• MPEG4

46
Handheld Audio/Video
47
MPEG-4 Video
48
Questions?

• ?

49
Media

• Media definitions
• Media parts
• Text, graphics, images, video, animation and
  sound.
• Might sound trivial but notions needed.
• Most from the human-computer interaction HCI
  world.

50
Text
This is an unformatted text also called plain
text. All the characters have the same style and
font and their pitch is the same. Regular human
computer interfaces use plain text. An advantage
is that v e r t i c a l a l i g n m e n t
is easier.

• This is an example of rich text.
• Characters can have different styles and their
  pitch is variable.
• They may also us different fonts.
• They may also respect certain formatting
  rules.

51
Graphics vs. Images
Graphics
Images

• graphics are revisable documents
• the document format must retain structural
  information
• the semantic content is preserved in the
  representation
• described as objects

images are not revisable the document format is
unaware of any structural information the
semantic content is not preserved described as
bitmaps formed of individual pixels
52
Still Images

• Format bitmap
• Picture elements pixels
• Pixel/Amplitude depth
• 1,8,16,24,32 bpp bits per pixel
• Common formats JPEG, GIF, PNG, TIFF
• Compression of images

53
Moving Images

• Movies consists of a series of frames or still
  images
• Frame rates and human perception
• lt10 sequence of still images
• 10-15 jerky effect
• gt15 movie effect
• Formats
• MPEG1,2,4
• AVI, QuickTime (containers)

54
Frame Rates

• Movies 24 fps
• TV
• American 30 fps
• European 25 fps
• HDTV 60 fps

55
Sound

• Speech vs. Non-speech sounds
• and computers
• Speech coding
• Use computers to communicate with humans
• Speech recognition and understanding
• to computer
• Speech synthesis
• from computer

56
Enabling Technologies

• We need some technologies to be able to do
  networked multimedia )
• Computers....
• CPU, RAM, storage space, display etc.

57
Central Memory
Memory bus
network
CPU
DMA
Bus Interface
Disk controller
Comm. Interface
Peripheral bus
Audio player
Audio A/D
Video compression
VCR
Display controller
Video A/D
58
Transmission

• Terrestrial transmission
• Metallic cables
• Twisted Pair, TP (shielded or unshielded)
• Optical fibers
• Modulated light

59
Aerial Transmission

• Surface transmission
• Radio, microwave, IR-light, laser
• Radio up to 200-400 Mbps
• Radio affordable at 11 to 108 Mbps
• Satellite transmission
• High delay, 100-300 ms

60
Base band Transmission

• Apply the signal to be sent directly on the
  cable.
• Only one bit can be sent at one time.

61
Synchronous Time Division
fixed time slots dedicated to data from A, B and
C
A
B
C
A
B
C
A
B
C
time
Synchronous Time Division Multiplexing System
Input line A
Input line B
Output line
Input line C
62
Asynchronous Time Division
variable time slots allocated to data from A, B
and C
A
B
C
B
C
A
c
C
time
Asynchronous Time Division Multiplexing System

Input line A
Input line B
Output line
Input line C
63
Broadband Transmission

• Modulate a carrier signal
• E.g. Change the amplitude of the carrier
• Allows us to send several signals in the same
  wire at the same time.
• We use a
• Modulator-demodulator modem

64
Network Topologies

• Full connection
• Star topology
• Tree/Mesh topology
• Routers and switches needed as junctions
• Bus
• Send the signal in both directions to everybody
• Ring

65
Bus Topology
station transmitting at a given instant
destination station
flow of transmitted data
66
Ring Topology
station transmitting at a given instant
flow of transmitted data
destination station
67
Tree structures

• Internet!
• Routers, switches, hubs
• Packets...

68
Communication Protocols

• Computers need a predefined language to
  communicate,
• Just like humans!
• Thus, protocols!
• Protocols are layered into stacks
• Suites Internet Protocol, OSI, DNA, SNA

69
Packet vs. Circuit switching

• Circuit
• Physical or virtual path created
• Dedicated bandwidth per circuit
• Wastes bandwidth
• Packet
• Chop up data into packets
• Send and forward them when necessary
• Multiplexing

70
Connections...

• Connection-oriented
• Create a virtual connection
• The network knows that A and B want to
  communicate
• Connectionless
• Just send data
• Network forwards the data

71
L,C,M,W,PAN

• Local Area Network LAN
• Campus Area Network - CAN
• Metropolitan Area Network MAN
• Wide Area Network WAN
• Personal Area Network PAN

72
Summary

• Course introduction
• Digital Media and Representation
• Computers
• Communications and Networks

73

• Questions?