Advanced Interactive Programming Graphics and Multimedia

1
Advanced Interactive ProgrammingGraphics and
Multimedia
2
Outline

• A quick overview on graphics
• Multimedia controls
• Multimedia MCI control
• ActiveMovie ActiveX control
• RealAudio ActiveX control
• Microsoft agent
• Animation control
• Programming multimedia controls

3
Graphics

• Two approaches to creating graphics in Visual
  Basic
• Graphical methods - the lowest primitive building
  blocks
• Graphical controls - These are like ordinary
  Visual Basic controls which are placed on your
  form and can be laid out interactively at design
  time.

4
Graphic Methods

• These are commands that enable you to draw
  directly on to your form at run time. Examples of
  these commands include
• Print draw text on a form or control
• Pset sets the color of an individual pixel
• Point returns the color value of a specified
  point
• Line draws a line or rectangle, or filled box
• Circle draws a circle, ellipse, or arc
• Graphics methods can be used to draw on certain
  objects including forms and picture boxes.

5
Graphical Controls

• There are four standard graphical controls in VB
  
• Two of the controls, image and picture box
  controls allow you to work with various image
  files
• While line and shape controls draw lines and
  shapes on your forms (you can then change the
  properties of these)

6
Colours

• Color as RGB value
• Colour properties can be applied to graphical
  controls and methods
• Colour properties
• BackColor
• ForeColor
• BorderColor
• FillColor

7
Multimedia MCI Control
8
Media Control Interface (MCI)

• A high-level API for controlling multimedia
  devices
• An intermediate layer between system software,
  application and hardware
• Software developers write programs using MCI
• Hardware suppliers provide drivers supporting MCI
• No interactions between Software developers and
  Hardware suppliers
• Applicable in both programming languages and
  scripting languages
• DirectX API for game programming

9
Multimedia MCI Control

• An ActiveX control
• Allows programs to interact with any MCI-enabled
  devices
• Such devices include sound boards, CD-ROM drives,
  audio players, video players and MIDI sequencers

10
Multimedia MCI Control

• Once connected to a device, buttons are
  automatically enabled for those supported
• The control can be customised to display only
  buttons supported
• Programmable in several ways
• The control can be visible or invisible at run
  time.
• You can augment or completely redefine the
  functionality of the buttons in the control.
• You can control multiple devices in a form

11
MCI Control Commands
12
MCI Control Properties
13
MCI Control Properties
14
MCI Control Events
15
Example MCI Control CD Player

• Use MCI control to control CD-ROM drive as an
  audio CD player
• The players GUI consists of three labels, three
  textboxes and a MCI control

16
Code
17
Example MCI Control AVI File Player

• Use MCI control to play AVI (audio-video
  interleave) files
• The player GUI contains a MCI control and a
  button

18
Code for AVI File Player
19
Microsoft ActiveMovie Control
20
Background

• Based on Microsoft DirectShow - an extensible
  media architecture, which defines how
  applications can control and process time-stamped
  multimedia data using modular components called
  filters connected in a configuration called a
  filter graph
• DirectShow SDK, APIs and filters APIs are
  available

21
Example Microsoft MPEG Filter Graph

• A source filter to read the data off the disk
• A splitter transform filter to separate the video
  and audio
• A video transform filter to decompress the video
  data
• A video rendering filter to display the data on
  the screen
• An audio transform filter to decompress the audio
  data
• An audio rendering filter to send the audio data
  to the sound card

22
Microsoft ActiveMovie Control

• Enables an application to play video and sound in
  many formats, including MPEG audio and video, AVI
  video, WAV audio, MIDI audio, and Apple
  QuickTime video.
• The elements of the controls GUI is shown below

23
Microsoft ActiveMovie Control

• The ActiveMovie control is programmable in
  several ways
• Each element can be shown or hidden.
• Each element can be enabled or disabled.
• You can augment or completely redefine the
  functionality of the buttons in the control.

24
ActiveMovie Control Properties
25
ActiveMovie Control Methods
26
ActiveMovie Control Events
27
Example Media Player

• Use ActiveMovie control to play media in an
  application
• The application contains a CommandButton, a
  commonDialog and an ActiveMovie control

28
Code for the Media Player
29
Using ActiveMovie Control in HTML Pages

• Two ways to play media files on HTML pages
• Embedded in a Web page
• Externally in a window displayed outside or on
  top of the page
• Four ways to activate media playback using HTML
  tags
• OBJECT
• EMBED
• IMG DYNSRC
• A HREF in an external window

30
RealPlayerRealAudio ActiveX Control
31
Streaming Technology

• A technique for transferring data such that it
  can be processed as a steady and continuous
  stream.
• The technology becomes increasingly important
  because
• Internet-based multimedia files or
  multimedia-oriented applications are popular
• Most users do not have fast enough access to
  download large multimedia files quickly or the
  storage to manage the files
• With streaming, the client browser or
  applications can start displaying the data before
  the entire file has been transmitted.

32
Streaming Technology

• Content creation, i.e. audio and video production
  using various tools
• Content encoding and authoring, i. e. how to
  convert and encode live and on demand audio and
  video files into the Internet accessible format
  and files, usually in .ram file
• Serving, i.e. the framework for content
  management and URL construction, delivering
  on-demand content, and live streaming over IP.
  This is usually achieved through a Web server or
  dedicated server
• See the following web site for details
• http//www.realnetworks.com/resources/tutorials/i
  ndex.html

33
RealAudio ActiveX Control

• The control enables VB applications to receive
  and play streaming audio and video clips over the
  Internet

Image window will appear when the source is video
clips
34
Position slider
Volume control
Information Summary
Internet congestion LED
Current position indicator
35
Example Playing a Audio

• Set the source to http//delphi.about.com/library/
  media/audio/IntroDel.ram
• No need to coding

36
Example Playing a Video Clip

• Set the source to http//play.rbn.com/?urlbps-out
  bound/bps-outbound/demand/videoseries/smil/intro.s
  milprotortsp
• No need to coding

37
Example Playing a Video Clip
38
Microsoft Agent

• Microsoft Agent is a technology for creating and
  using interactive animated characters in a
  Windows application or Web page.
• It enables software developers and Web authors to
  incorporate a new form of user interaction within
  the Microsoft Windows interface, known as
  conversational interfaces, that leverages natural
  aspects of human social communication.

39
Microsoft Agent

• Animated characters (agents) can be used as
  interactive assistants to introduce, guide,
  entertain, or otherwise enhance Web pages or
  applications such as museum presenter
• Microsoft Agent allows users of applications to
  interact with application using more natural
  human communication techniques

40
Microsoft Agent

• With support for speech recognition, characters
  can respond to voice commands using synthesized
  speech, recorded audio, or recorded video clip,
  etc.
• With motion capture and recognition, agents can
  even respond to gestures such as nodding,
  pointing, etc.
• http//www.microsoft.com/msagent/default.asp

41
Microsoft Agent Control

• Provides access to three predefined characters
• Allows users to interact with applications
  through the characters by mouse (pointing),
  keyboard (textual input) and voice commands
• The agent uses the icon at design time but will
  display one of the characters at execution time

42
Microsoft Agent

• Microsoft agent comes with a cast of three
  characters Merlin, Genie, and Robby
• It also includes tools for defining custom
  characters using Microsoft Agent Character Editor

43
Example Application

• Users can select and display a character from the
  ComboBox
• Users can type in text in the TextBox for a
  character to speak
• The CommandButton tell the selected character
  when to speak

10-1
44
Code for Initialisation
45
Code for Initialisation
The show event calls an animation
46
(No Transcript)
47
Code for ComboBox and Speak Button
Actions by selecting an agent from ComboBox
Actions by clicking the speak button
48
More Application Scenarios
49
Code for Clicking, DblClicking and Drag and Drop
50
Microsoft Agent Events
51
Character Object Properties Methods
52
Speech Output Tags
53
Animation Control

• The Animation control allows applications to play
  AVI file animations.
• The AVI files should not have sound, and are
  uncompressed or compressed using Run-Length
  Encoding (RLE). .
• At run time, the control does not have a visible
  frame.
• The control maintains a separate thread of
  execution while it is playing. Therefore, your
  application will not be blocked, and can continue
  to execute within its process.

54
Property and Methods
55
Example Animation Player

• The player contains a CommandButton, a
  CommonDialog and the Animation control

56
Code for the Player
57
Summary

• Extend your capability using many available
  third-party multimedia ActiveX controls
• For example, Shockwave Flash can be use to play
  Flash movies (.swf files)
• see some websites
• http//www.swiftsoft.de/activex.html
• http//activex.devdirect.com/ALL/Multimedia_PCAT_1
  954.aspx

58
Tutorial Questions

• Describe what is meant by the term streaming.
• Explain the function and usage of each of the
  following controls
• Microsoft Agent control
• ActiveMovie control
• RealAudio ActiveX control