Introduction to Media Computation

1
Introduction to Media Computation
Notes adapted from Introduction to Computing and
Programming with Java A Multimedia Approach by
M. Guzdial and B. Ericson, andinstructor
materials prepared by B. Ericson.
2
Learning Goals

• Understand at a conceptual level
• What is media computation?
• How do digital pictures work?
• How do digital sounds work?

3
What is Media Computation?

• Processing some collection of
• picture elements
• sound fragments
• movie frames
• text files and HTML pages
• The speed and storage capacity of modern
  computers makes this possible
• Even for beginning students just learning to
  program!

4
How Does Color Vision Work?

• Our eyes and brain work together to make sense of
  what we see
• The cones in our eyes are what allow us to see
  in color
• The rods allow us to see black, white, and shades
  of gray
• Our cones are sensitive to red, green, and blue
  light
• All other colors are combinations of these

5
Red, Green and Blue Light

• On a computer, we can produce white light as a
  combination of red, green, and blue
• Full intensity red, green, and blue combined
• Black is the absence of all light
• No red, green or blue light
• All other colors are combinations
• Of red, green, and blue
• Of different intensities

6
Color Exercise

• Start DrJava
• In the interactions pane type
• ColorChooser.pickAColor()
• Click on the RGB tab and move the sliders to
  change the intensity of red, green, and blue
• Make white, black, red, blue, green, yellow,
  violet, and orange

7
How Do Digital Cameras Work?

• There are red, green, and blue filters that
  capture the amount of each color at a position
• A part of a grid
• There are many positions
• picture element or pixel
• 640 x 480 is low resolution
• 1600 x 1200 is high resolution
• The more pixels, the better the picture (in
  theory)
• Can enlarge it without it looking grainy

8
How Do Computer Displays Work?

• A display has pixels (picture elements)
• Each pixel has a red, green, and blue component
• Combinations of red, green, and blue give the
  resulting color
• Black is 0 red, 0 green and 0 blue
• White is 255 red, 255 green, and 255 blue

9
Pictures are Made up of Pixels

• Digital cameras record light at pixels
• Monitors display pictures using pixels
• Our limited vision acuity helps us to see the
  discrete pixels as a smooth picture
• If we blow up the picture, however, we can see
  the pixels

10
Digital Pictures

• Capture the intensity of the red, green, and
  blue colors at each pixel
• Stored as a bunch of numbers
• Typically 8 bits for red, 8 bits for green, 8
  bits for blue
• Need nearly 1 million bytes to store a 640 x 480
  picture
• Need 3 million bytes to store an image from a 1
  megapixel (million pixel) camera
• Displayed as red, green, and blue colors on the
  computer display
• Lots of them close together
• Our brain sees a smooth color image

11
Digital Pictures in Java

• Java supports the use of digital pictures
• It does so by providing a Picture class
• To use pictures then, we create picture objects
  much in the same way we created world objects or
  turtle objects earlier
• We then have a collection of methods we can use
  to show and manipulate our pictures
• Lets take a quick look at what we can do with
  picture objects

12
Creating Picture Objects

• We can create picture objects in Java, much like
  other objects
• For example
• Picture picture1 new Picture()System.out.print
  ln(picture1)
• This creates a picture object, and prints
  information about it, but it doesnt
  actuallyshow anything at this point
• To do that, we do the following
• picture1.show()

13
Creating Picture Objects

• Why didnt our picture have anything in it?
• It was just a 200x100 pixel image of white
• Ultimately, thats because we didnt tell itto
  create a picture object for a specificimage
• So, instead, we got a default picture created
• If we want a picture createdfrom a specific
  image storedin a file on the computer,we need
  to do things a bitdifferently

14
Creating Picture Objects

• First, we pick a file name and save a reference
  to the resulting String object in a variable
  called fileName
• String fileName FileChooser.pickAFile()
• Next, we create a Picture object and save a
  reference to it in a variable called pictureObj
• Picture pictureObj new Picture(fileName)
• Now we invoke the show() method on the picture
  object
• pictureObj.show()

15
Creating Picture Objects
16
Doing it all at Once

• You can create a picture object
• by passing it the result of using the FileChooser
  to pick a file
• and then tell that picture object to show itself
• and you can do it all in one line
• new Picture(FileChooser.pickAFile()).show()
• But, there are problems with doing this
• You dont have a way to refer to the file or
  picture again, which is bad
• It looks ugly and hard to read, which is also bad

17
Show Picture Exercise

• Try both ways of creating a picture object and
  showing it.
• new Picture(FileChooser.pickAFile()).show()
• And do each piece one step at a time naming the
  result of each method
• String fileName FileChooser.pickAFile()
• System.out.println(fileName)
• Picture picture new Picture(fileName)
• System.out.println(picture)
• picture.show()

18
Substitution and Evaluation

• In programming you can
• Use a literal
• String name "Barb"
• Use a variable
• String myName "Barb"
• String name2 myName
• Use an expression
• String n3 "Ba" "rb"
• Use the result of a method invocation
• String n4 FileChooser.pickAFile()
• Values get substituted for variable names when
  expressions are evaluated

19
What is a Sound?

• Any object that produces vibrations in matter
  will make a sound
• The vibrations produce a sound wave
• The pitch is based on the frequency of the cycles
  in the sound wave
• The loudness is based on the amplitude (height)
  of the sound wave

20
How Does Hearing Work?

• The outer ear catches sounds
• The eardrum vibrates
• The inner ear translates the vibrations to nerve
  impulses for the brain to interpret

21
How does Recorded Sound Work?

• Phonograph recordings capture sound continuously,
  as an analog signal
• CDs and DVDs sample sounds and record numbers
  that represent the sound at the time of the
  sample
• With 44,100 samples per second!

22
Why Digitize Sound?

• High fidelity
• Reproduced sound is very similar to the original
• Perfect reproduction
• Sounds the same every time
• Easy to transmit
• Download as data
• Easier to manipulate on a computer
• Even though there are billions of bits

23
Playing a Sound

• We can create a Sound object just as we created a
  Picture object
• Get a file name and save a reference to it
• String fileName FileChooser.pickAFile()
• Create the sound object by asking the class to
  create a new Sound object and initialize it by
  reading data from the given file name
• Sound sound1 new Sound(fileName)
• Play the Sound
• sound1.play()

24
Play Sound Exercise

• Try creating a Sound object and playing it by
• Specifying it in steps
• Specifying it all at once
• How would you play the same sound twice?

25
Summary

• Pictures, sounds, text and movies can be
  digitized
• Media computation can mean processing millions to
  billions of bytes of data
• The speed of modern computers makes media
  computation possible even for beginners
• Java makes it easy to work with pictures and
  sounds using the Picture and Sound classes
• Well see more on this throughout the course!