Digital Media: Why Do We Need It, and How Do We Teach It as REAL Computer Science?

1
Digital Media Why Do We Need It, and How Do We
Teach It as REAL Computer Science?

• Jennifer Burg
• Department of Computer Science
• Wake Forest University
• Winston-Salem, NC 27109
• http//www.cs.wfu.edu/burg
• burg_at_wfu.edu

This material is based on work supported by the
National Science Foundation under Grants No.
DUE-0127280 and DUE-0340969.
2
Overview of Curriculum Material

• Primer
• Computer Science
• Art

Three companion books to be published by
Prentice-Hall, with on-line learning supplements.
Primer and Art books are being written by Dr.
Yue-Ling Wong, Wake Forest University.
3
Computer Science Material
4
Curriculum Material Creates a Bridge Between
Digital Media as Computer Science and Digital
Media As Art

• Primer (Yue-Ling Wong) foundational material
  for computer scientists and artists tied more to
  practice, but introducing the science behind the
  tools in a gentle way
• The Science of Digital Media (Jennifer Burg)
  for computer science majors focuses on the
  science and mathematics behind digital media
  tools like Photoshop, Audition, Illustrator,
  Premiere, Flash, and Director
• The Art of Digital Media (Yue-Ling Wong) for
  digital art students

5
Components of The Science of Digital Media

• Textbook
• Interactive tutorials
• Worksheets
• Programming exercises
• MATLAB mathematical modeling exercises
• (all currently available on the web at
• http//digitalmedia.wfu.edu)

6

• All material is freely available on the web at
  http//www.wfu.edu.
• Answer sheets to worksheets and implemented
  programs are at a password-protected site.
• This talk will focus on The Science of Digital
  Media.

7
How I Use the Material

• A course called Digital Media covers Chapters 1
  through 5 and is taught to computer science
  majors (juniors and seniors).
• Pre-requisites are Calc I, CS1 and CS2, and data
  structures.
• Chapters 1 through 5 cover digital imaging and
  digital audio.
• Digital video is introduced in the course
  (briefly).
• In class, we cover the science and mathematics of
  digital media.
• Behind the scenes, students do a game program
  as their semester-long term project, putting
  together digital photographs, vector graphics
  digital sound, and interactivity using Macromedia
  Director and its scripting language Lingo.
• Students are expected to learn hands-on things
  on their own.
• There is a pre-test and post-test for each
  chapter.
• Students also do worksheets and on-line
  tutorials. (Note that in this course, we do not
  do the programming exercises.)
• Second course called Digital Video and Animation,
  does an overview of central concepts of Chapters
  1 and 5 and then focuses on Chapters 6-8, which
  cover digital video and multimedia programming.

8
Software Needed for the Courses Described on
Previous Slide

• Image processing (e.g., Photoshop)
• Vector graphics (e.g., Illustrator or Freehand)
• Digital audio processing (e.g., Audition or Sound
  Forge)
• Video processing (e.g., Premiere or Final Cut)
• Multimedia programming (e.g., Director of Flash)
  (I use Director).
• Optional -- A programming environment (e.g., C
  or Java), if you choose to do the programming
  assignments.
• Optional MATLAB or Mathematica for mathematical
  modeling exercises. (If your students dont have
  access to MATLAB or Mathematica, but YOU do, you
  can do the mathematical modeling exercises as
  in-class demos.

9
Alternative Ways to Use Computer Science
Curriculum Material

• Teach a course in Algorithms for Digital Media
  and focus on the programming assignments.
• OR
• Focus on only one medium per course (digital
  imaging, audio, or video).
• OR
• Cover all of the book (The Science of Digital
  Media) in an undergraduate course, but not in
  complete depth. Then use the entire book again
  in a graduate course and go into more depth in
  the mathematics and science.

10
DiscussionAre courses in Digital Media suitable
for a rigorous computer science program?

• Is Digital Media REALLY computer science?
• Yes! Not only is it computer science, but its a
  more interesting way to engage the students
  interest in science.

11
Imagine

• You are students.
• Im your teacher teaching you about
  two-dimensional arrays.
• Two-dimensional arrays are data elements stored
  in contiguous blocks of memory, and blah blah
  blah
• Scribble scribble scribble on the board

12
OR

• I can show you what you can do with
  two-dimensional arrays.
• Image dithering

13
(No Transcript)
14
What can students learn in this exercise?

• Two-dimensional arrays, matrix operations
• Dynamically vs. statically allocated arrays
• Array/index vs. pointer/offset notation
• File I/O
• Not to mention
• Loops
• Control structures
• Function call and return
• Etc.

15
And there are other digital media topics that
deal with two-dimensional arrays

• Convolutions and unsharp mask

16
Imagine

• You are students.
• Im your teacher teaching you about trees.
• A tree consists of nodes. A non-empty tree has
  a root node, and n child nodes, where n gt 0, and
  blah blah blah
• Scribble scribble scribble on the board

17
OR

• I can show you what you can do with a tree.
• Octree algorithm for indexed color

18
indexed color by Photoshop
our own implementation of indexed color
19
This version has more color gradient.
With some fine-tuning of the algorithm, we can do
even better.
20
Octree Algorithm for Indexed Color
When the algorithm reaches a point where it wants
to create a 257th leaf node
This is what an image looks like when the node
with the least children is collapsed. Note that
there is more color detail in the sky.
This is what an image looks like when the node
with the most children is collapsed.
Photographs courtesy of Kristian Damkjer, who
used this curriculum material at the University
of Florida in the fall of 2005.
21
What can students learn in this exercise?

• The tree data structure
• Recursion and recursive descent of a tree

22
Students also can be motivated to learn
mathematics through digital media

• Show how colors can be modeled mathematically and
  understood as occupying a graphical space.

23
Students can be given a motivation for learning
logarithms
Human perception of sound intensity is
proportional to the logarithm of the air pressure
amplitude. This is why we use decibels to
measure sound. Decibels are defined in terms of
logarithms.
24
We can capture the students interest in fairly
complex mathematical operations e.g., the
Fourier transform.
25

• In short, relating concepts to images, sound,
  motion, and interactivity motivates students to
  learn.

26
Where does digital media currently fit in the
computer science curriculum?

• Not included in ACMs list of major
  subdisciplines within computer science
• computer science
• information systems
• software engineering
• computer engineering
• information technology
• (See http//www.acm.org/education/curric_vols/CC20
  05_Final_Report2.pdf.)
• Ill use the term digital media to denote a
  field of study and thus will consider it a
  singular noun. (Just didnt want you to think I
  dont know my grammar. ?)

27
Where does digital media currently fit in the
computer science curriculum?

• In Computer Science Computing Curricula 2001, not
  included as one of the essential courses or areas
  within traditional computer science
• i.e., architecture, graphics, databases,
  operating systems, algorithms
• The closest thing is Graphics and Visual
  Computing, with some elements covered in
  Net-Centric Computing, Algorithms and Complexity,
  and Programming Languages
• See
• http//www.computer.org/portal/cms_docs_ieeecs/iee
  ecs/education/cc2001/cc2001.pdf
• and
• http//www.cs.wfu.edu/burg/papers/BurgWongStrokSI
  GCSE2004.pdf

28
Where does digital media currently fit in the
computer science curriculum?

• Digital media courses in computer science
  departments exist mostly as electives.
• Digital media is sometimes covered superficially
  in CS100-type courses.
• Digital media also appears as
• an interdisciplinary field of study
• a course or courses within disciplines outside of
  computer science (e.g., art, communications)
• a department all its own

29
Why do we need digital media?

• Enrollments are falling in computer science.
• Even students who like working with computers may
  perceive computer science as irrelevant and
  detached from the REAL world of computers.
• We dont always create a good link between
  science and practice. Digital media is an ideal
  area in which to make this connection.

30
Why have computer scientists been reluctant to
include digital media in the computer science
curriculum?

• It seems inherently interdisciplinary.
• (Well, thats one of its charms!)
• It seems SOFT!
• (Really, its NOT! Consider the topics in the
  computer science curriculum material at
  http//digitalmedia.wfu.edu.)

31
Combining the science of digital media with the
practice of digital media

• Emphasize the science in classroom teaching,
  readings, worksheets, etc.
• In the background, students are working with
  application programs such as Photoshop, Audition,
  Premiere, Flash, Director, and/or Javascript.
• I have my students do a game program as their
  term project, developed incrementally as they
  study each medium.
• Another good assignment is an on-line
  resume/porfolio.

32
Just in case you wondered Why do I use
Director instead of Flash for on-line tutorials?

• Director is more powerful than Flash in the
  following ways
• Better support for video formats.
• Better support for sound.
• More Xtras.
• Allows for launching of external applications.
• Director now allows you to use Javascript syntax
  as an alternative to Lingo.
• See http//www.macromedia.com/software/director/re
  sources/integration/flash/quick_comparison.html

33
After the break

• Do tutorials and exercises
• Do a mathematical modeling worksheet
• Discuss ways to use the material
• Discuss your own curriculum needs
• Talk about good projects and assignments
• Talk about whether digital media really should be
  in computer science

34
The Challenge

• Link science to practice
• Show that knowing the science results in better
  practice
• Get scientists and artists communicating and
  collaborating on creative projects