Review Lecture

1
Review Lecture
IS 246Multimedia Information
Prof. Marc Davis Yahoo Inc. Thursday, March 15,
2007 330 pm 630 pm Spring 2007
2
Todays Review

• Foundations of Semiotics (15 min)
• Formalist Media Theory (15 min)
• Film Form
• Narrative Form
• Narration
• Editing (30 min)
• Kuleshov
• Isenhour
• Burch
• Barthes

3
Foundations of Semiotics

• Ferdinand de Saussure Course in General
  Linguistics (1906-1911)
• One of the founders of modern linguistics
• Established the structural study of language,
  emphasizing the arbitrary relationship of the
  signifier to signified and the diacritical nature
  of signs
• Distinguished synchronic linguistics (studying
  language at a given moment) from diachronic
  linguistics (studying the changing state of a
  language over time)
• Creation of semiology the study of sign systems
• Hugely influential on modern literary and media
  theory

4
Language and Linguistics

• The object of study of linguistics is language
  (langue) not human speech (langage)
• Linguistic study can be divided up into
• Diachronic linguistics
• How a system of values relates to and changes
  over time
• Synchronic linguistics
• How a system of values works per se (at a point
  in time)
• Linguistics is not just a subset of the general
  study of signs (semiology) but is its template

5
Linguistic Sign

• Sign, Signified, Signifier
• The linguistic sign is the unity of the signifier
  (a sound-image) and the signified (a concept)

6
Linguistic Sign

• The linguistic sign unites, not a thing and a
  name, but a concept and a sound-image. The
  latter is not the material sound, a purely
  physical thing, but the psychological imprint of
  the sound, the impression that it makes on our
  senses. The sound-image is sensory, and if I
  happen to call it "material," it is only in that
  sense, and by way of opposing it to the other
  term of the association, the concept, which is
  generally more abstract. (p. 66)

7
Linguistic Signs and Language

• The sign is arbitrary
• A multiplicity of signs is necessary to form any
  language
• Language is an over-complex system
• Language exhibits a collective inertia toward
  innovation

8
From Signification to Values

• ... to consider a term as simply the union of
  a certain sound with a certain concept is grossly
  misleading. To define it in this way would
  isolate the term from its system it would mean
  assuming that one can start from the terms and
  construct the system by adding them together
  when, on the contrary, it is from the
  interdependent whole that one must start and
  through analysis obtain its elements. (p. 113).
• Language is a system of interdependent terms in
  which the value of each term results solely from
  the simultaneous presence of the others ....
  (p. 114)

9
Linguistic Values

• Values are composed of
• A dissimilar thing that can be exchanged for the
  thing of which the value is to be determined
• Similar things that can be compared with the
  thing of which the value is to be determined

Signified
Signified
Signified
Signifier
Signifier
Signifier
10
Syntagmatic and Associative Relations

• In discourse, on the one hand, words acquire
  relations based on the linear nature of language
  because they are chained together. ...
  Combinations supported by linearity are syntagms.
  The syntagm is always composed of two or more
  consecutive units .... In the syntagm a term
  acquires its value only because it stands in
  opposition to everything that precedes or follows
  it, or to both.
• Outside discourse, on the other hand, words
  acquire relations of a different kind. Those
  that have something in common are associated in
  memory, resulting groups are marked by diverse
  relations. ...
• We see that the co-ordinations formed outside
  discourse differ strikingly from those formed
  inside discourse. Those formed outside discourse
  are not supported by linearity. Their seat is in
  the brain they are a part of the inner
  storehouse that makes up the language of each
  speaker. They are associative relations. (p.
  123).

11
Syntagmatic and Associative Relations
Associative (Paradigmatic) Axis
C
C
C
Syntagmatic Axis
A
B
C
D
E
12
Differences

• Everything that has been said up to this point
  boils down to this in language there are only
  differences. Even more important a difference
  generally implies positive terms between which
  the difference is set up but in language there
  are only differences without positive terms. (p.
  120).
• In reality the idea evokes not a form but a
  whole latent system that makes possible the
  oppositions necessary for the formation of the
  sign. By itself the sign would have no
  signification. (p. 130).

13
Discussion Questions (Saussure)

• Prof. Davis on Saussure
• How could signs (which are not positive units) be
  represented and manipulated computationally?
• How might Saussures ideas about linguistic signs
  apply to media such as video, photography, and
  music?

14
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

15
Why Study Formalist Film Theory?

• To provide a theoretical foundation for
  understanding the forms and functions of
  time-based media
• Unlike Film Studies per se, we use that
  understanding not to interpret films, but to
  analyze and design multimedia information systems
• Video capture
• Video analysis
• Video retrieval
• Video assembly
• Video reuse
• Video summarization (e.g., meeting recording)
• User interfaces to audio-visual content and that
  use audio-visual content

16
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

17
Expectations

• Suspense
• Delay in fulfilling an established expectation
• Surprise
• Result of an expectation that is revealed to be
  incorrect
• Curiosity
• Construct hypotheses about prior events

18
Perceiving Artistic Form

• Form
• The overall system of relations that we can
  perceive among the elements in the whole film
• In perceiving form, the spectator draws on
• Cues within the work
• Prior experiences
• Derived from everyday life
• From other artworks
• Conventions and norms

19
Principles of Film Form

• Function
• What is this element doing there?
• How does it cue us to respond?
• Motivation (justification for the presence of an
  element)
• Similarity and repetition
• Motif (any significant repeated element in a
  film)
• Parallelism (cues to compare two or more distinct
  elements by highlighting some similarity)
• Difference and variation
• Development
• Progression moving from beginning to middle to
  end
• Unity/Disunity

20
Viewers Activity

• The constant interplay between similarity and
  difference, repetition and variation, leads the
  viewer to an active developing awareness of the
  films formal system. (p. 56)

21
Todays Agenda

• Review of Last Time
• Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Discussion Questions
• Action Items for Next Time

22
Narrative Form

• Narrative
• A chain of events in cause-effect relationship
  occurring in time and space
• Story and Plot
• Story
• Set of all events in a narrative, both the ones
  explicitly represented and those the viewer
  infers
• Plot
• Everything visibly and audibly present in the
  film
• All the story events that are directly depicted

23
Story and Plot
24
Teeth Brushing Example

• Brushing Teeth
• Protagonist stands in front of bathroom mirror
• Protagonist opens medicine cabinet to remove
  toothbrush and toothpaste tube
• Protagonist squeezes out toothpaste on toothbrush
• Protagonist brushes teeth
• Protagonist drinks water from glass
• Protagonist spits out water and toothpaste
  residue

25
Time

• Temporal order
• Flashback
• Flashforward
• Temporal duration
• Story duration
• Plot duration
• Screen duration
• Temporal frequency
• Repetition of events

26
Temporal Duration

• Story Duration
• Example Brushing teeth in story world (5
  minutes)
• Plot Duration
• Example Brushing teeth in plot world (1 minute
  6 steps of 10 seconds each)
• Screen Duration
• Example Brushing teeth (12 seconds 3 shots of
  4 seconds each)

27
Space

• Story space
• Plot space
• Screen space and offscreen space

28
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

29
Narration

• Plots way of distributing story information in
  order to achieve specific effects
• Moment-by-moment process that guides us in
  building the story out of the plot
• Involves range and depth of story information

30
Range of Story Information

• Spectrum of knowledge of the story world that
  viewers and characters have
• Unrestricted (omniscient) narration
• Restricted narration
• Creates hierarchy of knowledge among viewer and
  characters
• Who knows what when?

31
Depth of Story Information

• How deeply the plot plunges into a characters
  psychological states
• Continuum between objectivity and subjectivity
• Subjectivity
• Perceptual subjectivity (hear and see what
  character perceives)
• Point-of-view shot
• Sound perspective
• Mental subjectivity (hear and see what character
  thinks)
• Internal voices
• Internal images
• How deeply do I know the characters
  perceptions, feelings, and thoughts?
• Range and depth of knowledge are independent
  variables

32
Classical Hollywood Cinema

• Action primarily arises from individual
  characters as causal agents
• The process of achieving goals desired by one or
  more characters drives the narratives
  development
• The protagonists goals come into conflict with
  other characters goals (antagonists) to create
  conflict

33
Classical Hollywood Cinema

• The cause-effect chain drives narrative events
• Plot time tends to depend on the storys
  cause-effect chain
• Dead time is rarely shown
• Appointments bring characters together at a
  specific time and usually place
• Deadlines makes plot duration dependent on the
  cause-effect chain
• Narration tends to be objective and
  unrestricted
• Narrative usually has strong closure at the end
  (cause-effect chain ends with final effect)

34
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

35
Dimensions of Film Editing

• Graphic relations between Shot A and Shot B
• Rhythmic relations between Shot A and Shot B
• Spatial relations between Shot A and Shot B
• Temporal relations between Shot A and Shot B

36
Continuity Editing

• Graphic continuity
• Smoothly continuous from shot to shot
• Figures are balanced and symmetrically composed
  in frame
• Overall lighting tonality remains constant
• Action occupies central zone of the frame
• Rhythmic continuity
• Dependent on camera distance of the shot
• Long shots last longer than medium shots that
  last longer than close-up shots

37
Spatial Continuity Editing

• 180 degree rule
• Ensures that relative positions in the frame
  remain consistent
• Ensures consistent eyelines (i.e., gaze vectors)
• Ensures consistent screen direction (i.e.,
  direction of character movement within the frame)

38
Use of 180 Degree Rule

• Establishing shot to establish axis of action
• Sequence of shot/reverse shots
• Focuses our attention on character reactions
• Eyeline match reinforces spatial continuity
  (Kuleshov Effect)
• Match on action reinforces spatial continuity
• Following 180 degree rule allows cheat cuts
• Continuity of action can override violations of
  180 degree rule

39
Temporal Continuity Editing

• Temporal order
• Forwardly sequential except for occasional use of
  flashbacks signaled by a dissolve or cut
• Temporal duration (seldom expanded)
• Usually in a scene plot duration equals story
  duration
• Punctuation (dissolves, wipes, fades), empty
  frames, and cutaways can elide time in shot and
  scene transitions
• Montage sequences can compress time

40
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

41
Kuleshov

• To determine the nature of montage is to solve
  the specific problem of cinema. (Eisenstein
  1949 48)
• Kuleshov
• First head of the Soviet State Film School after
  the October Revolution
• Lenin of all the arts for us the most important
  is cinema
• Kuleshov experiments
• Films without film
• Frame shots with hands
• Re-edit existing sequences

42
Pudovkin on Kuleshov Effect

• Kuleshov and I made an interesting experiment.
  We took from some film or other several close-ups
  of the well-known Russian actor Mosjukhin. We
  chose close-ups which were static and which did
  not express any feeling at allquiet close-ups.
  We joined these close-ups, which were all similar
  with other bits of film in three different
  combinations. In the first combination the
  close-up of Mosjukhin was immediately followed by
  a shot of a plate of soup standing on a table. It
  was obvious and certain that Mosjukhin was
  looking at this soup. In the second combination
  the face of Mosjukhin was joined to shots showing
  a coffin in which lay a dead woman. In the third
  the close-up was followed by a shot of a little
  girl playing with a funny toy bear. When we
  showed the three combinations to an audience
  which had not been let into the secret the result
  was terrific. The public raved about the acting
  of the artist. They pointed out the heavy
  pensiveness of his mood over the forgotten soup,
  were touched and moved by the deep sorrow with
  which he looked on the dead woman, and admired
  the light, happy smile with which he surveyed the
  girl at play. But we knew that in all cases the
  face was exactly the same.

43
Kuleshov Effect

• Kuleshov Effect
• Neutral Face ? Soup (Pensive Face)
• Neutral Face ? Dead Woman (Sad Face)
• Neutral Face ? Child playing with toy bear
  (Happy Face)
• How do you describe the face?
• Video has a dual semantics
• Sequence-independent stable semantics of shots
• Sequence-dependent variable semantics of shots

44
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

45
Isenhour on Context and Order

• A1B A2C ? A1 NOT EQUAL A2
• Shot context affects shot meaning
• The shot before affects the shot after
• BA1 CA2 ? A1 NOT EQUAL A2
• Shot context affects shot meaning
• The shot after affects the shot before
• AB NOT EQUAL BA
• Short order effects shot meaning
• (AB)C NOT EQUAL A(BC)

46
Metropolis Sequence
47
Metropolis Re-Sequence
48
Battleship Potemkin Sequence
49
Battleship Potemkin Re-Sequence
50
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

51
Burchs Transitions

• Temporal transitions
• Continuous
• Discontinuous
• Temporal ellipsis
• Measurable time ellipsis
• Indefinite time ellipsis
• Temporal reversal (flashback, overlapping cut)
• Measurable time reversal
• Indefinite time reversal

52
Burchs Transitions

• Spatial transitions
• Continuous
• Discontinuous
• Proximal
• Radically discontinuous

53
Todays Review

• Foundations of Semiotics
• Formalist Media Theory
• Film Form
• Narrative Form
• Narration
• Editing
• Kuleshov
• Isenhour
• Burch
• Barthes

54
Barthes Action Sequences

• Consecutive
• Temporal succession
• Consequential
• Causal succession
• Volitive
• Action results from an act of will
• Reactive
• Causal succession based on stimulus-response
• Durative
• Indicating the beginning, ending, or duration of
  an action
• Equipollent
• Necessarily paired actions (e.g., asking a
  question and answering a question)