Visual%20semiotics%20and%20space%20cognition:%20on%20the%20dynamics%20of%20two-dimensional%20representations%20of%20space%20and%20action

1
Visual semiotics and space cognition on the
dynamics of two-dimensional representations of
space and action
11th Early Fall School of Semiotics SEMIOTICS
AND GENRE September 10-16, 2005, Sozopol,
Bulgaria

• Wolfgang Wildgen

2
Part I Picture versus text

• The distinction between picture and text involves
  a set of basic semiotic challenges. First,
  pictures are linked in their production to the
  motricity of hands, in their receipt by the eye
  and the visual cortex. Language in its basic
  form, spoken language, is linked in its
  production to the motricity of the human vocal
  apparatus (from the vocal cords to the lips) and
  in its perception to the ear and the auditory
  cortex. The dynamics of these four subsystems and
  moreover the coordination of the pairs of
  subsystems in production and reception define the
  base line of any comparison of picture and text.

3

• At least two levels that must be separated
• the evolutionary basic distinction between a mode
  of manual/visual and a mode of phonetic/auditory
  communication,
• the application of manually based techniques to
  materials (with color or luminosity differences)
  and media (canvas, paper, glass, etc.) applying
  specific instruments (crayon, paint-brush, chisel
  etc.) is basic for products of the visual arts.
  This applies also to writing. As a consequence,
  pictures refer to a highly developed culture of
  materials, techniques and media

4
The linearity of language? A first confrontation
with pictures
Unidirectional process
Dependence on past or future steps of the process
Garden path and reanalysis
Major deviations of the unidirectional linearity
of language.
5
Fractal patterns(between d0 und d2
Cantor-set (d lt 1)
Sierpinki-triangle (d lt 2)
6
From one dimensional language to two-dimensional
pictures
Virtual dynamics of silence in communication
Virtual dynamics of a square (diagonal,
horizontal/perpendicular and spiral force-lines).
dynamics of corners
dynamics of sub-squares
Rotational dynamics (45o)
7
The flat structure of the atom gold
Historical mandalas as a demonstration of the
multiplicity of dynamical and symmetrical
structures in a circular space
8
Steady increase in the number of corners (and
therefore of implicit dynamic fields) with d
(dimension)

• two end points in a line segment
• four corners in a square
• eight corners in a cube
• sixteen corners in a four-dimensional cube
• 32, 64, 128 corners if we increase further the
  dimensionality of the cube

9
Implicit force-fields and the organization of
content

• In producing a picture these force-fields are
  relevant.
• A strong preference is given to rectangular
  frames which are near to the ideal (the square)
  but introduce a basic asymmetry.
• If we take the painting the Last Supper of
  Leonardo da Vinci (cf. Wildgen 2002, 2004b and
  2004a chapter 6), the prominent table of the
  supper fills the basic horizontal line and Christ
  marks the intersection with a vertical line of
  symmetry.

10
Part II Geometry and Dynamics in the Art of
Leonardo da Vinci
Leonardo da Vinci was born in 1452 in Vinci near
Florence. He made the first steps of his career
in Florence, where he worked under Verrocchio,
became master in his workshop and in 1477
independent master. In 1482 Leonardo asked
Ludovico Sforza in Milan for an employment. He
stayed in Milan until 1499, when Milan was taken
by the French troupes. Leonardo left Milan and
returned to Florence. In 1506 Leonardo was
invited to Milan by the French governor, Charles
d Amboise, and worked there until 1513. He left
Milan because of the pestilence and went to Rome
and left Rome invited by François I, king of
France, together with his pupils Melzi and Salai.
From 1516 until his death in 1519 Leonardo lived
in the Castel Cloux near Amboise in France.
11

• The two basic pillars of Leonardos semiotics of
  art are
• (linear, of color, vanishing) we shall deal with
  it under the topic of geometry.
• Dynamics (force, weight, counterpoise, balance,
  movement, percussion, etc.).

12
Last Supper by Leonardo da Vinci (Milan)
13
Force-fields in the Last Supper
..\Eigene Dateien\Präsentationen\Narrative
Struktur und visuelle Interpretation.ppt
14

• As this example shows, all three force fields we
  analyzed in the case of a void frame are used to
  organize specific contents (surfaces, figures,
  persons in space) in Leonardos mural painting.
  The head (ear) of Jesus is at the center of all
  force fields. The sub-centers of the groups of
  apostles lie in the intersections between the
  horizontal axis and the symmetric spiral which
  end at Jesus head (ear). The rectangle of the
  whole mural painting breaks the symmetry of the
  (ideal) square.
• The perspective generates a subdivision of the
  background space into three equal zones. In the
  central zone are situated Jesus, John (at the
  right of Jesus), and Thomas, James Major (at the
  left of Jesus) Judas is already outside of this
  field although he has the second position at the
  right of Jesus. Peter and Philip are at the
  intersections of these fields. 1

15
Basic content complexes organized in Leonardos
painting

• The table in the fore-ground.
• The perspective of the dining room, the windows,
  the landscape visible through the window, the
  subdivision of the background into three equal
  sub-fields.
• The arrangement of 12 apostles (grouped by 4 x 3)
  on both sides of Jesus.
• The gestures (body poses) and glances of Jesus
  and his apostles superimpose a further dynamical
  structure.

16
The geometrical scheme is that of a pyramid with
the angel on the second (visible) side of it. The
front side has on its vertex the face of the
Virgin, on its edges, the Jesus-baby at the right
and St. John Baptist as baby on the left (above
the base-line of Jesus and the angel).
17
Mary
protect
hold
Angel
John
point
bless
adore
Jesus
The pyramid in the Virgin in the Rocks.
18
The hand gestures alone define five different
force-lines and all four persons have different
gaze-lines involving different angles of the
head. All these forces imply some narrative
context, i.e., Leonardo tries to concentrate a
complex story in the static configuration of one
painting.
Drawing-study for the pointing gesture of the
angel
19
The semiotics of a figural composition
A sketch for the Epiphany
20
The triangle is asymmetric as the right side is
the line between the gaze of Mary and Jesus which
point to the face of a kneeing Magus. This is the
central force-line inscribed into the triangle.
21
Levels of semiotic analysis-  The space
(time) of the scene depicted (perspective,
light/shadow, outfit of the scenario).- The
thematic persons. These constitute a relational
schema which is represented by their relative
positions in space, the static relations and the
movements (e.g., the relative movement of the
head in relation to the trunk, the gestures of
the hands, the directions of gaze).- A
narrative content related to a known episode. The
scene may be identified as one moment resulting
from a series of prior events and having specific
(known) consequences
22
Part III Dimensional transitions, symmetry, and
transformations

• An architecture (3-space) is represented in an
  illusionist painting (trompe lœil), but one can
  neither enter the room nor move before it without
  destroying the illusion.
• A sculpture is represented in a mural painting
  one part may be sculpted, the other painted. In a
  proper position against the wall, it may be
  difficult to grasp the difference between 3-space
  and 2-space.
• A text describes a landscape, a building, a
  person either directly or as represented in a
  painting.
• A sentence contains an action scenario (in
  3-space time) in its valence structure, e.g.,
  Eve gives Adam an apple in the garden Eden. The
  action in 3-space time is flattened to a
  sentence with verb and case assignments/linear
  order.

23
Compression of information

• The technique of perspective (rediscovered in the
  Renaissance) codes artificially for the third
  dimension gestures, glances, frozen actions code
  for the temporal dimension.
• The technique of valence patterns (control of
  NPs), case assignment, etc., codes for the
  spatial parameters and allows their flattening
  into a sequence of verb (V) subject (S)
  object (O), etc. (in different orders dependent
  on the type of langue SVO, SOV, VSO, etc.).
• All the non-spatial or non-temporal dimensions
  are coded for by attributes/shape
  modifications/colors in a painting or by lexical
  differentiations in a sentence.

24
Symmetry and chaos-controllers as guarantees for
semiotic stability

• Group operation on a triangular star
• Identity (translation)
• Rotation (120)
• Mirroring on an axis

25
Hexagonal symmetry in a snow flake
26
Spatial symmetry in language (rare)
S A T O R
A R E P O
T E N E T
O P E R A
R O T A S
The sower (sator), called Arepo directs (tenet)
The carriage (the wheels rotas) with care
(opera)
27
Leytons generative geometry and the semiotics of
pictures

• GENERATIVE GEOMETRY A geometric object is one
  from which the transformations are recoverable
  i.e. a geometric object is a memory store.
• The regularity of the base space is the
  presupposition for the generation of
  information on deformations, asymmetries
  inscribed into the (pre-informative) space.

28
Translation
Translation
Rotation
The generation of a square and a rod via group
operations
29
Generation of complexity

• By a multiplication of basic types,
• By an intersection or blending of different
  entities, e.g. curved (sphere) and straight
  (cube).
• By an approximation of living forms (plants,
  animals, humans) in their geometry.
• By the addition of decorative patterns

30
Example from architecture the townhall in Bremen
Renaissance transformation of a gothic town hall
Decorative elements
31
New geometrical devices in the 20th century
Opera in Sidney, 1957-1973, and the spherical
geometry which underlies its construction
32
Conclusions

• Dimensionality fixes the basic geometry and
  dynamics which govern the organization of forms.
• Symmetries restrict possible constructions and
  thus contribute to the stability of the result.
• Capacities for meaning generation are created by
  possible deviations from the symplest and most
  symmetric forms (Leytons axiom).
• Natural semiosis is a process of memory
  generation and exploits systematically the
  geometry and dynamics of underlying forms.
• Decorative elements are inserted into the basic
  geometric and dynamic structure and profit from
  its memory function.

33
End of the presentation

• For further materials on the topic, cf.
• http//www.fb10.uni-bremen.de/homepages/homepageby
  id.asp?id34