Kein Folientitel

1
How much rhetoric does an account of
multimodal/hypermedial design need?
John Bateman
Document Design Conference 2004, Tilburg, January
24th
2
Overview of talk

• The role of rhetoric in document description
• Using rhetorical description multimodally
• Issues and discussion

3
Assumptions / Statements

• Rhetorical organisation is perceived as necessary
  for good document design and particulary web
  design and online interaction...
• An appropriate rhetorical organization is a
  prerequisite for effective communication.
• Communication should deploy resources
  consistently and this includes issues of
  rhetorical force.

... But what does this mean?
4
Rhetoric for organising information

• how do we systematise it for use in description
  and analysis?
• how do we teach it?
• is it really useful for design, or just post-hoc
  description?

5
Approaches to multimodal rhetoric

• Document Design / Analysis
• Roland Barthes
• Karen Schriver
• Automatic Multimodal Document Generation
• Elisabeth André
• Stephen Feiner

6
Research Goal to investigate the use of
different modes of information presentation
across genres and languages

• the GeM project
• Genre and Multimodality
• http//purl.org/net/gem

7
Genre Analysis Birds
http//purl.org/net/gem
8
Genre Analysis Birds
http//purl.org/net/gem
9
Genre Analysis Birds
http//purl.org/net/gem
10
1972
Redistribution of Information across modalities
and across time
The plumage is white with a tinge of buff on the
head and neck and dark brown, almost black,
wing-tips. Immatures are first dusky all
over, later piebald or white sprinkled with dark
spots.
Plumage white, save the wing quills, which are
dark brown, not black as usually stated. The
fledgling blackish-slate, spotted white. Immature
till the third or fourth year, and recognized by
the dark brown, chiefly on the back, wings, and
tail, which colour diminishes season by
season till at maturity reduced to the brown of
the wing quills.
1924
ADULT White, black wing-tips, yellow
nape. JUVENILE Grey, gradually becoming white
over 5 years.
1996
11
The GeM model

• Content structure
• Rhetorical structure
• Layout structure
• Navigation structure
• Linguistic structure
• Canvas constraints
• Production constraints
• Consumption constraints

http//purl.org/net/gem
12
The GeM annotation layers

• Content structure
• Rhetorical structure
• Layout structure
• Navigation structure
• Linguistic structure

http//purl.org/net/gem
13
The GeM annotation layers

• Content structure
• Rhetorical structure
• Layout structure
• Navigation structure
• Linguistic structure

http//purl.org/net/gem
14
Approaches to multimodal rhetoric

• Document Design / Analysis
• Roland Barthes
• Karen Schriver
• Automatic Multimodal Document Generation
• Elisabeth André
• Stephen Feiner

15
Relation between content, rhetorical organisation
and layout

• examine the page layout
• examine the rhetorical structure of the
  information as presented
• look for interrelationships

16
Rhetorical Structure Theory RST
NUCLEUS
SATELLITE
so you dont get the wires mixed up.
Replace spark plugs one at a time
Example relation motivation
(Honda Civic car manual)
17
motivation
presents an action in which the reader is the
actor and which is unrealized with respect to
the context
NUCLEUS
Replace spark plugs one at a time
comprehending the satellite increases the
readers desire to perform the action
presented in the nucleus
SATELLITE
so you dont get the wires mixed up.
EFFECT the readers desire to perform the
action presented in the nucleus is increased.
18
A simple case applied to layout
19
Layout Structure blocks
20
Layout Structure of the Page
the page
G
A
B
C
D
E
F
21
Rhetorical Structure of the Page
elaboration
joint
The artic explorer wears lots of warm clothes to
protect him from the cold
material
balaclava
trousers
mittens
windproof top
boots
woolen underclothing
22
Extensions to RST for multimodal analysis

• Linear adjacency replaced with 2D topological
  adjacency

23
Extensions to RST for multimodal analysis

• Linear adjacency replaced with 2D topological
  adjacency
• Not all elements of the page are subject to
  rhetorical relationships

24
Distribution of information across layers
Semantic Content
Layout
RST segments
layout units
navigational elements
base units
25
Distribution of information across layers
Semantic Content
Layout
RST segments
layout units
navigational elements
base units
26
Distribution of information across layers
Semantic Content
Layout
RST segments
layout units
navigational elements
base units
27
Extensions to RST for multimodal analysis

• Linear adjacency replaced with 2D topological
  adjacency
• Not all elements of the page are subject to
  rhetorical relationships

28
Extensions to RST for multimodal analysis

• Linear adjacency replaced with 2D topological
  adjacency
• Not all elements of the page are subject to
  rhetorical relationships
• Some layout units stand in the equivalent of
  intra-clausal relationships not inter-clausal.

• identification, class-ascription,
  property-ascription, possession, location

29
Three Examples of Using the Rhetorical Analysis
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
Page Layout Elements
39
Layout Structure
Types of element
Drawing
Intermediate
Caption
Textblock
40
Content Analysis
41
RST analysis
background
background
elaboration
elaboration
body parts
Tiger
pic
attributes
relationships
comparisons
lifespan
diet
weight
length
young
height
season
body
maturity
gestation
tail
joint
purpose
means
elaboration
joint
function
coat
joint
coating
of stripes
background
eyes
hearing
joint
Tiger mouth
(
pic
)
back of ears
white spots

claws
canines
molars
(
pic
)
function
purpose
(
pic
)
claws
why
retract
42
Layout Structure
Types of element
Drawing
Intermediate
Caption
Textblock
eyes ears coat mouth teeth
claws
43
(No Transcript)
44
Layout Structure
Types of element
Drawing
Intermediate
Caption
Textblock
eyes ears coat mouth teeth
claws
45
Layout Structure
Types of element
Drawing
Intermediate
Caption
Textblock
eyes ears coat mouth teeth
claws
46
(No Transcript)
47
navigation structure
rhetorical structure
layout structure
base level items
48
Extract from telephone instruction page
49
Extract from telephone instruction page
50
(No Transcript)
51
(No Transcript)
52
Extract from telephone instruction page
53
Does it matter?

• is there some rhetorical resolution below which
  users/readers are just not bothered?
• is it possible to find reading difficulties,
  comprehension difficulties where rhetorical and
  layout decisions are not made coherently?
• are the particular kinds of rhetorical relations
  that will be picked out for layout representation
  and others not? and in particular genres?