Technology as a liberator of communication skills

1
Technology as a liberator of communication skills

• John Vincent and Anne McDougall
• University of Melbourne

2
Writing as a monomodal activity

• Most education systems only value verbal writings
• This is shown by assessment regimes
• So when children fail to communicate verbally
  they are condemned

3
Kostas handwritten reflection
4
(No Transcript)
5
Crossing semiotic boundaries

• Computers allow text products which include many
  semiotic modes
• Words
• Sounds
• Music
• Colours
• Animations
• Navigation devices

6
Kosta SOSE project
7
Interview with Kosta

• Int. Ok so if youre describing a cat, what
  would be the first thing you would do?
• E Oh Id look at it and get out a piece of paper
  and write down the words.
• I So is it just words or do you see a picture?
• E Its really just words. Looking at the cat
• I So in a description or word picture, do you
  keep a picture? In your head?
• E Oh. Er. No.

8
Interview with Peter

• Interviewer When you write a description or a
  word picture, how do you find the right words to
  sound good?
• Peter mmm I just go through lots of words.
• Int And in a description word picture, do you
  keep a picture in your head?
• P No not really.
• Int So how did you do the writing about the
  candle?
• P So same as the go through words that sound
  right
• Int You just say them in your head?
• P Yeah
• Int So when you were looking at the candle what
  did you think when you were looking at it?
• P I looked at the way it burns and the way the
  wax melts off it.
• Int So do you write them down, or
• P No I just say them in my head and if they
  sound right I just write them down.

9
Is the computer the difference?First the
literacy theory

• Dual Coding (Sadoski and Paivio 2001)
• Through experience, we develop the ability to
  understand and use language and a remarkable
  ability to manipulate and transform the world
  around us mentally using a non-verbal code of
  images.
• There are direct interconnections between the
  modality-specific mental representations in the
  two systems, so that we can switch from one form
  of representation to another, or recode, both
  within a system (e.g. speech to writing) or
  between systems (e.g. language to mental
  images).

10
Linguistic transformation
Siegel Immersed in a world of words of our own
making, we have come to regard our reliance on
language as natural and, in doing so, fail to
recognize that there are multiple ways of knowing
music, dance, visual arts and so on each of
which offers a distinctive way of making meaning.
Each sign system is based on a unique
organizational principle and involves elements
that have no ready equivalents in the other sign
system. Transmediation involves this very
question of how to translate from one sign
system to another.
Sipe Whenever we move across sign systems, new
meanings are produced because we interpret the
text in terms of the pictures and the pictures in
terms of the text.
11
Transmediation, Transduction
Kress While transformation operates on the forms
and structures within a mode, transduction
accounts for the shift of semiotic material -
for want of a better word across modes.
It is in the realm of synaesthesia, seen
semiotically as transduction and transformation,
that much of what we regard as creativity
happens
12
Descriptive mode Nikita
Camp Fire It creeps and leaps into the night,
into the infinite stretch of black velvet, the
sky. It boasts of strength, it stretches and
towers, its man-made figure spits and shoots tiny
burning ash-bullets at unsuspecting marshmallow
roasters. Like fireflies they cascade the dark,
flowing orange fireworks that burn out in the
cool night air, then shatter into invisible
pieces to be blown away on the chill wind. A
lady watches from her throne of earth, stands
tall and stately as the fire burns itself to the
ground, the proud unmerciful furnace sucking the
life out of the wood until it has nothing. Her
tall, grand mannagum posture gracefully sweeps
the sky with the wind, and she laughs at the
campfire, in its shame, weakness and misery in
the early morning.
13
(No Transcript)
14
MultiMedia Nikita

• Int When you are planning with the computer
  work, what do you have to do?
•  
• K Which pages what I am going to put on each
  page like if I don't put enough or if I put too
  much which is normally the case, and I normally
  have to do the writing first otherwise I never
  actually go anywhere.
•  
• Int So you have to get the story line in and
  then you build the pictures.
•  
• K Which is quite the opposite to what I
  normally do. I prefer to just write with just a
  pen or just typing it out on my computer instead
  of MicroWorlds and stuff.
•  
• Int So it is quite a challenge for you to
  actually do the other way?
•  
• K Yes I don't like it much.

15
Interviews with skilled writers

• Interview fragment 4 Nikita
• Interviewer So you actually think of pictures
  that you like, you see them?
• Nikita Yes.
• Int In a description, do you keep a picture in
  your head?
• N Yes, then I change the picture to something
  else, like in my bonfire poem, like the tree
  became a lady, a tall lady, because I like
  describing things like that.
• Interview fragment 5 Tayla
• Interviewer When you write a description or word
  picture how do find the right words to sound
  good?
• Tayla I dont know - I think when I see
  something I look at it and just think of words to
  describe it really. You can picture it in your
  mind and just think what if you were right
  there what would you be feeling. Thats partly
  happening in my head, and thats the picture
  and the words together.
• Daniel and Alex talked of a film running in
  their heads.
• Interview fragment 6 Daniel
• Daniel With the playground writing I sort of
  saw the bird fly into a tree and then I sort of
  watched it again in my head and I wrote down
  words that reminded me of it and then what it
  looked like and everything and added it into what
  I was writing down.
• Interview fragment 7 Alex
• Interviewer So when youre trying to describe
  this particular thing, do you have a picture in
  your head or do you try words out or write
  things down?
• Alex When Im writing the scene or something in
  my book Im - usually have in my head what to
  happen but I usually rewind that in my head and
  do it again. And delete and cut and copy into
  different places.

16
Computers? Software? Teaching Style? Learning
Style?

• It became clear that of all the possible
  variables, it was the multimedia application on
  the computer that was the overwhelmingly powerful
  facilitator, but only for a certain learning
  style. It actually inhibited communication for
  those who normally succeed, the skilled
  verbalisers.
• The reason is suggested strongly as being
  different natural abilities to cross semiotic
  boundaries

17
If the limitation to one mode of representation
is a limitation, then we should do everything we
can to overcome that limitation. If it is a
limitation on the totality of human potential, if
it favours one aspect only, to the detriment of
others, then we have, I believe, no justifiable
reason for sustaining it. (Kress 1997, p. 29)
18
G MultiMedia