COMICS By Mikael Kindborg, September 5, 2001

1
COMICS! By Mikael Kindborg, September 5, 2001
I study how the semiotics of comics can be
applied to programming. Why have I chosen comics?
These are important reasons

• Comics have a rich visual language of signs for
  representing dynamics in a static form. A static
  represen-tation gives a good overview and it can
  be edited.
• Comics are fun! Comics are well known and
  standardised, and are enjoyed by people of all
  ages.

• Comics have a representational directness. For
  programs based on animated characters (e.g.
  social agents), comics have the potential to
  narrow the conceptual gap between the program
  description (the source code) and the runtime
  manifestation of the program (the gulf between
  the programmers intention and how to express
  it).

The main research questions are the
expressiveness of program descriptions based on
comics, and their understandability by children.
Mikael Kindborg, Programmable Toys Laboratory,
Computer Science Department, Linköping
University, Sweden. Web site www.ida.liu.se/mik
ki/comics/ Email mikki_at_ida.liu.se
2
Concurrent Comics!
Common for these systems is that they are based
on implicit iteration/ recursion and some form of
concurrency. Comic strips can be viewed as script
s/methods. There are no conditions in comics, but
the first frame in a strip could be used as the
precondition for the actions in the rest of the
frames of the strip. A program using comic strips
to represent events/methods could be viewed as
concurrent comics, comics where the strips are
interpreted concurrently rather than sequentially.
A program representation that uses elements from
comics could be based on different computation
models

• Graphical rewrite rules, as used in Stagecast
  Creator and AgentSheets.
  
• Guarded clauses and message passing, as used in
  ToonTalk (Concurrent Constraint Programming).
  
• Event-based scripts with conditionals, as used in
  Squeak eToys (the Tiles scripting system in
  Squeak).

3
Low Fidelity Prototypes
To learn about the way children understand
concurrent comics, qualitative studies have been
made with 10-11 year olds in a school in Sweden.
The programming activities involved creating soc
ial agent programs. A social agent is an embodied
agent that can express emotions and whose acts
are socially related. To the left is an example o
f a paper prototype of a social agent program
created by two boys. When the girls meet as in st
rip one, they say Hello you are a fool and No
but you are a bad and boring ant. When they meet
as in strip two, they say Hello my friend and
I am not your friend. When the girl meets the
monster in the last strip, they say Can you eat
Elisabeth for me? and No she is my wife.
The left/right order of characters in the
precondition was considered important when the
children made paper prototypes, as was the strip
order.
4
High Fidelity Prototypes
This is an example of social agents programmed by
an eleven year old girl. To the left is a gallery
of actors. To the right is the program for the
doctor actor. He has one appearance, shown in the
small frame at the top. The event below says that
when the doctor meets Kate and she has a cold,
the doctor makes her well. The elements from comi
cs used are

• Strips of frames.
• Frame text boxes provide feedback and reflect
  what happens in the frames.
  
• Characters.
• Voice balloons.

and there are many more signs in comics that
could be used!
5
Childrens Interpretations
Narrative interpretations dominated for the paper
prototypes, that is, the program was seen as a
story rather than as behaviours. For the computer
prototypes this interpretation was less frequent.
The diagrams below show the intended
interpretation vs. two types of narrative
interpretations made by the children. In addition
to the order of strips, the order of characters
in the frames were also interpreted differently
for paper and for the computer.
Meeting left/right vs. meeting right/left was
thought of as two different situations when
working with LOFI prototypes. However, when using
HIFI prototypes, this was not an issue. Probably
because the computer enforced the intended
interpretation of only the distance being
significant to the meeting condition, not the
order of the characters.
Intended interpretation
Narrative interpretation I
Narrative interpretation II
Events happen in any order depending on which
conditions trigger
Condition x
Action x
Condition 1
Action 1
Condition 1
Action 1
First event
Start event
Condition y
Action y
Condition 2
Action 2
Action 2
Action 3
Second event
Condition n
Action n
Condition n
Action n
Action n-1
Action n
Last event
6
More Comics!
Screenshots from a HIFI prototype that uses
sub-frames to include objects not spatially close
to each other into the same frame, and motion
markers in the form of ghost trails.
This is an example of how comic strips could be
used to express programs in ToonTalk, providing a
static representation for ToonTalk robots.