Flash Forums and ForumReader: Crafting an Interface for a New Kind of Online Discussion

1
Flash Forums and ForumReaderCrafting an
Interface for a New Kind of Online Discussion

• Kushal Dave, kushal_at_google.com
• Martin Wattenberg, mwatten_at_us.ibm.com
• Michael J. Muller, michael_muller_at_us.ibm.com
• IBM Research / Collaborative User Experience
• Cambridge, MA USA
• Work done at IBM

2
Outline

• Flash Forums
• What are they?
• Why do they matter?
• ForumReader
• Design decisions
• Demo
• User feedback
• Jam trial
• Lab study
• Conclusions

3
Reading 892 of these is hard!
4
Flash forum examples

• Slashdot
• News for nerds portal
• Several articles discussed daily
• Hundreds of posts per topic within a day
• IBM Jams
• Company-wide discussions
• Several broad forums
• Thousands of posts over 3 days
• Blog comments, news discussions, et al.
• As much text as a small novel

5
Flash Forums

• In contrast to Usenet and other ongoing forums
• Diffuse authorship
• Large size
• Focused topic
• Short duration
• Cf. flash mobs a large group of people who
  gather in a usually predetermined location,
  perform some brief action, and then quickly
  disperse wordspy.com
• Often, shallower threads

6
Flash forums are less conversational
90.00
80.00
Usenet
70.00
Slashdot
Jam
60.00
50.00
Percent of authors posting more than once
40.00
30.00
20.00
10.00
0.00
0.00
20.00
40.00
60.00
80.00
100.00
Percent of messages that are replies
7
Flash forum threads are shallower
70
60
50
40
Percent of messages
30
20
10
0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Thread depth
8
Why diffuse authorship matters

• Authors are relevant
• Some known users on Slashdot
• Some key personalities in IBM
• Metadata about users (reputation, seniority)
  provide cues
• but ideas become focus
• Users indicate ideas trump authors (unlike
  Usenet)
• Official and distributed moderation (try to)
  create meritocracy

9
Why time limits matter

• A Jams authenticity is derived from the fact
  that its a real-time and finite event, and that
  there are real, often serendipitous knowledge
  accidents among participants that emerge because
  of the time constraint imposed.
• Dorsett, Fontaine, ODriscoll

10
Why size and topic matter

• Many simultaneous posts, constrained topic, and
  shallow threads lead to
• Thread drift
• Particular themes split across threads
• Diminished utility of threads as filters

11
Discussion interfaces

• Much work on Usenet
• Conversation Map
• Netscan
• Our problem is different
• Authors are inadequate filters
• Threads are inadequate filters
• Everything happens at once
• Basically, the discussion is one big mess

12
ForumReader

• Easily move around the discussion
• Many ways to enter discussion
• Sense of orientation
• Integrate visualization and text analytics
• Builds on existing work in thumbnail interfaces
• SeeSoft/SeeSys, Readers Helper, Context Lens

13
DEMO!
14
Jam feedback

• 8,973 posts, 22,000 participants
• Survey of 1,248 participants
• 16 used ForumReader successfully
• Important (3.5 / 5) and satisfying (3.2 / 5)
• Value in orientation
• Ability to find themes came up repeatedly
• Amazing. To be able to locate commonalities,
  etc., and analyze the worth of this VALUABLE
  effort IS GREAT!

15
But we still didnt know

• How do users navigate discussions? What cues do
  they use?
• Does the interface really help them understand
  the scope of discussions and find information?
  Which features are most valuable?

16
Lab study design

• 2x2 varying visualization, text analysis
• Data Collected
• Describe expectation
• Explore discussion
• Identify key arguments
• Generate mindmap or outline
• Indicate relative amounts of discussion of topics
• Argue for or against
• Evaluation

Visualization Y N
4
4
Text Analytics N Y
4
4
17
Preference
Users consider map, search and moderation
highlighting as valuable as scrollbar and text
view.
18
Performance

• Visualization and text analytics might improve
  performance individually, but detract together.

Purple Text analytics Blue No text Left
columns No viz. Right columns Visualization
N.S.
plt.03
F(1,12)1.95
F(1,12)6.57
Plt.04
Plt.05
F(1,12)5.20
F(1,12)4.26
19
Navigation patterns
Users used the map (red dots) extensively, often
nonlinearly.
Rows are conditions map (top), NLP (second),
control (third), map NLP (bottom) Dots show
navigation events scrollbar/arrow keys (blue),
map (red), tree (cyan) Lines show click search
(black), typed search (cyan), highlighting (all
others)
20
Reading patterns
Users spent more time viewing starts of threads,
highly-moderated posts, and starts of discussions.
21
Summary

• Flash forums present novel structure and dynamics
  (diffuse, big, focused, short)
• Users value cues like readwear, moderation,
  threads, authors (especially themselves!)
• Navigation is idiosyncratic
• Our visualization and text analytics appear to
  help users see more of the discussion
• But too much complexity may be distracting

22
Future Work

• Much more to learn about flash forums
• Textual analysis
• User goals
• Moderation systems
• Opportunities for better interfaces
• Multi-dimensional filtering
• Anti-filtering emphasize novelty, variety

23
Thanks!