The Performance of Distributed News Aggregators

1
The Performance ofDistributed News Aggregators

• Wolf Richter Tobias Escher
• Oxford Internet Institute, University of Oxford
• republica
• 2 April 2008

2
Coming up

• Introduction to Netizen Journalism as an
  alternative to Traditional media
• Taxonomy of news aggregators
• Our methodology to assess performance
• Our Findings
• Remaining issues

3
Calling at

• Introduction to Netizen Journalism as an
  alternative to Traditional media
• Taxonomy of news aggregators
• Our methodology to assess performance
• Our findings
• Remaining issues

4
Critique of traditional or mainstream media

• Mass media effects on public
• opinion
• behaviour
• Gatekeeping
• Corruption
• Decline in quality
• Overall, our research suggests that 60 of press
  articles and 34 of broadcast stories come wholly
  or mainly from one of these pre-packaged
  sources. Lewis et al (2008)

5
The Internet as a Counter Public, Netizen
Journalism as the 5th Estate

• Netizen-journalism comes with big promises
• Reliable fact-checking
• Self-selection / become your own editor
• Empowerment of formerly passive audiences
• Ordinary citizens (Hindman)
• The blogosphere is smashing the old media
  monopoly and giving individuals power in the
  marketplace of ideas. (Hugh Hewitt Blog 2005)
• High profile cases have demonstrated the power of
  Netizen-journalism as a 5th estate
• Reutersgate
• Rathersgate
• Salman Pax
• US senator Trent Lott

6
New media environment empowers users at every
stage of the news production process
News production process

• Every user can gather and publish news on the
  Web

• Every user can search and aggregate news from an
  abundance of content available on the Web

Empower-ing effect
Criticism

• Dominance of old elites in the blogosphere
• Most publications taken from traditional media or
  trivial content

• Echo Chamber amplifies existing views and
  narrows perspectives

Source OII
7
What evidence do we have to fact-check the
promises of the participatory media environment?

• A-list bloggers traditional elites
• With only one exception, all of the bloggers in
  our census 75 A-list bloggers were either
  educational elites, business elites, technical
  elites, or traditional journalists. (Hindman,
  forth.)
• Major part of content about what I had for
  lunch
• 76 of bloggers say a reason they blog is to
  document their personal experiences and share
  them with others Lenhart,Pew (2006)
• Partisanship / inaccuracy

8
What do we actually look at?
9
Calling at

• Introduction to Netizen Journalism as an
  alternative to the traditional media
• Taxonomy of news aggregators
• Our methodology to assess performance
• Our findings
• Remaining issues

10
A simple taxonomy of news aggregators
News reporting Production process News selection Case studies
Professional reporters create original content Filter then publish Editorial Board
Citizen reporters create original content Filter then publish Editorial Board
Users create original content or comment on content from other sources Publish then filter Editorial Board
Users submit content from other sources or comments Publish then filter Editorial Board and moderators
Users submit content from other sources Publish then filter Public vote
Automatic algorithm collects content from other of sources Publish then filter Algorithm
11
Calling at

• Introduction to Netizen Journalism as an
  alternative to Traditional media
• Taxonomy of news aggregators
• Our methodology to assess performance
• Our findings
• Remaining issues

12
Measures of performance for news aggregators

• Appropriate quality measures for analysis depend
  on
• Function
• Target audience
• Genre (news, comment)
• Publication cycle
• Medium (print, television)
• Journalistic self-perception

13
Types of indicators for performance
performance
Held / Ruß-Mohl (2000)
14
PEJ study found Distributed News Aggregators to
be transitory and tech-heavy
Project for the Excellence in Journalism (2007)

• study compared distributed news aggregators (Digg
  News, del.icio.us, and Reddit) with aggregators
  of traditional news like Yahoo! News
• user-driven sites with no clear agenda
• In short, the user-news agenda, at least in this
  one-week snapshot, was more diverse, yet also
  more fragmented and transitory than that of the
  mainstream news media
• What were the favorite stories on the
  user-driven sites? For the most part, there were
  no dominant ones. The only story with any real
  traction was the release of the Apple iPhone, and
  that was just on one site (it accounted for 16
  of the stories on Digg that week). Otherwise,
  users put forth a mix of diverse and unconnected
  news events from day to day.
• user-driven sites more tech-heavy
• In a week when the mainstream press was focused
  on Iraq and the debate over immigration, the
  three leading user-news sitesReddit, Digg and
  Del.icio.uswere more focused on stories like the
  release of Apples new iphone and that Nintendo
  had surpassed Sony in net worth, according to the
  study.

15
How do we measure Performance?

• Timeliness and continuity chosen as performance
  measures
• Timeliness how often and when do new articles
  appear in the feed
• Continuity do different articles deal with the
  same topic
• Rationale
• Distributed News aggregators praised for flexible
  reaction to current news events no blackout of
  events due to frequency
• Blogosphere criticized for punctual and arbitrary
  focus of attention no set agenda no
  collective memory to pursue topics
• User sites criticized as being tech-heavy.
  Therefore distinguish World News and Technology
  News
• World News major events, emphasis on politics
  (e.g. US elections, war in Iraq)

16
Observed news sources
IHT Frontpage
IHT Technology and Media
BBC International Frontpage
BBC International Technology
DIGG World News
DIGG Technology
DIGG Technology Upcoming
DIGG All Popular
DIGG All Upcoming
OMN Main
OMN Technology
GV Main
GV Technology
SLASHDOT Main
17
Data collection process
Every 3 hours31st January 19th March
18
Calling at

• Introduction to Netizen Journalism as an
  alternative to Traditional media
• Taxonomy of news aggregators
• Our methodology to assess performance
• Our findings
• Remaining issues

19
World news Traditional media show higher
percentage of new articles per day
Percentage of new World news articles per day
20
Tech news Digg outperforms traditional media,
which still seem to do a decent job
Percentage of new Tech news articles per day
21
but lack even further behind if you compare the
absolute numbers of articles
Number of new Tech news articles per day
22
In absolute numbers the output of Traditional
media is higher for World News
Number of new World news articles per day
23
Timeliness World News show clearly visible
update cycles throughout the day
Percentage of new World News articles per cycle
(3h)
Weekend fatigue
24
Timeliness In Tech News more continuous update
activity visible at Digg and Slashdot
Percentage of new Tech News articles per cycle
(3h)
25
World news lifespan of articles
World News
Tech News
26
Summary findings Timeliness

• Traditional news show stronger performance
  (Percentage and absolute number of new articles)
  for World News and still fair performance for
  Tech News in comparison to distributed
  aggregators
• Both traditional as well as distributed
  aggregators have clearly defined update cycles
• Digg and Slashdot have more continous update
  patterns for their focus area technology
• Weekends and nights are marked by lower activity
  for both paradigms
• OhMyNews and Global Voices not competitive for
  World News and Tech News in timeliness of news
  production
• In their respective focus areas (World News and
  Tech News) BBC/IHT and Digg/Slashdot show
  remarkably similar patternsCritical vs
  subcritical mass?

27
Publish, then filter the example of Digg
filter for all content
filter for technology content
28
Digg Lifespan of articlessubmitted vs. filtered
mean(stddev) Upcoming Popular
All 0.1h (0.05) 3.9h (2.1)
Technology 0.22h (0.11) 25.4h (8.8)
every hour4 new articles
every hour200 new articles
every hour1 new articles
every hour100 new articles
29
Summary findings Diggs timeliness

• On average, 55 of stories are replaced on Diggs
  Front page (All popular) per update cycle.
• Wu Huberman (2007) This characteristic time
  is consistent with the fact that a story usually
  lives on the front page for a period between 1
  and 2 hours.
• However, some articles survive longer as during a
  day not all articles are replaced.
• Community applies tougher filter to submitted
  technology news than to world news
• 2 of articles make it from upcoming to popular
  for World News
• 1 of articles make it from upcoming to popular
  for technology news
• For Tech News, Digg produces a more stable
  agenda than for World News. Claims that
  distributed aggregators do not have an agenda
  should be corrected by looking at self-selected
  focus area of community

30
Continuity

• PEJ study user-news agenda () more fragmented
  and transitory than that of the mainstream news
  media
• collection process
• download articles
• Yahoo Term extractor to reduce to important key
  phrases
• further reduction by filtering stopwords
  stemming

They say representatives of Mr Mugabe, military
chiefs and the opposition have held meetings
chaired by South Africa since Saturday's
elections. The sources say Mr Mugabe is to give
an address to the nation but urge caution until
the announcement has been made. Zimbabwe and
South Africa's governments denied any
negotiations or a deal. But the US embassy in
the capital Harare said Mr Mugabe was expected to
address the people of Zimbabwe shortly. Claim
and counter-claim Western diplomats had been
summoned to a meeting with the government, the
embassy told the BBC.
president mugage zimbabwe election political
parties ....
mugabe zimbabwe elect politic part ....
31
Continuity Share of articles per key phrase
World News World News Technology Technology
unique phrases mean (stddev) unique phrases mean (stddev)
BBC 71 .29 (.0028) 87 1.90 (.0129)
IHT 71 .26 (.0031) 84 1.26 (.0102)
Digg 94 1.41 (.0051) 86 .51 (.0063)
OhMyNews 90 .89 (.0043) 98 19.23(0769)
Global Voices 91 1.42 (.0172) 96 8.37 (.0769)
Slashdot 88 .41 (.0032)
32
Summary Continuity

• World News
• Distributed more fragmented narrower agenda
• Traditional less fragmented, more connected
  broader agenda
• For Tech News
• Digg and Slashdot show best performance in this
  area
• BBC and IHT show average performance
• overall performance of traditional models not
  reached by new distributed models

33
Who captures the benefits created by the new
Participatory Media Environment?
Benefits

• Get access to news specifically catering to their
  interests

Users
Source OII
34
Calling at

• Introduction to Netizen Journalism as an
  alternative to the traditional media
• Taxonomy of news aggregators
• Our methodology to assess performance
• Our findings
• Remaining issues

35
Remaining issues

• OhMyNews International not as successful as
  original Korean version
• Hard to find a representative for the
  Blogosphere
• Measurement of continuity needs improvement
• Better keyword extraction
• Better matching algorithm
• Manual check

36
THANK YOU!

• Questions now gt ASK!
• Questions later gt WRITE!
• Wolf.Richter_at_oii.ox.ac.uk
• Tobias.Escher_at_oii.ox.ac.uk
• and read our blogs )
• http//people.oii.ox.ac.uk/escher/
• http//people.oii.ox.ac.uk/richter/

37
THE END
38
References

• Hindman, M. S. (forthcoming). Voice, Equality,
  and the Internet.
• Held, B. and S. Ruß-Mohl (2000). Qualität durch
  Kommunikation. Qualität durch Kommunikation
  sichern. Vom Qualitätsmanagement zur
  Qualitätskultur. Erfahungsberichte aus Industrie,
  Dienstleistung und Medienwirtschaft. B. Held and
  S. Ruß-Mohl. Frankfurt am Main.
• Lewis, J., A. Williams, et al. (2008). The
  Quality Independence of British Journalism.
  MediaWise Report, Cardiff University.
• Lenhart, A. and S. Fox (2006). Bloggers A
  portrait of the internet's new storytellers.
  Washington D.C, Pew Internet American Life
  Project.
• Project for Excellence in Journalism (2007). The
  Latest News HeadlinesYour Vote Counts, Project
  for Excellence in Journalism Understanding News
  in the Information Age.
• Schmidt, J. (2007). Online Mainstream Media and
  Weblogs Competing or Complementary Publics.
  Boundaries of Free Speech? Jerusalem, Israel.
• Schmidt, J., M. Paetzolt, et al. (2006).
  Stabilität und Dynamik von Weblog-Praktiken?
  Ergebnisse der Nachbefragung zur Wie ich
  blogge?!-Umfrage. Berichte der Forschungsstelle
  Neue Kommunikationsmedien. Bamberg, Germany.

39
Feeds
IHT Frontpage http//www.iht.com/rss/frontpage.xml
IHT Technology and Media http//www.iht.com/rss/technology.xml
BBC International Frontpage http//newsrss.bbc.co.uk/rss/newsonline_world_edition/front_page/rss.xml
BBC International Technology http//newsrss.bbc.co.uk/rss/newsonline_world_edition/technology/rss.xml
DIGG World News http//www.digg.com/rss/indexworld_news.xml
DIGG Technology http//www.digg.com/rss/containertechnology.xml
DIGG Technology Upcoming http//www.digg.com/rss/containertechnologydig.xml
DIGG All Popular http//www.digg.com/rss/index.xml
DIGG All Upcoming http//www.digg.com/rss/indexdig.xml
OMN Main http//media.ohmynews.com/rss/eng_ohmynews.xml
OMN Technology http//media.ohmynews.com/rss/eng_ohmynews_class4.xml
GV Main http//www.globalvoicesonline.org/-/weblog/feed/
GV Technology http//www.globalvoicesonline.org/-/topics/technology//feed/
SLASHDOT Main http//rss.slashdot.org/Slashdot/slashdot