Polling the Blogosphere: a RuleBased Approach to Belief Classification

1
Polling the Blogosphere a Rule-Based Approach to
Belief Classification

• Jason Kessler
• Indiana University, Bloomington

2
Belief Analysis of Blogs

• Polling the blogosphere on a controversial
  proposition
• Literal search on a proposition (e.g., Obama is
  electable)
• Which blog entries contain assert it? Which deny
  it?
• Aggregate results
• 243 bloggers assert it
• 616 bloggers deny it

3
Motivating Example

• Polling for the Moon landings were staged
• The theory that the Moon landings were staged is
  complete nonsense.
• The writer denies the Moon landings were
  staged.

4
Motivating Example

• If Obama is electable, the country is in good
  shape.
• Writer takes no stance toward Obama is
  electable.

5
Problem

• When a writer uses a declarative finite clause,
  does that writer assert, deny, or take no stance
  toward its truth value?
• This is the problem of identifying a writers
  stance toward a proposition.
• Veridicity or facticity of a proposition.

6
Example

• Everybody is sad that the bar closed.
• The writer asserts the bar closed.
• Belief ! Sentiment
• Negative sentiment toward the bar closed
• Positive stance.

7
Outline

• System Description
• Given a proposition, sentence
• Dependency Parse
• Syntactic Representation
• Hand written patterns over semantic classes
• Veridicality Elements
• Veridicality Transformations
• Evaluation
• Proof of concept
• Promising results

8
Dependency Parse

• Pipeline Stages
• Dependency Parse
• Tag Veridicality Elements
• Apply Veridicality Transformations

The theory that the Moon landings were staged is
complete nonsense.
9
Veridicality Elements (VEs)

• Pipeline Stages
• Dependency Parse
• Tag Veridicality Elements
• Apply Veridicality Transformations

The theory that the Moon landings were staged is
complete nonsense.
10
Veridicality Transformations (VTs)

• Pipeline Stages
• Dependency Parse
• Tag Veridicality Elements
• Apply Veridicality Transformations

The theory that the Moon landings were staged is
complete nonsense.
11
Veridicality Transformations (VTs)

• Pipeline Stages
• Dependency Parse
• Tag Veridicality Elements
• Apply Veridicality Transformations

The theory that the Moon landings were staged is
complete nonsense.
12
System Structure Veridicality Elements

• Find expressions that have the potential of
  changing the truth-value of a proposition or
  referring to it
• Different classes affect truth values differently
• Examples
• Assertion Positive
• The assertion that the sky is blue
• Nonsense Negative
• The idea that the sky is orange is nonsense
• If Neutral
• Pretend Counter-factive

13
Finding Veridicality Elements

• Manually created seed sets
• Search web for patterns likely to contain VEs
• I agree with the assertion that
• I with the assertion that
• I quibble with the assertion that
• I take issue with the assertion that
• Manually classify matches, form new queries
• I take issue with the that
• I take issue with the argument that
• Similar to Brin (1998)

14
System StructureVeridicality Transformations

• Relate these expressions to propositions
• Some expressions will not be related to
  propositions
• Why bag-of-Veridicality-Elements fails
• Templates over dependency graphs
• Select for a VE class and a proposition

15
System StructureVeridicality Transformations

• Examples
• Expression is a main verb, proposition is its
  comp. clause
• John pretended the monkey was harmless.
• Cleft construction, expression is an adjective
• It is inconceivable that two plus two equals five.

16
Another Example

• Pipeline Stages
• Dependency Parse
• Tag Veridicality Elements
• Apply Veridicality Transformations

If Bob goes to school, he realizes the Earth is
round.
17
Another Example

• Pipeline Stages
• Dependency Parse
• Tag Veridicality Elements
• Apply Veridicality Transformations

If Bob goes to school, he realizes the Earth is
round.
18
Another Example

• Pipeline Stages
• Dependency Parse
• Tag Veridicality Elements
• Apply Veridicality Transformations

If Bob goes to school, he realizes the Earth is
round.
19
Evaluation

• Primitive, proof-of-concept evaluation
• Can we poll the blogosphere?
• Google blog search for abortion is murder
• Unseen data
• Run the system on the first 100 hits.
• See if it does better baseline.

20
Evaluation

• Exclude a number of results
• Spam blogs
• Long, unparsable sentences
• Trivial sentences (no VEs)
• Abortion is murder!
• Questions

21
Evaluation

• Corpus Statistics
• 48 Sentences
• 27 positive
• 3 negative
• 18 neutral
• 39 classified correctly (81 accuracy)
• Majority class was positive, giving a baseline of
  56 accuracy

22
Related Work

• Nairn et al. (2006) focused on main verbs
• Complex behavior under negation
• Work on contextual polarity for sentiment
  analysis.
• Wilson et al. (2005)
• Statistical approach
• Polanyi and Zaenen (2006)
• Theoretical approach

23
Related Work

• Somasundaran et al. (2007)
• Statistical techniques used to detect presence of
  arguing in a sentence.
• Arguing writer takes a non-neutral stance
  toward some content

24
Future Work

• Annotate corpus
• Further testing
• Statistical approaches
• Augment VE/VTs
• Integrate Nairn et al. (2006)
• Take into account questions

25
Takeaways

• Belief analysis is a young field
• Bag-of-words is not enough
• Shallow linguistic methods show promise

26
Questions?

• Thank you.
• References
• Brin, S. 1998. Extracting patterns and relations
  from the world wide web. In WebDB Workshop at
  6th International Conference on Extending
  Database Technology, EDBT98.
• Nairn, R. Condoravdi, C. and Karttunen, L.
  2006. Computing relative polarity for textual
  inference. In ICoS-5.
• Polanyi, L. and Zaenen, A. 2005. Contextual
  valence shifters. In Shanahan, J. G. Qu, Y. and
  Wiebe J., eds,. Computing Attitude and Affect in
  Text.
• Somasundaran, S. Wilson, T. Wiebe, J. and
  Stoyanov, V. 2007. QA with attitude Exploiting
  opinion type analysis for improving question
  answering in on-line discussions and the news. In
  ICWSM.
• Wilson, T. Wiebe, J. and Hoffmann, P. 2005.
  Recognizing contextual polarity in phrase-level
  sentiment analysis. In HLT/EMNLP.

27
Implementation
Veridicality Element Classes
28
Veridicality Transformations