Subjectivity and Sentiment Analysis: from Words to Discourse

1
Subjectivity and Sentiment Analysis from Words
to Discourse

• Jan Wiebe
• Computer Science Department
• Intelligent Systems Program
• University of Pittsburgh
• I2R Singapore 2009

2
Burgeoning Field

• Quite a large problem space
• Several terms reflecting varying goals and models
• Sentiment Analysis
• Opinion Mining
• Opinion Extraction
• Subjectivity Analysis
• Appraisal Analysis
• Affect Sensing
• Emotion Detection
• Identifying Perspective
• Etc.

3
What is Subjectivity?

• The linguistic expression of somebodys opinions,
  sentiments, emotions, evaluations, beliefs,
  speculations (private states)

Private state state that is not open to
objective observation or verification Quirk,
Greenbaum, Leech, Svartvik (1985).
Note that this particular use of subjectivity is
adapted from literary theory E.G. Banfield 1982,
Fludernik 1993 Wiebe PhD Dissertation 1990.
4
Examples of Subjective Expressions

• References to private states
• She was enthusiastic about the plan
• He was boiling with anger
• References to speech or writing events expressing
  private states
• Leaders rounding condemned his verbal assault on
  Israel
• Expressive subjective elements
• That would lead to disastrous consequences
• What a freak show

5
Manually (human) Annotated News Data Wilson PhD
Dissertation 2008
I think people are happy because Chavez has fallen
6
Subjectivity and Sentiment Analysis

• Automatic extraction of subjectivity (opinions)
  expressed in text or dialog (newspapers, blogs,
  conversations, etc)
• Sentiment analysis specifically looking for
  postiive and negative sentiments

7
Why?

• Subjectivity analysis systems can provide useful
  input to several kinds of end applications

8
Why? Opinion Question Answering

• Answer Questions about Opinions
• Q What is the international reaction to the
  reelection of Robert Mugabe as President of
  Zimbabwe?

Stoyanov, Cardie, Wiebe EMNLP05 Somasundaran,
Wilson, Wiebe, Stoyanov ICWSM07
9
Why? Information Extraction (AAAI

• Filter out false hits for Information Extraction
  systems
• The Parliament exploded into fury against the
• government when word leaked out

Riloff, Wiebe, Phillips AAAI05
10
Why? Recognizing Stances in Debates

• Firefox is more respectful of W3C internet
  standards while µsoft sucks by trying to force us
  to use their own standards to keep their
  monopoly.
• IE is much easier to use. It also is more
  visually pleasing. It is much more secure as well.

Pro-Firefox
Pro-IE
11
Why? Product Review Mining

• Determine if the given product/movie review is
  positive or negative
• was billed as a suspense thriller along the
  lines of Hitchcock ..... the problem here is that
  writing has failed some very capable actors ....
• The last half of the film is very well done .
  Another thing that carries this film are the
  superb performances ... is a very entertaining
  and suspenseful film...

Positive review
12
And Several Others

• Tracking sentiments toward topics over time Is
  anger ratcheting up or cooling down?
• Prediction (election outcomes, market trends)
  Will Clinton or Obama win?
• Meeting summarization What were the main
  opinions expressed?
• Etcetera!

13
Focus

• Our focus is linguistic disambiguation how
  should language be interpreted?
• Is it subjective in the first place? If so, is
  it positive or negative? What is it about? Etc.
• Subjective language is highly ambiguous

14
Interpretation
Lexicon of keywords out of context
Full contextual Interpretation of words in text
or dialogue
The dream
15
Interpretation
Lexicon of keywords out of context
Full contextual Interpretation of words in
text or dialogue
Brilliant Difference Hate Interest Love
16
Subjectivity Lexicons

• Most approaches to subjectivity and sentiment
  analysis exploit subjectivity lexicons.
• Lists of keywords that have been gathered
  together because they have subjective uses

17
Automatically Identifying Subjective Words

• Much work in this area
• E.g. Hatzivassiloglou McKeown ACL97 Wiebe
  AAAI00 Turney ACL02 Kamps Marx 2002 Wiebe,
  Riloff, Wilson CoNLL03 Kim Hovy 2005 Esuli
  Sebastiani 2005

Subjectivity Lexicon http//www.cs.pitt.edu/mpqa
Entries from several sources (our work and
others)
18
However

• Consider the keyword Interest.
• It is in the subjectivity lexicon.
• But, what about interest rate, for example?

19
Dictionary Definitions senses

• Interest, involvement -- (a sense of concern
  with and curiosity about someone or something
  "an interest in music")
• Interest -- (a fixed charge for borrowing
  money usually a percentage of the amount
  borrowed "how much interest do you pay on your
  mortgage?")

20
Dictionary Definitions senses

• Interest, involvement -- (a sense of concern
  with and curiosity about someone or something
  "an interest in music")
• Interest -- (a fixed charge for borrowing
  money usually a percentage of the amount
  borrowed "how much interest do you pay on your
  mortgage?")

21
Senses

• Even in subjectivity lexicons, many senses of the
  keywords are objective 50 in our study!
• Thus, many appearances of keywords in texts are
  false hits

22
Senses

• His alarm grew as the election returns came in.
• He set his alarm for 7am.
• His trust grew as the candidate spoke.
• His trust grew as interest rates increased.

23
WordNet Miller 1995 Fellbaum 1998
24
Examples

• There are many differences between African and
  Asian elephants.
• dividing by the absolute value of the
  difference from the mean
• Their differences only grew as they spent more
  time together
• Her support really made a difference in my life
• The difference after subtracting X from Y

25
Subjectivity Sense Labeling

• Automatically classifying senses as subjective or
  objective

Wiebe Mihalcea ACL06 Gyamfi, Wiebe, Mihalcea,
Akkaya NAACL09 See also Esuli Sebastiani
EACL06, ACL07 Andreevskaia
Bergler EACL06, LREC06 Su
Markert Coling08, NAACL09
26
WordNet
27
WordNet
28
WordNet
If this sense is subjective, then maybe these
senses of brainy and smart-as-a-whip are as well
29
WordNet glosses
30
WordNet Examples
Glosses and examples contain clues as to the
subjectivity of a sense
31
WordNet Relations
32
WordNet Relations
33
Hierarchical Structure
34
Using Hierarchical Structure
Information content of the lowest common subsumer
Sim(t,s) -log(p(c)) (Resnik 1995)
The higher the IC of the LCS, the more specific
it is, and the more similar the seed and target
sense are
LCS c
Target Sense t

• Being similar to a
• subjective seed
• More likely the
• target is subjective

Seed Sense s
35
Using Hierarchical Structure
LCS
Target sense
Seed sense
36
Using Hierarchical Structure
LCS
voice1 (objective)
37
Sense Subjectivity LCS Feature
LCS c4
LCS c3
LCS c2
LCS c1
Seed Sense s4
Seed Sense s3
Seed Sense s2
Target Sense t
Seed Sense s1
38
Domains

• Several researchers have noted that subjectivity
  may be domain specific
• WordNet Domains (Gliozzo et al. 2005) assigns a
  domain label to each synset

39
Domains

• Over 80 of the subjective seed senses are in 6
  domains (rest are in 35)
• Factotum other 201 garishness2, racism1
• Psychological features 98 horror1,
  satisfaction1
• Person 68 meanie1,Francophobe1
• Law 61 swindler1, two-timer1
• Psychology 20 ecstasy1,indignity1
• Sociology 20 vandalism1,odium1

40
Sense Subjectivity LCS Feature
LCS c4
Saves computation
LCS c3
The score is the feature value for t
LCS c2
LCS c1
Seed Sense s4
Seed Sense s3
Domain D
Seed Sense s2
Target Sense t
Seed Sense s1
41
Using Hierarchical Structure Gyamfi, Wiebe,
Mihalcea, Akkaya NAACL09

• Hierarchical information is combined with other
  WordNet-Based knowledge to classify senses as
  Subjective or Objective

42
Interpretation
Lexicon of keywords out of context
Full contextual Interpretation of words in text
or dialog
continuum
Brilliant sense1 S sense2 S
Difference sense1 O sense2 O sense3
S sense4 S sense5 O
Now we will leave the lexicon and look at
disambiguation in the context of a text
or conversation
43
Contextual Subjectivity Analysis
Subjectivity Sentence Classifier
He spins a riveting plot which grabs and holds
the readers interest
S O?
Do the sentences contain subjectivity?
S O?
The notes do not pay interest.
E.g. Riloff Wiebe EMNLP03 Yu
Hatzivassiloglou EMNLP03
44
Contextual Subjectivity Analysis
Subjectivity Phrase Classifier
He spins a riveting plot which grabs and holds
the readers interest
S O?
Is a phrase containing a keyword subjective?
S O?
The notes do not pay interest.
Wilson, Wiebe, Hoffmann EMNLP05
45
Contextual Subjectivity Analysis
There are many differences between African
and Asian elephants.
S O?
Sentiment Phrase Classifier
Pos, Neg, Neutral?
Is a phrase containing a keyword
positive, Negative, or neutral?
Well return to this, topic after next. But first
Their differences only grew as they spent
more time together
S O?
Pos, Neg, Neutral?
Wilson, Wiebe, Hoffmann EMNLP05
46
Interpretation
Lexicon of keywords out of context
Full contextual Interpretation of words in text
or dialog
continuum
Brilliant sense1 S sense2 S
Difference sense1 O sense2 O sense3
S sense4 S sense5 O
Contextual Subjectivity analysis
Exploiting sense labels to improve the contextual
classifiers
47
Subjectivity Tagging using WSD
Subjectivity Classifier
He spins a riveting plot which grabs and holds
the readers interest
S O?
S O?
The notes do not pay interest.
48
Subjectivity Tagging using WSD
Subjectivity Classifier
He spins a riveting plot which grabs and holds
the readers interest
S O
Sense 4
WSD System
S O
Sense 1
The notes do not pay interest.
49
Examples

• There are many differences between African and
  Asian elephants. Sense1 O
• dividing by the absolute value of the
  difference from the mean Sense2 O
• Their differences only grew as they spent more
  time together Sense3 S
• Her support really made a difference in my life
  Sense4 S
• The difference after subtracting X from Y
  Sense5 O

50
Examples

• There are many differences between African and
  Asian elephants. Sense1 O
• dividing by the absolute value of the
  difference from the mean Sense2 O
• Their differences only grew as they spent more
  time together Sense3 S
• Her support really made a difference in my life
  Sense4 S
• The difference after subtracting X from Y
  Sense5 O

Or one of these?
51
Subjectivity Tagging using Subjectivity WSD
Subjectivity Classifier
There are many differences between African
and Asian elephants.
S O?
S O?
Their differences only grew as they spent
more time together
52
Subjectivity Tagging using Subjectivity WSD
Subjectivity Classifier
There are many differences between African
and Asian elephants.
S O
SWSD System
S O
Their differences only grew as they spent
more time together
53
SWSD Akkaya, Wiebe, Mihalcea EMNLP09

• SWSD Performance is well above baseline and the
  performance of full WSD
• SWSD is a feasible variant of WSD
• Subjectivity provides a natural course-grained
  sense grouping

54
SWSD in Subjectivity Tagging

• SWSD exploited to improve performance of
  subjectivity analysis systems
• Both S/O and Pos/Neg/Neutral classifiers

55
Sentiment Analysis using SWSD
There are many differences between African
and Asian elephants.
SWSD System
Their differences only grew as they spent
more time together
56
Interpretation
Lexicon of keywords out of context
Full contextual Interpretation of words in text
or dialog
continuum
Brilliant sense1 S sense2 S
Difference sense1 O sense2 O sense3
S sense4 S sense5 O
SWSD
Contextual Sentiment Analysis
Rest of the talk contextual processing not
bound to word senses
Return to contextual sentiment classification
57
Sentiment Analysis Wilson, Wiebe, Hoffman
EMNLP05, Computational Linguistics 2009

• Automatically identifying positive and negative
  emotions, evaluations, and stances
• Our approach classify expressions containing a
  keyword as positive, negative, both, or neutral

58
Phrase-Level Sentiment Analysis

• See also, E.G. Yi, Nasukawa, Bunescu, Niblack
  ICDM03 Polanyi Zaenen AAAI-SS04 Popescu
  Etzioni EMNLP05 Suzuki, Takamura, Okumura
  CICLing06 Moilanen Pulman RANLP07 Choi
  Cardie EMNLP08

59
Prior versus Contextual Polarity

• Many subjectivity lexicons contain polarity
  information
• Prior polarity out of context, positive,
  negative, or neutral
• A word may appear in a phrase that expresses a
  different polarity in context
• Contextual polarity

60
MPQA (Human) Polarity Annotations

• Judge the contextual polarity of the sentiment
  that is ultimately being conveyed in the context
  of the text or conversation

61
Contextual Interpretation

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

62
Contextual Interpretation

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

63
Contextual Interpretation

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

64
Contextual Polarity is Complex

• They have not succeeded, and will never succeed,
  in breaking the will of this valiant people.

65
Approach

• Step 1 Neutral or Polar?
• Step 2 Are the polar instances Positive or
  Negative?
• Combine a variety of evidence

66
Evidence

• Modifications and Conjunctions
• Cheers to Timothy Whitfield for the wonderfully
  horrid visuals
• Disdain and wrath
• Hatzivassiloglou McKeown ACL97
• Subjectivity of the surrounding context
  syntactic role in the sentence etc.

wonderfully horrid
67
Polarity Influencers

• Negation
• Local not good
• Longer-distance dependencies
• Does not look very good (proposition)
• No politically prudent Israeli could support
  either of them (subject)
• Phrases with negations may intensify instead
• Not only good, but amazing!

68
Polarity Influencers

• Modality
• No reason at all to believe that the economy is
  good

69
Polarity Influencers

• Contextual Valence Shifters Polanyi Zaenan 2004
  
• General polarity shifter
• Pose little threat
• Contains little truth
• Negative polarity shifters
• Lack of understanding
• Positive polarity shifters
• Abate the damage

70
Approach

• Step 1 Neutral or Polar?
• Step 2 Are the polar instances Positive or
  Negative?
• Combine a variety of evidence
• Still much to do in the area of recognizing
  contextual polarity

71
Interpretation
Lexicon of keywords out of context
Full contextual Interpretation of words in text
or dialog
continuum
Brilliant sense1 S sense2 S
Difference sense1 O sense2 O sense3
S sense4 S sense5 O
SWSD
Contextual Sentiment Analysis
Discourse
72
Discourse-Level Opinion Interpretation I Opinion
FramesSomasundaran, Wiebe, Ruppenhofer COLING08
Somasundaran, Namata, Wiebe, Getoor EMNLP09

• Interpretations involving multiple sentences
  within the discourse
• Opinion Frames are composed of 2 opinions and the
  relation between their targets (what they are
  opinions of)
• Larger structures emerge from interdependent
  frames
• Data task-oriented dialogues

73
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD

Like and must are clear positive clues
74
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD

targets what the opinion is about
75
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD
• The LCD is traditional

76
Discourse-Level Opinion Interpretation

• I like the LCD feature
• We must implement the LCD
• The LCD is traditional

Joint Interpretation of opinions in the discourse
If a coherent discourse expressing one overall
opinion ? traditional is also positive
77
Discourse-Level Opinion Interpretation

• Shapes should be curved, so round shapes.
  Nothing square-like.
• ... So we shouldnt have too square corners and
  that kind of thing.

78
Discourselevel interaction between opinions
Direct opinion
Direct opinion

• Shapes should be curved, so round shapes.
  Nothing square-like.
• ... So we shouldnt have too square corners and
  that kind of thing.

Direct opinion
79
Discourselevel interaction between opinions
Direct opinion
Direct opinion

• Shapes should be curved, so round shapes.
  Nothing square-like.
• ... So we shouldnt have too square corners and
  that kind of thing.

Direct opinion
What were the opinions regarding the curved
shape? Will the curved shape be accepted?
80
Discourselevel interaction between opinions
Opinions towards mutually exclusive option
(alternative)
Direct opinion

• Shapes should be curved, so round shapes.
  Nothing square-like.
• ... So we shouldnt have too square corners and
  that kind of thing.

Opinions towards mutually exclusive option
(alternative)
81
Discourselevel interaction between opinions
Opinions towards mutually exclusive option
(alternative)
Direct opinion

• Shapes should be curved, so round shapes.
  Nothing square-like.
• ... So we shouldnt have too square corners and
  that kind of thing.

Opinions towards mutually exclusive option
(alternative)
Opinions towards the square shapes reveal
additional information about the speakers
opinion of the curved shape
82
Discourse-level analysis for Recognizing stances
in debates Somasundaran Wiebe ACL-IJCNLP09

• Use our understanding of discourse-level opinion
  relations to recognize stances in debates
• Slides, part 2