Quasi-semantic Property IV: Sentiment and 7
7.2 Automatic algorithms for sentiment and opinion detection
this phenomenon are discussed. Furthermore, it is also shown that using the tech-nique in another context (here: news data analysis) can result in interesting findings even if this measures a slightly different quasi-semantic property.
• Visual analysis of the detected opinions
While there are many related approaches that automatically extract an opinion from customer feedback data, only few techniques exist for analyzing the documents with respect to the extracted properties. In this chapter, we are going to introduce a novel visual analysis method called summary reports that provides a compact but yet sufficiently detailed overview to compare the reviews of several products to each other. Furthermore, the Literature Fingerprinting technique that is introduced in chapter 4 is applied in the context of news feed analysis. Finally, we explore a set of customer reviews with respect to subgroups with similar opinions.
7.2 Automatic algorithms for sentiment and opinion detection
Section 7.2.1 presents a simple heuristic for detecting the sentiment of a text. Section 7.2.2 explains the basic algorithm for opinion detection that is employed in this thesis. Part of this process is the mapping between attributes and sentiment. Sections 7.2.3 and 7.2.4 present two different heuristics for performing this mapping: one is statistically-based, the other one incorporates linguistic knowledge about the sentence structure.
7.2.1 Algorithm for sentiment detection
To get a sentiment score for each article, a basic sentiment analysis algorithm was applied.
With the help of two lists with opinion signal words (one with negative signal words and the other with positive ones) from the General Inquirer Project [46], each word is classified as positive, negative or neutral1. We count the number of positive signal words in an article and subtract the number of negative signal words from it. To improve the accuracy of the method, negation is taken into account. This is done by inverting the value of a word if in a maximum distance ofX words a negation signal word is found (such as “no”, “not”,
“without”, . . .). In this case, the parameter X (the maximum distance to the negation signal word) was set to 3, a value that experimentally proved as minimizing the failures.
For alternative approaches please refer to section 3.5.
7.2.2 Attribute-based opinion analysis
Attribute-based opinion analysis is often done as a two-step process (in our case three steps):
1Note that the list contains signal words of all parts of speech. That means, not only opinion-bearing adjectives, but also nouns, verbs, etc. (e.g. “catastrophe”, “to like”).
Step 1: First, the opinion signal words and the attributes that are commented on are identified. In our experiments, we use the algorithm that is presented in chapter 6 to compile a list of attributes that are frequently commented on. This list is then used to identify mentions of attributes in the reviews. Note that alternative approaches for de-tecting attributes exist that could easily be substituted in our process (see section 3.4). As an opinion word list, we use the General Inquirer dictionary that was already introduced in the previous subsection.
Step 2: After annotating all the occurrences of attributes and opinion signal words in the text, a mapping between attributes and opinion words is performed. This means that we determine for each attribute in a sentence which opinion word(s) refer to it, clas-sifying each attribute as positive, neutral, or negative. Thus, we need to combine the quasi-semantic measure product attribute with the text feature opinion signal word that was already used in the previous subsection. Note that it is important to take the category
“neutral” into account as well, because in real-world data sets an attribute can also be mentioned in a non-evaluative way. In the next subsections, two different ways of per-forming the mapping are introduced.
Step 3: In our application scenario, we do not only want to know whether an attribute was mentioned positively or negatively in a specific sentence. Instead, we are interested in the overall opinion that was expressed about the attribute in the review. If an attribute is mentioned several times in the review, the majority vote of the sentence polarities for this attribute is determined to get its opinion value on the review-level.
Result: As a result, we get a feature vector for each review that summarizes the expressed opinions on the individual attributes. For each attribute, there is one feature dimension in the vector. The corresponding value of the vector for a particular attribute’s dimension indicates whether the attribute was mentioned positively (+1), negatively (-1) or neutrally / not at all (0).
Figure 7.1 exemplifies the process.
7.2.3 Statistically-based mapping
As a statistically-based approach to perform the mapping, we use a method that is sim-ilar to the one presented in [66, 28]. (The main difference is that we use an additional cutoff value and need attribute scores on document level.) The algorithm is based on the assumption that the closer an opinion signal word is to an attribute, the higher is the probability that it refers to the attribute. The opinion score for each attribute is therefore calculated by building a weighted sum of all opinion signal words that are in the same sentence than the attribute. The weight is determined by a weighting function that takes the distance to the attribute into account. An example for such a weighting function is given in equation 7.1.
rd-weight(A,o)=
1 if dist(A,o)≤cutoff/2,
0.5 if cutoff/2<dist(A,o)≤cutoff, 0 else.
(7.1)
7.2. Automatic algorithms for sentiment and opinion detection 119
Step 1: Identification of attributes and sentiment
I feel obligated to counter the badreviews. This printer is just fine. I don’t know what people are complainingabout regarding the software, but it installed seamlesslyand is intuitivein its operation. Even though the paper tray jams sometimes altogether I am happythat I bought this wonderfulprinter.
Step 2: Mapping between attributes and sentiment
[…] Even though the paper tray jamssometimes altogether
Iam happythat I bought this wonderfulprinter.
? ?
?
Step 3: Determining the overall sentiment of an attribute
[…] This printeris just fine. […] Even though the paper tray jams sometimes altogether I am happy that I bought this wonderful printer.
Overall sentiment for printer: positive Resulting Feature Vector
Printer Ink Software Paper tray Price
1 0 1 -1 0
Figure 7.1: The graphic above illustrates the different steps in the opinion analysis process. At-tributes are highlighted in bold face. Opinion signal words are colored in blue if they are positive and in red if they are negative.
Instead of a step function like the one above, other weighting functions, e.g. a gaussian function could be used as well. Our experiments showed that gaussian functions provide equally good results given that a cutoff-value is used that defines how many words before and after the attribute are taken into account at most. In our application, the cutoff threshold was set to 4. Experiments showed that this cutoff value is especially important in long sentences, because it prevents errors that are caused by very distant opinion signal words that are incorrectly mapped to the attribute. Instead of using a cutoff-value, [28]
proposes to determine sentence segments by searching for so called BUT-words/phrases (e.g. “but”, “except that”, . . .) that tend to invert the sentiment that is expressed in a sentence. Within a sentence segment, all the opinion signal words are taken into account and weighted by equation 7.2:
rd-weight(A,o)= 1
dist(A, o) (7.2)
In order to get an opinion value for an attribute A and a sentence S equation 7.3 is applied. The weighted polarity values for each opinion signal word o within S are summed up. The polarity value of an opinion signal word is either +1 or -1 depending on whether it is contained in the positive or negative word list. Thereby, negation is taken into account in the way described in section 7.2.1, inverting the polarity of an opinion signal word.
opinion-score(A,S)=X
o∈S
rd-weight(A,o)·polarity(o) (7.3)
If an attribute A gets a positive opinion score, the sentence is interpreted as talking positively about the attribute. Likewise, if this sum is negative, the sentence is supposed to talk in a negative manner about the attribute. If the sum is equal to 0, the polarity of the closest opinion signal word is decisive. Note that the opinion-score value cannot be used to determine the strength of the opinion, because the distance or the number of opinion signal words in a sentence do not necessarily correlate with intensity of opinion.
7.2.4 Linguistically-based mapping
camera (NN)
The (DT) det
useful (JJ) nsubj
has (VBZ) rcmod
is (VBZ)
cop otherwise (RB)
advmod
not (RB) neg
that (WDT) rel
size (NN) dobj
a (DT) det
convenient (JJ) amod Flow of positive affection Flow of
negated affection
opinion word
attribute negation
word tag (DT)
det grammatical relation
Figure 7.2: Output of the dependency parser for the input sentence “The camera that has a con-venient size is otherwise not useful.”
Additionally to the previously presented technique, we tried to gain as much as possible from including linguistic knowledge. It has to be assumed that the approach of measuring the distance between an attribute and an opinion signal words fails in sentences with a complex nesting of subordinate clauses. We therefore use a dependency graph built by the Stanford Parser [77, 76] to analyze the grammatical structure (incl. negations) of the input sentences. In figure 7.2 an example output of the parser for the sentence “The camera that has a convenient size is otherwise not useful.” is depicted. Different line types and shadings represent properties that have to be determined. First, we check which of the nodes (representing each of the tokens) is an opinion word and retrieve its polarity from a given opinion word list. In figure 7.2, all opinion words are painted as nodes with a dashed line. Afterwards, we mark each edge (representing a grammatical relationship between two tokens) that is negating or passing affections on. For instance, determiners would not let affections pass through, whereas the relative clause modifier (“rcmod”) would2.
After building our representation of the sentence structure, we traverse the graph starting from the opinion words and propagate their polarity through the graph. The influence of an opinion word decreases with increasing graph distance and the polarity is negated whenever
2Complete list of passing edges: conj (conjunct), amod (adjectival modifier), nsubj (nominal subject), dobj (direct object), prep (prepositional modifier), rcmod (relative clause modifier), xcomp (clausal com-plement with external subject), nn (noun compound modifier), dep (dependent), cop (copula), ccomp (clausal complement with internal subject). The following edges negate the opinion: neg (negation modi-fier), conj neg (negating conjunction), conj but (but conjunction).
7.3. Evaluation and improvement of the QSPs “Sentiment” and “Opinion” 121
negating edges are passed. Note that this traversal is done for each of the opinion words and consequently the polarities weighted by distance are summed up. Considering the previously mentioned example sentence, we would propagate a positive polarity from the token “convenient” to the attributes “size” and “camera”. Additionally, a negated positive polarity would be propagated from “useful” to the attributes “camera” and “size”. As the influence of the opinion words decreases with increasing graph distance, the resulting polarity for “camera” would be negative and for “size” positive.
To be able to work with compound nouns as well, we group tokens together which are directly connected in the dependency graph via a “NN”- (compound noun) or “amod”-(modifier) edge. The affection of those compound attributes is the average affection of all their tokens.
To the best of our knowledge, this is the first attempt to use the result of a dependency parser to measure the distance between an opinion signal word and an attribute. However, it is not the first approach that attempts to improve sentiment analysis techniques with a dependency parser. E.g., Ng et al. [99] use the output of the parser to get special n-grams that represent adjective-noun, subject-verb, or verb-object relations for their document-level sentiment analysis. Zhuang et al. [164] parse a sentence for candidate attributes and opinion words using a keyword list and subsequently, with the help of dependency relation templates, classify the attribute-opinion pairs as forming a valid attribute-opinion pair or not. Templates, such as attribute modifies noun, are automatically extracted from a training data set. While [99] states that the results are similar to the ones achieved with traditional n-gram feature vectors, Zhuang et al. report on significant improvements over comparative methods without a dependency parser. In contrast to [164] (which is also an attribute-based method), we use the dependency tree to measure the distance between two words in a way that takes the syntactic structure of the sentence into account but do not specify any templates.