Compositional rules of Korean auxiliary predicates for sentiment analysis
Most sentiment analysis systems count the number of occurrences of sentiment expressions in a text, and evaluate the text by summing polarity values of extracted sentiment expressions. However, linguistic contexts of the expressions should be taken into account in order to analyze sentimental orientation of the text meticulously. Korean auxiliary predicates affect meaning of the main verb or adjective in some ways while attached to it in their usage. In this paper, we introduce a new approach that handles Korean auxiliary predicates in the light of sentiment analysis. We classify the auxiliary predicates according to their strength of impact on sentiment polarity values. We also define compositional rules of auxiliary predicates to update polarity values when the predicates appear along with sentiment expressions. This approach is implemented to a sentiment analysis system to extract opinions about a specific individual from review documents which were collected from various web sites. An experimental result shows approximately 72.6% precision and 52.7% recall for correctly detecting sentiment expressions from a text.
Keywords:
Auxiliary predicates, Compositional rules, Polarity value, Sentiment analysis1. Introduction
Sentiment analysis is a process of assigning polarity to an entity on which a writer expresses his/her opinion in a text. By evaluating the polarity values, we can capture the intended sentiment on the entity. There are two major approaches to sentiment analysis [1]. One of them is a lexicon-based approach which utilizes a lexicon containing sentiment words annotated with their polarity. The other one is referred as a classification approach which is designed based on a machine learning approach. It involves generating a classifier from sentiment-annotated training texts. In this experiment, we adopt the former in which a sentiment lexicon is utilized to automatically extract a writer’s opinions from a text. The lexicon-based approach is preferable to a machine learning approach when a contextual word should be considered in a sentiment analysis.
The accurate meaning of an expression can be determined when it is interpreted in conjunction with its contextual words. Similarly, the context of a sentiment expression affects the meaning of the expressions, which eventually influences the polarity value of the expression. One of the most salient examples is a negative word which reverses the meaning of the sentiment expression when it occurs next to the expression. In addition to negative words, Korean presents various types of auxiliary predicates which affect the meaning of the main verb or adjective at various linguistic phases. They change the meaning, not just semantically but also pragmatically [2]. Accordingly, some of the auxiliary predicates play a significant role in shifting the polarity of a sentiment expression. If the sentiment expression is a verb or an adjective and followed by an auxiliary, the final polarity of the expressions should be evaluated in conjunction with the auxiliary. The following examples show the cases:
[EX]
(1) 공연이 만족스럽지 않았어.
(I) was not satisfied with the concert.
(2) 깔끔한 척하다.
(One) pretends to be neat.
(3) 게임에 연승해야 한다.
(We) should win every game we played.
The underlined parts in the Examples are the auxiliary predicates led by a sentiment expression. An adjective ‘만족스럽다’(satisfying) in (1) has a positive meaning in isolation. The overall polarity of the sentence, however, should be negative because the auxiliary predicate ‘~지 않’(not) is attached to ‘satisfying’. In the second example, ‘깔끔하’(neat) has a positive meaning in most cases while the sentence (2) is interpreted as negative. The sentiment expressed with the adjective does not determine the polarity of the sentence due to the auxiliary predicate ‘척하’(pretend). The semantic orientation of the sentence (2) is rather closer to the negative direction. The example (3) has a modal auxiliary ‘~어야 하’(should/must) expressing an obligation. The seed sentiment expression ‘win every game’ clearly denotes a positive meaning. However, the auxiliary adds deontic modality to the sentence meaning, which has to be interpreted as unreality, that is, an obligation to be met. Since the statement is not about something real, the effect of auxiliary predicates on a polarity value should be taken into account before the sentiment of (3) is evaluated as positive.
In this paper, we first list up auxiliary predicates that should be dealt with for sentiment analysis. Then, we categorize the auxiliaries into predefined classes according to how they affect a polarity value of sentiment expressions they co-occur with. We present a set of compositional rules for adjusting a polarity value depending on the classes of auxiliary predicates.
We first explore related studies in Section 2, and we present the categories of Korean auxiliary predicates and composition rules according to their categories in Section 3. We, then present experimental results in Section 4, followed by the conclusion in Section 5.
2. Related Studies
There exist two main approaches to the problem of extracting sentiment automatically [1]. One is a lexicon-based approach and the other is a machine learning approach. Local context of a sentiment word should be taken into consideration in sentiment analysis in order to get a better result. A lexicon-based approach is more advantageous than a machine learning approach from this point of view. SO-CAL (Sentiment Orientation CALculator) is introduced in [1]. This system uses lexicons of words annotated with their sentiment orientation and incorporates intensification and negation. This system deals with several linguistic contexts that can have impact on sentiment orientation (polarity and strength). Amplifiers such as 'very' in English increase the sentiment intensity while downtoners such as 'somewhat' decrease it. SO-CAL regards amplifiers and downtoners as a modifier that can adjust the sentiment orientation. While most systems change the sign of a polarity value for negations, SO-CAL shift the polarity value to the opposite orientation for negations by a fixed amount. The fact that 'not excellent' should have -5 polarity value because of 'excellent' with +5 polarity value is a little bit unacceptable. SO-CAL handles some markers that indicate that the words appearing in a sentence might not be reliable for the purpose of sentiment analysis. They are subjective or imperative markers. The approach of SO-CAL for these kinds of markers is to ignore the sentiment orientation for them. SO-CAL's performance is proved to be consistent across domains and on completely unseen data.
Reference [3] proposes a theoretical composition model, which calculates the polarity values of larger syntactic constituents as some function of the polarities of their subconstituents. The proposed sentiment composition model combines two input constituents and evaluates the output polarity for a composed constituent. A sentiment can be propagated, polarities can be reversed and polarity confliction can be resolved in a systematic way in this approach. The success of this approach depends on the quality of a syntactic parsing performance.
In research [4], a polarity value of sentiment expressions can be determined in two steps: firstly, evaluate the polarities of the constituents of the expressions, and secondly, apply a simple set of inference rules to combine the constituents recursively. In order to apply the inference rules, a syntactic pattern of a sentence is detected. For example, when we detect a syntactic pattern ‘[destroy] VP[theterrorism]NP’ the polarity value can be evaluated from the inference compose rule Compose([destroy],[terrorism]). The research [4] introduces new insights for content-word negators, which are content words that can flip the polarity of neighboring words.
3. Korean Auxiliary Predicate and Sentiment Analysis
3.1 Korean Sentiment Analysis
Sentiment analysis is a process to detect a sentiment expression from a given text by referencing a sentiment lexicon, and to evaluate the overall sentiment orientation of a target object. A sentiment lexicon consists of sentiment entries and their polarity. In order to extract a sentiment expression from a text, texts should be pre-processed first, which is the process of normalizing the texts. Since a text is a collection of users' opinions, there are plenty of proper nouns which include personal names and product names. In order to handle them properly in a sentiment process, named entity recognition had to be done. A morphological analyzer processes each sentence in order to find the stem of each word. Each sentence was also analyzed by a Korean dependency parser to identify relations between the words in a sentence. When a sentiment word is accompanied by an explicit negative word, the sentiment direction of the word should be reversed. There are three different types of explicit negative words in Korea [5]. In this paper, the polarity values are marked with an integer. The integer value +1 is assigned to a positive polarity, and the value -1 is assigned to a negative sentiment.
3.2 Korean Auxiliary Predicates
In this paper, we define a Korean auxiliary predicate as not only dependent auxiliary predicates that attach to the end of independent main predicates but also some of cognitive predicates, such as ‘think’, ‘believe’, ‘hope’, ‘imagine’, ‘know’, ‘suppose’, ‘understand’, ‘forget’, which can attribute a sentiment to an experiencer holding it. The reason why we include some of cognitive predicates into the Korean auxiliary predicates is that when theyappear with a sentiment expression in a text, they can decide who experiences the sentiment, and how the experiencer accept the sentiment.
We classify Korean auxiliary predicates into a set of classes according to their characteristics from a the sentiment, and how the experiencer accept the sentiment.
The classes of auxiliary predicates and their actions.
We classify Korean auxiliary predicates into a set of classes according to their characteristics from a sentiment analysis' point of view. Auxiliary predicates can change an overall polarity of a sentiment expression when they appear with it together in a sentence. Table 1 shows the classes of Korean auxiliary predicates and their actions along with motivating examples.
The class ‘negation’ is obvious. The auxiliary predicates in this class flip a polarity of a sentiment expression that they attach to. Auxiliary predicates in ‘near negation’ not flip a polarity of a sentiment expression but shift it to opposite direction less intensively than explicit negation makers do.
There are several Korean auxiliary predicates that nullify the polarity of a sentiment expression to which they attach. The class ‘unreality’ of Table 1 includes this kind of auxiliary predicates, whose usage is for a subjunctive expression, an intentional expression, and an imperative expression. In this case we should ignore the sentiment expression's polarity values when we calculate the sentiment orientations. Though auxiliary predicates in a class ‘attribution’ describe unreal facts same as those in an ‘unreality’ class do, we classify an ‘attribution’ class separately from an ‘unreality’ class. Main reason why we make difference between them is that a sentiment expression used with one of auxiliary predicates in an ‘attribution’ class can be real inside a specific person's mind. Therefore we keep the sentiment polarity for auxiliary predicates in the class ‘attribution’ instead of ignoring it. Their action is to change experiencer's state of mind.
Let us see the following examples. A sentiment expression ‘beautiful’ is both used in the examples (4) and (5). In example (4), a writer expresses his/her wish by using an auxiliary predicate ‘좋겠다’ (hope), which is included in the class ‘unreality’ of Table 1. It is apparent that we do not know if she is beautiful or not. Therefore a sentiment analyzer should ignore this expression when it tries to capture sentiment expression's polarities from a text. The example (5) shows that an auxiliary predicate ‘고 생각하다’(think that~) which is in the class ‘attribution’ attaches to a sentence ‘she is beautiful.’ In example (5), we are also not sure whether she is beautiful or not, but the fact “she is beautiful” is clearly true in a writer's state of mind.
[EX]
(4) [그녀가 아름다우]면 좋겠다.
(I) hope that she is beautiful.
(5) [그녀가 아름답다]고 생각하다.
(I) think that she is beautiful.
Auxiliary predicates in the class ‘weakening’ in Table 1 weaken the intensity of the polarity of the sentiment expression that they attach to. The class ‘emphasis’ has auxiliary predicates that can intensify the polarity of the sentiment expression that they attach to.
3.3 Compositional Rules of Auxiliary Predicates
When there is a sentiment expression with a Korean auxiliary predicate in a text, the polarity for the sentiment expression should be re-evaluated. Table 2 shows the definition of the rules that can re-evaluate the polarity according to the class of auxiliary predicates. Before we present how the compositional rules work, some notations should be introduced first.
A ‘SentUnit’ is defined as the triple [Attributor, Relation, Sentiment], in which Sentiment is defined as the triple [target, sentiment_expression, polarity]. Attributor is a person who experiences a Sentiment with the belief of Relation in Attributor's mind. A possible Relation is {‘fact’, ‘think’, ‘believe’, ‘hope’, ‘imagine’, ‘know’, ‘suppose’, ‘understand’, ‘forget’, ‘wonder’…}. A Sentiment is a unit of sentiment analysis result, which displays a sentiment expression found in a text, and a target object which the sentiment expression describes, and the polarity value of the sentiment expression.
We define a function ‘AuxCompose’ whose input is Aux_class and SentUnitin, and output is a SentUnitout that a compositional rule is applied to.
SentUnitoutt=AuxCompose(Aux_class, SentUnitin)
The input SentUnitin is [Attributor, Relation, [target, sentiment_expression, polarity]] and Aux_class is one of the classes shown in Table 1. The output SentUnitout which a compositional rule is applied to is shown in Table 2 according to the Aux_class.
In order to apply a rule, we first detect a sentiment expression and its target argument in a sentence, then we can look up the sentiment expression’s polarity from a sentiment lexicon. When there is an auxiliary predicate attached to the sentiment expression, its final SentUnit can be obtained by applying rules of Table 2.
[EX]
(6) 그 음악이 피곤을 날려버리다
(The music sweeps away stress.)
SentUnit6 =
[default, fact, [음악, 피곤을_날려버리다, +1]]
(7) 그 음악이 피곤을 날려버렸으면 좋겠다.
(I hope that the music sweeps away stress.)
AuxCompose(unreality, SentUnit6)
= [default, fact, [음악, 피곤을_날려버리다, 0]]
(8) 나는 그 음악이 피곤을 날려버렸다는 생각이 들다.
(I think that the music swept away stress.)
AuxCompose(attribution, SentUnit6)
= [I, think, [음악, 피곤을_날려버리다, +1]]
(9) 그 음악이 피곤을 날려버렸는지 모르겠다.
(do not know if the music sweeps away stress or not.)
AuxCompose(near_negation, SentUnit6)
= [default, fact, [음악, 피곤을_날려버리다, -0.1]]
(when alpha = -0.1)
Example (6) does not have an auxiliary predicate. The default values of Attributor and Relation in SentUnit are ‘default’ and ‘fact’, respectively. The Attributor ‘default’ means an anonymous writer of a text, and the Relation ‘fact’ means that the sentence for a SentUnit is an actual fact. The SentUnit of the Example (6) will be the inputs for the function of AuxCompose of the Example (7) ~ (9). An auxiliary predicate ‘으면 좋겠다’ (hope that~) is in the class ‘unreality’ in Example (7), therefore, the polarity of the SentUnit applied by the function AuxCompose with the input class ‘unreality’ will be zero. The Example (8) has an auxiliary ‘는 생각이 들다’(think that~) in the class ‘attribution’, which changes an Attributor and Relation of the output SentUnit. An auxiliary predicate ‘지 모르겠다’ (whether or not~) in Example (9) is in the class ‘near negation’, which shifts the sentiment orientation opposite.
The compositional rules according to the class of auxiliary predicates.
4. Evaluation
4.1 Experimental Setup
Korean dramas are very popular and Korean drama web sites to which audiences can upload their opinions are affluent in Korea. Therefore it is easy to get users’ opinions on Korean dramas from those web sites. Our corpus was collected by crawling various Korean drama web sites. The postings vary in topic, expressions, and their length. The audiences express diverse direct opinions regarding various entities, such as plots, actors/actresses, authors, directors, and so on. We have crawled bulletin pages of fifteen different web sites representing popular dramas and TV variety shows. We collected 370,693 reviews from 15 different drama sites, which were further segmented into 2,173,210 sentences or 18,711,095 words. The sentiment lexicon we adopted for this paper consists of 21,235 sentiment entries and their polarities, which was built manually for a previous work [6]. The lexicon has not only single entries but also multi-word entries. In order to have insight to the effect of compositional rules for handling auxiliary predicates, we built a following experimental environment. We gathered 1,000 reviews out of all review collection including the name of a specific actor in a text in order to extract the opinion of the actor from a drama web site1). Then annotators found the factual sentimental opinions describing the actor in a text and tagged them with polarity values. The polarity value is one of {-1,1}. Annotators were guided to evaluate the opinion about the actor not superficially but deeply in their meanings. In this experiment, we call this collection of reviews REPUTE.
The algorithm for extracting sentiment expressions from a review.
In order to extract the opinions of the actor from a review automatically, a sentiment analysis system does the following steps depicted in Figure 1. (We mark the name of the actor with ‘TARGET’.) In the line number 07 of the algorithm, sentiment expressions not describing the TARGET that we are interested in are excluded in the further processing.
Table 3 shows an example of the system’s output. In this table, ‘PERSON’ is a tag for a person’s name, and ‘SHOW’ is a tag for the name of shows. The SentUnits written in boldface are included in ‘Sentiment_OUT’ because they describe TARGET.
4.2 Experimental Results
We compared three sentiment analysis systems (A), (B) and (C) which were different from one another in terms of their analysis depth. The system (A) does not consider the effect of Korean auxiliary predicates in sentiment analysis. It just handles the simplest negations for reversing polarity values. The system (A) assumes that all sentiment expressions found in a text describe the ‘TARGET’. The system (B) and (C) are performed by the algorithm shown in Figure 1. While the system (B) includes the results Sentiment_OUT acquired by applying the compositional rules for ‘attribution’, the system (C) excludes them. In the example sentences depicted in Table 3, the final Sentiment_OUT of the system (B) includes the SentUnits of the line 1, 8, 9, and 10 while that of the system (C) includes those of line 1 and 10 only. The reason why the system (C) excludes the results is that the expressions of line 8 and 9 are not actual true. The main reason why we make a distinction between the system (B) and (C) is that sentiment expressions enclosed by the ‘attribution’ predicates are supposed to affect the overall result of sentiment analysis differently from others. The three systems perform collecting opinions about the actor from the corpus REPUTE. The results are described in Table 4. The measurements of precision and recall show that how many sentiment expressions with the correct polarity values for the actor the system can extract from a text. When the sign of the polarity value that the system outputs is the same as that of the REPUTE, manually as signed, the polarity value is considered as correct one.
The example of the result of sentiment analysis.
The experimental results.
As the systems do not adopt a refined syntactic analyzer in order to detect a relationship between a subject and its predicate, they cannot extract every sentiment predicate and its argument from a text, so their recall values are lower than the precision values. The reason why the system (A) has a low precision is that it extracts all sentiment expressions from a text regardless of objects sentiment expressions describe. Though the difference of the performance between the result of the system (B) and (C) is not a big deal, its difference says that the auxiliary predicates ‘attribution’ should be excluded when we are doing an analysis of deep sentiment meaning. Also we found that there is still much room for improvement in detecting TARGET for sentiment expressions.
5. Conclusion
In this paper, we have presented a new approach to handle Korean auxiliary predicates that affect sentiment polarity values. Some auxiliary predicates nullify an overall sentiment polarity, and some others intensify a sentiment polarity value. We have classified Korean auxiliary predicates according to how they impact on the sentiment polarity values. The compositional rules for handling auxiliary predicates were introduced. The experimental results showed that consideration of auxiliary predicates in sentiment analysis can improve the overall accuracy of the deep sentiment analysis.
The existing researches on sentiment analysis deal with the sentiments on the surface, so they are just interested in how many positive words occur or how many negative words occur in a text. The approach proposed in this paper, on the other hand, recognizes the in-depth sentiments, so it tries to identify who expresses which sentiments and how in a text.
There are a few issues left for completeness of handling Korean auxiliary predicates. A sentiment expression can be embedded by double auxiliary predicates, such as "방이 깨끗해야 한다고 생각하다." ("I think that the room should be clean.") or " 방이 깨끗하다고 생각해야 한다." ("I should think that the room is clean."). This will be the direction of our future research.
Acknowledgments
This work was supported by the National Research Foundation of Korea (No. 2012-0004132).
Notes
References
-
M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede, “Lexicon-based methods for sentiment analysis”, The Journal of Computational Linguists, 37(2), p267-307, (2010).
[https://doi.org/10.1162/COLI_a_00049]
- S. Hwang, “Semantic classification of cognitive verbs and their representation in lexical semantic network”, The Journal of Studies in Language, (2010).
- K. Moilanen, and S. Pulman, “Sentiment composition”, Proceedings of the Recent Advances in Natural Language Processing International Conference (RANLP-2007), p378-382, (2007).
- Y. Choi, and C. Cardie, “Learning with compositional semantics as structural inference for subsentential sentiment analysis”, Proceedings of the Conference on Empirical Methods in Natural Language Processing, (2008).
- S. Chang, Korean. John Benjamins Publishing Company, (1996).
-
Y. CHO, and K. Lee, “Automatic affect recognition using natural language processing techniques and manually built affect lexicon”, The Institute of Electronics, Information and Communication Engineers(IEICE) transactions of Information and Systems, E89D(12), p2964-2971, (2006).
[https://doi.org/10.1093/ietisy/e89-d.12.2964]
- R. Prasad, N. Dinesh, A. Lee, E. Miltsakaki, L. Robaldo, A. Joshi, and B. Webber, “The penn discourse treebank 2.0”, Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08), (2008).