Full Paper Int. J. on Recent Trends in Engineering and Technology, Vol. 7, No.

1, July 2012

A Novel Machine Learning Approach for Sentiment Analysis Based on Adverb-Adjective-Noun-Verb (AANV) Combinations
Souvik Sarkar1, Partho Mallick2, Tapas Kr. Mitra2
2

Department of Computer Science &Engineering, Jadavpur University, Kolkata Department of Computer Science &Engineering, Techno India Group College, Kolkata Email Id: souviksarkar@ieee.org, partho.mallick@gmail.com, mitra.tapas@gmail.com cally extracting target text and classifying the polarity of the text, that is whether the expressed opinion is positive, negative or neutral and also, by what extent. We define a set of general axioms for opinion analysis to determine a functional value of the sentiment analysis. In this analysis, Entropy, Conditional Entropy and Information Gain are some of the well-defined concepts that have been applied to evaluate our proposed opinion analysis system. II. PREVIOUS WORK Some prior studies have been made on Sentiment Analysis to focus on the document-level classification of sentiment (Turney, 2002; Pang et al., 2002) where researchers incorporated documents with single sentiment. Some other workers (Subasic and Huettner, 2001; Morinaga et al., 2002) trusted on quantitative information such as the frequencies of word associations or statistical predictions of favorability but they provided sentiment to words. Hatzivassiloglou and McKeown (1997) have investigated automatic acquisition of sentiment expressions, but they confined to adjectives and only one sentiment could be assigned to every word. Yi et al. (2003) rightly indicated about the necessity of extraction of the multiple sentiment aspects in a document.Kanayama Hiroshi, Nasukawa Tetsuya And Watanabe Hideo provides a method for translation from text documents to a set of sentiment units and remove two types of errors out of the four claimed by Nasukawa and Yi (2003) .In order to predict the orientation of a document [2][7] or the positive/negative/ neutral polarity of an opinion sentence within a document , Sentiment Analysis through light on assigning a polarity or a strength to subjective expressions (words and phrases that express opinions, emotions, sentiments, etc.) [4][9][10].Subsequent work has been done to focus on the strength of an opinion expression where each clause within a sentence can have a neutral, low, medium or a high strength [6]. Adverbs are used for opinion mining in [11] where adjective phrases such as excessively affluent were used to extract opinion carrying sentences. [10] uses sum based scoring with manually scored adjectives and adverbs, while [12] uses a template based methods to map expressions of degree such as sometimes, very, not too, extremely very to a [-2, 10] scale. Farah Benamara [1] shows that using adverbs and Adverb Adjective Combination produces significantly higher Pearson correlations (of opinion analysis 1

AbstractThe capability to study facts(data) about each living as well as non-living entity and derive conclusions(information) from those facts and then store them for future use and reference(knowledge), is an art which no other species has been gifted. This skill has been enriched over the time. With the advent of the internet, communicating across the globe has virtually been reduced to our palm. So, it is of utmost importance, to judicially use our vocabulary and grammar, to get the true feeling and sentiment across to the intended person/(s). Almost no research work has been done , as on date, based on adverb-adjective-noun-verb (AANV) combinations in sentiment analysis .We have proposed here for the first time , an AANV based sentiment analysis technique deploying linguistic analysis of adverbs, adjective, abstract noun and categorized verb, which has been a significant advancement from the previous research on this domain. We define a set of general axioms for opinion analysis to determine a functional value of the sentiment analysis. In this analysis, Entropy, Conditional Entropy and Information Gain are some of the well-defined concepts that have been applied to evaluate our proposed opinion analysis system. Index TermsSentiment analysis, Adverb-Adjective-NounVerb combinations, Entropy, Conditional Entropy, Information Gain.

I. INTRODUCTION Recently, there has been a trend towards a new concept of knowledge discovery. It is a field of computer science that deals with automatically extracting useful patterns from existing data about various entities which is considered to be knowledge. It is derived from the field of data mining and is also applied to knowledge discoveries of databases which contain large volumes of data about entities. This knowledge is iteratively input for further knowledge discoveries. The concept can be used for the process of reverse engineering and is also a part of software mining where existing software can be studied to form models like entity-relationship diagrams. The existing software artifacts contain enormous business value, so the process is not only important from engineering, but also from the business point of view. Here instead of studying the raw data, the focus is mainly on the meta-data. So, it is of utmost importance to device a mechanism, where the true sentiment of any sentence be correctly and efficiently evaluated, because knowledge itself is based solely on evaluations from data. The basic task is automati 2012 ACEEE DOI: 01.IJRTET.7.1.12

Full Paper Int. J. on Recent Trends in Engineering and Technology, Vol. 7, No. 1, July 2012 algorithms vs. human subjects) than these previously developed algorithms that did not use adverbs or Adverb Adjective Combination. V.S. Subrahmanian and Diego Reforgiato shows another way to identify intensity of opinion on any topic by analyzing the sentiments expressed by combinations of adjective , verb & Adverb [8] . We also previously shows that Adverb-Adjective Combination is very much important for opinion analysis but for achieving better result we must take in consideration domain categorization of adverb and Adverb-Adjective-Noun ( AAN )combination [5] instead of only Adverb-Adjective Combination. Our new approach extends AAN algorithm, which define a set of general axioms for opinion analysis taking Adverb, Adjective, and Noun & Verb in combination. In this analysis, Entropy, Conditional Entropy and Information Gain are some of the well-defined concepts that have been applied to evaluate our proposed opinion analysis system. AAN (AdverbAdjective-Noun) algorithm [5] as defined by us is shown in figure 1.

Example: SentiWordnet provide five Senses for the word LIKE in Verb Domain. Result from SentiWordNet: Sense 1: P: 0.125O: 0.875N: 0 Sense 2: P: 1O: 0N: 0 Sense 3: P: 0.375O: 0.625N: 0 Sense 4: P: 0.375O: 0.625N: 0 Sense 5: P: 0.125O: 0.875N: 0

IV UNARY AANV ALGORITHM Let AFF, DOUBT, WEAK, STRONG and MIN respectively be the sets of adverbs of affirmation, adverbs of doubt, adverbs of weak intensity, adverbs of strong intensity and minimizers. Suppose fSense is an unary AANV scoring function that takes as input, one adverb, one adjective, one noun and one verb and returns a number between -1 and +1. AAN scoring function fSense should satisfy the following 1. If adv AFF U STRONG & < 0, then

Figure 1. Steps of AAN Algorithm

III. VERB SCORING AXIOMS We examined all of Verb categories and found that verb can be categorized in two Auxiliary Verb (can, could, may, must, should, will, be, have, do, etc.) and Main Verb (it conveys a real meaning and doesnt depend on another verb) of which only Main Verb are useful in opinion analysis. And also, including verb with adverb-adjective-noun combination will provide us much better result in opinion analysis. Scoring of Verb will be either +1 or -1 depending upon SentiWordnet positive and negative polarity decision.

If adv AFF U STRONG &

>0, then

2. If adv WEAK U DOUBT &

> 0, then:

If adv WEAK U DOUBT & - Positive Score of a sense of a word obtained from SentiWordnet - Negative Score of a sense of a word obtained from SentiWordnet. n is the no of Sense obtained for that word in Verb domain of SentiWordNet. Scoring of verb must be in the range from -1 to +1. Final score of the verb (sc(verb)) will be obtained after mapping this score from [-1 - +1] scale to [0 1] scale. 2012 ACEEE DOI: 01.IJRTET.7.1.12 2

< 0, then:

3. If adv MIN,then:

In case a blog or an article consist of more than one sentence we consider two cases Case1: If Standard Deviation of fSense scores for all these sentences is above a certain threshold value. Then, Geometric

Full Paper Int. J. on Recent Trends in Engineering and Technology, Vol. 7, No. 1, July 2012 Mean of fSense scores for all these sentences is taken as final Sentiment scoring of that blog or article. Case2: If Standard Deviation of fSense scores for all these sentences is below that threshold value. Then, Arithmetic Mean of fSense scores for all these sentences is taken as final Sentiment scoring of that blog or article. Example: I like this product since it is very beautiful. sc(adj)=0.6875 , sc(adv)=0.125 , sc(noun)=1 , sc(verb)=0.2 fSense=0.6875+.2+(1-.6875)(1-.2)(.125)=0.91875 V. BINARY AANV ALGORITHM We assign a score to a binary AANV <adv1. adv2 ><adj. noun. verb> as follows. First, we compute the score fSense (adv2, adj, noun, verb). This gives us a score denoting the intensity of the unary AANV <adv2.adj. noun. verb> which we denote AANV1. We then apply fSense to (adv1, AANV1, 1, 0) and return that value as the answer. VI. INFORMATION GAIN The quantitative measure of information is based on our intuitive notation of the word information, i.e., the more unexpected an event is (with a priori probability p); the more information is obtained with the occurrence of that event (x) [4].

Here from, we can have Entropy for occurrence of a verb in a sentence given an adverb, an adjective; a noun is already present in the sentence.

where Pr(xi) is the prior probabilities for all values of X, and is the posterior probabilities of X given the values of Y. The amount by which the entropy of X decreases reflects additional information about X provided by Y and is called information gain (Quinlan, 1993), given by

Now, we can have information gain for occurrence of a verb in a sentence given an adverb, an adjective,and a noun is already present in the sentence.

VII. EXPERIMENT RESULT We implemented algorithm proposed in this paper by extending Opinion Analysis System (OASYS) as well as the algorithms described in [1, 2, 3, 4, 5]. The implementation was approximately 3000 lines of Java on a Intel Core2CPU 2.93GHz machine with 4GB RAM PC running Red Hat Enterprise Linux. Experimental purpose, a set of 100 news articles scored by 10 students and 600 blog posts scored by 40 students, is used. We then find scoring provided by AANV Algorithm on both blogs and news articles. We then compared our algorithm with those described in [1, 2, 3, 5].The table (Table. 1) below shows the Pearson correlations of the previous algorithms and also one proposed by us.
TABLE I. PEARSON CORRELATION FOR DIFFERENT ALGORITHMS

Now X={x1, x2.xm} The concept of Shannon entropy is defined then as

Here from, we can have quantitative measure of the amount of uncertainty associated with occurrence of a verb in a sentence. A number of Senses will be obtained for that word in Verb domain from SentiWordnet will be included in the set. Verb = {sense1, sense2, sense3 sensei.. sensen}. Now,

- Positive Score of a sense of a word obtained from SentiWordnet - Negative Score of a sense of a word obtained from SentiWordnet. n is the no of Sense obtained for that word in Verb domain of SentiWordNet. H(X) is the quantitative measure of the amount of uncertainty associated with X. Conditional Entropy signifies, the Entropy of X given that Y has occurred is defined as 2012 ACEEE DOI: 01.IJRTET.7.1.12 3

In course of our experiment one of the sampling set (Blog Review of Bollywood Movie DON 2) is shown along with the result in Table 2 & Table 3. In Table 2(A) and 2(B) each blog represents a set of sentences or a single sentence giving an opinion of a single person on some specific topic. Human Scoring as already mentioned done by 40 different students

Full Paper Int. J. on Recent Trends in Engineering and Technology, Vol. 7, No. 1, July 2012 and Machine scoring for that blog has been extracted using our AANV axioms. Table 3 shows the analysis of the results shown in Table 2(A) and Table 2(B). Final Pearson Correlation value for AANV is obtained after evaluating on all 100 news articles and 600 blog posts.
TABLE II(A). AANV OPINION ANALYSIS SYSTEM OUTCOME OF 50 COMMENTS FOR REVIEW OF B OLLYWOOD MOVIE DON 2 TABLE (II(B). AANV OPINION ANALYSIS SYSTEM OUTCOME OF ANOTHER 50 COMMENTS FOR REVIEW OF B OLLYWOOD MOVIE DON 2

TABLE III. ANALYSIS

OF OUR PROPOSED ALGORITHM DATASET

(AANV) (TABLE 2 A.&3B.)

FOR THE ABOVE

For better validation of our project Information Gain due to inclusion of verb for opinion analysis is calculated as shown in above section[VI] and compare(Figure 2) it with those of the algorithms described in [1,5]. The comparative study (Figure 2) shows that adverb-adjective-noun (AAN)[5] and adverb-adjective-combination(AAC)[1] both provide a good result on our test blogs and news articles but for defining a sentence there is a need of taking verb as consideration also. We have combined all of them (Adverb, Adjective, Noun and Verb) in our AANV axioms to obtained a result that will be very closer to the scoring done by a human being. Figure 2 shows the validation of our views of combining all of them in AANV axioms to extract a scoring of a blog review or a news article. 2012 ACEEE DOI: 01.IJRTET.7.1.12 4

Full Paper Int. J. on Recent Trends in Engineering and Technology, Vol. 7, No. 1, July 2012 REFERENCES
[1] F. Benamara et al., Sentiment Analysis: Adverbs and Adjectives Are Better than Adverbs Alone, Proc. 2007 Intl Conf. Welogs and Social Media (ICwsm 07), 2007. [2] P. Turney, Thumbs Up or Thumbs Down?Semantic Orientation applied to Unsupervised Classification of Reviews, In Proceedings of 2006 International Conference on Intelligent User Interfaces (IUI06), 2002. [3] C. Cesarano and B. Dorr and A. Picariello and D. Reforgiato and A. Sagoff and V.S. Subrahmanian, OASYS: An Opinion Analysis System, AAAI 06 spring symposium on Computational Approaches to Analyzing Weblogs, 2004. [4] S.O Kim and E. Hovy, Determining the Sentiment of Opinions, Coling04, 2004. [5] J.K. Sing , Souvik Sarkar ,Tapas Kr. Mitra, Development of a novel algorithm for Sentiment Analysis based on AdverbAdjective-Noun combinations , NCETACS-2012, National Conference on Emerging Trends and Applications in Computer Science - 2012. [6] T. Wilson and J. Wiebe and R. Hwa, Just how mad are you? Finding strong and weak opinion clauses, AAAI-04, 2004. [7] B. Pang and L. Lee and S. Vaithyanathan, Thumbs up? Sentiment Classification Using Machine Learning Techniques, 2002. [8] V.S. Subrahmanian and Diego Reforgiato, Adjective-VerbAdverb Combinations for Sentiment Analysis, Published by the IEEE Computer Society. 2008. [9] V. Hatzivassiloglou and K. McKeown, Predicting the Semantic Orientation of Adjectives, ACL-97, 1997. [10] H. Yu and V. Hatzivassiloglou, Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences, In Proceedings of EMNLP03, 2003. [11] S. Bethard and H. Yu and A. Thornton and V. Hativassiloglou and D. Jurafsky, Automatic Extraction of Opinion Propositions and their Holders, Proceedings of AAAI Spring Symposium on Exploring Attitude and Affect in Text, 2004. [12] T. Chklovski, Deriving Quantitative Overviews of Free Text Assessments on the Web, In Proceedings of 2006 International Conference on Intelligent User Interfaces (IUI06), January 29-Feb 1, 2006, Sydney, Australia, 2006.

Figure 2. Comparative Study of Different Algorithms

VIII. FUTURE WORK News articles sentiment analysis got high accuracy in respect to Pearson correlation since it contains a grammatically correct sentence but Blog / Twitter sentiment analysis is not an easy task since a tweet/blog can contain a significant amount of information in very compressed form, and simultaneously carry positive and negative feelings. AANV combination opinion analysis system introduce by us obtain substantial improvements from the earlier results but we still think advanced NPL & Machine Learning techniques should be employed in future efforts in this project.

2012 ACEEE DOI: 01.IJRTET.7.1.12