International Journal of Computer Trends and Technology- volume3Issue1- 2012

Behavior Study of Web Users Using Two-Phase Utility Mining and Density Based Clustering Algorithms
K. SIRISHA1, D.N.V.S.L.S.INDIRA2 M. V. CHAKRADHARA RAO3
1

M.Tech(CSE),sirishakunapuli@yahoo.co.in,Gudlavalleru Engineering College, Gudlavalleru

2

Associate professor,indiragamini@gmail.com, Gudlavalleru Engineering College,

Gudlavalleru
3

Associate professor, chakrimv@yahoo.com, Potti SriRamulu and Chalavadi Mallikharjuna Rao college of Engineering, Vijayawada

ABSTRACT With the recent explosive growth of the amount of content online, it has grown to become increasingly difficult for users to obtain and utilize information and for contents services to classify and catalog documents. Traditional web the search engines often return hundreds or thousands of results for a search, and that is time intensive for consumers to browse. Typically, in a data mining process, the number of patterns discovered can easily exceed the capabilities of a human user to identify interesting results. To address this problem, utility measures have been used to reduce the patterns prior to presenting them to the user. A frequent itemset just shows the statistical correlation between items, and it will not reflect the semantic significance of the items. This proposed approach utilizes a utility based itemset mining approach to overcome this limitation. This suggested system initial uses Dbscan clustering algorithm which identifies the behavior of the users page visits, order of occurrence of visits . After applying the clustering technique High Two phase utility mining algorithm is applied, targeted at finding itemsets that lead high utility. Excavation internet access sequences can discover extremely useful knowledge from internet logs with wideranging applications. Mining useful Internet path traversal patterns is a very important analysis issue in Internet technologies. Knowledge about the frequent Internet route traversal designs allows us to discover the most interesting Sites traversed of the users. However, considering sole the binary (presence/absence) occurrences of the Websites inside the Web traversal ways, real world situations might not be reflected. Therefore, if we consider the time spent by each owner as a utility value of the website, more interesting internet traversal paths can be discovered using proposed twophase algorithm. User page visits are sequential in nature.

In this paper MSNBC web navigation dataset is used to compare the efficiency and performance in web usage mining is finding the groups which share common interests. I. INTRODUCTION The Whole World Wide Internet serves as a huge, widely distributed, worldwide information service center. It contains an abundant and dynamic assortment of hyperlink information and Website access and usage information. Data mining, that could automatically discover useful and understandable designs from huge data designs, has been widely exploited in the Web. Internet mining can be broadly divided into 3 areas, i.e. content mining, use mining, and link structure mining [1]. Weblog mining is a specialized case of usage excavation, which mines Weblog entries to discover owner traversal patterns of Internet pages. An Internet server normally registers a log entry for every access of a Internet page. Each entry contains the URL requested, the IP address where which the request started, timestamp, etc. popular Websites, including Web-based e-commerce servers, might register entries in the purchase of hundreds of megabytes daily. Data excavation can feel performed on Weblog entries to obtain organization designs, sequential patterns, and fashions of Web accessing. Analyzing and exploring regularities in Weblog entries can identify possible customers for electronic commerce, enhance the quality of Internet information service, improve the performance of Internet provider system, and optimize the website architecture to cater to the liking of end users. Among the objectives of Weblog mining is to look for the frequent path traversal patterns in a Web environment. Path traversal pattern mining will be look for the ways that frequently cooccurred. It firstly converts the original sequence of log information as a set of traversal subsequences. Each traversal subsequence represents a maximal forward reference from the beginning aim of a user access. Furthermore, a sequence

ISSN: 2231-2803

http://www.internationaljournalssrg.org

Page 179

International Journal of Computer Trends and Technology- volume3Issue1- 2012

mining algorithm will be used to determine the frequent traversal patterns, called big reference sequences, from the maximal forward records, where a large reference sequence is a reference sequence that happens frequently sufficient inside the database. The challenge of clustering has become increasingly 2. RELATED WORK important in current years. The clustering problem has been addressed in a lot of contexts and by researchers in many disciplines; this shows its broad appeal and efficiency as among the procedures in exploratory data analysis. Clustering approaches aim at partitioning a set of information aspects in classes these that points that belong to the same class are more alike than aspects that belong to different classes. These classes are called clusters and their quantity is likely to be preassigned or can be a parameter to feel decided by the algorithm. There exist applications of clustering in such diverse areas as company, pattern recognition, communications, biology, astrophysics and numerous people. Cluster analysis is the company of the collection of designs (typically represented as a vector of measurements, or maybe a point within a multidimensional space) into clusters based on similarity. Usually, distance measures are utilized. Data clustering has its origins within a number of areas, including information mining, machine understanding, biology, and studies. Traditional clustering algorithms can feel categorized into two main categories: hierarchical and partitional . In hierarchical clustering, the quantity of clusters will not need to feel specified a priori, and issues because of initialization and town minima do not happen. However, since hierarchical techniques consider sole nearby friends in each step, they cannot incorporate a priori knowledge in regards to the worldwide profile or size of clusters. As a happen, they cannot always individual overlapping clusters. In addition, hierarchical clustering is static, and points committed to a given cluster within the early stages cannot move to a different cluster. Traditional information retrieval techniques express plaintext documents getting a show of numeric values for every document. Each value is connected through a certain term (word) that may appear for a document, and the set of possible provisions is shared across all documents. The values can be binary, indicating the position or lack of the corresponding phase. The values can even be a non-negative integers, which represents the amount of times a term appears on a document (ie. term frequency). Non-negative actual numbers can also be used, in this instance indicating the importance or weight of each term. Clustering is a widely used proficiency in information mining application for discovering patterns in underlying information. Most traditional clustering algorithms are limited in handling datasets that contain categorical attributes. Unfortunately, datasets with categorical kinds of characteristics are common in real being data excavation issue. In traditional versions, all of the Webpage within a database are managed equally by sole considering if a Web is present within a traversal path or otherwise not. We demonstrate the interesting ways we observed in our experiments, along with their significance to the conclusion In the past a decade, a lot of analysis work happens to be done to discover meaningful information from large scale of Internet server access logs. A Web mining system, called WEBMINER is presented in [2]. General Process of WUM Internet servers collect big volumes of information from the internet websites use. This data is retained in Web access log les. Together aided by the Internet access log les, different information can be applied in Web Use Mining like the whole world wide internet construction information, holder proles, Web website contents, etc. 3. Internet USE TERMINOLOGY We start aided by the definition of the group of conditions that results in the formal description of tall utility traversal route mining. 1. Amount of Hits: This amount usually signifies the number of occasions any resource is accessed within a Website. A hit is a demand on to a internet server for a file (site, image, JavaScript, Cascading Fashion Sheet, etc.). When a a an internet page is uploaded from a server the sheer amount of "hits" or "webpage hits" is comparable to the amount of data expected. Therefore, one page load does not regularly equal one hit because normally pages are composed of different pictures along with other information which stack up the quantity of hits counted. 2. Wide range of Visitors: A "visitor" is exactly what it sounds like. It's a human who navigates to your internet site and browses one or even more pages in your internet site. 3. Visitor Referring Website: The referring site provides the info or url of the site which referred the website in consideration. 4. Visitor Referral Website: The referral website provides the info or url of the site and is being known by the distinct site in consideration. 5. Time and Duration: These details within the server logs give the time and length for how long the Website was accessed from a certain owner. 6. Path Analysis: Path analysis provides the analysis of the path a distinct user has followed in accessing contents of the Website. producing process. This remainder of these paper is arranged as follows. Section 2 overviews the relevant work. In Section 3, we present the technical terms in utility excavation model. In Section 4, we present our proposed utility-based path traversal pattern mining algorithm. Section 5 presents the experimental results.

ISSN: 2231-2803

http://www.internationaljournalssrg.org

Page 180

International Journal of Computer Trends and Technology- volume3Issue1- 2012

7. Guest IP address: These details offers the Internet Protocol(I.P.) address of the visitors who visited the Website in consideration. 3.1. Traversal Path The data used for Web log mining is Weblog entry database. Each entry in the database consists of the URL requested, the IP address from which the request originated, timestamp, etc. The database can be stored on Web server, client or agent. The raw Weblog data need to be converted to a set of traversal paths. The goal of frequent traversal pattern mining is to find all the frequent traversal sequences in a given database. We give out the definition of some basic terms. X = <i1, i2, , im>is a m-sequence of traversal path. D = {T1, T2, , Tn} is a Weblog database, where Ti is a traversal path, 1 i n 3.2. Utility Mining Following is the formal definition of utility mining model. I = {i1, i2, , im} is a set of items. D={T1,T2,..Tn} is a transaction database where each transaction Ti belongs to D is a subset of I. O(Ip,Tp) objective value,represents the value of item Ip in Transaction Tq. S(Ip), Subjective value, is the specific value assigned by a user to express the users preference. 3.3. Utility-based Web Path Traversal Pattern Mining By introducing the concept of utility into web path traversal pattern mining problem, the subjective value could be the end users preference, and the objective value could be the browsing time a user spent on a given page. Thus, utility-based web path traversal pattern mining is to find all the Web traversal sequences that have high utility beyond a minimum threshold. A web page refers to an item, a traversal sequence refers to an itemset, the time a user spent on a given page X in a browsing sequence T is defined as utility, denoted as u(X, T). The more time a user spent on a Web page, the more interesting or important it is to the user. Table 1 is an example of a traversal path database. The number in the bracket represents the time spent on this Web page which can be regarded as the utility of this page in a given sequence. In Table 1, u(<C>, T1) is 2, and u(<D, E>, T8) = u(D, T8) + u(E, T8) = 7+2 = 9. From this example, it is easy to observe that utility mining does find different results with frequency based mining. The high utility traversal paths may assist Web service providers to design better web link structures, thus cater to the users interests. TID T1 T2 T3 T4 T5 User Traversal A(2),C(3) B(5), D(1)E(1) A(1)C(1)E(3) A(1)D(18)E(5) C(4),E(2)

4. PROPOSED ALGORITHMS

Algorithm for Utility-based Web Path Traversal Pattern Mining Utility-based path traversal pattern mining is targeted at finding sequences whose utility surpasses an owner specified minimal limit. The challenge of utility mining is the fact that it will not follow downward closure property (antimonotone property), that is, a tall utility itemset might comprise of some minimal utility sub-itemsets. Downward closure property plays a role in the success of ARM algorithms, including Apriori [8], where any subset of a frequent itemset must also be frequent. Two-Phase algorithm offered by Y. Liu et al. [6] is aimed at solving this difficulty. In Phase I, transaction-level utility is proposed and defined of the sum of the utilities of all the transactions containing X. (The factor of introducing this new concept is certainly not to define a new issue, but to make use of its property to prune the search area.) This model maintains a Transaction-level Downward Closure Property: any subset of the High transaction-level utility itemset must additionally feel High in transaction-level utility. In Phase II, single one database scan is performed to filter out the High transaction-level utility itemsets which are actually minimal utility itemsets. In this paper, we extend Two-Phase algorithm to traversal path mining issue. High transaction-level utility sequences are identified in Phase I. The dimensions of candidate set is reduced by sole considering the supersets of High transaction-level utility sequences. In Phase II, sole one database scan is performed to filter out the High transactionlevel utility sequences which are actually low utility sequences. This algorithm ensures that the complete group of High utility sequences is identified[10]. Algorithm1: Proposed new Dbscan clustering: Step 1: Build the similarity matrix using S3M measure(Definition 1). Step 2: choose all aspects from D that satisfy the Eps and Minpts C = 0 for each unvisited point P in dataset D mark P as visited N = get Neighbors (P, eps) if sizeof(N) < MinPts mark P as NOISE else start C = upcoming cluster mark P as visited end add P to cluster C for every aim P' in N if P' is certainly not visited mark P' as visited N' = getNeighbors(P', eps) if sizeof(N') >= MinPts N = N joined with N' if P' is certainly not however member of any cluster add P' to cluster C Step 3: Return C

ISSN: 2231-2803

http://www.internationaljournalssrg.org

Page 181

International Journal of Computer Trends and Technology- volume3Issue1- 2012

Step 4: For all Ti U Compute Si= R(Ti) Using description 2 for given threshold d. Step 5: Next compute the constrained-similarity top Approximations Sj for relative similarity r using description 3 if Si= SJ end if Step 6.: Repeat step 3 until U ; Return D. End Algorithm2: Two Phase Algorithm Input: 1. A set of m items I={i1,i2im},each Ij with a profit value Pj,j=1 to m; 2. A transaction database D={T1,T2,Tn} in which each transaction includes a subset of items with quantities; 3. The minimum threshold thres. Output: A set of utility itemsets. Step1:Calculate the utility value Ujk of each item Ij in each transaction Tk as Ujk=Qjk*Pj, Where Qjk is the quantity of Ij in Tk for j=1 to m and k=1 to n. Step2:Find the maximal utility value MUk in each transaction Tk as MUK=max{U1k,U2k,.Umk} for k=1 to n. Step3:Calculate the Utility upper bound UBj of each item Ij as the summation of the maximal utilities of the transactions which include Ij. That is :

where ujk is the utility value of each item ij in transaction Tk and |s| is the number of items in s. Step11:Check whether the actual average-utility value aus of each candidate average-utility itemset s is larger than or equal to thres. If s satisfies the above condition, put it in the set of high average-utility itemsets, H.

5. EXPERIMENTAL RESULTS The log data used in this paper is extracted from a research website at DePaul CTI (http://www.cs.depaul.edu). The data are randomly sampled from the Weblog data within 2 weeks in April, 2002. We performed preprocessing on the raw data. The original data included 3446 users, 10950 sessions, 105448 browses and 7051 Web pages. Among these Web pages, a large portion is URLs and, thus, data cleaning is needed. After data preprocessing, we carried out two groups of experiments, one for frequent traversal path mining and the other for utility-based traversal path mining. The minimum utility threshold is set at 1% of the total utility. Utility-based traversal path mining obtains 22 high utility sequences including twelve 1-sequences, seven 2-sequences and three 3-sequences. Frequency-based mining obtains 121 frequent sequences, including 1- sequence, 2-sequence, 3sequence and 4-sequence. Table 3 shows the top 10 sequences discovered by the two models. (We dont take 1-sequence into consideration). Dbscan cluster results:

Step4:Check whether the utility upper bound of an item Ij is larger than or equal to thres. If Ij satisfies the above condition,put it in the set of candidate utility 1- itemsets,C1. That is: Step5: Set r=1,where r is used to represent the number of items in the current candidate utility itemsets to be processed. Step6: Generate the candidate set Cr+1 from Cr with all the r- subitemsets in each candidate in Cr+1 must be contained in Cr. Step7: Calculate the Utility upper bound UBs of each candidate utility (r+1) itemset as the summation of the maximal utilities of the transactions which include s. That is :

Step8: Check whether the average-utility upper bound of each candidate (r+1)-itemsets s is larger than or equal to thres. If s does not satisfy the above condition, remove it from Cr+1. That is: Step9: IF Cr+1 is null, do the next step; otherwise, set r = r + 1 and repeat STEPs 6 to 9. Step10:For each candidate average-utility itemset s, calculate its actual average-utility value aus as follows:

Two Phase Results:

ISSN: 2231-2803

http://www.internationaljournalssrg.org

Page 182

International Journal of Computer Trends and Technology- volume3Issue1- 2012

{1=865} {1=1730} maxItem calc:2 HIGH UTILITY ITEMTIDSET:2 [] {1=1730, 2=865} maxItem calc:3 HIGH UTILITY ITEMTIDSET:3 [] {1=1730, 2=865, 3=865} maxItem calc:4 HIGH UTILITY ITEMTIDSET:4 [] {1=1730, 2=865, 3=865, 4=865} maxItem calc:5 HIGH UTILITY ITEMTIDSET:5 [] {1=1730, 2=865, 3=865, 4=865, 5=865} Optimal Two Phase Rules generated : class 984 IF : Maximum actions in one visit in {59} AND Actions in {3069} AND Visits in {1106} AND Bounce Rate in {0.65} AND Actions per Visit in {2.8} AND Avg. Visit Duration (in seconds) in {165} AND Actions by Returning Visits in {1202} AND Unique returning visitors in {225} AND Returning Visits in {308} AND Bounce Rate for Returning Visits in {0.49} AND Avg. Actions per Returning Visit in {3.9} AND Avg. Duration of a Returning Visit (in sec) in {281} AND Conversions in {100} AND Visits with Conversions in {69} AND Conversion Rate in {6.24} AND Revenue in {176} AND Outlinks in {128} AND Pageviews in {2943} AND Unique Outlinks in {122} AND Unique Pageviews in {2240} AND Downloads in {0} AND Unique Downloads in {0} AND Date in {12/28/2012,4/23/2013,5/17/2013,3/5/2012,9/2/2012,10/4/20 12,6/11/2012,10/20/2012,12/1/2012} Overall speaking, observed from our experiments, we realize that high utility traversal sequences are valuable, which can show the customers hidden behavior patterns to the web service providers, which in turn could be utilized to provide better services. 6. CONCLUSION AND FUTURE WORK This paper defines a new mining measure called the average utility and proposes three algorithms to discover high average-utility itemsets. The first algorithm discovers high utility itemsets from static databases in a batch way. This algorithm is divided into two phases. In phase I, it overestimates the utility of itemsets for maintaining the downward closure property. The property is then used to efficiently prune impossible utility itemsets level by level. In phase II, one database scan is needed to determine the actual high average-utility itemsets from the candidate itemsets generated in phase I. Since the number of candidate itemsets has been greatly reduced when compared to that by the traditional approaches, a lot of computational time may be saved. Traditional path traversal pattern mining discovers frequent Web accessing sequences from Weblog databases. It is not only useful in improving the website design, but also able to lead to better marketing decisions. However, since it is based on frequency, it fails to reflect the different impacts of different webpages to different users. The difference between webpages makes a strong impact on the decision

RESULTS: (0.)82,3227,954,1140,0.63,2.8,216,1512,248,393,0.48,3.8,32 0,53,39,3.42,81 --> NOISE (1.)48,1531,485,561,0.64,2.7,171,628,130,178,0.54,3.5,249, 47,32,5.7,85,66 --> NOISE (2.)63,2259,625,737,0.6,3.1,206,1190,176,253,0.41,4.7,350, 55,35,4.75,91,7 --> NOISE (3.)72,4201,1097,1298,0.6,3.2,234,1913,290,429,0.42,4.5,39 3,124,85,6.55,2 --> NOISE (4.)66,3931,1073,1263,0.6,3.1,218,1673,300,427,0.45,3.9,36 5,101,67,5.3,17 --> 0 (5.)46,3349,1028,1217,0.63,2.8,217,1638,282,415,0.46,3.9,4 20,65,45,3.7,11 --> NOISE (6.)85,3556,1040,1203,0.63,3,212,1381,268,367,0.48,3.8,33 4,97,69,5.74,173 --> 0 (7.)47,2728,887,1022,0.64,2.7,182,1247,251,344,0.5,3.6,301 ,75,48,4.7,135, --> 0 (8.)63,1544,531,599,0.62,2.6,185,654,130,177,0.44,3.7,343, 47,35,5.84,85,6 --> 30 (9.)52,1445,482,548,0.64,2.6,177,643,114,154,0.49,4.2,367, 40,24,4.38,70,5 --> 30 maxItem calc:1 HIGH UTILITY ITEMTIDSET:1 []

ISSN: 2231-2803

http://www.internationaljournalssrg.org

Page 183

International Journal of Computer Trends and Technology- volume3Issue1- 2012

makings in Internet Information Service Applications. In this paper, we presented high Two phase web utility mining, which introduced the concept of utility into web log. As utility measures the interesting or usefulness of a webpage, thus satisfies the Web Service Providers in quantifying the user preferences of ease in web data transactions. Hence, we explored a Two-Phase algorithm that discovered high on-shelf utility data on web pages highly efficiently, in which both the phases are carried out with effective algorithms and became responsible in giving us the effective results[10]. Our proposed algorithm was though a mixed approach of utility mining & twophase algorithm on web utility mining, both the mining algorithms were individually proved to be the best of many others, in yielding good experimental results when applied on a real-world Msndc Weblog database. Thus, our mixed approach can surely be declared as the best. We also demonstrated the interesting areas we observed, as well as their significance to the decision making process. On-shelf utility mining considered not only individual profit and utility of each item in a web transaction but also common on-shelf time periods of a product combination. In this study, a new on-shelf web utility mining algorithm was preferred in order to speed up the execution efficiency for mining high on-shelf utility web transactional itemsets. The experimental results also showed that the proposed high on-shelf utility approach had good impact when compared to the other traditional utility mining approaches. Finally In this paper we developed a new rough set dbscan clustering algorithm and presented a experimental results on msnbc.com which is useful in finding the user access patterns and the order of visits of the hyperlinks of the each user and the inter cluster similarity among the clusters. In the future, we will attempt to handle the maintenance problem of high on-shelf utility mining of transactions at the webpage level. Besides, the results from on-shelf utility mining on web transactional log are independent of the order of transactions. Another kind of knowledge called sequential patterns depends on the order of transactions. We will also extend our approach to mining out this kind of knowledge in the future. A number of interesting problems are open to discuss in the future research. For example, the accuracy, effectiveness and scalability of the proposed idea applied to larger databases need to be evaluated. Other factors can also be explored as utility in web transactional pattern mining. Besides, how to combine frequency and utility together to improve web transactional ease is still a problem need to be studied.
7.REFERENCES [1] Agrawal .R and Srikant .R (1994) : Fast algorithm for mining association rules in large databases, The 20th International Conference on Very Large Data Bases, pp. 487-99. [2] Lan G.C., Hong T.P., and Vincent S. Tseng.(2009) : A two-phased mining algorithm for high on-shelf utility itemsets, The 2009 National Computer Symposium, pp. 100-5. [3] D.N.V.S.L.S. Indira, Jyotsna Supriya .P, and Narayana .S(July 2011): A Modern Approach of On-Shelf Utility Mining with Two-Phase Algorithm on Web Transactions, IJCSNS International Journal of Computer Science and Network Security, VOL.11 No.7 [4] Lan G.C., Hong T.P., and Vincent S. Tseng (2011) : Reducing Database Scans for On-shelf Utility Mining, IETE Tech Rev 2011, vol 28,no. 2, pp.103-12. [5] Tseng V.S., Chu C.J., Liang T.(2006) : Efficient Mining of Temporal High Utility Itemsets from Data streams Proceedings of Second International Workshop on Utility-Based Data Mining, August 20, 2006. [6] Yao H., Hamilton H.J., and Butz C.J, (2004) : A foundational approach to mining itemset utilities from databases, Proceedings of the 3rd SIAM International Conference on Data Mining, pp. 482-486. [7] Yu-Cheng Chen and Jieh- Shan Yeh.(2010) : Preference utility mining of web navigation patterns, IET International Conference on Frontier Computing. Theory, Technologies & Applications (CP568) Taichung, Taiwan, pp.49-54. [8]. Chang, C.C., Lin, C.Y.: Perfect hashing schemes for mining association rules. The Computer Journal 48(2), 168179 (2005) [9]. Grahne, G., Zhu, J.: Fast algorithms for frequent itemset mining using fp trees. IEEE Transactions on Knowledge and Data Engineering 17(10), 13471362 (2005). [10]. G. Sunil Kumar, C.V.K Sirisha, Kanaka Durga.R, A.Devi,: Web Users Session Analysis Using DBSCAN and Two Phase Utility Mining Algorithms. International Journal of Soft Computing and Engineering (IJSCE) ISSN: 2231-2307, Volume-1, Issue-6.

ISSN: 2231-2803

http://www.internationaljournalssrg.org

Page 184