1 Introduction
The demand for accurate recommendation systems has dramatically increased with the growth of user engagement in varying e-Commerce portals and is facilitated by the rapid proliferation of personal and usage data on the Internet. An average user does not have enough time or resources to systematically explore the content on e-Commerce sites to identify items of interest. Hence, there is a pressing need for automated and personalized assistance for exploring content at online stores and service outlets.
The recommendation systems must be able to learn about both the preferences of the users as well as the properties of the contents to provide accurate personalized recommendations to the users. Both of these requirements present complex challenges.
The first challenge is collecting information about content properties. The information about the items used by current recommendation systems is generally provided by the creator of the content or by a few expert individuals. This source bias limits the usefulness of the available information for the purpose of recommendation. This is because effective recommendation systems need to leverage subjective information from the perspective of individual users of the system. To illustrate this problem, consider the following recommendation request by a user on the GoodReads website:
‘Id Like to read a page-turner book with a strong female lead! i like YA, with humore and some supernatural aspects to it.’ (sic)
The second critical challenge is learning the preferences of the users: humans are not perfectly rational beings and will not necessarily be aware of or be able to effectively articulate their own preferences. There are a number of issues that prevent the recommendation systems from accurately modeling the users. Studies have, for example, exposed paradoxical transitive preferences from users (such as
$A\,{\succ}\,B\,{\succ}\,C\,{\succ}\,A$
) when asked to compare a set of items (Davis, Reference Davis1958). In real life when people try to make decisions about candidate items we might be interested in, they usually base their decision about the candidates on what other people say about those items. They do not only consider what has been said about the items but the source of that information is also important. For example, a movie critique (C) saying ‘good’ for a movie might not mean anything for target user (T) while a blogger (B) using the same word for a movie might be very important for decision-making process of T. Another example about real life is, in some cases, it is not important if the source of information has liked the item or not. For example, if T likes every movie which was defined as a ‘World War II (WW2)’ movie by B, T will probably like the next movie which was defined as ‘WW2’ by B, regardless of B’s liking of that movie.
Existing systems interpret the reviews of users by relating them to their ratings. This introduces two problems: (i) since the rating system can differ among different web pages (different ranges, thumbs up/down, etc.), using reviews from different web pages is challenging and (ii) those systems can miss the fact that a feature of an item which causes one user to dislike an item, might cause another user to like the item. To illustrate this problem, we analyze a review by User A accompanied with a low rating:
‘Romantic chick-flick; total waste of time!’
While it is obvious that User A did not like the item, the same review can suggest that User B , who likes romantic movies, might like this movie. Another problem with existing review-based systems is that they extract the features of the items and subsequently discard the information about the reviewers. For example, suppose User A reviewed a hotel Hotel X as:
‘The view from the hotel was excellent!’
Let another user, User B , review a hotel Hotel Y using the exact same words. Existing review-based recommendation systems generally interpret these reviews as ‘User A and User B both care about the views from the hotels’ and ‘Hotel X and Hotel Y both have nice views.’ Therefore, these systems will recommend Hotel X to User B and Hotel Y to User A . But if Hotel X has a nice city view, which User A likes, and Hotel Y has a nice nature view, which User B likes, these recommendations are misleading. Language Independent Recommender Agent (LIRA) avoids this problem by learning that User B ’s reviews are not predictive of User A ’s ratings.
Another problem that existing review-based recommendation systems face is that not every user is willing to write reviews for the items they evaluate. Rating/review ratios in IMDB and GoodReads are only 1/400 and 1/20, respectively. Because of this sparsity, systems which require reviews from the target users will fail to produce recommendations for a large majority of the users. LIRA does not require target users to write reviews and can recommend items to users as long as they rate items.
Finally, most of the existing review-based systems depend on topic modeling approaches or finding similarities between different users’ reviews. This limits their applicability to situations where all the reviews are in the same language. They also fail to work when the writing style of the users change, such as using ‘gr8’ instead of ‘great’. LIRA can use reviews from different languages for recommendation, and even different writing styles, as long as users are consistent about their language usage or writing style.
In this paper, we present an agent-based recommender system, LIRA, which creates agents for target users (A T ) and reviewers to create accurate recommendations. LIRA consists of targets user’s agent, which processes messages from agents of other users, which sends the words sequences (N-Grams) in those users’ reviews to target user’s agent. LIRA agents, A T s, learn from those messages and then make predictions for unknown items based on the learnt knowledge.
We summarize the problems confounding other recommender systems but which LIRA can handle:
1. LIRA can make use of different perspectives of the users on different features of items, such as liking the different types of views in different hotels.
2. LIRA does not depend on other users’ or sources’ ratings, it can work on the ratings of a target user, given that reviews are available for items rated by the target user. This property also enables LIRA to utilize reviews from different sources, such as blogs, reviews from different websites, etc.
3. LIRA does not depend on social connections, explicit statement, or rating similarity for calculating trust for a review source. It can calculate these trust values by measuring the capability of those sources to predict target user’s rating in training set.
4. LIRA only needs ratings of target users and does not require them to write reviews to get recommendations.
5. LIRA is language independent: it can leverage reviews written in any language.
2 Background and related work
In this section, we provide the literature review on the methods we use and also about the state of recommendation systems research.
2.1 Term frequency-inverse document frequency
Term frequency-inverse document frequency (TF-IDF) is a measure of how important a word is to a document (Leskovec et al., Reference Leskovec, Rajaraman and Ullman2014). It is widely used in the text mining approaches. TF-IDF value is calculated by multiplying two measures: term frequency (TF) and inverse document frequency (IDF).
TF value is calculated using the following equation:
$${\rm TF}_{{ij}} {\equals}{{f_{{ij}} } \over {{\rm max}_{k} f_{{kj}} }}$$
In the equation above, TF ij is the term frequency value of word i in document j, f ij is the frequency of i in j (number of occurrences), max k f kj is the frequency of the most frequent word k in document j. Thus, the most frequent term in document j gets a TF of 1 and other terms get fractions as their term frequency for this document.
IDF value is calculated using the following equation:
$${\rm IDF}_{i} {\equals}{\rm log}_{2} {N \over {n_{i} }}$$
In the equation above IDF i is the inverse document frequency of word i, n i is the number of documents that word i appears in, and N is the total number of documents that we have in our collection.
Then TF-IDF value becomes:
$${\rm TF}{\minus}{\rm IDF}_{{ij}} {\equals}{\rm TF}_{{ij}} {\times}{\rm IDF}_{i} $$
The terms with the highest TF-IDF score are often the terms that best characterize the topic of the document. We use TF-IDF because it can eliminate too frequent words, as we do not want the system to be affected by words, which appears too frequently.
2.2 Regression algorithms
In this section, we are providing a list of regression algorithms which can be used with LIRA.
2.2.1 Gaussian process
A Gaussian process is a collection of random variables, any finite number of which have (consistent) joint Gaussian distributions (Williams & Rasmussen, Reference Williams and Rasmussen2006). Every Gaussian process can be fully specified with its mean function m(x) and co-variance function k(x, x'). Given N data points, X n , t n where the inputs x are vectors of some input dimension, and targets t are the output values, the Gaussian process tries to infer the underlying function, f(x) from the given data (MacKay, Reference MacKay1998). When that function is inferred, a Gaussian process can use that function to predict t n+1 for a new data point x n+1.
2.2.2 Support vector regression
The support vector machine, originally proposed by Vapnik (Reference Vapnik2013), is a learning algorithm (Cristianini & Shawe-Taylor, Reference Cristianini and Shawe-Taylor2000; Palaniswami & Shilton, Reference Palaniswami and Shilton2002) with the ability to provide function estimation. By using a mapping, Φ:X→F, where X is the domain and F is usually a high-dimensional feature space, support vector regression operates in feature space to approximate unknown functions in an output space Y, thereby using nonlinear functions to linearly estimate an unknown regression.
2.2.3 Neural networks
Artificial neural network is a widely used regression method (Wang, Reference Wang2003). It aims to simulate the working method of human brain to learn and make predictions about data. It is implemented by creating rather simple nodes, called perceptrons, and creates fully connected layers to form a network. Perceptrons are linear separators, which gets the weighted sum of inputs and checks if that some is over or under the threshold value for that perceptron.
Actual learning in neural networks is done by learning the correct weight values between perceptrons. At every iteration, weights between every pair of nodes are updated using
$$W_{{i,j}} \leftarrow W_{{i,j}} {\plus}\alpha {\times}a_{i} {\times}{\rm \Delta }_{j} ,$$
where Δ j is the error associated with node j, α is the learning rate, and a i is the output of node i. When a neural network is learning, the only correct value which is known is the output value, therefore the error of the system can only be measured for the output layer. So the error associated with output node, o, is
$${\rm \Delta }_{o} {\equals}{\rm Err}_{o} {\times}g({\rm input}_{o} ).$$
The error made by output nodes is back-propagated to the hidden nodes to find their contribution on the error. So error for a hidden layer node h becomes:
$${\rm \Delta }_{h} {\equals}g({\rm input}_{o} ){\times}\mathop \sum\limits_i {\rm \ztbnd }W_{{h,i}} {\rm \Delta }_{i} $$
2.3 Recommendation systems
Recommendation techniques can be broadly divided into two categories: non-personalized and personalized recommendation techniques. Non-personalized recommendation techniques compute the average statistics of the recommended items and recommend the same item set to all customers, that is, they do not consider individual user preferences. For instance, a bookstore might recommend a bestseller book to all of its customers. Non-personalized systems have the advantage of being fast. Moreover, they do not suffer from the cold start problemFootnote 1 , since they do not need an initial data (ratings, etc.) from each customer. Nevertheless, the quality of the results is low because of the lack of personalization.
Personalized recommendation techniques provide better results by considering individual user preferences. Two well-known approaches are content-based and collaborative filtering. Moreover, some hybrid approaches have emerged to overcome the problems of these approaches. Adomavicius and Tuzhilin (Reference Adomavicius and Tuzhilin2005) provide a detailed survey about recommender systems.
Content-based filtering algorithms recommend items that are similar to the ones that the user has liked. Content-based systems usually depend on textual content, for example, keywords. Information retrieval (IR) techniques, such as TF-IDF (Manning et al., Reference Manning, Raghavan and Schütze2008), can be applied on the keywords. The similarity among items can be determined by some scoring heuristics, such as cosine similarity, after the TF-IDF vectors are calculated. Apart from IR techniques, Bayesian classifiers and different machine learning techniques like clustering decision trees and artificial neural networks can be used for the similarity calculation (Adomavicius & Tuzhilin, Reference Adomavicius and Tuzhilin2005). Content-based recommendation systems have been used by the researchers on various topics (Lops et al., Reference Lops, De Gemmis and Semeraro2011), such as music (Soleymani et al., Reference Soleymani, Aljanaki, Wiering and Veltkamp2015), books (Mooney & Roy, Reference Mooney and Roy2000), movies (Debnath et al., Reference Debnath, Ganguly and Mitra2008), etc. However, these systems need the items to be marked with features, which is usually objective, and cannot make use of user similarities.
Collaborative filtering algorithms identify people with overlapping interests. These algorithms rely on the assumption that people who have exhibited similar interests in the past will continue to have similar interests in the future. Instead of considering correlation of items as in content-based algorithms, correlation between users’ preferences is examined. This is achieved by analyzing user ratings of items: if two users have provided similar ratings for many items, then it is concluded that these users have similar preferences. Another way of finding users with similar interests is to match demographic characteristics of users, for example, age, education, geographic locations, gender, etc. After determining like-minded users, the transitive relationship between users and items is utilized. For instance, if user c 1’s preferences are similar to that of user c 2’s and user c 2 likes item s 1, s 1 is recommended to c 1 if c 1 has not viewed item s 1 yet.
As in the case of content-based approaches, IR techniques, like cosine measure, can be used to calculate similarities between users. However, in collaborative filtering, similarity between vectors of the actual user-specified ratings is measured (Adomavicius & Tuzhilin, Reference Adomavicius and Tuzhilin2005). Moreover, different machine learning techniques are also applicable for identifying similarities between users (Mitchell, Reference Mitchell1997; Adomavicius & Tuzhilin, Reference Adomavicius and Tuzhilin2005; Alpaydin, Reference Alpaydin2014).
RegSVD, proposed by Paterek (Reference Paterek2007), improves on SVD by learning factor vectors directly on known ratings through a suitable objective function which minimizes prediction error. The proposed objective function is regularized in order to avoid over-fitting. Gradient descent is applied to minimize the objective function.
Collaborative filtering techniques have their own limitations. They work reliably only when there are sufficient number of users in the system with overlapping characteristics. Another concern is that when a new item is added to the system, it cannot be recommended to others until a number of people have rated it. Furthermore, these techniques are computationally expensive, and the recommendation process becomes cumbersome for millions of users and items. Collaborative filtering has also been tried on various domains such as e-commerce (Linden et al., Reference Linden, Smith and York2003), movies (Ungar & Foster, Reference Ungar and Foster1998), etc. Another shortcoming is that these systems cannot identify when two users rate an item with the same rating with different reasons.
2.3.1 Recommendation with topic models
A topic model is a statistical model that generates documents (strings of words) from some set of topic clusters. Parametric estimation techniques and unsupervised clustering algorithms such as the expectation-maximization algorithm (Dempster et al., Reference Dempster, Laird and Rubin1977) can fit models to observed documents. With the model, it is possible to estimate a likely distribution over topics on a document level and a distribution over words on the topic level. Topic models are an effective tool for characterizing documents since they are not necessarily dependent on a language’s underlying grammar, and yet can identify significant topic trends using the rate of each word’s appearance. Some of the more popular existing topic model frameworks are latent semantic analysis (Deerwester et al., Reference Deerwester, Dumais, Landauer, Furnas and Harshman1990) and latent Dirichlet allocation (LDA) (Blei et al., Reference Blei, Ng and Jordan2003).
Topic models are often used in recommender systems in a variety of creative ways (Jakob et al., Reference Jakob, Weber, Müller and Gurevych2009; Hariri et al., Reference Hariri, Zheng, Mobasher and Burke2011; Chen & Wang, Reference Chen and Wang2013; Liu et al., Reference Liu, He, Wang, Song and Du2013; Bao et al., Reference Bao, Fang and Zhang2014). Generally, such recommender systems use topic models to generate latent features in their documents, which can then be. supplied to some supervised learning algorithm (Paterek, Reference Paterek2007; Jakob et al., Reference Jakob, Weber, Müller and Gurevych2009; Chen & Wang, Reference Chen and Wang2013).
The fLDA topic model from Agarwal and Chen (Reference Agarwal and Chen2010) extended the LDA model to include ratings as well. The paper approaches the problem where a system has some users and items, and the goal of the recommender system is to make recommendations based on the user’s rating history and a textual description of each item. The fLDA model simultaneously learns: (a) the correlation between items’ documents and features using LDA, and (b) the users’ affinity towards each topic, to produce (c) a rating distribution for each user i and item j. The rating distribution inside the model is defined as a Normal (or binomial) distribution:
$$y_{{ij}} \sim{\rm Normal}\,(\alpha _{i} {\plus}\beta _{j} {\plus}s_{i}^{T} \bar{z}_{j} ,\sigma ^{2} )$$
where y
ij
is the rating, α
i
is the user’s rating bias, β
j
is the item’s rating bias, s
i
is the user’s affinity towards each topic as a vector, and
$\bar{z}$
is a latent variable measuring how each word in the description of item j correlates with a topic.
Similarly, the TopicMF model by Bao et al. (Reference Bao, Fang and Zhang2014) incorporates a topic model based on matrix factorization; this model considers both ratings and reviews of items in its recommendation. Their model combines this topic model using matrix factorization with another recommender system using matrix factorization to produce a combined model. Given I users, J items, D = I×J reviews available, W total words and an observed user-to-item rating matrix r and a review-to-word frequency matrix F, the algorithm attempts to find the best factorization for K hidden topics:
${\bf r}{\equals}{\bf u}_{{I{\times}K}} {\bf v}_{{J{\times}K}}^{T} ,{\bf F}{\equals}{\bf \Theta }_{{D{\times}K}} {\bf \Phi }_{{W{\times}K}}^{T} .$
The hidden matrices are learned by minimizing the least squares equation:
${\rm min}_{{{\bf u},{\bf v},{\bf \Theta },{\bf \Phi }}} \mid\,\mid{\bf r}{\minus}{\bf uv}^{T} \mid\,\mid{\plus}\mid\,\mid\,{\bf F}{\minus}{\bf \Theta \Phi }^{T} \mid\,\mid\plus\gamma $
where γ is an additional term that includes bias values and measures to prevent learned parameters from growing too large.
McAuley and Leskovec (Reference McAuley and Leskovec2013) have proposed the hidden factors as topics (HFT) algorithm, which creates topics to identify the hidden topics in the reviews to describe the items of interest, and also the users’ interest in those items. They are also using their algorithm to identify the categories of those items automatically.
These systems work on reviews provided by the users, but in the process of combining them into topics they lose the information about the reviewers. For example, if two people both mention that they like the ‘view’ of two different hotels, those items and users will be considered similar by these systems. However, the types of the views can be different and the users opinion of a ‘good view’ might also differ between users.
2.3.2 Hybrid recommendation systems
Sparsity of available data makes the personalization of rating prediction difficult, as it makes it hard to link users and products to each other. To overcome this problem, some researchers focused on generating hybrid recommendation systems to combine the strengths of different approaches (Kazienko & Musiał, Reference Kazienko and Musiał2006). Mostly, the base of these hybrids is a combination of content-based (Schein et al., Reference Schein, Popescul, Ungar and Pennock2002; Kazienko & Musiał, Reference Kazienko and Musiał2006; Levi et al., Reference Levi, Mokryn, Diot and Taft2012) and collaborative recommendation systems (Sarwar et al., Reference Sarwar, Karypis, Konstan and Riedl2001). Some researchers tried to incorporate the information from social networks to this hybrid recommendation systems either as social tags (Middleton et al., Reference Middleton, Shadbolt and De Roure2004) or social trust (Sugiyama et al., Reference Sugiyama, Hatano and Yoshikawa2004). Some of the more recent studies focused on creating hybrid recommendation systems which also make use of sentiment analysis approaches (Homoceanu et al., Reference Homoceanu, Loster, Lofi and Balke2011; Yang et al., Reference Yang, Zhang, Yu and Wang2013). Leung et al. (Reference Leung, Chan and Chung2008) proposed a method to extract association rules from data and use it as a supplement to enhance the recommendation system.
2.3.3 Recommendation using trust
The area of trust-based recommender systems has been the object of extensive study for the past years. Indeed, trust has been shown to provide significant improvements to classical collaborative filtering techniques. The main difference between most of these trust-enhanced methods is the acquisition of trust values between actor pairs.
Golbeck (Reference Golbeck2005) introduced TidalTrust algorithm, which uses a modified breadth-first search, to estimate the trust by using transitive rules. In this study, the trust of user u for user v is calculated using u’s trusted users’ trust on v. Massa and Avesani (Reference Massa and Avesani2007) and Massa and Bhattacharjee (Reference Massa and Bhattacharjee2004) proposed a similar method where instead of only looking at one level trust transitivity, they remove cycles from the trust network. Using a trust propagation horizon phase, they allow the trust to diffuse along the network.
While the studies above require an explicit statement about direct trust between users, it is very difficult to get that information from user explicitly. Therefore, some other studies focused on defining their own implicit trust values based on implicit information. O’Donovan and Smyth (Reference O’Donovan and Smyth2005) proposed a trust value based on the similarity of two user’s ratings, while Wang et al. (Reference Wang, Yin, Liu and Huang2011) generated a trust metric based on the similarity among users’ tastes.
3 Methodology
In this section, we define how LIRA works. We first describe the preprocessing steps, that is, how the data are prepared, and then explain how LIRA uses that data. While the dataset includes detailed meta-data about users, items, and reviews, our recommender system will only examine the textual content and numeric rating associated with the records. We do not need the rating to accompany the review so we create two sets of triples: (Reviewer, Item, Review) and (User, Item, Rating). This property shows the reviews and ratings are only related through the items. Note that sets of Users and Reviewers are not necessarily disjoint. If a person rates items but does not write reviews, they will only appear in the set of Users. Likewise, if a person writes reviews but do not provide ratings, such as bloggers or critiques, they only appear in the set of Reviewers. However, if a person both writes reviews and provides ratings, they appear both in Reviewers and Users sets.
3.1 Preparing data
For this study, we have used ‘Amazon Reviews’ dataset in five categories (McAuley et al., Reference McAuley, Pandey and Leskovec2015a,Reference McAuley, Targett, Shi and van den Hengelb). In addition to these, we have also used ‘GoodReads’ dataset that consists of book ratings and reviews from many different languages.
In all datasets we used, we run the following pre-processing steps:
1. Remove products which do not have any reviews, along with ratings for those products.
2. Remove users who have less than 10 ratings, along with ratings by those users.
3. Remove reviews for products which do not have any ratings.
Information about the resulting dataset is shown in Table 1.
Properties of datasets
Instead of following the traditional method of randomly splitting the data into two sets (training and testing), we use a slightly different approach. Because our algorithm works on every user separately, we split the data at the user level. We chose a random split of 80–20% of ratings for each individual user and use the former as the training and trust calculation set, and the latter for testing our algorithm. All of our datasets are large enough to validate with hold-out method.
3.2 LIRA details
LIRA uses a framework that utilizes separate learners for every available source (see Figure 1). It learns the writing behavior and predictiveness of reviews from each source separately. For this example, the rating scale used is [1, 5].
Overview of LIRA
We start the initial processing by creating data points that consist of the reviews by every source separately and marking them with target user’s rating for the corresponding item. That data point is then converted to a feature vector where the features are the words in the review and their values are their respective TF-IDF (Leskovec et al., Reference Leskovec, Rajaraman and Ullman2014) values. Note that LIRA is a general framework, within which any text to feature vector conversion approach can be used at this step.
3.2.1 Text regression
We choose a machine-learning regression algorithm, such as neural network (Wang, Reference Wang2003), Gaussian process (Williams & Rasmussen, Reference Williams and Rasmussen2006), etc. to learn to predict the target user’s ratings for unseen items. Any regression algorithm can be used at this step of LIRA. In Figure 1, Learner A1 will only find the common item I1 among target user’s ratings and source A1’s reviews. Learner A1 will learn that the word sequence ‘Chick-Flick’ is likely to result in the target user rating that item 5. Similarly, Learner A2 will find two common items, I1 and I3, getting the reviews from A2 for those items. Learner A2 will learn that the word ‘Romantic’ by A2 means high ratings by target user and ‘Historic’ by A2 means low ratings by target user. Learner A2 will also realize that word ‘Nice’ by A2 is not useful for predicting target user’s ratings.
3.2.2 Training
In the training phase, LIRA creates learners for every target user-source pair, to learn how the source’s reviews are predictive of the target user’s ratings. Hence, for every target user, User T , and source, S, pair, LIRA creates Learner TS and feeds (Review SI , Rating TI ) pairs to this learner. Here, Review SI is source S’s review of Item I and Rating TI is User T ’s rating of Item I .
Since the reviews are in free-form text, they are first converted to feature vectors using TF-IDF. It should be noted that text-to-feature conversion is done separately for every source in LIRA. Therefore, as long as a given source is consistent with its individual writing style and language usage, stop-word elimination will eliminate the words that are not informative only for that particular source’s reviews. For example, if the source starts every review with a ‘Merhaba Footnote 2 ’, this word will be identified as a stop-word, and will be eliminated.
These feature vectors are used to train, Learner
TS
, using a plug-and-play regression algorithm. The algorithm for training LIRA is presented in Algorithm 1. Note that if a source has not reviewed any items rated by the target user, it cannot be used as a predictor for this user’s rating, and is eliminated from consideration (see line 4).
3.2.3 Trust calculation
We have mentioned in Section 2.3.3 that existing recommendation systems assign a trust value between users either by using rating similarities between users or by analyzing social connections between them. Unlike those systems, we decided to assign trust values to represent their capability of making predictions for the target user.
As mentioned in Section 3.1, the available dataset is split into training and trust calculation sets. First, all the learners are trained using the training set. Then we make all the learners predict every item in the trust calculation set. This approach is similar to the use of validation set in common supervised machine learning approaches. Root mean square error (RMSE) for learner L is calculated as:
$${\rm RMSE}(L){\equals}\sqrt {{{\sum_{I\in{P}_{L} } ({\rm Prediction}(L,I)\hskip-1pt{\minus}r_{I} )^{2} } \over {\,\mid\,{P}_{L} \,\mid\,}}} .$$
In Equation (7), P L is the set of items which L can make predictions on, r I is the actual rating, and Prediction(L, I) is the rating prediction of L for item I.
This error measure is useful but not sufficient for a complete evaluation of our system. If we only consider the RMSE, a Learner A which made only one, albeit perfectly accurate, prediction will have the same trust value as a Learner B who was able to perfectly predict a large number of items. To address this issue, we include the number of predictions in our trust calculation. We use a sigmoid function to ensure that the magnitude of this count does not suppress the error value. As the number of predictions is non-negative, we choose the sigmoid function given in Equation (8) such that sigmoid(0)=0 and sigmoid(∞)=1. We introduce the β parameter to adjust how quickly our sigmoid function reaches ~1 (see Figure 2).
$${\rm sigmoid}(x){\equals}{2 \over {1{\plus}{\rm e}^{{{\minus}\beta x}} }}{\minus}1.$$
Behavior of adjusted sigmoid function with different β values
We need to combine the error value and number of common items for every learner, to get the final trust value. Trust should be positively correlated with the number of common items. So we choose the sigmoid function, with the number of common items as argument, as the numerator of the trust expression. The error measure calculated must be negatively correlated with the trust value. As the best case has 0 errors, to ensure the trust value lies between [0, 1] we use (1+errors) as the denominator term. For tuning the relative importance of the number of common items and error, we use an α parameter as the exponent of the denominator. The resultant trust value is then calculated as
$${\rm trust}(L){\equals}{{{\rm sigmoid}(\,\mid\hskip-2pt P_{L}\hskip-2pt \mid\,)} \over {(1{\plus}{\rm RMSE}(L))^{\alpha } }}.$$
Figure 3 shows how the importance of RMSE and the number of predictions change according to the values assigned to α and β parameters.
Behavior of trust function with different α and β values
In some cases, a prediction may be made by a new learner, where they do not have any previous predictions to calculate a trust value. To avoid losing the information from those learners, we assign them a default trust value. Since we do not know about the capabilities of this learner yet, we conservatively assign them the minimum possible trust value, which is calculated by assuming they have made a single prediction with maximal prediction error possible. Since the maximum value of error in a prediction can be the difference between maximum and minimum values in the rating scale we end up with the following expression (Equation (10)) for the trust value of the new learner (l new):
$${\rm trust}(l_{{{\rm new}}} ){\equals}{{{\rm sigmoid}(1)} \over {(1{\plus}{\rm Rating}_{{{\rm max}}} {\minus}{\rm Rating}_{{{\rm min}}} )^{\alpha } }}.$$
3.2.4 Predicting target user’s ratings for candidate items
In the prediction step of LIRA, we predict the rating for the unseen items by the target user. To achieve this, we will use the regression model trained in Section 3.2.2 and trust values calculated in Section 3.2.3.
For a candidate item, i, we first retrieve every source that has written a review for i and also has a learner trained for target user User t . Every source’s review is separately fed into their respective learner for t, which calculates predictions by individual sources. These predictions are then combined using a weighted average approach, where the weights correspond to trust values for each source. To avoid losing any predictive value in reviews from new sources, which did not have any reviews that can be used in the trust calculation step, we set their trust level to a default value explained in Equation (10).
$${\rm Prediction}(t,i){\equals}{{\sum_{s\in({\rm Sources}_{i} {\cap}{\rm Trained\,\,Sources}_{t} )} \,{\rm Trust}_{{ts}} {\times}{\rm Prediction}_{s} (t,i)} \over {\sum_{s\in({\rm Sources}_{i} {\cap}{\rm Trained\,\,Sources}_{t} )} \,{\rm Trust}_{{ts}} }}$$
The prediction LIRA makes for a Target User
t
−Item
i
pair is calculated using the formula in Equation (11). In this equation, Trust
ts
is the trust value calculated for Source
s
for Target User
t
’s ratings, as described in Section 3.2.3. Prediction
s
(t,i) is the prediction made by Source
s
’s review for Target User
t
−Item
i
. Source
i
is the set of sources who have provided reviews for Item
i
and Trained Sources
t
is the set of sources with a trained learner for Target User
t
. Note that, for some Target User
t
−Item
i
pairs,
$${\rm Sources}_{i} \,{\cap}^ \,{\rm Trained\,\,Sources}_{t} {\equals}{\Oslash}$$
. Therefore LIRA will not be able to make predictions for those pairs.
Learners which do not have a review written for the item, we are trying to predict the rating for cannot make prediction and we exclude those sources. For example, Learner A2 will not be able to make a prediction for I5.
The final prediction for an item I
X
,
$$r\hat{I}_{X} $$
is calculated as:
$$r\hat{I}_{X} {\equals}{{\sum_{u\in{\rm Rev}(I_{X} )} {\rm Trust}(L_{u} ){\times}{\rm Prediction}(L_{u} ,I_{X} )} \over {\sum_{u\in{\rm Rev}(I_{X} )} {\rm Trust}(L_{u} )}},$$
where Rev(I X ) is the set of reviewers who have provided review for item I X , Trust(u) is the trust value for that reviewer, and Prediction(L u, I X ) is the prediction by Learner u, L u , for item I X .
3.3 Scalability
Given the description of the approach, it may appear that this approach is computationally expensive because of the need for separately training a learner for every user-source pair, where every reviewer is used as a source for every other user. For a system with N users, where each user is also a reviewer, the number of learned models will be N×(N−1). However, some properties of the algorithm can be used to redress this problem as follows:
1. It is immediately obvious that creating the classifiers for separate users are independent processes. Therefore, this approach is easily parallelizable.
2. More importantly, LIRA is designed to be an agent-based system. So, every target user can run their own agent, and therefore their own learners, on their local computer, allowing them to get recommendations from the system without the burden of running LIRA for any other userFootnote 3 . This way they will be training only one classifier which contains N sub-learners, where N is the number of sources, which will not be computationally prohibitive.
Another approach, to make the system even more efficient and part of our planned future work is to train only a fixed number of most trustworthy sources for each target user. Assuming the top K sources per user is identified, the number of total models in the system will be reduced to N×K. Periodically, the pool of trusted sources can be updated by a process of exploration. This problem can be represented as an instance of multi-arm bandit problem (Gittins et al., Reference Gittins, Glazebrook and Weber2011) where reviewers correspond to the available options and their accuracy on predicting the target user’s ratings is the pay-off for the corresponding option.
4 Experimental setup
In this section, we explain the experiments we run, and the motivation behind them.
4.1 Selection of methods to use
We evaluate our algorithm on three datasets to demonstrate the general applicability and domain independence of our approach. We selected ‘Automotive’, ‘Baby’, and ‘Digital Music’ as the experimental domains because of their distinct characteristics. We are assuming that the opinions are mostly objective and based on the quality of the item for the ‘Automotive’ domain, that the opinions are mostly subjective and based on the preference of users for the ‘Music’ domain, and that the opinions will be a combination of objective quality and subjective personal preference in the ‘Baby’ domain.
For selecting appropriate method for regression and the tokenizer used in text to feature conversion, we run LIRA using the candidate methods on 100 randomly selected users and test which method yields the best results.
4.1.1 Regression method selection
To choose which regression method to use, we have tested LIRA using following regression learning algorithms: Neural network, Gaussian process, and SMOReg. For these algorithms, we used the default settings from WEKA. For the number of hidden nodes is (attributes+classes)/2, learning rate is 0.3, and sigmoid function has been used as the activation function. Kernel function used for both Gaussian Process and SMOReg is polynomial kernel function (Fan et al., Reference Fan, Heckman and Wand1995).
4.1.2 Tokenizer selection
To choose which tokenizer method to use, we have tested LIRA using following tokenizers: Word Tokenizer which only extracts words from a given text, and N-Gram Tokenizer, which tries to extract N-Grams, including unigrams (words) (Mayfield & McNamee, Reference Mayfield and McNamee2003). We limited the N-Grams to unigrams, bigrams, and trigrams, to reduce to complexity and prevent the data from getting too sparse.
4.2 All ratings experiment
For this set of experiments, we have used all the ratings and reviews in the dataset. LIRA works separately for every user, so for every user LIRA goes over every other users’ review to predict the rating of the target user.
As LIRA requires a separate trust calculation set, it splits the given training set 75–25%Footnote 4 , and then uses the larger subset for training and the smaller one for trust calculation. Similar to the split explained in Section 3.1, this split is also done at the user level. This process is also used for the next set of experiments.
We have chosen to compare our algorithm against state of the art algorithms in both rating-based and review-based categories. We have chosen RegSVD (Paterek, Reference Paterek2007) as the rating-based competitor, and HFT (McAuley & Leskovec, Reference McAuley and Leskovec2013) as the review-based competitor.
4.3 100 user experiment
This set of experiments are done to test LIRA’s capability of making use of the reviews from other sources. We randomly choose 100 users from the dataset as the group of target users. In our case, other sources then correspond to reviews from other users not in the chosen set. In general, however, these others sources can be taken from completely different platforms, such as websites, blogs, critique reviews, etc.
For the 100 users chosen, we only use their own ratings and not their own reviews, but use the reviews from all the users in the dataset, to predict their ratings. Hence, we consider all the users, except the selected 100, as outside sources for reviews. Note that the reviews from users, except the target user, are also used in the prediction, since those 99 users are considered on-site reviewers.
Since the competing algorithms are only capable of working with on-site data, we test them using ratings and reviews of the 100 selected users.
This approach is motivated by the following real-life scenario. Consider a newly built movie recommendation website which has few (in our example 100) users. As this system will only have the ratings and reviews of these few users, the only rating data that can be used for the recommendation systems is that. However, LIRA can make use of outside sources for the recommendation, such as critique reviews, fan blogs, etc.
Recommendation services with only such small number of users (100 users in this case), corresponds to very sparse datasets. LIRA is not adversely affected by this sparsity, as it is able to make use of reviews from other outside sources.
We made sure that the same set of 100 randomly selected users and exactly the same training and test sets are used to test competing algorithms and LIRA.
4.4 Error measurement
To calculate the error made by our algorithms and competing algorithms, we used the two most commonly used error measurement approaches in recommendation systems: mean absolute error (MAE) (Willmott & Matsuura, Reference Willmott and Matsuura2005) and RMSE (Levinson, Reference Levinson1946).
4.4.1 Mean absolute error
MAE is a very intuitive error measurement approach. As the name suggests MAE is an average of the absolute errors made while predicting each instance (see Equation (13)). MAE is primarily used when the values to be predicted are in the same scale and every error made by the predictor is equally important.
$${\rm MAE}{\equals}{{\sum_{(u,i)\,\mid\,R_{{u,i}} } \,\mid r_{{u,i}} {\minus}\hat{r}_{{u,i}} \mid\,} \over {\,\mid\{ (u,i)\mid R_{{u,i}} \} \mid\,}}$$
In this equation, R
u,i
is a Boolean value signaling whether user u has rated item i in our dataset, as we can only measure the error if we know the actual rating. Variables r
u,i
and
$\hat{r}_{{u,i}} $
denote the actual and predicted ratings, respectively. Finally, the sum of the absolute differences between actual and predicted ratings are divided by total number of predictions to obtain the average.
4.4.2 Root mean squared error
RMSE calculates the standard deviation of the errors on the instances by the predictor. RMSE is primarily used when the greater errors are to be penalized harsher. Since error values are squared, RMSE of uniformly distributed errors will be lower than randomly distributed errors, given that MAE remains constant.
The formulation for MAE is given in Equation (14).
$${\rm RMSE}{\equals}\sqrt {{{\sum_{(u,i)\,\mid\,R_{{u,i}} } (r_{{u,i}} {\minus}\hat{r}_{{u,i}} )^{2} } \over {\,\mid\{ (u,i)\mid R_{{u,i}} \} \mid\,}}} $$
In this equation, R
u,i
is a Boolean showing whether user u has a rating on item i in our dataset, as we can only measure the error if we know the actual rating. Variables r
u,i
and
$\hat{r}_{{u,i}} $
denote the actual and predicted ratings, respectively. Finally, the sum of the squared differences between actual and predicted ratings are divided by total number of predictions and square-rooted to get RMSE value.
5 Results
In this section, we will present and analyze the results from the above-mentioned set of experiments.
5.1 Selection of methods to use
5.1.1 Regression method selection
We compare error per user by means of MAE and RMSE for three candidate regression algorithms: neural network, Gaussian process, and SMOReg.
Box-plots in Figure 4 show that Gaussian process algorithm outperforms other two algorithms, and hence we choose Gaussian process algorithm to conduct our further experiments.
Box-plot of errors per user made by LIRA using each candidate regression algorithm
5.1.2 Tokenizer selection
We compare error per user by means of MAE and RMSE for two tokenizer, Word Tokenizer which only extracts words, and N-Gram tokenizer which also extracts N-Grams.
Box-plots in Figure 5 and data in Table 2 show that the average errors with or without N-Grams are approximately same. Given that including N-Grams do not increase the accuracy of the system, we follow the Occam’s Razor principle and conduct further experiments with only the Word Tokenizer.
Box-plot of errors per user made by LIRA using each tokenizer
Average error of LIRA using two tokenizers
MAE=mean absolute error; RMSE= root mean square error.
5.2 All ratings experiment
The results of this experiment compare LIRA against the competing algorithms, in the condition where all the data available come from the same environment, therefore all three algorithms are able to make use of all the data. The resulting MAE and RMSE values are presented in Table 3, and bold values mark the minimum MAE and RMSE in every domain. These results show that LIRA outperforms both competing algorithms on all six domains. Since LIRA is language independent, it is able to generate better predictions than competing algorithms, especially in the GoodReads domain. It can also be seen that while both competing algorithms suffer from limited data in small-sized domains, such as Automotive, LIRA is still able to make accurate predictions.
Results of LIRA against competing algorithms in ‘All ratings experiment’
HFT=hidden factors as topics; MAE=mean absolute error; RMSE=root mean square error; LIRA=Language Independent Recommender Agent.
As explained in Section 3.2.4, for Target User t −Item i pairs where t does not have learners trained for any of the sources who reviewed i, LIRA will fail to make any prediction. Therefore, the coverage of LIRA is not 100%.
We applied paired t-tests to verify if the improvements LIRA provides are significant. The results show that all the improvements are significant at p<0.001 level (see Table 4).
t-values for t-test comparison of LIRA against competing algorithms
HFT=hidden factors as topics; LIRA=Language Independent Recommender Agent.
5.3 100 user experiment
The results of this experiment compare LIRA against the competing algorithms, in the condition where data from external sources are available. Since only LIRA is able to make use of external sources, competing algorithms will only be able to make use of data from the selected 100 users.
The resulting MAE and RMSE values are presented in Table 5, and bold values mark the minimum MAE and RMSE in every domain. These results show that LIRA outperforms both competing algorithms on all six domains. These results show that LIRA continues to produce low error rates and is not significantly affected by small size of the dataset. On the other hand, error rate of the competing rating-based algorithm, RegSVD, drastically increases when the number of ratings is low. Error rate of competing review-based system, HFT, also increases, as it is only capable of making use of reviews which are accompanied with ratingsFootnote 5 .
Results of LIRA against competing algorithms in ‘100 user experiment’
MAE=mean absolute error; RMSE= root mean square error; LIRA=Language Independent Recommender Agent.
6 Conclusions
While most of the work on recommendation systems are based on ratings and rating similarities, limited representation power of ratings highlights the need for new recommendation systems which can make of use of the other information available to improve recommendation accuracy. With the introduction of Web 2.0 and burgeoning of online platforms, which rely on user-generated content, a rapidly increasing volume of on online and off-site sources make available item reviews that can provide valuable insights about user preferences and can be used to improve the coverage and accuracy of existing recommendation systems. Reviews written by users are more representative and detailed representation of a user’s preference for an item. Descriptive reviews also express users’ opinions about the features of candidate items and help us develop a deeper understanding of their preferences.
In this paper, we have proposed an agent-based approach, using the trust aspect of agent societies, that uses regression algorithms to predict the ratings of a target user for a candidate item. Our approach creates agents for both target users and reviewers to create accurate recommendations. LIRA consists of targets user’s agent, which uses messages, in the form of words or sequence of words in their reviews, from the other agents. The agent of the target user learns predictive knowledge from those messages, which are subsequently used for producing recommendations. LIRA assigns trust to each source’s agent using their accuracy on a set of unknown items. Final prediction of LIRA is calculated using a trust-weighted average of sources’ individual prediction. Among other benefits, this approach allows the recommendation system to differentiate two users, who use the same word for different reasons, by inferring from their past reviews where identical words have different predictive values for the target user when used by different sources.
Another key feature of LIRA is that it does not use the ratings of the sources and uses only target user’s ratings. This results in the LIRA approach having two key advantages. First, it can use reviews from different sources, such as different websites or fan blogs, without needing those texts to be accompanied with a rating value. This makes LIRA useful for addressing the ‘cold start’ problem of new recommender systems which do not have many users to write reviews for them. Second, by not taking the sources ratings into account, LIRA treats reviews as descriptive texts. For example, if one or more sources complain about a movie, because they found it ‘too romantic’, LIRA can infer that the movie might be recommended to a target user who does enjoy romantic movies.
Since LIRA does not deal with the meaning of the words, and only use them as signals between agents, the language in which the review was written in becomes irrelevant. As long as sources are consistent with their choice of language or writing style, LIRA will be able to effectively leverage their reviews. The language-independent property of LIRA, whereby reviews written in different languages can be gainfully utilized, is a distinctive advantage compared to other review-based recommender systems that combine reviews from different sources for purposes such as topic modeling (Agarwal & Chen, Reference Agarwal and Chen2010; McAuley & Leskovec, Reference McAuley and Leskovec2013).
Last, but not the least, a key advantage of LIRA is that it does not require the target users to write reviews. It is well known that people write reviews much more infrequently than they rate items online. Hence LIRA can be used in many scenarios where review-based recommender systems that require target users to write reviews can be used.
We presented experimental results from two sets of experiments. In the first set, there was a rating accompanying every review. This experiment showed that analyzing reviews from different sources separately, allows LIRA to outperform state-of-art algorithms. Domains where LIRA was more successful than its competitors are likely those where people are more capable and likely to write more descriptive reviews. The second set of experiments show that when there is very little user and rating data available, only LIRA can make use of other sources of information to make accurate recommendations and thus outperform its competitors.
One of the properties of LIRA, which might look like a disadvantage, is that it needs to train regression algorithms for every user-source pair. However, it should be mentioned that the agents work independently from each other, which makes this approach highly parallelizable. We are currently also working on an approach which provides the same advantages that LIRA provides, without having to create separate learners for every source.
