1 Introduction
Folksonomy is the other name for user-generated metadata. From implementation point of view, folksonomy is the complete group or set of tags. Tag consists of one or two keywords that users of a shared content management system assign to individual pieces of content for grouping and categorizing them, and for long-term retrieval of those contents. The public nature of these tags is a key factor; as it enables the users to find out how others had used the same tags in categorizing their own content. And in addition the users also view the terms others have added in the folksonomy (Sturtz, Reference Sturtz2004).
As users contribute towards it, it allows gathering of crowd intelligence. But if we look at the negative side, the users can also give incorrect tags, which will make folksonomy a mess constituting of erroneous words and also degrades its usefulness. Table 1 gives an overview of the types of erroneous tags encountered in the folksonomy identified by Solskinnsbakk and Gulla (Reference Solskinnsbakk and Gulla2001), Guy and Tonkin (Reference Guy and Tonkin2006), Nandipati (Reference Nandipati2011). Cattuto et al. (Reference Cattuto, Baldassarri, Servedio and Loreto2007) showed that how fast the growth of vocabulary takes place with the passage of time in Del.icio.us. By closely examining the growth, question arises regarding the quality of the vocabulary and about the worthiness of the folksonomy.
Erroneous tags in folksonomies
The limitations and drawbacks of folksonomies have been well documented; what is required is the in-depth analysis of efforts for keeping consistently well-maintained folksonomy in terms of quality. In this paper, we have investigated the research efforts to solve the said problem from four aspects namely selection of quality tags, tag management features provided by folksonomy applications, folksonomy cleaning and interoperability across the platforms. Existing related work by Peters and Weller (Reference Peters and Weller2008) covers two aspects while one aspect is covered in Dotsika (Reference Dotsika2009). To the best of our knowledge none of the reviews cover all four aspects, making our paper coverage more as compared with others.
The paper is structured as follows: in Section 2 the significance of the selected aspects has been justified. Analysis of the major approaches along with its pros, cons and challenges in the selected aspects are reviewed in Sections 3–6, respectively. Section 7 presents the summary and finally conclusion and future research directions are outlined in Section 8.
2 Significance of selected aspects
In this section, we will justify that how the selected aspects are important and play role in solving problem of keeping consistently well-maintained folksonomy in terms of quality. Let us have a look at each aspect.
2.1 Quality tag selection
Folksonomies do not limit its users to use a set of keywords already chosen by a group of experts for interpersonal information retrieval. However, selection of good tags not only enhances an individual’s analysis and view of the tags, it also encourages them to develop better tags in the future (Farhan & Sanderson, Reference Farhan and Sanderson2008). Conformity theory observed in folksonomies indicates that the tags that users observe from other users will influence the tags that they will choose for annotation of a resource (Asch & Solomon, Reference Asch and Solomon1956). The proof of this is that once a tag becomes popular it remains popular (Sen et al., Reference Sen, Lam and Rashid2006). So good tag selection by the users will influence other users tag selection.
A choice of low-quality tags in the user-generated folksonomy have negative effect on the quality of the tag-based retrieved search results and user’s experience (Li et al., Reference Li, Ma, Asano and Yoshikawa2012). Further pointed out by Geldart and Cummins (Reference Geldart and Cummins2010), low-quality tag creates problems, especially when looking at the subjects that have commercial significance. Once we have a collection of worthy tags we can utilize them in various ways, for example, displaying the good tags on tag cloud to enhance search and navigation. Tag suggestions when labelling a resource is improved if we are able to automatically find out quality tags.
Quality of indexing provided by the users is a key to measure the usefulness of the folksonomy (Sturtz, Reference Sturtz2004). For example, Layne (Reference Layne1994) computed depth or thoroughness of folks indexing of tagged images. Moreover, depth indexing increases precision in retrieval (Magnuson, Reference Magnuson2008).
2.2 Tag management facilitation provided by folksonomy applications
Information design of social networking applications influences the quality of the metadata collected. Floeck et al. (Reference Floeck, Putzke, Steinfels, Fischbach and Schoder2011) research focussed on this perspective and they found from experimentation on the social bookmarking website of BobrDobr.ru that if users were allowed to view the most popular tags, the overall variation of tags assigned to websites by all the users decreases.
At the time of resource submission if provided, for example, a set of helpful checklist of questions that could be applied to the object being tagged, or suggest tags, will promote quality tag selection. Type-ahead functionality and editing functions for the tags like changing or modifying it (Guy & Tonkin, Reference Guy and Tonkin2006), renaming tags (e.g. typos), deleting tags, merging tags (e.g. single and plural words) helps to avoid spell mistakes and ambiguity. Mandatory facets, meaning a user must categorize each item with at least a single tag from a facet. This keeps the folksonomy well organized and improves search and retrieval.
The precision of the retrieved search results can also be improved by the facilitation provided by the folksonomy application, when searching a resource, for example, query substitution improves performance. Use of ontology in the query expansion solves the problem, like, a user searching with a high-level concept such as plant retrieves resources, which are not normally tagged with such a high-level concept (Pan et al., Reference Pan, Taylor and Thomas2009). To address the same problem, in Flickr the user may choose types of relations (e.g. synonyms, hypernyms, hyponyms, holonyms or meronyms) for expanding the query. Thus, if a user made a query for ‘shoes’ the query could be expanded with the hyponyms ‘slippers’ and ‘trainers’ to retrieve all the pictures tagged with these subtypes of shoes from the Flickr collection. Accurate search results are one of the proofs of well-maintained quality folksonomy.
2.3 Folksonomy cleaning
Tags assigned to the resources often lack precision and completeness (Liu et al., Reference Liu, Zhang, Yang and Hong Jiang2009), in addition to the fact that some tags do not remain applicable all the time or in other words are expired tags. Tag weeding (the term used by Peters and Weller (Reference Peters and Weller2008)) is required to remove spam, invalid and expired tags, which will keep the folksonomy clean and also would result in precise search results.
2.4 Interoperability across platforms
It is often the case that a user has profiles on more than one folksonomy applications (e.g. Del.icio.us and Flickr) at a time. Interoperability allows utilizing a same tag set uniformly in different platforms. Furthermore, tags from different sources are compared in a way, that there must be an explicit agreement on the understanding of the promising patterns of the tags usage (Veres, Reference Veres2006). This will keep folksonomy consistent and non-ambiguous.
Concluding remarks: the above discussed four aspects are very important and interrelated for keeping quality-maintained folksonomy. The first reasons is that the knowledge of quality tag metrics help users on one hand to choose quality tag(s) and the designers of folksonomy-based systems on the other to design interfaces (e.g. facets, tag suggestions, query expansion) that ensure only good tags selection when entering and searching for a resource. Second, quality tag metrics enable the folksonomy cleansing mechanisms to distinguish between good and bad tags and to remove bad tags. Third, the tag management aspect of a folksonomy-based system enables, for example, the removal of expired tags by utilizing the time stamps associated with the tags. Lastly, the first three aspects ensure a quality folksonomy development and maintenance and the fourth aspect makes it possible to shift the same quality folks vocabulary across different platforms.
In Table 2 we have summarized the evaluation criteria to measure quality of folksonomies and mentioned as to which of our selected aspects cover which criteria. Criteria 1–7 are outlined by Farhan and Sanderson (Reference Farhan and Sanderson2008) for e-government folksonomy and 8–10 are pointed out in Helic et al. (Reference Helic, Krner and Granitzer2011, Reference Helic, Lerman, Rey, Strohmaier and Trattner2012), Al-Khalifa and Davis (Reference Al-Khalifa and Davis2006), Strohmaier et al. (Reference Strohmaier, Helic, Benz, Orner and Kern2012a), Solskinnsbakk et al. (Reference Solskinnsbakk, Atle, Haderlein, Myrseth and Cerrato2012). Here we feel the need that research is required to have updated evaluation criteria and folksonomy management metrics (Folksonomy Management, 2014) against which the folksonomies must be evaluated to measure and ensure worthiness of the folksonomy with passage of time (Qualitatively and Quantitatively).
Folksonomy quality measurement evaluation criteria
(1) quality tag selection, (2) tag management features, (3) folksonomy cleaning, (4) Interoperability.
In the next sections, we are going to analyze the approaches in each aspect, with pros, cons, analysis of techniques and challenges.
3 Quality tag selection
3.1 Tag literacy
One possible approach to have value tags can be that there should be rules and regulation for formation of the tags, as proposed in the article by Mejias (Reference Mejias2005) regarding tag literacy. Suggestions include the following:
∙ Use of plurals rather than singulars: the reason behind this is, the searchers make use of plural keywords more than twice as often as singular keywords (Baxter, Reference Baxter2009).
∙ Use lower case: the reason being is that mostly people write queries in lower case.
∙ Group words with an underscore.
∙ Follow tag conventions started by others.
∙ Add synonyms: for clarity and related option(s).
One might make a case for the establishment of a consortium to release a list of general guidelines (Guy & Tonkin, Reference Guy and Tonkin2006). An attempt in this direction by Spiteri (Reference Spiteri2007a) evaluated tags of three famous folksonomies including Del.icio.us, Technorati and Furl against the National Information Standard Organization (NISO) guidelines. They concluded that the tags corresponded closely to NISO guidelines but were not entirely compliant due to the problem of ambiguous tags(homographs and unqualified abbreviations). Some researchers have suggested after experiments to utilize the clear guidelines for construction of terms established for taxonomies, but all the guidelines are not significant. However, some guidelines for choice and formation of concept terms seem relevant and can be utilized (Spiteri, Reference Spiteri2007b).
Along with educating users, it is also required to outline metrics that can be used to justify whether a tag is a quality tag or not.
3.2 Tag quality measure metrics
The approaches discussed in this section are classified according to the metrics defined for the goodness of a tag.
3.2.1 Folk rating
Folk rating metric covers all the techniques in which users score is involved. The techniques can be seen from three perspectives, which are individual rating based, expert vs. non-expert opinion and inter-subjectivity based.
Analysis of the approach: the problem with individual rating-based techniques is that asking folks to rate tags as a quality measure is time consuming (Damme et al., Reference Damme, Hepp and Coenen2008). In addition, an another observation is that if a tag receives four consecutive thumbs down (indication of dislikeness) ratings (regardless of user), then 90% of the remaining ratings will be thumbs down as well (Sen et al., Reference Sen, Harper, Lapitz and Riedlet2007).
In expert vs. non-expert opinion-based techniques it is assumed that a tag is a quality tag if it is assigned by the more expert users as compared with the less experienced ones. The other name as suggested by Nauerz et al. (Reference Nauerz, Bakalov, Welsch and Birgitta2009) is ‘reputation-based tagging’. Kang and Lerman (Reference Kang and Lerman2011) investigated and concluded that accuracy and detailed folksonomy aspects are well covered by annotations from experts, however, annotations from non-expert or novice users lead to more comprehensive folksonomies than annotations of experts only.
Inter-subjectivity means tags that are understood and used by many members of a group. This is one of the indicators of quality and basis of inter-subjectivity-based techniques. Three metrics which are high-frequency tag, tag agreement and tag frequency inverse resource frequency (TF-IRF) have been proposed by Damme et al. (Reference Damme, Hepp and Coenen2008) and among these three, TF-IRF metric gave promising results. The metrics were tested on data set of Del.icio.us.
Same quality measure, that is, the number of users applying a particular tag is explored by Sen et al. (Reference Sen, Harper, Lapitz and Riedlet2007) using the num-user method. Summary of result suggested that tags being used once have lowest average rating, tags applied 16–31 times have the top average rating (2.53) and the tags applied most often (256 or more times) have an average rating of 2.14. Another inter-subjectivity quality measure explored is num-searches, which give weight and quality to the tag depending upon how many users did search using that tag. Next measure is based on aggregate user selection global-avg that gives position/weight to the tags with respect to their overall average rating across all users. Lastly, the consec-apps selection method, which ranks tags based on the number of initial identical ratings and the similar consec-users, which requires that the ratings come from different users.
The challenge to be considered is the systems may want users to rate tags rather than tag applications. Second, rating given by a user is influenced by the rating already given by other users (Sen et al., Reference Sen, Harper, Lapitz and Riedlet2007).
3.2.2 Validity of tags
The term validity of tags means that social terms provide added value relative to terms from the controlled vocabularies.
Analysis of the approach: one of the comparative studies on the potential role of tags vs. controlled vocabulary by Syn and Spring (Reference Syn and Spring2009) concluded that controlled vocabulary tends to represent the content more precisely and stably. Kiu and Tsui (Reference Kiu and Tsui2008) supported taxonomy-directed folksonomies to achieve the objective of consistency. However, a finding from Trant (Reference Trant2006) based on his work on museum vocabulary has discovered that >70% of the vocabulary used for tagging by the users was not listed in documentation. The reason being is the viewpoint and navigational behaviour of expert varies from that of (general) users.
3.2.3 Information value
Information value of tag means how well a tag is useful for thematic description, retrieval and distinction between resources.
Analysis of the approach: it is quite commonly observed that tags used to describe one resource may not be appropriate for another resource. The quality of a tag varies from resource to resource, therefore, there is a need to measure tag quality with respect to resource to which it is assigned (Krestel & Chen, Reference Krestel and Chen2008). The major techniques in literature to find appropriate tags are based on query log file and graph. In case of query log file-based techniques, appropriate tags are found by exploiting similarities between the tags used by the users in a query and those either in official vocabulary or used by the ordinary library patron in a query. This set of tags identifies associated library catalog records (Pera & Lund, Reference Pera and Lund2009; Daglas et al., Reference Daglas, Kakali and Kakavoulis2012). However, in graph-based techniques the relationship among user, tag and resource is utilized (Krestel & Chen, Reference Krestel and Chen2008).
3.2.4 Preference
Preference means the tags to which users give priority for annotation of a resource. There are many factors that influence the preference of the users. Nandipati (Reference Nandipati2011) suggested that in order to find out the preference, an important issue is to understand the motivation of users. Körner et al. (Reference Körner, Kern and Strohmaier2010) outlined two groups of people involved in tagging as categorizers and describers. Categorizers tag their resources using vocabulary based on their mental model, personal preference and mostly use less number of tags. On the other hand, describers keep easy retrieval in mind while annotating the content. Mostly, they use synonyms. Measures proposed for isolating people of each group are tag/resource ratio, conditional entropy, orphaned tags, overlap factor and tag/title intersection ratio. Authors concluded that all the measures are not equally helpful but tag/resource ratio appears to give good results in capturing human judgement. Strohmaier et al. (Reference Strohmaier, Christian and Kern2009) worked in the same direction, experimentally evaluated eight data sets and found out that tag agreement between users who are motivated by categorizing resources is less as compared with users who are motivated by describing resources.
Recent tags represent user’s present preference or interests. User significance for tags changes with the passage of time. In an environment in which the user gradually changes interests, the tag data close to the current temporal period is usually more significant than that which is temporally far from the current period (Zhang et al., Reference Zhang, Tang and Zhang2012). Here we feel that a study is required to highlight factors (one of them is user’s motivation (Strohmaier et al., Reference Strohmaier, Körner and Kern2012b)) that influence user’s tagging behaviour and then reflection of this on folksonomy quality.
An important aspect that needs consideration is that it is challenging to design interfaces that collect ratings in preference along with other quality dimensions (validity, information value) without puzzling users. Systems such as YouTube and Netix only gather ratings in preference dimension. However, it is necessary to explore interfaces for differentiating preference and other quality metrics, and examine the role they play in different domains (Sen et al., Reference Sen, Vig and Riedl2009). In addition, there are factors that are important in quality of tags, including user interaction level, organization level and interest in content (Nandipati, Reference Nandipati2011).
3.3 Tag generation models
Gupta et al. (Reference Gupta, Li, Yin and Han2010) outlined the key features of each tag model, focussing on the key aspect of how valued tags are selected in each model.
∙ Basic Polya Urn model: proposed by Golder and Huberman (Reference Golder and Huberman2006), captures already assigned tags but does not consider new tags.
∙ Yule Simon model: probability of new word selection is p, and of copied word is 1−p.
∙ Yule Simon model with long-term memory: copy using a distribution over past×time steps where probability decreases as a power law.
∙ Information value-based model-Focus on tags assigned previously, in addition to their information value.
∙ More parameters: background knowledge of a user, number of prior used tags the user has access to and most well-accepted/popular tags.
∙ Language model: model generation of tags and words collectively, using the model like Latent Dirichlet allocation (LDA).
3.4 Characteristics of ‘Good’ tags
In the light of above discussion and the features pointed out by Peters (Reference Peters2009), we can identify Good Tags from seven characteristics. First, frequently used or high-frequency tags because they depict users collective intelligence. Second, more distinctively appropriate metadata for a resource (distinctiveness of a tag). Third, the user should spend minimal effort to go through the tags of a particular resource so as to minimize cognitive load on the user. This is accomplished by having small number of tags per resource. Fourth, the tags which are specific to user’s perspective often do not have meaningful value to other users and hence should be removed from public folksonomy. Fifth, terms from controlled vocabulary such that they do not come strictly from any specific domain jargon. Sixth, spelling and semantic variant tags should be placed together to avoid confusion and get high recall. Seventh, recommended tags should be evenly selected from different categories (multiple facets) to ensure diversity of terms.
After having a brief discussion on quality tag selection metrics and good tags characteristics, in the next section our focus will be on the aspect of tag management features provided by the folksonomy applications.
4 Tag management features provided by the folksonomy applications
The term tag management refers to capability of managing user-generated tags or a folksonomy within a collaborative software so that quality folksonomy could be maintained. Microsoft Sharepoint is an example of applying tag management to a content management system. In addition, Jumper 2.0 (Perry, Reference Perry2009) and Knowledge Plaza (Whatever, Reference Wetzker and Bauckhage2009) are examples of tag management applied to enterprise bookmarking. For blogging wordpress is popular. We will review the approaches of tag management from two perspectives, first, the number of features considered at the time of resource submission, and second, the features which facilitate users in searching of resource(s) and to obtain precise search results.
4.1 Tag features for resource entry
Rather than developing efficient algorithms for making sense out of messy user-generated tags, clever algorithms should be developed and utilized to help users in selection of tags that make sense in the first place. Keeping this in view, one can improve systems which help in tag creation. These can be called preventive measures.
4.1.1 Choice of tags
Utilize predefined tags: rather than allowing users to create their own tags, the users can utilize predefined tags presented as suggested tags. The approach is common either when submitting or searching for a resource.
Analysis of the approach: manually defining tags for each resource is potentially time consuming (Lipczak, Reference Lipczak2008). Second, if the users are freely allowed to apply any tag to any resource, tagging systems will contain large numbers of unneeded, ambiguous and personal tags, which causes difficulty in resource searching (Gemmell & Shepitsen, Reference Gemmell and Shepitsen2008). It reduces cognitive load on the users, as the users need to choose one or more suggested tags instead of thinking and summarizing a source to find out the best representative tags. Tag suggestions support a common vocabulary surrounding all the users (Dattolo et al., Reference Dattolo, Ferrara and Tasso2010).
Major techniques in the literature for tag suggestion are based on considering tag–tag relationship (similar tags, co-occurring tags, alternative tags), tag–resource relationship (tags of similar resources), mapping (with ontology terms, folksonomy terms), user personomy and usage pattern.
Looking into the quality and coverage of the recommended tags is an important challenge. Because this will facilitate the users beyond the annotation process. For instance, techniques that suggest additional tags based on relations like ‘is-a’, ‘part-of’, ‘paradigmatic abstraction’ and ‘tag inheritance’ are particularly very significant in increasing a resources indexing depth (Peters, Reference Peters2009). Asymmetric tag co-occurrence provides a more suitable variety of candidate tags as compared with symmetric co-occurrence. In comparison, Jaccard symmetric co-occurrence gives good results in finding alike tags (Sigurbjrnsson & van Zwol, Reference Sigurbjrnsson and van Zwol2008).
In addition, quality and coverage of the suggested tags are particularly significant for the resources that are not evaluated by computers such as photos, videos and music. Indexing terms provided automatically by search engines work well for textual data but are less useful for describing and organizing resources like video, music and photos (Gemmell et al., Reference Gemmell, Schimoler, Ramezani and Mobasher2009; Harvey et al., Reference Harvey, Baillie, Ruthven and Elsweiler2009).
Tag editing/tag preprocessing: tag editing features allow users to change or modify tags, rename tags (e.g. typos), delete tags and merge tags (e.g. single and plural words).
Analysis of the approach: to handle lexical mistakes, such as misspelling, a tag editing feature is required. Type-ahead functionality helps the users to avoid misspelling. The other promising techniques used include Natural Language Processing (NLP) techniques, thesauri (aspell (GNU Aspell, 2011), Princeton University’s word lexical reference system (WordNet, Reference Whatever2010), Perl Lingua (Rossi, Reference Rossi2006; Franz & Richardson, Reference Franz and Richardson2007). These can be used independently or in combination for correction of misspelled tags, finding word class and stem.
The drawback of type-ahead is it fails to answer many of semantic-level and person-level overlap requirements (Muller, Reference Muller2007). One possible solution by Muller (Reference Muller2007) suggests that type-ahead functionality should provide the users with tags from his or her own personomy in order to perk up self-consistency.
4.1.2 Tags linkage
To keep contents organized, one prominent way is to allow users to organize and categorize tags into facets (Spiteri, Reference Spiteri2011). Keeping contents organized will enhance navigation and search precision. Tag relatedness can be viewed from two prospective, that is, Explicit and Implicit. Explicit relationship can be achieved by mapping tags with external knowledge sources such as ontologies (Laniado et al., Reference Laniado, Eynard and Colombetti2007) and implicit relationship is achieved by finding indirect connection among tags via usage of metrics, for example, subsumption.
4.1.2.1 Explicit connection
Community-driven ontology development: user interactive interface for representing the ontological concept with collaborative tags is one of the research directions to keep the folksonomy organized. Collaborative ontology evolution system allows community members to add, modify or delete existing and new ontology classes according to their own needs. The editing of the classes does not require any special skills and is as simple as adding tags to current tagging systems. Dimensions of workflow include size of the ontology, size of community (contributors, editors), discussion tools (mailing, chat, message boards), synchronization and editing techniques (Subversion (SVN), Concurrent versions system (CVS)) (Noy, Reference Noy2008).
Analysis of the approach: community involvement enables members to actively contribute towards the ontology evolution process and it works, as it is a key factor to accomplish a community common ground (Gendarmi et al., Reference Gendarmi, Lanubile, Informatica and Orabona2006; Ga et al., Reference Ga, Zouaq, Torniai, Jovanovi and Hatala2011). In other words, it characterizes the understanding of a particular community or domain area. The major challenges are, first, the choice of a suitable Ontology Meta-Model, appropriate for a large audience. Second, provision of helper and guidance feature to help the community for development and maintenance of collaborative ontology. Third, handling conflicts, for example, as if two communities have contradictory views on the same subject, or if two communities use analogous terms for different concepts (Hepp et al., Reference Hepp, Bachlechner and Siorpaes2006).
Community-based ontology matching: many ontologies to be matched are analogous to existing ontologies, mostly in the case when they are representing the same application domain. Matching is a precise solution to handle semantic heterogeneity and disambiguation problem (Zhdanova & Shvaiko, Reference Zhdanova and Shvaiko2006). Figure 1 represents the ontology alignment between two simple ontologies. ‘Senior Researcher’ and ‘Research Associate’ ontology on the left is mapped with ‘Researcher ontology’ on right.
Ontology alignment (Zhdanova & Shvaiko, Reference Zhdanova and Shvaiko2006)
Analysis of the approach: involving users in mapping process helps to handle challenges, such as appropriateness of mappings, especially when using them in the new application. The current challenge in the collaborative ontology matching is to find a way to handle the incomplete and contradictory alignments. Another prominent challenge is to handle appropriate annotation support and satisfactory description units that enable and facilitate user participation at large scale (Shvaiko & Euzenat, Reference Shvaiko and Euzenat2008).
Folk-based categorization: folksonomies with predefined organization, such as hierarchical, put limits on the flexibility of tagging because of its restricted and predefined vocabulary. In addition, it also requires an extra effort from users to select tags. These limitations divert direction of research to allow users to enter tags along with their categories.
Analysis of the approach: based on this approach users consensus-supported organized folksonomy is built. In addition to consensus it also gives flexibility to users in choosing tags and their corresponding categories. Yoo et al. (Reference Yoo, Choi, Suh and Kim2013) claimed in their work that by adding this sort of organization retrieval is improved as compared with flat folksonomy. However, developed folksonomy using this approach by Yoo and Suh (Reference Yoo and Suh2010) was limited to ‘has’ relationship.
4.1.2.2 Implicit connection
Explicit relationships are often inapplicable since they require that corresponding concepts must exist. For example, when a tag is mapped to an ontology it is required that corresponding concept must exist and also represents the meaning that was actually meant by the users. Apart from explicit relationships, there exists another linkage called implicit connection. Implicit relationships also play a key role in enhancing navigation, organization, portability and universal applicability (Matthes et al., Reference Matthes, Neubert and Steinhoff2012). Let us have a look at the approaches covering this aspect.
Grouping syntactical variation of tags: several reasons cause syntactical tag variations. That includes typographical mistakes, for example, semanticweb, semnticwev, zemantcweb, using singular or plural of the same word like semanticweb, semanticwebs, using separators, for example, semantic web, semanticweb or a combination of them like semntic-web, smanticweb, semntic-webs, etc.
Analysis of the approach: although all syntactically variant tags have identical sense, resources would be classified under diverse labels. This fact makes more confusion in locating the information, and navigating in the folksonomy. This problem can be rectified by identifying all of them as variations of the interchangeable label ‘semantic web’ and then group them under a single tag. The user could access this tag and would be able to obtain all the information relating to the resources associated with it and its syntactic variations (Echarte et al., Reference Echarte, Astrain, Crdoba and Villadangos2009). Tag clouds, for example, could show a representative of each group of related variations and provide a doorway to all resources annotated with them. Major techniques in the literature are based on pattern matching (Echarte et al., Reference Echarte, Astrain, Crdoba and Villadangos2009), similarity measures (Astrain et al., Reference Astrain, Echarte, Crdoba and Villadangos2009) and linguistic sources like Wikipedia/DBPedia. However, accuracy is still a major challenge, that is, how correctly syntactic variant tags are detected, especially when dealing with short length tags.
Tag community: tag community detection is to find out groups of tags that are either semantically close to each other or used in the same context (Papadopoulos et al., Reference Papadopoulos, Vakali and Kompatsiaris2011).
Analysis of the approach: Quattrone et al. (Reference Quattrone, Capra, De Meo, Ferrara and Ursino2011) proposed in their work that when a user is assigning tag(s) to a resource, similar and related tags already present in the folksonomy should be displayed. For example, in Del.icio.us and Flickr, when a user visits the page containing all the bookmarks/pictures tagged with a certain tag, a list of the tags related to that one are also shown inside a sidebar. The key benefit of displaying the tag community is to enhance help in searching via navigation.
There are many research efforts in the literature to compute similarity, like one based on linguistic resources (stemming/lemmatization/WordNet), or taking into account tag–tag relationships (clustering/co-occurrence), or tag–resource relations (tags associated to same resource are similar and vice versa).
As pointed out by Solskinnsbakk and Gulla (Reference Solskinnsbakk and Gulla2001), the pitfall of linguistic sources is limited flexibility, since they are language dependent, therefore, difficulty in handling multilingual tags (politics, politica) and sometimes proves to be expensive. However, it will help in organizing contents in hierarchy (Laniado et al., Reference Laniado, Eynard and Colombetti2007).
The clustering/co-occurrence needs certain number of resources annotated with the same tag in order to recognize linguistic variations effectively. Second, co-occurrence may identify dual topics like Barcelona and Football and such tags cannot be moulded to be a single tag (Solskinnsbakk & Gulla, Reference Solskinnsbakk and Gulla2001). In comparison, tag–resource relation gives better results for tag disambiguation than co-occurrence measures (Wetzker & Bauckhage, Reference Wei, Peng and Dumoulin2010; Quattrone et al., Reference Quattrone, Capra, De Meo, Ferrara and Ursino2011). Tag clustering techniques based on Hierarchical Agglomerative Clustering or K-mean results in good tag clusters if tags are distributed evenly but are not effective if distribution is not even (e.g. ‘S’ shape) (Xu et al., Reference Xu, Zong, Jin, Pan and Wu2013).
Other approaches: Equivalence includes subsumption, identicalness and mutual exclusion (Matthes et al., Reference Matthes, Neubert and Steinhoff2012). Abstract To Concrete Representation, a hypothesis proposed in Benz and Christian (Reference Benz and Christian2011) is that specialists mostly make use of more specific vocabulary as compared with generalists, who rely mainly on abstract tags. Relevancy Tag Ranking, relevant tags need to be ranked keeping in view different ranking criteria such as, importance, semantic closeness (Liu & Zhang, Reference Liu and Zhang2011).
4.1.3 Other related aspects
Tagging styles and their evaluation: cost-wise analysis and evaluation of different tagging approaches like ‘click2Tag’ and ‘SparTag.us’ have been tested by Hong et al. (Reference Hong, Chi and Budiu2008). Multimedia tagging by taking advantage of human intelligence and computers computation power is proposed by Wang et al. (Reference Wang, Ni, Hua and Chua2012). ‘Automatic Tagging’ or usage of alternatives such as a ‘Tag Signature’ in place of tags is proposed by Solskinnsbakk and Gulla (Reference Solskinnsbakk and Gulla2011). Here we feel there is a need to evaluate different tagging styles keeping both quality and cost of tagging perspective in mind.
To end this subsection, Table 3 compares features supported by five popular folksonomy sites at the time of submission of a resource. The term tag gardening has been used for editing and organizing of tags by Peters and Weller (Reference Peters and Weller2008).
Feature set support provided by five popular folksonomies when submitting a resource
4.2 Precise resource searching via tags
Precision and accuracy in search results can also be achieved by incorporating supporting features for a user when he or she searches for a resource. In this section, we will look at the major approaches to achieve this objective. There are two directions, either to use visual interface for navigation-based search, or to write textual queries.
4.2.1 Visual interfaces
Tag cloud: to facilitate users in navigation and searching, most of the applications provide visual interfaces. Tag cloud is one of the popular types of interface being used in folksonomies.
Analysis of the approach: tag cloud design depends upon the type of the folksonomy application and its requirement. As an example, Diaz and Tory (Reference Diaz and Tory2009) analyzed five diverse image and video search interfaces and gave a conclusion that search interfaces should maximize screen space used for visual representations of the media. In addition, it should present on-demand access to titles, tags and other metadata, and also make available contextual information about earlier viewed items, current keywords and alternate keyword possible options. The other important issues include selection of the tags and design (such as shape, colour and animation scheme and its interpretation) of the tag cloud.
Faceted browsing: with facets, resource collection can be retrieved and filtered in a structured way (Waltl, Reference Waltl2012).
Analysis of the approach: faceted browsing proved to be remarkable in organizing website resources. However, Choi and Street (Reference Choi and Street2009) argued that the user point of view is not reflected in those facets and proposed to incorporate the users perspective in design of the interface.
Personalized view and search: personalized search is based on computing search results, considering user taste, interests and needs.
Analysis of the approach: personalized view called ‘Folkview’ proposed by Dattolo and Pitassi (Reference Dattolo and Pitassi2011) works on creating personalized and adaptive paths for the users, so that association between the tags and resources can be modified. Major techniques to enhance precision and relevance in the search utilize user profiles and usage pattern.
By analyzing the profiles, users’ interests can be outlined. Usage pattern-based techniques take advantage of following set of factors: the total number of tags a user has provided (as some user may have used few tags while other have many tags credited to them), the frequency of use of the tags in both ways, that is, the number of times a tag has been used by the users and how many users have used the tag (Golder & Huberman, Reference Golder and Huberman2006), the types of the tags like Personal, Factual and Subjective (Al-Khalifa & Davis, Reference Al-Khalifa and Davis2007), and the patterns of resources in which the users are more interested (prediction of reoccurring patterns) (Kawase & Herder, Reference Kawase and Herder2011). In the same way, Kwak et al. (Reference Kwak, Shin, Yoon and Moon2009) proposed that based on tag usage across different folksonomies the user with the same interest can be found. User interest pattern in temporal direction has been proposed by Zhang et al. (Reference Zhang, Tang and Zhang2012) meaning finding first and the last time usage of a tag. The more times the tag has not been used the higher is the probability of it not reoccurring and vice versa.
4.2.1.1 Interface design evaluation
Efficiency, effectiveness and user satisfaction criteria for evaluation are considered by Ravendran (Reference Ravendran2012) for evaluating the usability of the interfaces. McCarthy and Conners (Reference McCarthy and Conners2006) evaluated tag clouds on a descriptive basis. However, to design a visual interface that could be used for retrieval of contents continuously, we need to have a better understanding of the users goals, objectives and strategies (Diaz & Tory, Reference Diaz and Tory2009).
4.2.2 Query-based exploratory search
Query reformulation: query reformulation means disambiguation of abbreviation in queries, or to enrich it with meaning. This can be done by substitution and/or expansion. Substitution means dropping the original terms given in the query and adding a new term to replace them. Query expansion means to enrich queries with senses and/or with related classes.
Analysis of the approach: removing words as a component of substitution improves performance. However, an expansion is a much safer form of query amendment as compared with substitution (Wei et al., Reference Xia, Peng, Feng and Fan2008; Dang & Croft, Reference Dang and Croft2010). Expanding queries will reduce ambiguity and improve search precision. Major techniques in the literature for query expansion make use of relationships (e.g. co-occurrence), mapping (with external sources (ontologies/thesauri/anchor text) and senses (WordNet, Tagora Sense Repository) and retrieving top-ranked search results (Pseudo relevance feedback (PRF)).
The work done by Abbasi (Reference Abbasi2011) compared different query expansion methods used in folksonomies and evaluated these techniques on large scale. Query expansion based on co-occurring tags normalized using cosine, Dice and Jaccard coefficient are compared. The Jaccard coefficient does not favour tags that do not co-occur very often. The Dice coefficient gives more promising results in finding co-occurring tags as compared with Jaccard. Furthermore, Abbasi (Reference Abbasi2011) concluded that cosine-based expansion for queries seem suitable for sparse search results, PRF for queries having many appropriate resources (~11–50) and overlap-based global expansion for queries having a lot of relevant resources (approximately >50).
Finding senses by using WordNet is not suitable because it is either too coarse (does not cover senses of a word in all domains) or too fine (many of the tags matches to single sense and not to all the senses available for a particular word) (Chen et al., Reference Chen, Feng and Liu2014).
Expanding abstract terms with more concrete terms is required to precise query and search results. Centrality and entropy measures are good at distinguishing abstract and concrete terms. Moreover, tag co-occurrence graph is more useful input to centrality measure than tag similarity graph to compute abstractness (Benz & Christian, Reference Benz and Christian2011). In an image domain considering similarity of both visual and textual features gives 8% more improvement in detecting abstract tags as compared with if they are used individually (Xia et al., 2013).
4.3 Other related aspects
Rank the search results/ranking the resources: given a keyword query (tag), the ranking strategy computes ranking of resources so that the resources that are most relevant to the keyword query become visible at the top of the ranking (Abel et al., Reference Abel, Kawase and Krause2010). As an example, Flickr offers search results considering relevancy, recentness and interestingness. Rásto et al. (Reference Rásto, Tvaroek and Bielikova2013) stated that faceted browsing navigational issues can be solved by classifying search results into hierarchical clusters. Hierarchical clusters make use of semantic properties of search results to get clusters and there is no need of domain experts, thus making them opt means for classifying search results.
Concluding remarks: folksonomies differ in results of precision and recall. The most likely reason behind these differences is their specific characteristics. Methods of collection building, tagging, socialization and ranking may differ from one folksonomy to the next (Morrison, Reference Morrison2008). Folksonomy-related difficulties outlined in Table 1 and discussed by Peters (Reference Peters2009) (such as, loss of indexing context, varying level of indexing and use of mixed languages) causes unsatisfactory search results, including inconsistencies in indexing depth.
Indexing depth affects both precision and recall. Indexing depth increases precision because more specific terms are used. Summarization increases recall because more broader terms are used (Taylor, Reference Taylor2004). According to Magnuson (Reference Magnuson2008) folksonomies may act as bridge between summarization and depth indexing by using folks analysis, paradigms and perspectives to index resources as shown in Figure 2. More suitably, the folksonomies make hierarchical power of information management more reasonable whose difficult classificatory taxonomies are beyond users vocabulary and understanding. However, in our opinion, quality of the folksonomy determines whether folksonomy is inclined more towards summarization or towards depth indexing. Three measures that are precision, recall and f measure are utilized by Vanderlei and Durão (Reference Vanderlei and Durão2007) to test the search accuracy in folksonomies. They propose to combine keyword, facet-based and folksonomy search techniques to get the best retrieved results.
Folksonomies: a bridge between summarization and depth indexing
To test the navigation effectiveness of the folksonomy, Helic et al. (Reference Helic, Krner and Granitzer2011) worked on pragmatic (task oriented) evaluation. Broad and narrow folksonomies are evaluated for navigation criteria by Helic et al. (Reference Helic, Lerman, Rey, Strohmaier and Trattner2012). Broad folksonomies result in better navigational utility since broad folksonomies give an opportunity of greater overlap among tags and also provide good alternatives for users to toggle between different portions of the network. Narrow folksonomies proved to be unable to provide such support. To end with, in Table 4 we have compared the searching features support provided by five popular folksonomies.
Searching features supported by five popular folksonomies
5 Folksonomy cleaning
Some tags expire with the passage of time (expired/invalid tags) while some are irrelevant to the resource to which they are assigned (false tag–resource relationship) or there is presence of some spam users. Folksonomy cleaning aspect deals with these cases to ensure folksonomy having resources tagged with correct and relevant tags, not with the noisy ones and also free from spam users. In this section, we will cover major research efforts to achieve the said objectives.
5.1 Removing nonsense tags/expired tags
In portals that change dynamically, the tags do not remain appropriate all the time. This leads to the concept of tag expiry.
Analysis of the approach: tag expiry helps the community to focus on what is currently really relevant. Moreover, tag expiry allows us to neglect ‘invalid’ tags from being considered when doing content adaptation or recommendation. Major techniques include manual or filter-based removal of expired tags. Manual technique allows users to assign a ‘lifetime’ to the tags (Nauerz et al., Reference Nauerz, Bakalov, Welsch and Birgitta2009). In filter-based technique frequency threshold values have been used to filter out infrequent/invalid tags (Kiu & Tsui, Reference Kiu and Tsui2008). However, identification of appropriate threshold values is important to judge the effectiveness of the filters.
5.2 Detection and refinement of noisy tags
The name folksonomy-based item representation is given to the practice of finding related tags for an item (resource) and utilizing these allied tags for illustration of each item. It is often the case that a tag is correct and is in frequent use but not relevant to the resource.
Analysis of the approach: noisy tags/spam posts or irrelevant tags assigned to a resource degrade performance because they give incorrect search results or low recall. Major techniques in the literature for finding relevancy (correct resource–tag relationship) are based on considering tag–tag relationship (similarity, co-occurrence). However, accuracy of tag similarity calculated from the corpus that is to be refined seems to be effected by noisy tags (Wang et al., Reference Wang, Hua and Li2009). Therefore, semantic and context-based approaches are more encouraged to use. The major challenges are, first, the decision of removal must be considered from view point of language dependency or society dependency. Second, refinement should not eliminate important tags.
5.3 Detection and removal of spam users
Spam users are major source of noisy tags. Spammers usually use different techniques for polluting the folksonomy. Examples include, by creating many accounts, publishing a specific post many times, using semantically diverse tags to annotate a resource and teaming up with other spammers to give good rating to each other (Yazdani et al., Reference Yazdani, Ivanow, Analoui, Berangi and Ebrahimi2012).
Analysis of the approach: pollution in the folksonomy can be reduced if there is restricted access (Preventing Spammers) (Maurer et al., Reference Maurer, Mcmillen, Abraham and Blum2008). Detection-based techniques do not put restrictions on users access, however, for detection, a model of feature set based on information about the users, activities/posts and interaction is developed to distinguish between legitimate and spam user. Furthermore, features (on basis of which model is developed) can also be defined considering different levels, including user level (spam user), post level (spam post/noisy tag) and resource level (spam resource). Most of the techniques in the literature for detection of the spam users are tested by defining features at post level and user level.
Cui et al. (Reference Cui, Han, Yao, Zhang and Zhou2011) pointed out that most of the research done in this area usually utilizes partial information about users, resources and tags. However, research is needed in a specific direction to make use of all the features related to the users, resources and tags in the folksonomies. This would result in more precise expert findings and is more resilient to spammers. Keeping this in mind, Poorgholami et al. (Reference Poorgholami, Jalali, Rehan and Asgari2013) outlined set of features by considering the tags, resources, users and relationship among them. The authors claimed that these features are used by many machine learning algorithms to detect spammers with 99% of accuracy.
6 Interoperability of tags
In this section, we will analyze the issue of interoperability of the tags from two points of view that are to use the same tag set and an agreement on meaning/usage pattern of the tags across different platforms.
6.1 Same tag set utilization across platforms
If we look from the user’s prospective then it is a more effective and friendly approach to use same set of tags on different platforms. TagCare (Golov et al., Reference Golov, Weller and Peters2008) allows users to bring their personal tags from Del.icio.us, Flickr and Bibsonomy platform into one place. It helps the users to apply the same tags uniformly in different platforms. In the same direction, Bibsonomy, a popular social bookmarking service allows to import Del.icio.us., Firefox bookmarks and local BibTex files (Herwig, Reference Herwig2008).
Analysis of the approach: in Tagcare the statistical features provide a summary that gives an idea regarding the usage of a certain tag. This can be represented as tag cloud or as a ranking of the most often or least frequently used tags. This can help to find out tags which have been used frequently or very infrequently which may necessitate that they be grouped with others.
6.2 Semantic interoperability using metadata vocabularies
Metadata vocabularies like Dublin Core (DC) and folksonomy ontologies play a fundamental role for metadata interoperability within any particular domain and across domains.
Analysis of the approach: one of the studies conducted in an attempt to assign properties of DC to the folksonomy tags reveals that percentage of tags unassigned to DC elements is higher, approximately ranging between 35 and 45%. The reason behind this fact is that their meaning cannot be identified. Some new properties were identified which include action, rate, depth and utility. These fresh properties can be assigned to most of the tags (~26.5%). However, still 13% of the tags remained unassigned (Catarino & Baptista, Reference Catarino and Baptista2010).
Saab (Reference Saab2011) user-generated tags in one application are exploited by another within a similar domain. Tags contained in RSS feed get accumulated during the interaction of the users with iCITY that provides services of cultural recommendations. These tags are mapped with the art-related concepts present in the CHIP, which is a personalized museum application. For mapping the relationships, skos:exactMatch, skos:broadMatch, skos:narrowMatch and skos:relatedMatch of Simple Knowledge Organization System (SKOS) have been utilized. However, this mapping is lexical mapping and still faces the problems of messy and ambiguous tags.
Knerr (Reference Knerr2001) developed ontology of the folksonomy using semantic web techniques. For validation purpose, tagging data from Del.icio.us is obtained and incorporated in the developed ontology. Automatic generation of the ontology for the folksonomy by software agents makes this process easy and efficient.
In conclusion, researchers have made various efforts with partial and inadequate success for combining ontologies and folksonomies to achieve the objective of semantic interoperability. However, the challenges faced in this include (1) structured nature of the ontologies and unstructured nature of the folksonomies, (2) lexicality, syntax and semantics, (3) social and cultural dimensions of the tags, and (4) appearance and reification (Saab, Reference Saab2011).
7 Summary
Brief summary of the major approaches and corresponding techniques in tag management facilitation provided by the folksonomy applications, quality tag selection, folksonomy cleaning and tags interoperability aspects are presented in Tables 5–8, respectively. We have also itemized the prominent feature set coverage of approaches falling in each of the aspect. For example, if we look at Table 6 the approaches cover features of good and quality tags selection that improves precision in search and navigation and also provides coverage to non-conventional tags. In the same way, feature set support provided by the approaches in rest of the aspects are outlined.
Feature set support, major approaches and corresponding techniques in tag management aspect
Feature set support, major approaches and corresponding techniques in tag selection aspect
Feature set support, major approaches and corresponding techniques in folksonomy cleaning aspect
Feature set support, major approaches and corresponding techniques in tags interoperability aspect
8 Conclusion and future directions
To the best of our knowledge based on an extensive survey, we have outlined the major research efforts done for quality folksonomy maintenance and have systematized it in four aspects. The important questions covered in the analysis are as follows.
Aspects: the core aspects that will cover features required for keeping quality of the folksonomy protected.
Significance of the aspects: the way in which, defined aspects will cover the features and ensure a well-maintained quality folksonomy.
Approaches in each defined aspect: major approaches to cover the defined features in each aspect along with its pros, cons and challenges.
Efforts have been made in setting metrics for good and worthy tags, techniques for entering good tags in folksonomy-based systems, organizing folksonomy, disambiguation, removal of low-quality, irrelevant and spam tags along with spam users. These all efforts have significant effect on folksonomy quality, which is reflected by achieving precision in search. However, we still need to focus on following aspects which need researchers attention. These are as follows.
Updated evaluation criteria: it is necessary to have updated evaluation criteria (benchmarks) to test and ensure a well-maintained folksonomy in terms of quality. In addition, evaluation criteria will also be used for classification of algorithms/techniques/methods and estimation of their effectiveness on set-benchmarks.
Folksonomy management metrics: defining metrics for folksonomy management like quantitative metrics, qualitative metrics, time metrics, utilization metrics and financial metrics.
Factors influencing tagging behaviour: study is required to find out what factors (like social, environmental) influence tagging behaviour of the users and how much passage of time plays its role in changing their tagging behaviour. For this objective development of quantitative and qualitative measures are also required to test the influence of each factor. This study will also give answer to the question that how the change of tagging behaviour influences quality of the folksonomy.
Approach for tagging: evaluation of effectiveness is required both for cost and quality perspective with regard to every tagging style. This may lead to an option of having folksonomy type-dependent tagging style.
Acknowledgement
This research is undertaken by the first author for partial fulfilment of PhD degree requirements with support of Higher Education Commission of Pakistan (HEC).
