2.0 RELATED WORK

Data mining is a multidisciplinary effort to extract nuggets of knowledge from data. The proliferation of large datasets within many domains poses unprecedented challenges to data mining as defined by Han and Kamber^{(Reference Han and Kamber14)}. Not only are datasets getting larger, but new types of data are also becoming prevalent, such as data streams on the web, microarrays in genomics and proteomics, and networks in social computing and systems biology. Researchers are realizing that in order to achieve successful data mining, feature selection is an indispensable component, as proposed by Guyon and Elisseeff^{(Reference Guyon and Elisseeff13)} and Liu and Motoda^{(Reference Liu and Motoda17)}.

Feature selection is normally done by searching the space of attribute subsets and evaluating each one. This is achieved by combining an attribute subset evaluator with a search method. In the present investigation, an evaluation of six filter feature subset methods with a rank search or greedy search method was performed to find out the best feature sets, and they are listed below for reference: (1) correlation-based attribute evaluation (CB); (2) chi-square attribute evaluation (CH); (3) Gain-Ratio attribute evaluation (GR); (4) Information Gain attribute evaluation (IG); (5) ReliefF attribute evaluation (RF); and (6) Symmetrical Uncertainty attribute evaluation (SU). That these entire filter techniques mentioned above could assess the relevance of features has been studied by Saeys et al^{(Reference Saeys, Inza and LarraÑaga31)} on the basis of the intrinsic properties of the data. Feature selection often increases classifier efficiency by reducing the size of the effective features.

Correlation feature subset selection evaluates the worth of a subset of attributes by considering the individual predictive ability of each feature along with the degree of redundancy between them. Correlation coefficients are used to estimate correlation between subset of attributes and class, as well as inter-correlations between the features. Relevance of a group of features grows with the correlation between features and classes, and a decrease with growing inter-correlation as has been studied by Asha et al^{(Reference Asha, Manjunath and Jayaram5)}. The study by Antonio et al^{(Reference Antonio, Benitez and Castro3)} established that, in order to predict the concept or class value of its instances, a dataset with the selected features alone must be consistent. The information gain measure is biased towards tests with many outcomes. That is, it prefers to select attributes having a large number of possible values over attributes with fewer values even though the latter is more informative, as defined by Asha et al.

Information Gain (IG) is a commonly used measure in the fields of information theory and machine learning. IG measures the number of bits of information gained about the class prediction by knowing the value of a given feature when predicting the class, according to the method proposed by Altidor et al^{(Reference Altidor, Khoshgoftaar, Van Hulse and Napolitano2)}. Rule-based algorithms provide ways to generate compact, easy-to-interpret and accurate rules by concentrating on a specific class. One way of generating classification rules is to use decision trees. The disadvantage of using a decision tree is its complex and incomprehensible nature, which has been noted in the literature^{(Reference Berry25)}. Principal component analysis (PCA) is a standard technique used to handle linear dependence among variables. A PCA of a set of m variables generates m new variables (the principal components), PC1. . .PCm. Each component is obtained by linear combination of the original variables and has been studied by Grimaldi et al^{(Reference Grimaldi, Cunningham and Kokaram18)}.

ReliefF is an easy-to-use, fast and accurate algorithm even with dependent features and noisy data as defined by Wang et al^{(Reference Wang, Khoshgoftaar and Napolitano29)}. Symmetric uncertainty is used to measure the degree of association between discrete features. It is derived from entropy that has been proposed by Chen et al^{(Reference Chen, Li, Cheng, Guo, Lipmaa, Yung and Lin7)}. Symmetrical uncertainty is calculated by the above equation. H(X) and H(Y) represent the entropy of features X and Y. The value of symmetrical uncertainty ranges between 0 and 1. The value of 1, indicating that one variable (either X or Y) completely predicts the other variable, has been studied by Ienco et al^{(Reference Ienco, Pensa and Meo15)}.

Feature selection is a process of selecting a subset of original features according to certain criteria and is an important and frequently used technique in data mining for dimension reduction. It reduces the number of features; removes irrelevant, redundant or noisy features; and brings about palpable effects for applications, thus speeding up a data mining algorithm, improving learning accuracy and leading to better model comprehensibility. Various studies show that some features can be removed without performance deterioration as shown by Ng^{(Reference Ng23)} and Donoho^{(Reference Donoho9)}.

Feature selection can be found in many areas of data mining such as classification, clustering, association rules and regression. For example, feature selection is called subset or variable selection in statistics^{(Reference Miller19)}. Guyon and Elisseeff^{(Reference Guyon and Elisseeff13)} have proposed the reducing of the number of dimensions by selecting variables or features which is effective in dealing with high-dimensional data. Variable and feature selection have become the focus of much research in areas of application for which datasets with many number of variables are available. The objective of variable selection is threefold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data. Feature selection has been an active research area in the pattern recognition, statistics and data mining communities.

A decision based on selected features about whether a dangerous surgery is necessary has been studied by Kim et al^{(Reference Kim, Street and Menczer30)}. Delimata and Suraj^{(Reference Delimata and Suraj28)} have dealt with feature selection, which is usually meant as a process of finding a subset of features from the original set of features forming patterns in a given dataset, optimised according to the defined goal and criterion of feature selection. Many classification problems involve features whose specificity demands some form of feature space transformation (pre-processing) coupled with post-processing consensus analysis in order to accomplish a successful discrimination between different classes. Pizzi and Pedrycz^{(Reference Pizzi and Pedrycz24)} have presented a new methodology that systematically addresses these design classification issues in their study. At the pre-processing phase, they offer a new approach of stochastic feature selection. This type of feature selection collates quadratically transformed feature subsets for presentation to a collection of respective classifiers. Feature selection has been an active and fruitful field of research area in the pattern recognition, machine learning, statistics and data mining communities and has been studied by Mitra et al^{(Reference Mitra, Murthy and Pal20)} and Miller^{(Reference Miller19)}.

Subset generation is a search process that produces candidate feature subsets for evaluation based on a certain search strategy. Each candidate subset is evaluated and compared with the previous best one according to a certain evaluation. If the new subset turns to be better, it replaces the best one. This process is repeated until a given stopping condition is satisfied. Ranking of features determines the importance of any individual feature, neglecting their possible interactions. Ranking methods are based on statistics, information theory or on some functions of the classifier's outputs, as studied by Duch et al^{(Reference Duch, Winiarski, Biesiada and Kachel10)}.

Decision tree induction as defined by Han and Kamber^{(Reference Han and Kamber14)} is a very popular and practical approach for pattern classification. Narasimha and Susheela^{(Reference Narasimha and Devi21)} have dealt with the Naïve Bayes (NB) classifier, a simple probabilistic classifier based on the Bayes theorem where every feature is assumed to be class-conditionally independent. Classification of instances gets difficult when the dataset contains a large number of features and classes because it takes enormous numbers of observations to estimate the probabilities. A Support Vector Machine (SVM) is a hyperplane that separates two different sets of samples with maximum distance of hyperplane to the nearest samples from both sets as by Han and Kamber^{(Reference Han and Kamber14)}. The KNN classifier is a non-parametric lazy learning algorithm. A data sample in KNN is classified on the basis of a selected number of k nearest neighbours as described by Han and Kamber.

The automatic and smart analysis of the complex structure of high-volume data in the aviation industry has been studied by Gürbüz et al^{(Reference Gürbüz, Özbakir and Yapici11)}. A predictive model to explain the development of a dispatch reliability prediction method for passenger aircraft was proposed by Bineid and Fielding^{(Reference Bineid and Fielding6)}. Nazeri and Zhang^{(Reference Nazeri and Zhang22)} have described the application of data mining to analysing severe weather impacts on National Airspace System (NAS). A model that allows investigation of non-linear effects of aviation safety factors and flexible assessment of aviation risk has been described in the literature^{(Reference Shyur26)}. The main goal of this work is to develop a model to provide relative risk probability inference and trend analysis among different kind of human errors which may cause any major aviation events. A data mining analysis method that can be applied to evaluate that monitor red-light-signal controlled intersections to improve traffic safety by reducing fatalities has been demonstrated by Solomon et al^{(Reference Solomon, Nguyen, Liebowitz and Agresti27)}.

Considering the high level of interaction between a service provider and its customers, the significance of a marketing approach cannot be overstated^{(Reference Appiah-Adu, Fyall and Singh4)}. Over the past few decades, considerable attention has been focused on cultivating a market-driven approach, which fosters the effective implementation of the marketing concept as an important source of competitive advantage in the tourism industry. Nazeri and Zhang^{(Reference Nazeri and Zhang22)} have defined data mining methods that have been successfully introduced in many fields. It is still a research topic, but of tremendous interest to the industry in order to solve its real-world problems. Statistics, artificial intelligence techniques, decision tree approaches, genetic algorithms and visualization techniques are the major data mining techniques used to solve these problems. Risk and safety have always been important considerations in aviation. With the rapid growth in air travel, flight delays, cancellations and incidents or accidents have also dramatically increased in recent years.

Aviation accidents may result in human injury or even death. An airline company collects several case reports including structural and textual data. Institutions make important investments in order to collect such information from several sources. Nowadays, analysis of such data cannot be done automatically, and analysts have difficulties in dealing with the growing data efficiently and on time. In conclusion, in the automatic and smart analysis of the complex structure of high-volume data in aviation industry tools with capable instruments are needed.

Data mining is a potentially useful instruments that was not known before, and using this instrument to reveal information hidden in the data was proposed by Jiawei and Kamber^{(Reference Jiawei and Kamber16)}. As a result, there is a large amount of knowledge and data accumulation in the aviation industry. These data could be stored in the form of pilot reports, maintenance reports, incident reports, component reports or delay reports. Also in the aviation industry, data mining applications have been used.

Bineid and Fielding^{(Reference Bineid and Fielding6)} have used data mining techniques to explain the development of a dispatch reliability prediction method for passenger aircraft. The U.S. Mine Safety and Health Administration (MSHA) developed a mine accident database from Part 50 of the federal mine safety regulations. This database has been used to track the numbers, rates and severity of mine accidents in the United States. Epidemiologists and mine safety researchers still use it to perform many analyses and help guide research, and best practices have been studied by Dessureault et al^{(Reference Dessureault, Sinuhaji and Coleman8)}.

Dessureault et al have explored the background of the Part 50 database, given a general background of data warehousing and data mining, and presented some of the interesting analyses that resulted from a modernized Part 50 data warehouse using data mining. A subset of data gathered from the Flight Safety Management Information Systems (FSMIS) developed by the office of the Taiwan Civil Aeronautics Administration takes aviation risk factors into account. Using the models presented, risk has been assessed as the probability of occurrence of a specific type of human error related to an aviation accident. The potential human-error-related risk could be identified and monitored in a timely fashion. The results can provide better information to the civil aviation communities to manage aviation safety risk so corrective action can be taken to reduce the occurrence of aviation accidents. It also allows identification of the fundamental causes of human-error-related accidents through the analysis of operational safety data. Finally, the results of the case study demonstrate that the proposed model is a more promising regression model with the potential of becoming very useful in practice as described by Shyur^{(Reference Shyur26)}.

This work represented several data modelling techniques such as decision trees, neural networks, market-basket analysis and K-means models. Decision trees build rule sets that can affect future decision making. Neural networks try to predict future outcomes by looking at the effects of historical inputs. Market-basket analysis shows the strength of the relationships between variables. K-means models weigh the impact of homogenous clusters on target variables. All of these models are demonstrated using real data gathered by the U.S. Department of Transportation from fatal accidents at red-light-signal controlled intersections in Maryland and Washington, DC, from 2000–2003 by Solomon et al^{(Reference Solomon, Nguyen, Liebowitz and Agresti27)}.