Comparative Analysis of Feature Selection and Dimensionality Reduction Techniques for High-Dimensional Datasets

Abstract

The existence of high-dimensional data is becoming widespread in fields including healthcare, bioinformatics, text analytics, and factory diagnostics, and has created difficulties in computational complexity, data redundancy, and model overfitting. Learning efficiency and predictive performance hence require effective techniques of feature selection and dimensionality reduction in order to enhance the efficiency of learning. The paper includes a comparative analysis of four popular methods, such as Chi-Square feature selection, Recursive Feature Elimination (RFE), Principal Component Analysis (PCA), and Linear Discriminant Analysis (LDA), on several high dimensions of data (gene expression, text and medical imaging data) and assessed their effectiveness. There are experimental findings that show that supervised techniques are always better as compared to unsupervised and filter techniques. The maximum classification accuracy of LDA was 90.5, precision was 89.7, recall was 89.2 and F1-score was 89.5 which minimized the feature space by 99.98. There was also good performance of RFE with an average accuracy of 88.9 but at a greater cost of computation. On the contrary, Chi-Square selection was the quickest to execute (in terms of time) (about 1.0 s) but less accurate (84.6%). PCA efficiently and accurately (balance between efficiency and performance) retained 95 percent variance. Altogether, the results show that the effectiveness of dimensionality reduction methods strongly depends on the data characteristics and the goal of application that can provide a reasonable guide to the choice of the adequate methods when working with big data set.