AN EMPIRICAL ANALYSIS OF FEATURE ENGINEERING TECHNIQUES TO REDUCE DIMENSIONALITY FOR NON-BINARY CLASSIFICATION PROBLEMS - A CASE STUDY WITH FOETAL HEALTH DATASET

Abstract

In today’s world, when AI and ML are deeply involved in our day-to-day lives, merely designing a machine learning model is insufficient. The complexity involved in training the model is vital in determining whether such systems would be deployed in real environments. Today, ML engineers strive to accomplish this, as the models that work well in academic research fail to work well in production. The ML code is a small segment of the ML infrastructure. While in Academia, the focus is on code and hyperparameters, the Industrial Product Team’s focus is on data.   Data engineering and feature engineering, often ignored during model creation and deployment, are two techniques to bridge this gap. To emphasize its importance, we have considered a non-binary classification problem - the Foetal Health Classification problem. We have applied different feature engineering techniques to reduce the number of significant features required for Model Training and have determined the best possible FE technique. From a set of 21 independent features, we could lower the feature count to nine and retain the accuracy score compared to training using the complete feature set. This paper showcases the performance of different prediction models on the dataset, selecting the best prediction model and applying feature engineering techniques for dimensionality reduction. Keeping the threshold at 0.025, we could achieve 96% accuracy, 92.9% precision score, 94.5% recall value, 93.7% F-score, and a dimensionality reduction of 29%. Maintaining a threshold of 0.013, a 95.1% accuracy, 91.3% precision value, 94.5% recall, and 92.8% F-score, and a dimensionality reduction of 57% could be achieved. The above indicates that equivalent results can be achieved with a subset of the Feature set, which can be further instrumental in reducing the model training and convergence time.