Impact of data normalization on the accuracy of supervised learning models

Original Article

Abstract

Feature normalization is a key step in supervised classification, especially when data are presented on heterogeneous scales. This study aims to evaluate the impact of two normalization strategies (MinMax and Z-Score) on the performance of three models: Logistic Regression, SVC/SVM, and Decision Tree, applied to four datasets: Adult Income, Heart Disease, Student Performance Math, and Student Performance Portuguese, obtained from the Machine Learning Repository. As a methodology, the models were trained using stratified cross-validation (k=5) and compared in terms of accuracy, precision, recall, F1-score, and ROC-AUC. The results showed that normalization with Z-Score had a significant effect on the Adult Income dataset, improving the performance of Logistic Regression (F1-score: 0.426 to 0.666; ROC-AUC: 0.641 to 0.904). In contrast, the Heart Disease dataset performed well even without normalization, but SVC/SVM with Z-Score improved its metrics with normalization (F1-score: 0.741 to 0.881; ROC-AUC: 0.785 to 0.922). However, these differences did not reach statistical significance according to the Wilcoxon test (p≈0.0625), although they do constitute moderate evidence. In the Student Performance datasets, the effects of normalization were minimal and statistically insignificant, which can be explained by the fact that the variables were already on comparable scales. Finally, it is confirmed that normalization does not affect all algorithms equally: its impact is more evident in socioeconomic and clinical contexts, where variables tend to use very different scales. This evidence provides practical elements to guide data preprocessing in areas such as health, education, and industry.