 |
advertisement |
|
|
|
|
|
|
|
Industrial Automatic Control Systems and Controllers Annotation << Back
|
Comparative Analysis of Data Quality Assessment
and Preprocessing Methods in Machine Learning Tasks |
Lygarev Ya.S., Lygarev M.S.
The article addresses machine learning algorithms improving the accuracy problem through comprehensive data preprocessing. An
approach is proposed that includes a formal statement of the preprocessing task as an operators’ composition, including imputation, outlier
handling, scaling, and noise filtering. Defect prioritization is carried out based on assessing their impact on model trainability. For missing
values, an adaptive approach is implemented. Noise suppression is performed using moving average and Gaussian filtering. Experimental
validation was conducted using data from the Federal Treasury of the Russian Federation, which contains regions’ financial indicators.
Three algorithms were evaluated: random forest, gradient boosting, and support vector regression. It is shown that the proposed preprocessing improves accuracy as measured by the R² metric. It is established that ensemble methods are more effective in the context of defect
elimination due to their aggregation mechanisms, whereas support vector regression remains more sensitive to residual data heterogeneity.
The obtained results confi rm comprehensive prioritized data preprocessing effectiveness and can be applied in practical data analysis tasks.
Keywords: data preprocessing, machine learning, data quality, random forest, gradient boosting, support vector regression, missing
value imputation, outliers, feature scaling, noise, Federal Treasury dataset.
DOI: 10.25791/asu.7.2026.1670
Pp. 12-20. |
|
|
|
Last news:
Выставки по автоматизации и электронике «ПТА-Урал 2018» и «Электроника-Урал 2018» состоятся в Екатеринбурге Открыта электронная регистрация на выставку Дефектоскопия / NDT St. Petersburg Открыта регистрация на 9-ю Международную научно-практическую конференцию «Строительство и ремонт скважин — 2018» ExpoElectronica и ElectronTechExpo 2018: рост площади экспозиции на 19% и новые формы контент-программы Тематика и состав экспозиции РЭП на выставке "ChipEXPO - 2018" |