Machine Learning-Optimized Bandwidth Selection for Kernel Density Estimation-Based Shewhart Control Charts in Educational Monitoring

Abstract

Early identification of students at risk of academic failure is a critical challenge in educational institutions. Kernel Density Estimation (KDE)-
based Shewhart control charts offer a flexible nonparametric approach for monitoring student performance without relying on normality
assumptions. However, the effectiveness of KDE-based charts depends critically on the choice of bandwidth parameter h. Classical bandwidth
selectors optimize for density estimation accuracy rather than control chart performance, leading to suboptimal detection of at-risk students.
This paper proposes a machine learning approach to bandwidth selection that directly optimizes control chart objectives. Using features
extracted from historical student score data, a random forest regressor is trained to predict the optimal bandwidth that minimizes a composite
loss function balancing in-control average run length (ARL₀) accuracy and out-of-control detection speed. Extensive Monte Carlo simulations
are conducted using Gamma-distributed data (shape = 5, rate = 0.15, scaled to mean 50, standard deviation 10) that realistically models student
examination scores. Six sample sizes (n = 40, 50, 80, 100, 200, 500) are investigated, representing classroom to institutional monitoring levels.
The ML-optimized bandwidth is compared with six classical selectors (LSCV, BCV, DPI, STE, SJ, CONT) using comprehensive performance
metrics: bandwidth prediction accuracy (RMSE), density estimation accuracy (MISE), in-control ARL (ARL₀), out-of-control ARL (ARL₁) for
six shift magnitudes (δ = 0.25, 0.5, 0.75, 1.0, 1.5, 2.0), standard deviation of run length (SDRL), median run length (MDRL), conditional
expected delay (CED), expected false discovery rate (EFDR), ROC curve analysis, precision-recall metrics, and extra quadratic loss (EQL).
Results demonstrate that the ML-optimized bandwidth consistently outperforms all classical methods across every metric and sample size. The
ML method achieves 10–15% lower RMSE, 2–3% lower MISE, maintains ARL₀ within 0.5% of nominal 370, and detects small shifts (δ =
0.5) 17–18% faster than the best classical method. Conditional expected delay is reduced by 17–20%, while EFDR is reduced from 24–42%
to below 4%. ROC analysis yields AUC of 0.96 versus 0.82, and F1-scores improve from 0.64 to 0.90. Feature importance analysis reveals
that sample size (35.2%), skewness (24.7%), and kurtosis (18.3%) are the most influential predictors. The proposed ML-optimized approach
provides educators with a powerful, data-driven tool for early identification of at-risk students, with practical significance of 1.5–2.3 weeks
earlier detection in a 15-week semester.