Comparison of algorithm performance, Random Forest Regression, SVR, and Gradient Boosting in predicting academic grades based on student lifestyle
Main Article Content
Abstract
This study examines the effectiveness of three machine learning algorithms—Random Forest Regression, Support Vector Regression, and Gradient Boosting—in predicting students’ academic grades based on lifestyle-related factors including study hours, sleep duration, social interaction, physical activity, and stress levels. Employing a quantitative experimental approach, model performance was evaluated using R², MSE, RMSE, and MAE, while SHAP analysis was applied to interpret feature importance. The results show that all models achieved reasonable predictive accuracy, with Gradient Boosting consistently outperforming the others across all metrics. Study duration was identified as the most influential predictor, whereas stress level and gender had minimal impact. These findings emphasize the importance of non-academic lifestyle factors in predicting academic achievement and provide insights for the development of data-driven, personalized decision support systems in education.
Downloads
Download data is not yet available.
Article Details
How to Cite
sihombing, D. (2025). Comparison of algorithm performance, Random Forest Regression, SVR, and Gradient Boosting in predicting academic grades based on student lifestyle. Journal of Intelligent Decision Support System (IDSS), 8(3), 188-197. https://doi.org/10.35335/idss.v8i3.314
References
Alj, Z., & Bouayad, A. (2024). Multidimensional determinants of academic performance: Insights from undergraduate students in Moroccan universities. JOTSE: Journal of Technology and Science Education, 14(2), 607–621.
Avdeef, A. (2021). Do you know your r2? ADMET and DMPK, 9(1), 69–74.
Bakır, R., Orak, C., & Yüksel, A. (2024). Optimizing hydrogen evolution prediction: A unified approach using random forests, lightGBM, and Bagging Regressor ensemble model. International Journal of Hydrogen Energy, 67, 101–110.
Becker, T., Rousseau, A.-J., Geubbelmans, M., Burzykowski, T., & Valkenborg, D. (2023). Decision trees and random forests. American Journal of Orthodontics and Dentofacial Orthopedics, 164(6), 894–897.
Doz, D., Cotič, M., & Felda, D. (2023). Random forest regression in predicting students’ achievements and fuzzy grades. Mathematics, 11(19), 4129.
Ekanayake, I. U., Meddage, D. P. P., & Rathnayake, U. (2022). A novel approach to explain the black-box nature of machine learning in compressive strength predictions of concrete using Shapley additive explanations (SHAP). Case Studies in Construction Materials, 16, e01059.
Gaftandzhieva, S., Talukder, A., Gohain, N., Hussain, S., Theodorou, P., Salal, Y. K., & Doneva, R. (2022). Exploring Online Activities to Predict the Final Grade of Student. Mathematics, 10(20), 1–20. https://doi.org/10.3390/math10203758
Han, C., Farruggia, S. P., & Solomon, B. J. (2022). Effects of high school students’ noncognitive factors on their success at college. Studies in Higher Education, 47(3), 572–586.
Hodson, T. O. (2022). Root mean square error (RMSE) or mean absolute error (MAE): When to use them or not. Geoscientific Model Development Discussions, 2022, 1–10.
Hodson, T. O., Over, T. M., & Foks, S. S. (2021). Mean squared error, deconstructed. Journal of Advances in Modeling Earth Systems, 13(12), e2021MS002681.
Huddar, N. M. (2021). A Study of Life Style of Low Achievers with Regard to Their Academic Life, Non Academic Life, Parents Involvement and Self Regulating Learning Habbit.
Khamis, G. S. M., Mohammed, Z. M. S., Alanazi, S. M., Mahmoud, A. F. A., Abdalla, F. A., & Bkheet, S. A. (2024). Prediction of Myocardial Infarction Complications using Gradient Boosting. Engineering, Technology & Applied Science Research, 14(6), 18550–18556.
Lisnyj, K. T., Pearl, D. L., McWhirter, J. E., & Papadopoulos, A. (2021). Exploration of factors affecting post-secondary students’ stress and academic success: Application of the socio-ecological model for health promotion. International Journal of Environmental Research and Public Health, 18(7), 3779.
Liu, Y., Fu, Y., Peng, Y., & Ming, J. (2024). Clinical decision support tool for breast cancer recurrence prediction using SHAP value in cooperative game theory. Heliyon, 10(2).
Montesinos López, O. A., Montesinos López, A., & Crossa, J. (2022). Support vector machines and support vector regression. In Multivariate Statistical Machine Learning Methods for Genomic Prediction (pp. 337–378). Springer.
Muhammad, Y., Hassan, M. A., Almotairi, S., Farooq, K., Granelli, F., & Strážovská, Ľ. (2023). The Role of Socioeconomic Factors in Improving the Performance of Students Based on Intelligent Computational Approaches. Electronics (Switzerland), 12(9), 1–18. https://doi.org/10.3390/electronics12091982
Najafzadeh, M., & Niazmardi, S. (2021). A novel multiple-kernel support vector regression algorithm for estimation of water quality parameters. Natural Resources Research, 30(5), 3761–3775.
Natras, R., Soja, B., & Schmidt, M. (2022). Ensemble machine learning of random forest, AdaBoost and XGBoost for vertical total electron content forecasting. Remote Sensing, 14(15), 3547.
Onyema, E. M., Almuzaini, K. K., Onu, F. U., Verma, D., Gregory, U. S., Puttaramaiah, M., & Afriyie, R. K. (2022). Prospects and challenges of using machine learning for academic forecasting. Computational Intelligence and Neuroscience, 2022(1), 5624475.
Otchere, D. A., Ganat, T. O. A., Gholami, R., & Ridha, S. (2021). Application of supervised machine learning paradigms in the prediction of petroleum reservoir properties: Comparative analysis of ANN and SVM models. Journal of Petroleum Science and Engineering, 200, 108182.
Rajendran, S., Chamundeswari, S., & Sinha, A. A. (2022). Predicting the academic performance of middle-and high-school students using machine learning algorithms. Social Sciences & Humanities Open, 6(1), 100357.
Robeson, S. M., & Willmott, C. J. (2023). Decomposition of the mean absolute error (MAE) into systematic and unsystematic components. PloS One, 18(2), e0279774.
Syam, N., & Kaul, R. (2021). Random forest, bagging, and boosting of decision trees. In Machine Learning and Artificial Intelligence in Marketing and Sales: Essential Reference for Practitioners and Data Scientists (pp. 139–182). Emerald Publishing Limited.
Veeramsetty, V. (2021). Shapley value cooperative game theory-based locational marginal price computation for loss and emission reduction. Protection and Control of Modern Power Systems, 6(4), 1–11.
Wilson, A., & Anwar, M. R. (2024). The Future of Adaptive Machine Learning Algorithms in High-Dimensional Data Processing. International Transactions on Artificial Intelligence, 3(1), 97–107.
Zhang, F., & O’Donnell, L. J. (2020). Support vector regression. In Machine learning (pp. 123–140). Elsevier.
Avdeef, A. (2021). Do you know your r2? ADMET and DMPK, 9(1), 69–74.
Bakır, R., Orak, C., & Yüksel, A. (2024). Optimizing hydrogen evolution prediction: A unified approach using random forests, lightGBM, and Bagging Regressor ensemble model. International Journal of Hydrogen Energy, 67, 101–110.
Becker, T., Rousseau, A.-J., Geubbelmans, M., Burzykowski, T., & Valkenborg, D. (2023). Decision trees and random forests. American Journal of Orthodontics and Dentofacial Orthopedics, 164(6), 894–897.
Doz, D., Cotič, M., & Felda, D. (2023). Random forest regression in predicting students’ achievements and fuzzy grades. Mathematics, 11(19), 4129.
Ekanayake, I. U., Meddage, D. P. P., & Rathnayake, U. (2022). A novel approach to explain the black-box nature of machine learning in compressive strength predictions of concrete using Shapley additive explanations (SHAP). Case Studies in Construction Materials, 16, e01059.
Gaftandzhieva, S., Talukder, A., Gohain, N., Hussain, S., Theodorou, P., Salal, Y. K., & Doneva, R. (2022). Exploring Online Activities to Predict the Final Grade of Student. Mathematics, 10(20), 1–20. https://doi.org/10.3390/math10203758
Han, C., Farruggia, S. P., & Solomon, B. J. (2022). Effects of high school students’ noncognitive factors on their success at college. Studies in Higher Education, 47(3), 572–586.
Hodson, T. O. (2022). Root mean square error (RMSE) or mean absolute error (MAE): When to use them or not. Geoscientific Model Development Discussions, 2022, 1–10.
Hodson, T. O., Over, T. M., & Foks, S. S. (2021). Mean squared error, deconstructed. Journal of Advances in Modeling Earth Systems, 13(12), e2021MS002681.
Huddar, N. M. (2021). A Study of Life Style of Low Achievers with Regard to Their Academic Life, Non Academic Life, Parents Involvement and Self Regulating Learning Habbit.
Khamis, G. S. M., Mohammed, Z. M. S., Alanazi, S. M., Mahmoud, A. F. A., Abdalla, F. A., & Bkheet, S. A. (2024). Prediction of Myocardial Infarction Complications using Gradient Boosting. Engineering, Technology & Applied Science Research, 14(6), 18550–18556.
Lisnyj, K. T., Pearl, D. L., McWhirter, J. E., & Papadopoulos, A. (2021). Exploration of factors affecting post-secondary students’ stress and academic success: Application of the socio-ecological model for health promotion. International Journal of Environmental Research and Public Health, 18(7), 3779.
Liu, Y., Fu, Y., Peng, Y., & Ming, J. (2024). Clinical decision support tool for breast cancer recurrence prediction using SHAP value in cooperative game theory. Heliyon, 10(2).
Montesinos López, O. A., Montesinos López, A., & Crossa, J. (2022). Support vector machines and support vector regression. In Multivariate Statistical Machine Learning Methods for Genomic Prediction (pp. 337–378). Springer.
Muhammad, Y., Hassan, M. A., Almotairi, S., Farooq, K., Granelli, F., & Strážovská, Ľ. (2023). The Role of Socioeconomic Factors in Improving the Performance of Students Based on Intelligent Computational Approaches. Electronics (Switzerland), 12(9), 1–18. https://doi.org/10.3390/electronics12091982
Najafzadeh, M., & Niazmardi, S. (2021). A novel multiple-kernel support vector regression algorithm for estimation of water quality parameters. Natural Resources Research, 30(5), 3761–3775.
Natras, R., Soja, B., & Schmidt, M. (2022). Ensemble machine learning of random forest, AdaBoost and XGBoost for vertical total electron content forecasting. Remote Sensing, 14(15), 3547.
Onyema, E. M., Almuzaini, K. K., Onu, F. U., Verma, D., Gregory, U. S., Puttaramaiah, M., & Afriyie, R. K. (2022). Prospects and challenges of using machine learning for academic forecasting. Computational Intelligence and Neuroscience, 2022(1), 5624475.
Otchere, D. A., Ganat, T. O. A., Gholami, R., & Ridha, S. (2021). Application of supervised machine learning paradigms in the prediction of petroleum reservoir properties: Comparative analysis of ANN and SVM models. Journal of Petroleum Science and Engineering, 200, 108182.
Rajendran, S., Chamundeswari, S., & Sinha, A. A. (2022). Predicting the academic performance of middle-and high-school students using machine learning algorithms. Social Sciences & Humanities Open, 6(1), 100357.
Robeson, S. M., & Willmott, C. J. (2023). Decomposition of the mean absolute error (MAE) into systematic and unsystematic components. PloS One, 18(2), e0279774.
Syam, N., & Kaul, R. (2021). Random forest, bagging, and boosting of decision trees. In Machine Learning and Artificial Intelligence in Marketing and Sales: Essential Reference for Practitioners and Data Scientists (pp. 139–182). Emerald Publishing Limited.
Veeramsetty, V. (2021). Shapley value cooperative game theory-based locational marginal price computation for loss and emission reduction. Protection and Control of Modern Power Systems, 6(4), 1–11.
Wilson, A., & Anwar, M. R. (2024). The Future of Adaptive Machine Learning Algorithms in High-Dimensional Data Processing. International Transactions on Artificial Intelligence, 3(1), 97–107.
Zhang, F., & O’Donnell, L. J. (2020). Support vector regression. In Machine learning (pp. 123–140). Elsevier.
Copyright and Licensing
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.