Comparative analysis of some linear predictive models in the presence of multicollinearity

  • Authors

    • Owoyemi Q.A University of Ilorin, Ilorin, Nigeria.
    • Bolakale A. University of Ilorin, Ilorin, Nigeria.
    2024-09-25
    https://doi.org/10.14419/r2bqgv16
  • Variance Inflation Factor; Multicollinearity; Regression Analysis; Regularization; Predictive Models.

  • Abstract

    This paper investigates the comparative performance of some linear predictive models in the presence of multicollinearity. By examining the efficacy of Ordinary Least Squares (OLS), Ridge Regression, Lasso Regression, and Elastic Net Regression, this study aimed to figure out the best method for building robust and interpretable models under such conditions. The research explores how these models address multicollinearity, focusing on coefficient stability, prediction accuracy, and variable selection. Through a rigorous analysis of simulated and real-world datasets, the study shows the strengths and weaknesses of each model, providing valuable insights for researchers and practitioners looking to mitigate the challenges posed by multicollinearity in selecting the most proper method for regression modeling. This will lead to the creation of a model with increased interpretability of the relationships between variables, less variance, and more dependable coefficient estimations.

  • References

    1. Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Pearson Education.
    2. James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An introduction to sta-tistical learning (Vol. 112, No. 1). Springer. https://doi.org/10.1007/978-1-0716-1418-1_1.
    3. Menard, S. (2020). Applied logistic regression (3rd ed.). Sage Publications.
    4. Montgomery, D. C., & Chatterjee, S. (2021). Design and analysis of experiments (10th ed.). John Wiley & Sons.
    5. Gujarati, D. N. (2004). Basic econometrics (4th ed.). McGraw-Hill.
    6. Fox, J. (1991). Regression diagnostics: An introduction. Sage Publications. https://doi.org/10.4135/9781412985604.
    7. Belsley, D. A., Kuh, E., & Welsch, R. E. (1980). Regression diagnostics: Identify-ing influential data and sources of collinearity. John Wiley & Sons. https://doi.org/10.1002/0471725153.
    8. Shen, X., Pan, W., & Ye, J. (2009). Ridge regression versus least squares for high-dimensional linear regression. Journal of Computa-tional and Graphical Statistics, 18(1), 158-17 Ridge Regression.
    9. Huang, Y., Wang, H., & Li, H. (2012). Finding consumer purchase drivers with LASSO regression: An empirical investigation in the mobile app market. Journal of Marketing Analytics, 1, 27-41.
    10. Liu, J., Wu, Y., & Zhang, R. (2010). A new approach to multicollinearity: Elastic Net regression. Journal of Machine Learning Research, 11, 2239-2258.
    11. Altelbany, M., Al-Azzani, A., & Al-Olayan, A. (2021). Evaluation of Ridge, Elastic Net and Lasso Regression Methods in Precedence of Multicollinearity Prob-lem: A Simulation Study. International Journal of Advanced Computer Science and Applications, 12, 17-26.

  • Downloads