A Comparison of OLS and Ridge Regression Methods in the Presence of Multicollinearity Problem in the Data

  • Authors

    • N S M Shariff
    • H M B Duzan
    2018-11-30
    https://doi.org/10.14419/ijet.v7i4.30.21999
  • Multicollinearity, OLS, Ridge Regression.
  • The presence of multicollinearity will significantly lead to inconsistent parameter estimates in regression modeling. The common procedure in regression analysis that is Ordinarily Least Squares (OLS) is not robust to multicollinearity problem and will result in inaccurate model. To solve this problem, a number of methods are developed in the literatures and the most common method is ridge regression. Although there are many studies propose variety method to overcome multicolinearity problem in regression analysis, this study proposes the simplest model of ridge regression which is based on linear combinations of the coefficient of the least squares regression of independent variables to determine the value of  k (ridge estimator in ridge regression model). The performance of the proposed method is investigated and compared to OLS and some recent existing methods. Thus, simulation studies based on Monte Carlo simulation study are considered. The result of this study is able to produce similar findings as in existing method and outperform OLS in the existence of multicollinearity in the regression modeling.

  • References

    1. [1] Hoerl AE & Kennard RW (1970), Ridge regression: applications to nonorthogonal problem. Technometrics, 12(1), pp. 69-78.

      [2] Hoerl AE & Kennard RW (1970), Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12(1), pp. 55-67.

      [3] Duzan H & Shariff NSM (2015), Ridge regression for solving the multicollinearity problem: review of methods and models. Journal of Applied Sciences, 15(3), pp. 392-404.

      [4] Duzan H & Shariff NSM (2016), Solution to the multicollinearity problem by adding some constant to the diagonal. Journal of Modern Appllied Statistical Methods, 15(1), pp. 752-773.

      [5] Mansson K, Shukur G & Kibria BMG (2010), A simulation study of some ridge regression estimators under different distributional assumptions. Communications in Statistics- Simulation and Computation, 39(8), pp. 1639-1670.

      [6] Kibria BMG (2003), Performance of some new ridge regression estimators. Communications in Statistics- Simulation and Computation, 32(2), pp. 419-435.

      [7] Khalaf G & Shukur G (2005), Choosing ridge parameter for regression problems. Communications in Statistics - Theory and Methods, 34(5), pp. 1177-1182.

      [8] Alkhamisi M & Shukur G (2008), Developing ridge parameters for SUR model. Communications in Statistics - Theory and Methods, 37(4), pp. 544-564.

      [9] Muniz G, Kibria BMG & Shukur G (2012), On developing ridge regression parameters: a graphical investigation. Department of Mathematics and Statistics 1.http://digitalcommons.fiu/math_fac/10

      [10] Muniz G & Kibria BMG (2009), On some ridge regression estimators: an empirical comparisons. Communications in Statistics- Simulation and Computation, 38(3), pp. 621-630.

      [11] Newhouse JP & Oman SD (1971), An evaluation of ridge estimators. Rand Report, No R-716-Pr, pp. 1-28.

  • Downloads

  • How to Cite

    Shariff, N. S. M., & Duzan, H. M. B. (2018). A Comparison of OLS and Ridge Regression Methods in the Presence of Multicollinearity Problem in the Data. International Journal of Engineering & Technology, 7(4.30), 36-38. https://doi.org/10.14419/ijet.v7i4.30.21999