Employing particle swarm optimization algorithm for shrinkage parameter estimation in generalized Liu estimator

  • Authors

    • Qamar Abdulkareem Abdulazeez University of Mosul
    • Zakariya Yahya Algamal University of Mosul
    2020-05-15
    https://doi.org/10.14419/ijasp.v8i1.30565
  • Multicollinearity, Shrinkage Parameter, Generalized Liu Estimator, Particle Swarm Optimization.

  • Abstract

    It is well-known that in the presence of multicollinearity, the Liu estimator is an alternative to the ordinary least square (OLS) estimator and the ridge estimator. Generalized Liu estimator (GLE) is a generalization of the Liu estimator. However, the efficiency of GLE depends on appropriately choosing the shrinkage parameter matrix which is involved in the GLE. In this paper, a particle swarm optimization method, which is a metaheuristic continuous algorithm, is proposed to estimate the shrinkage parameter matrix. The simulation study and real application results show the superior performance of the proposed method in terms of prediction error.

     

     

     

  • References

    1. [1] Al-Hassan, Y. M. (2010). Performance of a new ridge regression estimator. Journal of the Association of Arab Universities for Basic and Applied Sciences, 9(1), 23-26. https://doi.org/10.1016/j.jaubas.2010.12.006.

      [2] Alheety, M., & Kibria, B. G. (2009). On the Liu and almost unbiased Liu estimators in the presence of multicollinearity with heteroscedastic or correlated errors. Surveys in Mathematics and its Applications, 4, 155-167.

      [3] Alheety, M., & Kibria, B. G. (2014). A generalized stochastic restricted ridge regression estimator. Communications in Statistics-Theory and Methods, 43(20), 4415-4427. https://doi.org/10.1080/03610926.2012.724506.

      [4] Alkhamisi, M. A., & Shukur, G. (2007). A Monte Carlo study of recent ridge parameters. Communications in Statistics—Simulation and Computation®, 36(3), 535-547. https://doi.org/10.1080/03610910701208619.

      [5] Asar, Y., & Genç, A. (2015). New shrinkage parameters for the Liu-type logistic estimators. Communications in Statistics - Simulation and Computation, 45(3), 1094-1103. https://doi.org/10.1080/03610918.2014.995815.

      [6] Asar, Y., Karaibrahimoğlu, A., & Genç, A. (2014). Modified ridge regression parameters: A comparative Monte Carlo study. Hacettepe Journal of Mathematics and Statistics, 43(5), 827-841.

      [7] Batah, F. S. M., Ramanathan, T. V., & Gore, S. D. (2008). The efficiency of modified jackknife and ridge type regression estimators: a comparison. Surveys in Mathematics & its Applications, 3.

      [8] Bhat, S., & Raju, V. (2017). A class of generalized ridge estimators. Communications in Statistics-Simulation and Computation, 46(7), 5105-5112. https://doi.org/10.1080/03610918.2016.1144765.

      [9] Cervantes, J., Garcia-Lamont, F., Rodriguez, L., López, A., Castilla, J. R., & Trueba, A. (2017). PSO-based method for SVM classification on skewed data sets. Neurocomputing, 228, 187-197. https://doi.org/10.1016/j.neucom.2016.10.041.

      [10] Chen, K.-H., Wang, K.-J., Wang, K.-M., & Angelia, M.-A. (2014). Applying particle swarm optimization-based decision tree classifier for cancer classification on gene expression data. Applied Soft Computing, 24(0), 773-780. https://doi.org/10.1016/j.asoc.2014.08.032.

      [11] Dorugade, A. (2014). New ridge parameters for ridge regression. Journal of the Association of Arab Universities for Basic and Applied Sciences, 15(1), 94-99. https://doi.org/10.1016/j.jaubas.2013.03.005.

      [12] Dorugade, A., & Kashid, D. (2010). Alternative method for choosing ridge parameter for regression. Applied Mathematical Sciences, 4(9), 447-456.

      [13] Firinguetti, L. (1999). A generalized ridge regression estimator and its finite sample properties: A generalized ridge regression estimator. Communications in Statistics-Theory and Methods, 28(5), 1217-1229. https://doi.org/10.1080/03610929908832353.

      [14] Hocking, R. R., Speed, F., & Lynn, M. (1976). A class of biased estimators in linear regression. Technometrics, 18(4), 425-437. https://doi.org/10.1080/00401706.1976.10489474.

      [15] Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67. https://doi.org/10.1080/00401706.1970.10488634.

      [16] Kennedy, J., & Eberhart, R. C. (1995). Particle swarm optimization. Proceedings of IEEE Conference on Neural Network, 4, 1942–1948. https://doi.org/10.1109/ICNN.1995.488968.

      [17] Kiran, M. S. (2017). Particle swarm optimization with a new update mechanism. Applied Soft Computing, 60, 670-678. https://doi.org/10.1016/j.asoc.2017.07.050.

      [18] Lai, C.-M., Yeh, W.-C., & Chang, C.-Y. (2016). Gene selection using information gain and improved simplified swarm optimization. Neurocomputing, 218, 331-338. https://doi.org/10.1016/j.neucom.2016.08.089.

      [19] Lin, S.-W., Ying, K.-C., Chen, S.-C., & Lee, Z.-J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35(4), 1817-1824. https://doi.org/10.1016/j.eswa.2007.08.088.

      [20] Liu, K. (1993). A new class of biased estimate in linear regression. Communications in Statistics-Theory and Methods, 22(2), 393-402. https://doi.org/10.1080/03610929308831027.

      [21] Lu, Y., Wang, S., Li, S., & Zhou, C. (2009). Particle swarm optimizer for variable weighting in clustering high-dimensional data. Machine Learning, 82(1), 43-70. https://doi.org/10.1007/s10994-009-5154-2.

      [22] MÃ¥nsson, K., Shukur, G., & Golam Kibria, B. (2010). A simulation study of some ridge regression estimators under different distributional assumptions. Communications in Statistics-Simulation and Computation, 39(8), 1639-1670. https://doi.org/10.1080/03610918.2010.508862.

      [23] McDonald, G. C., & Galarneau, D. I. (1975). A Monte Carlo evaluation of some ridge-type estimators. Journal of the American Statistical Association, 70(350), 407-416. https://doi.org/10.1080/01621459.1975.10479882.

      [24] Mirjalili, S., & Lewis, A. (2013). S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization. Swarm and Evolutionary Computation, 9, 1-14. https://doi.org/10.1016/j.swevo.2012.09.002.

      [25] Nomura, M. (1988). On the almost unbiased ridge regression estimator. Communications in Statistics-Simulation and Computation, 17(3), 729-743. https://doi.org/10.1080/03610918808812690.

      [26] Qasim, M., Amin, M., & Amanullah, M. (2018). On the performance of some new Liu parameters for the gamma regression model. Journal of Statistical Computation and Simulation, 88(16), 3065-3080. https://doi.org/10.1080/00949655.2018.1498502.

      [27] Troskie, C., & Chalton, D. (1996). Detection of outliers in the presence of multicollinearity. Paper presented at the Multidimensional statistical analysis and theory of random matrices, Proceedings of the Sixth Lukacs Symposium, eds. Gupta, AK and VL Girko. https://doi.org/10.1515/9783110916690-022.

      [28] Wen, J. H., Zhong, K. J., Tang, L. J., Jiang, J. H., Wu, H. L., Shen, G. L., & Yu, R. Q. (2011). Adaptive variable-weighted support vector machine as optimized by particle swarm optimization algorithm with application of QSAR studies. Talanta, 84(1), 13-18. https://doi.org/10.1016/j.talanta.2010.11.039.

      [29] Yang, S.-P., & Emura, T. (2017). A Bayesian approach with generalized ridge estimation for high-dimensional regression and testing. Communications in Statistics-Simulation and Computation, 46(8), 6083-6105. https://doi.org/10.1080/03610918.2016.1193195.

      [30] Zhou, W., & Dickerson, J. A. (2014). A novel class dependent feature selection method for cancer biomarker discovery. Computers in Biology and Medicine, 47, 66-75. https://doi.org/10.1016/j.compbiomed.2014.01.014.

  • Downloads

  • Received date: 2020-03-27

    Accepted date: 2020-05-07

    Published date: 2020-05-15