Solving Classification Problem Using Ensemble Binarization Classifier

  • Authors

    • M. N. Shah Zainudin
    • Md. Nasir Sulaiman
    • Norwati Musapha
    • Thinagaran Perumal
    • Raihani Mohamed
    2018-12-09
    https://doi.org/10.14419/ijet.v7i4.31.23381
  • Ensemble decision tree, Bagging, Boosting, OVA, PAMAP2, WISDM.

  • Binarization strategy is broadly applied in solving various multi-class classification problems. However, the classifier model learning complexity tends to increase when expanding the number of problems into several replicas. One-Versus-All (OVA) is one of the strategies which transforming the ordinal multi-class classification problems into a series of two-class classification problems. The final output from each classifier model is combined in order to produce the final prediction. This binarization strategy has been proven as superior performance in accuracy than ordinal multi-class classifier model. However, learning model complexity (eg. Random Forest-RF ensemble decision trees) tends to increase when employing a large number of trees. Even though a large number of trees might produce a decent accuracy, generating time of the learning model is significantly longer. Hence, self-tuning tree parameter is introduced to tackle this matter. In such circumstances, a number of trees in the RF classifier are defined according to the number of class problem. In this paper, the OVA with self-tuning is evaluated based on parameter initialization in the context of RF ensemble decision tree. At the same time, the performance has also been compared with two classifier models such J48 and boosting for several well-known datasets.

     

     

  • References

    1. [1] Sun Z., Song Q., Zhu X., Sun H., Xu B., and Zhou Y. (2015), “A novel ensemble method for classifying imbalanced data,†Pattern Recognit., vol. 48, no. 5, 1623–1637.

      [2] Chandrashekar G. and Sahin F. (2014), “A survey on feature selection methods,†Comput. Electr. Eng., vol. 40, no. 1, 16–28.

      [3] Mohamed R., Zainudin M. N. S., Sulaiman N., and Perumal T. (2018), “Multi-label Classification for Physical Recognition from Various Accelerometer Sensor Positions,†J. Inf. Commun. Technol., vol. No. 2, no. 18, 209–231.

      [4] Krawczyk B. (2016), “Learning from imbalanced data : open challenges and future directions,†Prog. Artif. Intell..

      [5] Fürnkranz J. (2002), “Pairwise Classification as an Ensemble Technique,†Mach. Learn. ECML 2002, vol. 2430, no. 2000, 9–38.

      [6] Quinlan J. R. (1986), “Induction of Decision Trees,†Mach. Learn., vol. 1, no. 1, 81–106.

      [7] Mohamed R., Perumal T., Sulaiman N., Mustapha N., and Abd Manaf S. (2017), “Tracking and Recognizing the Activity of Multi Resident in Smart Home Environments,†J. Telecommun. Electron. Comput. Eng., vol. 9, no. 2–11, 39–43.

      [8] Breiman L. (2001), “Random forests,†Mach. Learn., vol. 45, no. 1, 5–32.

      [9] Abidine M. B.and Fergani B. (2012), “Evaluating C -SVM , CRF and LDA Classification for Daily Activity Recognition,â€.

      [10] Martínez J., Iglesias C., Matías J. M., Taboada J., and Araújo M. (2014), “Solving the slate tile classification problem using a DAGSVM multiclassification algorithm based on SVM binary classifiers with a one-versus-all approach,†Appl. Math. Comput., vol. 230, 464–472.

      [11] Ng S. S. Y., Tse P. W., and Tsui K. L. (2014), “A One-Versus-All Class Binarization Strategy for Bearing Diagnostics of Concurrent Defects,†sensors, vol. 14, 1295–1321.

      [12] Wu K. (2015), “AOSA-LogistBoost : Adaptive One-Vs-All LogistBoost for Multi-Class Classification Problems,†in 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), 654–662.

      [13] Gaikwad V. S. and Kulkarni P. J. (2012), “One Versus All classification in Network Intrusion detection using Decision Tree,†Int. J. Sci. Res. Publ., vol. 2, no. 3, 1–5.

      [14] Varpa K., Joutsijoki H., Iltanen K., and Juhola M. (2011), “Applying One-vs-One and One-vs-All Classifiers in k -Nearest Neighbour Method and Support Vector Machines to an Otoneurological Multi-Class Problem,†User Centred Networked Heal. Care, 579–583.

      [15] Li Y., Shi D., Ding B., and Liu D. (2014), “Unsupervised learning for human activity recognition using smartphone sensors,†Expert Syst. Appl., vol. 41, 6067–6074.

      [16] Zhang X., Xiong H., Zhou W., and Tian Q. (2016), “Fused One-vs-All Features With Semantic Alignments for Fine-Grained Visual Categorization,†IEEE Trans. IMAGE Process., vol. 25, no. 2, 878–892.

      [17] Wang X. and Niu Y. (2013), “New one-versus-all ν -SVM solving intra – inter class imbalance with extended manifold regularization and localized relative maximum margin,†Neurocomputing, vol. 115, 106–121.

      [18] Adnan M. N. and Islam M. Z. (2015), “One-Vs-All Binarization Technique in the Context of Random Forest,†in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning., 2015, no. April, 22–24.

      [19] Bolon-Canedo V., Sanchez-Marono N. and Alonso-Betanzos A. (2005), “Data classification using an ensemble of filters,†Neurocomputing, vol. 135, 13–20.

      [20] Ravi N., Dandekar N., Mysore P., and Littman M. L. (2005), “Activity Recognition from Accelerometer Data,†Am. Assoc. Artif. Intell., 1541–1546.

      [21] Ayu M. A., Ismail S. A., Abdul Matin A. F., and Mantoro T. (2012), “A comparison study of classifier algorithms for mobile-phone’s accelerometer based activity recognition,†Procedia Eng., vol. 41, no. Iris, 224–229.

      [22] Daghistani T. and Alshammari R. (2016), “Improving Accelerometer-Based Activity Recognition by Using Ensemble of Classifiers,†Int. J. Adv. Comput. Sci. Appl., vol. 7, no. 5, 128–133.

      [23] Kwapisz J. R., Weiss G. M., and Moore S. a. (2011), “Activity recognition using cell phone accelerometers,†ACM SIGKDD Explor. Newsl., vol. 12, 74.

      [24] Reiss A. and Stricker D., “Introducing a new benchmarked dataset for activity monitoring,†in Proceedings - International Symposium on Wearable Computers, ISWC, 2012, 108–109.

      [25] Bhatt J. (2014), “A Survey on One Class Classification using Ensembles Method,†Int. J. Innov. Res. Sci. Technol., vol. 1, no. 7, 19–23.

      [26] Zainudin M. N. S., Sulaiman N., Mustapha N., and Perumal T. (2015), “Activity Recognition based on Accelerometer Sensor using Combinational Classifiers,†in ICOS2015, 68–73.

      [27] Hall M. a. and Smith L. a. (1998), “Practical feature subset selection for machine learning,†Comput. Sci., vol. 98, 181–191.

      [28] Probst P. and Boulesteix A.-L. (2017), “To tune or not to tune the number of trees in random forest?,†no. 2001, 1–20.

      [29] Zainudin M. N. S., Sulaiman N., Mustapha N., and Perumal T. (2017), “Pairwise Classification Using Combination of Statistical Descriptors with Spectral Analysis Features for Recognizing Walking Activities,†J. Telecommun. Electron. Comput. Eng., vol. 9, no. 2, 55–60.

  • Downloads

  • How to Cite

    N. Shah Zainudin, M., Nasir Sulaiman, M., Musapha, N., Perumal, T., & Mohamed, R. (2018). Solving Classification Problem Using Ensemble Binarization Classifier. International Journal of Engineering & Technology, 7(4.31), 280-284. https://doi.org/10.14419/ijet.v7i4.31.23381