Parallel Self-Organizing Map Using Heterogeneous Uniform Memory Access

  • Authors

    • Muhammad Firdaus Mustapha
    • Noor Elaiza Abd Khalid
    • Azlan Ismail
    2019-01-18
    https://doi.org/10.14419/ijet.v8i1.7.25959
  • Graphic Processing Unit, Heterogeneous Computing, Heterogeneous Uniform Memory Access, Parallel Computing, Parallel SOM.

  • Abstract

    Self-organizing Map (SOM) is a useful data analysis method that consists of complex calculations. Numerous researchers have successfully improved online SOM processing speed using Heterogeneous Computing (HC). HC is a combination of Central Processing Unit (CPU) and Graphic Processing Unit (GPU) that work closely together by accessing separate memory blocks. Although the standard HC has an excellent performance, performing online SOM variant will cause computer hardware underutilized when number of neurons in SOM map is smaller than number of cores. Moreover, complexities of steps in SOM algorithm drive to increase the usage of high memory capacity. Nevertheless, this situation leads to communication latency that caused by “deep copies†method. Recently, Heterogeneous Uniform Memory Access (HUMA) platform has a noteworthy parallel processing capability. Therefore, this paper presents parallel SOM using HUMA platform and multiple stimuli approach. In this study, the main goal is to reduce computation time of SOM training via implementing parallel SOM on HUMA platform and increase GPU cores utilization. This study employed dataset from UCI repository. As a result, the proposed work was able to score up to 1.23 of speed up overall for large map size compared to standard parallel SOM. Hence, the proposed work is able to suggest a greater solution for small to medium sized of data analysis software.

  • References

    1. [1] W.-P. Tsai, S.-P. Huang, S.-T. Cheng, K.-T. Shao, and F.-J. Chang, “A data-mining framework for exploring the multi-relation between fish species and water quality through self-organizing map,†Sci. Total Environ., vol. 579, pp. 474–483, 2016.

      [2] J. Llanos et al., “Load estimation for microgrid planning based on a self-organizing map methodology,†Appl. Soft Comput., vol. 53, pp. 323–335, 2017.

      [3] F. Matic et al., “Oscillating Adriatic temperature and salinity regimes mapped using the Self-Organizing Maps method,†Cont. Shelf Res., vol. 132, no. September 2016, pp. 11–18, 2017.

      [4] T. Kohonen, “Essentials of the self-organizing map.,†Neural Netw., vol. 37, pp. 52–65, Jan. 2013.

      [5] S. McConnell, R. Sturgeon, G. Henry, A. Mayne, and R. Hurley, “Scalability of Self-organizing Maps on a GPU cluster using OpenCL and CUDA,†J. Phys. Conf. Ser., vol. 341, p. 012018, 2012.

      [6] S. Hasan, S. M. Shamsuddin, and N. Lopes, “Machine Learning Big Data Framework and Analytics for Big Data Problems,†Int. J. Adv. Soft Comput. Appl., vol. 6, no. 2, pp. 1–17, 2014.

      [7] K. Ben Khalifa, B. Girau, F. Alexandre, and M. H. Bedoui, “Parallel FPGA implementation of self-organizing maps,†in Proceedings. The 16th International Conference on Microelectronics, 2004. ICM 2004., 2004, pp. 709–712.

      [8] W. Kurdthongmee, “A novel Kohonen SOM-based image compression architecture suitable for moderate density {FPGAs},†Image Vis. Comput., vol. 26, no. 8, pp. 1094–1105, 2008.

      [9] W. Kurdthongmee, “A low latency minimum distance searching unit of the {SOM} based hardware quantizer,†Microprocess. Microsyst., vol. 39, no. 2, pp. 135–143, 2015.

      [10] T. Abe, Y. Hamano, S. Kanaya, K. Wada, and T. Ikemura, “A Large-Scale Genomics Studies Conducted with Batch-Learning SOM Utilizing High-Performance Supercomputers,†in Bio-Inspired Systems: Computational and Ambient Intelligence: 10th International Work-Conference on Artificial Neural Networks, IWANN 2009, Salamanca, Spain, June 10-12, 2009. Proceedings, Part I, J. Cabestany, F. Sandoval, A. Prieto, and J. M. Corchado, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 829–836.

      [11] S. J. Sul and A. Tovchigrechko, “Parallelizing BLAST and SOM Algorithms with MapReduce-MPI Library,†in 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum, 2011, pp. 481–489.

      [12] F. C. Moraes, S. C. Botelho, N. D. Filho, and J. F. O. Gaya, “Parallel High Dimensional Self Organizing Maps Using CUDA,†2012 Brazilian Robot. Symp. Lat. Am. Robot. Symp., pp. 302–306, Oct. 2012.

      [13] M. Mojarab, H. Memarian, M. Zare, A. Hossein Morshedy, and M. Hossein Pishahang, “Modeling of the seismotectonic provinces of Iran using the self-organizing map algorithm,†Comput. Geosci., vol. 67, pp. 150–162, Jun. 2014.

      [14] T. Richardson and E. Winer, “Extending parallelization of the self-organizing map by combining data and network partitioned methods,†Adv. Eng. Softw., vol. 88, pp. 1–7, 2015.

      [15] T. Kohonen, “The self-organizing map,†Neurocomputing, vol. 21, no. 1–3, pp. 1–6, 1998.

      [16] M. Takatsuka and M. Bui, “Parallel batch training of the self-organizing map using OpenCL,†Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 6444 LNCS, pp. 470–476, 2010.

      [17] L. Zhu, W. Guo, and Y. Bai, “Self-Organizing Maps Neural Networks on Parallel Cluster,†in 2009 First International Conference on Information Science and Engineering, 2009, pp. 384–388.

      [18] J. Fort, P. Letremy, M. Cottrell, I. Elie, and U. Nancy, “Advantages and drawbacks of the Batch Kohonen algorithm,†Proc Eur. Symp. Artif. Neural Networks ESANN, pp. 223–230, 2002.

      [19] G. Dzemyda and O. Kurasova, “Parallelization of the SOM-Based Integrated Mapping,†in Artificial Intelligence and Soft Computing - ICAISC 2004: 7th International Conference, Zakopane, Poland, June 7-11, 2004. Proceedings, L. Rutkowski, J. H. Siekmann, R. Tadeusiewicz, and L. A. Zadeh, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004, pp. 178–183.

      [20] C. Garcia, M. Prieto, and A. Pascual-Montano, “A Speculative Parallel Algorithm for Self-Organizing Maps,†Proc. Parallel Comput. 2005 (ParCo 2005), vol. 33, pp. 615–622, 2005.

      [21] D. MacLean and I. Valova, “Parallel Growing SOM Monitored by Genetic Algorithm,†in 2007 International Joint Conference on Neural Networks, 2007, pp. 1697–1702.

      [22] P. Gajdos and J. Platos, “GPU Based Parallelism for Self-Organizing Map,†in Advances in Intelligent Systems and Computing, 2013, vol. 179, no. Ihci 2011, pp. 3–12.

      [23] U. Seiffert and B. Michaelis, “Multi-Dimensional Self-Organizing Maps on Massively Parallel Hardware,†in Advances in Self-Organising Maps, London: Springer London, 2001, pp. 160–166.

      [24] V. Demian and J. C. Mignot, “Optimization of the self-organizing feature map on parallel computers,†in Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan), 1993, vol. 1, pp. 483–486 vol.1.

      [25] M. Yasunaga, K. Tominaga, and J. H. K. J. H. Kim, “Parallel self-organization map using multiple stimuli,†IJCNN’99. Int. Jt. Conf. Neural Networks. Proc. (Cat. No.99CH36339), vol. 2, pp. 1127–1130, 1999.

      [26] S. Q. Khan and M. A. Ismail, “Design and implementation of parallel SOM model on GPGPU,†2013 5th Int. Conf. Comput. Sci. Inf. Technol., pp. 233–237, Mar. 2013.

      [27] A. Faro, D. Giordano, and S. Palazzo, “Integrating unsupervised and supervised clustering methods on a GPU platform for fast image segmentation,†2012 3rd Int. Conf. Image Process. Theory, Tools Appl. IPTA 2012, pp. 85–90, 2012.

      [28] J. Lachmair, E. Merényi, M. Porrmann, and U. Rückert, “A reconfigurable neuroprocessor for self-organizing feature maps,†Neurocomputing, vol. 112, pp. 189–199, Jul. 2013.

      [29] A. De, Y. Zhang, and C. Guo, “A parallel image segmentation method based on SOM and GPU with application to MRI image processing,†Neurocomputing, vol. 198, pp. 180–189, 2016.

      [30] K. Perelygin, S. Lam, and X. Wu, “Graphics Processing Units and Open Computing Language for parallel computing,†Comput. Electr. Eng., vol. 40, no. 1, pp. 241–251, Jan. 2014.

      [31] D. B. Kirk and W. W. Hwu, Programming Massively Parallel Processors. Elsevier, 2013.

      [32] S. Mittal and J. S. Vetter, “A Survey of CPU-GPU Heterogeneous Computing Techniques,†ACM Comput. Surv., vol. 47, no. 4, p. 69:1--69:35, Jul. 2015.

      [33] S. Mukherjee, Y. Sun, P. Blinzer, A. K. Ziabari, and D. Kaeli, “A Comprehensive Performance Analysis of HSA and OpenCL 2.0,†2016 IEEE Int. Symp. Perform. Anal. Syst. Softw., no. April, 2016.

      [34] G. Kyriazis, “Heterogeneous System Architecture : A Technical Review,†pp. 1–18, 2012.

      [35] AMD Inc., “Compute Cores,†pp. 1–6, 2014.

      [36] S. Junkins, “The Compute Architecture of Intel ® Processor Graphics Gen9,†2015.

      [37] K. OpenCL, “OpenCL Specification,†2014.

      [38] M. Lichman, “UCI Machine Learning Repository,†Irvine, CA: University of California, School of Information and Computer Science, 2013. [Online]. Available: http://archive.ics.uci.edu/ml.

      [39] M. F. Mustapha, N. E. Abd Khalid, A. Ismail, and M. Manaf, “Enhancing Parallel Self-organizing Map on Heterogeneous System Architecture,†in Soft Computing in Data Science, 2017, pp. 162–174.

      [40] S. Moro, P. Cortez, and P. Rita, “A data-driven approach to predict the success of bank telemarketing,†Decis. Support Syst., vol. 62, pp. 22–31, 2014.

  • Downloads

  • How to Cite

    Firdaus Mustapha, M., Elaiza Abd Khalid, N., & Ismail, A. (2019). Parallel Self-Organizing Map Using Heterogeneous Uniform Memory Access. International Journal of Engineering & Technology, 8(1.7), 75-81. https://doi.org/10.14419/ijet.v8i1.7.25959

    Received date: 2019-01-16

    Accepted date: 2019-01-16

    Published date: 2019-01-18