Technology and Architecture for a System of High-Speed Sensor Data Stream Collection and Processing

  • Authors

    • Vladimir V. Kopytov
    • Pavel V. Kharechki
    • Vladimir V. Naumenko
    • Aleksey V. Savartsov
    • Oleg V. Dorofeyev
    • Mikhail V. Batashan
    2018-12-09
    https://doi.org/10.14419/ijet.v7i4.36.23745
  • facility monitoring, environmental monitoring, fog computing, cloud computing, information sensors.

  • Abstract

    Objective: The objective is to address the challenges of monitoring process facility and environmental parameters, which can be analyzed to anticipate dangerous and critical conditions.

    Methodology/approach: This article proposes the technology and architecture for a system of high-speed stream data collection and processing, which combines the advantages of both the cloud and fog computing models for data collection, storage and processing.

    Conclusion: The proposed technology and architecture for a system of high-speed stream data collection and processing make it possible to adapt to various monitoring and situation control challenges and can be used to set up centers for processing monitoring data of different levels.

    Originality/value: The originality of the proposed technology and architecture consists in the application of a set of universal programming solutions aiming to set up a data processing center. Such a center would require a minimum amount of work related to designing an automated data collection system and to developing additional software. Furthermore, it will provide ample opportunities for further scaling and expanding its functionality.

     

     

  • References

    1. [1] J. Gantz, D. Reinsel. The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East–United States, IDC Country Brief. URL: http://www.emc.com/collateral/analyst-reports/idc-digital-universe-united-states.pdf

      [2] OpenFog Reference Architecture for Fog Computing. URL: https://knect365.com/cloud-enterprise-tech/article/0fa40de2-6596-4060-901d-8bdddf167cfe/openfog-reference-architecture-for-fog-computing

      [3] Rob van der Meulen. 6.4 Billion Connected “Things Will Be in Use in 2016â€, Gartner. URL: https://www.gartner.com/newsroom/id/3165317

      [4] Tebueva F.B., Kopytov V.V., Petrenko V.I., Kharechkin P.V., Sidorchuk A.V. Method for Detecting and Eliminating Data Time Series Outlier in High-Speed Process Data Sensors. International Journal on Communications Antenna and Propagation (IRECAP), vol. 7, no. 7, 2017, pp. 603-612.

      [5] F. Bonomi, R. Milito, J. Zhu, and S. Addepalli. “Fog Computing and Its Role in the Internet of Things†in Proc. 1st Edition MCC Workshop Mobile Cloud Comput., Helsinki, Finland, 2012, pp. 13-16.

      [6] OpenFog Reference Architecture for Fog Computing. URL: https://www.openfogconsortium.org/wp-content/uploads/OpenFog_Reference_ Architecture_2_09_17-FINAL.pdf

      [7] Furht, B. and Escalante, A. Cloud Computing Fundamentals. In Handbook of Cloud Computing, Springer, 2010.

      [8] Manuel Díaz, Cristian Martín, Bartolomé Rubio. State-of-the-Art, Challenges, and Open Issues in the Integration of Internet of Things and Cloud Computing (Preprint), 2016. DOI: http://dx.doi.org/10.1016/j.jnca.2016.01.010

      [9] Apache Kafka. URL: https://kafka.apache.org/

      [10] Apache ActiveMQ. URL: http://activemq.apache.org/

      [11] Eclipse Mosquitto (an open source MQTT broker). URL: https://mosquitto.org/

      [12] RabbitMQ. URL: https://www.rabbitmq.com/

      [13] Apache ZooKeeper. URL: https://zookeeper.apache.org/

      [14] Dmitry Namiot. On Big Data Stream Processing. International Journal of Open Information Technologies, vol. 3, no. 8, 2015, pp. 48-51.

      [15] Rick Cattell. Scalable SQL and NoSQL Data Stores, ACM SIGMOD. Record 39, no. 4, 2011, pp. 12-27.

      [16] Kałużka, J., Napieralska, M., Romero, O., Jovanovic, P. Data Locality in Hadoop. International Journal of Microelectronics and Computer Science, vol. 8, no. 1, 2017, pp. 16-20.

      [17] OpenTSDB (The Scalable Time Series Database). URL: http://opentsdb.net/

      [18] Adam Kawa “Introduction to YARNâ€. URL: https://www.ibm.com/developerworks/library/bd-yarn-intro/index.html

      [19] Nandor Verba, Kuo-Ming Chao, Anne James, Daniel Goldsmith, Xiang Fei, Sergiu-Dan Stan Platform as a Service Gateway for the Fog of Things. Advanced Engineering Informatics, vol. 33, 2017, pp. 243-257.

      [20] Catalog of 98 Open-Spec, Hacker Friendly SBCs. URL: http://linuxgizmos.com/catalog-of-98-open-spec-hacker-friendly-sbcs/

      [21] Cisco IoT Networking. Deploy. Accelerate. Innovate. URL: https://www.cisco.com/c/dam/en/us/products/collateral/se/internet-of-things/brochure-c02-734481.pdf

      [22] AWS Greengrass. URL: https://aws.amazon.com/greengrass/

      [23] AWS Greengrass FAQs. URL: https://aws.amazon.com/greengrass/faqs/

      [24] P. V. Kharechkin, A. V. Savartsov. System for High-Speed Collection and Processing of the Sounding of the Earth’s Ionosphere // 3rd All-Russian Scientific Conference on the Fundamentals and Applications of Computer Technologies and Information Security, Taganrog, Russia, 2017, pp. 330-333.

      [25] K.A. Katkov, V.P. Pashintsev, E.K. Katkov, N.N. Gakhova, R.P. Gakhov, A.I. Titov. Forecast Accuracy of Determining Pseudo Range in Satellite Navigation System Through Analysis of Data from Ionosphere Monitoring. Journal of Fundamental and Applied Sciences, vol. 9, no. 1S, 2017. DOI: http://dx.doi.org/10.4314/jfas.v9i1s.744

      [26] V. P. Pashintsev, A. F. Chipiga, V. A Tsimbal, M. V. Peskov. A System for Determining Ionospheric Areas with Small-Scale Heterogeneities based on the GPS Monitoring Data // Izvestiya (News) of the Samara Scientific Center of the Russian Academy of Sciences vol. 18, no. 2(3), 2016, pp. 941-945.

  • Downloads

  • How to Cite

    V. Kopytov, V., V. Kharechki, P., V. Naumenko, V., V. Savartsov, A., V. Dorofeyev, O., & V. Batashan, M. (2018). Technology and Architecture for a System of High-Speed Sensor Data Stream Collection and Processing. International Journal of Engineering & Technology, 7(4.36), 197-208. https://doi.org/10.14419/ijet.v7i4.36.23745

    Received date: 2018-12-12

    Accepted date: 2018-12-12

    Published date: 2018-12-09