Machine Learning for the Communication Optimization in Distributed Systems
-
2018-09-12 https://doi.org/10.14419/ijet.v7i4.1.19491
-
machine learning, artificial intelligence, data communication optimization, distributed network. -
Abstract
The objective of the work that is presented in this paper was the problem of the communication optimization and detection of the issues of computing resources performance degradation [1, 2] with the usage of machine learning techniques. Computer networks transmit payload data and the meta-data from numerous sources towards vast number of destinations, especially in multi-tenant environments [3, 4]. Meta data describes the payload data and could be analyzed for anomalies detection in the communication patterns. Communication patterns depend on the payload itself and technical protocol used. The technical patterns are the research target as their analysis could spotlight the vulnerable behavior, for example: unusual traffic, extra load transported and etc.
There was a big data used to train model with a supervised machine learning. Dataset was collected from the network interfaces of the distributed application infrastructure. Machine Learning tools had been retained from the cloud services provider – Amazon Web Services. The stochastic gradient descent technique was utilized for the model training, so that it could represent the communication patterns in the system. The learning target parameter was a packet length, the regression was performed to understand the relationship between packet meta-data (timestamp, protocol, the source server) and its length. The root mean square error calculation was applied to evaluate the learning efficiency. After model was prepared using training dataset, the model was tested with the test dataset and then applied on the target dataset (dataset for prediction) to check whether it was capable to detect anomalies.
The experimental part showed the applicability of machine learning for the communication optimization in the distributed application environment. By means of the trained artificial intelligence model, it was possible to predict target parameters of traffic and computing resources usage with purpose to avoid service degradation. Additionally, one could reveal anomalies in the transferred traffic between application components. The application of techniques is envisioned in information security field and in the field of efficient network resources planning.
Further research could be in application machine learning techniques for more complicated distributed environments and enlarging the number of protocols to prepare communication patterns.
Â
Â
-
References
[1] A. F. Alam, A. Soltanian, S. Yangui, M. A. Salahuddin, R. Glitho, and H. Elbiaze. “A cloud platform-as-a-service for multimedia conferencing service provisioningâ€. In 21st IEEE Symposium on Computers and Communications (ISCC), pages 289--294. IEEE, 2016.
[2] E. Amazon. Amazon elastic compute cloud. Retrieved Feb, 10, 2009.
[3] W.-H. Bai, J.-Q. Xi, J.-X. Zhu, and S.-W. Huang. “Performance analysis of heterogeneous data centers in cloud computing using a complex queuing modelâ€. Mathematical Problems in Engineering, 2015, 2015.
[4] W. Li, L. Wu, Y. Xia, Y. Wang, K. Guo, X. Luo, M. Lin, and W. Zheng. On stochastic performance and cost-aware optimal capacity planning of unreliable infrastructure-as-a-service cloud. In Algorithms and Architectures for Parallel Processing, pages 644--657. Springer, 2016.
[5] “Amazon Machine Learningâ€, https://aws.amazon.com/machine- learning/
[6] Tim Roughgarden, Gregory Valiant, “CS168: The Modern Algorithmic Toolbox Lecture #6: Stochastic Gradient Descent and Regularizationâ€, http://theory.stanford.edu/~tim/s16/l/l6.pdf, Ðпрель 13, 2016
[7] “The M-Labâ€, https://www.measurementlab.net/data/
-
Downloads
-
How to Cite
Kazhmaganbetova, Z., Imangaliyev, S., & Sharipbay, A. (2018). Machine Learning for the Communication Optimization in Distributed Systems. International Journal of Engineering & Technology, 7(4.1), 47-50. https://doi.org/10.14419/ijet.v7i4.1.19491Received date: 2018-09-11
Accepted date: 2018-09-11
Published date: 2018-09-12