A Review on Robust Artificial Neural Network Forecasting Models towards Outliers Problem
 Abstract
 Keywords
 References

Abstract
Neurocomputing have been adjusted in time arrangement estimating field, however the nearness of exceptions that for the most part happen in information time arrangement might be hurtful to the information organize preparing. This is on the grounds that the capacity to consequently discover any examples without earlier suppositions and loss of all inclusive statement. In principle, the most wellknown preparing calculation for backpropagation calculations inclines toward lessening ordinary least squares estimator (OLS) or all the more particularly, the mean squared error (MSE). In any case, this calculation is not completely hearty when exceptions exist in preparing information, and it will prompt false estimate future esteem. In this paper, the effect of time series outliers in backpropagation training is discussed. The comparisons and related issues of autoregressive moving average (ARIMA) to artificial neural network (ANN) are also discussed briefly in this paper. Moreover, the background of the basic backpropagation neural network (BPNN) time series models; nonlinear autoregressive (NAR) and nonlinear autoregressive moving average (NARMA), is also discussed in this paper. The critical part of the paper is the application of metaheuristics algorithms, mainly the Firefly Algorithm (FFA) to improve the backpropagation models. There are also highlights of latest research works on the robustification of backpropagation using modern optimization algorithms.

Keywords
Outliers, Backpropagation, robust estimators, evolutionary algorithms, forecasting

References
[1] Z. Zhang, "Parameter estimation techniques: a tutorial with application to conic fitting",Image and Vision Computing, vol. 15, no. 1, pp. 5976, 1997.
[2] R. Pearson, "Outliers in process modeling and identification", IEEE Transactions on Control Systems Technology, vol. 10, no. 1, pp. 5563, 2002.
[3] A.Rusiecki, “Robust learning algorithm based on LTA estimator,”Neurocomputing, vol. 120, pp. 624632, 2013.
[4] M. ElMelegy, M. Essai, A. Ali, "Robust training of artificial feedforward neural networks" in Foundations of Computational Intelligence: Learning and Approximation (Studies in Computational Intelligence), USA, New York:SpringerVerlag, vol. 1, pp. 217242, 2009.
[5] Rusiecki, M. Kordos, T. Kamiński, K. Greń, "Training neural networks on noisy data" in Artificial Intelligence and Soft Computing, Berlin, Germany:SpringerVerlag, vol. 8467, pp. 131142, 2014.
[6] Abraham and B. Nath, "A neurofuzzy approach for modelling electricity demand in Victoria", Applied Soft Computing, vol. 1, no. 2, pp. 127138, 2001.
[7] D. Delen, G. Walker and A. Kadam, "Predicting breast cancer survivability: a comparison of three data mining methods", Artificial Intelligence in Medicine, vol. 34, no. 2, pp. 113127, 2005.
[8] R. Gareta, L. Romeo and A. Gil, "Forecasting of electricity prices with neural networks", Energy Conversion and Management, vol. 47, no. 1314, pp. 17701778, 2006.
[9] S. Hamid and Z. Iqbal, "Using neural networks for forecasting volatility of S&P 500 Index futures prices", Journal of Business Research, vol. 57, no. 10, pp. 11161125, 2004.
[10] S. Srinivasulu and A. Jain, "A comparative analysis of training methods for artificial neural network rainfall–runoff models", Applied Soft Computing, vol. 6, no. 3, pp. 295306, 2006.
[11] G. Zhang, M. Keil, A. Rai and J. Mann, “Predicting information technology project escalation: A neural network approach”,European Journal of Operational Research, vol. 146, pp. 115129, 2003.
[12] L. Ma and K. Khorasani, "New training strategies for constructive neural networks with application to regression problems", Neural Networks, vol. 17, no. 4, pp. 589609, 2004.
[13] H. Hippert, C. Pedreira and R. Souza, "Neural networks for shortterm load forecasting: a review and evaluation",IEEE Transactions on Power Systems, vol. 16, no. 1, pp. 4455, 2001.
[14] G. Zhang, "Business Forecasting with Artificial Neural Networks: An overview", in: G. P Zhang(Ed.), Neural Networks in Business Forecasting, Idea Group Publishing , Hershey, PA, 2004
[15] H. Allende, C. Moraga, and R. Salas, “Artificial neural networks in time Series forecasting: A comparative analysis”, Kybernetika, vol. 38, no. 6, pp. 685707, 2002.
[16] H. Allende, C. Moraga and R. Salas, "Robust Estimator for the Learning Process in Neural Networks Applied in Time Series", Artificial Neural Networks — ICANN 2002, pp. 10801086, 2002.
[17] H. Jang, E. Topal and Y. Kawamura, "Unplanned dilution and ore loss prediction in longhole stoping mines via multiple regression and artificial neural network analyses", Journal of the Southern African Institute of Mining and Metallurgy, vol. 115, no. 5, pp. 449456, 2015.
[18] A.Aliahmadi, M. JafariEskandari, A. Mozafari, and H. Nozari, “Comparing Linear Regression and Artificial Neural Networks To Forecast Total Productivity Growth In Iran”, International Journal of Information, Business and Management, vol. 8, no. 1, pp. 93113, 2016.
[19] E. Maasoumi, A. Khotanzed and A. Abaye, "Artificial neural networks for some macroeconomic series: A first report",Econometric Reviews, vol. 13, no. 1, pp. 105122, 1994.
[20] N. Kohzadi, M. Boyd, I. Kaastra, B. Kermanshahi, andD. Scuse, “Neural networks for forecasting: an introduction”, Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, vol. 43, no. 3, pp. 463474, 1995.
[21] N. Swanson and H. White, "A Model Selection Approach to RealTime Macroeconomic Forecasting Using Linear Models and Artificial Neural Networks", Review of Economics and Statistics, vol. 79, no. 4, pp. 540550, 1997.
[22] J. Hansen, J. McDonald and R. Nelson, "Time Series Prediction With GeneticAlgorithm Designed Neural Networks: An Empirical Comparison With Modern Statistical Models", Computational Intelligence, vol. 15, no. 3, pp. 171184, 1999.
[23] G. Box and D. Pierce, "Distribution of Residual Autocorrelations in AutoregressiveIntegrated Moving Average Time Series Models", Journal of the American Statistical Association, vol. 65, no. 332, pp. 15091526, 1970.
[24] J. McDonald and Y. Xu, "Some forecasting applications of partially adaptive estimators of ARIMA models", Economics Letters, vol. 45, no. 2, pp. 155160, 1994.
[25] G. Tkacz, S. Hu, "Forecasting GDP Growth Using Artificial Neural Networks", Working Paper 993 Department of Monetary and Financial Analysis Bank of Canada, 1999.
[26] G. Tkacz, "Neural network forecasting of Canadian GDP growth", International Journal of Forecasting, vol. 17, no. 1, pp. 5769, 2001.
[27] G. Zhang, "Time series forecasting using a hybrid ARIMA and neural network model", Neurocomputing, vol. 50, pp. 159175, 2003.
[28] M. Z. Mat Junoh, "Predicting GDP Growth in Malaysia Using KnowledgeBased Economy Indicators: A Comparison Between Neural Network And Econometric Approaches", Sunway College Journal, vol. 1, pp. 3950, 2004.
[29] K. Mohammadi, H. Eslami, and S. Dardashti, “Comparison of Regression, ARIMA & ANN Models for Reservoir Inflow Forecasting using Snowmelt Equivalent (a case study of Karaj)”,Journal of Agriculture, Science & Technology, vol. 7, no. 17, 2005.
[30] G. Rutka, "Network Traffic Prediction using ARIMA and Neural Networks Models", Electronics and Electrical Engineeging, vol. 4, pp. 4752, 2008.
[31] G. Cybenko, "Approximation by superpositions of sigmoidal functions", Math. Control Signals Syst., vol. 2, no. 4, pp. 303314, 1989.
[32] K. Funahashi, "On the approximate realization of continuous mappings by neural networks", Neural Networks, vol. 2, no. 3, pp. 183192, 1989.
[33] D. White and D. Sofge, “Handbook of Intelligent Control: Neural, Fuzzy, and Adaptative Approaches”, Van Nostrand Reinhold Company, 1992.
[34] H. White and M. Stinchcombe,“Approximating and learning unknown mappings using multilayer feedward networks with bounded weights”, In Arcial neural networks: approximations and learning theory, H. White ed., Blackwell, Oxford, UK, 1992.
[35] Gallant, H. White, "On learning the derivatives of an unknown mapping with multilayer feedforward networks", Neural Networks, vol. 5, no. 1, pp. 129138, 1992.
[36] H. White, Artificial neural networks: approximation and learning theory. Blackwell Publishers, 1992.
[37] Baccigalupi, L. Bedini, C. Burigana, G. De Zotti, A. Farusi, D. Maino, M. Maris, F. Perrotta, E. Salerno, L. Toffolatti and A. Tonazzini, "Neural networks and the separation of cosmic microwave background and astrophysical signals in sky maps", Monthly Notices of the Royal Astronomical Society, vol. 318, no. 3, pp. 769780, 2000.
[38] R. Fierro and F. Lewis, "Control of a nonholonomic mobile robot using neural networks", IEEE Transactions on Neural Networks, vol. 9, no. 4, pp. 589600, 1998.
[39] R. Lippmann, "An introduction to computing with neural nets", IEEE ASSP Magazine, vol. 4, no. 2, pp. 422, 1987.
[40] M. Khashei and M. Bijari, "An artificial neural network (p,d,q) model for timeseries forecasting", Expert Systems with Applications, vol. 37, no. 1, pp. 479489, 2010.
[41] M. Khashei and M. Bijari, "A novel hybridization of artificial neural networks and ARIMA models for time series forecasting", Applied Soft Computing, vol. 11, no. 2, pp. 26642675, 2011.
[42] Specht, "A general regression neural network", IEEE Transactions on Neural Networks, vol. 2, no. 6, pp. 568576, 1991.
[43] Alon, M. Qi, R. J. Sadowsik, "Forecasting aggregate retail sales: A comparison of artificial neural networks and traditional methods", J. Retailing Consumer Services, vol. 8, no. 3, pp. 147156, 2001.
[44] M. Thompson and M. Kramer, “Modeling chemical processes using prior knowledge and neural networks”, AIChE Journal, vol. 40, no. 8, pp. 13281340, 1994.
[45] KTam and M. Kiang, “Managerial applications of neural networks: the case of bank failure predictions”, Management science, vol. 38, no. 7, pp. 926947, 1992.
[46] G. Spellman, “An application of artificial neural networks to the prediction of surface ozone concentrations in the United Kingdom”, Applied Geography, vol. 19, no. 2, pp. 123136, 1999.
[47] Chau, “Particle swarm optimization training algorithm for ANNs in stage prediction of Shing Mun River”, Journal of hydrology, vol. 329, no. 3, pp. 363367, 2006.
[48] Chau, “Application of a PSObased neural network in analysis of outcomes of construction claims”, Automation in Construction, vol. 16, no.5, pp. 642646, 2007.
[49] StorrieLombardi, O. Lahav, L. Sodre and L. StorrieLombardi, “Morphological classification of galaxies by artificial neural networks”, Monthly Notices of the Royal Astronomical Society, vol. 259, no. 1, pp. 8P12P, 1992.
[50] G. Foody, "Using prior knowledge in artificial neural network classification with a minimal training set", International Journal of Remote Sensing, vol. 16, no. 2, pp. 301312, 1995.
[51] Guresen, G. Kayakutlu and T. Daim, "Using artificial neural network models in stock market index prediction", Expert Systems with Applications, vol. 38, no. 8, pp. 1038910397, 2011.
[52] P. Atkinson and A. Tatnall, "Introduction Neural networks in remote sensing",International Journal of Remote Sensing, vol. 18, no. 4, pp. 699709, 1997.
[53] G. Zhang, M. Y. Hu, B. Eddy Patuwo and D. C. Indro, "Artificial neural networks in bankruptcy prediction: General framework and crossvalidation analysis", European Journal of Operational Research, vol. 116, no. 1, pp. 1632, 1998.
[54] K. Fanning and K. Cogger, "A Comparative Analysis of Artificial Neural Networks Using Financial Distress Prediction", Intelligent Systems in Accounting, Finance and Management, vol. 3, no. 4, pp. 241252, 1994.
[55] O. Nelles, “Nonlinear system identification: from classical approaches to neural networks and fuzzy models”. Springer Science & Business Media, 2001.
[56] Zhang, Artificial higher order neural networks for economics and business. Hershey, IGI Global (701 E. Chocolate Avenue, Hershey, Pennsylvania, 17033, USA), 2008.
[57] Boely and R. Botez, "New Approach for the Identification and Validation of a Nonlinear F/A18 Model by Use of Neural Networks", IEEE Transactions on Neural Networks, vol. 21, no. 11, pp. 17591765, 2010.
[58] J. Michalkiewicz, "Modified Kolmogorov's Neural Network in the Identification of Hammerstein and Wiener Systems", IEEE Transactions on Neural Networks and Learning Systems, vol. 23, no. 4, pp. 657662, 2012.
[59] G. Wen and Y. Liu, "Adaptive fuzzyneural tracking control for uncertain nonlinear discretetime systems in the NARMAX form", Nonlinear Dynamics, vol. 66, no. 4, pp. 745753, 2011.
[60] M. Shafiq and N. Butt, "Utilizing higherorder neural networks in Umodel based controllers for stable nonlinear plants", International Journal of Control, Automation and Systems, vol. 9, no. 3, pp. 489496, 2011.
[61] Z. Chen and Y. Ni, "Onboard Identification and Control Performance Verification of an MR Damper Incorporated with Structure", Journal of Intelligent Material Systems and Structures, vol. 22, no. 14, pp. 15511565, 2011.
[62] M. GonzalezOlvera and Y. Tang, "BlackBox Identification of a Class of Nonlinear Systems by a Recurrent Neurofuzzy Network", IEEE Transactions on Neural Networks, vol. 21, no. 4, pp. 672679, 2010.
[63] L. YuMin, T. DaZhen, X. Hao and T. Shu, “Productivity matching and quantitative prediction of coalbed methane wells based on BP neural network”,Science China Technological Sciences, vol. 54, no. 5, pp. 12811286, 2011.
[64] M. Han, J. Fan and J. Wang, "A Dynamic Feedforward Neural Network Based on Gaussian Particle Swarm Optimization and its Application for Predictive Control", IEEE Transactions on Neural Networks, vol. 22, no. 9, pp. 14571468, 2011.
[65] X. Zongyi, Q. Yong, P. Xuemiao, J. Limin and Z. Yuan, "Modelling of the Automatic Depth Control Electrohydraulic System Using RBF Neural Network and Genetic Algorithm", Mathematical Problems in Engineering, vol. 2010, pp. 116, 2010.
[66] GómezGil, J. RamírezCortes, S. Pomares Hernández and V. AlarcónAquino, "A Neural Network Scheme for LongTerm Forecasting of Chaotic Time Series", Neural Processing Letters, vol. 33, no. 3, pp. 215233, 2011.
[67] X. ZhiBin, “Particleswarmoptimizationbased selective neural network ensemble & its application to modeling resonant frequency of microstrip antenna”.inMicrostrip Antennas, pp. 6982, 2011.
[68] Z. Chen and Y. Ni, “Onboard Identification & Control Performance Verification of an MR Damper Incorporated with Structure”,Journal of Intelligent Material Systems & Structures, vol. 22, pp. 15511565, 2011.
[69] B. Cosenza, “Development of a neural network for glucose concentration prevision in patients affected by type 1 diabetes”,Bioprocess & Biosystems Engineering, pp. 19, 2012.
[70] S. Chatterjee, S. Nigam, J. Singh and L. Upadhyaya, "Software fault prediction using Nonlinear Autoregressive with eXogenous Inputs (NARX) network", Applied Intelligence, vol. 37, no. 1, pp. 121129, 2012.
[71] M. Rashid, M. Frasca, A. Ali, R. Ali, L. Fortuna and M. Xibilia, "Nonlinear model identification for Artemia population motion", Nonlinear Dynamics, vol. 69, no. 4, pp. 22372243, 2012.
[72] S. Geman, E. Bienenstock and R. Doursat, "Neural Networks and the Bias/Variance Dilemma", Neural Computation, vol. 4, no. 1, pp. 158, 1992.
[73] C. Granger, “Strategies for modeling nonlinear timeseries relationships”,The Economic Record, vol. 69, no. 206, pp. 233238, 1993.
[74] R. Hoptroff, "The principles and practice of time series forecasting and business modelling using neural nets", Neural Computing & Applications, vol. 1, no. 1, pp. 5966, 1993.
[75] R. Ashley, "Statistically significant forecasting improvements: how much outofsample data is likely necessary?", International Journal of Forecasting, vol. 19, no. 2, pp. 229239, 2003.
[76] Azoff,“Neural Network Time Series Forecasting of Financial Markets”. Chichester, New York, Wiley, 1994.
[77] W. Gorr, D. Nagin and J. Szczypula, "Comparative study of artificial neural network and statistical models for predicting student grade point averages", International Journal of Forecasting, vol. 10, no. 1, pp. 1734, 1994.
[78] Komarek andA. Moore, “Making logistic regression a core data mining tool with trirls”,IEEE International Conference on Data Mining (ICDM'05), pp. 410, 2005.
[79] Zhang and M. Qi,“Predicting consumer retail sales using neural networks”, In Smith & Gupta (Eds.), Neural Networks in Business: Techniques and Applications, Hershey, PA: Idea Group Publishing, pp. 2640, 2002.
[80] Kaastra and M. Boyd, "Designing a neural network for forecasting financial and economic time series", Neurocomputing, vol. 10, no. 3, pp. 215236, 1996.
[81] Zhang and M. Hu, "Neural network forecasting of the British Pound/US Dollar exchange rate", Omega, vol. 26, no. 4, pp. 495506, 1998.
[82] Coakley and C. Brown, "Artificial neural networks in accounting and finance: modeling issues", International Journal of Intelligent Systems in Accounting, Finance & Management, vol. 9, no. 2, pp. 119144, 1999.
[83] W. Remus and M. O’Connor, “Neural networks for time series forecasting”,in Armstrong JS (ed) Principles of forecasting: a hand book for researchers & practitioners. Kluwer, Norwell, MA, pp 245–256, 2001.
[84] Y. Pan, T. Pohlen andS. Manago, “Hybrid neural network model in forecasting aggregate US retail sales”, inAdvances in business & management forecasting, vol. 9, 153170, 2013.
[85] S. Anbazhagan and N. Kumarappan, “Dayahead deregulated electricity market price forecasting using recurrent neural network”,Systems Journal, IEEE, vol. 7, no. 4, pp. 866872, 2013.
[86] Azad, S. Mekhilef and V. Ganapathy, "LongTerm Wind Speed Forecasting and General Pattern Recognition Using Neural Networks", IEEE Transactions on Sustainable Energy, vol. 5, no. 2, pp. 546553, 2014.
[87] Heydari and F. Keynia, “Prediction of Wind Power Generation Through Combining Particle Swarm Optimization & Elman Neural Network (ElPSO)”,International Energy Journal, vol. 15, no. 2, 2015.
[88] Chatfield,“TimeSeries Forecasting”. Boca Raton, Chapman & Hall/ CRC, FL, 2001.
[89] Lennon, G. Montague, A. Frith, C. Gent and V. Bevan, “Industrial applications of neural networksAn investigation”,Journal of Process Control, vol. 11, pp. 497507, 2001.
[90] G. Zhang, B. Patuwo and M. Hu, “A simulation study of artificial neural networks for nonlinear timeseries forecasting”,Computers & Operations Research, vol. 28, no. 4, pp. 381396, 2001.
[91] G. Zhang, “An investigation of neural networks for linear timeseries forecasting”,Computers & Operations Research, vol. 28, pp. 11831202, 2001.
[92] S. Arlot andA. Calisse, “A survey of crossvalidation procedures for model selection”, Statistic Surveys, vol. 4, pp. 4079, 2010.
[93] Bakirtzis, V. Petridis, S. Kiartzis, M. Alexiadis and A. Maissis, “A neural network short term load forecasting model for the Greek power system”, IEEE Transactions on Power Systems, vol. 11, no. 2, pp. 858863, 1996.
[94] Khotanzad, R. AfkhamiRohani, T. Lu, A. Abaye, M. Davis and D. Maratukulam, “ANNSTLFA neuralnetworkbased electric load forecasting system”, IEEE Transactions on Neural Networks, vol. 8 no. 4, pp. 835846, 1997.
[95] Bishop, “Neural networks for pattern recognition”, Oxford university press, 1995.
[96] S. Abid, F. Fnaiech and M. Najim, “A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm”, IEEE Transactions on Neural Networks, vol. 12, no. 2, pp. 424430, 2001.
[97] M. Hagan and M. Menhaj, “Training feedforward networks with the Marquardt algorithm”, IEEE Transactions on Neural Networks, vol. 5, no. 6, pp. 989993, 1994.
[98] T. Kwok and D. Yeung, “Objective functions for training new hidden units in constructive neural networks”,IEEE Transactions on Neural Networks, vol. 8, no. 5, pp. 11311148, 1997.
[99] Y. Yamamoto andP. Nikiforuk, “A new supervised learning algorithm for multilayered and interconnected neural networks”IEEE Transactions on Neural Networks, Vol. 11, pp. 36–46, 2000.
[100] Yam and T. Chow,“Extended least squares based algorithm for training feedforward networks”, IEEE Transactions on Neural Networks, Vol. 8, pp. 806–810, 1997.
[101] Yam and T. Chow, “A weight initialization method for improving training speed in feedforward neural networks”, Neurocomputing, Vol. 30, pp. 219–232, 2000.
[102] Yam and T. Chow, “Feedforward networks training speed enhancement by optimal initialization of the synaptic coefficients”,IEEE Transactions on Neural Networks, Vol. 12, pp. 430–434, 2001.
[103] X. Yu, M. Efe and O. Kaynak, “A general backpropagation algorithm for feedforward neural networks learning”, IEEE Transactions on Neural Networks, vol. 13, no. 1, pp. 251254, 2002.
[104] T. Kathirvalavakumar and P. Thangavel, “A Modified Backpropagation Training Algorithm for Feedforward Neural Networks*”,Neural Processing Letters, vol. 23, no. 2, pp. 111119, 2006.
[105] T. Williams, “Predicting changes in construction cost indexes using neural networks”, Journal of Construction Engineering and Management, vol. 120, no. 2, pp. 306–320, 1994.
[106] V. Asari, “Training of a feedforward multiplevalued neural networks by error backpropagation with a multilevel threshold function”,IEEE Transactions on Neural Networks, vol. 12, pp. 1519–1521, 2001.
[107] Chowdhury, Y. Singh and R. Chansarkar, “Dynamic tunneling technique for efficient training of multilayer perceptrons”,IEEE. Transactions on Neural Networks, vol. 10, pp. 48–55, 1999.
[108] G. Hinton, “Connectioniest Learning Procedure in Machine Learning: Paradigms & Methods”, In: J. G. Carbonell, (ed.), pp. 185–234, Cambridge MA: MIT press, 1989.
[109] T. Kathirvalavakumar andP. Thangavel, “A new learning algorithm using simultaneous perturbation with weight initialization”,Neural Processing Letters, vol. 17, pp. 55–68, 2003.
[110] Krogh and J. Hertz, “Generalization in a linear perceptron in the presence of noise”,Journal of Physics A – Mathematical & General, vol. 25, pp. 1135–1147, 1992.
[111] G. Lera andM. Pinzolas, “Neighborhood based Levenberg–Marquardt algorithm for neural network training”IEEE Transactions on Neural Networks, vol. 13, pp. 1200–1203, 2002.
[112] Parisi, E. Di Claudio, G. Orlandi and B. Rao, "A generalized learning paradigm exploiting the structure of feedforward neural networks", IEEE Transactions on Neural Networks, vol. 7, no. 6, pp. 14501460, 1996.
[113] Reed andR. Marks, “Neural Smithing Supervised Learning in Feedforward Artificial Neural Networks”, Cambridge: MIT, 1999.
[114] P. Thangavel and T. Kathirvalavakumar, “A new learning algorithm using simultaneous perturbation with weight initialization”, Neural processing letters, 17(1), 5568, 2003.
[115] P. Thangavel and T. Kathirvalavakumar, “Training feedforward networks using simultaneous perturbation with dynamic tunneling”, Neurocomputing, 48(1), 691704, 2002.
[116] Bryson, “Applied optimal control: optimization, estimation & control”,CRC Press, 1975.
[117] E. Vamsidhar, K. Varma, P. Rao and R. Satapati, “Prediction of rainfall using backpropagation neural network model”,International Journal on Computer Science & Engineering, vol. 2, no. 4, pp. 11191121,2010.
[118] Duy, Y. Sato and Y. Inoguchi, “Improving accuracy of host load predictions on computational grids by artificial neural networks”, International Journal of Parallel, Emergent and Distributed Systems, vol. 26, no. 4, pp. 275290, 2011.
[119] S. Ubaidillah, R. Sallehuddin and N. Ali, “Cancer Detection Using Aritifical Neural Network & Support Vector Machine: A Comparative Study”,JurnalTeknologi, vol. 65, no. 1, 2013.
[120] Elmi, S. Ibrahim andR. Sallehuddin, “Detecting sim box fraud using neural network”IT Convergence and Security 2012, pp. 575582, 2013.
[121] R. Alwee, S. Hj Shamsuddin and R. Sallehuddin, “Hybrid support vector regression and autoregressive integrated moving average models improved by particle swarm optimization for property crime rates forecasting with economic indicators”,The Scientific World Journal, 2013.
[122] H. Maier and G. Dandy, “Neural network models for forecasting univariate time series”, Neural Network World, vol. 5, pp. 747772, 1996.
[123] O. Valenzuela, I. Rojas, F. Rojas, H. Pomares, L. Herrera, A. Guillén and M. Pasadas, “Hybridization of intelligent techniques and ARIMA models for time series prediction”,Fuzzy Sets and Systems, vol. 159, no. 7, pp. 821845, 2008.
[124] Khashei, M. Bijari and G. Raissi Ardali,“Improvement of autoregressive integrated moving average models using fuzzy logic & artificial neural networks (ANNs)”,Neurocomputing, vol. 72, no. 4, pp. 956967, 2009.
[125] Weruaga, “Multilayer Neural Networks”, Beamer presentation, 2006.
[126] Bylander, “Universal Approximation”, 2003,http://www.cs.utsa.edu/~bylander/cs4793 /univapprox.pdf
[127] Bonnell, “ Implementation of a New Sigmoid Function in Backpropagation Neural Networks” (Doctoral dissertation, East Tennessee State University), 2011.
[128] T. Windeatt and C. Zor, “Minimising added classification error using Walsh coefficients”,IEEE Transactions on Neural Networks, vol. 22, no. 8, pp. 13341339,2011.
[129] Panklib, C. Prakasvudhisarn and D. Khummongkol, “Electricity Consumption Forecasting in Thail & Using an Artificial Neural Network & Multiple Linear Regression”,Energy Sources, Part B: Economics, Planning, & Policy, vol. 10, no. 4, pp. 427434, 2015.
[130] S. Khan, T. Ayub andS. Rafeeqi, “Prediction of compressive strength of plain concrete confined with ferrocement using artificial neural network (ANN) & comparison with existing mathematical models”, American Journal of Civil Engineering & Architecture, vol. 1, no. 1, pp. 714, 2013.
[131] T. Ayub, M. Nurrudin, S. Khan and F. Memon, “Prediction of Compressive Strength of Plain Concrete Confined with Ferrocement using Artificial Neural Network (ANN)”Special issues of the International Journal of Soft Computing & Software Engineering [JSCSE], vol. 3, no. 3, pp. 663667, 2013.
[132] Maia, F. de Carvalho and T. Ludermir, “Forecasting models for intervalvalued time series”,Neurocomputing, vol. 71, no. 16, pp. 33443352, 2008.
[133] Wang and Y. Mei, “Model for forecasting construction cost indices in Taiwan”Construction Management & Economics,vol. 16, no. 2, pp. 147157, 1998.
[134] P. Hartono and S. Hashimoto, “Learning from imperfect data”,Applied Soft Computing, vol. 7, pp. 353363, 2007.
[135] Pham and D. Karaboga, “Intelligent optimization techniques: genetic algorithms, tabu search, simulated annealing & neural networks”,Springer Science & Business Media, 2012.
[136] G. Zhang, M. Y. Hu, B. E.Patuwo andD. C. Indro, “Artificial neural networks in bankruptcy prediction: General framework and crossvalidation analysis”, European journal of operational research, 116(1), 1632, 1999.
[137] T. Kim, K. Oh, C. Kim and J. Do, “Artificial neural networks for nonstationary time series”,Neurocomputing, vol. 61, pp. 439447, 2004.
[138] Cadenas and W.Rivera, “Wind speed forecasting in three different regions of Mexico, using a hybrid ARIMA–ANN model”,Renewable Energy, vol. 35, no. 12, pp. 27322738, 2010.
[139] S. Sfetsos, “A novel approach for the forecasting of mean hourly wind speed time series”,Renewable energy, Elsevier, pp. 163174, 2002.
[140] C. Wu, K. Chau and C. Fan, "Prediction of rainfall time series using modular artificial neural networks coupled with datapreprocessing techniques", Journal of Hydrology, vol. 389, no. 12, pp. 146167, 2010.
[141] Wang, H. Zou, J. Su, L. Li and S. Chaudhry,“An ARIMA‐ANN Hybrid Model for Time Series Forecasting”,Systems Research & Behavioral Science,vol. 30, no. 3, pp. 244259, 2013.
[142] Yolcu, C. Aladag, E. Egrioglu and V. Uslu, “Timeseries forecasting with a novel fuzzy timeseries approach: an example for Istanbul stock market”,Journal of Statistical Computation & Simulation, vol. 83, no. 4, pp. 599612, 2013.
[143] Claveria and S. Torra, “Forecasting tourism dem & to Catalonia: Neural networks vs. time series models”, Economic Modelling, vol. 36, pp. 220228, 2014.
[144] K. Gairaa, A. Khellaf, Y. Messlem and F. Chellali, “Estimation of the daily global solar radiation based on Box–Jenkins & ANN models: A combined approach”,Renewable & Sustainable Energy Reviews, vol. 57, pp. 238249, 2016.
[145] Ardakani and M. Ardehali, “Longterm electrical energy consumption forecasting for developing & developed economies based on different optimized models & historical data types”,Energy, vol. 65, pp. 452461, 2014.
[146] J. Pati, K. K. Shukla, "A comparison of ARIMA neural network and a hybrid technique for Debian bug number prediction", Proc. Comput. Commun. Technol., pp. 4753, 2014.
[147] R. Sharda and R. Patil, “Neural networks as forecasting experts: An empirical test” In: Proceedings of the International Joint Conference on Neural Networks. Washington, D.C., vol. 2, pp. 491494, 1990.
[148] R. Sharda and R. Patil, “Connectionist approach to time series prediction: An empirical test”,Journal of Intelligent Manufacturing, vol. 3, pp. 317323, 1992.
[149] Z. Tang, C. Almeida and P. Fishwick, “Time series forecasting using neural networks vs BoxJenkins methodology”, Simulation, vol. 57, no. 5, pp. 303–310, 1991.
[150] Z. Tang and P. Fishwick, “Feedforward neural nets as models for time series forecasting”,ORSA Journal on Computing, vol.5, no. 4, pp. 374–385, 1993.
[151] S. Kang, “An Investigation of the Use of Feedforward Neural Networks for Forecasting”, Ph.D. Thesis, Kent State University, 1991.
[152] Kohzadi, M. Boyd, B. Kermanshashi and I. Kaastra, “A comparison of artificial neural network & time series models for forecasting commodity prices”,Neurocomputing, vol. 10, pp. 169181, 1996.
[153] T. Hill, M. O’ConnorandW. Remus, “Neural Network Models for Time Series Forecasts”,Management Science, 42, 7, 10821092, 1996.
[154] T. Hill, L. Marquez, M. O'Connor and W. Remus, “Artificial neural network models for forecasting and decision making”,International Journal of Forecasting, vol. 10, no.1, pp. 515, 1994.
[155] T. Hill, M. O’Connor and W. Remus, “Neural network models for time series forecasts”Management Sciences, vol. 42, no. 7, pp. 10821092, 1996.
[156] Caire, G. Hatabian and C. Muller, “Progress in forecasting by neural networks”,Proceedings of the International Joint Conference on Neural Networks, vol. 2, pp. 540–545, 1992.
[157] T. Hann andE. Steurer, “Much ado about nothing? Exchange rate forecasting: Neural networks vs. linear models using monthly & weekly data”,Neurocomputing,vol. 10, pp. 323339, 1996.
[158] S. Balkin and J. Ord, “Automatic neural network modeling for univariate time series”International Journal of Forecasting, vol. 16, no. 4, pp. 509515, 2000.
[159] Harun, “Short term load forecasting using Artificial Neural Network (ANN) with multiple time lags, stationary time series & Principal Component Analysis (PCA)”,(Masters dissertation, Universiti Teknologi MARA), 2011.
[160] B. Barbosa, L. Aguirre, C. Martinez and Braga, “Black and GrayBox Identification of a Hydraulic Pumping System”IEEE Trans. Control Systems Technology, vol. 19, no. 2, pp. 398406, 2011.
[161] S. Ahmadi and M. Karrari, “Nonlinear identification of synchronous generators using a local model approach”Przeglad Elektrotechniczny (Electrical Review), vol. 8, pp. 166170, 2011.
[162] L. Teslic, B. Hartmann, O. Nelles and I. Skrjanc, “Nonlinear system identification by gustafson–kessel fuzzy clustering and supervised local model network learning for the drug absorption spectra process” IEEE Transactions on Neural Networks, vol. 22, no. 12, pp. 19411951, 2011.
[163] C. Safak, V. Topuz and A. Baba, “Pneumatic motor speed control by trajectory tracking fuzzy logic controller”Sadhana, vol. 35, no. 1, pp. 7586, 2010.
[164] K. Tanabe, B. Lučić, D. Amić, T. Kurita, M. Kaihara, N. Onodera and T. Suzuki, “Prediction of carcinogenicity for diverse chemicals based on substructure grouping & SVM modeling”,Molecular diversity, vol. 14, no. 4, pp. 789802, 2010.
[165] K. Tanabe, B. Lučić, D. Amić, T. Kurita, M. Kaihara, N. Onodera and T. Suzuki, “Prediction of carcinogenicity for diverse chemicals based on substructure grouping & SVM modeling”, Molecular diversity, vol. 14, no. 4, pp. 789802, 2010.
[166] T. Tsai, C. Lee and C. Wei,“Neural network based temporal feature models for shortterm railway passenger demand forecasting”Expert Systems with Applications, vol. 36, no. 2, pp. 37283736, 2009.
[167] M. Cogollo and J. Velasquez, “Methodological advances in artificial neural networks for time series forecasting”,IEEE Latin America Transactions, vol. 12, no. 4, 764771,2014.
[168] Yassin, A. Zabidi, M. Salleh and N. Khalid, “Malaysian tourism interest forecasting using Nonlinear AutoRegressive (NAR) model”, 2013 IEEE 3rd International Conference on System Engineering & Technology (ICSET), pp. 3236, 2013.
[169] Pawlus, H. Karimi and K. Robbersmyr, “Databased modeling of vehicle collisions by nonlinear autoregressive model & feedforward neural network”,Information Sciences, vol. 235, pp. 6579, 2013.
[170] H. Chen and X. Liu, “Melt Index Prediction Based on Two Compensation by Compound Basis Function Neural Network & Hidden Markov Model”, 2014.
[171] Saberian, H. Hizam, M. Radzi, M. Ab Kadir and M. Mirzaei, “Modelling and prediction of photovoltaic power output using artificial neural networks”International Journal of Photoenergy, pp. 111,2014.
[172] S. Kamaruddin, N. Ghani and N. Ramli, “Modified nonlinear neural network forecasting models on Malaysian s & costs indices”, Proceedings of 2015 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), pp. 191196, 2015.
[173] Acuña, C. Ramirez and M. Curilem,“Comparing NARX and NARMAX models using ANN and SVM for cash demand forecasting for ATM”The IEEE 2012 International Joint Conference on Neural Networks (IJCNN), pp. 16, 2012.
[174] Goyal and D. Kaur, “A Novel Vehicle Classification Model for Urban Traffic Surveillance Using the Deep Neural Network Model”, I.J. Education & Management Engineering, vol.1, 1831,2016.
[175] M. Rahiman, M. Taib and Y. Salleh, “Selection of training data for modeling essential oil extraction system using NNARX structure”,In 2007 Int. Conf. Control, Automation & Systems (COEX 2007), Seoul, Korea, pp. 16911695, 2007.
[176] Chiras, C. Evans and D. Rees, “Nonlinear gas turbine modeling using NARMAX structures”,IEEE Trans. Instrumentation and Measurement, vol. 50, no. 4, pp. 893898, 2001.
[177] K. Ahn and H. P. H. Anh, “Inverse Double NARX Fuzzy Modeling for System Identification”,IEEE/ ASME Transactions on Mechatronics, 15(1), 136148, 2010.
[178] N. Kadir, N. Md Tahir, M. assin and A. Zabidi, “Malaysian tourism interest forecasting using Nonlinear AutoRegressive Moving Average (NARMA) model”,2014 IEEE Symposium on Wireless Technology & Applications (ISWTA), pp. 193198, 2014.
[179] Dimogianopoulos, J. Hios and S. Fassois, “FDI for Aircraft Systems Using Stochastic PooledNARMAX Representations, Design and Assessment”,IEEE Trans. Control Systems Technology, vol. 17, no.6, pp. 13851397, 2009.
[180] Hornik, M. Stinchcombe and H. White, “Multilayer feedforward networks are universal approximators”, Neural networks, vol. 2, no.5, pp. 359366, 1989.
[181] F. Hampel, E. Ronchetti, P. Rousseeuw and W. Stahel, “Robust Statistics, The approach based on influence functions”,Wiley, New York, 1986.
[182] C. Chen and G. Yin, “Computing the Efficiency & Tuning Constants for MEstimation”, In Proceedings of the 2002 Joint Statistical Meetings, vol. 478, p. 482, 2002.
[183] J. William, “ Introduction to Robust & QuasiRobust Statistical Methods”, Springer, Berlin, Heidelberg, 1983.
[184] Huber, “Robust Statistics”, John Wiley & Sons, New York, 1981.
[185] Farcomeni and L. Ventura, “An overview of robust methods in medical research”Statistical Methods in Medical Research,vol. 21, no. 2, pp. 111133, 2012.
[186] D. Adedia, Comparison of robust regression estimators. Unpublished MPhil thesis, KNUST, Kumasi, 2014.
[187] Kantar and V. Yildirim, “Robust Estimation for Parameters of the Extended Burr Type III Distribution”Communications in StatisticsSimulation and Computation, vol. 44, no. 7, pp. 19011930, 2015.
[188] Kordos and A. Rusiecki, “Reducing noise impact on MLP training”, Soft Computing, pp. 117, 2015.
[189] Kordos and A. Rusiecki, “Improving MLP neural network performance by noise reduction. In Theory & Practice of Natural Computing”, Springer Berlin Heidelberg, pp. 133144, 2013.
[190] Rusiecki, “Robust Learning Algorithm Based on Iterative Least Median of Squares”,Neural Process Lett, vol. 36, pp. 145160, 2012.
[191] S. Kamaruddin, N. Ghani and N. Ramli, “Enhancing Backpropagation of ANNNAR & ANNNARMA using Robust Estimators with Application on Real Industrial Data”,Proceedings of The 3rd International Conference on Computer Science & Computational Mathematics (ICCSCM 2014), vol. 3, pp. 256266, 2014.
[192] Bruna, “Short term load forecasting using nonlinear models”Master Thesis Report, Measurement & Control Group, Faculty of Electrical Engineering, Eindhoven University of Technology, 1994.
[193] Yang, “Engineering Optimization: An Introduction with Metaheuristic Applications”John Wiley and Sons, USA, 2010.
[194] X. Yang, “A New Metaheuristic Batinspired algorithm”,Nature Inspired Cooperative Strategies for Optimization, 284, pp. 6574, 2010.
[195] X. Yang, “NatureInspired Metaheuristic Algorithms” Luniver Press, UK, 2008.
[196] C. Blum, M. Aguilera, A. Roli and M.Sampels, “Hybrid Metaheuristics, An Emerging Approach to Optimization”, Springer, 2008.
[197] S. Pal, C. Rai and P. Amrit, “Comparative study of Firefly Algorithm & Particle Swarm Optimization for Noisy Nonlinear Optimization Problems”,International Journal of Intelligent Systems & Applications, vol. 10, pp. 5057, 2012.
[198] M. Yuanbin, “Particle Swarm Optimization for Cylinder Helical GearMultiobjective Design problems”,Applied Mechanics and Materials, vol. 109, pp. 216221, 2012.
[199] X. Yang, and X. He, “Firefly Algorithm: Recent Advances and Applications”, International Journal Swarm Intelligence, vol. 1, no.1, pp. 3650, 2013.
[200] Kennedy and R. Eberhart, “A discrete binary version of the particle swarm algorithm”,Systems, Man, and Cybernetics, vol. 5, pp. 41044108, 1997.
[201] X. Yang, “Firefly algorithms for multimodal optimization”, InStochastic algorithms: foundations and applications, pp. 169178, 2009.
[202] X. Yang and S. Deb, “Cuckoo Search via Levy Flights”,Proceedings of World Congress on Nature & Biologically Inspired Computing, pp. 210214, 2009.
[203] Bansal and K. Deep, “Optimization of directional overcurrent relay times by particle swarm optimization”,Swarm Intelligence Symposium, pp. 17, 2008.
[204] C. Floudas and P. Pardalos,“State of the art in global optimization: computational methods and applications”, Springer Science & Business Media, vol. 7, 2013.
[205] Gandomi, X. Yang and A. Alavi, “Mixed variable structural optimization using firefly algorithm”Computers & Structures, vol. 89, no. 23, pp. 23252336,2011.
[206] Parpinelli and H. Lopes, “New inspirations in swarm intelligence: a survey”,International Journal of BioInspired Computation, vol. 3, no. 1, pp. 116, 2011.
[207] X. Yang, “Multiobjective firefly algorithm for continuous optimization”,Engineering with Computers, vol. 29, no.2, pp. 175184, 2013.
[208] M. Spasos, K. Tsiakmakis, N. Charalampidis and R. Nilavalan, “RFMEMS switch actuation pulse optimization using Taguchi’s method”,Microsystem Technologies, vol. 17, pp. 13511359, 2011.
[209] F. Ducatelle, G. Caro and L. Gambardella, “Principles and applications of swarm intelligence for adaptive routing in telecommunications networks”,Swarm Intelligence, vol. 4, pp. 173198, 2010.
[210] M. Arjona, M.CisnerosGonzález and C. Hernández, “Parameter Estimation of a Synchronous Generator Using a Sine Cardinal Perturbation and Mixed Stochastic–Deterministic Algorithms”,IEEE Trans. Industrial Electronics, vol. 58, no. 2, pp. 486493, 2011.
[211] C. Liao, “Enhanced RBF Network for Recognizing NoiseRiding Power Quality Events”,IEEE Trans. Instrumentation & Measurement, vol. 59, no.6, pp. 15501561, 2010.
[212] M. Wan, L. Li, J. Xiao, C. Wang and Y. Yang, “Data clustering using bacterial foraging optimization”,Journal of Intelligent Information Systems, vol. 38, no.2, pp. 321341, 2012.
[213] D. Karaboga and B. Basturk, “A Powerful and Efficient Algorithm for Numerical Function Optimization: Artificial Bee Colony (ABC) Algorithm”,Journal of Global Optimization, vol. 39, no. 3, pp. 459471, 2007.
[214] Brajevic, and M. Tuba, “An upgraded artificial bee colony (ABC) algorithm for constrained optimization problems”,Journal of Intelligent Manufacturing, vol. 10 pp. 112, 2012.
[215] W. Yunna and S. Zhaomin, “Application of Radial Basis Function Neural Network Based on Ant Colony Algorithm in Credit Evaluation of Real Estate Enterprises”,Proc. 2008 Int. Conf. on Management Science and Engineering, pp. 13221329, 2008.
[216] Udhayakumar and S. Kumanan, “Sequencing & scheduling of job & tool in a flexible manufacturing system using ant colony optimization algorithm”,International Journal of Advanced Manufacturing Technology, vol. 50, pp. 10751084, 2010.
[217] M. AlMeshah, “Firefly algorithms with probabilistic neural network for classification problems”, PhD Thesis. Malaysian National University, 2013.
[218] X. S. Yang and N. I. M. Algorithms, Luniver Press. Beckington, UK, 242246, 2008.
[219] Zigiaris, “Business process reengineering BPR”,Report produced for the EC funded project, InnoRegio Project, BPR Hellas SA, January, 2000.
[220] M. Basu, “A simulated annealingbased goalattainment method for economic emission load dispatch of fixed head hydrothermal power systems”,International Journal of Electrical Power & Energy Systems, vol. 27, no. 2, pp. 147153, 2005.
[221] Chen and C. Huang, “Biobjective power dispatch using goalattainment method & adaptive polynomial networks”,IEEE Transactions on Energy Conversion, vol. 19, no. 4, pp. 741–747, 2004.
[222] Bouktir, R. Labdani and L. Slimani, “Economic power dispatch of power system with pollution control using multiobjective particle swarm optimization”,Journal of Pure & Applied Sciences, vol. 4, no. 2, pp. 57–77, 2007.
[223] ElWahed, A. Mousa, and M. Elsisy, “Solving economic emissions load dispatch problem by using hybrid ACOMSM approach”, The Online Journal on Power and Energy Engineering, vol. 1, no. 1, pp.3653, 2007.
[224] Xie, S. Wang and Z. Wu, “Study on economic, rapid and environmental power dispatch based on fuzzy multiobjective optimization”,Modern Applied Science, vol. 3, no. 6, pp. 3844, 2009.
[225] Alsumait and J. Sykulski, “Solving economic dispatch problem using hybrid GAPSSQP method”. Proceedings of the International Conference on Computer as a Tool (EUROCON ’09), pp. 351– 356, 2009.
[226] Sudhakaran, S. Slochanal, R. Sreeram and N. Chandrasekhar, “Application of refined genetic algorithm to combined economic & emission dispatch”, Journal of the Institution of Engineers, vol. 85, no. 2, pp. 115–119, 2004.
[227] Goncalves, C. Almeida, J. Kuk, and M. Delgado, “Solving economic load dispatch problem by natural computing intelligent systems”, in Proceedings of the 15th International Conference on Intelligent System Applications to Power Systems (ISAP ’09), vol. 1, pp.257264, 2009.
[228] E. Osaba, X. Yang, F. Diaz, E. Onieva, A. Masegosa and A. Perallos, “A discrete firefly algorithm to solve a rich vehicle routing problem modelling a newspaper distribution system with recycling policy”, Soft Computing, pp. 114, 2016.
[229] Alb, P. Alotto, C. Magele, W. Renhart, K. Preis and B. Trapp, “Firefly Algorithm for Finding Optimal Shapes of Electromagnetic Devices”, IEEE Transactions on Magnetics, vol. 52, no. 3, pp. 14, 2016.
[230] Gálvez and A. Iglesias, “New memetic selfadaptive firefly algorithm for continuous optimization”,International Journal of BioInspired Computation, vol. 8, no.5, pp. 300317, 2016.
[231] Łukasik and S. Żak, “Firefly algorithm for continuous constrained optimization tasks. In Computational Collective Intelligence”, Semantic Web, Social Networks & Multiagent Systems. Springer Berlin Heidelberg, pp. 97106, 2009.
[232] Saadi, M. Nor Azura and M. Norazan, “The Superiority of Evolutionary Algorithms to Robustify Backpropagation Neural Network Learning Algorithm”,Journal of Applied Environmental and Biological Sciences, vol. 6, no. 25, pp. 1731, 2016.
[233] M. M.Basri, N. M. Nawi, M. Mamat and N. A. Hamid, “A New Class of Nonlinear Conjugate Gradient for Training Back Propagation Algorithm”, In International Conference on Soft Computing and Data Mining, pp. 200212, 2018.
[234] R. Heravi and G. A. Hodtani, “ A New CorrentropyBased Conjugate Gradient Backpropagation Algorithm for Improving Training in Neural Networks”, IEEE Transactions on Neural Networks and Learning Systems, 2018.
[235] Hubara, E. Hoffer and D. Soudry, “Quantized BackPropagation: Training Binarized Neural Networks with Quantized Gradients”, (2018).
[236] F. Bordes, T. Berthier, L. Di Jorio, P. Vincent and Y. Bengio, “Iteratively unveiling new regions of interest in Deep Learning models”, 2018.
[237] Aljarah, H. Faris and S. Mirjalili, “ Optimizing connection weights in neural networks using the whale optimization algorithm”, Soft Computing, 22(1), 115, 2018.

View 
Download  Article ID: 25985 
