Gold price prediction using an evolutionary pi-sigma neural network

Gold Price Prediction has always been a fascinating area of study for researchers and decision makers who wish to determine its future value efficiently and accurately. In this study a predictor model is designed using a Pi-Sigma Neural Network (PSNN) for prediction of future gold price. Two evolutionary estimation paradigms such as Particle Swarm Optimization (PSO) and Differential Evolution (DE) are suggested during training step of the Pi-Sigma Neural Network to optimize the tunable weights of the network. The model is evaluated based on certain performance evaluation criteria such as Mean Square Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) over two dataset such as Gold/INR and Gold/AED accumulated within the same period of time. The result analysis illustrates the better prediction


Introduction
Since time immemorial, gold is termed as the oldest precious metal and for years it has been used as a global currency, an asset, a product or simply as an object of elegance. Gold is considered as a major commodity in financial and commercial market. Since the gold price varies within limited ranges thus reduces effect of inflation and so it is an investment product preferred by many investors. Therefore forecasting the gold price has become very significant for investors. An accurate gold price forecasting model is required to show the pattern of changes in gold price in future to carry out appropriate exchanges. Domain of Gold price prediction is already rich with various time series models, computational intelligence models and hybrid models. In last few years Artificial neural networks have been appeared as most promising predictor models compared to traditional time series models such as GARCH, EGARCH, GJR-GARCH, ARI-MA and soon [1][2][3]. A prediction model designed with back propagation neural network with an improved EMD online learning is proposed in [4] for gold price forecasting. Modeling and forecasting future gold price using a type-2 neuro fuzzy model has shown better performance compared to the ARIMA model in [5]. In [6] improved prediction results are observed by a feed forward network with ELM learning compared to feed forward back propagation networks, feed forward networks without feedback, radial basis function network and ELMAN network. In last decades, high order neural networks are receiving higher attention in different application domain such as function approximation, pattern recognition, classification, prediction and soon due to its exceeding computational capabilities with good learning and storage capacity than the other traditional neural networks [7,8]. This study focuses on the application of a high order neural network named as Pi-Sigma neural network (PSNN) to predict the one day ahead future gold price from the historical prices. PSNN is a sim-ple neural network, which consists of a single hidden layer of linear summation units and a product unit in the output layer. As PSNN includes adjustable weights in between input and hidden layer, where as a fixed weight of 1 in between hidden and output layer, so the network exhibits faster learning compared to other networks. In literature different meta-heuristic algorithms such as Genetic Algorithm, PSO, DE, Fire fly and soon have been proposed for training of PSNN [9][10][11][12]. In this study the parameters of the PSNN based predictor model are optimized by using the PSO and DE based learning algorithms. Historical daily gold prices of Gold/INR and Gold/AED data set, accumulated within 1st January 2015 to 13th July 2017 are taken for the empirical validation of the model. Practical analysis of results clearly suggests the PSNN with DE is showing better prediction result compared to PSO. The rest of the paper is organized as follows: Section 2 illustrates the description of Pi-Sigma Network. Section 3 explains about learning algorithm of PSNN. Section 4 discusses the experimental results corresponding to gold price predictions. Section 5 concludes the work.

PI-sigma neural network
Pi-Sigma Neural Network (PSNN) is a kind of Higher-Order Neural Network (HONN), introduced by Shin and Ghosh in 1991 which has the advantage of reduced number of adjustable weights and processing units compared to traditional MLPs. It is a special type of feed forward neural network having one input layer, a single hidden layer of summation units and product units in the output layer. Weights connected from the input layer to hidden layer are adjusted during the training process and the weights connecting from hidden layer to output layer are set to unity. Due to this reason the number of adaptable weights can startlingly diminish the intricacy of the hidden layer, for which the model can be easily executable and accelerated. Fig. 1 depicts the architecture of a PSNN. Here each xi represents the i th input of a d dimensional input vector X=[x1, x2 . . . xd] T passed through the nodes of input layer. The h1, h2. . . hd represents the nodes of the hidden layer containing summation units. The output of each summation unit is represented as a weighted sum of the input pattern with a bias value as follows: Where wij represents the weights between input i and hidden node j. Then the output of the product unit is obtained using the following equation: Finally the product of summation unit in the output layer is passed through an activation function (usually a tanh() or log sigmoid function) to produce output 'Y'. The error obtained by comparing the predicted output with actual output is used to update the weights of the PSNN structure by a weight updating algorithm during the training process.

Learning methods for pi-sigma neural network
Learning is a process of improving the performance of neural network by adjusting itself, responding to inputs for getting targeted or actual output. Training process of PSNN involves adjustment of unknown weights used in between the input and hidden layer, minimizing the error of prediction. Hence learning of PSNN can be cast as an optimization problem with a suitable error metric as an objective function. This section explains the details of PSO and DE based learning approach for PSNN.

Particle swarm optimization
Particle swarm optimization (PSO) is a meta-heuristic global optimization method based on the movement of flocks of birds and fishes [12][13][14][15]. In PSO, each individual in swarm (particle) has a position referring to a possible solution in the search space. Each particle flies over the solution space with a velocity, which is adjusted at each time step in search for the optimal solution. The position updating of a particle is dependent on its own past best position and the position of the best of its neighbors. The quality of a particle position depends on a problem specific objective function. For using PSO in the training step of PSNN, unknown weights of the network are represented as an array to form a particle. Particles are then initialized randomly and updated afterwards according to following equation: Where w, c1, c2 are inertia, cognitive and social acceleration constants respectively, r1 and r2 are random numbers within [0, 1]. pBest is the best solution of the particle achieved so far and gBest is the global best solution. Steps of PSO based PSNN learning is Step 1: Initialize the number of population (np), number of iterations (it), w, c1, c2. Initialize the particle's position and velocity.
Step 2: Find the fitness function value of each particle i.e. the error obtained by applying the weights specified in each particle in equation (2).
Step 3: Initialize the pBest and gBest position of the particles.
Step 4: Update the position and velocity of particle using equation (3) and (4).
Step 5: Update the pBest and gBest position by comparing the fitness value of new particles with previous one.
Step 6: Repeat steps 4 and 5 until some termination condition is reached, such as predefined number of iterations is reached or the error has satisfied the default precision value.
Step 7: Fixed the weight equal to the gBest position and use the network for testing.

Differential evolution
Differential Evolution (DE) is an evolutionary, stochastic, population-based optimization algorithm introduced by Storn and Price in 1996. In DE an optimal solution is explored from a randomly generated starting population by means of three evolutionary operations such as mutation, crossover and selection [14][15]. For each generation, the individuals of current population become target vectors. The mutation operation produces a mutant vector for each target vector which is further passed through the crossover operation to produce a trial vector. The trial vector replaces the target vector, if its fitness value is better than the target vector. So it can be summarized that mutation enlarges the search space, Crossover recapitulates previously successful individuals and selection encourages the survival of the fittest. DE has the advantage of reproducing the same results consistently over many trials unlike PSO which is more dependent on the randomized initialization of individuals.

Steps of DE based PSNN learning is
Step 1: Initialize the number of population (np), number of iterations (it), mutation scale (F), crossover probability (cr). Initialize the particle's position and velocity.
Step 2: Find the fitness function value of each vectore i.e. the error obtained by applying the weights specified in each vector in equation (2).
Step 3: Create a mutated individual mi for each individual target vector xi using the following formula: Where r1, r2, r3 are random and mutually exclusive integers generated in the range [1, np) Step 4: Create the trial vector by mixing the parameters of the mutant vector with those of the target vector by using the following crossover operation: Step 5: Compare the fitness value of the trial and target vector to select a vector having less fitness value in the next generation.
Step 6: Repeat steps 3 to 5 until some termination condition is reached, , such as predefined number of iterations is reached or the error has satisfied the default precision value.
Step 7: Fixed the weight equal to equal to the position of the vector having minimum fitness value and use the network for testing.

Experimental set up and result analysis
To test the predictive ability of PSNN, sample real life data comprising the daily closing prices of gold denoting price/gram in India (GOLD/INR) and United Arab Emirates Dirham (Gold/AED) during the period of 1/1/2015 to 13/8/2017 are gathered as experimental data and put forward for data pre-processing. The number of samples collected for each set is 664. Then the gold prices are normalized in the range 0 to 1 using the min max normalization. Then the input and output patterns to be used for the PSNN are prepared by a sliding window technique. In this experiment we have considered a window size of 5 and prediction horizon of 1. Accordingly an input sample consists of normalized gold price of 5 consecutive days and the corresponding output sample represents the next day normalized gold price. Total number of input output samples generated for both the data set using this approach are 659. Then first two-third of total samples i.e. 439 samples are taken for training purpose and remaining one-third i.e. 220 samples are taken for testing purpose. Next the structure of PSNN is built with 5 neurons in input layer expressing the 5dimensional input sample and 1 neuron in output layer to produce the predicted gold price for next day ahead. Simulation is done with different number of hidden layer neurons. Two learning algorithms such as DE and PSO are used for training the PSNN model, so that the network can adjust its unknown weights. This process of learning continues till the stopping criterion is met. Minimization of the mean square error (MSE) is considered as the fitness function for both the meta-heuristic learning algorithm, with the number of iterations set to 100 as the stopping criterion. The predictive ability of the model is accessed based on certain performance evaluation criteria such as Mean Square Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) described as follows: Where i gp is actual gold price, i gp  is predicted gold price, N is number of samples. The performance of the PSNN network is greatly influenced by the number of hidden layer neurons, which ultimately specifies the number of unknown weights need to be tuned and may affect the overall performance of the network. Hence simulation is done with different hidden layer size for both the data sets. The average and standard deviation of prediction errors obtained from 10 independent runs for PSNN with PSO and DE based learning algorithm with hidden layer size 2, 3, 4 and 5 is reported in Table 1 for Gold/INR data set and in Table 2  So other prediction results are observed with hidden layer size 2, which also provides less number of weights to be estimated by the learning algorithms. Figs. 2 and 3 represent the predicted gold price along with the actual price of both the data set obtained by the PSNN. The error statistics obtained over test data by using the PSNN with PSO and DE based learning is provided in Table 3.

Conclusion
In this paper, a predictor model is designed using a Pi-Sigma network for forecasting gold prices in India and United Arab Emirates Dirham using historical prices. The predictor model performance is observed with PSO and DE based two nature inspired learning algorithms. Three performance evaluation measures such as MSE, RMSE and MAE were adopted to analyze the performance of proposed predictor model. Analyzing the performance of the model with different hidden layer size, it is observed that PSNN is showing better results with DE based learning algorithm for hidden layer size 2 compared to PSO and other hidden layer sizes for both the dataset.