Learning is the fundamental capability of neural networks. Supervised learning adjusts network parameters by directly comparing the desired and actual network output. Supervised learning is a closed-loop feedback system in which the error is the feedback signal. The error measure, which indicates the difference between the output from the network and from training samples, is used to guide the learning process. The error measure is usually defined by the mean squared error (MSE) [5].

where Q is the number of pattern pairs in the sample set, t_q is the output part of the q_th pattern pair, and a_q is the network output corresponding to the pattern pair q. The MSE is calculated anew after each epoch. The learning process is terminated when MSE is sufficiently small or the failure criterion is met.

The Levenberg–Marquardt algorithm, which is a numerical optimization technique [6], was designed to minimize functions that are sums of squares of other nonlinear functions in neural network training, where the performance index is the MSE.

When the performance index is F(x) , the Levenberg–Marquardt algorithm for optimizing the performance index F(x) is represented as [7]:

Here, as the changing value of μ_K, the performance index F(x) of the network can be adjusted through the optimization algorithms with a small learning rate, where J(x_k) and V(x_k) are the matrix elements used to compute the gradient.

The complexity of a neural network is determined by the number of free parameters of weights and biases, which is determined by the number of neurons. If the network is too complex for a given dataset, then it is likely to overfit and have poor generalization [8]. The simplest method for improving generalization is early stopping. Another method is known as regularization [9]. One of two possible approaches can be used to improve the generalization capability of a neural network: restricting either the number of weights or the magnitude of weights, where the latter is called regularization.

The Bayesian regularization algorithm can be written as the sum of squares of network weights as follows:

where F(x) is the regularized performance index. Here, the regularization ratio α/β controls the effective complexity of the network solution.

The tanker freight market is characterized by interaction between many determinants of supply and demand in tanker transportation services [10]. To forecast the dynamics and fluctuations of freight rates in tanker freight markets, considerable research has been conducted using univariate or multivariate time-series analytical techniques [11] and ANN models [12].

For all data used in forecasting of oil tanker markets in this study, global oil production, world GDP, active fleets, new building prices, second-hand ship prices, demolition prices, time-charter rates, bunker prices, and crude oil prices were selected as independent variables, whereas dirty tanker earnings was selected as the dependent variable.

Thus, the aggregated data were composed of nine independent variables and one dependent variable, where each variable was based on 204 monthly observations from January 2000 to December 2016.

After the data for forecasting tanker markets were collected, the type of ANN architecture used to solve the problem of tanker market prediction, as well as the number of neurons and layers used in the network, were all determined. In ANN dynamic networks, the output depends not only on the current input to the network but also on previous inputs, outputs, and/or states of the network. Tanker markets prediction is part of an analysis that predicts the future value of a time series. Therefore, in this study, we selected dynamic networks as an appropriate ANN model to forecast dirty tanker markets. The non-linear autoregressive model with exogenous inputs (NARX networks) [5][8], which is a widely used network for applying predictions, is a recurrent dynamic network with feedback connections that encompass multiple layers of the network. This is shown in Figure 1.

Figure 1: 
NARX network (closed loop) for VLCC tanker market prediction

Until now, prediction using ANN for VLCC tanker markets has been performed using the Levenberg–Marquardt training algorithm [12]-[15]. However, no Bayesian regularization algorithm has been used to predict earnings for such markets. Therefore, this study focuses on using ANN for prediction of earnings for VLCC tanker markets and evaluates the accuracy of the ANN prediction model using the Levenberg–Marquardt and Bayesian regularization algorithms.

After we identified the network structure, the number of hidden layers in these two learning algorithms was determined to enable us to compare performance results and functions more easily.

To determine the number of neurons in the hidden layer to identify the best prediction performance for the VLCC tanker market without overfitting, we adjusted the number of neurons in the hidden layer of the ANN structure using the Levenberg-Marquardt algorithm. In addition, using the Bayesian regularization algorithm, we had the ANN perform prediction for two cases using eight and 10 neurons of the hidden layer, respectively, based on nine total input variable numbers [7]. We then evaluated the performance results.

Only one neuron was present in the output layer, which had the same size as that of the target.

The prediction for VLCC markets earnings with the advanced periods of one, three, six, nine, 12, and 15 steps (months) were performed using MATLAB with the neural network toolbox. The inputs of the testing ANN model had nine nodes for input signals. In addition, the hidden layer was composed of neurons, with the tan-sigmoid transfer function selected as the neuron activation function. The value of the tagged delay line as a time delay was adjustable to avoid producing correlation effects.

As an essential tool for neural network validation, the regression coefficient between the network output and target, known as the R value, should approximate 1 to ensure reliable ANN performance results. In addition, when applying dynamic networks for prediction, such as the focused time-delay neural network, two crucial factors must be considered when analyzing the trained prediction network: prediction errors should not be correlated either in time or with the input sequence.

The previously mentioned 204 data points of each variable in the period from January 2000 to December 2016 were randomly sampled during computation and divided into three datasets for training, validation, and testing. When the Levenberg–Marquardt algorithm was applied for iterative computing, the training set comprised approximately 70% of the full dataset, with the validation and test datasets each consisting of approximately 15%. In addition, when the Bayesian regularization training technique was applied, the testing data set was assigned as only 15% of the full dataset because the validation sequence was not applied to the algorithm.

A schematic of the ANN network for tanker market prediction is shown in Figure 2. When the Levenberg–Marquardt algorithm was applied, the number of neurons in the hidden layer was adjusted to improve the accuracy of the prediction performance. In addition, when the Bayesian regularization algorithm was applied, the number of neurons in the hidden layer was fixed to eight and 10 to compare the performance results of these two cases with those from the Levenberg–Marquardt algorithm.

Figure 2: 
ANN schematic of NARX network for VLCC tanker market prediction

The number of neurons in the output layer was the same as the size of the target. The output layer was composed of one neuron with the linear function as its activation function.

Each implementation for prediction was repeated several times to identify the optimal parameters and conditions of the network. When the results were unsatisfactory, training was repeated after weights and biases were initialized. When the Levenberg–Marquardt algorithm was trained, the number of neurons was adjusted to prevent overfitting or extrapolation. The value of the tagged delay line as a time delay was 2 (months) without any change made during implementation. The computational results from the Levenberg–Marquardt algorithm were considered as reasonable when the algorithm was fitted with the following considerations:

ㆍ The final mean performance index (MSE) was small
ㆍ The test set error (test performance index (MSE)) and validation set error (validation performance index (MSE)) had similar characteristics
ㆍ Expert judgment was used in determining various parameters and performance indices

When the Bayesian regularization technique was trained, 221 parameters in the 9-10-1 (input to number of neurons of the hidden layer to output) network, and 177 parameters in the 9-8-1 network were employed. The effective number of parameters was a minimum of 27 and a maximum of 172 when the 9-10-1 network was trained. The training of the 9-10-1 network effectively used less than 77% of the total number of weights and biases. The computational results for the training of the Bayesian regularization algorithm were considered as reasonable when the following were considered:

ㆍ The final mean performance index (MSE) was small
ㆍ The training error (training performance index (MSE)) was small
ㆍ Expert judgment was used in determining various parameters and performance indices

The computer used for the calculation had an Intel® Core ™ i5-5200U CPU @ 2.20 GHz.

The regression plots display the network outputs with respect to targets for training, validation, and test datasets. For this problem, the fit was reasonably good for all datasets, The validation for this problem was satisfactory for all data sets in each case with an R value of at least 0.93. The autocorrelation function of the prediction error and the cross-correlation function to measure the correlation between the input and prediction error were used during ANN prediction model validation.

All implementation results of the mean performance index (MSE) are presented in Table 2 - Table 7 and Figure 3 - Figure 8 for use in evaluating the accuracy of the ANN prediction model.

5.1.1 One-month-ahead prediction

Figure 3 shows that BRA.TDS-15.NN-10 network had relatively good convergence with the total progression of the observation values . The networks of BRA.TDS-15.NN-10 and BRA.TDS-15.NN-8 showed better convergence-to-target values than did LMA.TDS-15.NN-9. The mean performance indices of the BRA.TDS-15.NN-8 and BRA.TDS-15.NN-10 networks were nearly the same. However, the training performance error of the BRA.TDS-15.NN-10 network was 0.266, which was much lower than that of the BRA.TDS-15.NN-8 network (1.3874), as shown in Table 2.

Figure 3: 
One-month-ahead prediction of VLCC

Table 1 shows the performance results for one-month-ahead prediction using the Levenberg–Marquardt algorithm for different numbers of neurons in the hidden layers. Each case shows best performance results without overfitting. When the number of neurons was greater or less than the number of input variables of 9, the results did not change satisfactorily. Note that during testing, the magnitude of the μ value was changed from 0.01 to 0.1 to speed up the conjugation.

Table 1: 
Comparison of one-month-ahead predictions based on different neuron numbers with the Levenberg-Marquardt algorithm

[ One-month-ahead Prediction ]	ANN (Levenberg-Marquardt algorithm)
	NN-7	NN-8	NN-9	NN-10
Epoch	11	11	12	15
μ	0.1	0.01	0.1	0.01
Gradient	8.11	19.9	8.68	9.6
Regression	0.92432	0.93793	0.96639	0.94919
Mean Performance Error	28.2745	24.9203	12.2177	18.7857
Train Performance Error	10.1344	16.0031	9.2426	6.7629
Validation Performance Error	73.9551	44.9173	17.0812	62.3482
Test Performance Error	68.4571	47.1315	21.4366	32.1312

NN-: Number of neurons in the hidden layer (Unit of Performance Error: x 1000 USD/Day)

5.1.2 Three-months-ahead prediction

Table 3 shows that with the Bayesian regularization algorithm, as the size of the hidden layer neuron increased, both the number of iterations and computing time increased, and the gradient value and train performance error (MSE) converged to a smaller value. However, even though the size of the hidden layer neuron increased in the short-term-ahead prediction, the mean performance index of 8.6425 for eight neurons showed significantly better results than 8.9288 for 10 neurons in the same forecasting horizon.

Table 2: 
Comparison for one-month-ahead prediction of VLCC

[ One-month-ahead Prediction ]	ANN Architecture
	BRA .TDS-15. NN-8	BRA .TDS-15. NN-10	LMA .TDS-15. NN-9
Epoch /computing time(sec)	279/5	443/9	12
μ	-	-	0.1
Gradient	0.82	0.329	8.68
Effective number of Parameter(used/total)	120/177	147/221	-
Regression	0.98218	0.98175	0.96639
Mean Performance Error	6.7013	6.7026	12.2177
Train Performance Error	1.3874	0.266	9.2426
Validation Performance Error	-	-	17.0812
Test Performance Error	37.168	43.6063	21.4366

(Unit of Performance Error: x 1000 USD/Day)

Table 3: 
Comparison for 3-months-ahead prediction of VLCC

[ 3-months-ahead Prediction ]	ANN Architecture
	BRA .TDS-15. NN-8	BRA .TDS-15. NN-10	LMA .TDS-15. NN-9
Epoch /computing time(sec)	370/6	707/13	12
μ	-	-	0.1
Gradient	1.0	1.23	11.3
Effective number of Parameter(used/total)	114/177	121/221	-
Regression	0.97649	0.97605	0.93509
Mean Performance Error	8.6425	8.9288	23.8859
Train Performance Error	2.2021	1.9989	10.3963
Validation Performance Error	-	-	58.8623
Test Performance Error	46.6056	48.6599	54.7607

(Unit of Performance Error: x 1000 USD/Day)

Figure 4: 
3-months-ahead prediction of VLCC

5.1.3 Six-months-ahead prediction

Table 4 shows that even though the number of hidden layer neurons increased from eight to 10 neurons, the gradient value and training performance error (MSE) did not converge to a smaller value. The training, validation, and test performance errors with the Levenberg–Marquardt algorithm showed a significant error in size despite no indication of overfitting or extrapolation.

Table 4: 
Comparison for 6-months-ahead prediction of VLCC

[ 6-months-ahead Prediction ]	ANN Architecture
	BRA .TDS-15. NN-8	BRA .TDS-15. NN-10	LMA .TDS-15. NN-8
Epoch /computing time(sec)	347/6	707/15	17
μ	-	-	0.001
Gradient	0.999	1.23	37.8
Effective number of Parameter(used/total)	110/177	121/221	-
Regression	0.97983	0.97605	0.92713
Mean Performance Error	7.8063	8.9288	27.5454
Train Performance Error	2.0636	1.9989	15.7874
Validation Performance Error	-	-	56.4943
Test Performance Error	40.7316	48.6599	54.2511

(Unit of Performance Error: x 1000 USD/Day)

Figure 5: 
6-months-ahead prediction of VLCC

5.1.4 Nine-months-ahead prediction

Figure 6 shows that the BRA.TDS-15.NN-8 network had relatively good convergence with the total progression of the observation values. However, during sudden up and down changes in the market, unstable prediction also appeared with the 6-months-ahead prediction. The training, validation, and test performance errors with the Levenberg–Marquardt algorithm showed significant errors in size despite no indication of overfitting or extrapolation.

Table 5: 
Comparison for 9-months-ahead prediction of VLCC

[ 9-months-ahead Prediction ]	ANN Architecture
	BRA .TDS-15. NN-8	BRA .TDS-15. NN-10	LMA .TDS-15. NN-9
Epoch /computing time(sec)	279/4	247/5	15
μ	-	-	0.01
Gradient	0.82	0.602	99.6
Effective number of Parameter(used/total)	120/177	136/221	-
Regression	0.9828	0.97937	0.94738
Mean Performance Error	6.7013	7.6663	19.2948
Train Performance Error	1.3874	0.6623	7.5927
Validation Performance Error	-	-	44.3444
Test Performance Error	37.168	47.8225	49.6353

(Unit of Performance Error: x 1000 USD/Day)

Figure 6: 
9-months-ahead prediction of VLCC

5.1.6 15-months-ahead prediction

Table 7 shows that the mean performance index of 8.243 for 10 neurons with the Bayesian regularization algorithm generated better results than did the index of 10.5656 for eight neurons and as compared to the other training algorithm in the same forecasting horizon. As shown, similar performance results occurred with the 12-months-ahead prediction as with other long-term-ahead forecasting when the training algorithm was applied, where larger neurons exhibited better forecasting performance than did the smaller.

Figure 7: 
12-months-ahead prediction of VLCC

Table 7: 
Comparison for 15-months-ahead prediction of VLCC

[ 15-months-ahead Prediction ]	ANN Architecture
	BRA .TDS-15. NN-8	BRA .TDS-15. NN-10	LMA .TDS-15. NN-6
Epoch /computing time(sec)	229/4	341/7	15
μ	-	-	0.01
Gradient	0.839	0.298	30.3
Effective number of Parameter(used/total)	116/177	149/221	-
Regression	0.97111	0.97762	0.94432
Mean Performance Error	10.5656	8.243	20.1285
Train Performance Error	1.5528	0.2142	11.0865
Validation Performance Error	-	-	42.1246
Test Performance Error	62.239	54.2748	40.9310

(Unit of Performance Error: x 1000 USD/Day)

Figure 8: 
15-months-ahead prediction of VLCC

where:

LMA : Levenberg-Marquardt Algorithm
BRA : Bayesian Regularization Algorithm
TDS- : Test data set for full input data set (%)
NN- : Number of neurons of hidden layer

5.1.7 Comparison of performance index with the training algorithm

Figure 9 shows that the Bayesian regularization algorithm produced satisfactory results for all prediction horizons as compared to the Levenberg-Marquardt algorithm. In addition, in both training algorithms, the prediction results were generally satisfactory when the number of neurons in the hidden layer was similar to the number of input variables. The exception was 12-months-ahead prediction with the Levenberg-Marquardt algorithm. Particularly in the case of the Bayesian regularization algorithm, up to 9-months-ahead prediction showed more satisfactory prediction results with smaller neurons in the hidden layer as compared to the number of input variables. However, in 12- and 15-months-ahead predictions, results that are more satisfactory were obtained with greater numbers of neurons in the hidden layer as compared to the number of input variables.

Figure 9: 
Comparison of performance index for training algorithm