Dear Sergey!

My task is regression analisys. I have some hundreds of data sets and I train the same quantity of neural networks essembles on them (function mlpetraines() ).

There are the same quantity of input vectors for each data set of course. With function mlpeprocess() I define the result for each vector. Finally I define the maximum of all the results.

I use multi-threaded calculation using the CreateThread() function in C++. And depend on the numbers of using threads the results were different! Although the data sets and the vectors were equals! The different was the order of calculations only.

I assume that the reason is the different initial network weights that the function mlpecreate1() sets to random values. I made an experiment. In single thread I twice call mlpetraines() with the same data set (training set) and mlpeprocess() with the same vector and have got the different results:

**Code:**

alglib::mlpecreate1(nin, nhid, nout, ensemblesize, ensemble_A);

alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);

alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);

Result_1 = y_Out(0);

// Repeat the same code

alglib::mlpecreate1(nin, nhid, nout, ensemblesize, ensemble_A);

alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);

alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);

Result_2 = y_Out(0);

Result_2 is not equal to Result_1. Ok, this only confirms my assumption.

To order to make the initial random values of the neural net weights the same I remember ensemble_A immediately after function mlpecreate1() the first call and then restore them before the second call of mlpetraines():

**Code:**

alglib::mlpecreate1(nin, nhid, nout, ensemblesize, ensemble_A);

alglib::mlpeserialize(ensemble_A, string_mem);

alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);

alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);

Result_1 = y_Out(0);

alglib::mlpeunserialize(string_mem, ensemble_A);

alglib::mlpetraines(ensemble_A, XY_set, npoints, decay, restarts, info, rep);

alglib::mlpeprocess(ensemble_A, x_Vector, y_Out);

Result_2 = y_Out(0);

Result_2 is not equal to Result_1 again!

I do not understand anything. It looks like if one wants to calculate sinus of the some value twice and have got different results. Is not it ?

Is it possible to train the ensemble with the same data twice in the single code space and get the same trained ensemble in result? How do you achieve that effect and what is it – a bug or a feature ?