forum.alglib.net

Hi,
I am trying to use mlpbase and mlptrain properly. I'm not sure if this is a bug but when I run mlptrain in a loop with same dataset, same parameters, all variables are local to each iteration, the results are VERY different across different iterations, though they are supposed to be exactly the same. Furthermore, when I run one iteration on different compile and run, they show exactly the same result.
So I guess there are some static variables in mlpbase or mlptrain that is modified after each time some function is called, and the next time the function is called, maybe it is re-used wrongly.
Please help,
Thanks,
Tuan

Neural networks are randomized before training - it is the common practice in machine learning. Behavior you've described is just a consequence of randomization.

So even if I don't use mlprandomize() or mlprandomizefull(), are the weights still randomized?

Yes, they are randomized internally, inside mlptrain().

If so, I believe the randomization is not really good. Because the results are so different. The errors tend to be much higher for next iterations. The first iterations always perform much better than the next ones.

Can you give me some compact code which demonstrates this pattern - "first iterations always perform much better than the next ones"?

mlptrain() uses really simple RNG - one which is included into the standard library. It may lead to correlations in the adjacent weights, but not to the pattern you mentioned.