Hi

When I train my NN, I am using 10 sets of random multiple initialisations. When I cross-validate the results over a validation set, obviously I get different error values (taken over validation) for the 10 random initialisations.

I was thinking if I save the set of initial weights that gives me the BEST error over the validation set and then use this (saved set) to estimate the TEST set. Would this be legitimate?

Any ideas on this?

Any help is much appreciated.

regards
Rohit

Hi,

Do you mean to save the initial values of the weights or the values after training?
I would think you can save the values after training and use that network with lowest validation error to get the error on the test set.

Thanks,
Gal

Hi

Thanks so much for your reply. I meant saving the initial values of the weights (ie before training). So for example if I have 10 sets of initialisation, I will use all of them and select the one which gives me the lowest error on validation. For the test I will then use this best initialisation value set. Hope this makes it more clear. Thanks again ! regards, Rohit

But one thing I didn't get it:
To check the error on the test set, you need weights after training, so what good is it to save the initial value of the weights?

Thanks!
Gal