Say I have a NN that I want to train to perform a
function approximation problem (RBF, for example).
My problem is to choose the noise level in the
training.
One option is to train without noise at all or the
other option might be to add some noise to the
training set (say SNR = 30dB or SNR = 20dB).
The basic question is how to be able to compare
between the different training sets?
Can I use the test set error to pick the right
training set? (lets assume the test set has data with
SNR = -30dB:10:30dB).

I think it's unfair to use a validation set to compare
the different training sets, as the validation set is
basically very similar to the training set, so we can
expect a lower validation set error for the training
without noise, vs. the training with high levels of
noise, but it doesn't mean the generalization error
will be lower.

I'll appreciate any thoughts on this...

Thanks,
Gal