Hi,

I have been working on a neural network using a back propagation network for some time and it is a good prediction tool currently which is good. However, ideally, its three outputs would give me an output related to each other in such a way that I could derive a probability from each of the outputs.

Looking at my options, I dont see how this can be done with an activation function as it rationalizes the outputs and in a three layer network you get outputs such as 0.71, 0.72, 0.70 returned. However, I cannot find any evidence of working neural networks which work in this way returning a decent value for probability.

Is this possible? Has anyone got any pointers?

Thanks,

Adam

Check out Neural Networks for Pattern Recognition by Bishop http://www.amazon.com/exec/obidos/tg/detail/-/0198538642/104-7662266-1961534?v=glance particularly the chapter on error functions. If you are using linear activation functions for the output layer and the standard error formulas, the outputs can be interpreted as the conditional mean of a gaussian distribution with fixed variance. That is, the error formula is just the partial derivative of the negative log likelihood. If you apply soft max to the outputs of the network and use the standard error formulas, you can interpret the values as conditional class probabilities. If you use a logistic sigmoid activation on a single output node with the standard error formulas, you can interpret the value as a conditional probability. The chapter also talks about how to model conditional variances and mixtures of distributions.

Consider using the softmax activation function. See, for instance, the Usenet comp.ai.neural-nets FAQ:

http://www.faqs.org/faqs/ai-faq/neural-nets/part2/

-Will Dwinnell
http://will.dwinnell.com