I'm new to NN and I've been trying to fix this problem in a Boltzmann Machine for some time now.

My initial network had 2 layers, no hidden nodes and was unable to learn a AND problem. The nodes take values {-1, 1}

I1
.\ (w1)
..\
...> O1
../
./ (w2)
I2

The reason was that it all came down to this:
The weights are always different from eachother (for demonstration purposes I will consider w1 > w2)
So if the input vector is [-1, 1] then O1 = -1 (as it is expected), but if the inputs are [1, -1] then O1 will be 1 (which is incorrect). This happens independently of how small the difference between the weights is.
To fix this, I calculated a bias by:
- summing the differences between the expected output and real output during the training phase
- normalized that value according to the number of train examples
For the AND problem I ended up with: w1 = 0.612, w2 = 0.617, bias = -0.504. The bias is added to the output node before calling the sigmoid function.

But then I added a hidden layer with 2 nodes and got back to square zero... So now I'm unsure if the bias was the correct option.
What am I doing wrong? How do I solve this?

Thank you for your help!