to understand how a neural network is working I'm building a neural network sandbox.
The sandbox provides a tiny network with 4(+bias) Input nodes, 5 hidden nodes and 4 Output nodes.
You can draw the connections (when editing click hold a node and drag to dest node ), delete(double click) them and set their weight (when training click hold and drag hor.) and you can set backpropagading connections.
Each neuron(node) has it's own function for value, trigger, backprop.
You can train the network (to perfection) and play with it after training to test it.
With each interation you can check the resulting values and node states to see what happens while leaning.
Here you can give it a try (current [fix4all] functions: val:passthrough, trigger:binary, backp:delta*0.1)
There will also sigmoid, tanh and other functions be available soon (and you will able to change them for each/all nodes).
link: http://www.gameus.de/nn
mercuryus