What is a neural network?

Maybe this?

...or this?

But all more simpler

(simple neural network with 2 input and 5 hidden layers)

Structure of neural network

Structure of NN

Sinapse

Take the value from the input, multiply by specific weight and output the result.

Neurons

Their job is to act together the output from other synapse and apply the activation function 

(sigmoid activation function)

Neurons

And the apply the activation function to every neuron

Training of neural network

Error of NN

Cost function

Training rule

Gradient descent