Cost function is specially designed to measure how bad the network is at classifying the training examples.