cost function in deep learning

 

1. cost function should be able to average /sum up over each sample

2. cost function does not dependent on the activation function, which is required for back-propagation to work. 

reference:

https://stats.stackexchange.com/questions/154879/a-list-of-cost-functions-used-in-neural-networks-alongside-applications