cross-entropy

When reading tensorflow document, it reminds that this post could explain cross-entropy well.

http://colah.github.io/posts/2015-09-Visual-Information/

So, why should you care about cross-entropy? Well, cross-entropy gives us a way to express how different two probability distributions are.