Infinite surprise - the iridescent personality of Kullback-Leibler divergence
Kullback-Leibler divergence is not just used to train variational autoencoders or Bayesian networks (and not just a hard-to-pronounce thing). It is a fundamental concept in information theory, put to use in a vast range of applications. Most interestingly, it's not always about constraint, regularization or compression. Quite on the contrary, sometimes it is about novelty, discovery and surprise
Machine Learning
Best Practices
AI
Probabilistic Machine Learning
Probabilistic Deep Learning
Concepts