A Visual Guide to Natural Latents

Thanks to @Jeremy Gillen for reading and commenting on the draft. This was written while I was was funded by the Advanced Research + Invention Agency (ARIA) through project code MSAI-SE01-P005. I have tried to achieve two goals in this post. The first is to provide a self-contained explanation of Natural Latents using lots of pictures of probability distributions. The second is to frame Natural Latents in terms of statements about mutual information, rather than the KL-divergences and Bayes nets that Wentworth and Lorell normally use [1] . The two approaches are mathematically equivalent, but the different framings can bring slightly different way of thinking about the problem. This post will focus on what it means for a variable to satisfy the natural latent conditions and what those conditions correspond to intuitively. I'll also discuss a little bit about the motivation for studying natural latents. I'm not going to go through proofs or derivations, but hopefully by the end of this