# Chapter 12: Information Theory

This chapter covers basic information-theoretic concepts and discusses their relation to machine learning.

## Chapter 12.01: Entropy

## Chapter 12.02: Differential Entropy

## Chapter 12.03: Kullback-Leibler Divergence

In this section, we introduce the Kullback-Leibler divergence.

## Chapter 12.04: Entropy and Optimal Code Length

## Chapter 12.05: Cross-Entropy, KL and Source Coding

## Chapter 12.06: Information Theory for Machine Learning

In this section, we discuss how information-theoretic concepts are used in machine learning.

## Chapter 12.07: Joint Entropy and Mutual Information

In this section, we introduce joint entropy, conditional entropy, and mutual information.