From Bayesian Posteriors to JS Divergence: Complete Derivation of GAN Optimal Discriminator
1. Motivation: Why Derive the Optimal Discriminator?
1.1 Core Questions Addressed in This Chapter
In the preceding section, we formulated Generative Adversarial Networks (GANs) as a minimax zero-sum game: generator G aims to deceive the discriminator, while discriminator D classifies inputs. A fundamental question emerges:
“ 💡 Core Question: When generator G is held constant, what is the exact closed-form optimal strategy for discriminator D? At global equilibrium, what state does the game converge to, and which divergence metric does it optimize?”
These questions form the theoretical backbone of this chapter. The analytical conclusions are:
- The optimal discriminator has an exact closed-form solution: D*(x) = pdata(x)pdata(x) + pg(x);
- Under the optimal discriminator, the minimax objective reduces strictly to minimizing the Jensen-Shannon Divergence (JSD) between pdata and pg.
These two theorems construct the mathematical foundation of the original GAN framework (Goodfellow et al., 2014) and explain why training instability occurs when support manifolds are disjoint.
1.2 Theoretical Position in the GAN Curriculum
Chapter 2
Minimax Formulation
Current · Optimal D*
Closed-Form
Loss Reduction
-2log2 + 2JSD
Global Optimum: pg = pd → Ch 5 WGAN
Ch 4 · JSD Vanishing Gradients
