Denoising Diffusion Probabilistic Models
Diffusion 的关键就是两个步骤：前向加噪 (Forward Diffusion Process) 和反向去噪 (Reverse Diffusion Process)
该图来自于 DDPM
Forward Diffusion Process
原始的真实数据为 \(x_0\sim p(x)\) \(T\) \(x_1,\cdots, x_T\)
\[
x_0 \xrightarrow{\text{add noise }t=0} x_1 \xrightarrow{\text{add noise }t=1} \cdots \xrightarrow{\text{add noise }t=T-1} x_T
\]
令 \(T\to\infty\) \(x_T\) (isotropic Gaussian distribution)
在 DDPM(denoising diffusion probabilistic models) 马尔可夫过程 ，即后一步的分布仅依赖于前一步，有
\[
q(x_t \mid x_{t-1})=\mathcal{N}(\sqrt{1-\beta_t}x_{t-1},\beta_tI)
\]
\(I\) : (identity matrix); \(\beta_t\) :方差 (variance)，即 \(\sqrt{\beta_t}\) (standard deviation)
\(\{\beta_t\}_T\)
\(\{\beta_t\}_T\) 超参 (hyperparameter) 序列，是预先设定的。实际加噪过程中，会设置 \(\beta_1<\beta_2\cdots<\beta_T\)
DDPM 原论文中，给 \(\{\beta_t\}_T\) \(0.0001\) \(0.02\) denoising scheduler
直观感觉的 \(x_t\) \(x_{t-1}\) \(\sqrt{1-\beta_t}\) \(T\to\infty\) \(x_T\) \(\mathcal{N}(0, 1)\)
接下来的推导中将使用代换 \(\alpha_t=1-\beta_t\) \(\alpha\)
\[
x_t = \sqrt{\alpha _t}x_{t-1} + \sqrt{1-\alpha _t}\epsilon_{t-1}
\]
\(x_t-\sqrt{\alpha _t}x_{t-1}\sim \mathcal{N}(0, \beta _t I)\)
递推有
\[
x_t=\sqrt{\overline{\alpha}_t}x_0 + \sqrt{1-\overline{\alpha}_t}\varepsilon_t
\]
我们将在符号上，使用 \(\epsilon\) \(x_0\) \(x_1\) , \(x_1\) \(x_2\) , ... \(\varepsilon\) \(x_0\) \(x_t\)
推导：\(x_T\)
\[
\begin{aligned}
x_t
&= \sqrt{\alpha_t}x_{t-1} + \sqrt{1-\alpha_t}\epsilon_{t-1}\\
&= \sqrt{\alpha_t}(\sqrt{\alpha_{t-1}}x_{t-2} + \sqrt{1-\alpha_{t-1}}\epsilon_{t-2}) + \sqrt{1-\alpha_t}\epsilon_{t-1}\\
&= \sqrt{\alpha_t\alpha_{t-1}}x_{t-2}+\sqrt{\alpha_t(1-\alpha_{t-1})}\epsilon_{t-2}+\sqrt{1-\alpha_t}\epsilon_{t-1}
\end{aligned}
\]
这里有 \(\epsilon_i\sim \mathcal{N}(0, 1)\)
\[
\begin{gathered}
Var(c\xi+d)=c^2Var\xi\\
Var\left(\sum_i\xi_i\right)=\sum_iVar\xi_i+2\sum_{i<j}Cov(\xi_i,\xi_j)
\end{gathered}
\]
可知
\[
\begin{aligned}
&Var(\sqrt{\alpha_t(1-\alpha_{t-1})}\epsilon_{t-2}+\sqrt{1-\alpha_t}\epsilon_{t-1})\\
=\;& Var(\sqrt{\alpha_t(1-\alpha_{t-1})}\epsilon_{t-2})+Var(\sqrt{1-\alpha_t}\epsilon_{t-1})\\
=\;& \alpha_t(1-\alpha_{t-1}) Var(\epsilon_{t-2}) + (1-\alpha_t)Var(\epsilon_{t-1})\\
=\;& \alpha_t(1-\alpha_{t-1}) + (1-\alpha_t)
= 1-\alpha_t\alpha_{t-1}
\end{aligned}
\]
因此有
\[
x_t=\sqrt{\alpha_t\alpha_{t-1}}x_{t-2}+\sqrt{1-\alpha_t\alpha_{t-1}}\overline{\epsilon}_{t-2},\quad \overline{\epsilon}_{t-2}\sim \mathcal{N}(0, 1)
\]
以此类推就会有
\[
x_t=\sqrt{\overline{\alpha}_t}x_0 + \sqrt{1-\overline{\alpha}_t}\varepsilon_t
\]
其中 \(\overline{\alpha}_t=\prod_{i=1}^t\alpha_i\) , \(\varepsilon_t\sim \mathcal{N}(0, 1)\) \(\alpha_i\in(0, 1)\) \(t\to \infty\) \(\overline{\alpha}_t\to 0\) \(x_t\to \varepsilon_t\sim \mathcal{N}(0, 1)\)
Reverse Diffusion Process
如果能从 \(p(x_{t-1}\mid x_t)\) \(\mathcal{N}(0, 1)\) ( )
\[
x_T\sim \mathcal{N}(0, 1) \xrightarrow{\text{sample from }p(x_{T-1}\mid x_{T})} x_{T-1} \xrightarrow{\text{sample from }p(x_{T-2}\mid x_{T-1})} \cdots \xrightarrow{\text{sample from }p(x_{0}\mid x_{1})} x_0
\]
因此目标是求取 \(p(x_{t-1}\mid x_t)\) \(p_\theta(x_{t-1} \mid x_t)\to p(x_{t-1} \mid x_t)\)
\[
\begin{gathered}
p_{\theta}(x_{0:T})=p(x_T)\prod_{t=1}^Tp_\theta(x_{t-1} \mid x_t)\\
p_{\theta}(x_{t-1} \mid x_t)=\mathcal{N}(\mu_\theta(x_t, t), \Sigma_\theta(x_t, t))
\end{gathered}
\]
引入条件 \(x_0\) \(p_{\theta}(x_{t-1} \mid x_t)\) \(p(x_{t-1} \mid x_t, x_0)\) \(\beta_t\) \(p(x_{t-1}\mid x_t)\) Gaussian
在此不仔细检查提供其推导，详见 Feller, William. "On the theory of stochastic processes, with particular reference to applications." Proceedings of the [First] Berkeley Symposium on Mathematical Statistics and Probability. University of California Press, 1949.
在这个数学结论的支持下，我们就可以求解：
\[
p(x_{t-1} \mid x_t, x_0)=\mathcal{N}(\tilde{\mu}_t, \tilde{\beta}_tI)
\]
其中
\[
\begin{aligned}
\tilde{\mu}_t
&=\tilde{\mu}_t(x_t, x_0)=\frac{\sqrt{\alpha_t}(1-\overline{\alpha}_{t-1})}{1-\overline{\alpha}_t}x_t + \frac{\sqrt{\overline{\alpha}_{t-1}}(1-\alpha_t)}{1-\overline{\alpha}_{t}}x_0 \\
&=\tilde{\mu}_t(x_t, \varepsilon_t)=\frac{1}{\sqrt{\alpha}_t}\left(x_t-\frac{1-\alpha_t}{\sqrt{1-\overline{\alpha}_t}}\varepsilon_t\right)\\
\tilde{\beta}_t
&=\frac{1-\overline{\alpha}_{t-1}}{1-\overline{\alpha}_{t}}\beta_t
\end{aligned}
\]
推导：\(p(x_{t-1} \mid x_t, x_0)=\mathcal{N}(\tilde{\mu}_t, \tilde{\beta}_tI)\)
首先由 Bayes
\[
q(x_{t-1}, x_t \mid x_0) =
p(x_{t-1} \mid x_t, x_0) q(x_t \mid x_0) =
q(x_t \mid x_{t-1}, x_0) q(x_{t-1} \mid x_0)
\]
根据 Markov \(p(x_t \mid x_{t-1}, x_0)=p(x_t \mid x_{t-1})\)
\[
\begin{aligned}
p(x_{t-1} \mid x_t, x_0)
&= \underbrace{
p(x_t \mid x_{t-1})
}_{
\mathcal{N}(
\sqrt{\alpha_t}x_{t-1},
(1-\alpha_t)I
)
}
\cdot
\underbrace{
\frac{q(x_{t-1} \mid x_0)}{q(x_t \mid x_0)}
}_{
\mathcal{N}(
\sqrt{\overline{\alpha}_{t-1}}x_0,
(1-\overline{\alpha}_{t-1})I
)
\; / \;
\mathcal{N}(
\sqrt{\overline{\alpha}_{t}}x_0,
(1-\overline{\alpha}_{t})I
)
}\\
&\propto \exp\left(
-\frac{1}{2} \left(
\frac{
(x_t-\sqrt{\alpha_t}x_{t-1})^2
} {
1-\alpha_t
} +
\frac{
(x_{t-1}-\sqrt{\overline{\alpha}_{t-1}}x_0)^2
} {
1-\overline{\alpha}_{t-1}
} -
\frac{
(x_t-\sqrt{\overline{\alpha}_{t}}x_0)^2
} {
1-\overline{\alpha}_{t}
}
\right)
\right)
\end{aligned}
\]
整理 \(\exp\left(-\frac{1}{2}(*)\right)\) \(*\) \(x_{t-1}\)
\[
\left(\frac{\alpha_t}{1-\alpha_t} + \frac{1}{1-\overline{\alpha}_{t-1}}\right)x_{t-1}^2 -
\left(\frac{2\sqrt{\alpha_t}}{1-\alpha_t}x_t + \frac{2\sqrt{\overline{\alpha}_{t-1}}}{1-\overline{\alpha}_{t-1}}x_0\right)x_{t-1} +
C(x_t, x_0)
\]
想要求出 \(\tilde{\mu}_t, \tilde{\beta}_tI\) Gaussian
\[
\exp\left(-\frac{(x-\mu)^2}{2\beta}\right) =
\exp\left(-\frac{1}{2}\left(\frac{1}{\beta}x^2-\frac{2\mu}{\beta}x+\frac{\mu^2}{\beta}\right)\right)
\]
根据二次项系数就得到 \(\tilde{\beta}_t=\frac{(1-\alpha_t)(1-\overline{\alpha}_{t-1})}{1-\alpha_t\overline{\alpha}_{t-1}}=\frac{1-\overline{\alpha}_{t-1}}{1-\overline{\alpha}_{t}}\beta_t\)
\[
\begin{aligned}
\tilde{\mu}_t
&=\frac{1}{2}\left(\frac{2\sqrt{\alpha_t}}{1-\alpha_t}x_t + \frac{2\sqrt{\overline{\alpha}_{t-1}}}{1-\overline{\alpha}_{t-1}}x_0\right)\cdot \frac{1-\overline{\alpha}_{t-1}}{1-\overline{\alpha}_{t}}\beta_t\\
&=\boxed{\frac{\sqrt{\alpha_t}(1-\overline{\alpha}_{t-1})}{1-\overline{\alpha}_t}x_t + \frac{\sqrt{\overline{\alpha}_{t-1}}(1-\alpha_t)}{1-\overline{\alpha}_{t}}x_0}
\quad \boxed{ \tilde{\mu}_t(x_t, x_0) }
\end{aligned}
\]
Recall: \(\beta_t = 1-\alpha_t\) ，\(\overline{\alpha}_t = \prod_{i=1}^t\alpha_i\)
这里再对一对常数项系数就能验证是否是 Gaussian \(\beta_t\) Gaussian \(x_t=\sqrt{\overline{\alpha}_t}x_0 + \sqrt{1-\overline{\alpha}_t}\varepsilon \Rightarrow x_0=\frac{1}{\sqrt{\overline{\alpha}_t}}(x_t-\sqrt{1-\overline{\alpha}_t}\varepsilon)\)
\[
\begin{aligned}
\tilde{\mu}_t
&=\frac{\sqrt{\alpha_t}(1-\overline{\alpha}_{t-1})}{1-\overline{\alpha}_t}x_t + \frac{\sqrt{\overline{\alpha}_{t-1}}\beta_t}{1-\overline{\alpha}_{t}}\cdot\frac{1}{\sqrt{\overline{\alpha}_t}}(x_t-\sqrt{1-\overline{\alpha}_t}\varepsilon)\\
&=\boxed{\frac{1}{\sqrt{\alpha}_t}\left(x_t-\frac{1-\alpha_t}{\sqrt{1-\overline{\alpha}_t}}\varepsilon_t\right)}
\quad \boxed{ \tilde{\mu}_t(x_t, \varepsilon_t) }
\end{aligned}
\]
\(\tilde{\beta}_t\) \(\varepsilon\) (epsilon-prediction) \(x_0\) (v-prediction ) \(\tilde{\mu}_t\) \(\tilde{\mu}_t\) \(\mathcal{N}(0, 1)\) \(\epsilon_t\) \(x_t\)
\[
x_{t-1}=\tilde{\mu}_t + \sqrt{\tilde{\beta}_t}\epsilon_t
\]
这里应用了重参数化技巧，不直接从 \(\mathcal{N}(\tilde{\mu}_t, \tilde{\beta}_tI)\) \(x_{t-1}\) \(\mathcal{N}(0, 1)\) \(\epsilon_t\) \(\tilde{\mu}_t\) \(\tilde{\beta}_t\) \(x_{t-1}\)
事实上，大家后来往往用 \(\sigma_t^2\) \(\tilde{\beta}_t\) \(\tilde{\beta}_t\) \(\beta_t\) denoising scheduler \(\sigma_t^2\)
Interpretation from Maximum Likelihood
在极大似然视角下，DDPM 仍然是经典的通过优化 ELBO (Evidence Lower Bound)
\[
\log p(x_0) \geqslant \mathbb{E}_{q(x_{1:T}\mid x_0)}\left[\log \frac{p(x_{0:T})}{q(x_{1:T}\mid x_0)} \right]
\]
\(\mathbb{E}_{z\sim q(z \mid x)}\) \(\mathbb{E}_{q(z\mid x)}\)
推导：Evidence Lower Bound (ELBO)
将数据 \(x\) \(z\) \(z\) \(x\)
\[
\begin{aligned}
\log p(x)
&= \log p(x) \int q_{\phi} (z \mid x) dz\\
&= \int \log p(x) q_{\phi} (z \mid x) dz\\
&= \mathbb{E}_{z\sim q_{\phi} (z \mid x)}\left[\log p(x)\right] \\
&= \mathbb{E}_{z\sim q_{\phi} (z \mid x)}\left[\log \frac{p(x, z)}{p(z\mid x)}\right] \\
&= \mathbb{E}_{z\sim q_{\phi} (z \mid x)}\left[\log \frac{p(x, z)q_{\phi}(z\mid x)}{p(z\mid x)q_{\phi}(z\mid x)}\right] \\
&= \mathbb{E}_{z\sim q_{\phi} (z \mid x)}\left[\log \frac{p(x, z)}{q_{\phi}(z\mid x)}\right] + \mathbb{E}_{z\sim q_{\phi} (z \mid x)}\left[\log \frac{q_{\phi}(z\mid x)}{p(z\mid x)}\right] \\
&= \mathbb{E}_{z\sim q_{\phi} (z \mid x)}\left[\log \frac{p(x, z)}{q_{\phi}(z\mid x)}\right] + D_{\mathrm{KL}}(q_{\phi}(z\mid x) \;\|\; p(z\mid x)) \\
&\geqslant \mathbb{E}_{z\sim q_{\phi} (z \mid x)}\left[\log \frac{p(x, z)}{q_{\phi}(z\mid x)}\right]
\end{aligned}
\]
将前面反向去噪过程中建模的 \(p_\theta(x_{0:T})\) \(p(x_{0:T})\)
\[
\begin{aligned}
\log p(x_0)
&\geqslant \mathbb{E}_{q(x_{1:T}\mid x_0)}\left[
\log \frac {
p_{\theta}(x_{0:T})
} {
q(x_{1:T}\mid x_0)
}
\right]\\
&=\mathbb{E}_{q(x_{1:T}\mid x_0)}\left[
\log \frac {
p(x_T)
{\color{lime} \prod_{t=1}^T p_\theta(x_{t-1} \mid x_t)}
} {
{\color{lime} \prod_{t=1}^T q(x_t \mid x_{t-1})}
}
\right]\\
&=\mathbb{E}_{q(x_{1:T}\mid x_0)}\left[
\log \frac {
p(x_T)
{\color{skyblue} p_\theta(x_0 \mid x_1)}
{\color{red} \prod_{t=2}^T p_\theta(x_{t-1} \mid x_t)}
} {
{\color{skyblue} q(x_1 \mid x_0)}
{\color{red} \prod_{t=2}^T q(x_t \mid x_{t-1})}
}
\right]\\
&=\mathbb{E}_{q(x_{1:T}\mid x_0)}\left[
{\color{skyblue}
\log \frac {
p(x_T) p_\theta(x_0 \mid x_1)
} {
q(x_1 \mid x_0)
}
} +
\underbrace{ \color{red}
\sum_{t=2}^T\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
q(x_t \mid x_{t-1})
}
}_{\text{to be processed}}
\right]
\end{aligned}
\]
针对上面的连加项，由于马尔可夫性质有 \(q(x_t \mid x_{t-1})=q(x_t \mid x_{t-1}, x_0)\)
\[
\begin{aligned}
&{\color{red} \sum_{t=2}^T\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
q(x_t \mid x_{t-1})
}} \\
=& \sum_{t=2}^T\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
q(x_t \mid x_{t-1}, x_0)
} \quad \boxed{
q(x_t \mid x_{t-1}, x_0)
= p(x_{t-1} \mid x_t, x_0) \frac{
q(x_t \mid x_0)
} {
q(x_{t-1} \mid x_0)
}
}\\
=& \sum_{t=2}^T\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
p(x_{t-1} \mid x_t, x_0)
} + \sum_{t=2}^T \log \frac{
q(x_{t-1} \mid x_0)
} {
q(x_t \mid x_0)
} \\
=& { \color{red}
\sum_{t=2}^T\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
p(x_{t-1} \mid x_t, x_0)
}
} + { \color{skyblue}
\log \frac{
q(x_1 \mid x_0)
} {
q(x_T \mid x_0)
}
} \\
\end{aligned}
\]
\(\log \frac {q(x_1 \mid x_0)} {q(x_T \mid x_0)}\) \(\log \frac { p(x_T) p_\theta(x_0 \mid x_1) } { q(x_1 \mid x_0) }\) \(\log \frac {p(x_T) p_\theta(x_0 \mid x_1)} {q(x_T \mid x_0)}\)
\[
\begin{aligned}
\log p(x_0)
&\geqslant \mathbb{E}_{q(x_{1:T}\mid x_0)}\left[
{ \color{skyblue}
\log \frac {
p(x_T) p_\theta(x_0 \mid x_1)
} {
q(x_T \mid x_0)
}
} + { \color{red}
\sum_{t=2}^T\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
p(x_{t-1} \mid x_t, x_0)
}
}
\right]\\
&=\mathbb{E}_{q(x_1\mid x_0)}\left[
{\color{skyblue} p_\theta(x_0 \mid x_1)}
\right] +
\mathbb{E}_{q(x_{T}\mid x_0)}\left[
{ \color{skyblue}
\log \frac {
p(x_T)
} {
q(x_T \mid x_0)
}
}
\right] \\
&\quad + \sum_{t=2}^T
\mathbb{E}_{q(x_t, x_{t-1}\mid x_0)}\left[
{ \color{red}
\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
p(x_{t-1} \mid x_t, x_0)
}
}
\right]\\
&=\underbrace{\mathbb{E}_{q(x_1\mid x_0)}\left[
p_\theta(x_0 \mid x_1)
\right]}_{\text{reconstruction term, }L_0} -
\underbrace{D_{KL}\left(
q(x_T \mid x_0) \;\|\; p(x_T)
\right)}_{\text{prior matching term, }L_T} \\
&\quad -
\underbrace{\sum_{t=2}^T
\mathbb{E}_{q(x_{t}\mid x_0)}\left[
D_{KL} \left(
p(x_{t-1} \mid x_t, x_0)
\;\|\;
p_\theta(x_{t-1} \mid x_t)
\right)
\right]}_{\text{denoising matching term, }L_{1:T-1}}
\end{aligned}
\]
重建项 (reconstruction term) \(L_0\) : VAE DDPM 中视为从连续隐变量到离散图片数据的映射，进行了特殊处理先验匹配项 (prior matching term) \(L_{T}\) : DDPM 0去噪匹配项 (denoising matching term) \(L_{1:T-1}\) : \(p_\theta(x_{t-1} \mid x_t)\) \(p(x_{t-1} \mid x_t, x_0)\)
Training Objective
即
\[
L_{t-1} = \mathbb{E}_{q(x_t, x_{t-1} \mid x_0)}
\left[
\log \frac {
p_\theta(x_{t-1} \mid x_t)
} {
p(x_{t-1} \mid x_t, x_0)
}
\right]
\]
代入 \(p(x_{t-1} \mid x_t, x_0)=\mathcal{N}(\tilde{\mu}_t, \tilde{\beta}_tI)\) \(p_\theta(x_{t-1} \mid x_t)\) \(\mathcal{N}(\mu_\theta(x_t, t), \tilde{\beta}_tI)\)
\[
\begin{aligned}
L_{t-1}
&= \mathbb{E}_{q(x_t \mid x_0)} D_{KL}\left(
p_\theta(x_{t-1} \mid x_t) \;\|\; p(x_{t-1} \mid x_t, x_0)
\right)\\
&= \mathbb{E}_{q(x_t \mid x_0)} \left[
\frac{1}{2}(\mu_\theta(x_t, t)-\tilde{\mu}_t(x_t, x_0))^{\top}\tilde{\beta}_t^{-1}(\mu_\theta(x_t, t)-\tilde{\mu}_t(x_t, x_0))
\right] \\
&= \mathbb{E}_{q(x_t \mid x_0)} \left[
\frac{1}{2\tilde{\beta}_t}\|\mu_\theta(x_t, t)-\tilde{\mu}_t(x_t, x_0))\|^2
\right]
\end{aligned}
\]
原论文中还要加上一个与 \(\theta\) \(C\) \(p_\theta(x_{t-1} \mid x_t)\) \(\tilde{\beta}_tI\) \(\mu_\theta(x_t, t)\)
两个 \(d\) KL
设 \(p_1\) , \(p_2\) \(d\) \(\mu_1, \mu_2\) \(\Sigma_1, \Sigma_2\)
\[
\boxed{
D_{KL}(p_1 \;\|\; p_2) = \frac{1}{2}\left(
\log \frac{\det\Sigma_2}{\det\Sigma_1} + \mathrm{Tr}(\Sigma_2^{-1}\Sigma_1) +
(\mu_2-\mu_1)^{\top}\Sigma_2^{-1}(\mu_2-\mu_1) - d
\right)
}
\]
证明如下。考虑到
\[
p_i(x) = \frac{1}{(2\pi)^{d/2}(\det\Sigma_i)^{1/2}} \exp\left(-\frac{1}{2}(x-\mu_i)^{\top}\Sigma_i^{-1}(x-\mu_i)\right)
\]
有
\[
\begin{aligned}
\mathbb{E}_{p_1}[\log p_i(x)]
&= \mathbb{E}_{p_1}\left[-\frac{d}{2}\log(2\pi) -\frac{1}{2}\log \det\Sigma_i- \frac{1}{2}(x-\mu_i)^{\top}\Sigma_i^{-1}(x-\mu_i)\right]\\
&= -\frac{1}{2}\left(
{ \color{skyblue}
d\log(2\pi) + \log \det\Sigma_i
}
+ {\color{red}
\mathbb{E}_{p_1}[
(x-\mu_i)^{\top}\Sigma_i^{-1}(x-\mu_i)
]
}
\right)\\
\end{aligned}
\]
其中
\[
\begin{aligned}
{\color{red} \mathbb{E}_{p_1}[
(x-\mu_i)^{\top} \Sigma_i^{-1} (x-\mu_i)
]}
&= \mathbb{E}_{p_1}[\mathrm{Tr}((x-\mu_i)^{\top}\Sigma_i^{-1}(x-\mu_i))]\\
&= \mathbb{E}_{p_1}[
\mathrm{Tr}(
\Sigma_i^{-1} (x-\mu_i)(x-\mu_i)^{\top}
)
]\\
&= \mathrm{Tr}(
\Sigma_i^{-1} \mathbb{E}_{p_1}[
(x-\mu_i)(x-\mu_i)^{\top}
]
) \\
&= \mathrm{Tr}(
\Sigma_i^{-1} \mathbb{E}_{p_1}[
xx^{\top} + \mu_i\mu_i^{\top} - x\mu_i^{\top} - \mu_ix^{\top}
]
) \\
&= \mathrm{Tr}(
\Sigma_i^{-1} [
(\Sigma_1 + \mu_1\mu_1^{\top}) + \mu_i\mu_i^{\top} - \mu_1\mu_i^{\top} - \mu_i\mu_1^{\top}
]
) \\
&= \begin{cases}
{\color{red} d}, & i = 1\\
{\color{red} \mathrm{Tr}(\Sigma_2^{-1}\Sigma_1) + (\mu_2 - \mu_1)^{\top}\Sigma_2^{-1}(\mu_2 - \mu_1)}, & i = 2
\end{cases}
\end{aligned}
\]
第一行到第二行应用了迹的恒等式：\(\mathrm{Tr}(AB)=\mathrm{Tr}(BA)\)
因此最终有
\[
\begin{aligned}
D_{KL}(p_1 \;\|\; p_2)
&= \mathbb{E}_{p_1}\left[\log p_1(x) - \log p_2(x)\right]\\
&= \frac{1}{2}\left(
{\color{skyblue} \log\frac{\det\Sigma_2}{\det\Sigma_1}}
+ { \color{red}
\mathrm{Tr}(\Sigma_2^{-1}\Sigma_1)
+ (\mu_2 - \mu_1)^{\top}\Sigma_2^{-1}(\mu_2 - \mu_1)
- d
}
\right)\\
\end{aligned}
\]
前面我们已经得到了 \(\tilde{\mu}_t\) \(\tilde{\mu}_t(x_t, \varepsilon_t)\) \(\mu_\theta(x_t, t)\)
\[
\mu_\theta(x_t, t) = \mu_t(x_t, \varepsilon_{\theta}(x_t, t)) = \frac{1}{\sqrt{\alpha}_t}\left(x_t - \frac{1-\alpha_t}{\sqrt{1-\overline{\alpha}_t}}\varepsilon_{\theta}(x_t, t)\right)
\]
这样就有
\[
L_{t-1} = \mathbb{E}_{q(x_t \mid x_0), \varepsilon_t}\left[
\frac{\beta_t^2}{2\tilde{\beta}_t \alpha_t (1-\overline{\alpha}_t)}\|\varepsilon_{\theta}(x_t, t) - \varepsilon_t\|^2
\right]
\]
Recall \(x_t=\sqrt{\overline{\alpha}_t}x_0 + \sqrt{1-\overline{\alpha}_t}\varepsilon_t\) , \(\beta_t=1-\alpha_t\)
实践中，DDPM 略去了常数乘法系数（不同 \(t\) ） ，最小化
\[
L_{t-1} = \mathbb{E}_{t\sim \{1, ..., T\}, q(x_t \mid x_0), \varepsilon_t}\|\varepsilon_{\theta}(x_t, t) - \varepsilon_t\|^2
\]
综合观之，有
