PRML演習問題 2.11(基本) www - 機械学習基礎理論独習

問題

ディリクレ分布の下での $\ln \mu_j$ の期待値を、 $\alpha_j$ についての導関数として表すと、

$\begin{eqnarray} {\mathbb E}[\ln\mu_j]=\psi(\alpha_j)-\psi(\alpha_0)\tag{2.276} \end{eqnarray}$

になることを示せ。
ただし、 $\alpha_0$ は $(2.39)$ で定義され、 $\psi(\cdot)$ はディガンマ関数(digamma function)

$\begin{eqnarray} \psi(a)\equiv\frac{\rm d}{{\rm d}a}\ln\Gamma(a)\tag{2.277} \end{eqnarray}$

である。

参照

$\begin{eqnarray} {\rm Dir}({\boldsymbol\mu}|{\boldsymbol\alpha})=\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\prod_{k=1}^K\mu_k^{\alpha_k-1}\tag{2.38} \end{eqnarray}$

$\begin{eqnarray} \alpha_0=\sum_{k=1}^K\alpha_k\tag{2.39} \end{eqnarray}$

解答

${\mathbb E}[\ln\mu_j]$ を計算します。

$\begin{eqnarray} {\mathbb E}[\ln \mu_j]&=&\int\ln\mu_j{\rm Dir}({\boldsymbol\mu}|{\boldsymbol\alpha}){\rm d}{\boldsymbol\mu}\\ &=&\int\ln\mu_j\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\prod_{k=1}^K\mu_k^{\alpha_k-1}{\rm d}{\boldsymbol\mu}\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\int\ln\mu_j\prod_{k=1}^K\mu_k^{\alpha_k-1}{\rm d}{\boldsymbol\mu}\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\cdot\int\left(\frac{\partial}{\partial\alpha_j}\prod_{k=1}^K\mu_k^{\alpha_k-1}\right){\rm d}{\boldsymbol\mu}\tag{1}\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\cdot\frac{\partial}{\partial\alpha_j}\left(\int\prod_{k=1}^K\mu_k^{\alpha_k-1}{\rm d}{\boldsymbol\mu}\right)\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\cdot\frac{\partial}{\partial\alpha_j}\left(\int\prod_{k=1}^K\mu_k^{\alpha_k-1}{\rm d}{\boldsymbol\mu}\right)\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\cdot\frac{\partial}{\partial\alpha_j}\frac{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}{\Gamma(\alpha_0)}\tag{2}\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_K)}\cdot\Gamma(\alpha_1)\cdots\Gamma(\alpha_{j-1})\Gamma(\alpha_{j+1})\cdots\Gamma(\alpha_{K})\frac{\partial}{\partial\alpha_j}\frac{\Gamma(\alpha_j)}{\Gamma(\alpha_0)}\tag{3}\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_j)}\frac{\partial}{\partial\alpha_j}\frac{\Gamma(\alpha_j)}{\Gamma(\alpha_0)}\\ &=&\frac{\Gamma(\alpha_0)}{\Gamma(\alpha_j)}\frac{\left(\dfrac{\partial}{\partial\alpha_j}\Gamma(\alpha_j)\right)\Gamma(\alpha_0)-\Gamma(\alpha_j)\left(\dfrac{\partial}{\partial\alpha_j}\Gamma(\alpha_0)\right)}{\Gamma(\alpha_0)^2}\\ &=&\frac{1}{\Gamma(\alpha_j)}\dfrac{\partial}{\partial\alpha_j}\Gamma(\alpha_j)-\frac{1}{\Gamma(\alpha_0)}\dfrac{\partial}{\partial\alpha_j}\Gamma(\alpha_0)\\ &=&\frac{1}{\Gamma(\alpha_j)}\dfrac{\partial}{\partial\alpha_j}\Gamma(\alpha_j)-\frac{1}{\Gamma(\alpha_0)}\dfrac{\partial}{\partial\alpha_0}\Gamma(\alpha_0)\dfrac{\partial}{\partial\alpha_j}\alpha_0\tag{4}\\ &=&\frac{1}{\Gamma(\alpha_j)}\dfrac{\partial}{\partial\alpha_j}\Gamma(\alpha_j)-\frac{1}{\Gamma(\alpha_0)}\dfrac{\partial}{\partial\alpha_0}\Gamma(\alpha_0)\\ &=&\frac{\rm d}{{\rm d}a}\ln\Gamma(\alpha_j)-\frac{\rm d}{{\rm d}a}\ln\Gamma(\alpha_0)\tag{5}\\ &=&\psi(\alpha_j)-\psi(\alpha_0)\tag{6} \end{eqnarray}$

式 $(1)$ で、以下の式を使いました。

$\begin{eqnarray} \frac{\partial}{\partial\alpha_j}\prod_{k=1}^K\mu_k^{\alpha_k-1}&=&\left(\prod_{k=1,k\not=j}^K\mu_k^{\alpha_k-1}\right)\frac{\partial}{\partial\alpha_j}\mu_j^{\alpha_j-1}\\ &=&\left(\prod_{k=1,k\not=j}^K\mu_k^{\alpha_k-1}\right)\mu_j^{-1}\cdot\frac{\partial}{\partial\alpha_j}\mu_j^{\alpha_j}\\ &=&\left(\prod_{k=1,k\not=j}^K\mu_k^{\alpha_k-1}\right)\mu_j^{-1}\cdot\mu_j^{\alpha_j}\ln\mu_j\\ &=&\left(\prod_{k=1,k\not=j}^K\mu_k^{\alpha_k-1}\right)\mu_j^{\alpha_j-1}\ln\mu_j\\ &=&\ln\mu_j\prod_{k=1}^K\mu_k^{\alpha_k-1}\tag{7} \end{eqnarray}$