20190824

2つの互いに独立な確率変数

$(\Omega, \mathcal{F}, P)$
$(\mathcal{X}, \mathcal{M}_{\mathcal{X}})$
$(\mathcal{Y}, \mathcal{M}_{\mathcal{Y}})$

$X: \Omega \to \mathcal{X}$
$Y: \Omega \to \mathcal{Y}$
$(X, Y): \omega \mapsto (X(\omega), Y(\omega))$

$P_{X, Y} = P \circ (X, Y)^{-1}$
$P_X = P_{X, Y}(\cdot \times \mathcal{Y}) = P \circ X^{-1}$
$P_Y = P_{X, Y}(\mathcal{X} \times \cdot) = P \circ Y^{-1}$

$P_{X, Y}$も直積測度$P_X \times P_Y$も，$(\mathcal{X} \times \mathcal{Y}, \mathcal{M}_{\mathcal{X}} \times \mathcal{M}_{\mathcal{Y}})$上の測度．

$X$と$Y$が独立なら，$P_{X, Y} = P_X \times P_Y$．

$f: \mathcal{X} \times \mathcal{Y} \to [0, \infty)$

フビニ:

$\begin{aligned} \int_{\mathcal{X} \times \mathcal{Y}} f(x, y) d(P_X \times P_Y) &= \int_{\mathcal{X}} \left\{\int_{\mathcal{Y}} f(x, y) dP_Y\right\}dP_X\\ &= \int_{\mathcal{Y}} \left\{\int_{\mathcal{X}} f(x, y) dP_X\right\}dP_Y \end{aligned}$

$X$と$Y$が独立なら，$\mathbb{E}_{P_{X, Y}} \left[f(x, y)\right] = \mathbb{E}_{P_X} \left[ \mathbb{E}_{P_Y} \left[f(x, y)\right]\right] = \mathbb{E}_{P_Y} \left[ \mathbb{E}_{P_X} \left[f(x, y)\right]\right]$

empirical distributionとsample

$f: \mathcal{X} \to [0, \infty)$
$D^{(m)}_X = \frac{1}{m} \sum_{i=1}^m \delta_{x_i}$
$\mathbb{E}_{P_X} \left[f\right] \approx \mathbb{E}_{D^{(m)}_X} [f] = \frac{1}{m} \sum_{i=1}^m f(x_i) := g(\bm{x})$
$S: \omega \mapsto (X_1(\omega), \ldots, X_m(\omega))$
$P_S = P \circ S^{-1}$
$\mathbb{E}_{P_S} \left[g\right] = \mathbb{E}_P \left[g(S)\right] = \frac{1}{m}\sum_{i=1}^m \mathbb{E}_P \left[f(X_i)\right] = \frac{1}{m}\sum_{i=1}^m \mathbb{E}_{P_{X_i}} \left[f\right] = \frac{1}{m}\sum_{i=1}^m \mathbb{E}_{P_{X}} \left[f\right] = \mathbb{E}_{P_X} \left[f\right]$

Complexity

$X$と$Y$は独立とする．

$\sup_{g \in G} Y g(X)$が大きい方が複雑度が高い．

$\begin{aligned} \mathbb{E}_P \left[\sup_{g \in G} Y g(X)\right] &= \mathbb{E}_{P_{X, Y}} \left[\sup_{g \in G} y g(x)\right] \\ &\approx \mathbb{E}_{D^{(m)}_{X, Y}}\left[\sup_{g \in G} y g(x)\right] \\ &= \frac{1}{m} \sum_{i=1}^m \sup_{g \in G} y_i g(x_i) =: f(\bm{x}, \bm{y}) \end{aligned}$

$S_Y: \omega \mapsto (Y_1(\omega), \ldots, Y_m(\omega))$

for $\bm{x} \in \mathcal{X}^m$, $\hat{\mathscr{C}}(G) = \mathbb{E}_{P_{S_Y}} \left[f(\bm{x}, \cdot)\right]$を，empirical complexityと呼ぶことにする．

$S_X: \omega \mapsto (X_1(\omega), \ldots, X_m(\omega))$

$\mathbb{E}_{P_{S_X}} \left[\hat{\mathscr{C}}(G)\right]$を，complexityと呼ぶことにする．

complexityのイメージは多分こんな感じ．$\mathcal{Y} \in \left\{-1, 1\right\}$にして，$P_Y$をuniformだとすれば，Rademacher complexityになるはず．

うーんでも，ちょっとまだきになるところがあるので，明日やろうと思う．

Written with StackEdit.