Autoencoder

Denoising Autoencoder

Variational Autoencoder

• (*) Information: $I=-\log{P(x)} \rightarrow \begin{cases} 0, P(x) = 1 \\ \infin, P(x) \approx0 \end{cases}$
  • The intuition behind this is: assume that x is an event, if event x is sure to happen then no information will be gain from it. However, if the chance event x happens is really low, then its happening will give a lot of information.
  • $\log{1}=0$ and $\log{x\rightarrow0}=-\infin$