Probability-based Neural Network Systems

Abstract

Since Gaussian distribution may be employed as a universal approximator, it is clear that most modelling and optimisation problems could be solved by probabilitybased ANN systems. For this reason, chapter 9 concentrate on probability-based ANN systems. The first section introduces the random number generator, which has application in Markov-Chain and its hybrid, in subsequent sections. The fifth section describes the Restricted Boltzmann Machine (RBM) in detail. The Boltzmann machine may be a component network of Deep Belief Networks (DBN), which is described in the last section. The chapter has explained many algorithms related to DBN with great intuition, as this may facilitate better understanding and therefore implementation.

Keywords: Annealed Importance Sampling (AIS), Boltzmann machine, Contrastive divergence, Detailed balance, Distribution, Dynamic architecture, Energy function, Ergodic, Gibbs, Hamiltonian, Markov chain, Metropolis-Hasting criteria, Molecular dynamics, Momentum heat-bath, Partition function, Pseudorandom number, Random number, Sampling, Stationary distribution, Timereversible, Verlet integrator.