Automated adjustment system of restricted Boltzmann machine

Authors

  • V. M. Sineglazov National Aviation University, Kyiv
  • O. R. Tofaniuk National Aviation University, Kyiv

DOI:

https://doi.org/10.18372/1990-5548.60.13814

Keywords:

Deep Believe Network, Restricted Boltzmann Machine, Contrastive Divergence, Persistent Contrastive Divergence, Parallel Temperin

Abstract

In this paper the problem of learning the deep believe neural network with help of a restricted Boltzmann machine and the choose of an optimal algorithm for its training is considered. Different algorithms of restricted Boltzmann machine training, which are used for the pre-training of deep believe neural network, are considered, in order to increase the efficiency of this network and further solve the problem of structural-parametric synthesis of deep believe neural network. This task represents the task of justifying the necessity of optimal choice of the restricted Boltzmann machine adjustment algorithm for improving the quality of training of the neural network. To solve this problem, it is suggested to create an automated adjustment system of restricted Boltzmann machine, which choose the optimal training algorithm for this neural network.

Author Biographies

V. M. Sineglazov, National Aviation University, Kyiv

Aviation Computer-Integrated Complexes Department, Faculty of aeronavigation, electronics and telecomunication

Doctor of Engineering Science. Professor. Head of the Department

ORCID 0000-0002-3297-9060

 

 

O. R. Tofaniuk, National Aviation University, Kyiv

Aviation Computer-Integrated Complexes Department, Faculty of aeronavigation, electronics and telecomunication

Bachelor

References

. Golovko, “From multilayer perceptron to deep belief neural networks: training paradigms and application,” in book Lectures on neuroinformatics, Moscow, 2015, pp. 47–84. (in Russian)

G. E. Hinton and S. Osindero, “A fast learning algorithm for deep belief nets,” Neural computation № 18, 2006, pp. 1527–1554.

G. E. Hinton, R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, no. 313 (5786), 2006, pp. 504–507.

G. E. Hinton, “Training products of experts by minimizing contrastive divergence,” Neural Computation, pp. 1771–1800, 2002.

Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, and U. Montreal, “Greedy layerwise training of deep networks,” in: B. SchЁolkopf, J. Platt, T. Hoffman (eds.) Advances in Neural Information Processing, pp. 153–160, MIT Press, 2007.

Y. Bengio and O. Delalleau, “Justifying and generalizing contrastive divergence”, Neural Computation 21(6), pp. 1601–1621, 2009.

G. E. Hinton, “Learning multiple layers of representation,” Trends in Cognitive Sciences, 11(10), 428–434, 2007.

T. Tieleman, “Training restricted Boltzmann machines using approximations to the likelihood gradient,” in International Conference on Machine learning (ICML), pp. 1064–1071, 2008.

L. Younes, “Maximum likelihood estimation of Gibbs fields,” in Joint Conference on Spacial Statistics and Imaging, Lecture Notes Monograph Series, Institute of Mathematical Statistics, Hayward, California, 1991.

T. Tieleman and G. E. Hinton, “Using fast weights to improve persistent contrastive divergence,” International Conference on Machine Learning (ICML), pp. 1033–1040, 2009.

A. Fischer and C. Igel, “Empirical analysis of the divergence of Gibbs sampling based learning algorithms for Restricted Boltzmann Machines,” in International Conference on Artificial Neural Networks (ICANN), vol. 6354 of LNCS, pp. 208–217, Springer-Verlag, 2010.

K. Cho, T. Raiko and A. Ilin, “Parallel tempering is efficient for learning restricted Boltzmann machines,” Proceedings of the International Joint Conference on Neural Networks (IJCNN 2010), pp. 3246–3253. IEEE Press, 2010.

D. J. C. Mackay, Information Theory, Inference & Learning Algorithms, 1st ed. Cambridge University Press, June 2002.

Bengio Y., “Learning deep architectures for AI,” Foundations and trends in machine learning, no. 2(1), pp. 1–127, 2009.

H. Robert, Swendsen and Jian-Sheng Wang, “Replica Monte Carlo Simulation of Spin-Glasses”, Lett. 57, 2607, 24 November 1986.

Geyer and J. Charles, Markov Chain Monte Carlo Maximum Likelihood, 1991.

Downloads

How to Cite

Sineglazov, V. M., & Tofaniuk, O. R. (2019). Automated adjustment system of restricted Boltzmann machine. Electronics and Control Systems, 2(60), 43–52. https://doi.org/10.18372/1990-5548.60.13814

Issue

Vol. 2 No. 60 (2019)

Section

COMPUTER-AIDED DESIGN SYSTEMS

License

The scientific journal “Electronics and control systems” adheres to the principles of Open Access and provides free, immediate, and permanent access to all published materials without financial, technical, or legal barriers for readers.

All articles are published in Open Access under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Copyright

Authors who publish their works in the journal “Electronics and control systems”:

  • retain the copyright to their publications;

  • grant the journal the right of first publication of the article;

  • agree to the distribution of their materials under the CC BY 4.0 license;

  • have the right to reuse, archive, and distribute their works (including in institutional and subject repositories), provided that proper reference is made to the original publication in the journal.