Gaussian discrete restricted Boltzmann machine : theory and its applications : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Electronics and Computer Engineering at Massey University, Albany, New Zealand

Simple item page
dc.contributor.authorManoharan, Subha
dc.date.accessioned2015-07-29T02:19:58Z
dc.date.available2015-07-29T02:19:58Z
dc.date.issued2015
dc.description.abstractRestricted Boltzmann Machine (RBM) is a two-layer neural network, popular for its efficient training methodology in many applications involving data recall, classification, and recognition. Traditionally RBM is designed with binary neurons in both layers. RBMs with Gaussian (continuous-valued) neurons in visible layer have been introduced for ease of integration with real data. However, the hidden layer still consists of binary neurons. Recently, theoretical studies in discrete RBM with discrete visible and hidden nodes have shown that increasing the number of hidden states improves reconstruction error. Motivated by this finding, the research in this thesis aims to develop an RBM with a Gaussian visible layer and a discrete multi-state hidden layer, called the Gaussian Discrete RBM (GDRBM). The equations governing this new model have been worked out and a contrastive divergence training algorithm has been developed based on these equations. Performance results using the MNIST and CBCL benchmark datasets show that the performance of a GDRBM with 4-state hidden neurons is approximately the same as that of other Gaussian RBMs with binary hidden neurons when the size of the hidden layer is doubled. This GDRBM has also been used to form one layer of a deep autoencoder. This is the first time an autoencoder has been designed with a multi-state discrete layer. Initial experimental results show that a GDRBM-based deep autoencoder is able to reconstruct the inputs reasonably well. However the pretraining is not very effective and the amount of initial reconstruction error need to be reduced to make it perform at the same level of a traditional deep autoencoder. Further research will be needed to understand how GDRBM could be used in a deep autoencoder.en_US
dc.identifier.urihttp://hdl.handle.net/10179/6901
dc.language.isoenen_US
dc.publisherMassey Universityen_US
dc.rightsThe Authoren_US
dc.subjectRestricted Boltzmann machine (RBM)en_US
dc.subjectNeural networks (computer)en_US
dc.subjectMachine learningen_US
dc.subjectGaussian discrete RBMen_US
dc.titleGaussian discrete restricted Boltzmann machine : theory and its applications : a thesis presented in partial fulfilment of the requirements for the degree of Master of Engineering in Electronics and Computer Engineering at Massey University, Albany, New Zealanden_US
dc.typeThesisen_US
massey.contributor.authorManoharan, Subhaen_US
thesis.degree.disciplineElectronics and Computer Engineeringen_US
thesis.degree.grantorMassey Universityen_US
thesis.degree.levelMastersen_US
thesis.degree.nameMaster of Engineering (M.E.)en_US
Files
Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
01_front.pdf
Size:
57.84 KB
Format:
Adobe Portable Document Format
Description:
Download
Thumbnail Image
Name:
02_whole.pdf
Size:
1.78 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
license.txt
Size:
804 B
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Theses and Dissertations