Deep learning
Proud news flash!!! All three volumes of my Deep Belief Net trilogy have been purchased by a major international publisher. Thus, they are no longer available from CreateSpace. They are now available from Amazon and all major booksellers.
Deep belief nets are one of the most exciting recent developments in artificial intelligence. The structure of these elegant models is much closer to that of human brains than traditional neural networks; they have a 'thought process' that is capable of learning abstract concepts built from simpler primitives. A typical deep belief net can learn to recognize complex patterns by optimizing millions of parameters, yet this model can still be resistant to overfitting.
My three volume set, "Deep Belief Nets in C++ and CUDA C" presents the essential building blocks of several of the most common forms of deep belief nets. At each step the text provides intuitive motivation, a summary of the most important equations relevant to the topic, and concludes with highly commented source code for threaded computation on modern multiple-core CPUs as well as massive parallel processing on computers with CUDA-capable video display cards.
Volume I
Volume I focuses on restricted Boltzmann machines as the primary building block of deep belief nets.
Volume II
Autoencoders are interesting and powerful alternatives to restricted Boltzmann machines. Volume II presents the essential theory, equations, and computer code for a particular type of autoencoder which is extensible to the complex domain. Volume II also presents several algorithms for preprocessing time series and image data. Finally, Volume II extends the restricted Boltzmann machine of Volume I by providing a method for embedding class information in the input layer of the top-level RBM.
Volume III
Volume III presents convolutional nets, a form of deep belief net that is especially adapted to classification of images. Source code for all routines presented in the books, and the DEEP and CONVNET programs which implement these algorithms, are available for free download.
To see and optionally order Volume I from Amazon.com, click here.
To see the Table of Contents of Volume I, click here.
For Volume II Table of Contents, click here.
For Volume III Table of Contents, click here.
DEEP Program
DEEP is a free program which has the ability to train, analyze, and test deep belief nets. Some notable features of Version 2 include:
- Ability to read standard-format database files such as Excel CSV files
- Ability to read labeled image files in MNIST format and optionally apply a two-dimensional Fourier transform
- Ability to read a time series file and compute trend and velocity of trend change for a moving window
- Ability to read a time series file and compute full complex Fourier coefficients in a moving window
- Ability to read a time series file and compute full complex Morlet wavelet coefficients in a moving window
- Models can be numerically predictive or employ SoftMax classification
- Contrastive divergence training of restricted Boltzmann machines (RBMs)
- Real or complex-domain autoencoding and regression
- Greedy training of stacks of RBMs and autoencoders
- Conjugate gradient training of one or more 'supervised' layers
- Optional supervised fine tuning of the complete network
- Cross validation with any number of folds, optional shuffling, and optional boundary guards
CONVNET Program
CONVNET is a free program which reads MNIST and CIFAR-10 format image files and allows the developer to design, train, and test convolutional nets. It can also read a time-series file and convert the data path into a sequence of images. If the user's computer has a CUDA-capable device, it will be used for training.
DEEP and CONVNET are works in progress. New capabilities will likely be added on a regular basis. Stay tuned.
Last but not least, please understand that DEEP and CONVNET are experimental programs. They are provided free of charge to interested users for educational purposes only. In all likelihood these programs contain errors and omissions. The developer disclaims all responsibility for losses which the user may incur.
These programs contain no embedded advertising or malware of any sort, nor do they report back to me any information about your computer.
To download the DEEP User's Manual (Chapter 6 of Volume II), click here.
To download the CONVNET User's Manual (Chapter 5 of Volume III), click here.
To download a zip file containing the DEEP program, User's Manual, and core source code, click here.
Please click here if you wish to download CONVNET 2.0 (zip file with EXE, source, and supplement).