Background: Deep Neural Network (DNN) has attracted great attention in regression and
classification problems. However, the traditional DNN fails to provide favorable prediction performance
owing to their inherent shortcomings of the Back Propagation (BP) algorithm, such as slow
convergence and local optimum. To solve this problem, a novel DNN algorithm called Multi-Layer
Echo State Network (ML-ESN) is proposed.
Method: This method utilizes Echo State Network (ESN) based unsupervised and supervised learning
process. After discussing some related patents and methods, the ML-ESN structure is given. In
the ML-ESN with M+1 hidden layers, ESN based Auto-Encoder (ESN-AE) is employed by the front
M hidden layers for feature extraction in the unsupervised learning process, which can effectively
extract abstract features from the original data. To overcome the under-fitting and over-fitting problem
of ESN based classifier, the Penalty Factor ESN (PF-ESN) is presented to act as classifier by the
M+1th hidden layer in the supervised learning process.
Results: The experiments demonstrate that ML-ESN outperforms ESN, Tikhonov regularization
ESN (TR-ESN) and some DNN models such as Deep Belief Network (DBN) and stacked auto encoder
(SAE). Owing to the fast learning speed of ESN, the training time consumtion of ML-ESN is
also shorter than DBN and SAE.
Conclusion: The method provides references for fast and high accuracy classification and feature