Generic placeholder image

Recent Patents on Computer Science


ISSN (Print): 2213-2759
ISSN (Online): 1874-4796

Research Article

A New Deep Neural Network Based on Multi-Layer Echo State Network

Author(s): Ronghui Liu* and Junmin Zhao

Volume 11, Issue 1, 2018

Page: [44 - 51] Pages: 8

DOI: 10.2174/2213275911666180507112411

Price: $65


Background: Deep Neural Network (DNN) has attracted great attention in regression and classification problems. However, the traditional DNN fails to provide favorable prediction performance owing to their inherent shortcomings of the Back Propagation (BP) algorithm, such as slow convergence and local optimum. To solve this problem, a novel DNN algorithm called Multi-Layer Echo State Network (ML-ESN) is proposed.

Method: This method utilizes Echo State Network (ESN) based unsupervised and supervised learning process. After discussing some related patents and methods, the ML-ESN structure is given. In the ML-ESN with M+1 hidden layers, ESN based Auto-Encoder (ESN-AE) is employed by the front M hidden layers for feature extraction in the unsupervised learning process, which can effectively extract abstract features from the original data. To overcome the under-fitting and over-fitting problem of ESN based classifier, the Penalty Factor ESN (PF-ESN) is presented to act as classifier by the M+1th hidden layer in the supervised learning process.

Results: The experiments demonstrate that ML-ESN outperforms ESN, Tikhonov regularization ESN (TR-ESN) and some DNN models such as Deep Belief Network (DBN) and stacked auto encoder (SAE). Owing to the fast learning speed of ESN, the training time consumtion of ML-ESN is also shorter than DBN and SAE.

Conclusion: The method provides references for fast and high accuracy classification and feature extraction.

Keywords: Echo state network, auto-encoder, penalty factor, deep neural network, pattern identification, back propagation.

Graphical Abstract

Rights & Permissions Print Export Cite as
© 2023 Bentham Science Publishers | Privacy Policy