Toward Deep Neural Networks

WASD Neuronet Models, Algorithms, and Applications

Yunong Zhang, Dechao Chen, Chengxu Ye

• Publisher: Chapman and Hall/CRC
• Year: 2019
• Print ISBN: 9780367656492
• eBook ISBN: 9780429760983
• Topic: Economics
• Subtopic: Statistics for Business & Economics
• Index: Economics

Toward Deep Neural Networks: WASD Neuronet Models, Algorithms, and Applications introduces the outlook and extension toward deep neural networks, with a focus on the weights-and-structure determination (WASD) algorithm. Based on the authors' 20 years of research experience on neuronets, the book explores the models, algorithms, and applications of the WASD neuronet, and allows reader to extend the techniques in the book to solve scientific and engineering problems. The book will be of interest to engineers, senior undergraduates, postgraduates, and researchers in the fields of neuronets, computer mathematics, computer science, artificial intelligence, numerical algorithms, optimization, simulation and modeling, deep learning, and data mining.

Features

• Focuses on neuronet models, algorithms, and applications
• Designs, constructs, develops, analyzes, simulates and compares various WASD neuronet models, such as single-input WASD neuronet models, two-input WASD neuronet models, three-input WASD neuronet models, and general multi-input WASD neuronet models for function data approximations
• Includes real-world applications, such as population prediction
• Provides complete mathematical foundations, such as Weierstrass approximation, Bernstein polynomial approximation, Taylor polynomial approximation, and multivariate function approximation, exploring the close integration of mathematics (i.e., function approximation theories) and computers (e.g., computer algorithms)
• Utilizes the authors' 20 years of research on neuronets

Part IV

General Multi-Input Neuronet

Chapter 9

Multi-Input Euler-Polynomial WASD Neuronet

9.1Introduction

9.2Theoretical Basis and Analysis

9.3MIEPN Model

9.4WDD Subalgorithm

9.5WASD Algorithm with PWG and TP Techniques

9.6Numerical Studies

9.7Chapter Summary

Appendix C: Detailed Derivation of Normal Equation

Appendix D: Supplemental Theorems

Differing from the conventional back-propagation (BP) neuronets, a multi-input Euler-polynomial neuronet, in short, MIEPN (specifically, four-input Euler-polynomial neuronet, FIEPN) is established and investigated in this chapter. In order to achieve satisfactory performance of the established MIEPN, a weights-and-structure-determination (WASD) algorithm with pruning-while-growing (PWG) and twice-pruning (TP) techniques is built up for the established MIEPN. By employing the weights-direct-determination (WDD) subalgorithm, the WASD algorithm not only determines the optimal connecting weights between hidden layer and output layer directly, but also obtains the optimal number of hidden-layer neurons. Specifically, a sub-optimal structure is obtained via the PWG technique, then the redundant hidden-layer neurons are further pruned via the TP technique. Consequently, the optimal structure of the MIEPN is obtained. To provide a reasonable choice in practice, several different MATLAB computing routines related to the WDD subalgorithm are studied. Comparative numerical results of the FIEPN using these different MATLAB computing routines and the standard multi-layer perceptron (MLP) neuronet furth...

Table of contents

• Cover
• Half Title
• Series Page
• Title Page
• Copyright Page
• Dedication
• Contents
• Preface
• Authors
• Acknowledgments
• I: Single-Input-Single-Output Neuronet
• II: Two-Input-Single-Output Neuronet
• III: Three-Input-Single-Output Neuronet
• IV: General Multi-Input Neuronet
• V: Population Applications Using Chebyshev-Activation Neuronet
• VI: Population Applications Using Power-Activation Neuronet
• VII: Other Applications
• Bibliography
• Glossary
• Index