Novel artificial neural network architectures and algorithms for non-linear dynamical system modeling and digital communications applications

Hussain, Amir

Thesis

Novel artificial neural network architectures and algorithms for non-linear dynamical system modeling and digital communications applications

Download PDF

Creator

Hussain, Amir

Rights statement

Strathclyde Thesis Copyright

Awarding institution

University of Strathclyde

Date of award

1996

Thesis identifier

T9173

Qualification Level

Doctoral (Postgraduate)

Qualification Name

Doctor of Philosophy (PhD)

Department, School or Faculty

Abstract

This thesis presents novel Artificial Neural Network (ANN) based architectures and algorithms for solving two important signal processing applications. Firstly, in the context of non-linear dynamical system modeling applications, a new two-layer linear-in-the-parameters, feedforward ANN structure is developed, which is termed the Feedforward Functionally Expanded Neural Network (FFENN), in order to efficiently model chaotic and equation-error type non-linear dynamical processes. A general design strategy is presented for specifying the type and number of non-linear basis functions within the FFENN’s single hidden layer, for an arbitrary number of network inputs. The FFENN structure employing the proposed basis functions in its hidden layer is essentially a hybrid neural network incorporating to a variable extent, the combined modeling capabilities of the conventional Multi-Layered Perceptron (MLP), Radial Basis Function (RBF) and Volterra Neural Networks (VNN). Its output mean squared error surface is shown to be uni-modal allowing high speed single-run least squares based learning. A new pruning strategy based on an iterative pruning-retraining scheme coupled with statistical model validation tests, is also devised in order to optimise the size of FFENN structures for non-linear dynamical system modeling applications. Numerous case studies are performed using simulated chaotic, equation-error and a variety of real-world noisy, non-stationary time series processes which show that the new FFENN based predictor models consistently outperform other recently reported, feedforward and recurrent ANN structures, both in terms of non-linear prediction ability and relative computational complexity requirements. To enable efficient modeling of a more general class of non-linear dynamical systems, a new computationally efficient Recurrent Neural Network (RNN) structure is also developed, which is termed the Recurrent Functionally Expanded Neural Network (RFENN). Its learning algorithm is derived and a pruning strategy proposed. The development of the RFENN is shown to result in a new class of computationally efficient RNNs incorporating all linear-in-the-parameters feedforward ANNs (such as the RBF and VNN) adapted to employ local output feedback. Various case studies are performed using simulated chaotic, output-error and real-world noisy times series processes, which show that the RFENN based predictor models can significantly outperform the corresponding feedforward FENN and other ANN predictors in the modeling of certain types of non-linear dynamical processes. The FFENN and RFENN are also successfully applied to the task of real-time adaptive non-linear prediction of real non-stationary signals. A new hybrid RFENN-FIR adaptive structure comprising the non-linear RFENN subsection feeding into a linear Finite Impulse Response (FIR) subsection is also developed and shown to outperform the stand-alone FFENN and RFENN based adaptive predictors. Secondly, in the context of digital communications applications, two new adaptive non-linear Decision Feedback Equalizer (DFE) structures are developed, which are termed: the Decision Feedback Functional-Link Equalizer (DFFLE) with Expanded Feedback Terms (DFFLE-EFT); and the DFFLE with Unexpanded Feedback Terms (DFFLE-UFT). The DFFLE-UFT employs the recently reported non-linear-in-the parameters Feedforward Functional-Link Equalizer (FFLE) as its feedforward filter and a linear feedback filter. In contrast, the novel DFFLE-EFT structure non-linearly combines both the equalizer input and decision feedback samples. Learning algorithms and general design strategies are presented for both the structures. Pruning techniques for optimizing the sizes of the FFLE and DFFLE structures are also proposed. In the first digital communications application considered in the thesis, the new structures are employed for the equalization of linear and non-linear communication channels in the presence of ISI and additive (uncorrelated and correlated) noise. In the second digital communications application considered in the thesis, the FFLE and DFFLE structures are proposed as a novel solution to the problem of overcoming co-channel interference in digital communications systems. Various simulation case studies are performed for both applications, which show that the new DFFLE-EFT is a viable alternative to the optimal symbol Bayesian Transversal Equalizer (TE) and all other ANN based TE structures (which have been reported to date for approximating the Bayesian TE).

Advisor / supervisor