Expert Systems with Applications 38 (2011) 12189–12194

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

A two-stage algorithm integrating genetic algorithm and modified Newton method for neural network training in engineering systems Ching-Long Su a,⇑, S.M. Yang b, W.L. Huang b a b

Department of Information Management, Chang Jung Christian University Tainan, Taiwan Institute of Aeronautics and Astronautics, National Cheng Kung University, Taiwan

a r t i c l e

i n f o

Keywords: Adaptive genetic algorithm Modified Newton method Neural network Engineering systems

a b s t r a c t A two-stage algorithm combining the advantages of adaptive genetic algorithm and modified Newton method is developed for effective training in feedforward neural networks. The genetic algorithm with adaptive reproduction, crossover, and mutation operators is to search for initial weight and bias of the neural network, while the modified Newton method, similar to BFGS algorithm, is to increase network training performance. The benchmark tests show that the two-stage algorithm is superior to many conventional ones: steepest descent, steepest descent with adaptive learning rate, conjugate gradient, and Newton-based methods and is suitable to small network in engineering applications. In addition to numerical simulation, the effectiveness of the two-stage algorithm is validated by experiments of system identification and vibration suppression. Ó 2011 Elsevier Ltd. All rights reserved.

1. Introduction A three-layer feedforward neural network with sufficient neurons in the hidden layer has been shown to have function approximation capabilities to any degree of accuracy. There have been considerable interests in system identification and control by artificial neural network. Given a set of training patterns, a neural network can ‘‘learn’’ about the system dynamics by adapting the network parameters. Most of the network design efforts have been on algorithm selection for minimal iterations and better convergence in computation. Yang and Lee (1997) applied the backpropagation algorithm to three neural networks for system identification, state estimation, and vibration suppression of a smart structure. They then developed an optimal neural network by combining Taguchi method (Yang & Lee, 1999) and the backpropagation algorithm with adaptive learning rate and momentum factor (Yang & Lee, 1998). For applications to identification and control of engineering systems, one of the emphases is to find an effective network training algorithm. Many high order methods have thus been proposed. Jeng, Yang, and Lin (1997) employed the Levenberg–Marquardt algorithm in neural network design. Similar numerical method using the second order Jacobian matrix was also reported (Johansson & Runesson, 2005). Stager and Agarwal (1997) applied three gradient-based training methods: steepest descent, conjugate gradient, and Gauss–Newton to speed up network training.

The Newton-type iterative method is often applied to minimize the discrepancy between measured and simulated response of nonlinear systems (Nordstrom, Johansson, & Larsson, 2007). Neural network integrated with fuzzy logic has also been developed (Yang, Chen, & Huang, 2006). More recently, genetic algorithm has been applied to feedforward (Maniezzo, 1994) and radial basis function neural networks (Billing & Zheng, 1995). Integration of genetic algorithm with conjugate gradient, fuzzy logic, and Newton–Raphson method has also been proposed (deLima, Jacob, & Ebecken, 2005; Rovira, Valdes, & Casanova, 2005; Shapiro, 2002). The above studies are mainly in two categories. The first aims at developing ad hoc techniques such as varying the learning rate, momentum and rescaling variables to improve the training speed, while the second focuses on optimization techniques. In these algorithms, the initial connection weight and bias are known to be critical to network training, and hence learning performance. An adaptive genetic algorithm is applied in this paper to find the global minimum of the error function of a feedforward neural network. To accelerate the training process, a modified Newton method, similar to BFGS algorithm (AlBaali, 2000; Bortoletti, Difiore, Fanelli, & Zellini, 2003) is then applied to improve training speed. The two-stage algorithm for efficient training is to integrate the adaptive genetic algorithm in the first stage for better initial weight/bias and the modified Newton algorithm in the second stage for efficient training. 2. Neural network and genetic algorithm

⇑ Corresponding author. Tel.: +886 919 789 338. E-mail addresses: [email protected] (C.-L. Su), [email protected] (S.M. Yang). 0957-4174/$ - see front matter Ó 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2011.03.073

Artificial neural network is an information processing system with a large number of simple connection neurons to imitate the

12190

C.-L. Su et al. / Expert Systems with Applications 38 (2011) 12189–12194

biological neural network. The relationship between the input and output pattern of an artificial neuron can be represented by nj ¼ P i wij xi  hj and yj = g(nj) where nj is the jth linear combination output; xi is the ith input of the processing element; hj is the bias representing the transfer function threshold; wij is the connection weight between the ith input and the jth output for imitating the biological synapse strength; yj is the jth output of the processing element; and g(.) is the transfer function (or the activity function) for converting the weighted summation of input to output. The connection weight represents the relative strength between two artificial neurons. The training process is to repeatedly adjust the P0 connection weights for minimizing the error function Ep ¼ 12 ni¼1 p p 2 p p ðt i  oi Þ ; p ¼ 1; 2; . . . ; m where t i and oi are the desired and network output of the ith output neuron for the pth pattern, respectively, n0is the number of neurons of the output layer, and mis the number of training patterns. A neural network contains many processing neurons such that a higher degree of robustness and fault tolerance is possible. There remain few guideline in providing the initial weight and bias, and they are usually generated randomly. Backpropagation algorithm using the steepest descent method often converges to a local minimum. Genetic algorithm, by comparison, performs a multi-directional search by maintaining a population of potential solutions for optimal solution. The algorithm is based on the mechanism of natural selection and genetics by evolving a population of individuals (chromosomes.) Each individual vi(i = 1, . . . , n) of the population that represents a trial solution of the problem is usually represented by a vector with each element called a gene. Every gene controls the inheritance of one or several characters. Conventional genetic algorithm of binary representation suffers from sacrificing precision with prohibitively long representation in multidimensional problems (Maniezzo, 1994). An alternative is to represent the vector in floating point number of the same length as the solution vector. 3. Initial weight and bias by genetic algorithm An adaptive floating genetic algorithm is developed to find the global solution of the fitness function represented by the inverse of the error function, Ef = 1/Ep, for optimal wij of a feedforward network. The algorithm can be described by reproduction, crossover, and mutation: 3.1. Reproduction Reproduction is a process in which the most highly rated chromosomes in the current generation are reproduced in the new generation. A roulette wheel method is used in the selection process by calculating the fitness value for each chromosome vi, i = 1, . . . , n, where n is population size. The total fitness of the populaP tion is defined by f ¼ eðv i Þ , where e(vi) is the fitness function. The probability of a selection pi for each chromosome vi is pi = e(vi)/f, and the cumulative probability qi for each chromosome vi is P qi ¼ i pj . For a random (floating) number a within [0, 1], if a < q1, then select the first chromosome v1, otherwise select the ith chromosome v i ð25i5nÞqi  1 < a5i for reproduction. 3.2. Crossover Crossover operator provides a mechanism for chromosomes to mix and match by random process. The operator is selected among: (1) the arithmetic crossover



x01 ¼ ax1 þ ð1  aÞx2 ; x02 ¼ ax2 þ ð1  aÞx1 ;

ð1Þ

where x1, x2 are the parents in vector form, x01 ; x02 are the offspring, and a is a random value in [0, 1], (2) the simple crossover by imposing that if x1 = [x1, . . . , xq]T and x2 = [y1, . . . , yq]T are crossed after the kth position, then the offspring are

(

x01 ¼ ½x1 ; . . . ; xk ; ykþ1 a þ xkþ1 ð1  aÞ; . . . ; yq a þ xq ð1  aÞT ; x02 ¼ ½y1 ; . . . ; yk ; xkþ1 a þ ykþ1 ð1  aÞ; . . . ; xq a þ yq ð1  aÞT

ð2Þ

and (3) the heuristic crossover using the value of the error function in determining the direction of the search and producing only one offspring,

x3 ¼ aðx2  x1 Þ þ x2 :

ð3Þ

The three crossover operators are selected adaptively. 3.3. Mutation Mutation operator is a random alteration of some gene values in a chromosome. It is selected among: (1) the uniform mutation that if the element xk of a parent x is selected for this mutation, then x0 ¼ ½x1 ; . . . ; x0k ; . . . ; xq T , where x0k is a random value (uniform probability distribution) in [lk, uk], (2) the non-uniform mutation with x0 ¼ ½x1 ; . . . ; x0k ; . . . ; xq T where

x0k ¼



xk þ Dðt; uk  xk Þ;

if the random digit is 0;

xk  Dðt; xk  lk Þ;

if the random digit is 1;

xk 2 ½lk ; uk  ð4Þ

and

 b t Dðt; yÞ ¼ ay 1  ; T

ð5Þ

where D(t, y) returns a value in [0, y] such that its probability is close to zero as the generation number t increases, T is the maximal generation number and b is a system parameter of the degree of non-uniformity, (3) the multi-nonuniform mutation with x0 ¼ ½x01 ; . . . ; x0k ; . . . ; x0q  where

x0k ¼



xk þ Dðt; uk  xk Þ;

if the random digit is 0;

k 2 ½1; q;

xk  Dðt; xk  lk Þ;

if the random digit is 1;

xk 2 ½lk ; uk  ð6Þ

and (4) the boundary mutation that if element xkof a parent x is selected for mutation, then x0 ¼ ½x1 ; . . . ; x0k ; . . . ; xq T ; xk 2 ½lk ; uk  where x0k is either lk or uk in equal probability. A genetic algorithm is capable of maximizing the fitness function by the evolution steps on each individual of the population. The first stage of the training algorithm is to find the initial weight and bias by the adaptive genetic algorithm in floating number. The reproduction, crossover, and mutation operations are selected adaptively in each generation. The multi-directional search of genetic algorithm usually takes lengthy training time; however, the algorithm can be used in the early stage of training to detect the region where the global minimum is likely to be found for good starting point to efficient learning. 4. Accelerated convergence by modified newton algorithm Conventional backpropagation algorithm and conjugate gradient algorithm are the first-order training algorithm for they require only the first derivatives of the error function. The so called second order algorithm such as Newton-based method and Levenberg– Marquardt algorithm are considered to be more efficient in neural network with a relatively small number of weight and bias. The update rule is defined as

Dwij ðk þ 1Þ ¼ gH1 ðkÞrEðwij ðkÞÞ;

ð7Þ

12191

C.-L. Su et al. / Expert Systems with Applications 38 (2011) 12189–12194

where E is the error function, g is the network learning rate, rE is the gradient, and H is the second partial derivatives Hessian matrix of E. Eq. (7) requires calculation and store of the Hessian matrix and its inversion at each step. Yet there is no guarantee that the matrix is nonsingular. Moreover, the application is seriously limited because finding the first and second derivatives of the error function are difficult, if not impossible. A modified Newton method similar to BFGS algorithm in numerical analysis and to Levenberg–Marquardt algorithm can be applied, while without computing the inverse matrix, in neural network training. Start with the initial weight/bias calculated in the first stage and denote the estimate of the inverse Hessian matrix as B. The update rule can be written by

Dwij ðk þ 1Þ ¼ gBðkÞrEðwij ðkÞÞ;

ð8Þ

where g is the network learning rate and rE(wij(k)) is the gradient vector. The inverse of the Hessian matrix is updated by

  qT ðkÞBðkÞqðkÞ pðkÞpT ðkÞ Bðk þ 1Þ ¼ Bðk þ 1Þ þ 1 þ pT ðkÞqðkÞ pT ðkÞqðkÞ 

pðkÞqT ðkÞBðkÞ þ BðkÞqðkÞpT ðkÞ ; pT ðkÞqðkÞ

by the two-stage algorithm until the difference of the network’s estimated output and the desired output is small enough. During ^ðkÞ very the validation phase, the network will produce the output y close to the plant output y(k) if the training is successful. Fig. 1 shows the results of sum-squared error during training between the plant and the network model by (1) the backpropagation algorithm (Yang & Lee, 1997), (2) the backpropagation algorithm with momentum term and adaptive learning rate (Yang & Lee, 1998), (3) the conjugation gradient algorithm (Jeng et al., 1997), (4) the modified Newton method, and (5) the two-stage training algorithm with population size n = 500 and 100 training cycles in the genetic algorithm. The backpropagation algorithm is in essence the steepest descent method and an improved version is to add the momentum and adaptive learning rate by considering the weights of the previous step. The conjugate gradient method is to minimize the error function by a search that is mutually conjugate to the previous search direction. The training pattern consists

ð9Þ 10

3

10

2

10

1

10

0

Two-stage training Modified Newton Conjugate gradient BPN with momentum and adaptive learning rate Backpropagation (BPN)

pðkÞ ¼ wij ðk þ 1Þ  wij ðkÞ;

ð10Þ

qðkÞ ¼ rEðwij ðk þ 1ÞÞ  rEðwij ðkÞÞ

ð11Þ

T

or if kp (k)q(k)k 6 e, where e is a positive constant, then B(k + 1) is the identity matrix. The modified Newton method is essentially by substituting the inverse Hessian matrix with an approximation from weight and gradient changes in iteration process. It combines the stability of gradient descent with the speed of Newton-based method. In practice, most of the network parameters are determined either by engineering experience or trial-and-error, and the network size is preferably small for real-time, online applications. The number of neurons in the input and output layers are pre-defined by the number of sensor and actuator in engineering system and are thus small. The number of hidden layer neuron is often selected small as well for more neurons do not necessarily yield better result because they may reduce the network generalizability and increase training time/complexity. The two-stage algorithm integrating the adaptive genetic algorithm in the first stage for better initial weight/bias and the modified Newton method in the second section stage for efficient training is desirable to engineering applications when network size is small for practical purposes.

Sum-Squared Error (SSE)

where

10

-2

10

-3

-4

0

500

1000

1500

2000

2500

Number of training cycle Fig. 1. The variations of sum-squared error of the difference equation Eq. (12) by using different algorithms.

0.8 plant network

0.6

An example of nonlinear system identification is applied to illustrate the two-stage algorithm. The plant is governed by the difference equation (Narendra & Parthasarathy, 1990)

0.4

Output

ð12Þ

where

x1 x2 x3 x5 ðx3  1Þ þ x4 Fðx1 ; x2 ; x3 ; x4 ; x5 Þ ¼ : 1 þ x23 þ x22

-1

10

5. Numerical and experimental verifications

yðk þ 1Þ ¼ FðyðkÞ; yðk  1Þ; yðk  2Þ; uðkÞ; uðk  1ÞÞ:

10

0.2 0

ð13Þ

A [5 5  1] feedforward neural network model, representing 5 neurons in the input layer, 5 in the hidden layer, and 1 in the output layer, is constructed to simulate the plant represented by the difference equation. The input to the plant and network model is assumed independent and identically distributed random signal uniformly ranged in [1, 1]. System identification by neural network contains two phases: the network training phase and the validation phase. The training phase is to find the connection weights

-0.2 -0.4

-0.6

0

50

100

150

Time step

200

250

300

Fig. 2. Plant output and the network model output of the difference equation.

12192

C.-L. Su et al. / Expert Systems with Applications 38 (2011) 12189–12194

of 500 input/output pairs and the training goes until the sumsquared error is smaller than 0.0001. Training fails to converge when it reaches the maximum training time, defined by the number of epochs required to meet the stopping criterion, before reaching the error bound. Fig. 1 shows that the two-stage algorithm is superior to the other algorithms in training efficiency. The maximum training time, minimum training time, and the associated mean and standard deviation were also recorded to evaluate the computation performance. The algorithm combines not only better initial weight and bias but also faster convergence by the modified Newton algorithm. After the training process, the validation phase is carried out by

uðkÞ ¼ 0:6 sinðpk=20Þ; ¼ 0:6; ¼ 0:6;

k < 100;

100 6 k < 150; 150 6 k < 200;

¼ 0:6 sinðpk=15Þ;

ð14Þ

200 6 k < 300:

The output of the plant and the network model are shown in Fig. 2, where the neural network identification is so accurate. This benchmark test shows that a concise [5 ⁄ 5 ⁄ 1] three-layer network with the two-stage algorithm is effective in system identification of the highly nonlinear system. 24 neural networks in different number of neurons were also constructed, and the identification results showed that the two-stage algorithm remains having the best performance among the training algorithms. The added cost of stowing B(k + 1) is outweighed by the advantage of best convergence. On average, there is a reduction of 100 epochs in the training process because of the better initial weight/bias calculated by the genetic algorithm in the first stage. Another benchmark test of nonlinear system (Zhang & Morris, 1995)

Fig. 3. (a) Variations of the sum-squared error of Eq. (15) in the training process and (b) the plant output and the network output of the difference equation Eq. (16).

8 x1 < 3; x2 < 3; > < x1 þ x2 þ 5; y ¼ 2x1  x2 þ 2; x1 > 3; x2 > 3; > : x1 þ 2x2 þ 2; otherwise

ð15Þ

Fig. 4. Comparison of the time response between the smart structure and identification network model under random excitation, where both are almost identical indicating effective neural network model.

C.-L. Su et al. / Expert Systems with Applications 38 (2011) 12189–12194

is also used to validate the performance of the two-stage algorithm. Fig. 3(a) shows the variations of sum-squared error during the training process in which the error decreases substantially and effectively by the combined genetic algorithm and the modified Newton’s method. The network model is validated by



x1 ¼ 8 sinð2kp=25Þ; x2 ¼ 6 sinð2kp=15Þ  2 cosðkp=35Þ;

for k ¼ 1; . . . ; 200;

ð16Þ

and the output of the plant and the model are shown in Fig. 3(b). It is seen that the two-stage algorithm performs well. The adaptive genetic algorithm and the modified Newton method in O (n2) can be efficiently applied in small network suitable for engineering applications.–Many studies have been conducted to develop the mathematical model for smart structures with embedded piezoelectric sensor/actuator, but modeling the embedding interfaces has been known unreliable because of the interface mechanics. Neural network is thus preferable. The smart structure is a [90/ 90/90/90/0]x2 glass fiber composite laminated beam of 265 

12193

40.5  1 mm. The composite beam is made of S-glass/epoxy uni-directional pre-preg tapes for accommodating the embedded piezoelectric elements: two embedded 120  30  0.375 mm piezoelectric actuators and one 13.5  25  0.375 mm piezoelectric sensor. Details of the fabrication process, mechanical properties of the laminate and the electro-mechanical properties of piezoelectric material can be found in (Yang & Lee, 1997). The neural network identification does not intend to identify the structure parameters (such as mass, damping ratio, stiffness), but instead to find the connection weight and bias of the feedforward neural network through the experimental input/output. The input of the network model is the system state of displacement, velocity and the excitation force _  1Þ; uðk  1Þ; uðk  2ÞÞ. One piezoelectric actuator is ðxðk  1Þ; xðk employed to drive the system by an independent and identically distributed random signal uniformly distributed over ±9 V, while another sensor at 100 Hz sampling rate is to acquire the system state. Experiments are conducted on the smart structure under clamp-free boundary condition. A [3  3  2] network with the two-stage algorithm is constructed for modeling the input/output

Fig. 5. (a) Block diagram of vibration suppression by neural network identification and control with the two-stage algorithm, (b) experimental verification of the smart structure by an initial tip displacement and (c) by random excitation.

12194

C.-L. Su et al. / Expert Systems with Applications 38 (2011) 12189–12194

transfer function. The comparison of time response between the actual plant and the network model is shown in Fig. 4, where the network is shown to identify successfully the dynamics of a composite smart structure with embedded piezoelectric sensor and actuator. The neural controller design is shown in Fig. 5(a). All that required for controller design is a neural network model and a reference model for desired system performance. The neural controller can then be self-organized via the output error between the plant and reference model. The desired response of the controlled plant with respect to a command input is specified in the reference model. The training algorithm should adjust the weight and bias of the neural controller to meet the design requirement. A [3 ⁄ 3 ⁄ 1] feedforward neural network with the two-stage training algorithm is constructed to train the controller by the sampling data at 100 Hz. The reference model is chosen as

€x þ 5x_ þ 2400x ¼ 2400uðtÞ

ð17Þ

in which the un-damped natural frequency near 7.8 Hz. Fig. 5(b) shows the free vibration response of the composite smart structure under the neural controller. Fig. 5(c) shows the forced vibration response under random excitation. The experimental results indicate that the neural network controller with the two-stage algorithm can reduce the structure vibration efficiently in real time. The identification network and the neural controller are in minimal order, [3  3  2] and [3  3  1], while remaining effective in dynamics modeling and vibration suppression. 6. Conclusions 1. A two-stage training algorithm is developed for feedforward neural network. The first stage is to calculate better initial weight and bias by using an adaptive genetic algorithm for a near optimal solution, and the second stage is to train the network by using a modified Newton method with the specific initial weight and bias. This two-stage algorithm includes the advantages of genetic algorithm in having a critical and better initial weight/bias and the modified Newton algorithm in accelerating the convergence of network training. In engineering applications, the network size is preferably small for real-time, online applications, the numbers of input/output neurons are pre-defined by the number of sensor/actuator and are thus small, so that the two-stage algorithm is shown desirable for practical purposes. 2. The adaptive floating genetic algorithm is developed to search for the initial weight and bias. The crossover operator is the combination of arithmetic crossover, simple crossover, and heuristic crossover, while the mutation operator is the combination of uniform mutation, non-uniform combination, multi-nonuniform mutation, and boundary mutation. These combinations facilitate the genetic algorithm to reach the optimal solution efficiently. The modified Newton method includes the advantages of the steepest descent algorithm and the Newton-based method. It avoids the drawback of having the second derivatives of the error function (the Hessian matrix) and the associated inverse. The two-stage training algorithm provides better initial weight and bias and faster convergence.

3. The benchmark tests show the two-stage algorithm is efficient and effective in neural network training. A concise three-layer [5  5  1] feedforward neural network is effective in modeling a nonlinear system and 24 neural networks with different number of neurons also show that the two-stage algorithm remains having the best performance. On average, there is a reduction of 100 epochs in the training process because of the better initial weight/bias calculated by the genetic algorithm. Another test of [2  3  1] network also confirms that the two-stage algorithm is desirable. The algorithm is also validated experimentally in system identification and vibration control of a composite smart structure with embedded piezoelectric sensor and actuator. Experimental results demonstrate that the algorithm makes the identification network and controller network efficient and effective. It should be noted that in image processing where substantial number of input/output nodes may be needed in network model, the two-stage algorithm may not be suitable for the genetic algorithm and the modified Newton method are of O(n2).

References Yang, S. M., & Lee, G. S. (1997). Vibration control of smart structure by using neural networks. Transactions of the ASME Journal of Dynamic Systems, Measurement, and Control, 119(1), 34–39. Yang, S. M., & Lee, G. S. (1999). Neural network design by using Taguchi method. Transactions of the ASME Journal of Dynamic Systems, Measurement, and Control, 121, 560–562. Yang, S. M., & Lee, G. S. (1998). System identification of smart structures using neural networks. International Journal of Intelligent Material Systems and Structures, 8(10), 883–890. Jeng, C. A., Yang, S. M., & Lin, J. N. (1997). Multi-mode control of structures by using neural networks with Marquardt algorithms. International Journal of Intelligent Material Systems and Structures, 8(12), 1035–1043. Johansson, H., & Runesson, K. (2005). Parameter identification in constitutive models via optimization with a posteriori error control. International Journal for Numerical Methods in Engineering, 62(10), 1315–1340. Stager, F., & Agarwal, M. (1997). Three methods to speed up the training of feedforward and feedback perceptrons. Neural Networks, 10(8), 1435–1443. Nordstrom, L. J. L., Johansson, H., & Larsson, F. (2007). A strategy for input estimation with sensitivity analysis. International Journal for Numerical Methods in Engineering, 69(11), 2219–2246. Yang, S. M., Chen, C. J., & Huang, W. L. (2006). Structure vibration suppression by neural-network controller with active mass-damper. Journal of Vibration and Control, 12, 495–508. Maniezzo, V. (1994). Genetic evolution of the topology and weight distribution of neural networks. IEEE Transactions on Neural Networks, 5(1), 39–53. Billing, S. A., & Zheng, G. L. (1995). Radial basis function network configuration using genetic algorithms. Neural Networks, 8(6), 877–890. Shapiro, A. F. (2002). The merging of neural networks, fuzzy logic, and genetic algorithms. Insurance Mathematics & Economics, 31, 115. deLima, B. D. L. P., Jacob, B. P., & Ebecken, N. F. F. (2005). A hybrid fuzzy/genetic algorithm for the design of offshore oil production risers. International Journal for Numerical Methods in Engineering, 64(11), 1459–1482. Rovira, A., Valdes, M., & Casanova, J. (2005). A new methodology to solve non-linear equation systems using genetic algorithm. International Journal for Numerical Methods in Engineering, 63(10), 1424–1435. AlBaali, M. (2000). Improved Hessian approximations for the limited memory BFGSmethod. Numerical Algorithms, 22, 99–112. Bortoletti, A., Difiore, C., Fanelli, S., & Zellini, P. (2003). A new class of quasiNewtonian methods for optimal learning in MLP-network. IEEE Transactions on Neural Networks, 14(2), 263–273. Narendra, K. S., & Parthasarathy, K. (1990). Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks, 1, 4–27. Zhang, J., & Morris, A. J. (1995). A fuzzy neural network for nonlinear system modeling. IEE Proceedings of Control Theroy and Applications, 142(6), 551–561.

ID

388480

Title A two-stage algorithm integrating genetic algorithm and modified Newton method for neural network training in engineering systems

http://fulltext.study/journal/431

http://FullText.Study

Pages

6