IEEE TRANSACTIONS ON SYSTEMS, MAN,AND CYBERNETICS, VOL. 23, NO. 1, JANUARY/FEBRUARY 1993

204

Correspondence Dynamic System Identification Using Neural Networks Yd -

Takayuki Yamada and Tetsuro Yabuta

Abstract-Many studies have heen undertaken in order to apply both the flexibility and learning ability of neural networks to robot controllers. The application of neural networks to servolevel controllers has also been studied. Therefore, we have proposed a neural network direct controller as a servolevel controller and confirmed its characteristics such as stability. However, there is also possibility of applying to the identi6cation of a nonlinear plant. Thus, a practical neural network design method for the identification of both the direct transfer function and inverse transfer function of the object plant is proposed. A nonlinear plant simulator as a practical application of the direct transfer function identilier using a ueural network is also proposed. Simulated and experimental results for a second-orderplant show that these identifications can be satisfactorily achieved. They also confirm that neural network identiBen can represent nonlinear plant characteristics very well. The characteristics of a neural network direct controller with a feedback control loop is also proposed and confirmed, which uses the learning results of the inverse transfer tunetion identi6er.

Fig. 1. General control system (parallel type).

I. INTRODUCTION

A neural network is modeled on a biological neural network and has excellent capabilities such as nonlinear mapping. Recently, many studies such as Kawato’s work have been undertaken in order to apply both the flexibility and learning ability of neural networks to robot controllers [11-[6]. However, studies on neural networks as servolevel controllers are still in their early stages. On the other hand, there are many studies on the servolevel controller based on conventional control theories such as adaptive control, learning control and fuzzy control. If it is possible to make an interface between neural networks and conventional control theories, neural networks can not only use the results of conventional theories, but also develop in combination with them. Therefore, we have proposed a neural network direct controller as a servolevel controller in order to compare neural network controllers with conventional control theories. We have also confirmed its characteristics such as stability [7],[8]. However, it is necessary to take into account not only the controller but also the identification of the nonlinear dynamics system through the use of the neural network. However, no such studies have yet been completed and only Jordan has begun to study this problem [9]. This paper proposes practical design methods for the identification of both the direct and inverse transfer functions of a dynamic system through the use of a neural network. Simulated and experimental results confirm that these identifications can be satisfactorily achieved. This paper proposes a nonlinear simulator of a dynamic system as a practical application of the direct transfer function identifier. This paper also proposes a neural network controller with a feedback loop, which uses the learning results of the inverse transfer function identifier. With this control method, this feedback loop is expected to compensate for the remaining control error caused by identification error. Experimental results confirm the control characteristics of this scheme. Manuscript received March 16, 1990; revised March 6, 1992. The authors are with NTT TelecommunicationField Systems R&D Center, Tokai-Mura, Naka-Gun, Ibaraki-Ken, 319-11, Japan. IEEE Log Number 9202115.

Fig. 2. General control system (series type). 11. CLASSIFICATION OF CONTROLLERS

Figs. 1 and 2 show a parallel and a series controller, respectively. In this control scheme, both feedforward and feedback controllers are tuned through the use of plant parameters estimated by observers (identifiers). Kawato has applied a neural network to the parallel controller shown in Fig. 1 [4]. His controller uses the neural network as a feedforward block. At the beginning of learning, the feedback block guarantees stability. After learning, the system is mainly controlled by the neural network. With the controller shown in Fig. 2, the feedforward block acts as a dynamic filter of the desired value for control. One typical example is that the feedforward block is used as an inverse kinematics solver of robot manipulators. Although many studies have been concerned with the feedforward feedback control scheme, we can realize a neural network feedforward controller without a feedback block [7], [8]. There have been many attempts to design a controller using a neural network, but the characteristics and stability of neural network observers have not yet been clarified. As PDP type neural networks are trained to minimize error energy functions, observers (identifiers) using neural networks are classified into two types by definition of their error energy functions. One type is trained to minimize the squared error (energy function) between a neural network output and a plant output. Fig. 3 shows this type of observer that is defined as a direct transfer function identifier in this paper. In this type of identifier, the neural network output converges with the plant output after learning and the direct transfer function of the plant is composed in the neural network. The other type is trained to minimize the squared error (error energy function) between the neural network output and the plant input. Fig. 4 shows this type of observer that is defined as an inverse transfer function identifier. In this type of identifier, the neural network output converges with the plant input after learning and the inverse transfer function of the plant is composed in the neural network.

0018-9472/93$03.00 0 1993 IEEE

205

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 23, NO. 1, JANUARYffEBRUARY 1993

U

Y

Plant

L

Position

1

Actual Position 'I'ocess

Trajectory

Neural Network

Time Fig. 6 . Concept of learning type

Fig. 3. Scheme of direct transfer function identifiet

U

1

- Y

Plant

I *

Neural Network

Fig. 7. Concept of cost function type.

i Fig. 4. Scheme of inverse transfer function identifier.

Position

1

,Actual Position /Desired Trajectory

V

I

Time Fig. 5. Concept of adaptive type.

Observers are also classified into two types by learning method [lo].One type learns at every sampling time. Fig. 5 shows this type of observer that we call the adaptive type. As shown in Fig. 5, the neural network output converges with the teaching signal within one trial. The other type learns at every trial period. Fig. 6 shows this type of observer that we call the learning type. As shown in Fig. 6, the neural network output converges with the teaching signal after several trials. As the converging speed is of no importance in this observer, a cost function type identifier can be realized because a cost function needs much calculation. Fig. 7 shows one example of the learning type, which can operate as a self-tuning controller. In particular, this paper proposes practical design methods for both direct and inverse transfer function identifiers using neural networks and confirms their characteristics. 111. BASICCONCEPT

A neural network design based on a PDF' model is selected with a view to control systems application [ll]. There have been many studies on the PDP model with regard to such problems as clustering

and logical function. However, no studies on neural networks as servolevel controllers have yet been completed. When we attempt to realize a servolevel controller using neural networks, the following points must be considered. It is necessary to use a neural network that can deal with analog signals because both plant input and output are analog. It has been proved that a neural network can express any nonlinear logical function if the neural network is assumed to have infinite neurons [12]. However, it is difficult for an actual system to satisfy this assumption. Thus, the required size of the neural network has to be determined. A neural network has a nonlinear sigmoid function in order to achieve a nonlinear mapping ability. This makes it very difficult to analyze the neural network. When a neural network estimates an unknown plant, it needs the input and output of the plant for the teaching signal and the neural network input. However, it is impossible to predetermine the spectra of the input and output of an unknown plant. Thus, it is necessary to establish a practical design method, in which learning results do not depend on the input and output spectra of the plant. to point l), signals in the neural network are assumed to be analog. Regarding point 2), the required size of a neural network is defined as the minimum size in which the weights between the neurons are such that the neural network output matches the teaching signal. With regard to point 3), the dominant part of the object plant is assumed to be linear so that we can design the neural network for a linear system. In this semi-linear approach, we can analyze the main characteristics of the neural network theoretically. The nonlinear effect is studied on the basis of the linear neural network in addition to the nonlinear sigmoid function by use of both simulation and experiment. Regarding point 4), a teaching signal (which is the plant output for the direct transfer function identifier and the plant input for the inverse case) is divided into the parameter of an unknown plant and a signal vector, based on assumption 3). The unknown plant parameter is learned.

206

IEEE TRANSACTIONS ON SYSTEMS,MAN,AND CYBERNETICS, VOL. 23, NO. 1, JANUARYIFEBRUARY 1993

These are the basic assumptions of the practical design methods. Based on assumption 4),the teaching signal T is divided into both the unknown vector a and the signal vector I by the following:

The teaching signal Y(k)(the plant output) is divided into both the unknown vector a of the plant and the signal vector I(k) as (8)

Y(k)= a T I ( k ) T = aTI.

(1)

This signal vector I is the neural network input vector. If we use the assumption that the neural network is linear, we can divide the neural network output Noutinto both weight matrix W and neural network input vector I by the following:

Nout= W T I . Thus, error given by

E

(2)

between the neural network and the actual plant is

E

= (T

- Nout) = (aT - W T ) I .

(3)

As mentioned previously by using the linear assumption, we can prove the existence of the neural network weights, which can be obtained independent of the input signal I, where the error E is zero. A suitable input vector I is discussed in the following section.

IV. CONTROLLER DESIGN METHOD This section describes general design methods for both the direct and inverse transfer function identifiers. The object plant is a predominantly linear SISO plant system. The leaming is performed every sampling time. Since we assume that the dominant part of the plant is linear, the neural network identifiers are designed for a linear system. The basic neural network structure is a three layer, linear PDP type. As to the nonlinear effect, the sigmoid function is added to the designed linear neural network. The transfer function of the object plant is assumed to be expressed as

A(2-l) =

1

+ *=1 m ..

B(z-') = 1

+ 1b , C i

+ +

where I ( k ) and a are w[= n m 11 order vectors. On the other hand, when the neural network is linear, the neural network output y,(k)iS

YN(k) w T ( k ) W ( k ) r ( k )

(4)

(9)

where the weights between neurons w ( k ) and W ( k ) are w x 1 order vector and u x w order matrix, respectively. The error ~ ( k = ) Y(k)- YN(k)is revised as E(k)

= Y ( k )- YN(k) = a T I ( k ) - wT(k)W(k)I(k) = {a* - W T ( k ) W ( k ) } I ( k ) .

(10)

Equation (10) indicates that the error ~ ( k )becomes zero when the relation aT = w T ( k ) W ( k ) is satisfied. This fact implies the existence of a neural network weight matrix that makes the error ~ ( kzero. ) As shown in (lo),the selection of the weight matrix can be performed independent of both the plant input and output when the signal vector defined by (6) is used as the neural network input. The relation aT = w T ( k ) W ( k )indicates the existence of infinite weight matrixes w ( k ) and W(k), which satisfy this relation. This means that the weight matrixes w ( k ) and W ( k )directly correspond with vector a of the unknown parameter. To cause the weights w T ( k ) W ( k ) to converge with a*, the learning is undertaken every sampling time using the S rule as follows [ll].

W(k w(k

A(t-')Y(k) =~-~Gol?(z-~)U(k)

(8)

+ 1) = W ( k )+ qAW(k) + 1) = w ( k ) + qAw(k)

A W ( k )= - d E / d W ( k ) Aw(k) = - d E / d w ( k ) E = E2(k)

(11) (12)

(13) (14)

i=l

where q is the parameter related to the converging speed. Fig. 8 shows a block diagram of the direct transfer function identifier, which is applied to an n = 2, m = 1, d = 1 plant. Due to the result of (6), the neural network input is composed of both the plant input and output information. The neural network using the error E between the learning is performed with (11H14) n plant output and the neural network output. The broken line in Fig. Y(k)= - xa,Y(k- i) +Go 8 is a feedback loop to stabilihe the system during learning, which z=1 also guarantees the finiteness of the neural network input. If the plant m output is finite without the feedback loop, this feedback loop is not U(k - d) b,U(k - i - d ) (5) necessary. Although the previous design method is applied to a linear i=l system, the neural network can be extended into the nonlinear field when the nonlinear sigmoid function is added to this linear neural A. Direct Transfer Function Identifier network framework. A detailed investigation of the nonlinear sigmoid When the plant input is used as the neural network input, and function is presented in Section V. the neural network output Y N ( ~converges ) with the plant output, A nonlinear simulator that uses the result of the direct transfer the direct transfer function of the unknown plant is composed in the function identifier can be obtained when the object plant has nonlinear neural network. In this identifier, the plant output Y (k) becomes the and parasite terms. Fig. 9 shows a block diagram of a nonlinear teaching signal. By using the result of (5), both the neural network simulator applied to an n = 2, m = 1,d = 1 plant. This plant input vector I ( k ) and the unknown parameter vector a are defined simulator uses the result of the trained identifier shown in Fig. 8 by the following equations: without learning. The broken line of Fig. 9 is necessary when the identifier shown in Fig. 8 uses a feedback loop. As shown here, I T @ ) = [U(k-d),Y(kl),...,Y(k-n), we can obtain a nonlinear neural network simulator that can not be U ( k - d - 1) , U ( k m - d ) ] (6) obtained through the use of a conventional method such as adaptive aT = [Go,al,...,a,,Gobl,...,Gob,] . (7) control. where a, and b, are unknown parameters and d is dead-time. This is analyzed using the following assumptions 1. The upper limit orders of the object plant n,m are known, and 2. The dead-time of the object plant d is known. The plant output Y ( k )can be expressed by the following.

}

+

-

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 23, NO. 1, JANUARYFEBRUARY 1993

207

r----------- 1 I

w

I

Fig. 10. Block diagram of inverse transfer function identifier.

Fig. 8. Block diagram of direct transfer function identifier.

- - - - - - - - - - --- - 1

I

I

Fig. 9. Block diagram of simulator.

B. Inverse Transfer Function Identifier When the plant output I'(k) is used as the neural network input and the neural network output I" ( k ) converges with the plant input, the inverse transfer function of the unknown plant is obtained in the neural network. Thus, we select the plant input 1 7 ( k )as the teaching signal. We can obtain the following (15) using (2) n

U

111

U(k-d) =(l/Go)~(k)+Ca,l'(k-i)-GoCb,I-k-/-d. 2=1 ,=1 (15) Both the neural network input vector I ( k ) and the unknown parameter vector cy are defined in the same way as the direct transfer function identifier:

[ 1 7 ( k ) , I ' ( k - 1 ): . ' . I ' ( k - n ) . l r ( k - 1- d ) .. . . l - ( k - m - d ) ] aT = l / G o [ l , a l ;... a , . G o b l .....Gob,,,].

IT(k)=

(16) (17)

The teaching signal L ' ( k ) (the plant input) is divided into both unknown vector a and the signal vector I ( k ) . ( [ - ( k - d )= n ' l ( k ) ) . On the other hand, when the neural network is linear, its output U N ( ~is) obtained by

l - , v ( k )= d J T ( k ) W ( k ) I ( k ) .

(18)

The output error ~ ( k=) U ( k - d ) - l * v ( k ) is revised by the following equation.

&(IC) = U ( k - d - r v ( k ) = c r T I ( k )- w T ( k ) W ( k ) l ( k ) = {a' - J ( k ) W ( k ) } 1 ( k ) .

(19)

When the relation aT = w T ( k ) W ( k )is satisfied, the error ~ ( k becomes zero. This fact implies the existence of a neural network weight matrix that makes the error ~ ( kzero. ) The weight matrix

selection can be performed independent of both the plant input and output when the signal vector defined by (16) is used as the neural network input. In order to make the weights , ( k ) ' W - ( k ) converge with n T ,the learning is undertaken every sampling time using the 5 rule following (1 lF(14). Fig. 10 shows a block diagram of the inverse transfer function identifier applied to an 11 = 2, m = 1, d = 1 plant. As seen here, the neural network input is composed of both the plant input and output information. Neural network learning is performed in order to minimize the error c between the plant input C ( k - d ) and the neural network output 1 - x ( k ) .The broken line in Fig. 10 is a feedback loop to stabilize the system. This feedback loop also guarantees the finiteness of the neural network input. If the plant output is finite without the feedback loop, this loop is unnecessary. As mentioned above, we can realize an inverse transfer function identifier. The most important application of the identified inverse transfer function result is in the direct controller [SI. Only the local convergence of the weight matrix is guaranteed for the direct controller [SI. Thus, when the neural network begins to learn through the use of some initial weight matrixes, the error sometimes does not reach zero even after learning because of the local minimums problem. To avoid this problem, it is necessary to learn from initial weight matrixes in which the output error is nearly zero. If the direct controller uses the learning results of the inverse transfer function identifier as the initial weights, it is expected that the convergence of the direct controller will be guaranteed. This is because the initial weights stay within the local convergence condition. Therefore, the convergence of the neural network direct controller is guaranteed and the output error of the direct controller achieves zero. V. SIMULATION In this section, both direct and inverse transfer function identifiers are simulated using a second-order plant in order to confirm their realization and to discuss their characteristics. The differential equation of the second-order plant for this simulation is as follows:

where n3 and C,,,,,, are the coefficients of the parasite and nonlinear terms, respectively. In this simulation, we choose a1 = -1.3. a2 = 0.3 and b = 0.7. A . Simulation of Direct Transfer Function Identifier Fig. 11 shows the learning results of the direct transfer function identifier whose block diagram is shown in Fig. 8. In this simulation, the plant is linear ( a = 0 , C,,,= 0) and the neural network does not use the nonlinear sigmoid function. As shown in Fig. 11, the ) neural network output I r v ( k )converges with the plant output Y ( k ) as learning progresses. This result indicates that the direct transfer function identifier can be realized. Fig. 12 shows a learning result

IEEE TRANSACTIONS ON SYSTEMS, MAN,AND CYBERNETICS, VOL. 23, NO. 1, JANUARYFEBRUARY 1993

208

- Netwbrk

c,

a

P

xg=20

4-

output

a P c, a

z

c,

*g

*s

2

t

0 E

U

I

0

g=o.1

- Network output ---Plant output

0

%

c,

c

0 c,

(A

E

q=0.3 0

Q

I

I

100

200

h 0

200

100 Sampling number

Sampling number Fig. 11. Learning process of direct transfer function identifier.

' Fig. 13. Learning process of direct transfer function identifier applied to nonlinear plant.

10

q=o.1 h

K

v

w-

+. w

z o

a output ---- Network Plant output 0

100

-10 -10

0

10

Input x Fig. 14. Input-output relationship of nonlinear function.

200

Sampling number Fig. 12. Learning process of direct transfer function identifier using different initial value.

using different initial weights under the same simulation conditions.

As shown in Fig. 12, the output error decreases as learning progresses even if different initial weights are used. However, a relatively large output error remains after the learning has stopped. It is supposed that the neural network enters the local minimum state and leaming is stopped. The converged weights wT W in the Fig. 12 simulation are very different from the plant parameter a. This result means that the conventional control theories such as adaptive control can not use the leamed neural network weights as the identified plant parameters. This is because the leamed weights are different from the true plant parameters if inadequate initial weights are used. Fig. 13 shows the simulation result for a nonlinear plant where a3 = 0.05 and C,,, = 0.1 in (20). Only the hidden layer of the neural network has a nonlinear input-output function as follows.

Parameter X g changes the shape of the nonlinear function whose input-output relationship is shown in Fig. l.4. As shown in Fig. 14, when X g ---* 00, the nonlinear function (21) is equal to the linear function f(z) = z [7],[8]. The learning result in Fig. 13 shows that the nonlinear neural network output converges with the plant output after learning though the plant includes both the nonlinear and parasite terms. The follawing discussion is concerned with a nonlinear simulator that uses the result of the direct transfer function identifier. Fig. 15

U

0

I

100

J

200

Sampling number Fig. 15. Nonlinear simulator ushg results of direct transfer function identifier.

shows the relation between the true plant output and the nonlinear simulator output using the identified result from Fig. 13. The feedback loop is used to stabilize the system with h'p = 0.2 in this simulator. Although the nonlinear simulator output exhibits similar behavior to the plant output, a relatively small error between the nonlinear output and the plant output remains. This is because both the nonlinear and parasite terms prevent perfect identification. Fig. 16 shows the relation between the nonlinear simulator output and the true plant output where the weight matrixes are the same as those in Fig. 15 with a different feedback gain K p = 0.4. Both outputs

209

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS, VOL. 23, NO. I , JANUARYFEBRUARY 1993

- Plant output I

+-

x9=20

I

I

---- Simulator (Network) output

I

a

Neural

P c 3

-1

0

+E

m n U c m

b

Fig. 18. Direct controller using results of inverse transfer function identifier.

-cm

z

iij

100

200

-Plant output

---- Desired value

K,=O

Sampling number Fig. 16. Nonlinear simulator using results of direct transfer function identifier.

q=0.75

- Network output - _ - _Plant input 0

200

400

Sampling number Fig. 19. Simulation result of direct controller.

I

I

0

100

2 10

Sampling number Fig. 17. Process of inverse transfer function of identifier.

+-

a

P c 3 . 0

+r

m

are different from those in Fig. 15 because of the difference in the feedback gain lip. However, the nonlinear simulator exhibits behavior similar to that of the true plant though the feedback gain is different. This result indicates the usefulness of the proposed nonlinear simulator.

B. Simulation of Inverse Transfer Function Identifier Fig. 17 shows the learning result of the inverse transfer function identifier where both the plant and the neural network are linear. As shown in Fig. 17, the neural network output converges with the plant input as learning progresses. This result means that the proposed inverse transfer function identifier can be realized. As mentioned in Section IV, we can use the learned weights of the inverse transfer function identifier as the initial weights of the direct controller. Fig. 18 shows a block diagram of the direct controller without learning. In such a case, a proportional control feedback loop is added to compensate for the identification error. Fig. 19 shows the relation between the plant output and the desired value As shown in Fig. 19, where the feedback gain l