AE566F14 Project Report

Data Analysis and System Identification

Gaurav Kumar Singh

AE566 C OURSE P ROJECT

U NIVERSITY OF M ICHIGAN , A NN A RBOR
This project report is written as a part of the course requirements of AE566 Data Analysis
and System Identification taught by Prof. Dennis Bernstein in fall 2014.
Copyrights reserved

Contents

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.1

Motivation

1.2

Objective

1.3

Structure of the report

Data Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1

Background of the Experiment

2.2

A Brief Review of Aerospace Control Systems

2.3

Description of the experiment

11

2.4

Description of the data-set

12

Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1

Knowing our Data

13

3.2

Removing Trend from Data

13

3.3

Data Analysis begins

14

Linear Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.1

Frequency Domain Identification

20

4.2

MIMO Least Squares Fitting

20

4.3

Toy Example

21

4.4

Identification of Markov Parameters

24

4.5

Eigensystem Realization Algorithm

25

4.6

N4SID algorithm

26

Nonlinear Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5.1

NARMAX

29

5.2

Input Augmentation using Basis Functions

31

5.3

Hammerstein Wiener Model Using Simulink

31

Conclusions and Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

6.1

Logical flow of Identification

35

6.2

Final Comments

36

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

R Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
MATLAB

List of Figures

2.1
2.2
2.3

Plane (Body Axis) - based about aircrafts CG: . . . . . . .

Aircraft coordinate systems and configuration. . . . . . . . .
Test aircraft. The tested aircraft has conventional wing-tail
configuration . . . . . . . . . . . . . . . . . . . . . . . . .

3.1

First hands on data. [a] shows the hovering movement of the aircraft in
earth coordinates, [b],[c],[d] show the inputs and outputs of the data. . . .
Pairwise coherence of input with outputs. Top to bottom: coherence for
elevator input, coherence for aileron input and coherence for rudder input
Power-spectrum of inputs and outputs. From left to right: power spectrum
of elevator, aileron and rudder; power spectrum of pitch, yaw and roll . .
Pair-wise cross correlation between inputs and outputs. From left to right:
cross-correlation with respect to elevator, aileron and rudder . . . . . . .
Pairwise scatterplots. From top to down: elevator-pitch system, aileron-roll
system and rudder-yaw system . . . . . . . . . . . . . . . . . . . . . . .
Partition of data for outputs for estimation and validation purposes . . . .
Partition of data for inputs for estimation and validation purposes . . . . .

3.2
3.3
3.4
3.5
3.6
3.7
4.1
4.2
4.3
4.4
4.5

. . .
. . .
with
. . .

. . . . . . . 10
. . . . . . . 10
tractor
. . . . . . . 11
. . . 14
. . . 15
. . . 15
. . . 17
. . . 18
. . . 19
. . . 19

Frequency domain identification. Notice the first subplot shows the magnitude of the response while the second one is phase of the response. . . . .
Estimation of output using MIMO least squares algorithm. The fits are not
great, interesting though . . . . . . . . . . . . . . . . . . . . . . . . . . .
Estimation of output for validation input. The fits become worse as expected
FFT of fit-error for MIMO system. High peaks are seen at 0 frequency . . .
Application of the MIMO least squares algorithm on a known linear system.
The algorithm looks perfectly alright . . . . . . . . . . . . . . . . . . . . .

. . 21
. . 22
. . 22
. . 23
. . 23

6
4.6
4.7

FFT of fit-error for toy system. No high peaks are seen . . . . . . . . . .

Estimation of output for elevator-pitch SISO system. Again, estimation
looks a bit off. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.8 Markov parameters of the system obtained by providing impulse input to
the estimated system . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.9 Estimation of markov parameters through ERA . . . . . . . . . . . . . .
4.10 Application of N4SID algorithm on our data set. Looks like N4SID barely
manages to stick to the mean value of the response . . . . . . . . . . . .
5.1

5.2

5.3

5.4
5.5

5.6
5.7

NARMAX model structure algorithm on elev-pitch system. From top to

bottom: NARMAX algorithm on estimation data; estimation of output from
derived model structure for validation data . . . . . . . . . . . . . . . . .
Input augmentation using basis functions on data for estimation for elevpitch SISO system. From left to right: measured output and estimated
output, scatter plots for measured output and estimated output . . . . . .
Input augmentation using basis functions on data for validation for elevpitch SISO system. From left to right: measured output and estimated
output, scatter plots for measured output and estimated output . . . . . .
Block diagram of the Hammerstein Wiener system identification created in
Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hammerstein Wiener Identification for un-detrended elev-pitch SISO system in Simulink. From left to right: for estimation data; for validation
data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hammerstein Wiener Identification for detrended elev-pitch SISO system
in Simulink. From left to right: for estimation data; for validation data . .
Pole-zero map of the identified Hammerstein Wiener Model for elev-pitch
SISO system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . 24
. . . 25
. . . 26
. . . 27
. . . 28

. . . 30

. . . 31

. . . 32
. . . 32

. . . 33
. . . 33
. . . 34

1. Introduction

1.1

Motivation
Developing mathematical models of physical systems based on imperfect observations or
measurements is known as system identification. System identification lets us create and
use models of dynamic systems not easily modeled from first principles or specifications.
We can use time-domain and frequency-domain input-output data to identify continuoustime and discrete-time transfer functions, process models, and state-space models.A good
practical definition of system identification is that of Zadeh [1] :
System identification is the determination, on the basis of observation of input
and output, of a system within a specified class of systems to which the system
under test is equivalent.
When the system to be modeled is an aircraft, the models are generally dynamic with
multiple inputs and outputs, and the measurements are noisy. In this project report, we wish
to identify a system from the data obtained from a fixed-wing aircraft hovering over a place.

1.2

Objective
For many applications, an aircraft can be assumed to be a rigid body, whose motion
is governed by the laws of Newtonian physics. System identification can be used to
characterize applied forces and moments acting on the aircraft that arise from aerodynamics
and propulsion. Typically, thrust forces and moments are obtained from ground tests, so
aircraft system identification is applied to model the functional dependence of aerodynamic
forces and moments on aircraft motion and control variables. We are given a set of data
obtained from a series of initial flight test to perform a hover-control of a fixed wing aircraft.
The goal is to perform several tests, analysis and experiments on the data that we learned in
the AE566 Course on Data Analysis and System Identification.

Chapter 1. Introduction

1.3

Structure of the report

This chapter is an introduction that describes the goals of the report and a brief overview
of the contents of the report. Chapter 2 describes the application from which the data are
taken. Visual aids like diagrams and figures are shown explaining the source of the data and
its physical meaning. Chapter 3 analyzes the data, where we perform statistical and spectral
analysis of the data to get insight into its properties before we begin identification. Chapter
4 is on linear identification, including time-domain and frequency-domain identification.
Chapter 5 is on nonlinear identification. The next chapter includes discussion and conclusions. Chapter 6 is followed by bibliography and a short appendix. The appendix contains
the name of m-files that are used in this project.
R

Since, the data set I am working upon is provided by someone else, I agree to allow
the data provider to read my report.

2. Data Source

2.1

Background of the Experiment

In the field of UAVs, there is seen to be surge in research about vertical take-off / landing
(VTOL) and hovering capabilities. Though, existing rotary-wing aircraft satisfy VTOL
and hovering requirements but they are found to bepoor in cruise performance. But, with
high thrust-to-weight ratio (T/W), it is possible to hover a fixed-wing aircraft over a place.
However, the problem turns out to be difficult in the presence of wind-disturbances and
needs an active feedback to achieve hovering capability.

2.2

A Brief Review of Aerospace Control Systems

Aerospace engineers develop control systems for a vehicles orientation (attitude) about
its center of mass. The control systems include actuators, which exert forces in various
directions, and generate rotational forces or moments about the aerodynamic center of
the aircraft, and thus rotate the aircraft in pitch, roll, or yaw [2]. For example, a pitching
moment is a vertical force applied at a distance forward or aft from the aerodynamic center
of the aircraft, causing the aircraft to pitch up or down.
Roll, pitch and yaw (see fig 2.1) refer to rotations about the respective axes starting from
a defined steady flight equilibrium state. The equilibrium roll angle is known as wings level
or zero bank angle, equivalent to a level heeling angle on a ship. Yaw is known as "heading".
The equilibrium pitch angle in submarine and airship parlance is known as "trim", but in
aircraft, this usually refers to angle of attack, rather than orientation. However, common
usage ignores this distinction between equilibrium and dynamic cases. The most common
aeronautical convention defines the roll as acting about the longitudinal axis, positive with
the starboard (right) wing down. The yaw is about the vertical body axis, positive with the
nose to starboard. Pitch is about an axis perpendicular to the longitudinal plane of symmetry,

10

Chapter 2. Data Source

Figure 2.1: Plane (Body Axis) - based about aircrafts CG:

Figure 2.2: Aircraft coordinate systems and configuration.

positive nose up. A fixed-wing aircraft increases or decreases the lift generated by the wings
when it pitches nose up or down by increasing or decreasing the angle of attack (AOA). The
roll angle is also known as bank angle on a fixed-wing aircraft, which usually "banks" to
change the horizontal direction of flight. An aircraft is usually streamlined from nose to
tail to reduce drag making it typically advantageous to keep the sideslip angle near zero,
though there are instances when an aircraft may be deliberately "sideslipped" (see fig 2.2)

2.3 Description of the experiment

11

Figure 2.3: Test aircraft. The tested aircraft has conventional wing-tail with tractor configuration
for example a slip in a fixed-wing aircraft.

2.3

Description of the experiment

In this test, an aircraft (see fig 2.3) is being flown by a trained pilot. In all the flights, first it
is tried to manually achieve hovering and then switch to the control to maintain the hover
attitude of the aircraft. The data has two modes pilot mode and the autopilot mode.
Switch signal 3 stands for manual mode of operation, and swithc signal 1 and 2 are for
controller mode of operation.
The following variables are identified as inputs and outputs of this complex system.
Input Signals
Throttle
Elevetor
Aileron
Rudder
Output Signals
Euler Angles (pitch, roll, yaw)
3-axis rates (p, q, r)
Positions (x, y, z)
Quaternions (q0 , q1 , q2 , q3 )

12

2.4

Chapter 2. Data Source

Description of the data-set

The data provided has sufficiently large number of points (48000) for identification problem.
The time associated with each data-point is available as well. The variable cmd-mode
denotes a switch signal which indicates the mode in the which the aircraft is operating in.
Switch signal 3 denotes manual mode implying that a trained pilot is steering the aircraft.
While mode 1 and 2 denotes that the controller is trying to make the aircraft hover at a place.
Obviously, we would be interested in the controller mode, thus removing all the manual
mode data. After this operation, all variables are in ready-to-use format.
In almost all the aircraft related identification articles the triplets of elevator, aileron
and rudder are considered inputs, and the triplets of pitch angle, roll angle, yaw angle are
considered outputs to a 3-Input 3-Output system identification problem. We will be looking
to identify this system only.

3. Data Analysis

3.1

Knowing our Data

Before we go on analyzing the data, we first need to identify what our data looks like and
more importantly, what we should be looking for in our data. Firstly we take the given
R m-file called
data and generate plots of the variables of interest. We run the MATLAB
process.m accompanied by the data and get the plots shown in fig 3.1.

3.2

Removing Trend from Data

Now, with our data ready, we might to want to make it clean in the sense its more smooth
fro identification purposes. By cleaning we mean stripping off data from not so significant
R
hidden parts of it. The MATLAB function
detrend subtracts the mean or a best-fit line
(in the least-squares sense) from a given data. If the data contains several data columns,
detrend treats each data column separately [3].
Removing a trend from the data enables us to focus our analysis on the fluctuations in
the data about the trend. While trends can be meaningful, some types of analysis yield better
insight once you remove trend. So, we will create two data-sets, one with de-trended data,
and one as it is. We wish to apply all our algorithms on both of the data. Unless explicitly
mentioned otherwise, it should be assumed that the results of a particular algorithm was
similar for both the data sets.

Chapter 3. Data Analysis

14

[a]

[b]

[c]

[d]
Figure 3.1: First hands on data. [a] shows the hovering movement of the aircraft in earth
coordinates, [b],[c],[d] show the inputs and outputs of the data.

3.3

Data Analysis begins

Next thing to look for in a data-set is its sampling rate. Sampling rate can be easily identified
using the given formula and found to be 0.009999625.

T s = (max(time) min(time))/length(time);

(3.1)

Fs = 1/T s;

(3.2)

Then we wish to look for coherence between various inputs and outputs. This can be
R command mscohere. Results are shown in the fig 3.2. We notice
done using MATLAB
that there is strong coherence between, elevator and pitch, which is expected. However,
aileron and rudder signal do not show that much of coherence with roll and yaw respectively.
Maybe, its because of noise in the signal, or maybe its because of the feedback.

3.3 Data Analysis begins

15

Figure 3.2: Pairwise coherence of input with outputs. Top to bottom: coherence for elevator
input, coherence for aileron input and coherence for rudder input

Figure 3.3: Power-spectrum of inputs and outputs. From left to right: power spectrum of
elevator, aileron and rudder; power spectrum of pitch, yaw and roll

16

Chapter 3. Data Analysis

Next thing we do is to find the power spectrum of each of the inputs and outputs. Powerspectrum can be seen as a way to qualify the noise content of a signal. This can be done
R . Results are shown in fig 3.3.
using periodogram command of MATLAB

Figure 3.4: Pair-wise cross correlation between inputs and outputs. From left to right:
cross-correlation with respect to elevator, aileron and rudder
Similarly, we can check for cross correlation also, using xcorr command. The results
are shown in fig 3.4

3.3 Data Analysis begins

17

As we have mentioned, that our MIMO system is kind of pairwise SISO, we can figure
out how individual inputs result in individual outputs. We can show this graphically using
scatter plots. A scatter plot displays data points on a graph with inputs as abscissa and
outputs as ordinates. Scatter plots are shown in fig 3.5 .

Figure 3.5: Pairwise scatterplots. From top to down: elevator-pitch system, aileron-roll
system and rudder-yaw system

We mention in the previous section that the first thing to do is to remove the data which
corresponds to the manual mode of operation. This can be easily done using find command
R . Then, we remove the non-moving portions as doing ID on that portion
in MATLAB
may not be very sensible. This is followed by breaking the data into two portions; one for
estimation and other for validation. Fig 3.6 and fig 3.7 show this step.

Chapter 3. Data Analysis

18

Figure 3.6: Partition of data for outputs for estimation and validation purposes

Figure 3.7: Partition of data for inputs for estimation and validation purposes

We know our data well, and we know what kind of behavior these data should be
demonstrating since we know the source of the data. So, let us move on to the
identification.

4. Linear Identification

4.1

Frequency Domain Identification

To perform frequency domain identification, we first need to get our time-domain data
converted into frequency-domain. This is done by taking Discrete Fourier Transform (DFT)
of the data. There is a computationally effective algorithm to compute DFT, called Fast
Fourier Transform (FFT). We use fft command of MATLAB to do the needed. In this section
we perform frequency domain identification on a SISO system which is chosen to be the
elev-pitch SISO system. The DFTs of output and input are taken and an vector is formed
with their element-wise ratio. This ratio is called complex frequency response. Being a
complex quantity, it is possible to plot magnitude and phase of the complex frequency
response obtained. Such a plot is called Bode plot. The bode plot for frequency response
estimated is shown in fig 4.1.

4.2

MIMO Least Squares Fitting

In this section, we employ the MIMO least squares code we developed while working on
homework 3. MIMO least squares code is just least squares with elements of coefficient
matrix in block form. The algorithm assumes an input-output model structure with known
inputs and initial conditions for outputs. The algorithm is based on solving a set of linear
equations with the goal to minimize the errors between output and the estimated output and
thus produce a suitable coefficients matrix.
Now, we should remember that we have partitioned our data into two parts (see fig 3.6)
and fig 3.7, labeled as data for estimation and data for validation. To verify the accuracy and
fitness of this method, we perform this algorithm on the data for estimation, and by using
the same coefficients matrix, we make a guess about the data for validation. We measure
the error in the fitting and call the errors as fit error and prediction error. The outputs of our

20

Chapter 4. Linear Identification

Figure 4.1: Frequency domain identification. Notice the first subplot shows the magnitude
of the response while the second one is phase of the response.
code are shown in fig 4.2 and 4.3.
Now, we check the fourier transform of the fir error of this identification. High peaks in
the plot at a certain frequency indicate that there is mismatch of dynamics at that particular
frequency. We show this plot in fig 4.4.
In the plot we can see that there are high peaks at 0 frequency, which indicate that
our DC estimates are not accurate at all. Thus, it looks like the code failed to capture the
dynamics of the data provided. But, the code itself is correct. We show in the next section
that the code works by demonstrating it on a toy example. Also, using fit-error vs prediction
error, which normally gives us a convex plot, with minima at the best possible system order,
doesnt make sense, as both the errors will be large since our estimated system is not able to
match dynamics well.

4.3

Toy Example
Using the same algorithm, we perform ID on a known system, i.e. we create input vector
and the coefficient matrix, and using them we compute our output vector. Using the MIMO

4.3 Toy Example

21

Figure 4.2: Estimation of output using MIMO least squares algorithm. The fits are not great,
interesting though

Figure 4.3: Estimation of output for validation input. The fits become worse as expected

22

Chapter 4. Linear Identification

Figure 4.4: FFT of fit-error for MIMO system. High peaks are seen at 0 frequency
least squares code, we make an estimate of the coefficient matrix. This estimated coefficient
matrix along with the input vector will give us our estimated output. We see, how much we
are able to match the estimated output to the computed output (see fig 4.5).

Figure 4.5: Application of the MIMO least squares algorithm on a known linear system.
The algorithm looks perfectly alright
Looks like we did well. To confirm that we are not missing any of the dynamics, we
show that the fft of fit error for the toy example doesnt have high peaks anywhere (see fig

4.4 Identification of Markov Parameters

23

4.6). Then why didnt the algorithm work for the given data set. Probably, its not even
linear!!

Figure 4.6: FFT of fit-error for toy system. No high peaks are seen

4.4

Identification of Markov Parameters

The Markov parameter sequence for a state-space model is a kind of matrix impulse response
that easily found by direct calculation using Eq. (G.1):
h(0) = Cx(0) + D (0) = D
x(1) = A x(0) + B (0) = B
h(1) = CB
x(2) = A x(1) + B (1) = AB
h(2) = CAB
x(3) = A x(2) + B (2) = A2 B
h(3) = CA2 B
..
.
h(n) = CAn1 B,

n>0

Note that we have assumed x(0) = 0 (zero initial state or zero initial conditions). The
notation (n) denotes a qq matrix having (n) along the diagonal and zeros elsewhere.G.2
Since the system input is a q 1 vector, we may regard (n) as a sequence of q successive
input vectors, each providing an impulse at one of the input components.

Chapter 4. Linear Identification

24

The impulse response of the state-space model can be summarized as

(
h(n) =

D,

n=0

CAn1 B, n > 0

The impulse response terms CAn B for n 0 are known as the Markov parameters of the
state-space model.
Now, to identify markov parameters, we take the elev-pitch SISO system and we try
to identify it. After identification, we have an input-output model, to which if we supply
an impulse input, we get impulse response, and thus we get markov parameters. For
identification, we will use the same MIMO least squares code. The results are shown in fig
4.7 and fig 4.8

Figure 4.7: Estimation of output for elevator-pitch SISO system. Again, estimation looks a
bit off.
Since, our estimated response couldnt match the original response, we can safely
assume that the impulse response obtained are also not very much accurate. But, still, we
have got our Markov parameters. We can use these markov parameters in the next section.

4.5

Eigensystem Realization Algorithm

Eigensystem Realization algorithm is a system identification technique, originally proposed
in a seminal paper by Juang and Pappa [4]. ERA can be used as a modal analysis technique
and generates a system realization using the time domain response (multi-)input and (multi)output data.
We use the pulse response data to form the Hankel matrix

4.6 N4SID algorithm

25

Figure 4.8: Markov parameters of the system obtained by providing impulse input to the
estimated system
Y (k + 1)
...

Y (k + 1)
H(k-1) =
..

Y (k)

Y (k + p)
..

where Y(k) is the m X n pulse

Y (k + r)

Y (k + p + r)
response at time step k.
Now, by using singular value decomposition (SVD) followed by some analytical calculations we come up with a minimal state space system producing same markov parameters
as those were used to form the Hankel Matrix. We also know that the similar state space
systems have same markov parameters and same transfer functions, so, basically, we have
estimated our transfer function. It is also possible to estimate the order of the system through
ERA.
We use our elev-pitch SISO system to perform ERA upon. Now, we have two sources
of Markov parameters : we can get it by the impulse response of the estimated system we
developed in last section; also, we can get it by inverse Fourier transform of the complex
frequency response we computed in section on frequency domain identification. Let us use
the second method. The result for order 3, which we find to be most satisfactory, is shown
in the fig 4.9. Thus we can make aguess about the system order to be 3.

4.6

N4SID algorithm
In this section we try our hands on N4SID algorithm, which is based on subspace identification methods and is very popular in linear identification. The subspace identification

26

Chapter 4. Linear Identification

Figure 4.9: Estimation of markov parameters through ERA

4.6 N4SID algorithm

27

problem is: given a set of input/output measurements, estimate the system matrices (A, B,
C, D), Kalman filter gain K up to within a similarity transformation, and the innovation
R comman n4sid, we try to find a fit for our 3 inputcovariance matrix R. Using MATLAB
3 output system. Assuming various model orders is possible in n4sid; we present estimates
for order 3 (best than others) in fig 4.10.

Figure 4.10: Application of N4SID algorithm on our data set. Looks like N4SID barely
manages to stick to the mean value of the response
It looks like N4SID algorithm can only give mean value response, which is kind of
better, as we are more interested in fluctuations. But, there must be some better way!
R

Although, most of our methods could not live up to the expectation, but still, we have
got many insights about the system we wish to identify. The results of this section
indicate presence of non-linearity in the system, the prospects of which we explore in
the next section.

5. Nonlinear Identification

So far, we have tried our best to identify the system from the provided data. We did identify
some systems, but the fits are not satisfactory. We have also speculated the presence of
non-linearity in the system leading to failure of our linear least squares method. Hence, we
move to nonlinear identification.

5.1

NARMAX
The nonlinear auto-regressive moving average model with exogenous inputs (NARMAX
model) can represent a wide class of nonlinear systems, and is defined as

y(k) = F[y(k 1), . . . , y(k ny ), u(k d), . . . , u(k d nu ), e(k 1), . . . , e(k ne )] + e(k)
(5.1)
where y(k), u(k) and e(k) are the system output, input, and noise sequences respectively;
F[.] is some nonlinear function, d is a time delay typically set to d = 1; maximum lags for
the system output, input and noise are denoted by ny , nu , ne .
The model is essentially an expansion of past inputs, outputs and noise terms. Because
the noise is modelled explicitly unbiased estimates of the system model can be obtained in
the presence of unobserved highly correlated and nonlinear noise [5].
In our algorithm, we use several radial basis functions centered at several points on
a grid determined by range of input and output. Such radial basis functions together can
mimic system behavior provided proper scaling. Determining scale factors for each of the
functions is again a matter of least squares. We use this algorithm on our data for estimation
and data for validation as well. Results are shown in the fig 5.1.

5.1 NARMAX

29

Figure 5.1: NARMAX model structure algorithm on elev-pitch system. From top to bottom:
NARMAX algorithm on estimation data; estimation of output from derived model structure
for validation data

30

5.2

Chapter 5. Nonlinear Identification

Input Augmentation using Basis Functions

In this section we use the technique of augmenting input with basis functions such as
Gaussian, sinusoidal functions, etc. Basically, we assume that the input is unknown, and
it can be decomposed into a known and an unknown component. We find that unknown
component by augmenting another input to the system, thus changing the total number of
inputs to the system. After this, we just need to implement least squares to find the unknown
input. Let us take the example of elev-pitch SISO system with sum of sines and cosines as
basis functions to be used in input augmentation. The results are shown in fig 5.2.

Figure 5.2: Input augmentation using basis functions on data for estimation for elev-pitch
SISO system. From left to right: measured output and estimated output, scatter plots for
measured output and estimated output
The fits look good, and now we can try the obtained model for the data for validation
(see fig 5.3). For vaildation data, the fits, as always, are not too great, but are satisfactory
though.

5.3

Hammerstein Wiener Model Using Simulink

R codes to carry out system identification. Now, we wish to
So far, we have used MATLAB
experiment with the inbuilt blocks of the Simulink environment to identify our elev-pitch
SISO system. In this subsection we consider Hammerstein Wiener block [6] available in
system identification toolbox for our estimation purposes. The block diagram we need to
construct is shown in the fig 5.4 and the code to run the block is provided below:

ze =
zv =
mhw1
x0 =

i d d a t a ( p i t c h _ e , e l e v _ e , Ts , Name , e l e v p i t c h ) ;
i d d a t a ( p i t c h _ v , e l e v _ v , Ts , Name , e l e v p i t c h ) ;
= nlhw ( ze , [ 3 3 0 ] , p w l i n e a r , p w l i n e a r ) ;
f i n d s t a t e s ( mhw1 , ze , [ ] , m a x i t e r , 5 0 ) ;

5.3 Hammerstein Wiener Model Using Simulink

31

Figure 5.3: Input augmentation using basis functions on data for validation for elev-pitch
SISO system. From left to right: measured output and estimated output, scatter plots for
measured output and estimated output
p l o t ( mhw1 )

Figure 5.4: Block diagram of the Hammerstein Wiener system identification created in
Simulink
We first use the un-detrended data to create a system structure using iddata command.
Then by using nlhw command to feed data to the Hammerstein Wiener block. We do
this process first for estimation data, and after identifying the model structure we use
the validation input to create an estimation for validation input. By tweaking order and
non-linearity types, we are able to get the results as shown in fig 5.5.
Wait a second, this is interesting, looks like we are estimating the output well, its just
needs to be shifted. And we also know, doing what can help that. Yes, if we use detrended
data, we are able to eliminate this shift (see fig 5.6).
That has been a good identification. But what do we just learn about the system?
Hammerstein Wiener block allows us to interpret results of the identifications. In the code

32

Chapter 5. Nonlinear Identification

Figure 5.5: Hammerstein Wiener Identification for un-detrended elev-pitch SISO system in
Simulink. From left to right: for estimation data; for validation data

Figure 5.6: Hammerstein Wiener Identification for detrended elev-pitch SISO system in
Simulink. From left to right: for estimation data; for validation data

5.3 Hammerstein Wiener Model Using Simulink

33

we used for running the simulation block, we have assumed our both input and output
nonlinearities as piecewise linear, and the output-error model is assumed to be [3 3 0],
which indicates that there are 2 (3-1) zeros, 3 poles of the system, and there is no delay from
input to the output in terms of the number of samples. The pole-zero map of the identified
model is shown in fig 5.7.

Figure 5.7: Pole-zero map of the identified Hammerstein Wiener Model for elev-pitch SISO
system

6. Conclusions and Discussions

In this report we have seen how a data-set can be analyzed and worked upon so as to
identify the dynamics behind the data. There have been several steps and several methods
in identifying the system corresponding to given data. Some of the methods worked okay, a
few worked very well; but most of them failed. This is expected, otherwise who would need
these many methods. In this section we try to comprehend why a particular method worked
and why others didnt.

6.1

Logical flow of Identification

With the data-set in our hand, we just do not start working on the identification. First, we
need to understand the experiment from which the data has been obtained. Knowing the
underlying physics may not be absolutely necessary but having an idea of it helps in making
intuitive guesses about the system to be identified. Now, we focus on data, we try to qualify
data in terms of its sample rate, spectral content, noise content. Another thing to look for in
a data set is a linear trend. Removing a trend from the data enables us to focus our analysis
on the fluctuations in the data about the trend. Now we are all set for the identification.
Although almost everything in this world is non-linear, we begin ID by linear methods.
R to compute
First we do frequency domain identification by using fft tool of MATLAB
a complex frequency response. Now, here we come up with an algorithm without which
system identification is almost impossible, the least squares algorithm. The least squares
algorithm is explained in chapter 4. We do MIMO least squares on our data set to come up
with an input-output model which tries to give same output as the original system would
give if both are fed with same input. As it turns out, the estimated model doesnt lives up
to the expectations. But, we show in the next section, that the algorithm is highly capable
of estimating near to perfect model for a linear system. At this point, we start thinking
that the system might be non-linear. Next thing we do is to identify Markov parameters.

6.2 Final Comments

35

Markov parameters are used in Eigensystem realization algorithm to make a guess on the
system order. Now, we move to non-linear identification, where, we demonstrate three
techniques by which we can estimate a non-linear system. We start with NARMAX, where
we use radial basis functions with variable centering and spread to approximate for system
output. Secondly, we try our techniques of augmenting input with basis functions and do
identification accordingly. The last thing we do is using Hammerstein Wiener block of
Simulink to identify system. We show poles and zeros of the system and the associated
non-linearity with inputs and outputs. Most of the algorithms require working knowledge
of matrix mathematics. One great source on the subject is [7].

6.2

Final Comments
Out of several things we tried in this project, we wish to discuss few of them to conclude
this report.
Let us talk about detrend first. A linear trend typically indicates a systematic increase or
decrease in the data. A systematic shift can result from sensor drift, for example. Whether it
makes sense to remove trend effects in the data often is a subjective question and it largely
depends on the objectives of our analysis. We have performed this experiment using the
both sets of data, and in a few cases removing trend has helped us to obtain better fits.
In system identification, we need to come up with a model, which we think can be a
great fit for the data we have. For assuming a model structure, some knowledge of the
dynamics of the experiment which produced the data is always helpful. However, its not
absolutely necessary. Also, we do not have to stick with the model we choose. There are
lots of options available. We can pick an ARX model, or an input-output model, or even an
output-error model. Evaluating the model parameters is the core of the system identification.
If we have an idea of the underlying dynamics of the variables of the experiment, we might
actually guess order of the system. If we do not know the order of the system, we can
always play with various system orders to pick the right one.
Another important thing is maintaining the logical flow of identification. We do not
jump straight to non-linear identification unless there are strong evidences to do so. In this
report, we show that the inaccuracies in least squares, and in the ERA help us to decide on
whether we need non-linear identification or not. And then of course, there are many ways
to do non-linear identification.
Finally, getting a perfect identification in data from real experiments is almost impossible
because of effects from noise, feedback, and sensor and actuator inaccuracies. But, perfect
identification is not really required. In control systems, with so many tools, we just dont
require a perfect system, right ? ^

7. Bibliography

[1] Zadeh, L. A., From Circuit Theory to System Theory, Proceedings of the IRE, Vol.
50, May 1962, pp. 856865
[2] http://en.wikipedia.org/wiki/Flight_dynamics
[3] http://www.mathworks.com/help/matlab/ref/detrend.html
[4] J.-N. JUANG and R. S. PAPPA. "An eigensystem realization algorithm for modal
parameter identification and model reduction", Journal of Guidance, Control, and
Dynamics, Vol. 8, No. 5 (1985), pp. 620-627. doi: 10.2514/3.20031
[5] http://en.wikipedia.org/wiki/Nonlinear_system_identification#
NARMAX_methods
[6] http://www.mathworks.com/help/ident/ref/idnlhwmodel.html
[7] Bernstein, D. (2009). Matrix mathematics. Princeton, N.J.: Princeton University Press

R Codes
A. MATLAB

sys_id_main.m: This is a big file which contains almost all the codes divided into
sections.
hammerstein_weiner_simulink_runner: code to run hammerstein_wiener_
simulation_block.
process.m: file accompanied with data set.
narmax.m: code to run NARMAX for elevator-pitch system.
basis_function_augmentation.m: code for basis function augmentation.
detrend.matandundetrend.mat: data set without and with linear trend, respectively.
hammerstein_wiener_simulation_block.slx: simulink model for HammersteinWiener system.