Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
1
Multivariate Approaches to Modeling Batch Data
• Observation-Wise
Batch time
Unfolding (OWU) observations Batch
1
– method by which the data for
each batch is stacked on top
of one another
Batch
– Manner in which ProMV and 2
…
Time, T
Variables, K
N
Batch
Batches Trajectory
Batch N
Data
2
Batch-Wise Unfolding (BWU)
• Batch-Wise Unfolding (BWU)
– The method by which batch observations are extracted
horizontally in a time-wise fashion
– Each batch becomes a single row of data in the model
Time, T
Variables, K
N Batch
Batches Variable 1 Variable 2 Variable 3 Variable 4 Variable K
Trajectory
Data
K K
XTX
K
3
Kernel algorithms for PCA & PLS with unfolded data
K M N
X Y XXT
N N
N
YYT
N
N
XXTYYT
N
References: - Lindgren, Geladi, Wold, “The kernel algorithm for PLS”, Journal of Chemometrics, 1993.
- Dayal, MacGregor, “Recursive exponentially weighted PLS and its applications to adaptive
control and prediction” , Journal of Chemometrics, 1997.
4
Adv: Exponentially weighted recursive PLS
Based on the kernel algorithms
• New data often coming available periodically.
• Want to update the PLS model by simply correcting it for
new data and not rebuilding the entire model from scratch
each time (e.g. soft sensors).
• May also want to adapt model by exponentially discounting
past data.
• Can update the Kernels directly:
– Have kernels at time t-1 and new vectors of data become
available at t: xt, yt
– (XTX)t = λt (XTX)t-1 + xtTxt
– (XTY)t = λt (XTY)t-1 + xtTyt
• Update Kernels this way and then use fast Kernel algorithms
to update model
• λt is exponential forgetting factor (0 < λt ≤ 1.0)
(c) 2004-2012, ProSensus, Inc.
5
The Problem
A. Landmark Approach
6
Extraction of Landmarks from batch trajectories
Extract landmark features from the trajectories that summarize the major sources of
variation from batch to batch
Temp 2
Level 1
∆ Time 4
Batch trajectories
batches Z X Y
7
Score Plot for FMC Landmarks
Loading Plot
8
VIP Plot – ordered importance to model
9
B. PCA or PLS to extract features of batch trajectories
Time, T
Variables, K
N Batch
Batches Variable 1 Variable 2 Variable 3 Variable 4 Variable K
Trajectory
Data
10
Mean Centering and Scaling
11
PLS for batch-wise unfolded data
Batches
Variables
12
Industrial example: PCA of DuPont historical data
13
Loading Plot - p1
-Time varying loadings for all variables (grouped by variable)
- Reveal the relative importance of variables and their negative or
positive impact as a function of time.
SPE Plot
14
Summary of analysis of nylon autoclave data
15
PLS Analysis of FMC Reactor/drier data
Multi-block Structure
Z2 Initial Conditions
– Initial chemical analysis
– Weight of wet cake
X Process Trajectories
– 10 trajectories
– Batch duration
Y Final Quality
– 11 Quality variables from QC lab
– Solvent concentration
(c) 2004-2012, ProSensus, Inc.
16
Alignment of batch trajectories
Approaches to alignment
1. Linear interpolation of the time scale.
• Choose an average final time – resample the trajectories at a fixed
number of intervals for all batches
• Retain the varying batch duration information by including it in Z
• Often poor since most batches proceed faster or slower only in
certain phases.
• Could also fix time of each phase and resample to give fixed
number of points.
2. Use of an indicator/maturity variable (Nomikos & MacGregor)
• Indicator variable – e.g. conversion – resample at equal conversion
intervals
• Indicator variable must be monotonic with time in any phase (can
use different indicator variables in each phase)
• Other examples – cumulative amount of a key reactant fed to
batch, or cumulative heat release
• Compute a new time usage trajectory to retain the time variations
3. Dynamic time warping (Kassidas, MacGregor, Taylor)
• Speech recognition literature
(c) 2004-2012, ProSensus, Inc.
17
Trajectory alignment
Garcia, Kourti, MacGregor (2003) Ind. Eng. Chem. Res. 42, 3592-3601.
(c) 2004-2012, ProSensus, Inc.
18
Trajectory Alignment
• Indicator Variable Collector Level
Approach:
– Re-sample the batch
samples against a new
variable different from time
(e.g. conversion, reactant Dryer Temperature
feed). Monotonically
increasing/decreasing
37
(c) 2004-2012, ProSensus, Inc.
Final Level
100%
Collector Tank Level
Different Peak
Temp per Batch
0% 100%
Initial Temp
0%
STAGE 1 STAGE 2 STAGE 3 Dryer Temperature
Different Initial
Temp per Batch
19
Trajectory Alignment
Collector Level Collector Level
After Alignment
Agitator Speed Agitator Speed
After
20
Alternative preservation of timing information
– Record phase times
Put this timing variable information in the Z matrix
- Used in the Landmark approach
Temp 2
Level 1
∆ Time 4
21
The full dataset
Analysis step
Trouble-shooting the industrial batch reactor/dryer process
71 batches
(c) 2004-2012, ProSensus, Inc.
22
Batch-Wise Unfolding of the data arrays
– The batch trajectory observations are extracted horizontally in a time-
wise fashion
– Append both Quality (Y) and Initial condition data (Z)
– Each batch becomes a single row of data in the model
– PLS models built on these unfolded data matrices
– Scores for each batch – score plot shows differences among batches
Time, T
Variables, K
N Batch
Batches Variable 1 Variable 2 Variable 3 Variable 4 Variable K
Trajectory
Data
23
Analysis of combined Z1, Z2 blocks
Score plot Analysis reveals obvious clustering
among on- and off-spec batches.
t1
VIP’s for Z model
Analysis of X trajectories
• Clustering of good/bad batches
•Good batches have high t1
24
Score plot for Z1,Z2 Score plot for X
Good Batches
50
(c) 2004-2012, ProSensus, Inc.
25
Summary of trouble-shooting results
26
Inconsistent Batches
53
(c) 2004-2012, ProSensus, Inc.
Contribution Plots
54
(c) 2004-2012, ProSensus, Inc.
27
Contribution Plots
• Cooling Time, and high speed heating in stage 2 appear “strong” enough
to compensate for other adverse processing conditions?
55
(c) 2004-2012, ProSensus, Inc.
28
Observation-Wise Unfolding (OWU) Analysis
Unfold so that each row contains one • Mean center and scale columns
observation at each time for one batch to unit variance (e.g. Umetrics)
• Each latent variable t = Xw
provides a linear combination of
the original variables over all
batch times and all batches that
explain the greatest variance of
the raw time histories.
• Scores (t) are local – value at
every observation during the
batch. One score value per
time point instead of one
score value per batch.
29
Comparison of variance explained BWU vs OWU
30
BWU vs OWU Loading Plots (Nylon data)
BWU- shows the model loadings OWU- the model is built only on
for each time and each variable the variables. One loading per
in the batch. This plot reveals the variable. Does not provide any
major difference among the information on behaviour during
batches due to their variable- a batch.
time histories.
Recommendations
Batch-wise unfolding (BWU) recommended
– Necessary for analysis of differences among batches
• BWU looks at between batch variance, OWU looks at within batch variance (example – score
plots on FMC batches)
– Necessary to incorporate batches into PLS models with initial Z and
final Y values
– Necessary for control and prediction of product quality
– Preferable for on-line monitoring
• Use same model as for analysis, prediction and control
• Don’t need multiple models for phases
• Focuses on differences in performance of current batch from common cause variation (training
batches) which is what we want
• But OWU will still detect many gross deviations
Observation-wise (OWU) unfolding
– Mainly useful as a preprocessing step to reduce #of variables when
have multivariate sensors (Eg. NIR), etc.
• Then use scores from OWU on sensor data as variables in BWU
– More on this after the lab
(c) 2004-2012, ProSensus, Inc.
31
Umetrics – Simca_P software
• Uses both OWU and BWU
• Always leads user into OWU first
– As a result most users try to analyze differences among historical
batches at this level
– That is not good - OWU does not look at between batch difference
• In accompanying Comprehensive Chemometrics chapter on batches
Svante Wold also states that this is not good.
– On-line monitoring done on these scores (cf. Hahn’s approach)
• Then takes LV’s from OWU and refolds them batch-wise.
– First aligns them by linear time warping
– Then have aligned BWU scores from first OWU level as new variables
– Allows for PLS but variable loadings, contributions, etc are in terms of
these OWU level scores and not original variables – interpretation?
– Umetrics has introduced a way to put original variables into this BWU
matrix – We recommend that one should do this as first step in any
batch data analysis (ie. bypass OWU level).
(c) 2004-2012, ProSensus, Inc.
…
Time, T
Variables, K
N
Batch
Batches Trajectory
Batch N
Data
32
Workshop #2b: Comparison of PCA on BWU and
OWU Nylon data
• The purpose of this exercise is to look at the two methods for unfolding
batch data to determine how data can be interpreted using each
method:
• BWU – Batch wise unfolding
• OWU –Observation wise unfolding
There are approximately 31,000 data points per batch run in this
example.
(c) 2004-2012, ProSensus, Inc.
33
OWU makes sense to summarize Spectral variations
during batch
• PCA on OWU spectral data gives principal components that
apply at every instance of time over all batches
– Instantaneous (steady state model)
– This is what one wants for spectral data – no matter how the process
has reached this point, the spectra reflect the current composition of
the contents of the batch at that instant
– The new latent variables (t1, t2, …) reflect the number of detectable
constituents and their current concentrations
– These latent variables (t1, t2, …) just provide a small number of new
variables (eg. 3 or 4) that summarize the variations of the 1200 or so
spectral absorbances of the raw spectra.
– These principal components can then be used as other process
variables in a BWU model that captures the compositional history of
the batch over the entire batch duration and the effect on final Y
Product temp
Pressure
PCA
Summarizes ~99% of
Each row is a spectrum, taken the original spectra with
3 scores.
during the batch process (OWU) Each run reduces
down to 318 data points
(c) 2004-2012, ProSensus, Inc. (~100-fold reduction).
34
Time behavior of scores summarize spectral
information during batch
Progress of Spectral scores for a good batch
35
Interpretation of latent variables from OWU
• The LV’s (t1, t2, … tA) are simply A new variables that explain the most
variance in the original J raw trajectory variables.
• Since most batches vary widely over their duration, one often needs
almost as many new LV’s as one has original measured variables.
– If use OWU it would be better to subtract off the average trajectories first –
will get many fewer LV’s since no need to model the average trajectories
• If want to use the OWU approach for monitoring, it is simpler and often
just as good to use monitoring charts on a few of the key process
variables (i.e. univariate approach)
– (Gerald Hahn, Principal GE statistician in late 80’s presented this approach)
– OWU approach essentially is Hahn’s approach, but it uses PCA to select some
combinations of the variables instead of key original process variables
Contact details
Email: info@prosensus.ca
36