A Least Angle Regression Control Chart for Multidimensional Data G IOVANNA C APIZZI and G UIDO M ASAROTTO Department of Statistical Sciences University of Padua Italy
2 ND I NTERNATIONAL S YMPOSIUM ON S TATISTICAL P ROCESS C ONTROL Rio de Janeiro, Brazil July 13-14, 2011 A Least Angle Regression control chart. . .
1/ 22
Outline 1
Multivariate Statistical Monitoring
2
Variable-selection Methods in SPC
3
Proposed Procedure Reference Model Least Angle Regression Algorithm LAR-EWMA control chart
4
Simulation Results
5
Concluding Remarks and Future Research
A Least Angle Regression control chart. . .
2/ 22
Synopsis Framework: Phase II monitoring of a normal multivariate vector of product variables. Fault type: Persistent change in the process mean and/or increase in the total dispersion. Problem: Simultaneous monitoring of several variables: how many and which variables are really changed? Proposal: Combination of a variable-selection method (Least Angle Regression) with a multivariate control chart (multivariate EWMA). Results: The LAR-based EWMA is a very competitive statistical tool for handling several changepoint scenarios in the high-dimensional framework. A Least Angle Regression control chart. . .
3/ 22
Multivariate Statistical Monitoring Possible Frameworks
U NSTRUCTURED :
multivariate vector of quality characteristics
S TRUCTURED :
analytical models describing process quality 1 linear and non linear profiles 2 multistage processes
A Least Angle Regression control chart. . .
4/ 22
Multivariate Statistical Monitoring Possible Approaches
Monitoring the stability of the whole set of product variables (low sensitivity in a high-dimensional context) Monitoring a reduced set of out-of-control product variables. But the shifted components are obviously unknown!
A Least Angle Regression control chart. . .
5/ 22
Variable-selection Methods in SPC State of the art
Promising approach: combine a suitable variable selection method with the multivariate statistical monitoring Forward search algorithm with a Shewhart-type control chart (Wang and Jiang, 2009) for handling mean changes in the unstructured scenario. LASSO algorithm with a multivariate EWMA, for detecting 1
2
mean changes in the unstructured case (Zou and Qiu, 2009) changes in multivariate linear profiles (Zou et al. 2010)
A Least Angle Regression control chart. . .
6/ 22
Variable-selection Methods in SPC Our proposal
L EAST A NGLE R EGRESSION WITH A M ULTIVARIATE EWMA 1
2
3
4
general model formulation unifying the unstructured and structured framework use of a variable selection method yet unexplored in the SPC framework competitive procedure for detecting changes in process mean and total dispersion for a wide variety of change point-scenarios relatively simple tool for fault identification
A Least Angle Regression control chart. . .
7/ 22
Reference Model Characterization of mean and variance changes
C HANGE - POINT MODEL
yt ∼
µ , Σ) Nn (µ if t < τ µ + δ , Ω ) if t ≥ τ Nn (µ
(in-control) (out-of-control)
M EAN SHIFT
δ = Fβ F n×p : matrix of known constants; β p×1 : vector of unknown parameters . D ISPERSION INCREASE
EOC [ξt2 ] > n ⇐ Ω − Σ positive definite matrix Σ−1 (y t − µ )0 . with ξt2 = (y t − µ )Σ A Least Angle Regression control chart. . .
8/ 22
Reference Model Frameworks and characterization of δ
U NSTRUCTURED
[δi ] = [βi ]
⇐⇒
F = In
P ROFILE
" [δi ] = [g(xi )] =
p
#
∑ βj fj (xi )
j=1
e.g. δi = β1 + β2 xi + β3 xi2 for i = 1, . . . , n. M ULTISTAGE P ROCESS
Standard state space representations of a process with n stagescan be written in the desired form. yt,i = µi + ci xt,i + vt,i (i = 1, . . . , n) xt,i = di xt,i−1 + βi I{t≥τ} + wt,i A Least Angle Regression control chart. . .
9/ 22
Reference Model Monitoring stability ←→ Testing hypotheses on β
Multivariate EWMA control statistic z t = (1 − λ )z t−1 + λ (y t − µ ) with z 0 = 0n , 0 < λ ≤ 1. Linear model z t = F β + at , Σ) with at ∼ Nn (0n , λ /(1 − λ )Σ M ONITORING STABILITY OF PROCESS MEAN
β = 0} H0 = {the process is in control} ⇐⇒ {β H1 = {the process is out of control} ⇐⇒ {How many βj are non zero? One, two....all? } A Least Angle Regression control chart. . .
10/ 22
Least Angle Regression algorithm Main steps
1 2
3
Start with all the p coefficients equal to zero. Build up estimates of the unknown mean in successive steps: each step adding one variable. Reach the full least square solution in p steps (using all the variables).
S TEP k
{j1 , . . . , jk } | {z }
variables selected by LAR
A Least Angle Regression control chart. . .
⇐⇒
{βj1 , . . . , βjk } {z } |
just k parameters are assumed 6= 0
11/ 22
LAR-EWMA control chart Hypotheses systems and control statistics
M EAN C HANGE
H YPOTHESIS . βj1 6= 0, . . . , βjk 6= 0, βjk +1 = · · · = βjp = 0
C ONTROL S TATISTIC Likelihood ratio test: St,k
(k = 1, . . . , p)
VARIATION I NCREASE
H YPOTHESIS
C ONTROL S TATISTIC
β = 0p and E[ξt2 ] > n
St,p+1 = ξt2 x max 1, (1 − λ )St−1,p+1 + λ n with S0,p+1 = 1
A Least Angle Regression control chart. . .
12/ 22
LAR-EWMA control chart Control statistic, stopping rule, fault identification
Control statistic
Wt =
Alarm time
Fault Identification
St,k − ak bk k =1,...,p+1 max
t ? = min{t : Wt > h} St ? ,k − ak k = min k : >h bk ?
k ? ≤ p : {plausible mean shift} k ? = p + 1 : {plausible dispersion increase} A Least Angle Regression control chart. . .
13/ 22
LAR-EWMA control chart Control chart design
S MOOTHING C ONSTANT λ (0.1, 0.3) normal distribution (0.03, 0.05) skewed and heavy-tailed distributions C ONTROL LIMIT h h is determined to obtain a desired value of the in-control ARL via simulation, using the Polyak-Ruppert stochastic approximation algorithm.
A Least Angle Regression control chart. . .
14/ 22
Simulation Results Competitive control charts
M EAN CHANGE AND NO VARIANCE CHANGE U NSTRUCTURED SCENARIO REWMA (Hawkins, 1991,1993) MEWMA (Lowry et al.,1992)
M ULTISTAGE P ROCESSES MEWMA (Lowry et al.,1992) DEWMA (Zou and Tsung, 2008)
LEWMA (Zou and Qiu, 2009) M EAN CHANGE AND VARIANCE INCREASE PARAMETRIC P ROFILE KMW (Kim, et al. 2003)) PEWMA (Zou et al.,2007)
A Least Angle Regression control chart. . .
N ONPARAMETRIC P ROFILE NEWMA (Zou et al., 2008)
15/ 22
Simulation Results Investigation of a wide variety of out-of-control scenarios (unstructured and structured) 1 2
3
large range of size shifts different combinations of shifted components or locations: one, more than one. mean shifts and/or variance increases
performance measure: Average Run Length summary performance measure: the Relative Mean Index (Han and Tsung, 2006). Very small RMI values =⇒ best or close to the best chart.
A Least Angle Regression control chart. . .
16/ 22
Simulation Results The Relative Mean Index (Han and Tsung, 2006)
The RMI is a summary performance measure defined as RMI =
1 2 3
4
1 N 1 N ARLδ l − MARLδ l RMI = ∑ l N∑ N l=1 MARLδ l l=1
N total number of shifts ARLδ l out-of-control ARL for detecting δ l MARLδ l smallest out-of-control ARL, among the compared charts, for detecting δ l RMIl relative efficiency of a chart in detecting δ l compared to the best chart.
A Least Angle Regression control chart. . .
17/ 22
Linear Profile IC: yt,i ∼ N(0, 1). OC: yt,i ∼ N(β1 + β2 xi Shifts β1 = 0.1 β1 = 0.3 β1 = 1 β2 = 0.2 β2 = 0.4 β2 = 1.2 β1 = 0.1, β2 = 0.1 β1 =0.4,β2 = 0.2 β1 = 0.4 , β2 = 0.6 β1 = 0.6, β2 = 0.6 β1 = 0.1, ω = 1.2 β1 = 0.3, ω = 1.2 β1 = 1, ω = 1.2 β2 = 0.2, ω = 1.2 β2 = 0.4, ω = 1.2 β2 = 1.2, ω = 1.2 β1 = 0.1, β2 = 0.1, ω β1 = 0.4, β2 = 0.2, ω β1 = 0.4, β2 = 0.6, ω β1 = 0.6, β2 = 0.6, ω ω = 1.2 ω = 1.5 ω =2 RMI
= 1.2 = 1.2 = 1.2 = 1.2
A Least Angle Regression control chart. . .
PEWMA 293.08 46.16 4.65 180.73 46.84 5.41 230.79 20.69 10.39 7.11 45.88 21.55 4.44 38.68 21.80 5.12 42.47 13.74 8.60 6.41 53.39 11.06 4.37 0.13
, ω 2 ),
i = 1, . . . , 4, xi = −1, −1/3, 1/3, 1 KMW 296.40 43.04 4.29 182.79 43.66 4.99 243.60 21.14 12.34 8.16 50.95 23.77 4.29 43.46 23.93 4.97 47.36 15.14 9.96 7.29 58.17 12.35 4.71 0.19
LAR-EWMA 272.39 39.55 4.22 163.19 40.14 4.90 215.11 18.80 10.05 6.88 38.78 20.14 4.08 33.71 20.13 4.72 36.26 13.02 8.36 6.19 43.24 7.90 3.03 0.00
18/ 22
Simulation Results: RMI ARL0 = 500, λ = 0.2, τ = 1
“Unstructured” n = 15
LAR-EWMA MEWMA REWMA LEWMA 0.02 0.24 0.32 0.10
Linear Profile n=4 n = 10
LAR-EWMA PEWMA 0.00 0.13 0.01 0.07
KMW 0.19 0.11
Cubic Profile n=8 n = 15
LAR-EWMA PEWMA 0.00 0.24 0.00 0.23
KMW 0.36 0.31
Non-parametric profile n = 20 n = 40
LAR-EWMA NEWMA 0.05 0.28 0.06 0.30
Multistage process LAR-EWMA MEWMA DEWMA n = 20; ci = di = 1 0.02 0.24 0.10 n = 20; ci = 1.2; di = 0.8 0.06 0.22 0.18 A Least Angle Regression control chart. . .
19/ 22
Concluding Remarks and Future Research Advantages and Novel Aspects
The LAR-based multivariate EWMA offers an unifying approach for handling the multivariate statistical monitoring in both the unstructured and structured framework Flexible choices of F in a quite general model formulation
makes use of a relatively easier and faster variable selection method shows some appealing enhancements: a control statistic for detecting increases in total dispersion a simple tool for fault detection.
A Least Angle Regression control chart. . .
20/ 22
Concluding remarks and future research Future Research
A more general formulation for dispersion changes. Generalization of the LAR algorithm for non-normal data (but how is possible to handle dispersion changes in a distribution-free way?) Design of variable sampling interval version of the LAR-EWMA.
A Least Angle Regression control chart. . .
21/ 22
THANKS... S INCE 1222
“U NIVERSA U NIVERSIS PATAVINA L IBERTAS ” (Paduan Freedom is Complete and for Everyone)
Anatomy Theatre (1594) A Least Angle Regression control chart. . .
Galileo Galilei’s desk (≈1605) 22/ 22