Bootstrap-Based K-Sample Testing For Functional Data

arXiv:1409.4317v4 [math.ST] 28 Sep 2016

Efstathios PAPARODITIS∗ and Theofanis SAPATINAS† Department of Mathematics and Statistics, University of Cyprus, P.O. Box 20537, CY 1678 Nicosia, CYPRUS. September 29, 2016

Abstract We investigate properties of a bootstrap-based methodology for testing hypotheses about equality of certain characteristics of the distributions between different populations in the context of functional data. The suggested testing methodology is simple and easy to implement. It bootstraps the original functional dataset in such a way that the null hypothesis of interest is satisfied and it can be potentially applied to a wide range of testing problems and test statistics of interest. Furthermore, it can be utilized to the case where more than two populations of functional data are considered. We illustrate the bootstrap procedure by considering the important problems of testing the equality of mean functions or the equality of covariance functions (resp. covariance operators) between two populations. Theoretical results that justify the validity of the suggested bootstrap-based procedure are established. Furthermore, simulation results demonstrate very good size and power performances in finite sample situations, including the case of testing problems and/or sample sizes where asymptotic considerations do not lead to satisfactory approximations. A real-life dataset analyzed in the literature is also examined. Some key words: Bootstrap; Covariance Function, Functional Data; Functional ` eve Expansion; Mean Function; K-sample Principal Components; Karhunen-Lo problem.

1

INTRODUCTION

Functional data are routinely collected in many fields of research; see, e.g., Bosq (2000), Ramsay & Silverman (2002, 2005), Ferraty & Vieu (2006), Ramsay et al. (2009) and Horv´ath & Kokoszka (2012). They are usually recorded at the same, often equally spaced, time points, and with the same high sampling rate per subject of interest, a common feature of modern recording equipments. The estimation of individual curves (functions) from noisy data and the characterization of homogeneity ∗ †

Email: [email protected] Email: [email protected]

1

and of patterns of variability among curves are main concerns of functional data analysis; see, e.g., Rice (2004). When working with more than one population (group), the equality of certain characteristics of the distributions between the populations, like their mean functions or their covariance functions (resp. covariance operators) is an interesting and widely discussed problem in the literature; see, e.g., Benko et al. (2009), Panaretos et al. (2010), Zhang et al. (2010), Fremdt et al. (2012), Horv´ ath & Kokoszka (2012), Kraus & Panaretos (2012), Horv´ath et al. (2013), Fremdt et al. (2013) and Boente et al. (2014). For instance, Benko et al. (2009) and Horv´ath & Kokoszka (2012, Chapter 5) have developed asymptotic functional testing procedures for the equality of two mean functions. For the more involved problem of testing the equality of covariance functions, Panaretos et al. (2010) and Fremdt et al. (2012) have developed corresponding testing procedures in the two-sample problem. Critical points of these testing procedures are typically obtained using asymptotic approximations of the distributions of the test statistics used under validity of the null hypothesis. In this context, the main tools utilized are the functional principal components (FPC’s) and the associated Karhunen-Lo`eve expansion (KLE); see, e.g., Reiss & Ogden (2007), Gervini (2008), Yao & M¨ uller (2010), Gabrys et al. (2010) and Fremdt et al. (2013). For testing the equality of two covariance functions, Panaretos et al. (2010) have derived a functional testing procedure under the assumption of Gaussianity, while Fremdt et al. (2013) have extended such a functional testing procedure to the non-Gaussian case. Clearly, and due to the complicated statistical functionals involved, the efficacy of these functional testing procedures heavily rely on the accuracy of the obtained asymptotic approximations of the distributions of the test statistics considered under the null hypothesis. Simulation studies, however, suggest that the quality of some asymptotic approximations is questionable. This is not only true for small or moderate sample sizes, that are of paramount importance in practical applications, but also in situations where the assumptions under which the asymptotic results have been derived (e.g., Gaussianity) are not satisfied in practice; see, e.g., Fremdt et al. (2013, Section 4). To improve such asymptotic approximations, bootstrap-based testing inference for functional data have been considered by some authors in the literature. For instance, Benko et al. (2009) have considered, among other things, testing the equality of mean functions in the two-sample problem and have applied a bootstrap procedure to obtain critical values of the test statistics used. This bootstrap procedure resamples the original set of functional observations themselves without imposing the null hypothesis and, therefore, its validity rely on the particular test statistic used. That is, the bootstrap approach used does not generate functional pseudo-observations that satisfy the null hypothesis of interest. Therefore, it is not clear if this procedure can be applied to other test statistics, to different testing problems or to the case where more than two populations of functional observations are compared. Similarly, and for the case of comparing the mean functions of two populations of functional observations, Zhang et al. (2010) have considered a bootstrap procedure that generates functional pseudo-observations which do not satisfy the null hypothesis. Thus, the validity of this approach depends on the specific test statistic used. A different idea for improving asymptotic approximations 2

has been used by Boente et al. (2014) in the context of testing the equality of several covariance functions, by applying a bootstrap procedure in order to calibrate the critical values of the test statistic used. Again, this bootstrap approach is taylor made for the particular test statistic considered and does involve any resampling of the functional observations themselves. Finally, permutation tests for equality of covariance operators applied to different distance measures between two covariance functions have been considered by Pigoli et al. (2014). We investigate properties of an alternative and general bootstrap-based testing methodology for functional data, which is potentially applicable for different testing problems, different test statistics and for more than two populations. Among other things, the bootstrap-based procedure proposed can be applied to the important problem of comparing the mean functions or the covariance functions between several populations. The basic idea behind this testing methodology is to bootstrap the observed functional data set in such a way that the obtained functional pseudo-observations satisfy the null hypothesis of interest. This requirement leads to a particular bootstrap scheme that automatically generates pseudo-functional observations with identical mean functions (when testing the equality of mean functions) or identical covariance functions (when testing the equality of covariance functions) among the different populations. This common mean function is the estimated pooled mean function (when testing the equality of mean functions) and the common covariance function is the estimated pooled covariance function (when testing the equality of covariance functions) of the observed functional data. A given test statistic of interest is then calculated using the bootstrap functional pseudo-observations and its distribution is evaluated by means of Monte Carlo simulations. As an example, we show that this bootstrap-based functional testing procedure consistently estimates the distribution of the test statistics under the null hypothesis, proposed by Benko et al. (2009) and Horv´ath & Kokoszka (2012, Chapter 5), for the problem of testing the equality of two mean functions, and by Panaretos et al. (2011), Fremdt et al. (2013) and Boente et al. (2014), for the problem of testing the equality of two covariance functions. The theoretically established asymptotic validity of the suggested bootstrap-based functional testing methodology is further gauged by extensive simulations and coincides with accurate approximations of the distributions of interest in finite sample situations. These accurate approximations lead to a very good size and power behavior of the test statistics considered. The paper is organized as follows. In Section 2, we first present the suggested bootstrap-based methodology applied to the problem of testing equality of the covariance functions in the functional set-up. We then extend the discussion to the problem of testing equality of the mean functions between several groups. Some pertinent observations through remarks on the testing methodology considered are also included. In Section 3, we provide theoretical results which justify the validity of the suggested bootstrap-based testing methodology applied to some test statistics recently considered in the literature. In Section 4, we evaluate the finite sample behavior of the proposed bootstrap-based testing procedures by means of several simulations and compare the results obtained with those based on classical asymptotic approximations. An application to a real-life dataset is also presented. Some concluding remarks are made in Section 5. Finally, auxiliary results and proofs of the main results 3

are compiled in the Appendix.

2

Bootstrap-based Functional Testing Methodology

2.1

Model and Assumptions

We work with functional data in the form of random functions X := X(t) := X(ω, t), defined on a probability space (Ω, A, P) with values in the separable Hilbert-space H = L2 (I), the space of squaredintegrable R-valued functions on the compact interval I = [0, 1]. We denote by µ(t) := E[X(t)], (for almost all) t ∈ I, the mean function of X, i.e., the unique function µ ∈ L2 (I) such that E < X, x >=< µ, x >, x ∈ L2 (I). We also denote by C(t, s) := Cov[X(t), X(s)] := E[(X(t) − µ(t))(X(s) − µ(s))], t, s ∈ I, the covariance function (kernel) of X, and by C(f ) = E[hX − µ, f i(X − µ)], for f ∈ L2 (I), the R covariance operator of X. It is easily seen that C(f )(t) = I C(t, s)f (s)ds, i.e., C is an integral operator with kernel C; note that C is a Hilbert-Schmidt operator provided that E||X||2 < ∞. Throughout R the paper we assume that hf, gi = I f (t)g(t)dt, kf k2 = hf, f i, and that all functions considered are elements of the separable Hilbert-space L2 (I). Finally, the operator u ⊗ v : L2 7→ L2 is defined as (u ⊗ v)w = hv, wiu, u, v ∈ L2 , and we denote by kCkS the Hilbert-Schmidt norm of the covariance operator C. Throughout the paper, it is also assumed that we have available a collection of random functions satisfying Xi,j (t) = µi (t) + i,j (t), i = 1, 2, . . . , K, j = 1, 2, . . . , ni , t ∈ I, (1) where K (2 ≤ K < ∞) denotes the number of populations (groups), ni denotes the number of PK observations in the i-th population and N = i=1 ni denotes the total number of observations. We also assume that the K populations are independent and, for each i ∈ {1, 2, . . . , K} and j = 1, 2, . . . , ni , the i,j are independent and identical distributed random elements with E[i,j (t)] = 0, t ∈ I, and Eki,j k4 < ∞. Denote by (λk , ϕk ), k = 1, 2, . . . , the eigenvalues/eigenfunctions of the covariance operator C, i.e. Z λk ϕk (t) = C(ϕk )(t) := C(t, s)ϕk (s)ds, t ∈ I, k = 1, 2, . . . . I

Throughout the paper it is assumed that λ1 > λ2 > · · · > λp > λp+1 , i.e., there exists at least p distinct (positive) eigenvalues of the covariance operator C.

2.2

Testing the Equality of Covariance Functions

In this section, we describe the suggested bootstrap-based functional testing methodology for testing the equality of covariance functions (resp. covariance operators) for a (finite) number of populations. Since testing the equality of covariance functions is equivalent to testing the equality of covariance operators, as in Fremdt et al. (2012), we confine our attention to the former test.

4

Let XN = {Xi,j (t), i ∈ {1, 2, . . . , K}, j = 1, 2, . . . , ni , t ∈ I} be the observed collection of random functions satisfying (1). Let Ci (t, s), t, s ∈ I, be the covariance functions in the i-th population, i.e., for each i ∈ {1, 2, . . . , K}, Ci (t, s) := Cov[Xi,j (t), Xi,j (s)] := E[(Xi,j (t) − µi (t))(Xi,j (s) − µi (s))], where µi (t) := E[Xi,j (t)], j = 1, 2, . . . , ni . Our aim is to test the null hypothesis H0 : C1 = C2 = . . . = CK

(2)

H1 : ∃ (k, l) ∈ {1, 2, . . . , K} with k 6= l such that Ck 6= Cl .

(3)

versus the alternative hypothesis

Notice that the equality in the null hypothesis (2) is in the space (L2 (I × I), k · k), i.e., Ck = Cl , for any pair of indices (k, l) ∈ {1, 2, . . . , K}, with k 6= l, means that kCk −Cl k = 0, and the alternative hypothesis (3) means that kCk − Cl k > 0. 2.2.1

The Bootstrap-based Testing Procedure

Let TN be a given test statistic of interest for testing hypothesis (2) which is based on the functional observations XN . Assume, without loss of generality, that TN rejects the null hypothesis H0 when TN > dN,α , where for α ∈ (0, 1), dN,α denotes the critical value of this test. The bootstrap-based functional testing procedure for testing hypotheses (2)-(3) can be described as follows: Step 1: First calculate the sample mean functions in each population X i,ni (t) =

ni 1 X Xi,j (t), ni

t ∈ I,

i ∈ {1, 2, . . . , K}.

j=1

Step 2: Calculate the residual functions in each population, i.e., for each i ∈ {1, 2, . . . , K}, ˆi,j (t) = Xi,j (t) − X i,ni (t), Step 3:

j = 1, 2, . . . , ni .

∗ (t), t ∈ I, i ∈ {1, 2, . . . , K}, Generate bootstrap functional pseudo-observations Xi,j

j = 1, 2, . . . , ni , according to where

t ∈ I,

∗ Xi,j (t) = X i,ni (t) + ∗i,j (t),

t ∈ I,

(4)

∗i,j (t) = ˆI,J (t), t ∈ I,

and (I, J) is the following pair of random variables. The random variable I takes values in the set {1, 2, . . . , K} with probability P (I = i) = ni /N for i ∈ {1, 2, . . . , K}, and, given I = i, the random variable J has the discrete uniform distribution in the set {1, 2, . . . , ni }, i.e., P (J = j | I = i) = 1/ni for i ∈ {1, 2, . . . , K}, j = 1, 2, . . . , ni . Step 4: Let TN∗ be the same statistic as TN but calculated using the bootstrap functional pseudo∗ ∗ , i ∈ {1, 2, . . . , K}; j = 1, 2, . . . , n . Denote by D ∗ observations Xi,j i N,T the distribution function of TN given the functional observations XN . Step 5: For any given α ∈ (0, 1), reject the null hypothesis H0 if and only if TN > d∗N,α , 5

∗ , i.e., D ∗ (d∗ ) = 1 − α. where d∗N,α denotes the α-quantile of DN,T N,T N,α ∗ Notice that the distribution DN,T can be evaluated by means of Monte-Carlo.

Clearly, and since the random functions ∗i,j (t) are generated independently from each other, for any two different pairs of indices, say (i1 , j1 ) and (i2 , j2 ), the corresponding bootstrap functional pseudoobservations Xi∗1 ,j1 (t) and Xi∗2 ,j2 (t) are independent. Furthermore, observe that the random selection of the error function ∗i,j (t) in Step 4 of the above bootstrap algorithm, is equivalent to selecting ∗i,j (t) randomly with probability 1/N from the entire set of available and estimated residual functions {b r,s (t) : r = 1, 2, . . . , K and s = 1, 2, . . . , nr }. Hence, conditional on the observed functional data ∗ (t) have the following first and second order properties: XN , the functional pseudo-observations Xi,j ∗ E[Xi,j (t)]

= X i,ni (t) +

E[∗i,j (t)]

K ni 1 XX = X i,ni (t) + b i,j (t) = X i,ni (t), N

t ∈ I,

i=1 j=1

since

Pni

i,j (t) j=1 b

=

Pni

j=1 (Xi,j (t)

− X i,ni (t)) = 0, within each population i ∈ {1, 2, . . . , K}.

Moreover, ∗ ∗ Cov[Xi,j (t), Xi,j (s)]

=

E[∗i,j (t)∗i,j (s)]

K ni 1 XX = b i,j (t)b i,j (s) N i=1 j=1

=

K X i=1

=

K X ni i=1

where

ni 1 N ni

N

ni X

(Xi,j (t) − X i,ni (t))(Xi,j (s) − X i,ni (s))

j=1

bi,n (t, s) = C bN (t, s), C i

t, s ∈ I,

ni 1 X b Ci,ni (t, s) = (Xi,j (t) − X i,ni (t))(Xi,j (s) − X i,ni (s)), ni

t, s ∈ I,

j=1

bN (t, s) is the is the sample estimator of the covariance function Ci (t, s) for the i-th population and C corresponding pooled covariance function estimator. Thus, and conditional on the observed functional data XN , the bootstrap generated functional ∗ (t) have, within each population i ∈ {1, 2, . . . , K}, the same mean function pseudo-observations Xi,j

X i,ni (t), which may be different for different populations. Furthermore, the covariance function in bN (t, s). That is, each population is identical and equal to the pooled sample covariance function C ∗ (t), satisfy the null hypothesis (2). This basic property of the functional pseudo-observations Xi,j ∗ (t)’s allows us to use these bootstrap observations to evaluate the distribution of some test the Xi,j

statistic TN of interest under the null hypothesis. This is achieved by using the distribution of TN∗ as an estimator of the distribution of TN , where TN∗ is the same statistical functional as TN calculated using ∗ (t), i = 1, 2, . . . , K, j = 1, 2, . . . , n , t ∈ I}. the bootstrap functional pseudo-observations X∗N = {Xi,j K

6

Since, as we have seen, the set of pseudo-observations used to calculate TN∗ satisfy the null hypothesis, we expect that the distribution of the pseudo random variable TN∗ will mimic correctly the distribution of TN under the null. In the next section we show that this is indeed true for two particular test statistics proposed in the literature. However, since our bootstrap methodology is not designed or tailor made for any particular test statistic, its range of validity is not restricted to these two particular test statistics.

2.3

Testing the Equality of Mean Functions

We assume again that we have available a collection of curves XN , satisfying (1). Recall that µi (t), t ∈ I denote the mean functions of the curves in the i-th population, i.e., for each i ∈ {1, 2, . . . , K}, µi (t) := E[(Xi,j (t)], j = 1, 2, . . . , ni . The basic idea used in the bootstrap resampling algorithm of Section 2.2.1 enables its adaption/modification to deal with different testing problems related to the comparison of K populations of functional observations. For instance, suppose that we are interested in testing the null hypothesis that the K populations have identical mean functions, i.e., H0 : µ1 = µ2 = · · · = µK

(5)

H1 : ∃ (k, l) ∈ {1, 2, . . . , K} with k 6= l such that µk 6= µl .

(6)

versus the alternative hypothesis

As in the previous section, equality in the null hypothesis (5) is in the space (L2 (I), k · k), i.e., µk = µl , for any pair of indices (k, l) ∈ {1, 2, . . . , K}, with k 6= l, means that kµk − µl k = 0, and the alternative hypothesis (6) means that kµk − µl k > 0. Such a testing problem can be easily addressed by changing appropriately Step 3 of the bootstrap resampling algorithm of Section 2.2.1. In particular, we replace equation (4) in Step 3 of this algorithm by the following equation + Xi,j (t) = X N (t) + + i,j (t),

where X N (t) =

K ni 1 XX Xi,j (t), N

t ∈ I,

(7)

t ∈ I,

i=1 j=1

is the pooled mean function estimator and + i,J (t), t ∈ I, where J is a discrete random i,j (t) = b variable with P (J = j) = 1/ni for every j = 1, 2, . . . , ni , i ∈ {1, 2, . . . , K}. Thus, the bootstrap error functions + i,j (t) for population i appearing in equation (7) are generated by randomly selecting a residual function from the set of estimated residual functions b i,j (t) belonging to the same population i ∈ {1, 2, . . . , K}. This ensures that the covariance structure of the functional observations in each 7

population is retained by the bootstrap resampling algorithm, which may be different for different populations, despite the fact that the bootstrap procedure generates K populations of independent bootstrap functional pseudo-observations that have the same mean function. In particular, conditional + on the observed functional data XN , we have for the bootstrap functional pseudo-observations Xi,j (t),

that, + E[Xi,j (t)] = X N (t), t ∈ I,

and + + Cov[Xi,j (t), Xi,j (s)]

=

+ E[+ i,j (t)i,j (s)]

ni 1 X b i,j (t)b i,j (s) = ni j=1

=

1 ni

ni X

(Xi,j (t) − X i,ni (t))(Xi,j (s) − X i,ni (s))

j=1

t, s ∈ I.

bi,n (t, s), =C i

◦ (t) = X (t) + ∗ (t), t ∈ I, to generate the bootstrap pseudoRemark 2.1 Notice that if we use Xi,j N i,j

observations instead of equation (7) with ε∗i,j (t) defined as in Step 3 of the algorithm in Section 2.2.1, ◦ (t) will have in the K groups an identical mean function equal to X (t) and an identical then the Xi,j N b covariance function equal to CN (t, s). This may be of particular interest if one is interested in testing

simultaneously the equality of mean functions and covariance functions between the K populations. Remark 2.2 If distributional assumptions have been imposed on the observed random functions Xi,j (t), then efficiency considerations may suggest that such assumptions should be also taken into account in the implementation of the bootstrap resampling algorithms which are used to generated the bootstrap functional pseudo-observations. For instance, the assumption of Gaussianity of the random paths Xi,j (t), t ∈ I, can be incorporated in our bootstrap testing algorithm by allowing for the functional bootstrap pseudo-observations to follow a Gaussian processes on I with a mean and covariance function specified according the null hypothesis of interest.

3

Bootstrap Validity

In this section, we establish the validity of the introduced bootstrap-based functional testing methodology applied to some test statistics recently proposed in the literature for the important problems of testing the equality of mean functions or covariance functions between two populations.

8

3.1 3.1.1

Testing the equality of two covariance functions Test Statistics and Limiting Distributions

For testing the equality of two covariance operators, it is natural to evaluate the Hilbert-Schmidt norm of the difference of the corresponding sample covariance operators Cb1 and Cb2 , defined as ni 1 X Cbi = (Xi,j − X i,ni ) ⊗ (Xi,j − X i,ni ), ni

i = 1, 2,

j=1

or, equivalently, as ni 1 X b Ci (f )(t) = hXi,j − X i,ni , f i(Xi,j (t) − X i,ni (t)), ni

f ∈ L2 (I),

t ∈ I,

i = 1, 2.

j=1

Such an approach has been recently proposed by Boente et al. (2014) by considering the test statistic TN = N kCb1 − Cb2 k2S , where N = n1 +n2 . If n1 /N → θ1 ∈ (0, 1), EkXi,1 k4 < ∞, i ∈ {1, 2}, and the null hypothesis H0 given in (2) with K = 2 is true, then, as n1 , n2 → ∞, Boente et al. (2014) showed that TN converges weakly P 2 to ∞ l=1 λl Zl , where Zl are independently distributed standard Gaussian random variables and λl are the eigenvalues of the pooled operator B = θ1−1 B1 + (1 − θ1 )−1 B2 , where Bi is the covariance operator √ of the limiting Gaussian random element Ui to which ni (Cbi − Ci ) converges weakly as ni → ∞, i = 1, 2. Since the limiting distribution of TN depends on the unknown infinite eigenvalues λl , l ≥ 1, implementation of this asymptotic result for calculating critical values of the test is difficult. With this in mind, Boente et al. (2014) have proposed a bootstrap calibration procedure of the distribution of the test statistic TN . Another, related, approach for testing the equality of two covariance functions, is to evaluate the b1,n (t, s) and C b2,n (t, s), t, s ∈ I, of each group distance between the sample covariance functions C 1 2 b and the pooled sample covariance function CN (t, s), t, s ∈ I, based on the entire set of functional b1,n (t, s)− C b2,n (t, s), t, s ∈ I, on certain directions observations. Therefore, looking at projections of C 1 2 reduces the dimensionality of the problem. Such approaches have been considered by Panaretos et al. (2010) (for Gaussian curves) and Fremdt et al (2012), (for non-Gaussian curves), where the asymptotic distributions of the corresponding test statistics proposed under the null hypothesis have been derived bk , ϕ More specifically, denote by (λ bk ), k = 1, 2, . . . , N , the eigenvalues/eigenfunctions of the pooled bN (t, s), i.e., sample covariance operator CbN defined by the kernel C Z ck ϕ λ bk (t) = CbN (ϕ bk )(t) =

0

1

bN (t, s)ϕ C bk (s)ds,

9

t ∈ I,

k = 1, 2, . . . , N,

b1 ≥ λ b2 ≥ · · · . (We can and will assume that the ϕ with λ bk (t), k = 1, 2, . . . , N , t ∈ I, form an orthonormal system.) Select a natural number p and consider, for i = 1, 2, . . . , p, the projections Z bi >= (Xk,j (t) − X k,nk (t))ϕ bi (t)dt, j = 1, 2, . . . , nk , k = 1, 2. b ak,j (i) =< Xk,j − X k,nk , ϕ I

bk,n , k = 1, 2, with elements For 1 ≤ r, m ≤ p, consider the matrices A k nk X bk,n (r, m) = 1 b ak,j (r)b A ak,j (m), k nk

k = 1, 2.

j=1

b2,n (r, m) is the projection of the difference b N (r, m) := A b1,n (r, m) − A Notice that, for 1 ≤ r, m ≤ p, ∆ 2 1 b2,n (t, s) in the direction of ϕ b1,n (t, s) − C br (t)ϕ bm (s), t, s ∈ I. C 2

1

Panaretos et al. (2010) considered then the test statistic (G)

Tp,N =

n1 n2 N

X 1≤r,m≤p

b 2 (r, m) ∆ N . br λ bm 2λ

They showed, that, under the assumption that Xi,1 (t), i ∈ {1, 2}, t ∈ I, are Gaussian processes, and if n1 , n2 → ∞ such that n1 /N → θ1 ∈ (0, 1), EkXi,1 k4 < ∞, i ∈ {1, 2} and the null hypothesis H0 (G)

given in (2) with K = 2 is true, then, Tp,N converges weakly to a χ2p(p+1)/2 distribution. The non-Gaussian case has been recently investigated by Fremdt et al. (2012). In particular, b N = (∆ b N (r, m))r,m=1,2,...,p and defined ξbN = vech(∆ b N ), i.e., the vector they considered the matrix ∆ b N . Fremdt et al. (2012) proposed then containing the elements on and below the main diagonal of ∆ the test statistic Tp,N =

n1 n2 bT b −1 b ξ L ξN , N N N

b N is an estimator of the (asymptotic) covariance matrix of ξbN . They showed that if n1 , n2 → ∞ where L such that n1 /N → θ1 ∈ (0, 1), EkXi1 k4 < ∞, i ∈ {1, 2}, and the null hypothesis H0 given in (2) with K = 2 is true, then, T2,N converges weakly to a χ2p(p+1)/2 distribution. Furthermore, under the same set of assumptions, consistency of the test Tp,N has been established under the alternative, that is when the covariance functions C1 and C2 differ. 3.1.2

Consistency of the Bootstrap

We apply the bootstrap procedure introduced in Section 2.2.1 to approximate the distributions of the (G)

∗ (t), i ∈ {1, 2}, test statistics TN , Tp,N and of Tp,N under the null hypothesis. To this end, let Xi,j

j = 1, 2, . . . , ni , t ∈ I, be the bootstrap functional pseudo-observations generated according to this bootstrap procedure. Let TN? = N kCb1? − Cb2? k2S ,

10

where Cb1? and Cb2? are the sample covariance operators of the two groups but calculated using the the ∗ (t), i ∈ {1, 2}, j = 1, 2, . . . , n , t ∈ I. Let bootstrap functional pseudo-observations Xi,j i ∗(G)

Tp,N =

n1 n2 N

2

X 1≤r,m≤p

b ∗ (r, m) 1∆ N , b∗ λ b∗ 2 λ r m

b∗ are the same statistics as ∆ br appearing in T (G) but calculated b ∗2 (r, m) and λ b 2 (r, m) and λ where ∆ r N N p,N ∗ (t), i ∈ {1, 2}, j = 1, 2, . . . , n , t ∈ I. Similarly, using the bootstrap functional pseudo-observations Xi,j i

let n1 n2 b∗T b ∗−1 b∗ ξ L ξ , N N N N b N appearing in Tp,N but calculated using the are the same statistics as ξbN and L ∗ Tp,N =

∗ and L b∗ where ξbN N

∗ (t), i ∈ {1, 2}, j = 1, 2, . . . , n , t ∈ I. The following bootstrap functional pseudo-observations Xi,j i

results are then true. Theorem 3.1 If EkXi,1 k4 < ∞, i ∈ {1, 2}, and n1 /N → θ1 ∈ (0, 1), then, as n1 , n2 → ∞,

sup P TN∗ ≤ x | XN − PH0 TN ≤ x → 0, in probability, x∈
τp > τp+1 , i.e., that there exists at least p distinct (positive) eigenvalues of the operator Z. Let Z ˆ a ˆi =< X 1,n1 − X 2,n2 , φi >= (X 1,n1 (t) − X 2,n2 (t))φˆi (t)dt, I

12

i = 1, 2, . . . , p,

be the projection of the difference X 1,n1 (t) − X 2n2 (t), t ∈ I, into the linear space spanned by φˆ1 (t), φˆ2 (t),. . . ,φˆp (t), t ∈ I, the eigenfunctions related to the sample estimator ZˆN (t, s) of the kernel Z(t, s), t, s ∈ I. Based on the above, Horv´ ath & Kokoszka (2012, Chapter 5) considered the following test statistics (1)

Sp,N =

p ˆ2k n1 n2 X a N τˆk

(2)

and Sp,N =

k=1

If n1 /N → θ1 ∈ (0, 1), EkXi,1

k4

p n1 n2 X 2 a ˆk . N k=1

< ∞, i ∈ {1, 2}, and the null hypothesis H0 given in (5) with K = 2 (1)

(2)

is true, then, as n1 , n2 → ∞, they showed that, Sp,N converges weakly to a χ2p -distribution while Sp,N P converges weakly to pk=1 τk Nk2 . Under the assumption that µ1 (t) − µ2 (t), t ∈ I, is not orthogonal to the linear span of φ1 (t), φ2 (t), . . . , φp (t), t ∈ I, they have also showed consistency, in the sense that, if n1 /N → θ1 ∈ (0, 1), EkXi,1 k4 < ∞, i ∈ {1, 2}, and the alternative hypothesis H1 given in (6) with (1)

(2)

K = 2 is true, then, as n1 , n2 → ∞, Sp,N → ∞ and Sp,N → ∞, in probability. 3.2.2

Consistency of the Bootstrap (1)

(2)

To approximate the distribution of the test statistics SN , Sp,N and Sp,N , we apply the bootstrap + procedure proposed in Section 2.3. To this end, let Xi,j (t), i ∈ {1, 2}, j = 1, 2, . . . , ni , t ∈ I, be the

bootstrap functional pseudo-observations generated according to this bootstrap algorithm and define n1 n2 + + kX 1,n1 − X 2,n2 k2 , N 2 p ˆ+ n1 n2 X a k , and = N τˆ+ k=1 k

+ = SN +(1)

Sp,N

+(2)

Sp,N =

p n1 n2 X +2 a ˆk N k=1

(1)

(2)

be the same statistic as SN , Sp,N and Sp,N , respectively, but calculated using the bootstrap functional + pseudo-observations Xi,j (t), i ∈ {1, 2}, j = 1, 2, . . . , ni , t ∈ I. The following results are then true.

Theorem 3.4 If EkXi,1 k4 < ∞, i ∈ {1, 2}, and n1 /N → θ1 ∈ (0, 1), then, as n1 , n2 → ∞,

+ sup P SN ≤ x | XN − PH0 SN ≤ x → 0, in probability, x∈R

where PH0 (SN ≤ x), x ∈ , b a∗2,j (r) =< X2,j − X 2,n2 , ϕ b∗r > b a∗1,j (r) =< X1,j − X 1,n1 , ϕ Pni ∗ ∗ b∗ b∗r , r = 1, 2, . . . , N , and X i,ni (t) = n−1 j=1 Xi,j (t), t ∈ I, i = 1, 2. Furthermore, λr and ϕ i b ∗ (t, s) = denote the eigenvalues and eigenfunctions, respectively, of the pooled covariance matrix C N P b ∗ (t, s) + (n2 /N )C b ∗ (t, s), where C b ∗ = n−1 ni (X ∗ (t) − X ∗ (t))(X ∗ (s) − X ∗ (s)), (n1 /N )C i,ni i,ni 1,n1 2,n2 i,j i,j i,n1 j=1 i ∗ ∗ ∗ ∗ b b b b t, s ∈ I, i = 1, 2. We assume that λ ≥ λ ≥ λ ≥ · · · ≥ λ , and recall that N = n1 + n2 . We first 1

2

3

N

establish the following useful lemmas. Lemma 6.1 Under the assumptions of Theorem 3.3 we have, conditionally on XN , that, for 1 ≤ i ≤ p, b∗ − λ bi | = OP (N −1/2 ) (i) |λ i

and

(ii) kϕ b∗i − b c∗i ϕ bi k = OP (N −1/2 ),

where b c∗i = sign(< ϕ b∗i , ϕ bi >). Proof: We fist show that for r ∈ {1, 2}, nr

∗ ∗

−1/2 X  ∗ ∗ bN (t, s) (Xr,j (t) − X r,nr (t))(Xr,j (s) − X r,nr (s)) − C

nr

= OP (1)

(9)

j=1

and

nr

∗

−1/2 X  ∗

n X (t) − X (t)

r

= OP (1). r,j r,nr j=1

20

(10)

∗ (t) = X ∗ (t) − X ∗ (t) and Y e ∗ (t) = X ∗ (t) − X r,nr (t). Then Let Yr,j r,nr r,j r,j r,j

Z 1Z 1X nr nr

 2 X  ∗ 2 2

∗ ∗ ∗ bN (t, s) bN (t, s) dtds E nr−1/2 Yr,j (t)Yr,j (s) − C E Yer,j (t)Yer,j (s) − C

≤ nr 0 0 j=1

j=1

nr



−1/2 X  ∗

2 ∗ ∗ ∗ + 2 E nr Yr,j (t)Yr,j (s) − Yer,j (t)Yer,j (s) j=1

Z

1Z 1

=2 0

 2 bN (t, s) dtds + OP (1), E ε∗r,1 (t)ε∗r,1 (s) − C

0 ∗

−1/2

∗ (t) = Y ∗ e ∗ (t) + (X r,nr (t) − X using the fact that Yr,j ). Now, since r,nr (t)) = εr,j (t) + OP (nr r,j P P nr 2 2 ∗ ∗ 2 −1 b b [b εr,j (t)b εr,j (s)− CN (t, s)] = OP (1), assertion (9) follows E(ε (t)ε (s)− CN (t, s)) = N r,1

r,1

r=1

j=1

by Markov’s inequality. Assertion (10) follows by the same inequality and because nr nr

X  ∗ ∗

−1/2 X e ∗ 2

2 X (t) ≤ 2 E n Y E n−1/2 X (t) −

r,nr r r,j + OP (1) r r,j j=1

j=1

Z =

1

E(ε∗r,1 (t))2 dt + OP (1) = OP (1).

0

Using (9), we get that ∗ ∗ ∗ bN bN kS ≤ n1 kC b1,n b2,n bN kS + n2 kC bN kS = n1 OP (n−1/2 ) + n2 OP (n−1/2 ) = OP (N −1/2 ). kC −C −C −C 1 2 1 2 N N N N (11)

Using (11) and Lemmas 2.2 and 2.3 of Horv´ath & Kokoszka (2012), we have that, for 1 ≤ i ≤ p, ∗ b∗ − λ bi | ≤ kC bN bN kS = OP (N −1/2 ), |λ −C i

and ∗ bN bN kS ) = OP (N −1/2 ). kϕ b∗i − b c∗i ϕ bi k = OP (kC −C

This completes the proof of the lemma. Let



nr X ∗ ∗ e∗ (i, j) = 1 A < X1,k − X 1,n1 , b c∗i ϕ bi >< X1,k − X 1,n1 , b c∗j ϕ bj >, 1,n1 nr k=1

n2 1 X ∗ ∗ ∗ e A2,n2 (i, j) = < X2,k − X 2,n2 , b c∗i ϕ bi >< X2,k − X 2,n2 , b c∗j ϕ bj > n2 k=1

b ∗ = ((∆ b ∗ (i, j)), 1 ≤ i, j ≤ p)), where ∆ b ∗ (i, j) = (A e∗ (i, j) − A e∗ (i, j)). and ∆ 1,n1 2,n2 N N N Lemma 6.2 Under the assumptions of Theorem 3.3 we have, conditionally on XN , that r  n 1 n2  b ∗ e ∗ = oP (1). ∆N − ∆ N N 21

∗ −X ∗ −X Proof: Let a ˘∗r,j (i) =< Xr,j b∗i >, e c∗i ϕ a∗r,j (i) =< Xr,j bi >. We first show that we can r,nr , ϕ r,nr , b b ∗ (i, j) by a replace b a∗ (i) in ∆ ˘∗ (i). For this, notice that, for r ∈ {1, 2}, we have r,j

r

N

r,j

r nr nr

∗ n1 n2 1 X n1 n2 1 X ∗ ∗ ∗ ∗ b ar,k (i)b ar,k (j) − a ˘r,k (i)˘ ar,k (j) = b a∗r,k (i) − a ˘∗r,k (i) b ar,k (j) N nr N nr k=1 k=1 r nr
∗ n1 n2 1 X − a ˘∗r,k (j) − b a∗r,k (j) a ˘r,k (i) N nr k=1

and r

nr
∗ n1 n2 1 X b a∗r,k (i) − a ˘∗r,k (i) b ar,k (j) = N nr

r

k=1

×

1 nr

n1 n2 N nr X

Z 0

1Z 1 0

∗

(X r,nr (t) − X r,nr (t))ϕ b∗i (t)dt ∗

∗ (Xr,k (s) − X r,nr (s))ϕ b∗j (s)ds

k=1

= 0. ˘ ∗ (i, j) be the same expression as ∆ b ∗ (i, j) with b Let ∆ a∗r,j (l) replaced by a ˘∗r,j (l), l ∈ {i, j}, and notice N N p b ∗ (i, j) − ∆ ˘ ∗ (i, j)| = oP (1). Furthermore, that by the previous considerations, n1 n2 /N |∆ N N r r Z 1Z 1 n1 h i X 1 n1 n2 ˘ ∗ ∗ ∗ e ∗N (i, j)) = n1 n2 bN (t, s) (s) − X 1,n1 (s)) − C (∆N (i, j) − ∆ (t) − X 1,n1 (t))(X1,k (X1,k N N 0 0 n1 k=1   × ϕ b∗i (t)ϕ b∗j (s) − b c∗i b c∗j ϕ bi (t)ϕ bj (s) dtds r Z Z n2 h i n1 n2 1 1 1 X ∗ ∗ bN (t, s) (t) − X 2,n2 (t))(X2,k (X2,k (s) − X 2,n2 (s)) − C − N 0 0 n2 k=1   × ϕ b∗i (t)ϕ b∗j (s) − b c∗i b c∗j ϕ bi (t)ϕ bj (s) dtds = V1,N + V2,N , with an obvious notation for Vr,N , r ∈ {1, 2}. Now, r Z Z nr h i n1 n2 1 1 1 X ∗ ∗ bN (t, s) (t) − X r,nr (t))(Xr,k (s) − X r,nr (s)) − C (Xr,k |Vr,N | ≤ N nr 0 0 k=1   ∗ ∗ × ϕ bi (t) − b ci ϕ bi (t) ϕ b∗j (s) dtds r Z Z nr h i n1 n2 1 1 1 X ∗ ∗ bN (t, s) + (s) − X r,nr (s)) − C (Xr,k (t) − X r,nr (t))(Xr,k N nr 0 0 k=1   ∗ ∗ × b cj ϕ bj (s) − ϕ bj (t) b c∗i ϕ bi (t) dtds r nr h i 1 n1 n2

1 X ∗ ∗ bN (t, s) ≤√ (Xr,k (t) − X r,nr (t))(Xr,k (s) − X r,nr (s)) − C

√

nr N nr k=1 n o × kϕ b∗i − b c∗i ϕ bi k + kϕ b∗j − b c∗j ϕ bj k  1 r n n n  1 2 = OP √ OP (kϕ b∗i − b c∗i ϕ bi k) + OP (kϕ b∗j − b c∗j ϕ bj k) = OP (N −1/2 ), nr N 22

because of (9) and Lemma 6.1. This completes the proof of the lemma.



Proof of Theorem 3.1: Let S be the Hilbert space of Hilbert–Schmidt operators endowed with P the inner product hΨ1 , Ψ2 iS = ∞ j=1 hΨ1 (ej ), Ψ2 (ej )i for Ψ1 , Ψ2 ∈ S, where {ej : j = 1, 2, . . .} is an orthonormal basis in H. Notice that Cb∗ ∈ S, i = 1, 2. Since i

∗ X i,ni

= X i,ni +

−1/2 OP (ni )

and

−1/2 ni

ni X ∗ ∗ (Xi,j − X i,ni ) = OP (1),

i = 1, 2,

j=1

we get Cbi∗ =

ni 1 X ∗ ∗ ∗ ∗ (Xi,j − X i,ni ) ⊗ (Xi,j − X i,ni ) ni j=1

=

ni 1 X ∗ ∗ (Xi,j − X i,ni ) ⊗ (Xi,j − X i,ni ) + OP (n−1 ), ni

i = 1, 2,

j=1

∗ − X ∗ where the random variables (Xi,j i,ni ) ⊗ (Xi,j − X i,ni ) are, conditional on XN , independent

and identically distributed. By a central limit theorem for triangular arrays of independent and identically distributed S-valued random variables (see, e.g., Politis & Romano (1992, Theorem 4.2)), √ we get, conditionally on XN , that ni (Cbi∗ − CbN ) converges weakly to a Gaussian random element U in S with mean zero and covariance operator B = θ1 B1 + (1 − θ1 )B2 as ni → ∞. Here, Bi is √ the covariance operator of the limiting Gaussian random element Ui to which ni (Cbi − Ci ) converges weakly as ni → ∞. By the independence of the bootstrap random samples between the two populations, we have, conditional on XN , TN∗ = N kCb1∗ − Cb2∗ k2S = N hCb1∗ − CbN , Cb1∗ − CbN iS + N hCb2∗ − CbN , Cb2∗ − CbN iS =

N √ N √ k n1 (Cb1∗ − CbN )k2S + k n2 (Cb2∗ − CbN )k2S . n1 n2

Hence, taking into account the above results and that n1 /N → θ1 , we have that N kCb1∗ − Cb2∗ k2S P ˜ 2 ˜ converges weakly to ∞ λ l Z as n1 , n2 → ∞, where λl , l ≥ 1, are the eigenvalues of the operator l=1

l

Be = θ1−1 B + (1 − θ1 )−1 B and Zl , l ≥ 1, are independent standard (real-valued) Gaussian distributed random variables. Since B1 = B2 , the assertion follows.

 0

∗ ∗ L b ∗−1 ξb∗ with ξb∗ = vech(∆ b ∗ ) and L b ∗ is an Proof of Theorem 3.3: Recall that Tp,N = (n1 n2 /N )ξbN N N N N N p ∗ b estimator of the covariance matrix of n1 n2 /N ξN . The element of the latter matrix corresponding to p p the covariance of n1 n2 /N ξb∗ (i1 , j1 ) and n1 n2 /N ξb∗ (i2 , j2 ), 1 ≤ i1 ≤ j1 ≤ p and 1 ≤ i2 ≤ j2 ≤ p, N

N

23

∗ (i , j ), ξb∗ (i , j )) and is estimated by is denoted by l(ξbN 1 1 N 2 2 ∗ ∗ b l(ξbN (i1 , j1 ), ξbN (i2 , j2 )) =

n1 n2 n 1 X ∗ b∗ ϕ b a∗1,j (i1 )b a∗1,j (j1 )b a∗1,j (i2 )b a∗1,j (j2 )− < C b∗j1 > 1,n1 bi1 , ϕ n1 + n2 n1 j=1

n2 n1 n 1 X b a∗2,j (i1 )b a∗2,j (j1 )b a∗2,j (i2 )b a∗2,j (j2 ) n1 + n2 n2 j=1 o ∗ ∗ b2,n b2,n −< C ϕ b∗ , ϕ b∗j2 > . 2 i1 2 i2 ∗ b∗ ϕ × 1,n1 bi2 , ϕ

o

+

b ∗ . To establish the b2,n (r, m) be the (r, m)th element of the matrix ∆ b N (r, m) = A b1,n (r, m) − A Let ∆ 2 1 N theorem, it suffices to show that, under the assumptions made, the following assertions (12) and (13) are true.

 rn n   1 2 b∗ L ( ∆N (i, j), 1 ≤ i, j ≤ p) XN ⇒ L((∆(i, j), 1 ≤ i, j ≤ p) , N

(12)

where ∆ = (∆(i, j), 1 ≤ i, j ≤ p) is a Gaussian random matrix with E(∆(i, j)) = 0, 1 ≤ i, j, ≤ p having a positive definite covariance matrix Σ with elements σ(i1 , j1 , i2 , j2 ) = Cov(∆(i1 , j1 ), ∆(i2 , j2 )), 1 ≤ i1 , i2 , j1 , j2 ≤ p, given by n σ(i1 , j1 , i2 , j2 ) =(1 − θ) E(< X1,j − µ1 , ϕi1 >< X1,j − µ1 , ϕj1 >< X1,j − µ1 , ϕi2 >< X1,j − µ1 , ϕj2 >) o − < Cϕi1 , ϕj1 >< Cϕi2 , ϕj2 > n + θ E(< X2,j − µ2 , ϕi1 >< X2,j − µ2 , ϕj1 >< X2,j − µ2 , ϕi2 >< X2,j − µ2 , ϕj2 >) o − < Cϕi1 , ϕj1 >< Cϕi2 , ϕj2 > , where C = (1 − θ)C1 + θC2 . Furthermore, ∗ ∗ b l(ξbN (i1 , j1 ), ξbN (i2 , j2 )) − b c∗i1 b c∗j1 b c∗i2 b c∗j2 σ(i1 , j1 , i2 , j2 ) → 0, in probability,

(13)

for all 1 ≤ i1 , i2 , j1 , j2 ≤ p. To establish (12), recall that by Lemma 6.2 it suffices to consider the asymptotic distribution of p ∗ −X e ∗ . Let Ye ∗ (i, j) =< X ∗ − X r,nr , b n1 n2 /N ∆ c∗i ϕ bi >< Xr,k c∗j ϕ bj > and notice that r,nr , b N r,k r,k n1 n2 1 X  p p 1 X ∗ ∗ ∗ e e n1 n2 /N ∆N (i, j) = n1 n2 /N Y1,k (i, j) − Ye2,k (i, j) . n1 n2 k=1

k=1

∗ (i, j), they do not affect the limiting distribution of the two Since b c∗i and b c∗j change solely the sign of Yer,k bi >< sums above. Thus, without loss of generality, we set b c∗ = b c∗ = 1. Let Y˘ ∗ (i, j) =< X ∗ −X r,nr , ϕ i

∗ Xr,k

j

r,k

r,k

∗ (i, j)) = λ bi 1{i=j} , and consider instead of the distribution of − X r,nr , ϕ bj >, notice that E(Y˘r,k

24

p p e ∗ the distribution of the asymptotically equivalent sum n1 n2 /N Z ∗ (i, j), given by n1 n2 /N ∆ N N n1 n2   1 X p p 1 X ∗ ∗ ∗ ˘ Y1,k (i, j) − Y˘2,k (i, j) n1 n2 /N ZN (i, j) = n1 n2 /N n1 n2 k=1

k=1

n1 n2 p p 1 X 1 X ∗ ∗ b bi ) ˘ = n2 /N √ (Y1,k (i, j) − 1{i=j} λi ) − n1 /N √ (Y˘2,k (i, j) − 1{i=j} λ n1 n2 k=1 k=1 p p ∗ ∗ = n2 /N Z1,N (i, j) − n1 /N Z2,N (i, j), ∗ (i, j), r ∈ {1, 2}. Notice that, conditionally on X , Z ∗ (i, j) is with an obvious notation for Zr,N N N ∗ (i, j) and Z ∗ (i, j), where for r ∈ {1, 2}, distributed as the difference of the two independent sums Z1,N 2,N bi 1{i=j} , Z ∗ (i, j) is a sum of the independent and identically distributed random variables Y˘ ∗ (i, j)− λ r,N

r,k

k = 1, 2, . . . , nr . Furthermore,

∗ (i, j)) E(Zr,N

= 0 and since

ε∗r,k

=

∗ Xr,k

− X r,nr , we get

Cov(Zr,k (i1 , j1 ), Zr,k (i2 , j2 )) = E(Zr,k (i1 , j1 )Zr,k (i2 , j2 )) h i bi1 >< ε∗1,k , ϕ bj1 >< ε∗1,k , ϕ bi2 >< ε∗1,k , ϕ bj2 > = E < ε∗1,k , ϕ bi λ b −λ 1 i2 1{i1 =j1 } 1{i2 =j2 } =

nr 2 X nr 1 X < εbr,k , ϕ bi1 >< εbr,k , ϕ bj1 >< εbr,k , ϕ bi2 >< εbr,k , ϕ bj2 > N nr r=1

k=1

bi λ b −λ 1 i2 1{i1 =j1 } 1{i2 =j2 } h → (1 − θ) E < X1,k − µ1 , ϕi1 >< X1,k − µ1 , ϕj1 >< X1,k − µ1 , ϕi2 > i h × < X1,k − µ1 , ϕj2 > + θ E < X2,k − µ2 , ϕi1 >< X2,k − µ2 , ϕj1 > i × < X2,k − µ2 , ϕi2 >< X2,k − µ2 , ϕj2 > − λi1 λi2 1{i1 =j1 } 1{i2 =j2 } = σ(i1 , j1 , i2 , j2 ),

(14)

bi >. Thus, in probability, by the weak law of large numbers and using < εbr,l , ϕ bi >=< Xr,l − X r,nr , ϕ by a multivariate central limit theorem for triangular arrays of real valued random vectors, we get ∗ ) ⇒ Z, where Z is a Gaussian distributed p × p random matrix, with E(Z(i, j)) = 0 and that L(Zr,N

Cov(Z(i1 , j1 ), Z(i2 , j2 )) = σ(i1 , j1 , i2 , j2 ). To conclude the proof of (12), notice that p p √ √ ∗ ∗ ∗ + n1 /N Z2,N ) ⇒ L( 1 − θZ1 + θZ2 ) = L(Z), L(ZN ) = L( n2 /N Z1,N where Z1 and Z2 are two independent copies of the Gaussian random matrix Z. To establish (13), notice that, for r ∈ {1, 2}, nr nr 1 X 1 X ∗ ∗ ∗ ∗ b ar,j (i1 )b ar,j (j1 )b ar,j (i2 )b ar,j (j2 ) = e a∗r,j (i1 )e a∗r,j (j1 )e a∗r,j (i2 )e a∗r,j (j2 ) + OP (nr−1/2 ) nr nr j=1

j=1

25

and that, for σ (1) (i1 , j1 , i2 , j2 ) = (1 − θ) E[< X1,k − µ1 , ϕi1 >< X1,k − µ1 , ϕj1 >< X1,k − µ1 , ϕi2 >< X1,k − µ1 , ϕj2 >] + θ E[< X2,k − µ2 , ϕi1 >< X2,k − µ2 , ϕj1 >< X2,k − µ2 , ϕi2 >< X2,k − µ2 , ϕj2 >], we have nr 1 X e a∗r,j (i1 )e a∗r,j (j1 )e a∗r,j (i2 )e a∗r,j (j2 ) − b c∗i1 b c∗j1 b c∗i2 b c∗j2 σ (1) (i1 , j1 , i2 , j2 ) → 0, nr

(15)

j=1

in probability, since as in obtaining (14), nr  1 X E e a∗r,j (i1 )e a∗r,j (j1 )e a∗r,j (i2 )e a∗r,j (j2 ) − b c∗i1 b c∗j1 b c∗i2 b c∗j2 σ (1) (i1 , j1 , i2 , j2 ) nr j=1 i h c∗j2 ϕ bj2 > c∗i2 ϕ bi2 >< ε∗r,k , b c∗j1 ϕ bj1 >< ε∗r,k , b = E < ε∗r,k , b c∗i1 ϕ bi1 >< ε∗r,k , b

−b c∗i1 b c∗j1 b c∗i2 b c∗j2 σ (1) (i1 , j1 , i2 , j2 ) → 0,

independence of the random

Pnr

a∗r,j (i1 )e a∗r,j (j1 )e a∗r,j (i2 )e a∗r,j (j2 )) j=1 e variables e a∗r,j (i1 )e a∗r,j (j1 )e a∗r,j (i2 )e a∗r,j (j2 ), for

in probability, and also V ar(n−1 r

= OP (n−1 r ), due to the different j’s. Furthermore,

bN ϕ by the triangular inequality and because | < C bi , ϕ bj > − < Cϕi , ϕj > | → 0 in probability, it yields that ∗ b∗ ϕ | −b c∗i b c∗j < Cϕi , ϕj > | → 0, r,nr bi , ϕ

(16)

in probability, since ∗ ∗ br,n bN ϕ br,n bN )ϕ bN (ϕ | −b c∗i b c∗j < C bi , ϕ bj > | ≤ | < (C −C b∗i , ϕ b∗j > | + | < (C b∗i − b c∗i ϕ bi ), ϕ b∗j > | r i r

bN ϕ + | < (C bi , ϕ b∗j − b c∗j ϕ bj > |   ∗ ∗ ∗ ∗ b∗ − C bN k + kϕ = OP kC b − b c ϕ b k + k ϕ b − b c ϕ b k → 0, i j r,nr i i j j by (9) and Lemma 6.1. Equations (15) and (16) imply then assertion (13). This completes the proof of the theorem.



Proof of Theorem 3.2: Notice that under Gaussianity of the random functions X1,j and X2,j , the random variables < X1,j − µ1 , ϕi > and < X2,j − µ2 , ϕi > are independent Gaussian distributed with mean zero and variance λi , i = 1, 2, . . . , p. From assertion (12) in the proof of Theorem 3.3, we get that in this case, the random variables ∆(i, j) are for 1 ≤ i ≤ j ≤ p independent with mean zero and

26

V ar(∆(i, j)) = 2λ2i if i = j and V ar(∆(i, j)) = λi λj if i 6= j. We then have that ∗(G)

p X ∆ b ∗2 (r, r) b ∗2 (r, m)  n1 n2 1  X ∆ + 2 b∗2 b∗ λ b∗ N 2 λ λ r r m r=1 1≤r