Independence Distribution-preserving Covariance Structures for the Likelihood Ratio Test for LX β = 0 in the General Linear Model

Research has been ongoing for over fifty years with respect to violating the assumption of i.i.d. observations in the error covariance matrix. However, there exist test statistics and dependency structures for which the sampling distribution of the test statistic is identical to the test statistic’s distribution under the assumption of i.i.d. observations. We derive an explicit representation of the general non-i.i.d. error covariance matrix of the general linear model error vector such that the likelihood ratio test statistic for testing certain linear restrictions on the parameter vector is robust against certain forms of dependency and heteroscedasticity. In doing so, we correct two proposed explicit covariance matrix characterizations given in Khatri (1981).


The Problem
Consider the general linear model where y is an n × 1 vector of observations, X is an n × p known fixed non-null model (design) matrix with rank (X) = r ≤ p, β is a p×1 vector of unknown model parameters, and ϵ is an n×1 vector of random perturbations such that ϵ ∼ N ( 0, σ 2 Σ ) , where Σ is symmetric nonnegative definite (s.n.d.) and σ 2 > 0. We assume that (1) is consistent, i.e., y ∈ C (Σ : X), where C (A) represents the column space of a matrix A.
The usual assumptions of the general linear model include the restriction Σ = I on the error covariance structure for (1).Ignoring this assumption can result in poor statistical references.However, for certain statistics, conditions on the error covariance matrix exist that allow particular types of perturbations of the usual i.i.d.covariance Var (y) = σ 2 I for sampled observations without affecting the characteristics of the sampling distribution of the statistic.That is, non-i.i.d.covariance structures for the data exist for which the sampling distribution of a statistic is identical to the sampling distribution under the assumption of i.i.d.observations.We refer to such covariance structures as independence distribution-preserving (IDP) covariance matrices.
The form of IDP dependency structures yields insight into model-misspecification and error-term dependence robustness for a statistic of interest.That is, one can see the form of covariance matrices for which the statistic of interest holds in addition to the often-assumed i.i.d.covariance structure.The existence of IDP covariance structures for a statistic implies that normally distributed observations need not be independent nor must the marginal variances be equal for the usual i.i.d.-induced properties of the statistic to hold.Hence, for certain statistics, some degree of robustness against dependent observations and heteroscedasticity with non-i.i.d.covariance matrices exists.Khatri (1981) has proposed one implicit and two explicit expressions for the set of s.n.d.error covariance structures such that the distributional properties of the likelihood ratio (LR) statistic under Σ = I for testing are preserved.Unfortunately, his two proposed explicit IDP covariance matrix characterizations are not general IDP covariance matrices.Here, we explicitly characterize the set of s.n.d.IDP dependency structures Σ ≥ IDP and symmetric positive definite (s.p.d).dependency structures Σ > IDP such that the distribution of the LR statistic for testing (2) is identical to the distribution of the LR statistic under the i.i.d.covariance structure Σ = I for all nonzero y ∈ C [Σ : X].

Background
Research on IDP covariance structures for univariate statistics has been ongoing for at least fifty years.In one of the first papers published in this area, Walsh (1947) studied the effect of equicorrelated data on certain two-group hypothesis test statistics.Other authors, such as Baldessari (1965), Baldessari and Gallo (1981), Stadje (1984), and Jensen (1989), have considered the derivation of the general s.p.d.IDP covariance structures for the univariate sample variance.In addition, Bhat (1962), Rogers (1980), Stadje (1984), and Young, Thompson, and Turner (2003) have derived the general s.p.d.IDP covariance structures for the univariate sample variance such that the sample variance is a multiple of a chi-squared random variable and is independent of the sample mean.
Recently, Young, Thompson, and Turner (2003) have characterized the general s.n.d.IDP covariance matrix for the multivariate sample variance and Young and Turner (2001) have characterized the general s.n.d.IDP joint covariance structure for the multivariate two-group problem.Also, Young et al. (2003) have characterized the s.n.d.covariance structures such that the sample covariance matrix is distributed as a central Wishart random matrix with n − 1 degrees of freedom and is independent of the sample mean vector.
In addition to Khatri (1981), research has been published on test statistics for the univariate dependent-variable regression case.For instance, Halperin (1951) has described an IDP covariance structure for a statistic used to test model adequacy, and Jeyaratnam (1982) has provided a sufficient IDP covariance structure for a statistic in testing linear hypotheses of the form (2) where L ∈ R s×n and rank (LX) = s, and X is the design matrix so that LXβ is a set of estimable functions.Tranquilli and Baldessari (1988) have devised an IDP structure for a test of model adequacy in multiple regression analysis, and Ghosh and Sinha (1980) have derived conditions on the design matrix X to yield the usual F-statistic for testing (2).
While these contributions are significant, we note that our primary focus is on the solutions obtained in Khatri (1981).We improve upon Khatri's (1981) results in three ways.First, we provide a correct general representation of the general IDP covariance structure.Second, our IDP characterization has no indeterminate matrices.In other words, we rigorously define all matrices in our general representation of the IDP covariance structure so that the structure is s.n.d.Third, our representation is not restricted such that (Σ − θI) is s.n.d., where θ > 0.

Outline
We have organized the remainder of the paper as follows.In Section 2, we define notation and give two LR statistics of interest for testing (2).In Section 3, we present Khatri's (1981) proposed explicit IDP general covariance structure characterization along with six lemmas needed for the derivation of our main results.We derive our IDP characterization of the general s.n.d.IDP dependency structure for the LR statistic for testing (2) in Section 4. Finally, we provide some brief concluding remarks in Section 5.

General Mathematical Notation
We use the following notation throughout the remainder of the paper.The symbol C m×n (R m×n ) represents the vector space of all m × n matrices over the complex (real) field C (R).The notation R S n represents the set of symmetric matrices in R n×n .The symbol denotes the cone of all Hermitian s.n.d.matrices in C n×n (R n×n ), and represents the interior cone composed of the set of Hermitian s.p.d.matrices in C n×n (R n×n ).The notation A * represents the conjugate transpose of A ∈ C n×m , and the symbol A ′ denotes the transpose of A ∈ R n×m .Also, B − ∈ R n×m represents a generalized inverse of B ∈ R m×n , and B + ∈ R n×m denotes the Moore-Penrose pseudoinverse of B ∈ R m×n .Additionally, C (A) and R (A) denote the column space and row space, respectively, of A, and C (A) ⊥ and R (A) ⊥ represent the orthogonal complement of C (A) and R (A), respectively.Also, P A denotes the orthogonal projection matrix onto C (A), R A denotes the orthogonal projection onto R (A), and P ⊥ A and R ⊥ A represent the orthogonal projection matrix onto C (A) ⊥ and R (A) ⊥ , respectively.

IDP Notation
We need the following notation and terms to specifically define the IDP problem we address.Let L ∈ R s×n such that rank (LX) = s, and let Assuming the Gauss-Markov model (1), Khatri (1981) has derived the LR statistic for testing the hypothesis in (2), which is where F Σ ∼ F s 1 ,n−r 1 if H 0 holds and Σ is known.If Σ = I, then because C (L ′ ) ⊂ C (X), s = s 1 , and r = r 1 , the test statistic (3) becomes where F I ∼ F s,n−r .

Mathematical Preliminaries
3.1 Khatri's (1981) IDP Solutions Khatri (1981) has implicitly characterized the set of s.n.d.IDP covariance structures Σ ∈ R ≥ n such that F Σ = F I as given θ > 0 is fixed.Khatri (1981) has also proposed two explicit characterizations for the set of IDP covariance structures such that F Σ = F I .Khatri's (1981) first IDP characterization, which he found by deriving the general solution for [Σ − θI] to the matrix equation ( 5), is where Khatri's (1981) IDP general covariance matrix (6) is not an explicit characterization of the set of IDP covariance structures such that F Σ = F I (see appendix).Khatri's (1981) second proposed general IDP covariance structure is purportedly a sufficient IDP covariance structure because it is restricted to the case where (Σ − θI) ∈ R ≥ n .This explicit general IDP covariance matrix is where θ > 0 and W ∈ R ≥ n is arbitrary.We remark that (7) has the unnecessary restriction that (Σ − θI) ∈ R ≥ n .Because (7) follows from (6), neither the covariance matrices given in ( 6) nor ( 7) represents an explicit characterization of the set of s.n.d.IDP covariance matrices for F Σ = F I .

Lemmas
We now state six lemmas that we use in the proof of our IDP covariance-matrix characterization result.The first lemma gives a representation of the general s.n.d.solution to the matrix equation AXA * = B. We remark that Groß (2000) gives another form of a general n.
n.d.solution to AXA * = B. Lemma 3.1.(Baksalary 1984, Theorem 3) Let A ∈ C m×n and B ∈ C ≥ m such that C (B) = C (A) .Then, a represntation of the general Hermitian n.d.solution to AXA* = B is X = [ A − D + ( I n − A − A ) Z ] [ A − D + ( I n − A − A ) Z ] * ,where D ∈ C m×n is an arbitrary but fixed matrix such that B = DD * , and Z ∈ C n×n is free to vary.