Longitudinal data analysis autoregressive linear mixed effects models

Longitudinal Data

Longitudinal data refers to repeated measurements or observations collected from multiple subjects over time, and when multiple response variables are involved, it is termed multivariate longitudinal data Analyzing longitudinal data is crucial for understanding changes in response variables over time, exploring differences in these changes based on various factors or covariates, and examining the relationships between changes in multiple response variables Given that measurements from the same subjects are correlated, it is essential to employ analytical methods that account for this correlation and the variance-covariance structure of the data.

I Funatogawa and T Funatogawa, Longitudinal Data Analysis, JSS Research Series in Statistics, https://doi.org/10.1007/978-981-10-0077-5_1

Longitudinal data analysis has evolved significantly, particularly through the development of linear mixed effects models, which account for random effects across subjects when analyzing continuous response variables Pioneering research in this field emerged in the 1980s, notably through Laird and Ware's influential paper in 1982 Since the 1990s, the publication of books and the creation of statistical software have further advanced the analysis of longitudinal data The choice of analytical methods varies based on study design; for instance, if blood pressure measurements are taken once from 100 subjects, t-tests or regression analysis are appropriate Conversely, if measurements are taken 100 times from a single subject, time series analysis is utilized, and for repeated measurements from multiple subjects, longitudinal data methods are employed.

Figure 1.1a illustrates hypothetical longitudinal data from a randomized controlled trial (RCT), where solid and dotted lines represent changes in a response variable for subjects in two randomly assigned groups: a new treatment and a placebo In RCTs, the distributions of the response variable at baseline should be comparable between groups, ensuring internal validity and comparability This randomization is crucial, particularly for balancing unknown confounders, a significant advantage of RCTs over observational studies, where adjusting for confounding factors is challenging, especially unknown ones This book focuses on RCTs and experimental studies, rather than observational research.

Figure1.1b shows an example of time series data Time series data usually have a large number of time points Typical longitudinal data have a large or moderate

Nikke i a ve ra ge c lo sing pr ic e ( 10,000 ye n)

Figure 1.1 illustrates hypothetical longitudinal data from a randomized controlled trial comparing two groups, with solid and dotted lines representing changes in a response variable for each subject Additionally, the figure includes time series data of the Nikkei average closing price.

Longitudinal data can vary significantly in structure, featuring either a limited number of subjects with few time points or a larger sample size with extensive time points This flexibility allows for diverse analyses, accommodating both scenarios where there are many subjects and many time intervals, or fewer subjects with more frequent measurements.

Since the 1990s, there has been a significant increase in publications focused on longitudinal data analysis, including key works such as Diggle et al (1994, 1st edn and 2002, 2nd edn.), Dwyer et al (1992), and Fitzmaurice et al (2004, 1st edn and 2011, 2nd edn.).

This book primarily focuses on autoregressive linear mixed effects models, a topic that is seldom covered in other literature It references significant works from various authors, including Gregoire et al (1997), Hand and Crowder (1996), and Jones (1993), among others, highlighting its unique contribution to the field The book builds on foundational texts such as Littell et al (1996, 2006) and includes insights from recent studies by Tango (2017) and Vonesh (2012), making it a comprehensive resource for understanding these advanced statistical methods.

This article explores linear mixed effects models and marginal models in Section 1.2, followed by specific examples in Section 1.3 Section 1.4 outlines the commonly used mean structures and variance-covariance structures, while Section 1.5 focuses on inference based on maximum likelihood methods.

Linear Mixed Effects Models

Linear mixed effects models are used for the analysis of longitudinal data of continuous response variables LetY i

T be the vector of the response corresponding to theith(i1,ã ã ã,N)subject measured from 1 ton ioccasions.Y ij is thejth measurement.A T denotes the transpose ofA Linear mixed effects models are expressed by

The model can be expressed as Y_i = X_iβ + Z_i b_i + ε_i, where β represents a p×1 vector of unknown fixed effects parameters In this equation, X_i denotes a known n_i×p design matrix for fixed effects, while b_i is a q×1 vector of unknown random effects parameters Additionally, Z_i is a known n_i×q design matrix for random effects, and ε_i is an i×1 vector of random errors, comprising ε_i1, ε_i2, , ε_in_i.

It is assumed thatb i andε i are both independent across subjects and independently follow a multivariate normal distribution with the mean zero vector,0, and variance covariance matricesGand

The distributions are defined as \( b_i \sim MVN(0, G) \) and \( \epsilon_i \sim MVN(0, R_i) \), where \( G \) is a \( q \times q \) square matrix and \( R_i \) is an \( n_i \times n_i \) square matrix In these matrices, the diagonal elements represent variances, while the non-diagonal elements indicate covariances Both matrices contain unknown parameters and are presumed to follow specific structures Additionally, it is assumed that responses from different subjects are independent.

4 1 Longitudinal Data and Linear Mixed Effects Models

Following the above assumptions, the marginal distribution ofY i is a multivariate normal distribution The mean vector is the marginal expectation E( Y i ) X i β, and the variance covariance matrixV i Var( Y i )is

V i Var( Z i b i +ε i ) Z i GZ T i +R i (1.2.4) The distribution is expressed by

Y i ∼MVN( X i β , V i ), (1.2.5) where V i is ann i×n i square matrix The E( Y i )and the expectation for a typical subject withb i 0, E( Y i |b i 0 ), are the same.

Recent advancements in linear models now permit correlations and unequal variances among responses These models have been enhanced by incorporating random effects and varying structures of the variance-covariance matrix Traditionally, linear models operated under the assumption of an independent structure with equal variances in the variance-covariance matrix.

The term "mixed effects" refers to models that incorporate both fixed and random effects Additionally, linear mixed effects models can also encompass structures that involve specific configurations of R i without random effects, represented as V i R i instead of V i Z i GZ T i + R i.

Marginal models are defined by the equation Y_i = X_iβ + ε_i, where ε_i follows a multivariate normal distribution with mean 0 and variance R_i In these models, we focus on the marginal expectation E(Y_i), the marginal variance, and the correlation of the response variable The mean structure represents the first moment, while the variance-covariance structure represents the second moment of a multivariate normal distribution Linear mixed effects models can be adapted to marginal models, but for discrete responses like binary or count data, higher moments are necessary to define the likelihood, leading to the use of generalized estimating equations (GEE) in marginal models Additionally, the interpretation of fixed effects parameters β varies between mixed effects models and marginal models.

Examples of Linear Mixed Effects Models

Means at Each Time Point with Random Intercept

In clinical trials and experimental studies, it is common for observation time points to be standardized across all subjects, although the intervals between these observations do not need to be uniform When the observation time points are consistent, a model can be applied that includes means for each time point along with a random intercept.

In this study, μj represents the mean at the jth time point, where j ranges from 1 to J Each subject is associated with a random intercept, bi, which is assumed to follow a normal distribution alongside a random error, εij Both bi and εij are independently distributed with a mean of zero, having variances of σG² and σε², respectively.

In the case of four time points, the model for the response,Y i , and the variance covariance matrix of the response vector,V i Var( Y i ), are

Figure 1.2 presents data from a schizophrenia trial, highlighting selected subjects It illustrates the means at each time point using a random intercept, where the estimated means are represented by a thick line with closed circles, while the predicted values for each subject are depicted by thin lines.

In this context, \( I_a \) represents an \( a \times a \) identity matrix The variance-covariance structure \( Z_i G Z_i^T \), which arises from a random intercept, forms a square matrix where all elements are equal to \( \sigma_G^2 \) The total variance-covariance matrix \( V_i = \text{Var}(Y_i) \) incorporates the random error vector's variance-covariance matrix \( \sigma_\epsilon^2 I_{n_i} \) Consequently, the diagonal elements of \( V_i \) are \( \sigma_G^2 + \sigma_\epsilon^2 \), while the non-diagonal elements equal \( \sigma_G^2 \) Here, \( \sigma_G^2 \) is referred to as between-subject, inter-subject, or inter-individual variance, whereas \( \sigma_\epsilon^2 \) is known as within-subject, intra-subject, or intra-individual variance.

Figure 1.2a presents longitudinal data from selected subjects in the schizophrenia trial by Diggle et al (2002), which will be analyzed in Chapter 3 In Figure 1.2b, an example utilizing model (1.3.1) is illustrated, where the thick line with closed circles represents the estimated means at each time point (μˆ j) The thin lines depict the predicted values (ˆ μ j + bˆ i) for each subject, demonstrating parallel individual lines due to the random intercept, which indicates the assumption of mutual parallelism among subjects over time.

The above model,Y ij μ j +b i +ε ij , is also expressed in another design matrix withJ−1 dummy variables,x 1j ,ã ã ã,x J − 1j Letx kj 1 ifkj−1 and 0 otherwise. The model is

In the case of four time points, the model is

1.3 Examples of Linear Mixed Effects Models 7

Balanced data refers to datasets that have consistent observation time points and no missing values, while unbalanced data consists of datasets with varying observation time points or missing values Despite the presence of missing values, it is possible to analyze the entire dataset without excluding subjects with incomplete data A detailed discussion on handling missing data can be found in Section 3.2.

Group Comparison Based on Means at Each Time

In this study, we analyze group comparisons by examining the means at each time point using a linear mixed effects model that includes a random intercept We define two groups, A and B, where an indicator variable, x gi, takes the value of 0 for group A and 1 for group B The model incorporates the main effects of both time and group.

In the case of four time points, this model is

The design matrices of the fixed effects for groups A and B are

The parameter β g highlights the differences between the two groups, with the model presuming that these differences remain consistent over time However, this assumption may not hold true in randomized controlled trials (RCTs), as illustrated in Sect 1.1 and Fig 1.1a While baseline distributions at j1 are anticipated to be similar across groups, variations in distributions are expected for j > 1.

When the interaction between time and the group is added because the differences between groups are not constant over time, the model becomes

In the case of four time points,X i for groups A and B andβin the model are

In this model, time courses are not assumed to be parallel between the groups. The expected values at the last time point for groups A and B are

The expected difference at the last time point between groups A and B is

To estimate the difference, we use the following contrast vector,L,

In Sect.1.5.5, we discuss the estimation and test using contrasts.

Because the distributions of baseline atj1 are expected to be the same, we can omitβ g0from (1.3.4) in RCTs The model is

In the case of four time points,X i for groups A and B andβare

The difference at the last time point between groups A and B isβ g3.

Means at Each Time Point with Unstructured Variance

Accurate model specification is crucial when dealing with missing data, as it involves both the mean structure and the variance-covariance structure Previous sections have discussed the use of means at each time point with a random intercept; however, this approach assumes constant variance and covariance, which can be overly restrictive Recently, models employing an unstructured (UN) variance-covariance approach have gained popularity, as they impose no constraints on the parameters related to mean structure and variance-covariance components.

In this study, the mean at the jth time point, denoted as μ j, is analyzed alongside the UN for R i, referred to as R UN i It is assumed that the random error vector, ε i, adheres to a multivariate normal distribution characterized by a mean of zero and the corresponding UN, R UN i.

In the case of four time points, the model for the response,Y i , and the variance covariance matrix of the response vector,V i Var( Y i ), are

The UN is not parsimonious, and the number of parameters increases largely when the number of time points is large.

For the two group comparison, the models corresponding to (1.3.4) and (1.3.7) are

Y ij β 0+β 1 x 1j +ã ã ã+β J − 1 x J − 1j + β g0+β g1 x 1j +ã ã ã+β gJ − 1 x J − 1j x gi +ε ij ε i ∼MVN( 0 , R UN i ) ,

Y ij β 0+β 1 x 1j +ã ã ã+β J −1 x J −1j + β g1 x 1j +ã ã ã+β gJ −1 x J −1j x gi +ε ij ε i ∼MVN( 0 , R UN i )

We can assume different UN for each group, but the number of parameters is doubled.

Linear Time Trend Models with Random Intercept

Linear time trend models are commonly utilized for both the mean structure and random effects, under the assumption that changes occur at a constant rate over time In this context, let Y_ij represent the responses for the jth measurement of the ith subject, where i ranges from 1 to N, and t_ij denotes time as a continuous variable.

In the context of mixed-effects models, the random effects for the intercept and slope of the ith subject, denoted as b0i and b1i, are referred to as random intercepts and random slopes, respectively These random effects, represented as a bivariate vector bi (b0i, b1i) T, are assumed to follow a bivariate normal distribution characterized by a mean vector of zero, variances σG0² and σG1², and covariance σG01 It is also possible to assume that the random intercept and slope are independent, indicated by σG01 equaling zero Additionally, the variance of the random error term εij is denoted as σε², with the assumption that these random errors are mutually independent and normally distributed.

In the case of four time points, the model and the variance covariance matrix of the response vector are

The diagonal element at thejth time inV i , that represents the variance, is σ G0 2 + 2σ G01 t ij +σ G1 2 t ij 2 +σ ε 2

The non-diagonal element at thej,kth time inV i , that represents the covariance, is σ G0 2 +σ G01 t ij +t ik

Based on these equations, when covariate values of random effects are different across time points, t ij t ik for j k, the variance is also different across time points, Var

Var(Y ik ), unless σ G1 2 0 and σ G01 0 The covariance also depends on the time.

When the time points \( t_{ij} \) are consistent across subjects, the variance-covariance structure remains uniform and is represented within the UN model Conversely, when the time points \( t_{ij} \) vary among subjects, the variance-covariance structure diverges, leading to its exclusion from the UN model.

The linear time trend model illustrated in Figure 1.3a features a random intercept with a fixed slope, indicating that while intercepts vary among subjects, the slopes remain constant Conversely, Figure 1.3b presents a model with both random intercepts and slopes, demonstrating that response variance can increase over time, although it may also decrease within certain observation periods.

The growth curve model illustrates a linear time trend, with variations including quadratic and higher-order equations that also qualify as growth curve models While these curves depict nonlinear changes over time, they maintain linearity in their parameters Conversely, nonlinear models like Gompertz and logistic curves exhibit both nonlinear changes over time and nonlinearity in their parameters.

We discuss nonlinear growth curves further in Chap.5 Furthermore, growth curves for child development, such as height, weight, and body mass index (BMI), are estimated by other methods.

The linear time trend model features a random intercept, as illustrated in Figure 1.3a, while Figure 1.3b presents the model with both a random intercept and a random slope The estimated means are represented by the thick line with closed circles, and the predicted values for each subject are depicted by the thin lines.

Group Comparison Based on Linear Time Trend Models

Models with Random Intercept and Random Slope

This section presents group comparisons utilizing the linear time trend model, incorporating time as a continuous variable, group as a qualitative variable, and their interaction In the case of two groups, A and B, an indicator variable, x gi, is defined, where x gi is 0 for group A and 1 for group B The analysis employs a linear mixed effects model that includes this interaction to assess the differences between the groups over time.

In the analysis, group A is characterized by an intercept of β 0 and a slope of β 1, while group B has an intercept of β 0 + β g0 and a slope of β 1 + β g1 The coefficient β g1 represents the interaction term between time and the group, highlighting the differences in slopes between the two groups When considering four time points, the model can be expressed accordingly.

The design matrices for the fixed effects of groups A and B are

In this study, the variance covariance parameters, σ G0 2, σ G1 2, σ G01, and σ ε 2, are assumed to be consistent across groups, although they can also vary The slope serves as a key summary measure for group comparisons, with extensive research conducted since the late 1980s on methods to estimate and test differences in slopes, β g1, particularly in the context of missing data and dropouts Conversely, the asymptote or equilibrium in autoregressive linear mixed effects models offers valuable and interpretable summary measures.

(2008) studied the estimation of asymptotes focusing on missing data.

Mean Structures and Variance Covariance Structures

Mean Structures

In the model, let μ_ij represent the mean for the ith subject at the jth time point, with the mean structure defined as μ_j As the number of time points increases, the number of parameters also rises For instance, in the linear time trend model, the mean is expressed as μ_ij = β_0 + β_1 t_ij, which includes two parameters: the intercept and the slope Additionally, a quadratic time trend can be represented as μ_ij = β_0 + β_1 t_ij + β_2 t_ij², incorporating an extra parameter for the quadratic term.

A quadratic equation can exhibit either a maximum or minimum value, calculated as β 0 − β 1² /(2β 2), occurring at the time t ij − β 1 / 2β 2 Beyond this point, the response transitions from increasing to decreasing or vice versa, indicating non-monotonic changes However, this assumption may not hold true, leading to poor fits for subsequent data points To enhance accuracy, not only linear and quadratic equations but also higher-order polynomials can be employed The l-th order polynomial is represented as μ ij = β 0 + β 1 t ij + β 2 t ij² + + β l t ij l, allowing for more flexible modeling of time-related data.

A piecewise linear function has linear trends and slope changes at some breakpoints.

In regression analysis, applying a log transformation to response or explanatory variables can enhance the fit of linear models, particularly when the response variable exhibits a log-normal distribution This technique is commonly utilized in pharmacokinetics, where drug concentrations typically follow right heavy-tailed distributions Additionally, log transformation is effective in stabilizing variance when it is dependent on the mean.

The Box–Cox transformation is y (λ) y λ −1 λ , (λ0), y (λ) logy, (λ0).

Here, the following formula holds: λ→0lim y λ −1 λ logy.

In certain situations, alternative frameworks to linear mixed effects models may be more appropriate This article will explore autoregressive linear mixed effects models in upcoming chapters, while Chapter 5 will focus on nonlinear mixed effects models Additionally, nonparametric regression analysis and smoothing techniques are also utilized in the analysis.

Variance Covariance Structures

This section provides an in-depth analysis of variance covariance structures, illustrated in Table 1.1, which presents various structures across four time points The variance covariance matrix of the response vector \( Y_i \) is defined as \( V_i = Z_i G Z_i^T + R_i \), where \( Z_i G Z_i^T \) represents the components induced by random effects, and \( R_i \) accounts for random errors Additionally, variance covariance structures influenced by a random intercept are detailed in Section 1.3.1, while the unstructured (UN) approach is discussed in Section 1.3.3.

1.4 Mean Structures and Variance Covariance Structures 15 intercept and a random slope are shown in Sect.1.3.4 The variance covariance structure induced by a random intercept and a random slope with t 1 t 2 t 3 t 4

0 1 2 3 is given in Table1.1o Structures popularly used are independent equal variances,

UN, compound symmetry (CS), and first-order autoregressive (AR(1)).

The independent structure with equal variances is utilized alongside random effects to address the correlation of data measured from the same subject This approach, known as conditional independence given random effects, ensures that the inherent correlations within a subject are accounted for Additionally, the independent structure with unequal variances is also employed in this context.

The UN structure in statistical modeling imposes no restrictions on variance or covariance parameters, leading to inefficiencies due to its large number of parameters, particularly as the number of time points increases Specifically, when the number of time points is n_i, the parameters amount to (n_i(n_i + 1))/2, and this increases by n_i + 1 with each additional time point In contrast, the CS (compound symmetry) structure, characterized by two parameters—variance (σ²) and covariance (σ₁)—maintains consistency across time points The correlation (ρ), defined as σ₁/σ², is known as the intra-class correlation coefficient The CS structure also encompasses a random intercept and independent random errors, with diagonal elements representing the sum of between-subject and within-subject variance, while non-diagonal elements reflect only the between-subject variance Although the CS structure constrains covariance and correlation to be positive, it can exhibit negative values, making it a narrower framework Additionally, the heterogeneous CS (CSH) structure introduces a CS correlation structure with varying variances across time points.

The AR(1) structure exhibits a typical pattern of serial correlation, where the correlation diminishes with increasing time intervals In contrast, the heterogeneous AR(1) (ARH(1)) structure incorporates AR(1) correlation while allowing for varying variances across different time points The AR(1) structure for R_i and R_ARi can be implemented in three distinct ways: using random errors alone as V_i R_ARi, incorporating random effects as V_i Z_i GZ_T_i + R_ARi, or combining random effects with independent errors as V_i Z_i GZ_T_i + R_ARi + σ² I_n_i Various methodologies exist to simultaneously address random effects, serial correlations, and independent errors (Diggle 1988; Heitjan 1991; Jones 1993; Funatogawa et al 2007; Funatogawa et al 2008).

(1993) used serial correlations for continuous time.

The Toeplitz structure features consistent variances across time points and uniform covariances for equal time intervals In contrast, the heterogeneous Toeplitz structure maintains a Toeplitz correlation while exhibiting varying variances over time Specifically, the j,kth element is defined as σ j σ k ρ | j − k | Additionally, the two-band Toeplitz structure imposes a constraint where covariances are zero when the time difference exceeds two points This similar approach is also applied in autoregressive linear mixed effects models, as detailed in Section 2.4.1 and Table 2.3a, to address measurement errors effectively.

Kenward (1987) used the first-order ante-dependence (ANTE(1)) Thej,kth element is

Table 1.1 Variance covariance structures for four time points

(e) Random intercept and independent equal variances, inter- and intra-variances

(f) Compound symmetry: CS, variance and covariance

1.4 Mean Structures and Variance Covariance Structures 17

(n) First-order ante-dependence: ANTE(1) a

(o) Random intercept and random slope with t 1 t 2 t 3 t 4

⎠ a Lower triangular elements are omitted σ j σ k k − 1 l j ρ l

In statistical modeling, when the number of time points is denoted as \( n_i \), the total number of parameters is calculated as \( 2n_i - 1 \) The ANTE(1) model is characterized by non-stationarity, meaning that variances fluctuate over time and correlations vary with specific time points Conversely, the AR(1), CS, and Toeplitz models are considered stationary, as they maintain constant variances across time and their correlations rely solely on the time distance between observations.

Variance covariance structures using a stochastic process are applied In the Orn- stein–Uhlenbeck process (OU process), the variance and covariance betweenY(s) andY(t)are

This continuous time process corresponds to AR(1) for discrete time with ρ e −α The following process that integrates the OU process is called the integrated Ornstein–Uhlenbeck process (IOU process),

The variance ofW(t)and the covariance betweenW(s)andW(t)are

Taylor et al (1994) and Taylor and Law (1998) used the IOU process and Sy et al.

Variance-covariance matrices can vary across different levels of a factor, such as groups, which can double the number of parameters when comparing two groups This assumption significantly impacts the outcomes of statistical tests and estimations The implications of unequal variances in analysis of covariance are explored in the works of Funatogawa et al (2011) and Funatogawa and Funatogawa (2011).

Inference

Maximum Likelihood Method

Maximum likelihood (ML) methods are commonly employed for estimating linear mixed effects models To simplify the explanation, this section focuses on the likelihood for independent data In this context, let Y represent a variable that follows a normal distribution characterized by a mean (μ) and variance (σ²) The probability density function of Y is denoted as f(Y).

The probability density function is a function of a random variable Y given the parameters μ, σ 2

It shows what value ofY tends to occur Now, letY 1 ,ã ã ã,Y N be

N random variables that follow independently an identical normal distribution with the meanμand varianceσ 2 Then, the probability density function ofY 1 ,ã ã ã,Y N is f(Y 1 ,ã ã ã,Y N )

The likelihood function is mathematically equivalent to the probability density function, differing only in that it is expressed as a function of the parameters μ and σ² based on the observed data This relationship arises from the independent variables, resulting in a straightforward multiplication of the probability density function.

Y 1 ,ã ã ã,Y N The ML method maximizes the log-likelihood (ll) for the estimation of unknown parameters The log-likelihood is ll ML −N

Minus two times log-likelihood (−2ll) is used for the calculation,

The ML estimator (MLE) ofμis the arithmetic mean,Y¯ N i 1 Y i /N The MLE of σ 2 is N i 1

The estimator of σ² is biased as it fails to account for the reduction of one degree of freedom when estimating the mean parameter An unbiased estimator for σ² is given by the formula N - 1.

/(N−1) When N is infinite, both converge to the same value In this simple example, the MLE ofσ 2 is biased but consistent.

Longitudinal data analysis involves the examination of data collected from various subjects, where it is typically assumed that the data from different subjects are independent, while data from the same subject are not In linear mixed effects models, the marginal distribution of the response variable \(Y_i\) follows a multivariate normal distribution characterized by a mean of \(X_i \beta\) and a variance-covariance matrix represented as \(V_i Z_i G Z_i^T + R_i\) Consequently, the probability density function for the observed data \(Y_1, \ldots, Y_N\) can be expressed as \(f(Y_1, \ldots, Y_N)\).

(1.5.5) where|V i |is the determinant ofV i The marginal log-likelihood function,ll ML, and

−2ll ML are ll ML −

Maximum Likelihood Estimates (MLEs) are derived by maximizing the log-likelihood function or minimizing its negative counterpart concerning unknown parameters When the variance-covariance parameters are established, the MLEs for the fixed effects parameters, denoted as β, can be obtained by minimizing the negative log-likelihood function.

To estimate the variance-covariance parameters, which are typically unknown, we apply the generalized inverse denoted by X T i V −1 i Y i The maximum likelihood (ML) estimates are obtained by concentrating out β from the likelihood function, substituting the equation for βˆ into the concentrated −2 log-likelihood This leads us to minimize the resulting concentrated −2 log-likelihood to derive the estimates.

In the context of statistical modeling, the equation (1.5.9) represents the relationship between the response variable \( r_i \), the predictors \( Y_i \), and the fixed effects parameters \( \beta \) To streamline the analysis, we can minimize the number of unknown parameters, denoted as \( p \), which corresponds to the fixed effects parameters Typically, variance-covariance parameters are not directly computed; instead, iterative methods such as the Newton-Raphson method, the expectation-maximization (EM) algorithm, or Fisher’s scoring algorithm are employed Ultimately, the maximum likelihood estimates (MLEs) of the fixed effects parameters are represented as \( \hat{\beta} \).

X T i Vˆ − i 1 Y i , (1.5.10) whereV i in Eq (1.5.8) is replaced with the ML estimatesVˆ i

The variance \( \sigma^2 V_i \) can be expressed as \( \sigma^2 V_{ci} \), allowing \( \sigma^2 \) to be factored out of the likelihood The closed form of \( \sigma^2 \) is determined by \( \beta \) and \( V_{ci} \), which is then incorporated into the negative log-likelihood function In this context, \( V^{-1}_i \) is replaced by \( V^{-1}_{ci} \), and the estimate \( \hat{\beta} \) is also included in the negative log-likelihood calculation.

−2ll ML CONCisV −1 ci This reduces further the number of optimization parameters by one The MLEs of the fixed effects parameters are βˆ

An example in the state space form is given in Sect.6.5.2.

The maximum likelihood (ML) estimates of variance covariance components are biased as they do not account for the reduction in degrees of freedom caused by estimating fixed effects parameters To address this issue, the restricted maximum likelihood (REML) method is employed, which utilizes a log-likelihood function specifically designed for REML analysis.

The REML (Residual Maximum Likelihood) method estimates variance covariance parameters through the residual contrast, which is a linear combination of observations that does not rely on fixed effects This method is detailed in Section 1.6 with vector representations However, it is important to note that the REML method is not suitable for comparing the goodness of fit between two models with differing fixed effects, as their log-likelihoods are based on distinct residual contrasts.

Variances of Estimates of Fixed Effects

When variance covariance parameters are known, the variance covariance matrix of the ML estimates of fixed effects vector is

The variance and covariance are replaced by the ML estimatesVˆ i ,

The variance-covariance matrix in a linear mixed effects model relies on the accuracy of the mean structure, variance-covariance structure, and distribution assumptions However, the standard errors of the estimated coefficients are likely underestimated, as this approach fails to account for the uncertainty in estimating variance and covariance.

Even if V i Var( Y i )is wrongly specified, the following sandwich estimator provides a consistent estimator of Var(β):ˆ

It is also called robust variance.

Prediction

The joint distribution ofY i andb i is the following multivariate normal distribution:

From this distribution, the conditional expectation ofb i givenY i is

ReplacingV i ,G, andβby the ML estimates,Vˆ i ,G, andˆ βˆ, the predictors of random effects are bˆ i ˆGZ T i Vˆ i − 1 (Y i −X i βˆ) (1.5.19)

When the variance covariance parameters are known, the estimator βˆ in Eq (1.5.8) serves as the best linear unbiased estimator (BLUE), while E( b i |Y i ) functions as the best linear unbiased predictor (BLUP) It is important to note that, since b i is a random vector, it is referred to as a predictor rather than an estimator The designation of "best" signifies that it has the minimum error variance among linear unbiased estimators or predictors However, when the variance covariance parameters are unknown, they must be substituted accordingly.

The estimates βˆ in Eq (1.5.10) are referred to as empirical BLUE (EBLUE), while bˆ i is known as empirical BLUP (EBLUP), both derived from empirical Bayes methods The predicted response profile Yˆ i for the ith subject is calculated accordingly.

The weighted mean of the population mean \(X_i \beta\) and the observed response \(Y_i\) is referred to as shrinkage, as it causes predicted values to converge towards the population mean The degree of shrinkage is influenced by the ratio of intra-subject variance \(R_i\) to the total variance \(V_i Z_i GZ^T_i + R_i\) When intra-subject variance is larger than inter-subject variance, the weight on \(X_i \beta\) increases, while a larger inter-subject variance leads to a greater emphasis on \(Y_i\) Additionally, a higher number of observations \(n_i\) for the ith subject results in reduced shrinkage.

The predicted values of the random effects for the ith subject, denoted as bˆ i, are derived from a weighted average of the REML estimates of the fixed effects parameter βˆ and the OLS estimates βˆ OLSi based solely on the ith subject's data When the conditions X i Z i and R i σ 2 I n i are met, the estimator for β i can be expressed as βˆ i = β + b i.

The greater inter-subject variance relative to intra-subject variance leads to βˆ i estimates that are more aligned with βˆ OLSi Conversely, a reduced inter-subject variance causes bˆ i estimates to approach zero.

Goodness of Fit for Models

Several indicators assess the goodness of fit in statistical models, notably Akaike’s Information Criterion (AIC) and Schwartz’s Bayesian Information Criterion (BIC) Both AIC and BIC impose penalties for increasing the number of parameters, helping to prevent overfitting Additionally, the maximum log-likelihood values for Maximum Likelihood (ML) and Restricted Maximum Likelihood (REML) are denoted as ll MLmax and ll REMLmax, respectively.

K N i 1 n i be the number of data,pbe the number of parameters for fixed effects, andqbe the number of parameters for random effects For ML and REML, AIC and BIC are

AICML −2ll MLmax+ 2(p+q), (1.5.23) AICREML −2ll REMLmax+ 2q, (1.5.24) BICML −2ll MLmax+(p+q)logK, (1.5.25) BICREML −2ll REMLmax+qlog(K−p) (1.5.26)

The REML method cannot be used to compare two models with different fixed effects as described in Sect.1.5.1.

Estimation and Test Using Contrast

LetLbe a 1×qcontrast vector and consider the estimation ofLβ.Lβis assumed to be estimable such that Lβ kE( Y ) kXβfor some vector of constants, k, whereY

The estimator isLβ, and theˆ two-sided 95% confidence interval is

In statistical analysis, the test statistic follows a t-distribution with ν degrees of freedom, where ν(0.975) represents the upper 97.5th percentile of the t-distribution Utilizing the contrast vector L, one can conduct a test against the null hypothesis of Lβ0.

Estimating the degree of freedom, ν, often requires approximation methods Notable techniques include the Satterthwaite approximation, established in 1946, and the Kenward–Roger method introduced in 1997, which employs an adjusted estimator of the variance-covariance matrix to mitigate small sample bias.

When there are multiple contrasts using ak×p(p≥k)full rank matrixL, anF test with the null hypothesis ofLβ0can be performed The following test statistic approximately follows anF distribution: βˆ T L T

The numerator degree of freedom in statistical analysis is represented by the rank of matrix L, denoted as rank(L) Conversely, the denominator degree of freedom typically requires estimation through an approximation method Additionally, when k equals 1, the F test statistic can be expressed as the square of the t test statistic.

Vector Representation

In the previous sections, linear mixed effects models are shown using the vector Y i for each subject This section shows the representation using the vector Y

,Zdiag( Z i ) The linear mixed effects models shown in Sect.1.2 are expressed by

The variance covariance matrices,VVar( Y ),G AVar( b ), andRVar( ε ), are

−2ll ML,−2ll REML,β, Var(ˆ β), andˆ bˆshown in Sect.1.5are expressed by

(1.6.10) βˆandbˆ i as shown in Sect.1.5can be also derived from the following mixed model equation:

In Section 1.5.1, we discuss residual maximum likelihood (REML), focusing on a full rank matrix K of dimensions N_i × (N_i - p) that meets the condition K^T X = 0 The term K^T Y represents the residual contrast, which follows a multivariate normal distribution with a mean vector of zero and a variance-covariance matrix of K^T V K, independent of β The log-likelihood for K^T Y is denoted as ll REML.

Diggle PJ (1988) An approach to the analysis of repeated measurements Biometrics 44:959–971 Diggle PJ, Heagerty P, Liang KY, Zeger SL (2002) Analysis of longitudinal data, 2nd edn Oxford University Press

Diggle PJ, Liang KY, Zeger SL (1994) Analysis of longitudinal data Oxford University PressDwyer JH, Feinleib M, Lippert P, Hoffmeister H (eds) (1992) Statistical models for longitudinal studies of health Oxford University Press

Fitzmaurice GM, Davidian M, Verbeke G, Molenberghs G (eds) (2009) Longitudinal data analysis. Chapman & Hall/CRC Press

Fitzmaurice GM, Laird NM, Ware JH (2004) Applied longitudinal analysis Wiley

In their 2011 work, Fitzmaurice et al presented a comprehensive guide to applied longitudinal analysis, emphasizing its significance in statistical research Funatogawa and Funatogawa (2011) explored the intricacies of analysis of covariance in randomized trials, specifically comparing scenarios with equal and unequal slopes Additionally, in a 2007 study, Funatogawa, Funatogawa, and Ohashi developed an autoregressive linear mixed effects model tailored for analyzing longitudinal data that exhibit profiles approaching asymptotes, contributing valuable insights to the field of statistical medicine.

Funatogawa T, Funatogawa I, Shyr Y (2011) Analysis of covariance with pre-treatment measurements in randomized trials under the cases that covariances and post-treatment variances differ between groups Biometrical J 53:512–524

Funatogawa T, Funatogawa I, Takeuchi M (2008) An autoregressive linear mixed effects model for the analysis of longitudinal data which include dropouts and show profiles approaching asymptotes Stat Med 27:6351–6366

Gregoire TG, Brillinger DR, Diggle PJ, Russek-Cohen E, Warren WG, Wolfinger RD (eds) (1997) Modelling longitudinal and spatially correlated data Springer-Verlag

Hand D, Crowder M (1996) Practical longitudinal data analysis Chapman & Hall

Heitjan DF (1991) Nonlinear modeling of serial immunologic data: a case study J Am Stat Assoc 86:891–898

Jones RH (1993) Longitudinal data with serial correlation: a state-space approach Chapman & Hall Kenward MG (1987) A method for comparing profiles of repeated measurements Appl Stat 36:296–308

Kenward MG, Roger JH (1997) Small sample inference for fixed effects from restricted maximum likelihood Biometrics 53:983–997

Laird NM (2004) Analysis of longitudinal & cluster-correlated data IMS

Laird NM, Ware JH (1982) Random-effects models for longitudinal data Biometrics 38:963–974 Littell RC, Miliken GA, Stroup WW, Wolfinger RD (1996) SAS system for mixed models SAS Institute Inc

Littell RC, Miliken GA, Stroup WW, Wolfinger RD, Schabenberger O (2006) SAS for mixed models, 2nd edn SAS Institute Inc

Satterthwaite FE (1946) An approximate distribution of estimates of variance components Biomet- rics Bull 2:110–114

Sy JP, Taylor JMG, Cumberland WG (1997) A stochastic model for analysis of bivariate longitudinal AIDS data Biometrics 53:542–555

Tango T (2017) Repeated measures design with generalized linear mixed models for randomized controlled trials CRC Press

Taylor JMG, Cumberland WG, Sy JP (1994) A stochastic model for analysis of longitudinal AIDS data J Am Stat Assoc 89:727–736

Taylor JMG, Law N (1998) Does the covariance structure matter in longitudinal modeling for the prediction of future CD4 counts? Stat Med 17:2381–2394

Verbeke G, Molenberghs G (eds) (1997) Linear mixed models in practice—a SAS oriented approach. Springer-Verlag

Verbeke G, Molenberghs G (2000) Linear mixed models for longitudinal data Springer-Verlag Vonesh EF (2012) Generalized linear and nonlinear models for correlated data Theory and appli- cations using SAS SAS Institute Inc

Wu H, Zhang J-T (2006) Nonparametric regression methods for longitudinal data analysis Mixed- effects modeling approaches Wiley

Zimmerman DL, Nỳủez-Antún VA (2010) Antedependence models for longitudinal data CRCPress

This chapter delves into autoregressive linear mixed effects models, expanding on previous discussions of linear mixed effects models by regressing current responses on past responses, fixed effects, and random effects These models exhibit two key characteristics: approaching asymptotes and state-dependence, which can be modeled through fixed and random effects The current response is influenced by both present covariates and historical covariate data Three vector representations are introduced: an autoregressive form, a response with asymptotic changes, and a marginal form that does not rely on prior responses The marginal interpretation aligns with subject-specific interpretations found in linear mixed effects models Additionally, the chapter presents variance-covariance structures for AR(1) errors, measurement errors, and random effects at baseline and asymptote, alongside likelihood estimations for both marginal and autoregressive forms Notably, the marginal form accommodates intermittent missing values, and the chapter distinguishes between autoregressive models focusing on the response itself and those utilizing autoregressive error terms.

Autoregressive linear mixed effects modelãLongitudinalãState-dependence

Autoregressive Models of Response Itself

Introduction

Three primary approaches for modeling longitudinal data include mixed effects models, marginal models, and transition models (Diggle et al 2002; Fitzmaurice et al 2011) This article focuses on linear mixed effects models and marginal models with linear mean structures, as outlined in the framework of linear mixed effects models, published under exclusive license to Springer Nature Singapore Pte Ltd in 2018.

I Funatogawa and T Funatogawa, Longitudinal Data Analysis, JSS Research Series in Statistics, https://doi.org/10.1007/978-981-10-0077-5_2

28 2 Autoregressive Linear Mixed Effects Models

In this chapter, we explore nonlinear mixed effects models, which incorporate both fixed and random effects to account for subject variability Unlike marginal models that focus solely on the marginal distribution without random effects, nonlinear mixed effects models address complex time trends that linear models may fail to capture Autoregressive linear mixed effects models, discussed here, effectively represent nonlinear trends approaching an asymptote while maintaining clear interpretations of both marginal and subject-specific outcomes These models highlight the importance of the marginal profile, as it is often more relevant than the conditional profile based on prior responses By integrating autoregression, we transform a static mixed effects model into a dynamic one, considering the influence of past covariate history.

There are two primary types of autoregressive models for continuous response variables: those that focus on the response itself and those that incorporate autoregressive error terms In this context, let Y i,t represent the response variable, X i,t denote the design vector, and ε i,t signify the error term for subject i at time t In autoregressive models of the response, the current response, Y i,t, is regressed on its previous value, Y i,t−1.

In linear models with the first-order autoregressive error, AR(1) error, an error term, ε ei , t , is regressed on the previous error term,ε ei , t − 1,

In Section 1.4.2, we explore the use of AR(1) errors in linear mixed effects models and marginal models, with the model in equation (2.1.2) being a part of the marginal models The interpretation of the fixed effects parameter β e remains consistent across both linear mixed effects and marginal models However, transition models of the response yield different interpretations for the fixed effects parameter This book delves into autoregressive models of the response in detail, highlighting differences from models with AR(1) errors as discussed in Section 2.6 Notably, when the response variable is discrete, such as binary or count data, the interpretation of the fixed effects parameter varies among the three main approaches Additionally, we clarify the difference in notation from Chapter 1, specifically regarding Y i.

In autoregressive linear mixed effects models, the response for the ith subject, represented as a vector T, is measured over multiple occasions Each individual measurement is denoted as Yij, where j indicates the specific measurement instance The baseline plays a crucial role in these models, influencing the overall analysis and interpretation of the data.

2.1 Autoregressive Models of Response Itself 29 concept, and we model the baseline measurement separately from the measurement at later time points For these models in the following chapters, we defineY i ,0 is a baseline measurement, and Y i , t (t 1,2,ã ã ã,T i )is thetth measurement after the baseline measurement.Y i

T is the(T i + 1)×1 vector of the response.

Regression models that analyze the current response based on previous responses and covariates are commonly referred to as autoregressive models or conditional models, as noted by Rosner et al (1985) and Rosner and Muñoz (1988).

In this book, we refer to various statistical frameworks as autoregressive models, including conditional autoregressive models (Schmid, 1996), state-dependence models (Lindsey, 1993), transition models (Diggle et al., 2002), dynamic models (Anderson and Hsiao, 1982; Schmid, 2001), Markov models, autoregressive response models, and lagged-response models (Rabe-Hesketh and Skrondal, 2012).

While AR(1) processes are commonly used in longitudinal data analysis, various other processes find application in time series analysis One such process is the autoregressive moving average process of order (p,q), known as ARMA(p,q).

The autoregressive model can be expressed as Y_t = ρ_1 Y_{t-1} + + ρ_p Y_{t-p} + ξ_t + θ_1 ξ_{t-1} + + θ_q ξ_{t-q}, where Y_t, Y_{t-1}, , Y_{t-p} are observed values and ξ_t, ξ_{t-1}, , ξ_{t-q} are random variables with a mean of zero and constant variance The parameters ρ_1, , ρ_p represent the autoregressive coefficients, while θ_1, , θ_q denote the unknown parameters in the model Specifically, an AR(1) model is a type of ARMA(1,0), and a higher-order autoregressive process of order p, denoted as AR(p), corresponds to an ARMA(p,0) model.

A moving average process of orderq(MA(q)) is ARMA(0,q) and

In this book, we use only the AR(1) process.

This article explores various aspects of autoregressive models, beginning with Section 2.1, which focuses on models for individual subjects Section 2.2 introduces random effects to address variability among subjects in longitudinal data In Section 2.3, autoregressive linear mixed effects models are presented, highlighting their vector representations and connections to linear mixed effects models Section 2.4 delves into variance-covariance structures, while Section 2.5 outlines estimation methods Finally, Section 2.6 discusses models incorporating autoregressive error terms.

Response Changes in Autoregressive Models

This section explores the variations in response levels within an autoregressive model, focusing on parameter interpretation To simplify the analysis, we examine a scenario involving a single subject without any random effects or errors Initially, we present an overview of the findings.

30 2 Autoregressive Linear Mixed Effects Models autoregressive model with an intercept The response at timet(t 1,2,ã ã ã,T),Y t , is a linear function of the previous response,Y t −1, as

Y t ρY t −1+β int , (2.1.3) whereρis a regression coefficient of the previous response andβ intis an intercept. These are unknown parameters.

Assumingρ1, this model can be transformed as

IfY t − 1 equals (1−ρ) −1 β int, the change is zero, and(1−ρ) −1 β int ≡ Y Asycan be interpreted as an asymptote if 0< ρ 0) asY 0 β base, the marginal form is

2.1 Autoregressive Models of Response Itself 31

The autoregressive model is represented as Y_t = ρY_{t-1} + β_int, with Y_0 defined as β_base The asymptote, denoted as Y_Asy, is calculated using the formula Y_Asy ≡ (1 - ρ)^{-1}β_int The change in Y, expressed as Y_t - Y_{t-1}, is directly proportional to the distance remaining to the asymptote, represented by Y_Asy - Y_{t-1}, with a proportionality constant of (1 - ρ) In this model, Y_t serves as a critical dividing point between Y_{t-1} and Y_Asy.

Fig 2.2 Autoregressive model, Y t ρ Y t − 1 + β int a Effects of the autoregressive coefficient ρ b Effects of the autoregressive coefficient ρ (0 < ρ < 1) ρ affects both the asymptote,

( 1 − ρ) −1 β int , and the proportion of the change, ( 1 − ρ)

This equation also shows that the asymptote,Y Asy, can be expressed by (1−ρ) − 1 β int,becauseY t →(1−ρ) − 1 β intwhent→ ∞if 0< ρ 1\), the response diverges without converging to an asymptote Additionally, when \(-1 < \rho < 0\), the response stabilizes at an asymptote with amplitude However, this discussion focuses solely on the scenario where \(0 < \rho < 1\), and in the absence of the intercept term \(\beta_{int}\), the equation simplifies to \(Y_t = \rho Y_{t-1}\), resulting in a response that trends towards zero.

Figure2.2b shows how changes in the response over time depend on the autoregressive coefficient, ρ, in the model, Y t ρY t −1 + β int, under the constraint

In the range of 0 < ρ < 1, the asymptote is defined as (1−ρ) − 1 β int Changes in response are directly proportional to the remaining size, with a constant factor of (1−ρ) Consequently, a smaller ρ results in a greater absolute difference between the baseline value and the asymptote, leading to a quicker approach to the asymptote Therefore, ρ significantly influences both the value of the asymptote and the rate of change towards it.

Fig 2.3 Autoregressive model, Y t ρ Y t − 1 + β int with Y 0 β base a Effects of the baseline. b Effects of the intercept The asymptote, ( 1 − ρ) − 1 β int , depends on the intercept

2.1 Autoregressive Models of Response Itself 33

In this book, the parameter ρ is treated as a constant value across subjects, but it can also be a random variable in broader contexts It's important to recognize that the asymptotes adjust simultaneously with changes in ρ, and the model is interpreted as representing profiles approaching the asymptotes under the constraint of 0 < ρ < 1.

Figure 2.3a and 2.3b illustrate how variations in the response are influenced by the baseline and intercept in the model Y_t = ρY_{t-1} + β_int with Y_0 = β_base The baseline parameter establishes the initial response at time zero, though its effect diminishes over time Given the limited number of time points in longitudinal data, the baseline plays a crucial role Meanwhile, the intercept determines the asymptote, calculated as (1−ρ)⁻¹β_int Although the proportion of change towards the asymptote remains constant at (1−ρ), the magnitude of change increases as the remaining size grows.

Y Asy −Y t −1 ρ t − 1 (1−ρ) − 1 β int−β base ρ t −1 Y Asy−β base

Figure 2.4a illustrates the relationship between response changes and time-independent covariates in the model, represented by the equation Y t = ρY t−1 + β int + β cov x The asymptotes, calculated as (1−ρ) − 1 (β int + β cov x), exhibit a linear dependency on the covariate x, with a coefficient of (1−ρ) − 1 β cov The proportion of change to the asymptote remains constant at (1−ρ), although larger changes occur when the remaining size is greater Conversely, Figure 2.4b demonstrates how response alterations are influenced by time as a continuous variable, resulting in a linear change in the asymptote over time.

Fig 2.4 a Autoregressive model, Y t ρ Y t−1 + β int + β cov x Effects of a time-independent covariate.

The asymptote, ( 1 − ρ) − 1 (β int + β cov x ) , linearly depends on the covariate, x, with the coefficient

( 1 − ρ) − 1 β x b Autoregressive model, Y t ρ Y t−1 + β int + β cov t The time-dependent covariate is time, t, as a continuous variable

The autoregressive model illustrated in Fig 2.5 demonstrates how a time-dependent covariate influences the response variable In this model, the response at time t (Yt) is determined by its previous value (Yt-1), along with contributions from the intercept (βint) and the covariate (βcov xt) The figure presents two scenarios: one where the covariate alters at a specific time point and another where it remains constant Notably, the response variable adjusts to a new asymptote following a change in the covariate value, highlighting the dynamic relationship between the covariate and the response.

Changes in response levels are influenced by time-dependent covariates, such as varying drug doses As illustrated in Figure 2.5a, the response can fluctuate based on covariate changes at specific time points, while other scenarios show stability without such changes This phenomenon, known as state-dependence, indicates that current responses are affected by both present and past covariate values Figure 2.5b further demonstrates that when covariate values shift, the response may initially adjust to a new asymptote, but if the covariate remains constant, the response gradually stabilizes at this new level.

Examples of Autoregressive Linear Mixed Effects Models

Example Without Covariates

An example of autoregressive linear mixed effects models without covariates is

The equation Y_i,t = ρY_i,t-1 + (β_int + b_int i) + ε_i,t (for t > 0) illustrates the relationship between baseline and response changes, where b_base i and b_int i represent random effects that account for individual differences and are assumed to follow a normal distribution This model captures both the initial state and the subsequent variations in response over time.

These equations can also be represented by the marginal form,

Y i , t ρ t (β base+b base i ) + t l 1 ρ t − l (β int+b int i ) +ε m i , t ,(t >0), (2.2.3) whereε m i , t is ε m i , t t l 0 ρ t − l ε i , l (2.2.4)

In particular,ε m i ,0 ε i ,0 Here, m in subscript means the marginal The following expression holds: ε m i , t ρε m i , t − 1+ε i , t (2.2.5)

Figure 2.6 illustrates instances where error terms are excluded By incorporating random effects, we can effectively capture the shifts from a baseline response level to alternative response levels for each individual Such changes are commonly observed in studies assessing the impacts of interventions The variability among subjects at both the baseline and asymptote is represented by Var(b base i) and Var.

The mean structures in Figures 2.6a and 2.6b are identical, yet the variability among subjects is greater at baseline in Figure 2.6a, while it is reduced at baseline in Figure 2.6b Additionally, the correlation between baseline measurements and asymptote values varies significantly, with some showing a strong correlation and others a weak one When assessing the effects of an intervention, the variability among subjects shifts due to differing responses to the intervention.

The autoregressive linear mixed effects model, as illustrated in Fig 2.6, incorporates random effects with omitted error terms, represented as Y_i, 0β_base + b_basei, Y_i, tρY_i, t−1 + β_int + b_inti In this model, b_basei signifies a random baseline effect, while b_inti denotes a random intercept Notably, the model exhibits larger variance in both the random baseline and the random asymptote, expressed as (1−ρ)−1b_inti.

Example with Time-Independent Covariates

This article presents an example of autoregressive linear mixed effects models incorporating time-independent covariates, where the baseline and subsequent time points are modeled separately In this framework, the value of the time-independent covariate remains constant for each subject over time The analysis focuses on comparing response changes between groups A and B, using dummy variables to represent each group Specifically, let \(x_{\text{base}, i} = 1\) for subject \(i\) in group B at time \(t_0\) and 0 otherwise, and \(x_{\text{int}, i} = 1\) for subject \(i\) in group B at time \(t > 0\) and 0 otherwise The model captures these dynamics effectively.

Y i , t ρY i , t − 1+β int+β int g x int i +b int i +ε i , t , (t >0) (2.2.6) The asymptote of the subjecti,Y Asy i , is

Y Asy i (1−ρ) − 1 β int+β int g x int i +b int i

The expected values of the asymptotes of groups A and B are

2.2 Examples of Autoregressive Linear Mixed Effects Models 37

Fig 2.7 Autoregressive linear mixed effects model under randomization—error terms are omitted,

The study examines the response changes in two groups, identified as xg0 and xg1, revealing that while baseline distributions are comparable, later distributions diverge significantly Notably, the variances of a random baseline (b base i) are consistent across groups; however, the variances of a random asymptote ((1 - ρ) - 1 b int i) show considerable differences.

The expected difference between the asymptotes of groups A and B is

In a randomized controlled trial (RCT) example presented in Sect 3.1, a comparison across three treatment groups, including a placebo group, is illustrated It is important to note that variance covariance matrices may vary between treatment groups For instance, Fig 2.7 depicts an RCT where error terms are excluded Typically, the response distributions at baseline should be similar across groups; however, significant differences may arise at later time points In such scenarios, we can assume equal variances for baseline responses while allowing for different variances in the intervention responses between the groups.

Example with a Time-Dependent Covariate

In clinical studies, the asymptotes of a model are influenced by time-dependent covariates, such as drug dosing For a subject at time t, the drug dose can be represented as x_i,t This scenario is illustrated through autoregressive linear mixed effects models that incorporate time-dependent covariates, highlighting their significance in analyzing dynamic changes in response to treatment.

Y i , t ρY i , t − 1+(β int+b int i )+(β cov+b cov i )x i , t +ε i , t , (t>0), (2.2.8) whereb cov i , a coefficient of the covariatex i , t , is an additional random variable The asymptote of the subjectiat timet,Y Asy i , t , is

Y Asy i , t (1−ρ) −1 β int+b int i +(β cov+b cov i )x i , t

The asymptote is influenced by the covariate \( x_{i,t} \), where the term \( (1-\rho)^{-1} b \, \text{cov}_i \) illustrates varying sensitivity to dose modifications among subjects While some individuals exhibit significant changes in response levels with dosing adjustments, others show minimal variation Examples of clinical studies where treatment dose acts as a time-dependent covariate are discussed in Sections 3.3 and 4.3.

Autoregressive Linear Mixed Effects Models

Autoregressive Form

The response vector \( T \) is defined as a \((T_{i+1}) \times 1\) vector corresponding to the \(i\)th subject, measured from 0 to \(T_i\) The baseline measurement is represented as \(Y_{i,0}\), while \(Y_{i,t}\) (for \(t = 1, 2, \ldots, T_i\)) indicates the \(t\)th measurement following the baseline We introduce a \((T_{i+1}) \times (T_{i+1})\) matrix \(F_i\) characterized by 1s just below the diagonal and 0s elsewhere The product \(F_i Y_i\) yields the vector of previous responses.

2.3 Autoregressive Linear Mixed Effects Models 39

Table 2.1 Representations of autoregressive linear mixed effects models

(d) Nonlinear mixed effects models (without covariate) b

For a comprehensive understanding of the definitions of J i, M x, and M z, refer to Section 2.3.2 For an in-depth exploration of nonlinear mixed effects models and differential equations, consult Chapter 5 Additionally, Chapter 6 provides detailed information on state space representation.

Autoregressive linear mixed effects models are expressed as

The model is defined by the equation Y_i = ρF_i + X_iβ + Z_ib_i + ε_i, where ρ represents an unknown regression coefficient, β is a vector of fixed effects parameters, and X_i and Z_i are design matrices for fixed and random effects, respectively The vector b_i contains random effects parameters, while ε_i represents random errors It is assumed that both b_i and ε_i are independent across subjects and follow a normal distribution with a mean of zero, characterized by their respective variance-covariance matrices G and R_i Specifically, b_i follows a multivariate normal distribution MVN(0, G) and ε_i follows MVN(0, R_i).

The variance-covariance matrix of the response vector \( Y_i \), conditioned on the previous response \( F_i Y_i \), is denoted as \( \text{LetV}_i \) Similar to the linear mixed effects models discussed in Section 1.2, this variance-covariance matrix is expressed in a specific format.

The key distinction from linear mixed effects models lies in the incorporation of the term ρ F i Y i Additionally, there are variations in how model parameters are interpreted, as well as shifts in response levels and variance-covariance structures These aspects will be explored in the subsequent sections.

Table 2.2a presents the vector representation of the autoregressive linear mixed effects model (2.2.8) that incorporates a time-dependent covariate for T i 3 In this context, X i and Z i are defined as block diagonal matrices, where the blocks represent the baseline components (t0) and additional components for time periods greater than zero (t > 0).

Table 2.2 Three representations of an example of autoregressive linear mixed effects models for

⎜ ⎝ β base + b base i β int + b int i + ( β cov + b cov i )x i,1 β int + b int i + ( β cov + b cov i )x i,2 β int + b int i + ( β cov + b cov i )x i,3

Y Base Asy i X i M x β + Z i M z b i X i β ∗ + Z i b ∗ i (2.3.13) where *(asterisk) shows the parameters for the asymptote Model (2.2.8) with the representation (2.3.7)

⎜ ⎝ β base + b base i β ∗ int + b int ∗ i β ∗ cov + b cov ∗ i

Model (2.2.8) with the marginal form (2.3.19)

Representation of Response Changes

The model (2.2.8) can be represented by response changes with asymptotes as

Y Asy i , t (1−ρ) −1 β int+b int i +(β cov+b cov i )x i , t β int ∗ +b ∗ int i − β cov ∗ +b cov ∗ i x i , t

, (2.3.7) where * (asterisk) shows the parameters for the asymptote The asymptote linearly depends on the covariate.

Changes at each time point can be shown in the following vector representation:

, (2.3.8) where X i , t andZ i , t are the corresponding 1×p and 1×q row vectors ofX i and

In this analysis, we focus on scenarios where both \(X_i\) and \(Z_i\) are block diagonal matrices, with blocks representing baseline components (\(t_0\)) and subsequent components (\(t > 0\)) The parameters \(\beta_s\) and \(b_s\) are redefined as \(\beta^*_s\) and \(b^*_s\) for the asymptote by applying a transformation that involves multiplying by \((1 - \rho)^{-1}\) To facilitate the representation of these parameter transformations, we introduce a \(p \times p\) diagonal matrix for the vectors \(\beta^* M x \beta\) and \(b^*_i M z b_i\).

In the model represented by equation (2.3.7), the diagonal matrices M x and M z feature diagonal elements of 1 for baseline parameters and (1−ρ) − 1 for later time point parameters Specifically, the model includes three fixed effects parameters: β base, β int, and β cov.

Changes at all the time points can be shown in the following vector representation:

Y i −F i Y i is a(T i+ 1)×1 vector in which the first element is a baseline response,

In the analysis of response changes, we denote the differences as \( Y_{i,t} - Y_{i,t-1} \) To represent this in vector form, we introduce a diagonal matrix \( J_i \) of size \( (T_{i+1}) \times (T_{i+1}) \) The first element of \( J_i \) is set to 1, representing the baseline at \( t_0 \), while the remaining elements are defined by the proportional constant \( (1 - \rho) \), applicable for subsequent time points where \( t > 0 \) Therefore, for \( T_i = 3 \), the structure of \( J_i \) is established.

LetY Base Asy i be the(T i + 1)×1 vector, and the first element corresponds to the baseline, and the other elements correspond to the asymptotes,

Then, the representation of response changes with asymptotes of autoregressive linear mixed effects models (2.3.3) is

The expected value ofY Base Asy i is

The anticipated shift from \( Y_{i, t-1} \) to \( Y_{i, t} \) under random effects is directly related to the distance remaining to the asymptote, represented as \( Y_{\text{Base Asy}, i} - F_i Y_i \), excluding the initial element Table 2.2b illustrates the vector representation of this relationship for \( T_i = 3 \).

Marginal Form

The marginal form (unconditional form) of autoregressive linear mixed effects models (2.3.3) is

The equation Y_i (I_i - ρ F_i)^{-1} (X_i β + Z_i b_i + ε_i) is presented in this section, where I_i denotes a (T_i + 1)×(T_i + 1) identity matrix The subscript i of I_i indicates the subject rather than the size of the identity matrix This equation is derived by multiplying both sides of the preceding equation by (I_i - ρ F_i)^{-1}.

The expectation given random effectsb i is E( Y i |b i )( I i −ρ F i ) −1 ( X i β+Z i b i ), and the marginal expectation is E( Y i ) ( I i −ρ F i ) −1 X i β The expectation for a typical subject and the marginal expectation are the same, E( Y i |b i 0 )E( Y i ).

Subject specific interpretation and marginal interpretation are the same Let i be the marginal variance covariance matrix of the response vectorY i i is i Var( Y i )

Given the marginal variance covariance matrix i , the variance covariance matrix of the autoregressive formV i can be expressed as

V i ( I i −ρ F i ) i ( I i −ρ F i ) T (2.3.18)The marginal form of the model (2.2.8) is

, (2.3.19) whereε m i , t t l 0 ρ t − l ε i , l Table2.2c provides the vector representation of (2.3.19) forT i3.

Variance Covariance Structures

AR(1) Error and Measurement Error

In linear mixed effects models featuring random effects, it is commonly assumed that random errors are independent Specifically, this independence manifests as an autoregressive AR(1) error in the marginal framework In practice, data analysis frequently encounters measurement errors that are independent over time Consequently, assuming independent errors in a marginal context is justifiable, especially when the measurement method lacks precision.

We consider the error structures induced by an AR(1) error and a measurement error simultaneously We consider two different assumptions for an AR(1) error term in

First,ε i ,0does not include an AR(1) error The error structure in the autoregressive form such as (2.2.1) is ε i ,0 ε (ME)i ,0 ε i , t ε (AR)i , t +ε (ME)i , t −ρε (ME)i , t −1 , (t>0), (2.4.1)

46 2 Autoregressive Linear Mixed Effects Models whereε (AR)i , t andε (ME)i , t independently follow a normal distribution with the mean

0 and the variancesσ AR 2 andσ ME 2 , respectively Here, AR means autoregressive, and

ME means a measurement error In the marginal form such as (2.2.3), this error structure is

From these equations, we can confirm that ε (ME)i , t is an independent error in the marginal form The variance of ε m i , t is not constant and this AR(1) error is not stationary.

In this study, we analyze a stationary AR(1) error structure represented by the equation ε_i,0 = ε_(AR,ST)i,0 + ε_(ME)i,0 + ε_i,t = ε_(AR,ST)i,t + ε_(ME)i,t - ρε_(ME)i,t-1 for t > 0 Here, ε_(AR,ST)i,0, ε_(AR,ST)i,t (for t > 0), and ε_(ME)i,t are assumed to follow a normal distribution with a mean of 0 and specific variances.

1−ρ 2 − 1 σ AR 2 , ST ,σ AR 2 , ST , andσ ME 2 , respectively Here, ST in subscript means stationary In the marginal form, this error structure is ε m i , t t j 0 ρ t − j ε (AR,ST)i , j +ε (ME)i , t (2.4.4)

The variance ofε m i , t is Var ε m i , t

1−ρ 2 −1 σ AR 2 , ST +σ ME 2 and it is constant. The model (2.2.1) (t >0) with the error structure (2.4.1) is

Y i , t ρY i , t − 1+(β int+b int i )+ε (AR)i , t +ε (ME)i , t −ρε (ME)i , t − 1 , (t >0).

The following transformation makes clear thatε (ME)i , t is a measurement error,

In the context of measurement errors, the latent variable \( Y_{i,t - \epsilon (ME)} \) represents the ideal scenario without such errors, as discussed in Section 6.3.1's state space representation We now present the variance-covariance matrix of these error structures, denoted as \( R_{ME_i} \), which reflects the autoregressive nature of the measurement error \( \epsilon(ME)_{i,t} \) Additionally, the corresponding variance-covariance matrix in marginal form, \( ME_i \), indicates that the error term is independent and maintains a constant variance, expressed as \( ME_i \sigma_{ME}^2 I_i \) Therefore, the autoregressive form is represented by \( R_{ME_i} \).

Table 2.3 Variance covariance structures for autoregressive linear mixed effects models in the autoregressive form and marginal form for T i 3

When j is infinite, the ( j, j)th element is

When j and k are infinite, the ( j , k ) th element is 0 (i) Random baseline and random intercept

(j) Random baseline and random intercept

When j and k are infinite, it is

Table2.3a and b shows these matrices forT i 3.R ME i is a structure similar to two-band Toeplitz in Table1.1m except for the (1, 1)th element.

The variance-covariance matrices of AR(1) errors, denoted as LetR AR i and R AR,ST i, are presented in autoregressive form, while their corresponding marginal form matrices are represented as AR i and AR,ST i These matrices are detailed in Table 2.3c–f When a random baseline effect b base i is included in the model, it becomes challenging to determine the stationarity of the AR(1) error based solely on model fit, as discussed in Section 2.4.2 Additionally, the parameter σ AR 2 is influenced by the unit time, with the jth diagonal element of AR i representing σ AR 2.

1−ρ 2 −1 σ AR 2 and does not depend on the unit time.

Variance Covariance Matrix Induced by Random

In this analysis, we examine the role of random effects in the variance-covariance matrices of the response vector According to model (2.2.1), the baseline level, denoted as \( b_{base,i}(t_0) \), and the intercept, \( b_{int,i}(t>0) \), are treated as random effects It is assumed that the vector \( b_i(b_{base,i}, b_{int,i})^T \) follows a bivariate normal distribution characterized by a zero mean vector and a variance-covariance matrix \( G \), which includes variances \( \sigma_{G0}^2 \) and \( \sigma_{G1}^2 \), along with a covariance \( \sigma_{G01} \) This relationship is represented as \( b_{base,i} \) and \( b_{int,i} \).

ForT i 3, the autoregressive form of the variance covariance matrix of the response vector induced by the random effects is

This is a block diagonal matrix The transformation to the marginal form is

The(j+ 1,k+ 1)th element of this matrix is ρ j+k σ G0 2 + ρ j +ρ k −2ρ j+k

When both j andk are infinite, the (j,k)th element is

1−ρ 2 − 1 σ G1 2 ; this value represents the inter-individual variance of the random asymptotes Table2.3i and j shows these matrices.

In this analysis, we examine a model with three random effects: the baseline level \( b_{\text{base}, i}(t_0) \), the intercept \( b_{\text{int}, i}(t > 0) \), and a covariate effect \( b_{\text{cov}, i} \) These random effects, represented as \( b_i (b_{\text{base}, i}, b_{\text{int}, i}, b_{\text{cov}, i})^T \), are assumed to follow a trivariate normal distribution.

⎝ σ Gb 2 σ Gbi σ Gbc σ Gbi σ Gi 2 σ Gic σ Gbc σ Gic σ Gc 2

ForT i 3, the variance covariance matrix of the response vector induced by the random effects is

Table2.3g shows the autoregressive form of the variance covariance structure induced by a random baseline, b base i , forT i 3 σ G0 2 is the (1, 1)th element of

V i , and the difference in the assumptions betweenR AR i andR AR , ST i is also on the

The presence of a random baseline effect in the model prevents us from determining the stationarity of the AR(1) error solely based on model fit The marginal form related to this is detailed in Table 2.3h.

Variance Covariance Matrix Induced by Random

In this analysis, we examine the variance-covariance matrix of the response vector under the simultaneous assumptions of random effects, including b base i and b int i, along with an AR(1) error term, ε (AR)i , t, and a measurement error, ε (ME)i , t We begin by considering a non-stationary AR(1) error with ε (AR)i ,0 set to 0 For T i equal to 3, the autoregressive structure is illustrated in Table 2.4a, providing insights into the variance-covariance relationships within the model.

Z i GZ i T +R i Table2.4b shows the corresponding marginal form i , whereAis a

4×4 matrix, and the(j+ 1,k+ 1)th element is (2.4.8) Whenjandkare infinite, the random effects and the measurement errors produce a compound symmetry (CS) structure with the diagonal elements being

1−ρ 2 −1 σ G1 2 +σ ME 2 and the nondiagonal elements being

In this analysis, we consider a stationary AR(1) error term, denoted as ε(AR,ST)i,t, in place of the previously used ε(AR)i,t The autoregressive structure of the variance-covariance matrix for the response vector V_i is presented in Table 2.4c, while Table 2.4d illustrates the corresponding marginal form.

Table 2.4 Examples of variance covariance matrices of the response vector induced by random effects and random errors in the autoregressive form and marginal form for T i 3

Representation, assumption on AR(1) error

(a) Autoregressive form V i , non-stationary AR(1)

(b) a Marginal (unconditional) form i , non-stationary AR(1)

⎠ (d) a Marginal (unconditional) form i , stationary AR(1)

+ AR , STi + ME i A + σ AR,ST 2

⎠ a The ( j + 1 , k + 1 ) th element of A is ρ j+k σ G0 2 + ( ρ j + ρ k − 2 ρ j+k )(1 − ρ ) −1 σ G01 + (1 −ρ j )

Variance Covariance Matrix for Asymptotes

In Section 2.3.2, the fixed effects and random effects parameters, β and b_i, were adjusted using matrices M_x and M_z, resulting in the transformed parameters β* and b*_i This section focuses on analyzing the variance-covariance matrix of the transformed random effects parameter b*_i.

First, we consider the case of two random effects,b i (b base i ,b int i ) T , as shown in the model (2.2.1) LetM z be a diagonal matrix with diagonal elements

. Then,b i is transformed tob ∗ i M z b i , b ∗ i b base i b ∗ int i

Here, * (asterisk) shows the parameters for the asymptote.b i andb ∗ i follow multivariate normal distributions, b i ∼MVN(0 , G), (2.4.11) b ∗ i ∼MVN

The variance covariance matrix ofb ∗ i is

(1−ρ) −1 σ G01is the covariance of the random baseline and random asymptote The correlation is

Next, we consider the case of three random effectsb i (b base i ,b int i ,b cov i ) T as shown in the model (2.2.8) LetM z be a diagonal matrix with diagonal elements

The variance covariance matrix ofb ∗ i , M z GM T z is

Estimation in Autoregressive Linear Mixed Effects Models

Missing Data

Multivariate Longitudinal Data and Vector Autoregressive

Multivariate Autoregressive Linear Mixed Effects Models

Appendix

Nonlinear Mixed Effects Models

Nonlinear Curves

Time Series Data

Longitudinal Data

Autoregressive Linear Mixed Effects Models

Linear Mixed Effects Models

Tiêu đề	Longitudinal Data Analysis Autoregressive Linear Mixed Effects Models
Tác giả	Ikuko Funatogawa, Takashi Funatogawa
Người hướng dẫn	Naoto Kunitomo, Akimichi Takemura
Trường học	The Institute of Statistical Mathematics
Chuyên ngành	Statistical Data Science
Thể loại	springer briefs
Năm xuất bản	2018
Thành phố	Tachikawa

Định dạng
Số trang	150
Dung lượng	3,39 MB
File đính kèm	139. Longitudinal Data Analysi.rar (2 MB)