Estimating Measurement and Structure: Factor Scores and SAM

From Measuring Latent Variables to Estimating Relationships

Attributes that are difficult to observe directly, such as motivation and skills, are measured through multiple indicators, and their relationships with other variables are estimated from these measurements. Structural equation modeling (SEM) allows measurement and structural models to be estimated simultaneously. A sequential approach is also possible: factor scores are first calculated and then used in regression analysis. Factor scores are predicted values of latent variables. Estimates of structural coefficients depend on whether the scores’ variances and covariances match those of the latent variables. When measurement and structural models are estimated separately, the central task is to correct for measurement error and carry uncertainty from the measurement stage into the subsequent analysis.

Bias and Hypothesis Testing in Factor Score Regression

A study comparing four methods of factor score regression (FSR) explains bias in terms of how scores are calculated and the roles variables play in the regression equation (Devlieger et al., 2016). Infinitely many methods for calculating factor scores are consistent with the same factor analysis results. This factor indeterminacy decreases when relationships between indicators and latent variables are strong or when there are many indicators. In the simple regression equation η = γξ + ζ, coefficient properties depend on which variables are replaced with factor scores and how the latent variable scale is set. Regression FSR yields an unbiased unstandardized coefficient when only the independent variable is a factor score. Bartlett FSR yields an unbiased coefficient when only the dependent variable is a factor score. The bias avoiding method uses a regression predictor for the independent variable and a Bartlett predictor for the dependent variable. It avoids bias in the unstandardized coefficient. Bias remains in the standardized coefficient because the standard deviations of the factor scores differ from those of the latent variables. Variable roles are specified in advance. The method cannot apply to mediation models in which a single variable serves as both an independent and a dependent variable. The bias correcting method recovers the variances and covariances of latent variables from those of the factor scores. It removes bias in both unstandardized and standardized coefficients at the population level.

To test hypotheses about the corrected coefficients, an approximate standard error adds the prediction errors of the factor scores for the independent and dependent variables to the regression error. The stronger the relationship between variables, the greater the contribution of the independent variable’s prediction error. The statistic obtained by dividing the corrected coefficient by this standard error is compared with a t-distribution with n−(p+1) degrees of freedom. Here, n is the sample size, and p is the number of independent variables. The simulation included 3 latent independent variables and 2 latent dependent variables. The independent variable variance was set to 100, the dependent variable residual variance to 400, and all factor loadings to 1. To compare the four FSR methods and SEM, the research team generated 1000 datasets for each of 216 conditions that varied the measurement model’s coefficient of determination (CD), regression coefficient γ, sample size n, and number of items I. The higher the CD, the lower the factor indeterminacy. The item responses generated in R for Study 1 followed a normal distribution with a skewness of 0 and kurtosis of 3. In Study 2, measurement errors were generated using a t-distribution with 3 degrees of freedom and a χ² distribution with 1 degree of freedom. The average skewness and kurtosis of item responses in the two non-normal conditions were 0.001 and approximately 10.057, and 0.693 and 4.785, respectively. SEM was specified to match the data generation model and estimated using maximum likelihood. The evaluation covered success rate, coefficient bias, efficiency, mean squared error (MSE), standard error bias, Type I error rate, and statistical power.

With normally distributed data, the success rate across all methods and conditions was 0.989 or higher, and the Type I error rate for the five methods was close to the expected value of 0.05. Regression FSR and Bartlett FSR underestimated unstandardized coefficients, and increasing the sample size did not reduce the bias. Bias decreased as the number of items increased but remained even at 10 items. The bias avoiding method also underestimated standardized coefficients. The bias correcting method and SEM showed little bias on both scales, with similar efficiency, MSE, and statistical power under normally distributed data. Standard error bias was smallest for SEM and generally decreased as sample size, CD, and the number of items increased. With non-normal data, efficiency and success rates declined in the t-distribution condition with high kurtosis. SEM had a low success rate when CD=0.3 or I=3. SEM had the smallest bias and MSE, followed closely by the bias correcting method. The Type I error rate exceeded 0.05 for all methods and was particularly high for the bias correcting method when CD=0.30. The bias correcting method was considered an appropriate alternative to SEM among the FSR methods compared. The implementation presented at the time had limitations concerning overall fit indices, mediation relationships, and non-recursive structural models. The simulation design set all loadings to 1 and correctly specified SEM. Differences between methods disappeared across all performance indicators at CD=0.99.

SAM Expanded to Measurement Blocks and Two-Step Estimation

A study formalizing Structural–After–Measurement (SAM) extends the correction used in factor score regression into sequentially estimating measurement and structural models (Rosseel & Loh, 2024). SAM first estimates measurement parameters, fixes their values, and then estimates structural parameters; the unit used to organize the measurement model is defined as a measurement block. For models with continuous latent variables and observed indicators, all latent variables can be placed in a single CFA block, assigned to separate blocks, or grouped into subsets. Residual correlations between indicators in different blocks are set to 0. When equality constraints, cross-loadings, or correlated residuals are present, the associated latent variables are grouped together. This configuration limits how misspecification in one block affects estimation in other blocks. A block with two indicators for a single factor is not identified, whereas a block with three indicators is saturated, so interpreting fit at the block level requires considering both the number of indicators and the model configuration. In a previous example reviewed in the study, estimating and fixing measurement parameters using individual CFAs yielded a regression coefficient of 0.878, compared with the true population value of 0.700.

Local SAM estimates the structural model after calculating latent variable means and covariances from observed means and covariances and estimated measurement parameters. It uses a mapping matrix M that satisfies MΛ = I for the factor loading matrix Λ. Latent means are calculated by subtracting indicator intercepts from the observed means and applying M. The latent covariance matrix is calculated as M(S−Θ)Mᵀ. The measurement error covariance matrix Θ is subtracted from the observed covariance matrix S. The result is multiplied by M on the left and its transpose on the right. M can be constructed using ML, GLS, or ULS criteria. The expression under the ML criterion is identical to the Bartlett factor score matrix. Unlike entering factor scores directly into a regression analysis, this procedure corrects the covariance for measurement error. Using the recovered summary statistics, OLS can be applied to recursive models, and 2SLS with instrumental variables can be applied to non-recursive models. Local SAM applies when the measurement and structural parts do not overlap, Λ has full column rank, and the estimated latent covariance matrix is positive definite. Global SAM does not explicitly calculate latent summary statistics or M. It estimates structural parameters using methods such as ML, GLS, and WLS while holding measurement parameters fixed in the overall SEM. It can be applied to models in which an indicator directly influences other latent variables. When local SAM is applicable and the same estimation method is used in both steps, the two approaches yield identical structural parameter estimates.

The standard errors of the structural parameters reflect uncertainty in the measurement parameters estimated in the first step. The sam() implementation in lavaan 0.6-9 described in the study applies the same two-step standard error correction to both local SAM and global SAM. A non-parametric bootstrap was presented as an alternative. Fit is evaluated separately for the measurement blocks and the unsaturated structural model. ‘Pseudo’ versions of existing indices are also calculated using the ML discrepancy function after SAM estimates are substituted into the overall model to obtain model-implied means and covariances. This procedure also applies when estimation methods other than ML are used in the two steps. The study’s formal treatment focuses on single-group and single-level applications. The implementation at the time also supported multigroup models with equality constraints across groups and 2-level SEM with random intercepts. Missing data, categorical indicators, interaction terms, and quadratic terms in the structural model were presented as areas for extension.

Reading notes

When standardized coefficients are used to compare relationships among learning motivation, self-efficacy, and competence in HRD and medical education, interpretation of their magnitude depends on the scale’s measurement quality and the method used to correct covariances. The choice of how to divide measurement blocks also determines where residual correlations and cross-loadings among indicators are allowed. Coefficient bias and Type I error rates followed different patterns when non-normality coincided with low CD. Small bias alone provides a limited basis for assessing hypothesis testing performance.

References

• Devlieger, I., Mayer, A., & Rosseel, Y. (2016). Hypothesis testing using factor score regression: A comparison of four methods. Educational and Psychological Measurement, 76(5), 741–770. https://doi.org/10.1177/0013164415607618
• Rosseel, Y., & Loh, W. W. (2024). A structural after measurement approach to structural equation modeling. Psychological Methods, 29(3), 561–588. https://doi.org/10.1037/met0000503