Latent Variable Models in Educational Research

Modeling item response profiles using factor models, latent class models, and latent variable hybrids Dena Pastor James Madison University pastorda@jmu.edu

Purposes of the Presentation To present the model-implied item response profiles (IRPs) that correspond to latent variable models used with dichotomous item response data To provide an example of how these models can be used in practice

Item Response Profiles (IRPs) 1 Proportion Responding Correctly 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4 Item Number

Pattern Differences IRPs for classes of examinees with different patterns

Elevation Differences IRPs for classes of examinees with the same pattern, but differences in elevation

Latent Variable Model C NON-PARALLEL PARALLEL Latent Class Model C is a latent categorical variable with as many levels as # of classes C is a nominal latent variable C is a ordinal latent variable

Exploratory Process In latent class modeling a variety of models are fit to the data with differing numbers of classes 1-class model, 2-class model, 3-class model, etc. Use fit indices and a priori expectations to determine the number of classes to retain Can allow latent categorical variable to be nominal and examine resulting profiles; can also constrain latent categorical variable to be ordinal

Alternative Model for Parallel Profiles Do we have 3 classes, with no variability within class? Factor Model F is a latent continuous variable OR F Do we have 1 profile with systematic variability within class?

Different Models for Different IRPs + within profile variability 1 profile 1 1 0.9 0.9 0.8 0.8 0.7 0.7 LCM: 1 class Factor Model 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 1 2 3 4 1 2 3 4 2 parallel profiles + within profile variability 1 1 0.9 0.9 LCM: 2 classes (C is ordinal) Semi-parametric Factor Model 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 1 2 3 4 1 2 3 4 2 non-parallel profiles + within profile variability 1 1 0.9 Factor Mixture Model 0.9 0.8 LCM: 2 classes (C is nominal) 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 Latent Variable Hybrids 0.2 0.1 0.1 0 1 2 3 4 0 1 2 3 4

Number of profiles? (number of classes) 1+ 1 Nature of profile differences? Decisions Non- parallel Parallel Systematic variability within profiles? Systematic variability within profiles? Systematic variability within profiles? no yes no yes no yes Semi- parametric factor model (SPFM) Latent class model (LCM) Factor model (FM) LCM with parallel profiles LCM with non- parallel profiles Factor mixture model (FMM) Models 1 1 1 1 1 1 0.9 0.9 0.9 0.9 0.9 0.9 0.8 0.8 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 IRPs 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3 0.3 0.3 0.3 0.2 0.2 0.2 0.2 0.2 0.2 0.1 0.1 0.1 0.1 0.1 0.1 0 0 0 0 0 0 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

1 1 1 Latent class model (LCM) C 0.9 0.9 0.9 0.8 0.8 0.8 0.7 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0.4 0.4 0.3 0.3 0.3 0.2 0.2 0.2 0.1 0.1 0.1 0 0 0 1 2 3 4 1 2 3 4 1 2 3 4 2 classes 2 classes 1 1 F C 0.9 0.9 F C 0.8 0.8 0.7 0.7 0.6 0.6 Semi-parametric factor model (SPFM) 0.5 0.5 Factor mixture model (FMM) 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 1 2 3 4 1 2 3 4 1 class: Factor Model! 1 class: Factor Model! 1 1 F C F C 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 1 2 3 4 1 2 3 4 2 classes, w/in class factor variance = 0 2 classes, w/in class factor variance = 0 1 1 F C 0.9 0.9 0.8 F C 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 1 2 3 4 0 1 2 3 4

Marginal probability of getting an item correct is sum across classes of probability of getting item correct conditional on class membership K Conditional probability differs across models = 1| ) = = ( ( 1| ) P u P u c i k i = 1 k F + + C exp( + ) F Factor mixture model (FMM) ~ (0, ) F N ki ki k F k k 1 (exp( )) ki ki k F C Semi- parametric factor model (SPFM) + exp( + ) F N ~ ( , ) F i + i k F k k k 1 (exp( )) i i k Latent class model (LCM) exp( + ) C ki 1 (exp( )) ki

Latent class model (LCM) Latent Variable Distribution IRP Path diagram 1 0.9 0.8 0.7 C 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4 1 0.9 0.8 C 0.7 C is ordinal 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4 1 0.9 C C is 0.8 0.7 0.6 nominal 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4

Semi-Parametric Factor Model (SPFM) Latent Variable Distribution IRP Path diagram 1 0.9 0.8 F 0.7 C 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4 1 0.9 0.8 F 0.7 C 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4 Measurement Invariance Quantitative differences between classes Same measurement model parameters (thresholds, loadings) for each class

Factor Mixture Model (FMM) Latent Variable Distribution IRP Path diagram 1 0.9 F C 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4 1 0.9 0.8 F C 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 3 4 Measurement Non-Invariance Qualitative differences between classes Different measurement model parameters (thresholds, loadings) for each class

Example 9 dichotomously scored items measuring 3 aspects of psychosocial research: 1. Confidentiality 2. Generalizability 3. Informed Consent Sample 2,259 incoming freshmen tested in low- stakes conditions prior to start of classes

Exploratory Model Selection Exploratory model selection approach to answer the question, What type and number of latent variables are most salient for our data? Reasons to believe that IRPs would differ in pattern and/or elevation because students differ in: Completion of psychosocial coursework Effort they put forth on test

Model Fit Indices Model FM LL # paras 18 26 33 BIC 25395 25352 25348 SSA-BIC 25338 25270 25243 LMR NA NA NA 1f 2f 3f -12628 -12576 -12547 LCM 1c 2c 3c 4c 5c -12938 -12649 -12591 -12538 -12520 9 25946 25445 25406 25377 25418 25918 25384 25314 25253 25262 NA 0.00 0.07 0.01 0.14 19 29 39 49 SPFM 1f2c -12618 21 25399 25332 0.01 FMM 1f2c -12539 35 25348 25237 0.00

IRPs of 4 Class LCM 0.18 0.36 0.25 0.20 generalizability

2-class FMM Factor Variability Within Each Class = = 0.27 3.96 X Y 0.44 0.56 26. Which ethical practice is not considered by Marty? a) She failed to obtain informed consent from her participants b) She failed to randomly select participants c) d)

Visually Conveying Loading Information Standardized Loadings = = 0.27 3.96 Item Content Confidentiality Item Number 17 34 39 25 27 36 23 26 32 Class X 0.26 0.28 0.16 0.46 0.49 0.46 0.51 0.88 0.44 Class Y 0.36 0.74 0.81 -0.03 -0.07 0.27 0.30 0.35 0.31 X Y Generalizability Informed Consent X Y

Validity Evidence for 2-class FMM Solution X Y Students with higher SAT-V scores, who reported put forth more effort on the test, and who have completed psychosocial coursework more likely to be in Class X Positive relationship between SAT-V, coursework completion and factor scores in that class (negative relationship with effort) Negative relationship between number of missing responses and factor scores in Class Y

Correspondence Between Models C & D from LCM, Y from FMM A & B from LCM, X from FMM X & Y from FMM with intervals

Parting Thoughts These models are like potato chips It was so much easier to settle on a brand of chip when I had a limited number of brands to choose from But I also like having more brands because it increases my chances of finding the brand that is right for me With all these brands, it is possible that some are selling essentially the same chip .but which ones? When two brands are essentially the same chip, what criteria do I use to choose between the two brands?

Questions? pastorda@jmu.edu Pastor, D. A., & Gagn , P. (2013). Mean and covariance structure mixture models. In G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd Ed.). Greenwich, CT: Information Age. Pastor, D. A., Lau, A. R., & Setzer, J. C. (2007, August). Modeling item response profiles using factor models, latent class models, and latent variable hybrids. Poster presented at the annual meeting of the American Psychological Association, San Francisco.