Assessing Psychological Flexibility: CompACT-15 Confirmatory Measurement
Explore the Confirmatory Measurement Modeling and Longitudinal Invariance of the CompACT-15, a short-form assessment of psychological flexibility. Learn about its structure, strengths, weaknesses, and comparisons with other multidimensional measures in the field.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Confirmatory Measurement Modeling and Longitudinal Invariance of the CompACT-15: A Short-Form Assessment of Psychological Flexibility Ti Hsu1, Lesa Hoffman2, and Emily B.K. Thomas1 University of Iowa 1Department of Psychological and Brain Sciences 2Department of Psychological and Quantitative Foundations
Awareness Openness
Awareness Openness Engagement
Unidimensional Measurement of PF The Acceptance and Action Questionnaire-II (Bond et al., 2011).
Weaknesses Strengths Bond et al., 2011 Well-fitting 1 factor structure after addition of residual variance Good alpha reliability Construct validity Fledderus et al., 2012 Replicated 1 factor structure Incremental validity No DIF by gender Flynn et al., 2016 Marginal fit for 1 factor structure after addition of residual variance Incremental validity Wolgast., 2014 AAQ-II items more strongly related to items measuring distress rather than acceptance/nonacceptance Tyndall et al., 2019 AAQ-II factor more strongly related to internalizing symptoms and stress than other measure of PF Rochefort et al., 2018 AAQ-II items also measure neuroticism and negative affect
Multidimensional Experiential Avoidance Questionnaire (MEAQ; G mez et al., 2011) 62 items. Multidimensional Psychological Flexibility Inventory (MPFI; Rolffs et al., 2016) 60 items. Multidimensional Measurement of PF Psy-Flex (Gloster et al., 2021) 6 items. Comprehensive assessment of Acceptance and Commitment Therapy processes (CompACT; Francis et al., 2016) 23 items.
23-item measure with a three- factor structure: Openness to Experience Behavioral Awareness, Valued Action. Has demonstrated acceptable psychometrics: Acceptable alpha reliability Construct validity when examined with internalizing symptoms and well-being. Relative brevity compared to other multidimensional PF measures. The CompACT
Dimensionality has only been assessed through exploratory factor analysis (EFA). Reliability has been largely assessed through coefficient alpha, which makes 3 untested assumptions about the items. Psychometric Limitations Stability over time of the CompACT items has not yet been formally examined.
Assess the dimensionality of the CompACT with both Confirmatory Factor Analysis (CFA) and Item Factor Analysis (IFA). Assess the model-based reliability of the CompACT. Further assess the construct validity of the CompACT by examining the association between the 3 factors and related constructs. To examine the measurement equivalence of the CompACT over the span of three months. Study Aims
Data were part of a larger longitudinal MTurk study examining coping strategies, risk factors, and psychiatric symptoms during COVID-19. Inclusion criteria included being a United States resident; fluency in English; age 18 years; and previous completion of 100 Human Intelligence Tasks (HITs) with an approval rate 95%. Several validity and attention checks were included in surveys to reduce inattentive, straight-line, or automated responding. Method: Procedure
Method: Participants Completion of all three surveys were not related to race, sex, or employment. Completers more likely to identify as Hispanic/Latino at T1 and T2. Completers more likely to identify as heterosexual at T2. N = 523 T0 (May 2020); n = 485 T1 (June 2020); n = 360 T2 (July 2020); n = 266 Recruited via Mturk 78.1% White 82.9% non-Hispanic 59.6% Male 86.5% Heterosexual 67.3% Employed full time Mean Age = 37.42 (SD = 11.46)
The 23-item CompACT (Francis et al., 2016). The 7-item AAQ-II (Bond et al., 2011) Method: Measures The 6-item Brief Resilience Scale (BRS; Smith et al., 2008) The 12-item Intolerance of Uncertainty Scale-12 (IUS-12; Carleton et al., 2007) The 10-item Kessler Psychological Distress Scale (K10; Kessler et al., 2002)
Method: Analytic Strategy Confirmatory Factor Analysis Item Factor Analysis/IRT Predict a continuous, normally distributed item response for at least one latent factor score. Predict the probability of each of K total response options using a cumulative link function and K-1 binary submodels. Parameters: item intercept, factor loading, and residual variance. Parameters: all submodels for an item share 1 factor loading, but each submodel has their own threshold. Factor means and variances set to 0 and 1 respectively to estimate all parameters. We also converted IFA parameters into IRT parameters of difficulty and discrimination. Robust full-information maximum likelihood (MLR) used to estimate all CFA models. Diagonally weighted least squares (WLSMV with a THETA parameterization that fixed item residual variances to 1) used to estimate all IFA models.
Results: Dimensionality and Scale Revision We first tested the extent to which covariances between items of the 23-item CompACT at T0 could be predicted by the 3 hypothesized factors (i.e., OE, BA, and VA) Found poor model fit by every index using both CFA and IFA. Three design issues may have contributed to model misfit: Uneven use of positive or negative wording across factors, Redundancy in content, Interpretability of item content (e.g., thoughts are just thoughts ).
CompACT Items # Openness to Experience Subscale One of my big goals is to be free from painful emotions I try to stay busy to keep thoughts or feelings from coming I tell myself I shouldn't have certain thoughts I go out of my way to avoid situations that might bring difficult thoughts, feelings, or sensations I work hard to keep out upsetting feelings I get so caught up in my thoughts that I'm unable to do the things that I most want to do. Even when something is important to me, I'll rarely do it if there's a chance it will upset me. I am willing to fully experience whatever thoughts, feelings, and sensations coming up for me, without trying to change or defend against them 13 Thoughts are just thoughts, they don't control what I do. I can take thoughts and feelings as they come, without attempting to control or avoid them Behavioral Awareness Subscale I rush through meaningful activities without being really attentive to them I find it difficult to stay focused on what's happening in the present Even when doing the things that matter to me, I find myself doing them without paying attention I do jobs or tasks automatically, without being aware of what I'm doing It seems I'm "running on automatic" without much awareness of what I'm doing Valued Action Subscale I can identify the things that really matter to me in life and pursue them I make choices based on what's important to me, even if it is stressful I behave in line with my personal values I am able to follow my long term plans times when progress is slow I can keep going with something when it's important to me I act in ways that are consistent with how I wish to live my life I undertake things that are meaningful to me, even when I find it hard to do so My values are really reflected in my behavior 2 4 8 11 15 6 18 20 22 3 9 12 16 19 1 7 10 17 23 5 14 21
CompACT Items # Openness to Experience Subscale One of my big goals is to be free from painful emotions I try to stay busy to keep thoughts or feelings from coming I tell myself I shouldn't have certain thoughts I go out of my way to avoid situations that might bring difficult thoughts, feelings, or sensations I work hard to keep out upsetting feelings I get so caught up in my thoughts that I'm unable to do the things that I most want to do. Even when something is important to me, I'll rarely do it if there's a chance it will upset me. I am willing to fully experience whatever thoughts, feelings, and sensations coming up for me, without trying to change or defend against them 13 Thoughts are just thoughts, they don't control what I do. I can take thoughts and feelings as they come, without attempting to control or avoid them Behavioral Awareness Subscale I rush through meaningful activities without being really attentive to them I find it difficult to stay focused on what's happening in the present Even when doing the things that matter to me, I find myself doing them without paying attention I do jobs or tasks automatically, without being aware of what I'm doing It seems I'm "running on automatic" without much awareness of what I'm doing Valued Action Subscale I can identify the things that really matter to me in life and pursue them I make choices based on what's important to me, even if it is stressful I behave in line with my personal values I am able to follow my long term plans times when progress is slow I can keep going with something when it's important to me I act in ways that are consistent with how I wish to live my life I undertake things that are meaningful to me, even when I find it hard to do so My values are really reflected in my behavior 2 4 8 11 15 6 18 20 22 3 9 12 16 19 1 7 10 17 23 5 14 21
CompACT Items # Openness to Experience Subscale One of my big goals is to be free from painful emotions I try to stay busy to keep thoughts or feelings from coming I tell myself I shouldn't have certain thoughts I go out of my way to avoid situations that might bring difficult thoughts, feelings, or sensations I work hard to keep out upsetting feelings I get so caught up in my thoughts that I'm unable to do the things that I most want to do. Even when something is important to me, I'll rarely do it if there's a chance it will upset me. I am willing to fully experience whatever thoughts, feelings, and sensations coming up for me, without trying to change or defend against them 13 Thoughts are just thoughts, they don't control what I do. I can take thoughts and feelings as they come, without attempting to control or avoid them Behavioral Awareness Subscale I rush through meaningful activities without being really attentive to them I find it difficult to stay focused on what's happening in the present Even when doing the things that matter to me, I find myself doing them without paying attention I do jobs or tasks automatically, without being aware of what I'm doing It seems I'm "running on automatic" without much awareness of what I'm doing Valued Action Subscale I can identify the things that really matter to me in life and pursue them I make choices based on what's important to me, even if it is stressful I behave in line with my personal values I am able to follow my long term plans times when progress is slow I can keep going with something when it's important to me I act in ways that are consistent with how I wish to live my life I undertake things that are meaningful to me, even when I find it hard to do so My values are really reflected in my behavior 2 4 8 11 15 6 18 20 22 3 9 12 16 19 1 7 10 17 23 5 14 21
Results: CompACT-15 Dimensionality The CompACT-15 demonstrated good fit, without cross-loadings or residual covariances by most fit indices using both CFA and IFA. Standardized loadings ranged from .66 to .89. OE and BA factors were strongly related (? .7), but VA was less related to OE (? .1) or BA (? .4).
Results: Model-based Reliability in Three Ways 1. Omega reliability for each dimension: .850 for OE .929 for BA .851 for VA 2. Factor reliability as the proportion of true trait variance for each factor score over the total variance: .886 for OE .940 for BA .878 for VA 3. Item information/(Item information +1)
Results: Measurement Stability Over Time Used longitudinal measurement invariance procedures in which unstandardized factor loadings, intercepts or thresholds, and residual variances were constrained equal over time in successive models. Resulting decreases in global fit were examined using nested model comparisons. Majority of item parameters functioned equivalently across three occasions, separated by 1-month intervals. This is to be expected in a non-treatment setting in which minimal true change or reactivity is likely.
Latent Factors 1 2 3 4 5 6 7 8 9 1. Openness to Experience (CompACT-15) 2. Behavioral Awareness (CompACT-15) 3. Valued Action (CompACT-15) 4. Brief Resilience Scale Positively Worded Items (BRS) 5. Brief Resilience Scale Negatively Worded Items (BRS) 6. Acceptance and Action Questionnaire-II (AAQ-II) 7. Prospective Intolerance of Uncertainty (IUS-12) 8. Inhibitory Intolerance of Uncertainty (IUS-12) 9. Kessler Psychological Distress Scale (K 10) .72 .45 .23 .69 -.81 -.34 -.67 -.77 .12 .35 .63 .43 -.39 -.14 -.44 -.39 .25 .16 .60 .70 -.37 -.49 -.52 -.35 .67 .66 .37 .67 -.77 -.58 -.78 -.74 -.73 -.78 -.32 -.34 -.74 .49 .76 .95 -.49 -.32 -.13 -.52 -.58 .45 .81 .52 -.67 -.64 -.40 -.51 -.76 .73 .80 .76 -.65 -.75 -.36 -.33 -.71 .93 .48 .71 .74 .16 .25 .68 -.76 -.50 -.68 -.69 Results: Construct Validity All CFA correlations were significant at the p < .001 level except between Prospective Anxiety and Valued Action (p = .033), between positively-worded BRS items and Behavioral Awareness (p = .003), and between Openness to Experience and Valued Action (p = .042). All IFA correlations were significant at the p < .001 level.
Latent Factors 1 2 3 4 5 6 7 8 9 1. Openness to Experience (CompACT-15) 2. Behavioral Awareness (CompACT-15) 3. Valued Action (CompACT-15) 4. Brief Resilience Scale Positively Worded Items (BRS) 5. Brief Resilience Scale Negatively Worded Items (BRS) 6. Acceptance and Action Questionnaire-II (AAQ-II) 7. Prospective Intolerance of Uncertainty (IUS-12) 8. Inhibitory Intolerance of Uncertainty (IUS-12) 9. Kessler Psychological Distress Scale (K 10) .72 .45 .23 .69 -.81 -.34 -.67 -.77 .12 .35 .63 .43 -.39 -.14 -.44 -.39 .25 .16 .60 .70 -.37 -.49 -.52 -.35 .67 .66 .37 .67 -.77 -.58 -.78 -.74 -.73 -.78 -.32 -.34 -.74 .49 .76 .95 -.49 -.32 -.13 -.52 -.58 .45 .81 .52 -.67 -.64 -.40 -.51 -.76 .73 .80 .76 -.65 -.75 -.36 -.33 -.71 .93 .48 .71 .74 .16 .25 .68 -.76 -.50 -.68 -.69 Results: Construct Validity All CFA correlations were significant at the p < .001 level except between Prospective Anxiety and Valued Action (p = .033), between positively-worded BRS items and Behavioral Awareness (p = .003), and between Openness to Experience and Valued Action (p = .042). All IFA correlations were significant at the p < .001 level.
Latent Factors 1 2 3 4 5 6 7 8 9 1. Openness to Experience (CompACT-15) 2. Behavioral Awareness (CompACT-15) 3. Valued Action (CompACT-15) 4. Brief Resilience Scale Positively Worded Items (BRS) 5. Brief Resilience Scale Negatively Worded Items (BRS) 6. Acceptance and Action Questionnaire-II (AAQ-II) 7. Prospective Intolerance of Uncertainty (IUS-12) 8. Inhibitory Intolerance of Uncertainty (IUS-12) 9. Kessler Psychological Distress Scale (K 10) .72 .45 .23 .69 -.81 -.34 -.67 -.77 .12 .35 .63 .43 -.39 -.14 -.44 -.39 .25 .16 .60 .70 -.37 -.49 -.52 -.35 .67 .66 .37 .67 -.77 -.58 -.78 -.74 -.73 -.78 -.32 -.34 -.74 .49 .76 .95 -.49 -.32 -.13 -.52 -.58 .45 .81 .52 -.67 -.64 -.40 -.51 -.76 .73 .80 .76 -.65 -.75 -.36 -.33 -.71 .93 .48 .71 .74 .16 .25 .68 -.76 -.50 -.68 -.69 Results: Construct Validity All CFA correlations were significant at the p < .001 level except between Prospective Anxiety and Valued Action (p = .033), between positively-worded BRS items and Behavioral Awareness (p = .003), and between Openness to Experience and Valued Action (p = .042). All IFA correlations were significant at the p < .001 level.
In Sum We propose a new short-form of the CompACT, the CompACT-15, which demonstrates good model fit of a correlated 3 factor model with simple structure and adequate reliability. Majority of CompACT-15 items demonstrated measurement stability across three occasions in a two-month period. CompACT-15 dimensions also show expected patterns of correlation to support construct validity.
Study Limitations Recruitment of respondents using Mturk, which has been subject to scrutiny due to increased activity from bots and farmers. Followed recommendations for best practice to safeguard data quality (Chemielewski & Kucker, 2020). Reported influx of new MTurk respondents during the COVID-19 pandemic who are less cognitively reflective, White, Democratic, and experienced compared to respondents prior to the pandemic (Arechar & Rand, 2021). Unclear what effects this more diverse but less attentive subject pool may have on data quality.
Study Strengths and Future Directions Our use of two different types of confirmatory measurement models (CFA and IFA/IRT). The brevity of the CompACT-15 shorter instrument with better model fit and good reliability. Highlight the importance of wording direction in items which measure PF PF is not just the inverse of psychological inflexibility. Future examination of different kinds of measurement equivalence needed.
Latent Factors 1 2 3 4 5 6 7 8 9 1. Openness to Experience (CompACT-15) 2. Behavioral Awareness (CompACT-15) 3. Valued Action (CompACT-15) 4. Brief Resilience Scale Positively Worded Items (BRS) 5. Brief Resilience Scale Negatively Worded Items (BRS) 6. Acceptance and Action Questionnaire-II (AAQ-II) 7. Prospective Intolerance of Uncertainty (IUS-12) 8. Inhibitory Intolerance of Uncertainty (IUS-12) 9. Kessler Psychological Distress Scale (K 10) .72 .45 .23 .69 -.81 -.34 -.67 -.77 .12 .35 .63 .43 -.39 -.14 -.44 -.39 .25 .16 .60 .70 -.37 -.49 -.52 -.35 .67 .66 .37 .67 -.77 -.58 -.78 -.74 -.73 -.78 -.32 -.34 -.74 .49 .76 .95 -.49 -.32 -.13 -.52 -.58 .45 .81 .52 -.67 -.64 -.40 -.51 -.76 .73 .80 .76 -.65 -.75 -.36 -.33 -.71 .93 .48 .71 .74 .16 .25 .68 -.76 -.50 -.68 -.69
Study Strengths and Future Directions Our use of two different types of confirmatory measurement models (CFA and IFA/IRT). The brevity of the CompACT-15 shorter instrument with better model fit and good reliability. Highlight the importance of wording direction in items which measure PF PF is not just the inverse of psychological inflexibility. Future examination of different kinds of measurement equivalence needed.
Acknowledgements Dr. Lesa Hoffman and Dr. Emily Thomas The creators of the CompACT, Dr. Ashley Francis, Dr. David Dawson, and Dr. Nima Golijani-Moghaddam The THRIVE Lab, especially Dr. Anne Roche, Jenna Adamowicz, M.A., and Manny Stegall, B.S. University Iowa Graduate College University of Iowa Graduate and Professional Student Government
Thank you for listening! Any questions? You can also reach me at: ti-hsu@uiowa.edu, @TiHsu1
More on the AAQ-II Wording directly assesses EA or PiF, not PF. Evidence that PF and PiF may not lie on the same continuum. Positive construct should not be measured through the absence of a negative construct. AAQ-II items measure PiF as a unidimensional construct, but PF is in theory multidimensional.
Study Strengths II Our results also highlight the importance of wording direction in items which measure PF. We show that negatively-keyed items do not function as the unidimensional opposite of positively keyed items, since we observed lower correlations with the positively worded VA items with the negatively worded OE and BA items. We also observed larger correlations among CompACT-15 factors and those measured by similarly worded items. PiF is not the inverse of PF!
Future Directions Examine Examine Examine Examine the measurement equivalence of the CompACT-15 in individuals receiving psychotherapy and those who are not. Examine the measurement equivalence of the CompACT-15 across longer (or shorter!) temporal contexts. Examine the measurement equivalence of the CompACT-15 across different cultural groups.
