Understanding SEM Concepts - Measurement Reliability and Model Evaluation

Understanding Basic SEM Concepts

Understanding Basic Concepts Factor Loadings How well an item measures the underlying construct? It shows the representation of the item in the construct How will I know, if the representation is adequate? The loading value should preferably be over .70 What if the values are less than .70? Indicator Reliability (Square of Loadings) R-Square Change in the Endogenous (Dependent) variable explained by the exogenous variables. 0.615 61.5% change in the dependent variable PQ is being explained by ER, RDR, and PR. Beta (Path) Coefficient Weight of impact/How much an impact is there of one variable on another. How strongly one variable influences another variable. Normally, a value over .20 would mean a significant impact of IV on DV in a relationship. No impact with negative impact This is not right, they are not equal. If there is a negative sign, this means that there is obviously an impact but in an opposite direction.

Understanding Basic Concepts Construct Reliability Reliability means consistency If we go back and collect the data from the same respondents under similar conditions, if I get similar results, this means that my measure was reliable. How would I know statistically, if my measure was reliable? The Traditional Measure Cronbach s Alpha (A more Conservative Approach) The Modern Measure Composite Reliability (A more Liberal Approach) With in these two is Rho_a (Something in the Middle/between the two) Recommended value is >= .70

Evaluating Measurement Model Construct Reliability

Indicator Reliability The first step in reflective measurement model assessment involves examining how much of each indicator s variance is explained by its construct, which is indicative of indicator reliability. To compute an indicator s explained variance, we need to square the indicator loading, which is the bivariate correlation between indicator and construct. As such, the indicator reliability indicates the communality of an indicator. Indicator loadings above 0.708 are recommended, since they indicate that the construct explains more than 50 percent of the indicator s variance, thus providing acceptable indicator reliability.

Improving Indicator Reliability Researchers frequently obtain weaker indicator loadings (< 0.708) for their measurement models in social science studies, especially when newly developed scales are used (Hulland, 1999). Rather than automatically eliminating indicators when their loading is below 0.70, researchers should carefully examine the effects of indicator removal on other reliability and validity measures. Generally, indicators with loadings between 0.40 and 0.708 should be considered for removal only when deleting the indicator leads to an increase in the internal consistency reliability or convergent validity (discussed in the next sections) above the suggested threshold value. Another consideration in the decision of whether to delete an indicator is the extent to which its removal affects content validity, which refers to the extent to which a measure represents all facets of a given construct. As a consequence, indicators with weaker loadings are sometimes retained. Indicators with very low loadings (below 0.40) should, however, always be eliminated from the measurement model (Hair, Hult, Ringle, & Sarstedt, 2022).

Assessing Indicator Reliability We need to estimate the relationships between the reflectively measured constructs and their indicators (i.e., loadings). All indicator loadings of the reflectively measured constructs are well above the threshold value of 0.708 (Hair, Risher, Sarstedt, & Ringle, 2019), which suggests sufficient levels of indicator reliability. Variance is explained by taking Square of the loadings, indicating indicator reliability if its is over .50

Internal Consistency Reliability The second step in reflective measurement model assessment involves examining internal consistency reliability. One of the primary measures used in PLS-SEM is J reskog s (1971) composite reliability rhoc. Higher values indicate higher levels of reliability. Values over .70 are normally considered reliable. Reliability values between 0.60 and 0.70 are considered acceptable in exploratory research, whereas values between 0.70 and 0.90 range from satisfactory to good. Values above 0.90 (and definitely above 0.95) are problematic, since they indicate that the indicators are redundant, thereby reducing construct validity (Diamantopoulos, Sarstedt, Fuchs, Wilczynski, & Kaiser, 2012). Reliability values of 0.95 and above also suggest the possibility of undesirable response patterns (e.g., straight-lining), thereby triggering inflated correlations among the error terms of the indicators.

Internal Consistency Reliability Cronbach s alpha is another measure of internal consistency reliability, which assumes the same thresholds as the composite reliability (rhoc). A major limitation of Cronbach s alpha, however, is that it assumes all indicator loadings are the same in the population (also referred to as tau-equivalence). The violation of this assumption manifests itself in lower reliability values than those produced by rhoc. While Cronbach s alpha is rather conservative, the composite reliability rhoc may be too liberal, and the construct s true reliability is typically viewed as within these two extreme values. As an alternative and building on Dijkstra (2010), subsequent research has proposed the exact (or consistent) reliability coefficient rhoA (Dijkstra, 2014; Dijkstra & Henseler, 2015). The reliability coefficient rhoA usually lies between the conservative Cronbach s alpha and the liberal composite reliability and is therefore considered and acceptable compromise between these two measures.

Reliability in SmartPLS4 The internal consistency reliability values are displayed in a matrix format. With rhoA values in between Alpha and Composite Reliability. If the results for Cronbach s alpha and Composite Reliability are above the 0.70 threshold (Hair et al., 2019), indicating that all construct measures are reliable.

Reference Hair Jr, J. F., Hult, G. T. M., Ringle, C. M., Sarstedt, M., Danks, N. P., & Ray, S. (2021). Partial Least Squares Structural Equation Modeling (PLS-SEM) Using R: A Workbook.

Construct Validity

Concept of Construct Validity Validity Accuracy How accurately the items/constructs are measured? Is the construct measuring what you intend it to measure? Is the construct measuring what it was intended to measure? Statistical validity (Construct validity) Content validity/face validity (through existing research/experts) Construct Validity Convergent Validity Established through Average Variance Extracted (AVE) How much variance is extracted by the latent construct based on the indicators. The items are converging/coming together to represent the underlying construct. Convergent Validity is established if the AVE value is greater than .50 Discriminant Validity Discriminant validity help establish the distinctiveness of the constructs Whether or not the constructs have their own individual identity. The Three Methods of Discriminant Validity Fornell & Larcker Criterion (1981) Heterotrait-Monotrait Ratio Less than 0.85 (Conservative) or a more liberal approach is < 0.90 Cross-Loadings

Issues in Convergent Validity Check for Items Loadings First, only delete items with loadings less than 0.40 Next, if this doesn t solve the issues, remove items that are less than 0.50. If the AVE and CR values are found adequate after removal of items less than 0.40 than items with loadings between 0.40 0.70 may not be removed. Items with loadings between 0.40 0.70 shall only be removed if they improve the Composite Reliability and Average Variance Extracted values from below the threshold level to the required level.

Discriminant Validity

Discriminant Validity - SmartPLS Discriminant Validity is utilized to establish distinctiveness of the different constructs in the study. Is the extent to which a construct is empirically distinct from other constructs in the model. In order to establish discriminant validity, following methods may be utilized Fornell & Larcker Criterion Fornell and Larcker (1981) proposed the traditional metric and suggested that each construct s Square root of AVE (Variance within) should be compared to the inter-construct correlation (as a measure of shared variance between constructs) of that same construct and all other reflectively measured constructs in the structural model the shared variance between all model constructs should not be larger than their AVEs (Square root). Recent research indicates, however, that this metric is not suitable for discriminant validity assessment. Heterotrait-Monotrait Ratio Heterotrait monotrait ratio (HTMT) Is a measure of discriminant validity. The HTMT is the mean of all correlations of indicators across constructs measuring different constructs (i.e., the heterotrait heteromethod correlations) relative to the (geometric) mean of the average correlations of indicators measuring the same construct (i.e., the monotrait heteromethod correlations). Cross Loadings An item in a construct shall load substantially well onto its own construct instead of the other constructs in the study.

Issues in Discriminant Validity

Issues in Discriminant Validity - Before collecting the data Make sure to have adequate No. of Items in each scale/construct. There should be at least 4-6 items, since, in SEM items are deleted if they fail to load or due to cross-loading. Make sure that the Items/Statements are easy to understand. Make sure there is no overlap in the statements of different constructs.

Solve Discriminant Validity Issues Make sure to clean the data. Check for Standard Deviation in Responses. Standard Deviation value of Less than 0.25 mean that there is probability of respondent misconduct and the responses for the questions are almost similar. Assess cross-loading, if an item is loading well onto another construct instead of its own construct, that particular items shall be removed. For example, an item has loading over 0.70 on two constructs Check for Cross-loading, if an item is cross-loading, and the difference is less than .10, REMOVE the item(s). Check for low loadings, remove items that have low loadings (start with < 0.40). If the problem still persists, try collecting more data Combine the constructs that have lack of discriminant validity as they are measuring the same concept (This is appropriate for dimensions of the same construct). If problems still persist, dropping one (or more) independent variables (i.e., collinear variables that demonstrate insufficient discriminant validity) from the model may also help.

Reference Farrell, A. M. (2010). Insufficient discriminant validity: A comment on Bove, Pervan, Beatty, and Shiu (2009). Journal of business research, 63(3), 324- 327.