Disproportionate Impact Analysis: Understanding PPG Method

Disproportionate Impact Analysis (2020-21) Damien Hoffman Research Analyst, Oxnard College, Office of Institutional Effectiveness

Definition and Background When one subgroup of students attains an outcome such as degree completion at a rate that is substantially lower than the benchmark rate, that subgroup may be referred to as disproportionately impacted (CCCCO, 2017). Differences in rates between subgroups may suggest that one group has less access to support services and/or is in need of relatively greater student support services to attain outcomes at rates comparable to other groups (CCCCO, 2017). Information can be used variety of purposes on college campuses In this case, the information is used to address equity gaps in student equity plans Requirements for SSSP funding

Percentage Point Gap Index There are three different statistical indices used to assess disproportionate impact: 80% Index, Proportionality Index, and Percentage Point Gap Index. CCCCO strongly recommends to use the Percentage Point Gap Index (i.e. PPG) in DI analyses.

The rate for the reference group on an outcome is then subtracted from the rate for the subgroup on the same outcome to get a PPG value Reference group rate could be overall rate, rate for all other groups not including the group of interest ( all but current ), or highest performing group rate We will be using all but current rate for the reference group for all analyses Choice of reference group makes a huge difference is whether a subgroup gets flagged for DI, so choose wisely

PPG value is compared to its margin of error, which leads to flagging for disproportionate impact. Below is a diagram of the decision rules.

Margin of Error The detection of disproportionate impact in the PPG method uses a margin of error (E) that is adjusted by the sample size (n) or cohort size of the subgroup. The standard margin of error is 3% if the sample size of the subgroup is at least 800 (n 800). The margin of error decreases as the sample size increases. Margin of error can be calculated by hand or looked up in a table MOE is also very important is determining whether a subgroup gets flagged for disproportionate impact

Disadvantages of PPG Becomes unstable when using small subgroup samples Analyses with subgroup sample sizes <30 are not recommended even when adjusting the MOE Lots of decision points on margin of error, reference group, denominator choices Sometimes can obscure results when larger group rates are compared with small group rates

Education Code The Education Code requires, at a minimum, for colleges to disaggregate by gender and each of the following groups when performing equity analyses: disabled, current/former foster youth, low-income, LGBTQ, ethnicity and veterans. We will be doing three different disaggregated analyses for three of the Vision for Success goals (Successful Enrollment, Transfer-Level Math + English Completion in 1stYear, and Fall-to-Spring Retention) using the following disaggregations: 1. Gender separately 2. Ethnicity separately, and 3. Gender and Ethnicity together

Before we dive into the data summaries, the following groups were flagged for adverse disproportionate impact on the following metrics: 1. For Successful Enrollment: Whites in general, Unreported in general, Asian males and females, White males, White females, Unreported males, Unreported females 2. For Fall-Spring Retention: Asian students in general, whites in general, Asian males, white males, white females 3. For Transfer-Level Math and English Completion: males in general, Asians in general,

Successful Enrollment at the Same Community College (By Gender) cohort 2020-2021Female 2020-2021Male 2020-2021Other/Unknown167 group n success pct reference reference_group moe pct_lo pct_hi di_indicator 10889 7598 3335 0.306272 0.302898 all but current 0.03 0.276272 0.336272 2269 0.298631 0.309153 all but current 0.03 0.268631 0.328631 83 0.497006 0.303132 all but current 0 0 0 0.075835 0.421171 0.572841

By Ethnicity group American Indian/Alaskan Native Asian 3376 Black or African American Filipino Hispanic 6285 Pacific Islander Two or More Ethnicities Unreported White 6910 n success pct reference reference_group moe 0.035856 -0.01845 0.053259 pct_lo pct_hi di_indicator 747 13 0.017403 0.316859 all but current 221 0.065462 0.357769 all but current 0.03 0.035462 0.095462 126 0.490272 0.302278 all but current 140 1 0.299611 all but current 4088 0.650438 0.129275 all but current 0.03 0.620438 0.680438 19 0.256757 0.305059 all but current 150 0.581395 0.300989 all but current 54 0.088962 0.312129 all but current 876 0.126773 0.409656 all but current 0.03 0.096773 0.156773 1 1 0 0 0 0 0 1 1 257 140 0.061131 0.429142 0.551403 0.082825 0.917175 1.082825 74 258 607 0.113923 0.142834 0.370679 0.061012 0.520383 0.642407 0.039777 0.049185 0.128739

By Gender + Ethnicity disaggregationgroup Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender Ethnicity + Gender n success pct ppg_reference ppg_reference_group moe pct_lo pct_hi di_indicator_ppg American Indian/Alaskan Native, Female 204 American Indian/Alaskan Native, Male 543 Asian , Female 2131 Asian , Male 1232 Asian , Other/Unknown 13 Black or African American , Female 121 Black or African American , Male 128 Black or African American , Other/Unknown 8 Filipino , Female 71 Filipino , Male 66 Filipino , Other/Unknown 3 Hispanic , Female 3762 Hispanic , Male 2452 Hispanic , Other/Unknown 71 Pacific Islander , Female 40 Pacific Islander , Male 34 Two or More Ethnicities , Female 127 Two or More Ethnicities , Male 125 Two or More Ethnicities , Other/Unknown 6 Unreported , Female 282 Unreported , Male 305 Unreported , Other/Unknown 20 White , Female 4151 White , Male 2713 White , Other/Unknown 46 1 0.004902 0.308184 all but current 12 0.022099 0.313345 all but current 127 0.059596 0.336501 all but current 0.03 0.029596 0.089596 88 0.071429 0.321375 all but current 0.03 0.041429 0.101429 6 0.461538 0.304758 all but current 61 0.504132 0.303567 all but current 59 0.460938 0.303789 all but current 6 0.75 0.304677 all but current 71 1 0.302212 all but current 66 1 0.302399 all but current 0.12063 3 1 0.304756 all but current 2507 0.666401 0.213537 all but current 0.03 0.636401 0.696401 1542 0.628874 0.255833 all but current 0.03 0.598874 0.658874 39 0.549296 0.303934 all but current 12 0.3 0.304878 all but current 7 0.205882 0.305048 all but current 66 0.519685 0.303395 all but current 80 0.64 0.302607 all but current 4 0.666667 0.304751 all but current 23 0.08156 0.308295 all but current 21 0.068852 0.308791 all but current 10 0.5 0.304658 all but current 467 0.112503 0.359926 all but current 0.03 0.082503 0.142503 394 0.145227 0.332037 all but current 0.03 0.115227 0.175227 15 0.326087 0.304815 all but current 0.068614 -0.06371 0.073516 0.042056 -0.01996 0.064155 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 1 0 0.271803 0.189735 0.733342 0.089091 0.415041 0.593223 0.086621 0.374317 0.547558 0.346482 0.403518 1.096482 0.116305 0.883695 1.116305 0.87937 0.565803 0.434197 1.565803 1.12063 0.116305 0.432991 0.154952 0.145048 0.454952 0.168069 0.037814 0.373951 0.086961 0.432724 0.606646 0.087654 0.552346 0.727654 0.400083 0.266583 0.058358 0.023202 0.139918 0.056115 0.012738 0.124967 0.219135 0.280865 0.719135 0.6656 1.06675 0.144493 0.181594 0.47058

Fall to Spring Retention (By Gender) disaggregation Gender Gender Gender #NAME? #NAME? group F M N n success ppg_reference 0.61728 all but current 0.03 0.601148 0.661148 1831 0.617538 0.630707 all but current 0.03 0.587538 0.647538 48 0.607595 0.625474 all but current 4497 0.625278 NA all but current 0.03 0.595278 0.655278 pct ppg_reference_group moe pct_lo pct_hi di_indicator_ppg 4148 2965 79 7192 2618 0.631148 0 0 0 0 0.110259 0.497336 0.717854

By Ethnicity disagg IPEDS IPEDS IPEDS IPEDS IPEDS IPEDS IPEDS IPEDS IPEDS #NAME? group American Indian/Alaskan Native Asian Black or African American Filipino Hispanic Pacific Islander Two or More Ethnicities 183 Unreported White #NAME? n success pct ppg_reference ppg_reference_group moe pct_lo pct_hi di_indicator_ppg 13 195 143 217 5283 13 5 0.384615 0.625714 all but current 101 0.517949 0.628269 all but current 78 0.545455 0.626897 all but current 127 0.585253 0.626523 all but current 3472 0.657202 0.53693 all but current 0.03 0.627202 0.687202 8 0.615385 0.625296 all but current 102 0.557377 0.627051 all but current 75 0.681818 0.6244 all but current 529 0.511111 0.64447 all but current 4497 0.625278 NA all but current 0.03 0.595278 0.655278 0.271803 0.112812 0.656418 0.070179 0.447769 0.588128 0.081952 0.463503 0.627406 0.066527 0.518727 0 1 0 0 0 0 0 0 1 0 0.65178 0.271803 0.343582 0.887188 0.072444 0.484933 0.629821 0.093439 0.588379 0.775258 0.030462 0.480649 0.541573 110 1035 7192

By Ethnicity + Gender disagg IPEDS + Gender American Indian/Alaskan Native, F IPEDS + Gender American Indian/Alaskan Native, M IPEDS + Gender Asian, F IPEDS + Gender Asian, M IPEDS + Gender Asian, N IPEDS + Gender Black or African American, F IPEDS + Gender Black or African American, M IPEDS + Gender Black or African American, N IPEDS + Gender Filipino, F IPEDS + Gender Filipino, M IPEDS + Gender Filipino, N IPEDS + Gender Hispanic, F IPEDS + Gender Hispanic, M IPEDS + Gender Hispanic, N IPEDS + Gender Pacific Islander, F IPEDS + Gender Pacific Islander, M IPEDS + Gender Two or More Ethnicities, F IPEDS + Gender Two or More Ethnicities, M IPEDS + Gender Two or More Ethnicities, N IPEDS + Gender Unreported, F IPEDS + Gender Unreported, M IPEDS + Gender Unreported, N IPEDS + Gender White, F IPEDS + Gender White, M IPEDS + Gender White, N #NAME? group n success ppg_reference 0.6 0.625296 all but current 0.25 0.625696 all but current 57 0.537736 0.626588 all but current 43 0.494253 0.626882 all but current 1 0.5 0.625313 all but current 41 0.569444 0.625843 all but current 34 0.5 0.626474 all but current 3 1 0.625122 all but current 67 0.54918 0.626591 all but current 59 0.634409 0.625158 all but current 1 0.5 0.625313 all but current 2071 0.662084 0.596949 all but current 0.03 0.632084 0.692084 1372 0.650854 0.614673 all but current 0.03 0.620854 0.680854 29 0.617021 0.625332 all but current 4 0.571429 0.625331 all but current 4 0.666667 0.625244 all but current 44 0.564103 0.625949 all but current 57 0.57 0.626058 all but current 0.098 1 0.2 0.625574 all but current 34 0.693878 0.624808 all but current 0.14 0.553878 0.833878 35 0.636364 0.625193 all but current 6 1 0.624965 all but current 297 0.511188 0.635305 all but current 225 0.511364 0.632701 all but current 0.04672 0.464644 0.558083 7 0.5 0.625522 all but current 4497 0.625278 NA all but current 0.03 0.595278 0.655278 pct ppg_reference_group moe pct_lo pct_hi di_indicator_ppg 5 8 3 2 0.438269 0.161731 1.038269 0.346482 -0.09648 0.596482 0.095186 0.44255 0.632922 0.105067 0.389186 0.692965 -0.19296 1.192965 0.115494 0.45395 0.684939 0.118842 0.381158 0.618842 0.565803 0.434197 1.565803 0.088725 0.460455 0.637905 0.101621 0.532787 0.692965 -0.19296 1.192965 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 106 87 0.59932 2 72 68 3 122 93 0.73603 2 3128 2108 47 0.142948 0.474074 0.759969 0.370405 0.201023 0.941834 0.400083 0.266583 0.110963 0.453139 0.675066 0.472 0.438269 -0.23827 0.638269 7 6 1.06675 78 100 0.668 5 49 55 6 581 440 14 7192 0.132143 0.400083 0.599917 1.400083 0.040657 0.47053 0.551845 0.50422 0.768507 0.261916 0.238084 0.761916 #NAME?

Transfer-Level Math + English Completion in 1stYear, By Gender disaggregation Gender Gender Gender #NAME? #NAME? group F M N n success pct ppg_reference ppg_reference_group moe pct_lo pct_hi di_indicator_ppg 657 592 12 1261 312 0.474886 0.428808 all but current 254 0.429054 0.473842 all but current 5 0.416667 0.453163 all but current 571 0.452815 NA 0.038233 0.436652 0.513119 0.040278 0.388776 0.469332 0.282902 0.133765 0.699568 all but current 0.03 0.422815 0.482815 0 1 0 0

By Ethnicity disagg IPEDS IPEDS IPEDS IPEDS IPEDS IPEDS IPEDS IPEDS #NAME? group Asian Black or African American Filipino Hispanic Pacific Islander Two or More Unreported White #NAME? n success pct ppg_reference 0.44878 all but current ppg_reference_group moe pct_lo 0.43689 0.788916 0.11948 0.547187 0.224827 0.248857 0.698512 0.031162 0.412721 0.475045 0.692965 -0.19296 1.192965 0.170596 0.283949 0.625142 0.253035 0.413632 0.919702 0.079751 0.390447 all but current 0.03 0.422815 0.482815 pct_hi di_indicator_ppg 31 21 19 989 19 0.612903 7 0.333333 0.454839 all but current 9 0.473684 0.452496 all but current 439 0.443883 0.485294 all but current 1 0.5 0.45274 all but current 15 0.454545 0.452769 all but current 10 0.666667 0.450241 all but current 71 0.470199 0.45045 all but current 571 0.452815 NA 0.176013 0.213854 0 0 0 1 0 0 0 0 0 2 33 15 151 1261 0.54995

By Ethnicity + Gender disaggregation IPEDS + Gender Asian, F IPEDS + Gender Asian, M IPEDS + Gender Asian, N IPEDS + Gender Black or African American, F IPEDS + Gender Black or African American, M IPEDS + Gender Filipino, F IPEDS + Gender Filipino, M IPEDS + Gender Hispanic, F IPEDS + Gender Hispanic, M IPEDS + Gender Hispanic, N IPEDS + Gender Pacific Islander, F IPEDS + Gender Two or More Ethnicities, F IPEDS + Gender Two or More Ethnicities, M IPEDS + Gender Two or More Ethnicities, N IPEDS + Gender Unreported, F IPEDS + Gender Unreported, M IPEDS + Gender Unreported, N IPEDS + Gender White, F IPEDS + Gender White, M IPEDS + Gender White, N #NAME? group n success ppg_reference 0.449 all but current 0.238 0.452 all but current 0.272 0.453 all but current 0.980 0.454 all but current 0.370 0.454 all but current 0.262 0.451 all but current 0.327 0.454 all but current 0.310 0.448 all but current 0.043 0.469 all but current 0.046 0.452 all but current 0.438 0.453 all but current 0.693 0.452 all but current 0.253 0.454 all but current 0.238 0.452 all but current 0.980 0.452 all but current 0.566 0.451 all but current 0.295 0.452 all but current 0.980 0.449 all but current 0.115 0.452 all but current 0.114 0.454 all but current 0.490 571 0.453 NA pct ppg_reference_group moe di_flag 17 13 1 7 14 9 10 531 453 12 0.706 7 0.538 0 0.000 2 0.286 5 0.357 6 0.667 3 0.300 244 0.460 192 0.424 3 0.600 1 0.500 8 0.533 6 0.353 1 1.000 2 0.667 7 0.636 1 1.000 37 0.507 34 0.459 0 0.000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 2 15 17 1 3 11 1 73 74 4 #NAME? 1261 all but current 0.030

Conclusions Successful Enrollment groups tagged for DI (on basis of gender, ethnicity, or both) Whites Asians Unreported Asian males and Asian females White males and White females Unreported males and Unreported females Fall-to-Spring Retention groups tagged for DI Asians Whites Asian males, White males and White females Transfer-Level Math and English Completion groups tagged for DI Males Hispanics*

Conclusions In addition to DI flagging, one could also calculate the number of students needed to close the equity gap Then look up those students individually to see why their rates might be a lot lower in that metric Good starting point, but much more work needs to be done to look at every single disaggregation for each of the five metrics No one is certain when directions from the state will be out about which disaggregations we should be reporting and whether or not we re going to be using data sets provided by them for examining DI If we are using their data, data reported here will be different to some extent, due to differing data sources and denominators for rates

References California Community Colleges Chancellor s Office (2017). Percentage point gap method. Retrieved from http://extranet.cccco.edu/Portals/1/TRIS/Research/Analysis/PercentagePoi ntGapMethod20 17.pdf Sosa, G. (2017). Using Disproportionate Impact Methods to Identify Equity Gaps. Retrieved from: https://www.sdmesa.edu/about-mesa/institutional- effectiveness/institutional-research/data-warehouse/data- reports/Equity%20Calculations%20Explained.pdf