Achieving School Improvement Through Demonstrable Growth

Demonstrable Improvement for Schools in Receivership January 9, 2019 Ira Schwartz Associate Commissioner, Office of Accountability

Goals for This Presentation 1. Provide overview of Demonstrable Improvement (DI) Process. 2. Describe selection of Demonstrable Improvement indicators. 3. Explain DI Process for current and newly identified Receivership schools. 2

1. Demonstrable Improvement Process Overview

Demonstrable Improvement Process All schools in newly identified for Receivership are designated as Struggling. No new Persistently Struggling schools. Struggling Schools must annually make Demonstrable Improvement (DI) after a two-year period, and annually thereafter, or they will be placed in Independent Receivership. Schools that make Demonstrable Improvement continue under Superintendent Receivership. In deciding whether Demonstrable Improvement has been made, Commissioner shall consider: Performance on Metrics (Indicators). Number of Years Schools have been Identified. Superintendent s Successful use of Receivership Powers to Implement the School s Plan. 4

What is Demonstrable Improvement? For schools operating under Superintendent Receivership: Demonstrable Improvement (DI) is the process used to determine whether an Independent Receiver shall be appointed to a school in Superintendent Receivership. DI does not determine whether a school is removed from Receivership. Removal from Receivership occurs at the end of the school year in which a school under Superintendent Receivership is removed from Comprehensive Support and Improvement status. 5

DI Methodology: Guiding Principles One or more indicator(s) have been established for each Metric specified in legislation. For each indicator, a school can make progress by achieving a school specific Progress Target. Progress Targets increase over the three-year period. Most indicators are based on student performance these are classified as Tier 1 indicators. Other indicators are based on implementation of programs and/or processes these are classified as Tier 2 indicators. 6

DI Methodology: Guiding Principles The State Education Department will select and assign some of the school indicators, and the School Receiver in consultation with the Community Engagement Team shall select some. Selected indicators will be based primarily on where the school s performance needs the most improvement. A Superintendent Receiver may seek to have local measures approved by the Commissioner. Schools should avoid selecting indicators that measure the same performance as another indicator or indicators that only apply to a small percentage of school s students. The result of the process shall be a judgment that a) the school made Demonstrable Improvement, b) the school did not make Demonstrable Improvement unless there are shown to be extenuating or extraordinary circumstances, or c) the Commissioner shall review the totality of the school's record to make a determination as to whether or not the school made Demonstrable Improvement. 7

DI Methodology: Revision The existing DI methodology has been utilized from the 2015-16 school through the 2017-18 school year. Beginning with the 2018-19 school year, NYSED has revised the methodology to a) align the DI indicators with the ESSA indicators, b) update and streamline the annual Progress Targets, and c) differentiate the targets for current vs. new Receivership schools. Implementation of the revised methodology is differentiated for schools that are continuing in Receivership, referred to as Cohort 1 Schools, vs. newly identified Receivership schools, referred to as Cohort 2 Schools (discussed in more detail in a separate section). Schools will be assigned and asked to select new indicators based on the revised methodology. However, DI determinations based on the revised methodology will first be made using 2019-20 school year results. 8

2. Selecting Demonstrable Improvement Indicators

Selecting Indicators Schools are measured on indicators selected both by NYSED and the district. There are two indicator types: Level 1: Indicators that are mostly performance-based (e.g., Performance Index, Growth, Graduation Rate). Selected by the State based on a school s baseline performance. A Level 1 indicator will be assigned when a school performs either below the State Measure of Interim Progress (MIP) or state average for the indicator, whichever is applicable. Level 2: Indicators selected by the individual school from a state-approved list. A school may choose a Level 2 indicator if the school performs either below the State Measure of Interim Progress (MIP) or state average for the indicator, whichever is applicable. Local indicators may be added by districts. All local indicators for Cohort 1 schools must be resubmitted to the Department for approval. 10

Selecting Indicators: Level 1 A minimum of 5 Level 1 Indicators (7 for schools serving grades 3-12) and a minimum of 5 (or 7) Level 2 Indicators. Total number of Level 1 indicators is determined by school s performance on each indicator. Schools are assigned all Level 1 indicators on which they fall below the current year target for the indicator. After year 1, if a school s Level 1 indicator falls below the State MIP or State average, then that indicator will be assigned to the school as an additional Level 1 indicator for the 2019-20 and 2020-21 school years. If the number of such indicators is less than 5 (or 7 for schools serving grades 3-12): First, alternative Level 1 indicators are assigned. These are indicators where school s current baseline exceeds 1st year State MIP, but does not exceed 3rd year State MIP. If multiple alternative Level 1 indicators are available, the State will assign the indicator with the largest gap between current performance and the Year 3 MIP. Second, if total available Level 1 indicators and Level 1 alternative indicators <5 (or <7 for grades 3-12 schools), Level 2 indicators may be used as Level 1 indicators. 11

Selecting Indicators: Level 1 Elementary and Middle Schools Indicator Name Note School Safety 3-8 ELA All Students MGP2 3-8 Math All Students MGP2 Student Suspension Rate Chronic Absenteeism ELA Core Subject Performance Index1 Math Core Subject Performance Index1 Science Core Subject Performance Index1 Progress in ELP (ELP Success Ratio) ELA Weighted Average Performance Index1 Math Weighted Average Performance Index1 Science Weighted Average Performance Index1 Making Priority School Progress Existing Existing; Revised baseline Existing; Revised baseline Currently a Level 2 indicator; New as Level 1 Currently a Level 2 indicator; New as Level 1 New, Replaces percentage of students scoring at Level 2 or above New, Replaces percentage of students scoring at Level 2 or above New, Replaces percentage of students scoring at Level 2 or above New New New New Eliminated 1 Core Subject and Weighted Average Index calculations use the same formula in the numerator. For the denominator, Weighted Academic Achievement uses the greater of: (1) # of continuously enrolled students who are tested or (2) 95% of continuously enrolled students with or with test scores. The Core Subject Performance denominator is the # of continuously enrolled students who are tested. School will be assigned either the Core Subject or Weighted Average Indicator for a subject, not both. 2 Baseline MGP based on 3-year average. 12

Selecting Indicators: Level 1 High Schools Indicator Name Note School Safety 11th Graders Passing ELA Regents 10th Graders Passing Math Regents Total Cohort 4-Year Grad Rate - All Students Total Cohort 5-Year Grad Rate - All Students Student Suspension Rate Chronic Absenteeism Existing Existing (2016 Total Cohort) Existing (2017 Total Cohort) Existing Existing Currently Level 2; New as Level 1 Currently Level 2; New as Level 1 Currently Level 2; New as Level 1 Replaces percentage of students scoring at Level 2 or above Currently Level 2; New as Level 1 Replaces percentage of students scoring at Level 2 or above New; Replaces Total Cohort 4-Yr Grad Rate w/ Adv Designation New New New New Eliminated ELA Performance Index Math Performance Index College, Career, & Civic Readiness (CCCR) Index Progress in ELP (ELP Success Ratio) Science Performance Index Social Studies Performance Index Total Cohort 6-Year Grad Rate - All Students Making Priority School Progress 13

Selecting Indicators: Level 2 Schools can select a maximum of 10 Level 2 indicators. Level 2 indicators are categorized into two tiers: Tier 1 = Focused on student and teacher outcomes. (List on slide 15.) Many of these indicators are based on accountability subgroups (i.e., English language learners, low-income students, students with disabilities, and racial/ethnic subgroups). Tier 2 = Focused on organizational and school process changes. For Cohort 1 Schools these indicators are each weighted at 1% of the Level 2 component of the DI Index. (List on slide 16.) The following Level 2 indicators have been eliminated: Post-graduation plans of completers (Indicators #83 and #84). Classes taught by teachers without appropriate certification (Indicator # 87). Percent of newly hired teachers with state-provided growth ratings of effective or above (Indicator #96). 14

Selecting Indicators: Level 2, Tier 1 Indicator Code(s) 3 34 - 38 40 - 44 45 - 49 50 - 54 55 - 59 60 - 64 65 66 68 71 - 75 89-93 82 86 95 97 10 - 14 16 - 20 tbd tbd 22 - 26 28 - 32 tbd tbd tbd 77 - 81 Indicator Name Tier Student Attendance 3-8 ELA MGP 3-8 Math MGP 3-8 ELA Level 2 and above Gap with non-similarly classified Student 3-8 Math Level 2 and above Gap with non-similarly classified Students HS ELA Level 2 and above Gap with non-similarly classified Students HS Math Level 2 and above Gap with non-similarly classified Students 2018 Total Cohort (9th Graders) with 5 or more credits 2017 Total Cohort (10th Graders) with 5 or more credits 2016 Total Cohort (11th Graders) with 5 or more credits Total Cohort 4-Year Grad Rate Total Cohort 5-Year Grad Rate Drop Out Rate Teacher Turnover Teacher Attendance HS Student Promotion Rate (promoted from grades 9,10 & 11) 3-8 ELA Core Subject Performance Index 3-8 Math Core Subject Performance Index 3-8 ELA Weighted Average Performance Index 3-8 Math Weighted Average Performance Index HS ELA Performance Index HS Math Performance Index Science Performance Index Social Studies Performance Index Total Cohort 6-Year Grad Rate College, Career, & Civic Readiness (CCCR) Index Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 Tier 1 These Level 2 indicators are based on the accountability subgroups. Shading indicates New or Revised indicators. 15

Selecting Indicators: Level 2, Tier 2 Tier 2 Indicators: Focused on organizational and school process changes. For Cohort 1 Schools each indicator is weighted at 1% of the Level 2 component of the DI Index. For Cohort 2 Schools these indicators are weighted in the same way as other Level 2 indicators. Indicator Code Indicator Name Tier 2 Plan for and implement quality Community School Model Tier 2 6 Family and Community Engagement (DTSDE Tenet 6) Tier 2 7 Teacher Practices and Decisions (DTSDE Tenet 4) Tier 2 8 Curriculum Development and Support (DTSDE Tenet 3) Tier 2 94 Providing 200 Hours of quality Extended Day Learning Time (ELT) Tier 2 16

Computing the DI Index Schools get credit on an indicator for meeting or exceeding the annual Progress Target. Schools get full credit or no credit for meeting or not meeting target or goal. No partial or extra credit. DI Index is a weighted average of Level 1 and Level 2 indicators. Level 1 indicators weighted at 50% of the DI Index and Level 2 indicators weighted at 50% of the DI Index, regardless of the number of Level 1 and Level 2 indicators. Each indicator within Level 1 and Level 2 shall be weighted equally (exception is Tier 2 indicators for Cohort 1 Schools). DI Index ranges from 0% to 100%, with cutpoints as follows: >= 67% >= 40% & < 67% Commissioner s Decision < 40% Preliminarily Determination: Did Not Make DI Preliminarily Determination: Made DI 17

Computing the DI Index: Cohort 2 Example Indicator Level Outcome Progress Target Met Target? Weighting Math Core Subject Performance Index Level 1 45 45 Yes 10% 3-8 ELA All Students MGP Level 1 48 47 Yes 10% Progress in ELP (ELP Success Ratio) Level 1 35% 40% No 0% 10% Chronically Absent (For Example) Chronic Absenteeism Level 1 25% of students chronically absent No 0% Grades 3-8 Math All Students MGP Math Core Subject Performance Index SWD Students Implement Community School Model 3-8 ELA SWD Students MGP Grades 3-8 ELA low-income MGP Total Cohort 5-Year Grad Rate - SWD Students Level 1 45% 47% No 0% Level 2 44 42 Yes 8.3% First Year Implementation Level 2 First Year Implementation Yes 8.3% Level 2 49 51 No 0% Level 2 51% 52% No 0% Level 2 67% 65% Yes 8.3% Student Attendance Level 2 77% 75% Yes 8.3% DI Index 53% 18

Demonstrable Improvement Determinations In this example, because the school s Demonstrable Improvement Index is at or above 40% but below 67%, the Commissioner reviews the entire performance of the school. After the review, the Commissioner determines the school has made Demonstrable Improvement. Note: Progress targets become more rigorous in Years 2 and 3. 19

Demonstrable Improvement Determinations NYSED will make preliminary determinations for each indicator as to whether school met the progress target. Districts may appeal the preliminary determinations for any indicator. NYSED will use school s DI Index and review of additional data for schools with DI Index at or above 40% but below 67% to make preliminary determinations. Commissioner will consult with stakeholders who may appeal DI preliminary determination. Commissioner makes final determination as to whether the school made DI. 20

Additional Information Once selected, indicators remain in place for three years. If a determination cannot be made in the current year about an indicator, the weights for that year will be adjusted. If a determination cannot be made in a future year about an indicator, another indicator may be selected. After year 1, if a school s Level 1 indicator falls below the 2018-19 target, then that indicator will be assigned as an additional Level 1 indicator to the school for 2019-20 and 2020-21 school years. After year 1, the Department will be review the progress targets for 2019-20 and 2020-21 school years. If any are determined to be insufficiently or overly rigorous, the Department reserves the right to adjust them. 21

3. DI Process for Cohort 1 & Cohort 2 Schools

Demonstrable Improvement Cohort 1 Schools: Current Receivership Schools that are identified as CSI schools under the ESSA accountability system. Cohort 2 Schools: Current Priority Schools not in Receivership that are identified as CSI schools under ESSA. Differentiation between Cohorts 1 and 2: 2018-19 school year determinations. Indicators available for selection. Progress targets. 23

DI Determinations for 2018-19: Cohort 1 Cohort 1 Schools: DI Determinations for Cohort 1 Schools will be made using the existing methodology and indicators selected for these schools. New Progress Targets will be determined and provided to these schools. If necessary, changes may be made to DI indicators assigned to these schools in a manner consistent with current methodology. Additionally, Cohort 1 Schools will be assigned new Level 1 indicators and will have the opportunity to select new Level 2 indicators based on the revised methodology for DI determinations beginning with the 2019-20 school year. 24

DI Determinations for 2018-19: Cohort 2 Cohort 2 Schools: DI Determinations for Cohort 2 Schools will not be made for the 2018-19 school year. Schools will receive a DI index for informational purposes only. Schools will be assigned new Level 1 indicators and will have the opportunity to select new Level 2 indicators based on the revised methodology for DI determinations beginning with the 2019-20 school year. Schools that select a DTSDE indicator will not receive a determination for that indicator in the 2018-19 school year. 25

DI Determinations for 2019-20 and 2020-21 Cohort 1 and Cohort 2 Schools: DI Determinations for both Cohorts will be made using the revised methodology and the new indicators selected for these schools. More rigorous Progress Targets will be used in each subsequent year. 26

How a School Exits Receivership Cohort 1 and Cohort 2 Schools: Schools will be removed from Receivership at the end of the school year in which the school is not identified as a Comprehensive Support and Improvement School. The first opportunity for removal will be as of June 2021 based on 2018-19 and 2019-20 school year results. Making DI does not determine whether a school is placed into or removed from Receivership. Other Schools: For current schools in Receivership that are not identified as CSI based on 2017-18 school year results, removal will occur at the end of the 2018-19 school year. 27

Timeline for Indicator Selection Cohort 1 Schools: For the 2018-19 school year, schools will be assigned Level 1 indicators and will be allowed to make revisions to Level 2 indicators with applicable goals and targets no later than for 4 weeks following notification regarding indicators, using the existing methodology. Cohort 1 and Cohort 2 Schools: For 2019-20 and subsequent school years, schools will be assigned Level 1 indicators and allowed to select Level 2 indicators for use for with applicable targets no later than 6 weeks following identification. 28

Next Steps Participate in NYSED Webinar for schools, which will occur once information on assigned Level 1 indicators and eligible Level 2 indicators have been provided to districts. Discuss selection of Demonstrable Indicators with the school s Community Engagement Team. Submit the file with the selected indicators to: accountinfo@nysed.gov. Call (718) 722-4553 or send an e-mail to accountinfo@nysed.gov for technical assistance. Submit questions to: accountinfo@nysed.gov. 29

Thank You

Appendix: Progress Targets for New Indicators Level 1 Indicators: Level 1 Indicators are automatically assigned to the school by NYSED if the school s baseline falls below the State Measure of Interim Progress (MIP) for that indicator (if a State MIP is available) or below the State Average (if a State MIP is not available). Level 2 Indicators: Schools have autonomy to select Level 2 Indicators. However, the Indicators must be eligible for selection. An Indicator is eligible for selection if the State MIP is available and the school baseline falls below the State MIP or, if State MIP is not available, the school baseline falls below the state average for that indicator. Cohort 1 schools have more rigorous progress Targets than Cohort 2 schools for Level 2 indicators where State MIP is not available. 31

Progress Targets: Level 1 Indicators Level 1 Indicators: Indicator Types & Associated Target-Setting Approach ESSA indicator; MIP currently calculated. currently calculated. Required Annual MIP progress provides reference point for annual target. ESSA indicator; No MIP Non-ESSA indicator; No MIP currently calculated. Calculated Annual MIP progress provides reference point for annual target. Indicator End Goal --- Fractional decrease in gap between state average & school baseline Indicator State Average --- --- Year Close gap between school baseline & end goal by 2% (see Table 2 for example and MIPS comparison) Close gap between school baseline & end goal by 4% (see Table 2 for example and MIPS comparison) Close gap between school baseline & end goal by 6% (see Table 2 for example and MIPS comparison) 1/6 reduction in gap between school baseline & state average. Year 1 DI Target Cohorts 1 & 2 1/4 reduction in gap between school baseline & state average. Year 2 DI Target Cohorts 1 & 2 1/3 reduction in gap between school baseline & state average. Year 3 DI Target Cohorts 1 & 2 32

Progress Targets: Level 2 Indicators Level 2 Indicators: Indicator Types & Associated Target-Setting Approach ESSA indicator; MIP currently calculated. ESSA indicator; No MIP currently calculated. Non-ESSA indicator; No MIP currently calculated. Indicator End Goal4 Indicator State Average Same as Level 1 Indicator. See above. Same as Level 1 Indicator. See above. --- Fractional decrease in gap between state average & school baseline 1/4 reduction in gap between school baseline & state average. 1/6 reduction in gap between school baseline & state average. 1/3 reduction in gap between school baseline & state average. 1/4 reduction in gap between school baseline & state average. 1/2 reduction in gap between school baseline & state average. 1/3 reduction in gap between school baseline &state average. --- --- Year Cohort 1 Year 1 DI Target Same as Level 1 Indicator. See above. Cohort 2 Cohort 1 Year 2 DI Target Same as Level 1 Indicator. See above. Cohort 2 Cohort 1 Year 3 DI Target Same as Level 1 Indicator. See above. Cohort 2 33

Progress Targets: Comparison with Measures of Interim Progress Example of Expected Annual DI Progress for Indicators for which End Goals are Available: Compared with MIP Calculations. School Baseline Year 1 Year 2 Year 3 Year 4 Year 5 End Goal PI 100 104 108 112 116 120 200 4% (12% over baseline) 4% (16% over baseline) 4% (20% over baseline) Expected Annual % Gains 4% 4% MIP --- (4% over baseline) (8% over baseline) --- PI 100 102 104 106 200 --- --- Expected Annual % Gains 2% 2% 2% DI Target --- (2% over baseline) (4% over baseline) (6% over baseline) --- --- --- 34