Oregon Department of Education: Equity in Education Partnerships

2022-23 Graduation and Dropout Reporting Part 2 of 2: Graduation and Dropout Validations 10/25/2023 Paul Skomsvold paul.skomsvold@ode.Oregon.gov Oregon Department of Education 1

About Us The Oregon Department of Education works in partnership with school districts, education service districts and community partners to foster equity and excellence for every learner; Together, we serve over 553,012* K-12 students and support 81,826* school employees, including teachers, administrators and classified staff; We believe every student should have access to a high-quality, well-rounded learning experience; and We work to ensure every student in Oregon graduates with a plan for their future. *Data from October 2022 Oregon Department of Education 2

Equity Education equity is the equitable implementation of policy, practices, procedures, and legislation that translates into resource allocation, education rigor, and opportunities for historically and currently marginalized youth, students, and families including civil rights protected classes. This means the restructuring and dismantling of systems and institutions that create the dichotomy of beneficiaries and the oppressed and marginalized. Oregon Department of Education 3

Todays Schedule Dates & Review Validations Overview Viewing Validations & Making Corrections Methodology/Rate Calculations Additional Tips/Resources Oregon Department of Education 4

Dates & Review: Key Dates Validations Closing: NCES Dropout Rates 22-23 4-Year Cohort Graduation Rates 22-23 5-Year Cohort Graduation Rate 22-23 First Period Cumulative ADM 23-24 Audit Review Closes Public Cohort Graduation and Dropout Report Release First Period Cumulative ADM 23-24 Closed December 8 October 23 November 3 TBD October 26 December 1 December 21 Validations Opening: NCES Dropout Rates 22-23 4-Year Cohort Graduation Rates 22-23 5-Year Cohort Graduation Rate 22-23 Annual Cumulative ADM 22-23 Exit Adjustment Closes Final Graduation and Dropout Rate Refresh First Period Cumulative ADM 23-24 Audit Review Opens Oregon Department of Education 5

Validation Overview 1. National Center for Education Statistics (NCES) Dropout and Graduation Rates Dropout and Graduation events occurring in 2022-23 2. Cohort Graduation Rates for 4-year cohort Students with HS Entry year of 2019-20 (on-time graduation) 3. Cohort Graduation Rates for 5-year cohort Students with HS Entry year of 2018-19 (extended 1 extra year) Oregon Department of Education 6

Validation Overview: Uses of the Data Publicly released in media files Will appear on school and district At-a-Glance Reports in Fall 2024 EdFacts (Federal Reports) Special Education (SPR&I) Ad Hoc requests as needed Oregon Department of Education 7

Validation Overview: Quick Reminders A few quick reminders before we dive in Exit Adjustment Fall Membership Oregon Department of Education 8

Validation Overview: Exit Adjustment Opened October 5, 2023. Closes December 1, 2023. Not a collection just a window into Annual Cumulative ADM 2022- 23 No new records! If you need to add records, contact your Regional ESD Partner and ask that they open Annual Cumulative ADM. See training part 1, from October 5, for more information: https://odedistrict.oregon.gov/CollectionsValidations/Validations/Pages/4YearCoh ort.aspx Oregon Department of Education 9

Validation Overview: Calculated End Date Codes Attach to the student s latest record. Refresh Nightly Depend on all data being reported Do not assume the calculated end date code is necessarily the correct one for this student. See training part 1, from October 5, for more information: https://odedistrict.oregon.gov/CollectionsValidations/Validations/Pages/4YearCohort.as px Oregon Department of Education 10

Validation Overview: Public Releases Important! Although you can see both preliminary and (later) final data during the validation window and before the public release, graduation, dropout, and fall membership data are EMBARGOED until the public release dates. Do not share embargoed data with anyone other than authorized district staff. Oregon Department of Education 11

Viewing Validations & Making Corrections: Purpose of Report Validation Report validation window allows districts to: Preview counts and rates Review student level data used for counts Ensure counts of diplomas or dropouts match districts student information system Correct or adjust data Understand report rules and impact to upcoming accountability reports (with targets and ratings) and plan for local news releases Oregon Department of Education 12

Viewing Validations & Making Corrections: How to Validate Log into the Achievement Data Insight (ADI) Application Training on the application is posted at ODE District Data Site : Achievement Data Insight (ADI) : Applications : State of Oregon Make sure you have permissions to these validations: Fall Membership 4-Year Cohort Graduation Rate 5-Year Cohort Graduation Rate NCES Dropout and Graduation Rates Contact your DSA if you need access To look up your DSA https://district.ode.state.or.us/apps/login/searchSA.aspx Remember: there is no Accept button Oregon Department of Education 13

Viewing Validations & Making Corrections: Anatomy of an ADI Tile Title of the Validation Date the current user last opened this tile Example Date the tile refreshed Different from the Data As-of Date Oregon Department of Education 14

Viewing Validations & Making Corrections: Data As-of-Date In this example, changes made by 10/9/2018 were included in the data currently displayed Oregon Department of Education 15

Viewing Validations & Making Corrections: Data As-of-Date You add a record for Bruce Banner on Friday, November 3 ODE begins processing a validation refresh on Tuesday, Nov 7 You add another record for Melinda May on Wednesday, Nov 8 ODE refreshes the validation in ADI on Thursday, Nov 9 The Data As-of Date will be Tuesday, Nov 7 (processing begins early Tuesday morning so we encourage folks to get changes in by Monday at 5PM) Bruce Banner s record will be included in this refresh, but Melinda May s will not. Oregon Department of Education 16

How to Validate Cohort: Step 1 Review the student-level data to ensure that outcomes are accurate for each student If you find an outcome that you believe is not accurate: First, check to make sure that you have reported the correct outcome to ODE if you have not, report it. Next, if you reported the outcome, ensure that the date reported on the ADM record is correct Check to make sure a cohort refresh has occurred since you made the correction, using the As-of Date Oregon Department of Education 17

How to Validate Cohort: Step 2 If you have reported the correct outcome to ODE, prior to the as-of date that is currently displayed, but this outcome is not displayed as the student s outcome, review rules in the manual or ask a question to find out why. Include SSID Number (do not combine SSIDs and names) Do not include Confidential Information Oregon Department of Education 18

How to Validate Cohort: Step 3 Next, review student-level data to ensure that student groups, accountable institutions, and other data are accurate If student group data are inaccurate: If you need to correct a student s Gender or Ethnicity, and the student was enrolled during the 2022-23 or 2023-24 school years, contact your Regional ESD partners, and request to have the relevant ADM collection opened so that you can edit the student s records For other student groups, use the appeal/update process. Oregon Department of Education 19

Viewing Validations & Making Corrections: Additional Info on Corrections Student Enrolled during 2022-23 and outcome data needs editing: Use Exit Adjustment to correct data. Non-Outcome data needs editing: Contact Regional ESD Partners and edit Annual Cumulative ADM. Add outcomes before 2022-23: Contact Regional ESD Partners and edit Annual Cumulative ADM; Add a Program Type 14 Record. Change a High School Entry Year: Send appeal to Regional ESD Partners through Secure File Transfer. Note: some data is from Spring Membership or LEP and can no longer be edited (see the calculations sections of this WebEx or the manuals for more information) Oregon Department of Education 20

Viewing Validations & Making Corrections: Cohort Updates and Appeals If you can t correct your data in one of these ways, you must submit an appeal. Appeals can be accepted through the end of the day on December 1, but no later than this. Examples of Appeals: Student Group Identification is inaccurate Student was previously reported in ADM with an outcome or institution that is not accurate Student was matched to GED data incorrectly Use the Ask-a-Question feature in the ADI Validation to submit an appeal. All appeals must be student-level. Oregon Department of Education 21

Viewing Validations & Making Corrections: Change Deadlines Appeals must be submitted to ODE by 5:00 PM on the Friday before a refresh is scheduled in order to be included in that refresh. Data edits (in the Cumulative ADM Collection) must be made by 5:00 PM on the Monday before a refresh is scheduled in order to be included in that refresh. Oregon Department of Education 22

Viewing Validations & Making Corrections: Cohort Update/Appeals Deadlines Appeals (5pm) Updates (5pm) ADI Refresh (first refresh data edits only) October 23 October 26 (Validation Opening) October 27 October 30 November 2 November 3 November 6 November 9 November 10 November 13 November 16 November 24 November 27 November 30 December 1 December 4- last week validations open December 7 December 8 deadline for guaranteed review December 11 December 14 December 15 December 18 December 21 Oregon Department of Education 23

Viewing Validations & Making Corrections: Accepting and Appealing Validations ODE s system logs each visit to a validation. Validations are an opportunity for districts to review and correct their data prior to publication. While it is ODE s expectation that all data submitted are accurate, and validation is encouraged, it is not mandatory. If you do not visit the validation, you will receive a reminder. Remember: Make sure charter school staff, high school principals and registrars, and any other interested parties are also able to visit the validation during this time. Oregon Department of Education 24

Methodology/Rate Calculations: Differences between Reports Definition NCES 4-year Cohort 5-year Cohort ODE s system logs each visit to a validation. Denominator Dropout Rate: Fall Membership 2022- 23 in Grades 9-12 Adjusted Cohort (students with 2019-20 cohort, minus students who were deceased or transferred out) Adjusted Cohort (students in 2018-19 cohort, minus students who were deceased or transferred out) Validations are an opportunity for districts to review and correct their data prior to publication. While it is ODE s expectation that all data submitted are accurate, and validation is encouraged, it is not mandatory. Graduates Regular or Modified Diploma awarded (4A or 4G) between 9-1-22 and 8-31-23 Grad Rate: (Dropouts + Graduates) Regular or Modified diploma earned by students in the adjusted cohort between 7- 1-19 and 8-31-23 Regular or Modified diploma earned by students in the adjusted cohort between 7-1-18 and 8-31-23 If you do not visit the validation, you will receive a reminder. Remember: Make sure charter school staff, high school principals and registrars, and any other interested parties are also able to visit the validation during this time. 23, plus students reported as 3F between 5-1 and 6-30-22. Dropouts Students reported as school-year dropouts between 7-1-22 and 6-30- No cohort dropout rate No cohort dropout rate Oregon Department of Education 25

Methodology/Rate Calculations: 22-23 Cohort Changes New student groups! Military Connected Students Recently Arrived Students with Experience in Incarceration or Detention Oregon Department of Education 26

Methodology/Rate Calculations: Cohort Definitions Unadjusted Cohort: All the students with the 2019-20 cohort year who were enrolled during the four years being measured Adjusted Cohort: The Unadjusted Cohort, minus any students whose outcome was Transferred Out or Deceased Graduates: All students in the Adjusted Cohort who earned/were awarded a Regular or Modified High School Diploma by August 31, 2023 Completers: All students in the Adjusted Cohort who earned/were awarded a Regular, Modified, or Extended High School Diploma, or were awarded an Adult High School Diploma or a GED, by August 31, 2023 Oregon Department of Education 27

Methodology/Rate Calculations: Cohort Methodology 1: Identify all the students in the (unadjusted) cohort. 2: Look at all of the outcome records we have for those students. 3: Assign outcome based on lowest outcome rank (i.e., the best outcome rank). Hierarchy shown on the next slide.. Oregon Department of Education 28

Methodology/Rate Calculations: Cohort Methodology Regular High School Diplomas (Awarded (4A) or Earned (4F) by 8-31-23) Post Graduate Scholars (4G) by August 31, 2023 Modified High School Diplomas (Awarded (4A) or Earned (4F) by 8-31-23) Transfer Out Events, or Student Deceased (Occurred on or before 8-31-23) Student moved to another state/country Student transferred to homeschool Student transferred to private school Student deceased or permanently incapacitated Other Completers (Awarded by 8-31-23) Adult High School Diploma (AHSD) Extended Diploma GED Non-completers Alternative Certificates Continuing Enrollment (student enrolled on or after 9-1-23) Dropouts/non-completers. Oregon Department of Education 29 Outcome Rank Descriptions 1819 (oregon.gov)

Methodology/Rate Calculations: Transfer Out Events Transfer Out Events or Student Deceased are only counted if they are the student s last event, or close to the student s last event. Example: Mary is enrolled from September 2019 through May 2020, then transfers to private school (and is coded 2B). In October 2020, she re-enrolls with her previous school district, and earns a modified diploma in June 2023. Mary would be counted as a modified diploma, not a transfer out. Oregon Department of Education 30

Methodology/Rate Calculations: Cohort Rate Calculation Cohort Graduation Rate: Graduates Adjusted Cohort Cohort Completer Rate: Completers Adjusted Cohort Oregon Department of Education 31

Methodology/Rate Calculations: Accountable Institutions Accountable to the last degree-granting institution they were enrolled in. At the district level, it will be the last degree-granting school they were enrolled in within this district. Oregon Department of Education 32

SB 923 and Accountable Institutions Updated ORS 327.061 to say: a student may be included in average daily membership only if the student is reported with a school identification number, as issued in accordance with rules adopted by the State Board of Education. Cohort will continue to use district-only accountable school cohorts for: ADM records reported prior to 23-24 Accountable school has closed SB 923 Guidance SB 923 FAQ Oregon Department of Education 33

Methodology/Rate Calculations: Accountable Institutions Student enrolls in Magenta SD, transfers to Orange SD in September 2020, and graduates from Orange SD in June 2023. Orange SD is accountable. Student enrolls in Aqua SD, at Daisy Johnson Charter School and remains enrolled there through June 2023 without earning a credential. In September 2023, the student transfers to Chartreuse SD. Student is accountable to Aqua SD and Daisy Johnson Charter School. Student enrolls in Crimson SD, at Coulson Memorial HS. Student transfers into a Crimson SD district- level program in May 2021, and remains there until withdrawing from the school in June 2022. Student is accountable to Crimson SD and to Coulson Memorial HS. Oregon Department of Education 34

Accountability for 4F and 4G: 4F = Students earned a diploma, but were not awarded the diploma, have been enrolled for fewer than four years of high school, and remain enrolled for the current (23-24) school year. 4G = Students were awarded a diploma and are continuing into a Post Graduate Scholars program. A student attends Carol Danvers High School from September 2019 to May 2022. The student meets diploma requirements, but elects to remain enrolled for a fourth year, so she is reported with a 4F ADM End Date Code. In September 2022, the student enrolls in Kamala Khan Early College Academy for her fourth year of high school (note: both schools are in the same district). She completes the year there, and in June 2023 is awarded her diploma and reported with a 4A ADM End Date Code. In the 2022-23 Cohort Graduation Rates, the student s outcome will be High School Diploma Awarded, and her Accountable School will be Carol Danvers High School. Oregon Department of Education 35

Students Accountable to the State of Oregon: Outcome Last Degree-Granting Accountable Always uses the resident institutions, not the attending. Oregon Department of Education 36

Students Accountable to the State of Oregon: Example In this example, Portland SD was the last degree-granting district to enroll the student, so they will see the student s record However, this student is not included in Portland s (or any other district s) unadjusted cohort Oregon Department of Education 37

Additional Tips/Resources: Cohort Summary & Student Level Student Level Make sure all of the diplomas awarded have been reported. Check students who are listed as dropouts/non-completer . Confirm that schools have the required documentation for students who transferred out. Review ADM Exit Adjustment and Dropout Reporting manual and training (Oct. 5) for more information. Aggregate (Summary) level Review totals first, then student group counts Make sure all the high schools and combined schools with high school grades in your district are included on the report Remember that the district total is not necessarily the sum of the school totals. Oregon Department of Education 38

Additional Tips/Resources: Cohort Student Level, ADM/SSID, and Access ADM/SSID Verify accuracy for First Period Cumulative ADM Check for duplicate SSIDs Verify Accurate Date for Program Type 14 records Access Verify all schools have access and are reviewing their data Verify that the correct people have access to review data Begin early! Oregon Department of Education 39

Additional Tips/Resources: Additional Resources, Part 1 AA Update newsletter http://www.oregon.gov/ode/educator-resources/assessment/Pages/Assessment- and-Accountability-Update.aspx Video Training 2023-24 (to participate, get handouts, and review recorded trainings) ODE District Data Site : Previous Trainings : Training : State of Oregon Oregon Department of Education 40

Additional Tips/Resources: Additional Resources, Part 2 Cumulative ADM Resources: ODE District Data Site : ADM Collections : Collections : State of Oregon Cohort Graduation Rates: http://www.oregon.gov/ode/reports-and- data/students/Pages/Cohort-Graduation-Rate.aspx Dropout & Exit Adjustment Manual: dropout_exitadj_manual_2223.pdf (oregon.gov) Cohort Graduation Rate Manual CohortGraduationRate_PolicyManual_20222023.pdf (oregon.gov) Dropout Rates: http://www.oregon.gov/ode/reports-and- data/students/Pages/Dropout-Rates.aspx Program Type 14 Guidance: CumADM Program Type 14 Guidance.pdf (oregon.gov) Fall Membership: http://www.oregon.gov/ode/reports- and-data/students/Pages/Student-Enrollment- Reports.aspx Post Graduate Scholars Guidance: https://www.oregon.gov/ode/schools-and- districts/grants/Pages/Post-Graduate-Scholar- Programs.aspx Oregon s ESSA Plan http://www.oregon.gov/ode/rules-and- policies/ESSA/Pages/default.aspx High School Entry Year Assignment Guidance: High School Entry Year Assignment Guidance.pdf (oregon.gov) School and District At-a-Glance Profiles: https://www.ode.state.or.us/data/reportcard/reports.asp x Oregon Department of Education 41

Contacts For help, contact your Regional ESD Partners: Karen Brown Smith 1-800-706-4447 x3124 karen.brown@imesd.k12.or.us Peter Campbell 1-800-706-4447 x3203 peter.campbell@imesd.k12.or.us http://www.oregon.gov/ode/educator- resources/assessment/Documents/esdpartners.pdf Paul Skomsvold 971-208-0296 Paul.Skomsvold@ODE.Oregon.gov Oregon Department of Education 42

Questions Oregon Department of Education 43

Thank You Oregon Department of Education 44