Child Outcomes Data Highlights: FFY 2019 Insights

FFY 2019 Child Outcomes Data Highlights Presenters: Cornelia Taylor, Haidee Bernstein and Nicholas Ortiz 1

Webinar Intended Outcomes Learn about the latest national child outcomes findings where to find more information about the national analysis, and how to find resources to support state and local child outcomes data use 2

General Background Looking at compliance is not enough to produce good results, so the focus has shifted to looking at results. All federal agencies are required to report on the outcomes achieved by their programs. The Office of Special Education Programs (OSEP) uses child outcomes data to: Justify the funding for Part C and Part B Preschool. Monitor state results through Results Driven Accountability processes (Part C only). 3

Three Child Outcomes In 2005, OSEP required states to report data on 3 child outcomes: Children have positive social emotional skills (including social relationships) Children acquire and use knowledge and skills (including early language/communication [and early literacy]) Children use appropriate behaviors to meet their needs 4

5

The Summary Statements 1. Of those children who entered or exited the program below age expectations in each outcome, the percent who substantially increased their rate of growth by program exit. (c+d)/(a+b+c+d) 2. The percent of children who were functioning within age expectations in each outcome by program exit. (d+e)/(a+b+c+d+e) 6

State Approaches to Measuring Child Outcomes FFY 2019 Approach Part C Part B 619 Child Outcomes Summary (COS) Process One Tool Statewide 40 (67%) 40 (71%) 8 (14%) 9 (15%) 4 (7%) 5 (8%) Publisher s Online System Other 4 (7%) 5 (8%) 56 59 TOTAL 7

Method for Calculating National Estimates Weighted average of states that met minimum quality criteria Minimum quality criteria for inclusion in national analysis: Data Completeness Part C: 28% or more of exiters Part B Preschool: 12% or more of 3-5 child count Validity category a not greater than 10% category e not greater than 65% 8

Reasons for Excluding States (FFY 2019) Started with 51 states in each program Part C Part B 619 Number Excluded 3 1 Reason for Exclusion Sampling Missing data* Unusual patterns in progress categories a and e ** Number Excluded 2 2 1 5 Total number of states included in the national analysis 46 40 *For Part C: reported outcomes data on less than 28% of exiters. For Part B 619: reported outcomes data on less than 12% of child count. **Had at least one outcome with category a greater than 10% or category e greater than 65%. 9

Number of States that Met Criteria for Inclusion in the National Analysis 60 Part C Part B Preschool 48 50 47 47 46 46 45 45 41 45 39 39 44 43 43 40 42 Number of States 41 40 40 33 36 30 33 29 19 20 15 10 0 08-09 09-10 10-11 11-12 12-13 13-14 14-15 15-16 16-17 17-18 18-19 19-20 School Year 10

National Child Outcomes Data for Children Exiting in FFY 2019 Part C Early Intervention Part B Preschool Outcome Summary Statement 1 Summary Statement 2 Summary Statement 1 Summary Statement 2 Social Relationships 64% 52% 81% 56% Knowledge and Skills 69% 44% 82% 53% Action to Meet Needs 71% 54% 81% 62% Note: Data are based on 46 states' Part C programs and 40 states' Part B Preschool programs. Only states with high-quality data were included. 11

Part C Child Outcomes Data Trends: FFY 2012-2019 Percent making greater than expected growth (Summary Statement 1) Percent exiting at or above age expectations (Summary Statement 2) 76% 76% 76% 75% 75% 73% 71% 71% 74% 74% 73% 74% 73% 71% 72% 69% 61% 61% 68% 67% 59% 67% 66% 59% 66% 57% 66% 58% 65% 57% 64% 54% 59% 59% 58% 57% 57% 57% 55% 52% 52% 52% 50% 50% 49% 47% 46% 44% 2012 2013 2014 2015 2016 2017 2018 2019 2012 2013 2014 2015 2016 2017 2018 2019 Social Relationships Knowledge and Skills Actions to Meet Needs Social Relationships Knowledge and Skills Actions to Meet Needs 12

Part B Preschool Child Outcomes Data Trends: FFY 2012-2019 Percent making greater than expected growth (Summary Statement 1) Percent exiting at or above age expectations (Summary Statement 2) 82% 82% 82% 81% 80% 81% 81% 81% 79% 80% 79% 79% 81% 81% 78% 80% 77% 78% 81% 80% 80% 77% 77% 77% 65% 65% 65% 64% 64% 64% 63% 62% 60% 59% 59% 60% 58% 58% 57% 56% 56% 56% 55% 54% 53% 53% 53% 52% 2012 2013 2014 2015 2016 2017 2018 2019 2012 2013 2014 2015 2016 2017 2018 2019 Social Relationships Knowledge and Skills Actions to Meet Needs Social Relationships Knowledge and Skills Actions to Meet Needs 13

State-Level Variation in Completeness of Child Outcomes Data 14

Part C: Completeness of Child Outcomes Data FFY 2019 Inclusion criterion = 28% and higher Completeness = Total with outcomes data/total exiters Based on Outcome A 88% 63% 19% N=51 15

Part B Preschool: Completeness of Child Outcomes Data FFY 2019 Inclusion criterion = 12% and higher Completeness = Total with outcomes data/child count Based on Outcome A 99% 4% N=49; One state was excluded because they did not report outcomes data, one state was excluded because they did not report age 3-5 child count data. 16

Examining State-Level Variation in Child Outcomes Summary Statements 17

Part C State Variation: Exited within Age Expectations (Summary Statement 2) Outcome: Knowledge and Skills FFY 2019 71% 45% 11% N = 51 18

Part B 619 State Variation: Exited within Age Expectations (Summary Statement 2) Outcome: Knowledge and Skills FFY 2019 76% 48% 14% N = 49. One state was excluded because they reported data on only 5 children. Another state excluded because they did not report outcomes data. 19

What could this difference mean? Interpreting Differences Across States State 2 61% State 1 80% 20

States serving a higher proportion of their overall population ages 0-3 in Part C are more likely to have a higher percentage of children exiting Part C within age expectations (FFY 2019) % who exited within age expectations Values represent average % within each group 60% Part C, Summary Statement 2: 57% 53% 51% 50% 49% 45% 41% 40% Social Relationships Knowledge and Skills Action to Meet Needs < 2.5% (6 states) 2.5-3.9% (28 states) > 3.9% (17 states) % of the state s population ages 0-3 served in Part C Total N = 51 21

States serving a higher proportion of their overall population ages 3-5 in Part B 619 were no more or less likely to have a higher percentage of children exiting Part B 619 within age expectations (FFY 2019) % who exited within age expectations Part B 619, Summary Statement 2: 60% 59% Values represent average % within each group 58% 55% 55% 54% 50% 49% 48% Social Relationships Knowledge and Skills Action to Meet Needs < 5.7% (13 states) 5.7-7.5% (17 states) >7.5% (18 states) Total N = 48 % of the state s population ages 3-5 served in Part B 619 22

2019 State Child Outcomes Data Quality Profiles 23

2019 State Data Quality Profiles Figure 3. Preschool Special Education State Trends for Summary Statement 1, Outcome A Figure 1. Preschool Special Education National and State Percentages for Summary Statement 1 100% 100% 91% 88% 87% 90% 90% 86% 86% 85% 82% 82% 81% 81% 80% 80% 78% 80% 76% 80% 74% 72% 71% 69% 70% 70% 60% 60% 50% 50% 40% 40% 30% 30% 20% 20% 10% 10% 0% 0% A Social Relationships B Knowledge and Skills C Action to Meet Needs 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 State National State -1 STD Average +1 STD 24

Child Outcomes Resources IDEA Child Outcomes Highlights for FFY2019 (ECTA/DaSy) A 2-page summary of the national results Indicator-Specific Guidance for Target Setting (DaSy) Guidance, considerations, suggested analyses, and further resources for setting C3/B7 targets Child Outcomes Approach Maps (DaSy) Data visualization displaying the child outcomes approach by state Take a Look at Your Child Outcomes Data Profile (DaSy) Guidance on how to use your Child Outcomes Data Quality Profile Year-to-Year Changes in State Child Outcomes Data: What Do They Mean?(ECTA/DaSy) A 4-page brief designed to help stakeholders learn what questions to ask to understand year-to- year changes in child outcomes data 25

Child Outcomes Resources Breadth of the Three Child Outcomes (ECTA/DaSy) An infographic defining six clusters of functional behaviors within each of the three child outcomes. Child Outcomes Summary (COS) Process Professional Development Resources (ECTA/DaSy) A collection of national resources to support training and TA on the COS process Remote Screening, Evaluation, and Assessment (ECTA/DaSy) Three lists of tools compiled to help states, programs, and staff identify assessment tools for use when practitioners cannot be in the room with the child and family COS Completion When Teams Can't Meet In Person (ECTA/DaSy) How to use telecommunication devices to have discussions with team members in different locations through teleconferencing Remote Service Delivery and Distance Learning (ECTA) Online information to assist programs in addressing the challenges of the pandemic 26

Contact us Cornelia Taylor cornelia.taylor@sri.com Haidee Bernstein Haidee.Bernstein@sri.com Nicholas Ortiz nicholas.ortiz@sri.com 27

Find out more at ectacenter.org and dasycenter.org The ECTA Center is a program of the FPG Child Development Institute of the University of North Carolina at Chapel Hill, funded through cooperative agreement number H326P170001 from the Office of Special Education Programs, U.S. Department of Education. Opinions expressed herein do not necessarily represent the Department of Education's position or policy. Project Officers: Julia Martin-Eile The DaSy Center is a program of SRI International, funded through cooperative agreement number H373190002 from the Office of Special Education Programs, U.S. Department of Education. Opinions expressed herein do not necessarily represent the Department of Education's position or policy. Project Officers: Meredith Miceli & Amy Bae 28