Importance of School Climate and Delaware School Surveys

School Climate and the Delaware School Surveys: Review of importance of school climate and overview of tools used to measure it plus more DELAWARE POSITIVE BEHAVIOR SUPPORT PROJECT

Why is school climate important? School Climate is linked to a wide range of academic, behavioral, and socio- emotional outcomes for students and teachers: Academic achievement Student academic, social, and personal attitudes and motives Student attendance and school avoidance Student behavior problems, delinquency, victimization Student and teacher emotional well-being Teachers greater implementation fidelity of new curriculum and interventions

Why is school climate important? The Every Student Succeeds Act (ESSA) explicitly recognizes the strong relationship between positive school climate and student learning and success. Law requires states to include data related to school climate and safety in annual school report cards

How are we doing in Delaware? DE School Climate Longitudinal Study 2012-2017 Examined changes in students perceptions of school climate using the DE School Climate Scale -Student version Guiding question: Did students perceptions of school climate improve from 2012 to 2017 in elementary, middle, and high schools? If so, were improvements found across all seven aspects of school climate measured by the Delaware School Climate Scale-Student?

Major Findings Total school climate score: Students perceptions quite favorable; especially in elementary schools Improved significantly from 2012 to 2017 All seven subscales: Scores improved significantly (elementary, middle, and high schools) Exception: Bullying School-wide subscale scores in middle schools. Most impressive: Improvements in School Safety and Bullying (elementary & high schools) For more information, an executive summary and full report for the longitudinal study are available.

What do I need to know about the DE School Surveys? THIS SECTION WILL: REVIEW THE 5 SCALES INCLUDED IN THE SURVEY LIST THE SUBSCALES INCLUDED IN EACH OF THE 5 SCALES PER POPULATION PROVIDE ITEM EXAMPLES FROM EACH SCALE AND SUBSCALE.

Delaware School Survey Scales (Note: School Climate is 1 of the 5 scales included in the survey) Student Scales Teacher/Staff Scales Home Scales School Climate Techniques (School Discipline) Bullying Victimization Student Engagement Student Social and Emotional Competencies School Climate Techniques (School Discipline) School Climate Bullying Victimization Student Engagement

Delaware School Climate Scale: Subscales per Population Student Teacher/Staff Home Teacher-Student Relations Teacher-Student Relations Teacher-Student Relations Student-Student Relations Student-Student Relations Student-Student Relations Clarity of Expectations Clarity of Expectations Clarity of Expectations Fairness of Rules Fairness of Rules Fairness of Rules School Safety School Safety School Safety Student Engagement School- wide Student Engagement School- wide Bullying School-wide Bullying School-wide Teacher-Home Communications Teacher-Home Communications Staff Relations Total School Climate Total School Climate Total School Climate Satisfaction with School

School Climate Scale Subscale Item Examples Teacher-Student Relations Teachers care about their students. Student Engagement School-wide Most students try their best. Fairness of Rules The school rules are fair. Bullying School-wide (Note: A high score on this subscale is unfavorable) Students threaten and bully others. Staff Relations Teachers, staff, and administrators work well together.

Delaware Positive, Punitive, and SEL Techniques Scale: Subscales per Population Student Survey Teacher/Staff Survey Home Survey Positive Behavior Techniques Positive Behavior Techniques Punitive Techniques Punitive Techniques Social Emotional Learning Techniques Social Emotional Learning Techniques

Techniques Scale Subscale Item Examples Use of Positive Techniques Students are praised often. Use of Punitive Techniques (Note: A high score for this subscale is unfavorable) Students are punished a lot. Use of Social Emotional Learning (SEL) Techniques Students are taught to feel responsible for how they act.

Bullying Victimization Scale: Subscales per Population Student Teacher/Staff Home Verbal Bullying Verbal Bullying Physical Bullying Physical Bullying Bullying Victimization1 Social/Relational Bullying Social/Relational Bullying Cyberbullying2 Cyberbullying2 1 Grades 6-12 only for the printed version. Optional for grades 4-5 with computer version. 2Grades 6-12 only.

Bullying Scale Subscale Item Examples Verbal Bullying A student said mean things to me. Physical Bullying I was pushed or shoved on purpose. Social/Relational Bullying A student told or got others to not like me. Cyberbullying (grades 6-12) A student sent me a mean or hurtful message about me using email, text messaging, instant messaging, or similar electronic messaging.

Student Engagement Scale: Subscales per Population Student Teacher/Staff Home Behavioral Behavioral Student Engagement Cognitive Cognitive Emotional Emotional

Student Engagement Subscale Item Examples Behavioral Engagement I pay attention in class. I follow the rules at school. Cognitive Engagement I try my best in school. Emotional Engagement I feel happy in school.

Social and Emotional Competencies Scale: Subscales per Population Teacher/Staff Survey Student Survey Home Survey Responsible Decision-making /Responsibility Understanding how others think and feel/Social Awareness Self-management of Emotions and Behavior Relationship Skills

Social & Emotional Competencies Subscale Item Examples Responsible Decision-making/Responsibility I feel responsible for how I act. Understanding how others think and feel/Social Awareness I think about how others feel. Self-management of emotions and behavior I can control how I behave. Relationship skills I am good at solving conflicts with others.

Are the DE School Surveys Reliable and Valid? THIS SECTION WILL: REVIEW EVIDENCE OF SURVEY RELIABILITY REVIEW EVIDENCE OF SURVEY VALIDITY PROVIDE REFERENCES TO PEER-REVIEWED JOURNAL STUDIES.

Reliability Are the scores consistent or stable? Generally want reliability scores (alpha coefficients) to be above .70. Reliable scales and subscales indicate that if the same students, teachers/staff, or parents in a school took the survey again, similar results would be found. CAUTION: Some scores are not very reliable with 3rdgraders.

School Climate and Techniques: Reliability (alpha coefficients) School Climate and Techniques: Reliability (alpha coefficients) Subscale Student .88 .87 .77 .77 .80 .82 .74 N/A N/A .91 N/A Teacher/Staff .88 .91 .87 .90 .82 .87 .86 .90 .96 .94 N/A Home .92 .95 .91 .93 .91 N/A N/A .91 N/A .98 .88 Teacher-Student Relations Student-Student Relations School Safety Clarity of Expectations Fairness of Rules Student Engagement School-wide Bullying School-wide Teacher-Home Communications Staff Relations Total Climate Parent Satisfaction Use of Positive Behavioral Techniques .84 .83 N/A Use of Punitive Techniques .75 .80 N/A Use of Social Emotional Learning Techniques .84 .90 N/A

School Climate: School Climate: Student StudentReliability (alpha coefficients) by Reliability (alpha coefficients) by Grade Grade Teacher Student Relations Clarity of Expect- ations Student Engagement School-wide Bullying School- wide Student Relations School Safety Fairness of Rules Total Score Grade Third .75 .82 .62 .64 .65 .71 .64 .85 Fourth .81 .85 .68 .71 .71 .73 .70 .86 Fifth .83 .86 .71 .74 .77 .76 .77 .87 Sixth .87 .87 .78 .78 .81 .80 .79 .89 Seventh .87 .88 .79 .81 .82 .82 .80 .91 Eighth .87 .87 .80 .80 .83 .81 .80 .90 Ninth .86 .87 .82 .81 .83 .82 .77 .91 Tenth .86 .88 .82 .77 .82 .80 .81 .90 Eleventh .87 .88 .84 .82 .83 .83 .81 .91 Twelfth .87 .90 .86 .82 .83 .82 .82 .91

Student Engagement and Bullying Victimization: Student Engagement and Bullying Victimization: Reliability (alpha coefficients) Reliability (alpha coefficients) Subscale Student Home Behavioral Engagement .80 .86 Cognitive Engagement .75 .85 Emotional Engagement .86 .91 Verbal Bullying .91 .91 Physical Bullying .84 .78 Social/Relational Bullying .90 .90 Total Bullying .95 .93 Cyberbullying .89 .90

Student Social and Emotional Competencies Scale: Student Social and Emotional Competencies Scale: Reliability (alpha coefficients Reliability (alpha coefficients) ) Subscale Student Responsible Decision Making .77 Social Awareness .86 Self-Management .78 Relationship Skills .77 Total Score .92

Validity Does the test yield the factors predicted? Are the scores related to other variables as one might predict (e.g., grade level, gender and race, academic achievement, suspensions)? Caution: Correlation does not mean causation. Direction of influence is likely to be bidirectional. Extensive studies have been conducted to test validity of DE School Surveys, and they are deemed VALID! For details, please see the Technical Manual for the Delaware School Survey. Bear, G., Yang, C., Harris, A., Mantz, L., Hearn, S., & Boyer, D. (2019). Technical Manual for the Delaware School Survey: Scales of School Climate; Bullying Victimization; Student Engagement; Positive, Punitive, and Social Emotional Learning Techniques; and Social and Emotional Competencies. Newark, DE: Center for Disabilities Studies.

Concurrent Validity Are the scores related to other variables as one might predict (e.g., grade level, gender and race, academic achievement, suspensions)? In the tables on the following slides, subscale scores from the student and teacher/staff survey are correlated with academic achievement (in ELA and Math) and suspensions/expulsions Positive and significant correlations indicate that a higher scores on a subscale are related to higher scores on the academic achievement or suspension/expulsion variables Negative and significant correlations indicate that lower scores on a subscale are related to higher scores on the academic achievement or suspension/expulsion variables, and vice versa Non-significant correlations indicate a weak relationship between variables and relationships should not be interpreted as being meaningful Again, Correlation does not mean causation. Direction of influence is likely to be bidirectional. *The data in the following slides is from the 2015-16 school year.

Evidence of Concurrent Validity Student Survey and School-level Data: School Climate Scale Elementary Schools Middle Schools High Schools Subscales ELA Math S/E ELA Math S/E ELA Math S/E Teacher Student Relations Student Student Relations Engagement School-wide Clarity of Expectations Fairness of Rules School Safety .540** .485** -.420** .714** .715** -.583** -.565* -.041 .108 .691** .649** -.682** .751** .755** -.740** .526* -.837** .286 .531** .530** -.585** .663** .644** -.623** .546** -.819** .355 .463** .445** -.316** .605** .614** -.408* -.432* -.021 .077 .500** .463** -.366** .690** .616** -.772** -.431* -.296 -.135 .558** .500** -.512** .657** .669** -.579** .451* .528* -.691** Bullying School-wide Total School Climate Note. ELA= English Language Arts. S/E = Suspensions and Expulsions. ELA = % passing ELA. Math = % passing math. -.782** -.687** .574** -.708** -.760** .676** -.510* -.686** -.381 .694** .639** -.598** .746* .743** -.698** -.749** .223 .393 77 Elementary Schools, 28 Middle Schools, 17 High Schools. *p < .05. **p < .01, ***p < .001One tailed.

Evidence of Concurrent Validity Teacher Survey and School-level Data: School Climate Scale Elementary Schools Middle Schools High Schools Subscale ELA Math S/E ELA Math S/E ELA Math S/E Teacher Student Relations .522** .648** -.574** .577** .590** -.463** .431* -.661** .353 Student Student Relations Schoolwide Engagement Clarity of Expectations Fairness of Rules School Safety .716** .746** -.753** .683** .635** -.586** .629** .704** -.668** .743** .816** -.734** .727** .746** -.683** .731** .775** -.774** .498** .624** -.541** .444* .396* .420* .397* -.424* -.246 .559** .591** .611** .683** -.566** -.696** .506** .590** .468** .543** -.400* -.486** .559** .574** .461* .590** -.419* -.580** -.687** -.700** .690** -.660** -.612** .463* -.534** -.547** .500* Bullying Schoolwide Teacher-Home Communications .604** .698** -.555** .551** .614** -.530** .547** -.700** .330 .307** .270** -.211* Staff Relations .214 .259 -.198 .230 .222 -.360 .655** .622** -.527** .617** .603** -.508** .587** .613** -.676* Total School Climate Note. ELA= English Language Arts. S/E = Suspensions and Expulsions. ELA = % passing ELA. Math = % passing math. 75 Elementary Schools, 27 Middle Schools, 20 High Schools. *p < .05. **p < .01, One tailed.

Evidence of Concurrent Validity Student Survey: Techniques Scale Elementary Schools Middle Schools High Schools Subscale ELA Math S/E ELA Math S/E ELA Math S/E Punitive Techniques -.764** -.714** .634** -.790** -.822** .735** -.147* -.473* .726** Positive Techniques -.033 -.030 -.016 .113 .102 -.118 -.460 -.332 -.262 SEL .374** .325** -.325** .580** .610** -.619** -.199 -.181 -.151 Techniques Note. ELA= English Language Arts. S/E = Suspensions and Expulsions. ELA = % passing ELA. Math = % passing math. 76 Elementary Schools, 28 Middle Schools, 18 High Schools. *p < .05. **p < .01, **p <.001.One tailed.

Evidence of Concurrent Validity Teacher Survey: Techniques Scale Elementary Schools ELA Math Middle Schools ELA Math High Schools Math Subscale S/E S/E ELA S/E Positive Techniques Punitive Techniques SEL Techniques .319** .347** -.244* -.542** .258 .278 -.234 .137 .261 .692** .688** -.672** .655** .649** -.674** .396* .483* -.627** .544** .540** -.415** .390* .386* .607** .529** -.619** -.288 Note. ELA= English Language Arts. S/E = Suspensions and Expulsions. ELA = % passing ELA. Math = % passing math. 75 Elementary Schools, 27 Middle Schools, 20 High Schools. *p < .05. **p < .01,One tailed.

Peer-Reviewed Journals Bear, G. G., Gaskins, C., Blank, J. , & Chen, F. F. (2011). Delaware School Climate Survey-Student: Its factor structure, concurrent validity, and reliability. Journal of School Psychology. Bear, G., Yang, C., Pell, M., & Gaskin, C. (2014).Validation of a brief measure of teachers perceptions of school climate: Relations to student achievement and suspensions. Learning Environments Research. Bear, G.G., Yang, C., & Pasipanodya, E. (2015). Assessing school climate: Validation of a brief measure of the perceptions of parents. Journal of Psychoeducational Assessment. Yang, C., Bear, G. G., Chen, F.F., Zhang, W., Blank, J.C., & Huang, X.S. (2013). Students perceptions of school climate in the U.S. and China. School Psychology Quarterly. Bear, G.G., Mantz, L., Glutting, J., Yang, C., & Boyer, D. (2015). Differences in bullying victimization between students with and without disabilities. School Psychology Review.

Peer-Reviewed Journals Bear, G.G., Holst, B., Lisboa, C., Chen, D., Yang, C., & Chen, F.F. (2016). A Brazilian Portuguese survey of school climate: Evidence of validity and reliability. International Journal of School and Educational Psychology, 4, 165-178. Bear, G.G., Chen, D.D., Mantz, L., Yang, C., Huang, X., & Shiomi, K. (2016). Differences in classroom removals and use of praise and rewards in American, Chinese, and Japanese schools. Teaching and Teacher Education, 53(1), 41-50. Mantz, L. S., Bear, G. G., Yang, C., & Harris, A. (2016). The Delaware Social-Emotional Competency Scale (DSECS-S): Evidence of Validity and Reliability. Child Indicators Research. 1-21. Bear, G.G., Yang, C., Mantz, L., & Harris, A. (2017). School-wide practices associated with school climate in elementary, middle, and high school. Teaching and Teacher Education, 63, 372- 383. Bear, G. G., Slaughter, J. C., Mantz, L. S., & Farley-Ripple, E. (2017). Rewards, praise, and punitive consequences: Relations with intrinsic and extrinsic motivation. Teaching and Teacher Education: An International Journal of Research and Studies, 65(1), 10-20.

Peer-Reviewed Journals Bear, G. G., Yang, C., Chen, D., He, X., Xie, J. S., & Huang, X. (2018). Differences in school climate and student engagement in China and the United States. School Psychology Quarterly, 33(2), 323-335. Yang, C., Bear, G. G., & May, H. (2018). Multilevel Associations Between School-Wide Social Emotional Learning Approach and Student Engagement Across Elementary, Middle, and High Schools. School Psychology Review, 47(1), 45-61. Bear, G. G., Harris, A., Saraiva de Macedo Lisboa, C., & Holst, B. (2019). Perceptions of engagement and school climate: Differences between once-retained and multiple- retained students in Brazil. International Journal of School & Educational Psychology, 7(1), 18-27. Harris, A. B., Bear, G. G., Chen, D., de Macedo Lisboa, C. S., & Holst, B. (2019). Perceptions of Bullying Victimization: Differences between Once-Retained and Multiple-Retained Students in Public and Private Schools in Brazil. Child Indicators Research, 1-20.

Additional Resources Delaware School Survey Tool overview FAQ Professional development materials designed to provide information to schools that can lead to improvements in school climate and behavioral outcomes for students. Topics include: Teacher-Student Relations, Student Relations, Bullying Prevention and more! Delaware Positive Behavior Support Project - The DE-PBS Project is a collaborative project with the Delaware Department of Education, the University of Delaware Center for Disabilities Studies, and Delaware Public Schools. This statewide initiative is designed to build the knowledge and skills of Delaware educators in the concepts and practices of a Positive Behavior Support framework as an example of a multi-tiered System of support.