Survey Administration and Feedback for Teacher Preparation Programs at University of South Florida

College of Education University of South Florida Annual Survey Administration Alumni & Employer Surveys: Initial Teacher Preparation Programs & Advanced Graduate Programs

These surveys were designed to capture the perspectives of our alumni and their school principals. Initial Teacher Preparation Programs Alumni and Principal Surveys The surveys contain items employing a 5-point Likert scale (with 1=Strongly disagree, 2=Disagree, 3=Neither agree nor disagree, 4=Agree, and 5=Strongly agree). These surveys are parallel in nature and are administered annually. The data are provided to program faculty alongside other important data points in order to triangulate the results and examine trends across time. Team members in the Office of Continuous Improvement Support (OCIS) meet with program faculty to do a deep dive into other data sources and collaboratively develop distinct student-learner and program outcomes.

The survey questions are closely articulated with the InTASC standards measuring: 1. Learner Development 2. Learning Differences 3. Learning Environments 4. Content Knowledge 5. Application of Content 6. Assessment 7. Planning for Instruction 8. Instructional Strategies 9. Professional Learning and Ethical Practice 10. Leadership and Collaboration Initial Teacher Preparation Programs Alumni and Principal Surveys

Initial Teacher Preparation Programs Alumni and Principal Surveys The survey data are aggregated and reported by program and degree level across the four InTASC Domains: 1. The Learner & Learning 2. Content Knowledge 3. Instructional Practice 4. Professional Responsibility

Initial Teacher Preparation Programs Alumni and Principal Surveys Sample Selection For this reporting period, we selected 2020-2021 and 2021-2022 graduates employed by Florida public schools during the 2022-2023 school year and their principals.

What our principals say: A sample of responses Initial Teacher Preparation Programs Alumni and Principal Surveys She has been a wonderful addition to our school's family. She works great with her team, colleagues, students, and families. We are thrilled to have her as part of our team. In her 3rd year of teaching, she won our Teacher of the Year award this year. She is an amazing educator that is an example to all others. With the partnership of USF and very strong mentor teachers, I feel like we have found the winning combination to prepare new teachers!! What a great year with our interns and USF!!!

Initial Teacher Preparation Programs Alumni and Principal Surveys Descriptive Statistics: Alumni Survey InTASC Domain Domain 1: The Learner & Learning Domain 2: Content Knowledge Domain 3: Instructional Practice Domain 4: Professional Responsibility N Mean 167 167 167 167 SD 0.5 0.6 0.6 0.6 Min 3.0 2.0 2.2 2.0 Max 5.0 5.0 5.0 5.0 4.5 4.4 4.4 4.4 Descriptive Statistics: Principal Survey N Mean 4.2 4.2 4.1 4.2 SD 0.9 0.9 0.9 0.9 Min 1.0 1.0 1.0 1.0 Max 5.0 5.0 5.0 5.0 InTASC Domain Domain 1: The Learner & Learning Domain 2: Content Knowledge Domain 3: Instructional Practice Domain 4: Professional Responsibility 320 320 320 320

Initial Teacher Preparation Programs Alumni and Principal Surveys Initial Teacher Preparation Programs InTASC Domains 5.0 4.0 3.0 2.0 1.0 Domain 1: The Learner & Learning Domain 2: Content Knowledge Domain 3: Instructional Practice Domain 4: Professional Responsibility Alumni Principal

Initial Teacher Preparation Programs Alumni and Principal Surveys Initial Teacher Preparation Principal Survey The Educator Preparation Program (EPP) that prepared this teacher is responsive to district needs Percent of respondents reporting agree or strongly agree 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2020 2021 2022 2023 2024

Initial Teacher Preparation Programs Alumni and Principal Surveys Initial Teacher Preparation Programs: Alumni Survey The results were overwhelmingly positive, with many of our Initial Teacher Preparation Program completers rating their performance as quite well or extremely well on the majority of the survey items. The only area where our Initial Teacher Preparation Program completers rated themselves less than 90%, as performing quite well or extremely well was Planning instruction that supports every student in meeting rigorous learning goals (87%). Initial Teacher Preparation Programs: Principal Survey The results were overwhelmingly positive, with many of our School Principals rating our program completers as quite well or extremely well on the majority of the survey items. The only areas where our Principals rated our program completers less than 85%, as performing quite well or extremely well was Planning instruction that supports every student in meeting rigorous learning goals and Adapting his/her practice to meet the needs of each learner (considering learners, families, other professionals, and the community) both 83%, and Using multiple methods of assessment to engage learners in their own growth (84%). The results across this set of surveys were very similar.

Initial Teacher Preparation Programs Alumni and Principal Surveys Psychometrically Speaking: Alumni Survey Cronbach s alpha was estimated to be .93, suggesting a high level of internal consistency (reliability) Correlations between InTASC domains ranged from .64 to .90 demonstrating evidence of the construct validity of the inferences made based on these ratings Pearson Correlation Coefficients, N = 167 Prob > |r| under H0: Rho=0 Domain 1 The Learner & Learning Domain 2 Content Knowledge Domain 3 Instructional Practice Domain 4 Professional Responsibility Domain 1: The Learner & Learning 1.00 Domain 2: Content Knowledge 0.73 1.00 <.0001 Domain 3: Instructional Practice 0.77 0.90 1.00 <.0001 <.0001 Domain4: Professional Responsibility 0.68 0.64 0.71 1.00 <.0001 <.0001 <.0001

Initial Teacher Preparation Programs Alumni and Principal Surveys Psychometrically Speaking: Principal Survey Cronbach s alpha was estimated to be .98, suggesting a very high level of internal consistency (reliability) Correlations between InTASC domains ranged from .86 to .97 demonstrating very strong evidence of the construct validity of the inferences made based on these ratings Pearson Correlation Coefficients, N = 320 Prob > |r| under H0: Rho=0 Domain 1 Domain 2 Domain 3 Domain 4 The Learner & Learning Content Knowledge Instructional Practice Professional Responsibility 1.00 Domain 1: The Learner & Learning Domain 2: Content Knowledge 0.91 1.00 <.0001 0.93 <.0001 0.91 <.0001 Domain 3: Instructional Practice 0.97 1.00 0.89 <.0001 0.86 <.0001 <.0001 1.00 Domain 4: Professional Responsibility 0.89 <.0001

These surveys were designed to capture the perspectives of advanced preparation program graduates and their employers. The surveys contain items employing a 5-point scale (with 1=Not well at all, 2=Slightly well, 3=Somewhat well, 4=Quite well, and 5=Extremely well). These surveys are parallel in nature and are administered annually. The data are provided to program faculty alongside other important data points in order to triangulate the results and examine trends across time. Team members in the Office of Continuous Improvement Support (OCIS) meet with program faculty to explore other potentially useful data sources. Advanced Graduate Programs Alumni and Employer Surveys

The survey questions are closely articulated with the CAEP standards for Advanced Graduate Programs measuring: 1. Applications of data literacy 2. Use of research and understanding of qualitative, quantitative and/or mixed methods research methodologies 3. Employment of data analysis and evidence to develop supportive school environments 4. Leading and/or participating in collaborative activities with others such as peers, colleagues, teachers, administrators, community organizations, and parents 5. Supporting appropriate applications of technology for their field of specialization 6. Application of professional dispositions, laws and policies, codes of ethics and professional standards appropriate to their field of specialization Advanced Graduate Programs Alumni and Employer Surveys

Advanced Graduate Programs Alumni and Employer Surveys Sample Selection For this reporting period, we selected 2020-2021 and 2021-2022 graduates employed by Florida public schools during the 2022-2023 school year and their employers.

What our employers say: A sample of responses She has become an excellent Assistant Principal for Hillsborough County Public Schools. She will make an outstanding principal. Advanced Graduate Programs Alumni and Employer Surveys Communication is her strength. She does an amazing job working with teachers and plays a key role in the academic success of our students. She is a reflective thinker and a better action planner.

Advanced Graduate Programs Alumni and Employer Surveys Descriptive Statistics: Advanced Graduate Programs Alumni Survey CAEP Advanced Graduate Programs Domain Domain 1: Applications of data literacy Domain 2: Understanding research Domain 3: Data analysis for supportive environments Domain 4: Collaboration Domain 5: Applications of technology Domain 6: Ethics N Mean 4.4 3.9 4.4 4.4 4.4 4.6 SD 0.8 0.8 0.8 0.6 0.9 0.6 Min 1.1 2.0 1.7 3.0 2.0 2.5 Max 5.0 5.0 5.0 5.0 5.0 5.0 40 37 37 37 37 37 Descriptive Statistics: Advanced Graduate Programs Employer Survey CAEP Advanced Programs Domain Domain 1: Applications of data literacy Domain 2: Understanding research Domain 3: Data analysis for supportive environments Domain 4: Collaboration Domain 5: Applications of technology Domain 6: Ethics N Mean SD 0.9 1.0 0.8 1.0 0.8 0.8 Min 1.7 1.0 2.0 2.0 2.0 2.0 Max 5.0 5.0 5.0 5.0 5.0 5.0 54 51 54 54 54 54 4.2 4.0 4.4 4.2 4.4 4.5

Advanced Graduate Programs Alumni and Employer Surveys Advanced Graduate Programs CAEP Advanced Standards 5.0 4.0 3.0 2.0 1.0 Domain 1: Applications of data literacy Domain 2: Domain 3: Data analysis for supportive environments Domain 4: Collaboration Domain 5: Applications Domain 6: Ethics Understanding research of technology Alumni Employer

Advanced Graduate Programs Alumni and Employer Surveys Advanced Graduate Programs: Alumni Survey The results were overwhelmingly positive, with many of our Advanced Graduate Programs Alumni rating their performance as quite well or extremely well on many of the survey items. The only area where our Advanced Graduate Programs Alumni rated themselves less than 75%, as performing quite well or extremely well was with respect to Understanding qualitative research methods and Understanding mixed research methods (i.e., integration of both qualitative and quantitative research methods) both 73%. Advanced Graduate Programs: Employer Survey Once again, the results were overwhelmingly positive, with many of our Advanced Graduate Programs Employers rating our alumni as performing quite well or extremely well on most of the survey items. The only area where our Advanced Graduate Programs Employers rated our alumni less than 75% as performing quite well or extremely well was with respect to Understanding qualitative research methods and Understanding mixed research methods (i.e., integration of both qualitative and quantitative research methods) both 71%. These results are parallel in nature.

Advanced Graduate Programs Alumni Survey Psychometrically Speaking: Alumni Survey Cronbach s alpha was estimated to ranged from .68 to .97, suggesting an acceptable level of internal consistency across the six domains (reliability) Correlations between CAEP domains ranged from .31 to .83 demonstrating some degree of construct validity of the inferences made based on these ratings Pearson Correlation Coefficients N=37 Prob > |r| under H0: Rho=0 Domain 1 Domain 2 Use of research Domain 3 Data analysis for supportive environments Domain 4 Collaboration Domain 5 Applications of technology Domain 6 Ethics Applications of data literacy Domain 1: Applications of data literacy 1.00 Domain 2: Use of research 0.45 0.01 0.74 1.00 Domain 3: Data analysis for supportive environments 0.65 1.00 <.0001 0.60 <.0001 0.66 <.0001 0.54 0.00 <.0001 0.31 0.07 0.36 0.03 0.37 0.02 Domain 4: Collaboration 0.55 0.00 0.61 1.00 Domain 5: Applications of technology 0.83 1.00 <.0001 0.65 <.0001 Domain 6: Ethics 0.46 0.00 0.50 0.00 1.00

Advanced Graduate Programs Employer Survey Psychometrically Speaking: Employer Survey Cronbach s alpha was estimated to ranged from .92 to .98, suggesting a high level of internal consistency across the six domains (reliability) Correlations between CAEP domains ranged from .69 to .89 demonstrating construct validity of the inferences made based on these ratings Pearson Correlation Coefficients N=54 Prob > |r| under H0: Rho=0 Domain1 Domain2 Use of research Domain3 Data analysis for supportive environments Domain4 Collaboration Domain5 Applications of technology Domain6 Ethics Applications of data literacy Domain 1: Applications of data literacy 1.00 Domain 2: Use of research 0.89 1.00 <.0001 0.88 <.0001 0.89 <.0001 0.76 <.0001 0.75 <.0001 Domain 3: Data analysis for supportive environments 0.81 1.00 <.0001 0.79 <.0001 0.69 <.0001 0.69 <.0001 Domain 4: Collaboration 0.84 1.00 <.0001 0.87 <.0001 0.82 <.0001 1.00 Domain 5: Applications of technology 0.77 <.0001 0.78 <.0001 Domain 6: Ethics 0.77 1.00 <.0001

College of Education University of South Florida Annual Survey Administration Next Steps: We are pleased with the overwhelmingly positive results on this set of surveys We will continue to collect data and as our sample sizes grow with each survey administration, we will be better positioned to disaggregate our data to examine trends over time and continue to examine the psychometric characteristics of these data We have asked our faculty, staff, and program coordinators to reach out to recent graduates and encourage them to provide their valuable feedback on our surveys so that we can continue to improve our programs