Efficient Learning Outcome Development for STEM Education

ASSESSMENTS & RUBRICS MODIFIED FROM PRESENTATIONS BY ELLEN IVERSON (SERC) AND DAVID STEER (U OF AKRON) This work is supported by the National Science Foundation s Transforming Undergraduate Education in STEM program within the Directorate for Education and Human Resources (DUE-1245025). Questions, contact education-AT-unavco.org

LINKING GOALS AND PROCESS: THE MATERIALS DESIGN RUBRIC A. B. Grand Challenges Interdisciplinary problems (geoscience & social science tied together) Nature and methods of science Authentic geodesy data and inquiry [System thinking] C. D. 1. Guiding Principles 2. Learning Goals and Outcomes 3. Assessment and Measurement 4. Resources and Materials 5. Instructional Strategies 6. Alignment 7. GETSI-specific Instructional Strategies E. Must score 100% - 12/12

Identify Module Learning Goals THE APPROACH Plan Identify teaching & learning outcomes for individual units Instructional Strategies to implement teaching resources Determine how to assess and measure student success on goals and outcomes Design teaching resources and materials to match assessments

LEARNING GOALSAND OUTCOMES What is required from the Materials Development Rubric? Learning outcomes describe measureable geoscience literacy goals Instructions and/or rubrics provide guidance for how students meet learning goals Learning outcomes and goals are appropriate for the intended use of the course/module Learning outcomes and goals are clearly stated for each module in language suitable for the level of the students Learning outcomes and goals address the process and nature of science and development of scientific habits of mind Must score 13/15 on this section

LEARNING OUTCOMES Learning Outcomes are the intended results of the teaching activities Cognitive: What do they know? Affective: What do they care about? Behavioral: What can they do?

WRITING LEARNING OUTCOMES Describe conditions under which behavior is to be performed Use action verbs State Criteria Add the product, process or outcome From Climate Unit: After completing this unit, students will be able to correctly distinguish between forced and unforced climate change.

LEARNING OUTCOMES Learning Outcomes target different levels of learning Mastery: complex tasks likely to have varying levels of progress Developmental: lower level tasks required before moving on

LEARNING GOALSAND OUTCOMES What is required from the Materials Development Rubric? Learning outcomes describe measureable geoscience literacy goals Instructions and/or rubrics provide guidance for how students meet learning goals Learning outcomes and goals are appropriate for the intended use of the course/module Learning outcomes and goals are clearly stated for each module in language suitable for the level of the students Learning outcomes and goals address the process and nature of science and development of scientific habits of mind Must score 13/15 on this section

LINKING GOALSAND PROCESS: THE MATERIALS DESIGN RUBRIC 1. Guiding Principles 2. Learning Goals and Outcomes 3. Assessment and Measurement 4. Resources and Materials 5. Instructional Strategies 6. Alignment

ASSESSMENTSAND MEASUREMENTS What is required from the Materials Development Rubric? Assessments measure the learning goals Assessments are criterion referenced Assessments are consistent with course activities and resources expected Assessments are sequenced, varied and appropriate to the content Assessments address goals at successively higher cognitive levels Must score 13/15 on this section

ASSESSMENTSAND MEASUREMENTS There are two broad categories of assessments: Formative: While the learning is occurring Purpose to monitor student learning Immediate feedback Help students & faculty identify weaknesses Low stakes Summative: After learning has occurred Purpose to evaluate learning against some benchmark High stakes (graded)

ASSESSMENTSAND MEASUREMENTS What are formative and summative assessment strategies that you find most effective? FORMATIVE SUMMATIVE

LOOKATSOMECURRENT GETSI EXAMPLES

ASSESSMENTSAND MEASUREMENTS What will the assessment team need? Enough assessment opportunities to conclusively demonstrate the level of learning achieved Unit-level (learning outcomes) Both formative and summative Module-level (module goals) SUMMATIVE These assessments need to show what students know and are able to do as related to the broader goals

LEARNING GOALSAND OUTCOMES What is required from the Materials Development Rubric? Learning outcomes describe measureable geoscience literacy goals Instructions and/or rubrics provide guidance for how students meet learning goals Learning outcomes and goals are appropriate for the intended use of the course/module Learning outcomes and goals are clearly stated for each module in language suitable for the level of the students Learning outcomes and goals address the process and nature of science and development of scientific habits of mind Must score 13/15 on this section

ASSESSMENTSAND MEASUREMENTS What is required from the Materials Development Rubric? Assessments measure the learning goals Assessments are criterion referenced Assessments are consistent with course activities and resources expected Assessments are sequenced, varied and appropriate to the content Assessments address goals at successively higher cognitive levels Must score 13/15 on this section

WHATARE SCORING RUBRICS? Scoring rubrics are descriptive scoring schemes used to evaluate effort Two Major Types Holistic: set of descriptions used to assign a score to the whole Analytic: Set of components that are independently evaluated (sum for score)

DESIGNING RUBRICS Determine the most important attributes needed to evaluate the Learning Outcome Decide analytic or holistic Define levels Success Part Way There Needs Work

GOOD RUBRICS Have clear criteria Each criteria is distinct, clearly delineated and fully appropriate for the outcome Have distinctive levels Each level is distinct and progresses in a clear and logical order Can be used reliably by multiple raters Provide guidance to learners Rubric serves as primary reference point for discussion and guidance as well as evaluation of assignment(s) Support Metacognition Rubric is regularly referenced and used to help learners identify the skills and knowledge they are developing throughout the program

HOLISTIC EXAMPLE Outcome: Students will demonstrate the ability to properly process and interpret data 0 1 2 3 4 Unacceptable Poor Acceptable Good/Solid Exemplary Score Student unable to process or interpret data sets without significant errors. Student processes and interprets data sets with significant errors. Student processes and interprets data sets with some errors. Student properly processes and interprets each data set. Student processes and interprets data sets with only minor errors.

ANALYTICAL EXAMPLE Outcome: Material developers will demonstrate the ability to incorporate GETSI Guiding Principles in their curriculum