Assessing Elementary Students Functional Thinking Skills: The Case of Function Tables

Assessing Elementary Students Functional Thinking Skills: The Case of Function Tables Katherine L. McEldoon & Bethany Rittle-Johnson

Project Goals Develop an assessment of elementary students functional thinking abilities, an early algebra math skill Develop a model of knowledge progression 2

Functional Thinking The table shows how the In numbers are related to the Out numbers. When a 38 goes in, what number comes out? A.41 B.51 C. 54 D. 77 A type of mathematical thinking which focuses on the relationship between two (or more) varying quantities, specifically the kinds of thinking that lead from specific relationships to generalizations of that relationship across instances. (Smith, 2008) 77 Out = (In x 2) + 1 Out = (In x 2) + 1 Y = 2X + 1 Y = 2X + 1 Encapsulates important core components of early algebraic reasoning, such as generalization generalization and covariation. (Carraher, Martinez, & Schliemann, 2008) covariation. 3

Functional Thinking Performance Grade 4 The table shows how the In numbers are related to the Out numbers. When a 38 goes in, what number comes out? A.41 B.51 C.54 D.77 National Assessment of Educational Progress (NAEP), National Performance results in Mathematics at Grade 4; 2007 4

Function Tables Focus: Functional Tables Determining Values and Rules Column Column A A 2 Column Column B B 6 Typical Tasks Typical Tasks (Carraher & Earnest, 2001; Schliemann & Carraher, 2000) 3 7 4 8 Fill in the missing values in this table 5 9 6 10 10 14 18 18 What is the rule for this table? Asked to select a rule from several choices Asked to write the rule verbally or symbolically 21 25 25 41 45 45 5

Function Table Competencies Within function table problems, we isolated required competencies and used this as a basis for our assessment Column Column A A 2 Column Column B B 6 3 7 Loosely hypothesized order of difficulty: 1. Apply a Given Rule (prerequisite) 2. Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule (verbal/symbolic) 6. Generate Rule Verbally 7. Generate Rule Symbolically 4 8 5 9 6 14 21 41 Column B = Column A + 4 Column B = Column A + 4 B = A + 4 B = A + 4 6

Wilsons Construct Modeling Approach Item Desig n Construct Map Measure ment Model Item Score Wilson s Four Building Blocks 1) Construct Map 2) Item Design 3) Item Score 4) Measurement Model Assess the student performance data to evaluate your construct map and items 7

Item Design: Assessment We designed items that tapped each of these competencies Modified from e.g. Blanton; Schliemann; Warren, Cooper & Lamb Items varied in operation used in underlying function 33 33 responses to 11 11 items 16 16 of which had an additive additive underlying function Y = X + 2 10 10 had a combination combination underlying function Y = 2X + 2 8

Item Design: Assessment We developed items that tapped each of these competencies 1. 1. Apply a Given Rule Apply a Given Rule 2. Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically 9

Item Design: Assessment We developed items that tapped each of these competencies 1. Apply a Given Rule 2. 2. Determine Next Y value in Sequence Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically Column Column A A 2 Column Column B B 6 3 7 4 8 5 9 6 14 21 41 10

Item Design: Assessment We developed items that tapped each of these competencies 1. Apply a Given Rule 2. Determine Next Y value in Sequence 3. 3. Determine Near Y value in Sequence Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically Column Column A A 2 Column Column B B 6 3 7 4 8 5 9 6 14 21 41 11

Item Design: Assessment We developed items that tapped each of these competencies 1. Apply a Given Rule 2. Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. 4. Determine Far Y value in Sequence Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. Generate Rule Symbolically Column Column A A 2 Column Column B B 6 3 7 4 8 5 9 6 14 21 41 12

Item Design: Assessment We developed items that tapped each of these competencies 1. Apply a Given Rule 2. Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. 5. Recognize a Rule Recognize a Rule Verbal & Symbolic 6. Generate Rule Verbally 7. Generate Rule Symbolically What is a rule used in the table above to get the numbers in column B from the numbers in column A? Verbal & Symbolic A) B) Multiply the number in column A by 2. Divide the number in column A by 2. Subtract 2 from the number in column A. Add 2 to the number in Add 2 to the number in column A. column A. C) D) D) 13

Item Design: Assessment We developed items that tapped each of these competencies 1. Apply a Given Rule 2. Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. 6. Generate Rule Verbally Generate Rule Verbally 7. Generate Rule Symbolically Column Column A A 2 Column Column B B 6 3 7 4 8 5 9 6 14 21 41 What is a rule for figuring out what number belongs in column B? The rule is that you add 4 to the The rule is that you add 4 to the A number to get the B number A number to get the B number 14

Item Design: Assessment We developed items that tapped each of these competencies 1. Apply a Given Rule 2. Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule 6. Generate Rule Verbally 7. 7. Generate Rule Symbolically Generate Rule Symbolically Column Column A A 2 Column Column B B 6 3 7 4 8 5 9 6 14 21 41 Write this rule as a number sentence, using A to stand for any number in column A and B to stand for any number in column B. B = A + 4 B = A + 4 15

Item Scores: Coding Coding Each response only tapped one competency Each was coded as correct or incorrect Item Desig n Construct Map Measure ment Model Item Score 16

Data Collection: Procedure 231 second through sixth grade students Middle class suburban community Predominantly Caucasian population During one 40 minute class period 17

Measurement Model Based on Item Response Theory Item Response Theory Item Response Theory encompasses a set of ways to mathematically model how both Student Ability Estimate Estimate and Item Difficulty Item Difficulty are related to a student s Item Responses Item Responses Student Ability It is a useful methodology to use when evaluating an assessment instrument an assessment instrument both in terms of its ability to accurately estimate student ability student ability but it also give metrics of the quality of each item the instrument. evaluating accurately estimate quality of each item on 18

Measurement Model Wright Map Student Ability Scores Student Ability Scores Item Difficulty Scores Item Difficulty Scores --------------------------------------------------------------- 5 XX| XXXX| XX| 4 XXXXXX| XXXXXXXX| XXXXXX| XXXXXXXXXXXXXXXX| 3 XXXXXXXXXXXX| XXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXX| XXXXXXXXXXX| 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L4 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 0 XXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L3 XXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|L2 L2 -2 XXXXXXXXXXXXXXXXXXXX|L2 L2 L2 XXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXX| -3 XXXXXXXXXXXXXX| XXXXXXX|L1 L1 L1 XXXX| XXX|L1 L1 -4 X| X| XX| | ================================================================== An Wright map generated by a Rasch model (a type of item response model) and was used in this evaluation Logit Scale (log-odds ratio) Student Ability Estimates Item Difficulties 19

Measurement Model Wright Map Wright Map Wright Map An Wright map was generated by a Rasch model (a type of item response model) and was used in this evaluation Item difficulties based on the Wright maps were used in the development of our Construct map Student Ability Scores Student Ability Scores Item Difficulty Scores Item Difficulty Scores This item difficulty is 3.1logits, or the average student has a ~0.28 probability of getting it correct XXXXXXXX| XXXXXXXXXXX|Item E 3 XXXXX-3-Item C Item D XXXXXXXXXX| XXXXXXXXXXXX| 2 XXXXXXXXX-2- XXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|Item B XXXXXXXXXXXXXXXXXXXXXXXXXX| 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|Item A This item has a difficulty level of .98, meaning that the average student has a ~0.47 probability of getting it correct 20

Wright Map Addition Functions XXXXXXXXXXXXXXXXXX| XXXXXXXXXXX| 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|7-16 XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX|6-6 0 XXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|6-14 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-15 -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-13 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|3-12 XXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|3-4 3-5 -2 XXXXXXXXXXXXXXXXXXXX|5a-9 2-10 2-11 XXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXX| -3 XXXXXXXXXXXXXX| XXXXXXX|1-3 1-7 1-8 XXXX| XXX|1-1 1-2 -4 X| X| XX| ======================================================================================= Each 'X' represents 0.4 cases ======================================================================================= 7) Generate Rule Symbolically 6) Generate Rule Verbally 5) Recognize a Rule 4) Determine Far Y value in Sequence 3) Determine Near Y value in Sequence 2) Determine Next Y value in Sequence 1) Apply a Given Rule 21

Wright Map Combination Functions --------------------------------------------------------------------------------------- 4 XXXXXXXXX| XXXXXXXX| XXXXXXXXXXX|4-8 3 XXXXX|3-6 4-7 6-9 7-10 XXXXXXXXXX| XXXXXXXXXXXX| 2 XXXXXXXXX| XXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXX| 1 XXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|2-5 XXXXXXXXXXXXXXXXXXXXXXXXXX| 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5b-4 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5a-3 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| -2 XXXXXXXXXXXXXXXXXXXXXXXXXXX| XXXXXXXXXXXXXXXXXXXXXXXXXXX|1-2 XXXXXXXXXXXXXXX| -3 XXXXXXXXX| XXXXXXXXXXX|1-1 XXXXXXX| -4 XXXXXXXXX| ======================================================================================= Each 'X' represents 0.4 cases 7) Generate Rule Symbolically 6) Generate Rule Verbally 5) Recognize a Rule 4) Determine Far Y value in Sequence 3) Determine Near Y value in Sequence 2) Determine Next Y value in Sequence 1) Apply a Given Rule 22

Construct Map Construct Map A representation of the continuum of knowledge that people are thought to progress through for the target construct (Wilson, 2005) Placed competencies into a hierarchy based on We used item difficulty scores from IRT measures Their clumping on the Wright maps From theory 23

Mapping of Competencies into Construct Map Levels Level Level Description Description Competencies Competencies 7. Generate Rule Symbolically 6. Generate Rule Verbally - Generate an explicit symbolic rule Level 4: Level 4: Generate Symbolic Rule Level 3: Level 3: Generate & Use Verbal Rule 5. Recognize a Rule (verbal/symbolic) - Generate an explicit verbal rule - Complete a function table with missing values 4. Determine Far Y value in Sequence - Select a correct rule out of several choices - Determine the next Y value in a function sequence Level 2: Level 2: Recognize Rule & Determine Next 3. Determine Near Y value in Sequence 2. Determine Next Y value in Sequence - Use a given rule to determine new Y values Level 1: Level 1: Apply Rule 1. Apply a Given Rule 24

Benefits of a Construct Modeling Approach First, it elucidated the relative difficulty of functional thinking abilities, and at times this was not in line with our predictions. Second, the resulting assessment is a criterion referenced measure which is particularly appropriate for assessing Students ability estimate levels Learning gains from an intervention 25

Summary Identified key competencies that are important for elementary-level functional thinking, with a focus on function table problems These competencies were then incorporated into an assessment Student performance data was used to develop a construct map, or proposed knowledge progression, of elementary-level functional thinking abilities The resulting construct map provided insight into the acquisition of functional thinking knowledge in elementary-school students This can be used as a research tool, and to guide This can be used as a research tool, and to guide instructional sequences for students instructional sequences for students 26

Thank you Thank you For more information: http://peabody.vanderbilt.edu/earlyalgebra.xml http://peabody.vanderbilt.edu/earlyalgebra.xml The first author is supported by a predoctoral training grant provided by the Institute of Education Sciences, U.S. Department of Education, through Grant R305B040110 to Vanderbilt University. The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education. 27

28

Wright Map - Multiplication 1. Apply a Given Rule 2. Determine Next Y value in Sequence 3. Determine Near Y value in Sequence 4. Determine Far Y value in Sequence 5. Recognize a Rule (symbolic) 6. Generate Rule Verbally 7. Generate Rule Symbolically --------------------------------------------------------------------------------------- 5 XXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXX| | 4 XXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | 3 XXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|7-7 XXXXXXXXXXXXXXXXXX|3-4 2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXX|4-3 3-5 XXXXXXXXXXXXXXXXXX|6-6 1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|5-1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | 0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX|2-2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -1 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | -2 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| | XXXXXXXXXXXXXXXXXXXXXXXXXX| -3 XXXXXXXXXXXXXXXXXXXXXXXXXX| | ======================================================================================= Each 'X' represents 0.3 cases ======================================================================================= 29