Understanding Rescorla Wagner Model in Classical Conditioning

Rescorla Wagner Model Classical Conditioning and prediction

Theories of Classical Conditioning: WHY do organisms respond to predictability? Ruled out Pavlov: Stimulus substitutability theory Perceptual Gating Theory Kamin: Surprise theory: Right, but wrong Provided some really puzzling results Rescorla and Wagner: Computational Model Current Attentional Models

Kamins work: 1967-1974 Blocking and overshadowing Overshadowing: Use one "weak" and one "strong" CS CS1+CS2 US Reaction to weaker stimulus is blotted out by stronger CS Demonstrated by Pavlov CS(strong)+ CS(weak) US(shock) UR(avoidance) CR (avoidance) CR is STRONGEST CS(strong) US(shock) UR(avoidance) CR (avoidance CR is STRONG CS(strong) US(shock) UR(avoidance) CR (avoidance CR is WEAK

Kamins work: 1967-1974 Blocking and overshadowing Blocking: Train 1 CS, then add a second CS to it: CS1(light) US(shock) UR(avoidance) CR(avoidance) CS1(light)+CS2(tone) US(shock) UR (avoidance) CR (avoidance) Test each individually after training CS1(light) US(shock) UR(avoidance) CR(avoidance) CS2(tone) US(shock) UR(avoidance) CR(NO avoidance) Find that only one supports a CR One stimulus blocks learning to second CS Demonstrated by Kamin

The Rescorla Wagner Equation!: Yields an equation: THE Rescorla Wagner (1974) model!!!!! or Vi= i j( j-Vsum) V=k( -V0 Vi= amount learned (conditioned) on a given trial i= the salience of the CS j= the salience of the US ( j-Vsum) = total amount of conditioning that can occur to a particular CS-US pairing

The Rescorla Wagner Equation!: Yields an equation: THE Rescorla Wagner (1974) model!!!!! Vi= i j( j-Vsum) What does this equation say? The amount of conditioning that will occur on a given trial is a function of: The size of the salience of the CS multiplied by The size of the salience of the US multiplied by (The maximum amount of learning) - (the amount of learning that has already occurred).

Can say this easier! How much you will learn on a given trial (Vi) is a function of: ior how good a stimulus the CS is (how well it grabs your attention) jor how good a stimulus the US is (how well it grabs your attention j or how much can learning can be learned about the CS- US relationship AND Vsumor how much you have learned ALREADY!

Assumptions of Rescorla-Wagner (1974) model Model developed to accurately predict and map learning as it occurs trial by trial Assumes a bunch of givens: Assume animal can perceive CS and US, and can exhibit UR and CR Helpful for the animal to know 2 things about conditioning: what TYPE of event is coming the SIZE of the upcoming event Thus, classical conditioning is really learning about: signals (CS's) which are PREDICTORS for important events (US's)

Assumptions of R-W model Assumes that with each CS-US pairing 1 of 3 things can happen: The CS might become more INHIBITORY The CS might become more EXCITATORY There is no change in the CS How do these 3 rules work? If US is larger than expected: CS = excitatory If US is smaller than expected: CS= inhibitory If US = expectations: No change in CS The effect of reinforcers or nonreinforcers on the change of associative strength depends upon: The existing associative strength of THAT CS AND on the associative strength of other stimuli concurrently present

More assumptions Explanation of how an animal anticipates what type of CS is coming: Direct link is assumed between "CS center" and "US center": E.g. between a tone center and food center In 1970 s: other researchers thought R and W were crazy with this idea Now: neuroscience shows formation of neural circuits! Assumes that STRENGTH of an event is given The conditioning situation is predicted by the strength of the learned connection THUS: when learning is complete: The strength of the association relates directly to the size or intensity of the CS Asymptote of learning = max learning that can occur to that size or intensity of a CS Maximum amount of learning that a given CS can support

More assumptions The change in associative strength of a CS as the result of any given trial can be predicted from the composite strength resulting from all stimuli presented on that trial: Composite strength = summation of conditioning that occurs to all stimuli present during a conditioning trial If composite strength is LOW: the ability of reinforcer to produce increments in the strength of component stimuli is HIGH More can be learned for this trial If the composite strength is HIGH: reinforcement is relatively less effective (LOW) Less can be learned for this trial- approaching max of learning

More assumptions: Can expand to extinction, or nonreinforced trials: If composite associative strength of a stimulus compound is high, then the degree to which a nonreinforced presentation will produce a decrease in associative strength of the components is LARGE If composite associative strength is low- nonreinforcement effects reduced

WHY is this equation important? We can use the three rules to make predictions about amount and direction of classical conditioning j> Vsum= Excitatory Conditioning The degree to which the CS predicted the size of the US was GREATER than expected, so you react MORE to the CS next trial j < Vsum= Inhibitory Conditioning The degree to which the CS predicted the size of the US was LESS than expected, so you react LESS to the CS next trial j= Vsum= no change: The CS predicted the size of the US exactly as you expected

The Equation: Lets USE it to Explain Learning, Overshadowing and Blocking!: Vi= i j( j-Vsum) Vi= amount learned (conditioned) on a given trial i= the salience of the CS j= the salience of the US ( j-Vsum) = total amount of conditioning that can occur to a particular CS-US pairing

Okay, you got all that? Let s put this baby to work .. .we will try a few examples

The equation: Vi =ij( j-Vsum) Vi= change in associative strength that occurs for any CS, i, on a single trial i= stimulus salience (assumes that different stimuli may acquire associative strength at different rates, despite equal reinforcement) j= learning rate parameters associated with the US (assumes that different beta values may depend upon the particular US employed) Vsum= associative strength of the sum of the CS's (strength of CS-US pairing) j= associative strength that some CS, i, can support at asymptote In English: How much you learn on a given trial is a function of the value of the stimulus x value of the reinforcer x (the absolute amount you can learn minus the amount you have already learned).

Acquisition FIrst conditioning trial: Assume (our givens) CS = light; US= 1 ma Shock Vsum= Vl; no trials so Vl= 0 Thus: j-Vsum= 100-0 = 100 First trial must be EXCITATORY BUT: must consider the salience of the light: i= 1.0 j= 0.5

Acquisition Plug into the equation: for TRIAL 1 VL1= (1.0)(0.)(100-0) = 0.5(100) = 50 thus: VL1only approaches 50% of the discrepancy between j and Vsumis learned for the first trial

Acquisition TRIAL 2: Same assumptions! VL2 = (1.0)(0.5)(100-50) = 0.5(50) = 25 Vsum= (50+25) = 75

Acquisition TRIAL 3: VL3= (1.0)(0.5)(100-75) = 0.5(25)= 12.5 Vsum= (50+25+12.5) = 87.5

Acquisition TRIAL 4: VL= (1.0)(0.5)(100-87.5) = 0.5(12.5) = 6.25 Vsum= (50+25+12.5+6.25) = 93.75 TRIAL 10: Vsum= 99.81, etc., until reach ~100 on approx. trial 14 When will you reach asymptote?

120 100 80 Amount of Learning 60 Vsum (Total Learning) Vi (Individual Trial Learning) 40 20 0 0 5 10 15 20 25 Trial

Now: Back to Explaining Blocking and Overshadowing Overshadowing: use one "weak" and one "strong" CS reaction to weaker stimulus: less CR Reaction to stronger stimulus: more CR Blocking: 1st CS blocks learning to 2ndCS At issue: What is predicting what? Does LT give any more information/predictability than L alone? If not, then L blocks learning to LT

How to explain overshadowing? Yep, it is good old Rescorla-Wagner to the rescue!

Remember Overshadowing Pavlov: Compound CS with 1 intense CS, 1 weak CS after a number of trials found: strong CS elicits strong CR Weak CS elicits weak or no CR Note: BOTH CSs are presented at same time Why would one over shadow or overpower the other? Why did animal not attend equally to both?

Overshadowing Rescorla-Wagner model helps to explain why: Assume L= light = 0.2; T= tone = 0.5 L= light = 1.0 ; t= tone = 1.0 Plug into equation: Vsum= Vl+ Vt= 0 on trial 1 VL = 0.2(1)(100-0) = 20 Vt= 0.5(1)(100-0) = 50 after trial 1: Vsum= 70

Overshadowing TRIAL 2: VL= 0.2(1)(100-(50+20)) = 6 Vt= 0.5(1)(100-(50+20)) = 15 Vsum= (70+(6+15)) = 91 TRIAL 3: VL = 0.2(1)(100-(91)) = 1.8 Vt = 0.5(1)(100-(91)) = 4.5 Vsum = (91+(1.8+4.5)) = 97.3 and so on thus: reaches asymptote (by trial 6) MUCH faster w/2 CS's NOTE: CSttakes up over 70 units of assoc. strength CSltakes up only 30 units of assoc. strength

Overshadowing R-W explains 2 CS learning 120 100 80 Amt of learning 60 40 20 0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 Trials Vsum for light V sum for tone V sum total

Blocking Similar explanation to overshadowing: Does not matter whether VLhas more or less saliency than Vt, CS has basically absorbed all the associative strength that the CS can support Why?

Blocking Give trials of CS A-alone to asymptote: Reach asymptote: VL= j=100 =Vsum Let s say we did 100 trials to ensure VLis as close to asymptote as possible. NOW add trials to compound stimuli: CS of the light has salience: L=.5465 CS of tone has salience of: t=0.464 Note that CStonehas higher salience! Eh, oh, the math is going to be TOO HARD to do!!!!!

Blocking Or IS the math to hard to do? First compound VLT1Trial: Vt= ( j-Vsum) What is Vsumfor the light after training to the CS light? That s right Vsum= ___________ Vt=0.*1.0*(100-100)= _____________ No learning!

How could one eliminate blocking effect? Increase the intensity of the US to 2 mA with jnow equals = 160 Learning so far: Vsumstill equals 100 (learned to 1 mA shock) But now: TOTAL learning is increased to 160 because we changed the US!

How could one eliminate blocking effect? Plug into the equation: (assume Vland Vtequally salient) Vt= 0.2(1)(160-100) = 0.2(60) = 12 Vl= 0.2(1)(160-100) = 0.2(60) = 12 Vsum= 100+12+12 =124

How could one eliminate blocking effect? on trial 2: Vsum= 124 Vt= 0.2(1)(160-124) = 0.2(36) = 7.2 Vl= 0.2(1)(160-124) = 0.2(36) = 7.2 Vsumnow = (124+14.4) = 138. Again, monotonically increasing curve. Thus, altering the salience of the US alters the learning Does altering the CS make the same or similar change?

The Rescorla Wagner Equation!: Yields an equation: THE Rescorla Wagner (1974) model!!!!! Vi= i j( j-Vsum) What does this equation say? The amount of conditioning that will occur on a given trial is a function of: The size of the salience of the CS multiplied by The size of the salience of the US multiplied by (The maximum amount of learning) - (the amount of learning that has already occurred).

Can say this easier! How much you will learn on a given trial (Vi) is a function of: ior how good a stimulus the CS is (how well it grabs your attention) jor how good a stimulus the US is (how well it grabs your attention j or how much can learning can be learned about the CS- US relationship AND Vsumor how much you have learned ALREADY!