Modern Twists on Rescorla-Wagner Model of Classical Conditioning

Comparative Models of Classical Conditioning Modern Twists on Rescorla-Wagner

Model developed to accurately predict and map learning as it occurs trial by trial Assumptions Assumptions of Rescorla of Rescorla- - Wagner (1974) Wagner (1974) Assumes a bunch of givens: Assume animal can perceive CS and US, and can exhibit UR and CR Helpful for the animal to know 2 things about conditioning: what TYPE of event is coming the SIZE of the upcoming event model model Thus, classical conditioning is really learning about: signals (CS's) which are PREDICTORS for important events (US's)

Assumes that with each CS-US pairing 1 of 3 things can happen: The CS might become more INHIBITORY The CS might become more EXCITATORY There is no change in the CS How do these 3 rules work? If US is larger than expected: CS = excitatory If US is smaller than expected: CS= inhibitory If US = expectations: No change in CS Assumptions of R-W model The effect of reinforcers or nonreinforcers on the change of associative strength depends upon: The existing associative strength of THAT CS AND on the associative strength of other stimuli concurrently present

j> Vsum= Excitatory Conditioning We can use the three rules to make predictions about amount and direction of classical conditioning The degree to which the CS predicted the size of the US was GREATER than expected, so you react MORE to the CS next trial WHY is this WHY is this equation equation important? important? j < Vsum= Inhibitory Conditioning j= Vsum= no change: The CS predicted the size of the US exactly as you expected The degree to which the CS predicted the size of the US was LESS than expected, so you react LESS to the CS next trial

Explaining loss Explaining loss of Associate of Associate What if associative value of US changes across trials? Is an unusual prediction: Associative value of a CS can decrease despite continued pairings with the US Value Value despite despite Show this with three-phase experiment: Phases 1 and 2 CSA US (1 food pellet) CSB US (1 food pellet) Train on separate trials (equal #) pairings with pairings with the US: the US: Phase 3: CSAand CSBpaired together 1 food pellet Same US, so same amount of conditioning, right?? RW model predicts that conditional properties of A and B individually will DECREASE in phase 2 . In a way, the animal only got of what it expected. Thus, this is a DECREASE

Why: Overexpectation effect VA= ; VB= for phase 1; Explaining loss of Explaining loss of Associate Value Associate Value despite pairings despite pairings with the US: with the US: VA+B= how much ? This is an over-expectation effect (we will see this in operant, too!) Got 1 pellet with A Got 1 pellet with B So how many pellets when A+B? Not get 2 pellets, so Phase 2 = decremental conditioning Animal responds less to combined pellets!

Critique of Critique of the the Rescorla Rescorla- - Wagner Wagner Model: Model: R-W model really a theory about the US effectiveness: Says little about CS effectiveness Doesn t really deal with how CS effectiveness may vary How WELL a CS predicts as a combo of salience and probability but doesn t really evaluate this States that an unpredicted US is effective in promoting learning, whereas a well- predicted US is ineffective because nothing is left to learn.

Fails to predict the CS-pre-exposure effect: Critique of the Critique of the Rescorla Rescorla- - Wagner Wagner Model: Model: Two groups of subjects Grp I Grp II Expos ----- CS alone CS-US pairings Control CS-US pairings PRE- Bob and Tom effect Bob always hangs with Tom You are dating Tom You have a BAAAAAD breakup with Tom Now you hate Bob .why?

Critique of the Critique of the Rescorla Rescorla- - Wagner Wagner Model: Model: Original model didn t address how exposure could change salience/learning to CS Incidental CS exposure (before it was a CS) can affect its salience and learn-ability We learn about stimuli even if they aren t predictive We learn they AREN T predictive or ARE predictive We learn who they hang around ..stimuli that group together

In pre-exposure effect, simply being around a neutral stimulus alters its ability to become conditioned Critique of the Critique of the Rescorla Rescorla- - Wagner Wagner Model: Model: Original R-W model doesn't predict any difference, Assumes no conditioning trials occur when CSs presented in absence of US so Vsum= 0 This appears to be wrong Conditioning likely occurring any time 2 stimuli are together Form an incidental association Need to modify the equation to account for this They have, but we won t!

Original R-W model implies that salience is fixed for any given CS R-W assume CS salience doesn't change w/experience More recent data strongly suggest CS salience DOES change w/experience Critique of the Critique of the Rescorla Rescorla- - Wagner Wagner Model: Model: Why changes in CS salience? Data suggest that Salience to a CS DECREASES when CS is repeatedly presented without consequence CS that is accidentally paired with another CS INCREASES in salience Thus: Appears that CS and US effectiveness are both highly important As a result of these types of data: Given birth to attentional models of CC

Attentional models Three Primary Three Primary Extensions and Extensions and Alternatives to Alternatives to RW model RW model Timing and Information models Comparator hypothesis

Question: How well does the CS command attention? Assumes that increased attention facilitates learning about a stimulus Procedures that disrupt attention to CS disrupt learning Attentional Attentional Models of CC Models of CC Different attentional models differ in assumptions about what determines how much attention a CS commands on any given trial Single attentional mechanisms: Kamin s surprise Multiple attentional mechanisms: Current models examine changes in multiple types of attention

Multiple attentional mechanisms: Multiple attentional mechanisms: Assume that the outcome of a given trial alters the degree of attention commanded by the CS on future trials Three attentions: Looking for action: attention a CS commands after it has become a good predictor of the CS Surprise? Then an increase in looking for learning on next trial Looking for learning: how well the organism processes cues that are not yet good predictors of the US, and thus have to be learned about Good predictor? Then increase in attention on next trial Pleasant outcome? Increases emotional value of CS on next trial Looking for liking: the emotional/affective properties of the CS

Recognized that time is important factor in CC: Timberlake, et al 1980 Focal search Responses become conditioned when CS-US interval is short Short CS-US intervals condition responses congruent with a focal search Immediate contact with the CS; food preparation responses Timing/Information Timing/Information Theory Models Theory Models General search Different responses become conditioned when CS-US interval is long Long CS-US intervals condition responses congruent with a general search Longer intervals result in general search responses such as looking, searching, etc. Suggests that organisms learn both WHAT is predicted HOW MUCH is predicted WHEN what is predicted will occur.

Organisms learn when the US occurs in relation to the CS Use this information in blocking, second-order conditioning, etc. Not pay attention to the tone, because it occurs with the light If the tone were to occur before/after the light, then different CS situation Temporal coding Temporal coding What is learned in one phase of training influences what is learned in subsequent phase hypothesis hypothesis Overshadowing Pre-exposure effect Blocking Large literature supports this

Importance of Inter Importance of Inter- -trial interval trial interval More conditioned responding observed with a longer inter-trial interval Inter-trial interval is the time BETWEEN trials If very short; less learning If longer; more learning BUT: Intertrial interval and CS duration (CS-US interval) act in combination to determine responding Critical factor: Relative duration of these two temporal intervals rather than absolute value of either one by itself Need quick CS-US pairing (delayed conditioning) then a longer inter-trial interval Why?

Holland (2000) Conditioned rats to an auditory cue that was presented just before delivery to food CR to CS: Nosing of food cup (goal tracking) THEN .Each group conditioned with: 1 of 2 CS durations: 10 or 20 sec 1 of 6 intertrial intervals: 15 to 960 sec Importance of Importance of Inter Inter- -trial interval interval trial Characterized responses in terms of the ratio of the intertrial interval (I) and the CS duration (T). DV = Time spent nosing the food cup during CS plotted as function of relative value of I/T Results: As I/T ratio increases, the percentage of time the rats spend with the nose in the food cup increases When intertrial interval > Tone interval, spend more time nosing the cup When the intertrial interval < Tone interval, spend LESS time nosing the cup Expecting a new CS-US pairing to occur during long intertrial intervals

Organism is making a comparison between events during the Intertrial interval (I) and Trial interval (T) How long one has to wait for the US during the CS Versus How long one has to wait for the US during the intertrial interval Relative Waiting Time Hypothesis Why is the Why is the Inter Inter- -trial interval so interval so important? important? trial When US waiting time during CS is shorter than intertrial interval: I/T ratio is high: CS is highly informative about the next occurrence of the US Lots of responding When US waiting time during CS is same/longer than the intertrial interval waiting time: I/T ratio is low CS is not highly informative Less responding

Comparitor Comparitor Hypothesis Hypothesis Ralph Miller, et al.: Organism compares across learning situations Assumes: Conditional responding depends on What happens during CS ALSO what happens in other aspects of experimental situation (when CS is not presented) That is, animal compares presence of CS to absence of CS Revaluation effects: Can better explain blocking What is blocked is RESPONDING to CSAB, not learning of CSAB Can unmask blocking to CSABby presenting CSAalone without US (Extinction for CSA). Changes conditional value of CSA Now CSABhas different, predictive value Animal revalues CSABand now responds as it is predictive, whereas CSA is not.

Comparator Hypothesis Comparator Hypothesis Note: This is a theory of PERFORMANCE, not learning! Assumes conditioned responding depends on BOTH associations between CS-US and between US and other stimuli when CS not there These other stimuli form the comparator cues Also assumes formation of excitatory associations with US ONLY Instead of forming inhibitory associations with CS, form stronger excitatory associations with not CS Animal compares the relative associative value of CS vs. not CS Whatever is stronger is what you respond to

Why are these theories important? Why are these theories important? Helps explain the PROCESS of classical conditioning Describes conditions and allows predictions of Who is likely to form associations between stimuli To What kinds of stimuli it will occur When classical conditioning will occur Where associations form: what settings/conditions are important How well: how fast/slow, what quality of associations are formed. Important for predicting/controlling behavior in applied settings Understanding and treatment of pedophilia, sexual fetishes, etc. Phobia treatment Commercials and advertising Social behaviors