L&M Test I
Is there learning in simultaneous conditioning?
No
Is temporal contiguity (repeated CS-US pairings) enough to establish a connection between two stimuli?
No - contingency (the establishment of a contingent relationship) is necessary
Blocking (conditioning)
Occurs when a previous association prevents another association from being formed (first association tells you all you need to know to predict an outcome, therefore making it unnecessary for the animal to learn that a new stimulus is also predictive of reward - animal thus does not make that association).
How could CR timing be mediated by circuit that makes an engram?
One hypothesis: "delay line" - plasticity occurs only for CS-carrying cells that fire at the right time
How does surprise seem to be a key element of conditioning?
Surprise causes the animal to adjust their expectations and update their learning models. It is not enough that an unconditioned stimulus is contingent on a conditioned stimulus - you have to be surprised that the unconditioned stimulus occurred.
Trace conditioning
Temporal interval between CS and US
"Engram found" - experiment to gauge implication of cerebellum
- Experimenters inflicted a lesion to the part of the cerebellum of rabbits thought to be involved in eyeblink conditioning to observe the effect on behavior; - After the lesion, the unconditioned response is fine (meaning that destroying that area of the cerebellum didn't affect the reflex), but the conditioned response is GONE (lesion effectively destroyed memory of conditioned response); - When you attempt to retrain the rabbit to re-learn the conditioned response after the lesion, the eye that is contralateral to the lesioned side of the cerebellum is able to be retrained [laterality shows that there is a motor component to this learning].
Classical conditioning in honeybees
- Honeybees can be classically conditioned to protrude their proboscis when they sniff the aroma of explosive chemicals. - They are exposed to the particular odour and then rewarded with a sugar solution. Within five minutes, they learn to associate the smell with an impending food supply and this triggers the proboscis extension reflex.
"Ingredients" necessary for learning during eyeblink conditioning
- Puff information (unconditioned stimulus) - Tone information (conditioned stimulus) - Plasticity - Timing (neurons need to keep track of time between tone and puff - if you blink too early you'll get puffed in the eye)
Mauk et al 1986 - How to make an engram; what did this experiment show?
- Showed that conditioned eyelid responses develop when unconditioned stimulus is electrical stimulation (using electrode) of the inferior olive - generates blink AND could be a motor learning event. - Stimulus behaves like a real learning experience - animal learns to predict it in the real world. (When compared to conditioning using a standard unconditioned stimulus - air puff - the conditioning produced in this manner appears quite normal: responses develop at a similar rate, are of comparable magnitude and topography...)
Hesslow et al 1999 - How to make an engram; what did this experiment show?
- Suggested that the CS in classical conditioning reaches the cerebellum via the mossy fiber system; - Ferrets eye-blink conditioned to respond with blinks to forelimb stimulus - when direct mossy fiber stimulation was given, animals responded with conditioned blinks immediately (without ever having been trained to the mossy fiber stimulation) - Hinted that memory trace is probably located in cerebellar cortex.
2 different possible kinds of reinforcement
1. Something good happens 2. Something bad is removed
2 basic concepts underlying the Rescorla-Wagner model
1. The associative strength ("value") of a given CS-US relationship is updated based on the difference between the predicted and observed outcomes ("prediction error"). 2. The prediction on a given trial is determined by the sum of all currently present CSs.
Operant conditioning
A type of learning in which behavior is strengthened if followed by a reinforcer or diminished if followed by a punisher. Behavior that maximizes reward (generalized positive outcome) or minimizes punishment (generalized negative outcome)
Second order conditioning
A type of learning where a CS is paired with a stimulus that became associated with the US in an earlier procedure [i.e.: learning that light predicts another stimulus that predicts salivation; "stimulus-stimulus" learning]
Phases of spontaneous recovery
A. CS-US learning in Context X: relationship that the CS carries information about probability of puff. B. Extinction in Context Y: learns that tone, which had always predicted the puff, now predicts nothing. C. Renewal in Context X: recovery. Animal recovers CS-US association. [Could be associated to other neural circuits - neural system responsible for basic associative learning is not the same as the one involved in this recovery]
Two features of conditioning
Acquisition and extinction
What is learning like when backward conditioning is followed by classical conditioning?
Animal doesn't learn [Backward conditioning tells you the end of the meaningful event, not how to best predict it. If you then try to condition the animal to interpret the second stimulus as the predictive one, it takes a while for them to establish that connection - informational content is at conflict]
How do animals with damage to the dorso-lateral striatum respond to habitual learning?
Animals with lesions (damage) to the dorso-lateral striatum never develop habits despite extensive training
What kind of learning does long-delay conditioning produce?
Attenuated leraning
Rescorla-Wagner as a case study in blocking
Blocking happens when a novel stimulus - which has no predictive value, in light of its novelty - is presented along with a well-established CS (whose predictive value is essentially equal to λ, that is, 1). Neither the well established CS nor the novel stimulus will change their predictive value because no surprise occurs: the two stimuli together totally predict the US because their Vs sum to λ. (λ - Vt) = 1 - (1 + 0) = 1 - 1 = 0 No surprise means no learning.
Delay conditioning
CS precedes US, but they overlap in time
Shortcoming of Hull's Systematic Theory of Behavior
Cannot explain blocking/compound conditioning/overshadowing
Role of the cerebellum in classical conditioning & eyeblink response
Cerebellum is thought to be where memories learned for motor responses (such as conditioned eyeblink responses) are stored.
Extinction (conditioning) - learning curve behavior
Descending slope in curve - rabbit stops exhibiting the conditioned response
Latent Inhibition
Difficulty in establishing classical conditioning to a conditioned stimulus we've repeatedly experienced alone - that is, without the unconditioned stimulus. ["Familiar stimulus takes longer to acquire meaning than new stimulus". Inhibition effect due to relatively poor learning. e.g. Group exposed to tone without puff (before learning) learns much slower than group that had not been exposed to the tone.]
Role of dopamine in goal-directed learning
Dopamine gives you the salient incentive to go accomplish a goal and get a reward, but its not itself communicating a pleasure signal to your brain about a certain reward
Backward conditioning
Flipping the normal conditioning procedure such that the unconditioned stimulus precedes the conditioned stimulus
Goal-directed vs habitual behavior (learning stages)
Goal-directed behavior dominates early in learning, habitual behavior dominates later in learning
What concept informed Tolman's work?
Idea that if you train a mouse on a maze to their left, and then insert it on the maze to the right, it will not be making the same turns. Instead, it will be going in the direction of the food. Mouse thus clearly understands the relationship between the direction it is heading and the location of the food.
Tolman's position on establishing associations between stimuli and outcomes
Idea that relationship between the stimulus and the outcome is being mediated by a cognitive map. In this way, response is not being directly informed by the stimulus, but by the goal that the stimulus will accomplish for you.
Temporal discounting
In decision making, the greater weight given to the present over the future - rewards that are farther away in time are worth less
What is the difference between delay and trace conditioning?
In delay, there is some overlap between CS and US; in trace, there is a wider interval between CS and US
Can we see blocking in the brain?
Inferior olive provides ERROR SIGNAL. During conditioning, the cerebellum INHIBITS the olive (shuts off the expected error signal) - if you stop this inhibition, BLOCKING doesn't occur.
What does Clark Hull's Systematic Theory of Behavior summarize?
It sums up the process of learning in animals to measure degree of conditioning expected to occur in a given situation.
What is the effect of lesioning a rat's prefrontal cortex when it comes to speed of learning of habits?
Lesioning (damaging) the rat's prefrontal cortex induces faster learning of habits
Acquisition (conditioning) - learning curve behavior
Monotonic increase over time
Why is a variable ratio of actions and outcomes less effective as a conditioning method?
More average-based; animal will understand that average and know that around that rough number of actions they will receive a reward, but not specified (Steady but slow responding - variable intervals; allows animal to be lazy until reward is approaching)
Key factors that influence effectiveness of classical conditioning
Predictiveness (the CS reliably predicts US - contingency & timing), US strength (the US is sufficiently rewarding/aversive), CS salience
Primary vs Secondary Reinforcers
Primary reinforcers - those you don't need to learn are rewarding; they are innately rewarding to you (food, sex, direct stimulation of dopamine neurons in your brainstem...) Secondary reinforcers - those you learn are rewarding over time. Accrue value (e.g. money)
Fixed ratios (most successful method for conditioning)
Reinforcement after fixed number of responses - after a certain specified number of actions, a reward is given. Maximizes an animal's operant response - they know they'll get a reward bc there is a highly predictive relationship between taking those actions and being rewarded. (Step functions - saves energy; no need to take extra actions between intervals bc they don't maximize rewards)
Prediction error (RW model)
Subtraction of what is expected (= sum of the value terms of all the CS's that just appear in the world) from the outcome [Brain should generate an expectation (hear a tone, generate an expectation of the puff) - if the puff comes, there's nothing to learn; you are well-calibrated in the world, making predictions that are true. BUT, if you don't expect the puff, you have something to learn - maybe you could have done something better with regards to predicting.]
What does associative bias suggest?
Suggests intrinsic machinery for classical conditioning that seeks to find an explanation for why something happens [E.g.1.: Poison group = tone + taste -> poisioning; almost all learning attributed to flavor stimulus - ingestion (poisoning) related to flavor. E.g.2.: Shock group = tone + taste -> shock; tone has stronger predictive value for CR; acoustic stimulus more reasonable for predicting shock - startle; could be associated with intrinsic responses to dodging predators]
What is suggested by the spontaneous recovery of a learning association that had been extinguished?
That memory is probably not singular (neatly traceable to a single brain region) or else it would go up and down more regularly. It is thought that some spontaneous recovery effects could be associated to other neural circuits. (Seems like neural system responsible for basic associative learning is not the same one involved in this recovery)
Context reinstatement in learning
The context reinstatement effect refers to the enhanced memory performance found when the context information paired with a target item at study is re-presented at test.
Compound conditioning/overshadowing
Training an animal that, where two stimuli occur simultaneously, they are predictive of a specific stimulus. When you do compound conditioning, the amount of CR you get generally sums to the total value of learning you get when you do the complete conditioning. [If you just show light or just play tone, individual animals can respond to different extents. With compound conditioning, the amount of CR obtained generally sums to the total value of learning you get when you do the complete conditioning. The response produced by the dog to each stimuli reflects their brain's valuation of each of them. CSs compete.]
4 key elements of classical conditioning
Unconditioned Stimulus (food), Unconditioned Response (salivation), Conditioned stimulus (tone), Conditioned Response (salivation)
What does eye-blink conditioning entail?
Unconditioned stimulus (air puff) prompts eye.id closure in rabbits. Conditioned stimulus (tone) has no meaning to animal before conditioning. Tone and puff are associated, and then tone alone is sufficient to make rabbit close its eye. Day 5 - conditioned response happens before puff; shows that association has been learned.
Behavioral evidence that LTP is involved in learning and memory - fear conditioning & Morris water maze (+ main takeaway)
Used genetic breeding technique to make a mouse line with enhanced LTP (amount of LTP between 2 neurons, given the same stimulation, much higher in transgenic mice than in wildtype -> assumption that there should be a measurable effect of this re: learning). FEAR: Animal learns that tone precedes shock. Animals with enhanced LTP have better retention of fear association. MORRIS WATER MAZE: Put animal in water with hidden platform; earlier in task animal wanders trying to find it. Transgenic mice with enhanced LTP are better at learning how to situate the platform in the maze than wild-type mice. LTP may be a mechanism contributing to the formation of the association between CS and US stimuli during fear conditioning
Where does the "badly timed", but still learned CR, seen in LTD knockout mice come from?
When you pharmacologically BLOCK purkinje cell input to deep cerebellar nuclei -> get (badly timed) learning. + mossy fiber to CN plasticity - LTP
Rescorla-Wagner Model Equation
Δ V = α β (λ - Vt), Vt= V1 + V2...Vn Δ V -> change in associative strength for CS in one trial α -> salience of CS β -> learning rate λ -> observed outcome / maximum associative strength/magnitude of US Vt -> predicted outcome given all current CS's / associative strength already accrued by CS (λ - Vt) [together] -> quantifies prediction error
