116 Readings 6-12

7 Expectancy of reward and the S-O association - Classical conditioning in instrumental behavior

How expectancy may motivate instrumental behavior via S-O association -Whatever the stimuli may be, reinforcement of the instrumental response will inevitably result in pairing these stimuli (S) with the reinforcer or response outcome (O) -Such pairings provide the potential for classical conditioning and the establishment of an association between S and O - Classical conditioning in instrumental behavior First the presence of S comes to evoke the instrumental response directly through Thorndike's S-R association Second, the instrumental response also comes to be made in response to as S-O association that creates the expectance of reward

11 Differences between learning and memory

- clarified by considering the three stages of information processing studies of learning and studies of memory all involve stages of acquisition, retention, and retrieval. ->Learning experiments involve manipulations of the conditions of acquisition, Memory experiments are varied retention and retrieval.

6 Concurrent-chain schedules (choice with commitment) choice link vs terminal link

-Concurrent chain schedules of reinforcement are used to study how organisms make choices that involve commitment to one alternative or another Two stages: -Choice link- chooses between right or left which leads to alternative A (VI 3-min) or alternative B (FI 3-min) -> Responding on either does not yield food -Terminal link- once the participant has made a choice, it is stuck with that alternative until the end of the terminal link of the schedule or the end of the trial -> Terminal link becomes a conditioned reinforcer - differences in the value of the conditioned reinforcer will then determine the relative rate of each choice response in the choice link Variety is the spice of life—pigeons prefer variable over fixed schedules even if the VI schedule requires more time on average for the reinforcer to become available than the FI alternative Concurrent chain response schedules as an important tool for studying conditioned reinforcement

9 Frustrative non-reward behaviors

-Extinction procedures also produce strong emotional effects such as frustration induced by the withdrawal of an expected reinforcer -Frustrative non-reward energizes behaviors such as aggression (when candy machine breaks) When extinction was introduced with pigeons the previously reinforced pigeon began to attack its innocent partner (usually happens early in extinction then subsides)

9 Spontaneous Recovery

-If a rest period is introduced after extinction training, responding comes back ->Nothing specific is done during the rest period to produce the recovery Critical factor is introducing a period of rest between the end of extinction training and assessments of responding - typical finding is that behavior that have become suppressed by extinction recovers with a period of rest

7 Applications of the Premack Principle

-In trying to find an appropriate reinforcer for schizophrenia patients they tried allowing them to sit down because that was something they normally did Taking advantage of the high-probability sitting response worked very well -The high probability of echolalia and perseverative behavior in children with autism suggests that these responses may be effectively used as reinforcers in treatment procedures -The opportunity to engage in a prevalent stereotyped response resulted in better performance on the training tasks than food reinforcement Results indicate that high probability responses can serve to reinforce lower probability responses even if the reinforcer responses are not characteristic of normal behavior -Premack principle encouraged thinking about reinforcers as responses rather than as stimuli so that any behavior could serve as a reinforcer provided that it was more likely than the instrumental response

9 Renewal of Conditioned Responding

-Mark Bouton Renewal refers to recovery of conditioned responding when the contextual cues that were present during extinction are changed Recovery of responding with a change in context is called the renewal effect and it occurs under a wide range of circumstances -appetitive and aversively conditioned responding -Pavlovian and instrumental conditioning -Contextual control of extinction can also occur with interoceptive cues (such as those created by drug states) A CS that has undergone conditioning and extinction has ambiguous meaning and the ambiguity allows the CS to come under contextual control (cues) more easily -After just acquisition training, the CS is not ambiguous because it only signifies one thing

9 Immediate versus delayed extinction

-Memory consolidation takes time which raises the possibility that introducing extinction before the memory of acquisition has been fully consolidated might change the memory and promote extinction performance -Conducting extinction trials immediately after acquisition produces a more rapid loss of conditioned behavior than conducting extinction one day later but the enhanced decline of conditioned behavior is deceptive because with immediate extinction the behavior is more likely to show spontaneous recovery and renewal

6 Maximizing rates of reinforcement

-Organisms distribute their actions among response alternatives so as to receive the maximum amount of reinforcement possible (animals switch back and forth between response alternatives so as to receive as many reinforcers as they possibly can) ->This idea has been used to explain choice behavior at both molecular and molar levels of analysis

7 Antecedents of the Response-Allocation Approach -Consummatory-Response Theory

-Reinforcers are special stimuli that strengthen instrumental behavior Consummatory-Response Theory (Sheffeld) - species-typical consummatory responses (eating, drinking) are themselves the critical feature of reinforcers -Reinforcer responses assumed to be special because they involved consummation or completion of an instinctive behavior sequence -Consummatory responses are fundamentally different from various potential instrumental responses (running, opening a latch, etc.) -David Premack took issue with this idea reinforcer responses are only special because they are more likely to occur than the instrumental responses they follow

6 Undermatching, overmatching, and response bias

-Relative rates of responding do not always match relative rates of reinforcement exactly s- sensitivity parameter (when s=1 perfect matching occurs) -> undermatching: reduced sensitivity of the choice behavior to the relative rates of reinforcement ->influenced by many variables such as species, the effort of difficulty to switch from on alternative to the other, etc. b- response bias occurs when the response alternatives require different amounts of effort or if the reinforcer provided for one response is much more attractive than the reinforcer for the other response

6 Self-control choice and delay discounting

-Self-control is a matter of choosing a large reward over an immediate small reward (ex: extra hour in bed vs higher GPA) -Delay discounting—the value of a reinforcer declines as a function of how long you wait to obtain it (reinforcers lose their value the longer you wait for them) -> Hyperbolic decay function - if the reinforcer is delivered with no delay, the value of the reinforcer is directly related to its magnitude. The longer the reinforcer is delayed, the smaller its value -The value of a reinforcer declines as a function of how long you have to wait to obtain it -Problem of self-control -Different results occur depending on the size of the reinforcer and how rapidly its value is discounted -People with higher education level, higher income, and higher IQ show slower reward discounting -Students who engaged in unprotected sex were found to have steeper discounting functions than those who used condoms -Addictive behaviors with drugs associated with steeper discounting functions - don't know causal direction

8 Sensory capacity and orientation

-The animal will only pay attention to the stimuli which it can actually detect in the first place. If I don't have ears it won't matter what variation you put on a sound for me. -Orientation is just like saying you can't pay attention to what is behind you, you must be attending to the stimuli in order to have stim generalization or differentiation.

6 Mechanisms of the Matching Law -molar vs molecular theories

-The matching law is a descriptive law of nature rather than a mechanistic law - it ignores when individual responses are made Molar theories—ignore what might occur at the individual level of responses -Explain aggregates of responses and deal with the distribution of responses and reinforcers in choice situations during an entire experimental session Molecular theories—operate on a shorter time frame and focuses on what happens at the level of individual responses -Views the matching relation as the net result of these individual choices

9 Effects of Extinction procedures

-The target response decreases when the response no longer results in reinforcement It produces an increase in response variability -Extinction produced a decline in the number of response sequences the participants completed, but it increased the variability of those sequences ->sometimes do response to check if it will work

Working vs Reference memory

-Working memory is operative when information has to be retained only long enough to complete a particular task, then it is discarded after task is completed -Reference memory- long-term retention of information necessary for the successful use of incoming and recently acquired information -> Need to have some sort of previous knowledge to understand

11 Advocates of cognitive ethology

-claim that animals are capable of conscious thought and intentionality- debated and rejected for centuries based on the complexity, flexibility, and cleverness of various examples of animal behavior

7 The Response-Allocation Approach

-considers a broad range of activities that are always available to an individual (instead of just focusing on the instrumental and reinforcer responses). How an individual distributes his or her responses among the various options that are available -> Starting point = unconstrained baseline: how the individual allocates his or her responses to various behavioral options when there are no restrictions and presumably reflects the individual's unique preferences -> Unrestricted baseline is also called the behavioral bliss point

7 Imposing an Instrumental Contingency

-individuals will generally defend their response allocations against challenges to the unrestricted baseline or bliss point condition -the baseline response allocation usually cannot be reestablished after an instrumental contingency has been introduced -study: In a real life situation, doing something like restricting someone's time on facebook might not make it reinforcing because the person might just decide to go outside if facebook is restricted. It works best in isolated situations.

11 The Morris Water Maze

-start position is varied randomly from one trial to the next so that the platform cannot be found by always swimming in the same direction -largest improvements in performance occurred from the first to the second day of training -useful technique for the study of the neural bases of spatial memory, but it is not a task that rats or mice are likely to encounter in their natural environment

12 Behavioral theory of timing (BET)— -Adjunctive behaviors

-time-related behaviors emerge in situations where the primary basis for the delivery of a reinforcer is the passage of time. -adjunctive behaviors- Systematic activities or responses that occur when reinforcers are delivered at fixed intervals. Ex) finger tapping when forced to wait emerge automatically when forced to wait for something important. -Right after food delivery- focal search, middle of interval- general search bc least likely to get food, then closer to end- focal search -Bc diff adjunctive behaviors emerge at diff intervals in a forced waiting period- responses can be used to tell time (clock process) instead of reading an internal clock, participants are assumed to "read" their adjunctive behavior states to tell time SET is retrospective whereas BET is prospective

10 Two-Process theory of avoidance

-treats conditioned fear as a source of motivation for avoidance learning Part 1: classical conditioning of fear to the CS Part 2: instrumental reinforcement of the avoidance response through fear reduction -classical and instrumental processes depend on each other -Classical conditioning makes instrumental negative reinforcement possible but successful instrumental avoidance responding can result in extinction of the classically conditioned fear -Explains avoidance behavior in terms of escape from conditioned fear rather than in terms of prevention of the shock -Instrumental response is reinforced by a tangible event (fear reduction) rather than merely the absence of something

7 Two perspectives of Motivation of instrumental behavior -Associative structure of instrumental conditioning -Response-allocation approac

1. Associative structure of instrumental conditioning- (by Thorndike) relies on concept of associations and is similar to Pavlovian conditioning -Molecular perspective—focuses on individual responses and the specific stimulus antecedents and outcomes of those responses -Examines instrumental learning in detailed isolated behavioral preparations 2. Response-allocation approach (Skinnerian) involves considering instrumental conditioning within the broader context of the numerous activities that organisms are constantly doing -Molar perspective—considers long-term goals and how organisms manage to achieve those goals within the context of all their behavioral options -More functional perspective

10 Characteristics of the Aversive Stimulus

1. Time-out— removal of the opportunity to obtain positive reinforcement 2. Overcorrection— requiring a subject not only to rectify what was done badly but to overcorrect for the mistake (ex: child put foreign object in mouth, then wash out mouth) 3. Point loss- effective in human studies 4. Shocks in pigeons—the more intense and longer shock, the more effective punishment response -Individuals habituate to mild punishment

11 Procedural Determinants of Delayed Matching to Sample

1. nature of the stimulus that serves as the sample Some types of sample stimuli are more effective than others 2. duration of exposure to the sample stimulus and the delay interval -Longer delay= less retention -presentation of a stimulus creates a stimulus trace in the brain that automatically fades with time, leading to more memory errors 3. Size of reward- greater reward= greater retention -test performance depends on the similarity between the conditions of testing and the conditions of training

11 5 Types of Memory

1. procedural memory- reflects knowledge about relationships among features of the environment and mediates the learning of behavioral and cognitive skills that are performed automatically, without the requirement of conscious control. -Used in research on classical and instrumental conditioning 2. perceptual memory- memory for specific events or memory for what, when, and where something happened 3. semantic memory 4. primary or working memory 5. episodic or declarative memory

12 Possible Bases of Serial Order Performance - response chain -paired associate learning -serial representation learning

1. response chain—each response produces the stimulus for the next response in the sequence, and correct responses occur because the organism has learned a series of S—R associations -does not require actually learning the stimulus sequence or forming a mental representation of the order - do not require cognitive mechanisms -Seeing the new stimulus stimulates the response, not the previous stimulus 2. paired associate learning—set of independent S—S associations knowing that after A follows B and after B follows C (A-B, B-C, C-D,etc.) 3. serial representation learning—forming a mental representation of the entire sequence A-> B-> C-> D->E-> F -You know the ordinal position of each stimulus

11 Explanations of retrograde amnesia -Consolidation failure explanation of retrograde amnesia -Retrieval failure hypothesis -amnesia reversal

A neural insult during the consolidation window disrupts the consolidation process and interferes with establishing a long-term record of the learning experience Stimulus short term memory store (information is vulnerable and can be lost because of interfering stimuli) if proper conditions are met consolidation into long-term memory store -Consolidation failure explanation of retrograde amnesia—attributes amnesia to a disruption in the original acquisition of the memory -Retrieval failure hypothesis—an amnesic agent alters the coding of new memories so as to make them more difficult to recover -thus amnesia is attributed to a problem in the retrieval phase amnesia caused by hypothermia can be reversed by returning to normal body temp that they had right after the extinction procedure -in this case, amnesic treatment did not disrupt the consolidation of memory of extinction

10 Experimental Analysis of Punishment

An aversive stimulus is presented after a target instrumental response and if it effective, this response will become suppressed. -Since it involves suppression of behavior, it can only be observed with responses that occur in the first place -Lab studies usually begin with the response being positively reinforced then punishment superimposed which can become complicated and make it difficult to predict what will happen

8 multiple schedule of reinforcement

An instrumental conditioning procedure in which responding is reinforced in the presence of one stimulus and not reinforced in the presence of another cue -Animals can tell which stimuli to pay attention to if you vary the reinforcement across stimuli in terms of on getting fixed ratio and one getting variable interval.. they'll be able to tell.

11 The Radial Arm Maze

Animals in the wilderness find food somewhere- they remember to avoid that spot for some time because they already took the food from there radial arm maze was developed to test memory for places where an animal recently obtained food and depleted that food source has eight arms radiating from a central choice area, with a food cup at the end of each arm most efficient way for a rat to get all eight pellets is to enter only those arms of the maze that it had not yet visited- rat does this Rats do not require much training to perform efficiently bc foraging is in their nature- evolutionary roots rats appear to use distinctive features of the environment to keep track of where they've been- if landmarks around arm moved- go to arm as if it is new Rats can go up to 16-24 spatial locations Radical arm maze with interruption procedure Shows that memory deteriorates with time

7 Two-Process Theory -Rescorta and Solomon

Assumes there are two types of learning: Pavlovian and instrumental -two different neural mechanisms underlying respondent and operant behavior -Assumes that they are RELATED in a special way -During the course of instrumental conditioning, the stimuli (S) in the presence of which the instrumental response is reinforced becomes associated with the response outcome (O) through Pavlovian conditioning and this results in an S-O association -Rescorta and Solomon - the S-O association activates an emotional state that motivates the instrumental behavior (emotional state assumed to be positive or negative depending on if the reinforcers appetitive or aversive)

10 Avoidance procedure vs Punishment Procedure

Avoidance procedure—individual has to make a specific response to prevent an aversive stimulus from occurring -Negative contingency between an instrumental response and the aversive stimulus: if the response occurs, the aversive stimulus is omitted -Increase the occurrence of instrumental behavior -Active avoidance- Safety is achieved by doing something Punishment procedures -Positive contingency: the target response produces the aversive outcome -Suppresses instrumental responding -Passive avoidance—increased safety is achieved by not doing something -In both cases, individuals learn to minimize their exposure to aversive stimulation

9 Resistance to change and behavioral momentum -Two major conclusions

Behavioral momentum (Nevin and Grace)—behavior that has a great deal of momentum will also be hard to "stop" or disrupt by various manipulations, including extinction -Research on multiple schedules of reinforcement enables investigators to compare the susceptibility of behavior to disruption under two different conditions in the same session with the same individual Two major conclusions: 1. Behavioral momentum is directly related to the rate of reinforcement higher rates of reinforcement produce behavior that has greater momentum and is less susceptible to disruption 2. Behavioral momentum is unrelated to response rate two behaviors that occur at similar rates do not necessarily have similar degrees of behavioral momentum effects of reinforcement rate on behavioral momentum (Pavlovian conditioning or S-O associations) Study with videogames and sprites: responding was less disrupted in the presence of the sprite that associated with higher reinforcement rate

12 Food Caching and Recovery

Caching behavior is related to ecological factors and varies among species and within populations of those species Food caching and recovery involves many factors First you have to decide what food items to catch Perishable vs nonperishable Decide where to store the food because the location has to be recalled at time of recovery Caching involves a social component: storing food is only useful if you rather than a competitor gets to eat what you stored Decision to cache the food can be influenced on the presence of a competitor who might steal it Cache location influenced too -> Spatial Memory in Food caching and recovery Cache recovery reflects spatial memory

6 Can self-control be trained?

Can self-control be trained? -Shaping- present the large reward immediately and then increase the delay for the large reward in small steps -Distract attention from the large reward -e.g. 2nd and 3rd grade students, pretest - 2 cents now or 3 at the end of the day; then went through training -Training with the delayed reward increased subsequent preference of the children for the larger delayed reward

12 Components of linguistic skill -Apes and grammar

Chimps and bonobos raised in a language-rich social environment typically acquire a vocabulary of about 100—200 words Language involves arrangement of words into sequences according to certain rules of grammar or syntax Evidence of "grammar" in great apes -Declarative language does not occur in great apes

7 Response interactions in Pavlovian Instrumental transfer -alcohol

Classically conditioned stimuli elicit not only emotional states but also overt responses such as sign tracking -The overt responses elicited by a Pavlovian CS may influence the results in a Pavlovian instrumental transfer experiment -Study on effects of a pavlovian CS for alcohol on instrumental responding reinforced by alcohol ->Results show that pavlovian CS for ethanol will increase instrumental responding reinforced by ethanol. The increased lever pressing during the CS shows that an independently established S-O association can facilitate instrumental responding reinforced by that outcome -However, if the lever on the right and the CS appeared on the left, the rat approached and sniffed the light on the left -The results of the transfer test depended on the compatibility of the Pavlovian CR and the instrumental response

8 Control by contextual cues

Contextual cues control behavior in a variety of ways (you are more likely to study in the library than at home during break) -Study with quails and sexual reinforcement in two different settings - the birds that received CS context paired with sexual reinforcement came to prefer that context -The association of contextual cues with sexual reinforcement increased preference for those cues -Drug-conditioned place preference Contextual cues do not have to signal reinforcement to gain control over behavior

8 Importance of type of reinforcement

Different types of reinforcement... ie eliminating aversive stimuli or providing positive stimuli will cause different stimuli to be attended to and therefore different controlling stimuli Pigeons pay more attention to visual stimuli than auditory when the reinforcer is food -in contrast, a signal for an aversive outcome activates the defensive behavior system which is usually responded to by auditory cues to avoid danger

8 Spence's Theory of Discrimination Learning

Differential responding to S+ and S- reflects BOTH conditioned excitation to S+ and conditioned inhibition to S- -Discrimination training makes S- inhibitory -Study: tone was made to be inhibitory (S-) and suppressed responding even in the presence of the S+ -Negative discriminative stimuli (S- cues) can exert a powerful inhibitory influence on drug seeking and drug self-administration behavior

8 Contextual cues and conditional relations

Discrete stimulus—presented for a brief period, has a clear beginning and end, and can be easily characterized (such as a light or a tone) -A more comprehensive analysis of the stimuli organisms experience during the course of conditioning indicates that discrete discriminative stimuli occur in the presence of back-ground contextual cues -Contextual cues are various features (visual, auditory, and olfactory) of the room or place where the discrete discriminative stimuli are presented

10 The Discriminated Avoidance Procedure -Avoidance trial vs Escape trial

Discriminated or signaled avoidance- occurrences of the aversive stimulus are signaled by a CS. Responding during the CS terminates the CS and prevents the delivery of the aversive US. Avoidance trial—if the participant makes the target response before the shock is delivered, the CS is turned off and the US is omitted on that trial -If the participant fails to make the response during the CS—US interval, the scheduled shock is delivered and remains on until the response is made (escape trial) Escape trial—after the CS—US interval if the response isn't made, thy receive a shock until the response is made -escape trials Occur mostly During early stages of training -With more training, avoidance trials come to predominate

9 Hypothesis of the Partial-reinforcement extinction effect -Discrimination hypothesis -Jenkins study

Discrimination hypothesis- introduction of extinction is easier to detect after continuous reinforcement than after partial reinforcement Jenkins study—the subjects who initially received partial reinforcement training responded more in extinction -Possible that partial reinforcement seems to teach individuals not to give up in the face of failure and this learned persistence is retained across an unbroken string of successes

12 Techniques for Studying the Temporal control of behavior -Duration estimation -Peak procedure

Duration estimation—a discrimination procedure in which the discriminative stimulus is the duration of an event Pigeons trained to peck red if the light is presented for 2 second and peck green if the light is presented for 10 seconds -Establishes temporal discrimination Peak procedure—involves a duration production instead of duration estimation -Each trial begins with the presentation of a discriminative stimulus (noise or light) and after a time interval, the food pellet is set up for delivery -peak response rates occurred near the times that food became available during training, with lower response rates evident before and after that point. Discrete-trial variation of a fixed-interval schedule -Temporal control of behavior is evident in the frequency of head pokes into the food cup rather than lever presses reinforced with food

12 Episodic memory in food caching and recovery

Episodic memory was originally thought to be exclusive to humans There is evidence showing that scrub jays are able to remember what, where, and when they stored particular food items Challenging the notion that episodic memory is uniquely human

11 Genes and Learning

Establishing a long-term memory depends on the activation of genes that manufacture new protein products that produce a lasting change in how a synapse works

10 The Conditioned Emotional Response Theory of Punishment

Estes proposed that punishment suppresses behavior through the same mechanism that produces conditioned suppression -Conditioned suppression involves a suppression of ongoing behavior elicited by a stimulus that has been associated with aversive stimulation Punished responses become suppressed due to cues that acquire conditioned aversive properties

10 Expectancy Theory of Avoidance

Expectancy Theory—encounters with aversive events trigger a conscious process of threat appraisal that generates expectations of future threat (or lack of threat) based on cues and responses

9 Exposure therapy

Exposure therapy is basically an extinction procedure in which participants are exposed to cues that elicit fear in the absence of the aversive US -> diff from flooding bc cues -Used in treating phobias, drug addiction with the aim of extinguishing cues associated with drug-taking behavior

9 Extinction of conditioned behavior

Extinction can only be conducted after a response or association has been established using Pavlovian or instrumental conditioning. A true reversal of acquisition is rarely achieved and may not be possible. -S-O and R-O associations survive extinction procedures; rather than erasing old learning, extinction seems to involve learning a new inhibitory S-R association

9 What is learned in extinction? -unlearning -emotion

Extinction does not involve unlearning; possible that it produces an inhibitory S-R association motivated by the unexpected absence of the reinforcer in extinction -Effects of nonreinforcememnt depend critically on the individual's prior history Extinction involves both behavioral and emotional effects -Emotional effects stem from the frustration triggered when an expected reinforcer is not forthcoming -Aversive emotion serves to discourage responding during the course of extinction through the establishment of an inhibitory S-R association Study: responding during the stimulus with which the response had been extinguished was less than responding during the alternate stimulus Extinction procedure produced an inhibitory S-R association that was specific to a particular stimulus and response

9 Classical conditioning vs Instrumental conditioning: extinction -forgetting

Extinction involves omitting the US, or reinforcer -Classical conditioning: extinction = repeated presentations of the CS without the US -> present tone without food -> CR undergoes extinction -Instrumental conditioning: extinction = no longer presenting the reinforcer when the response occurs The behavior change that occurs in extinction is the reverse of what was observed in acquisition however this is not correct -Forgetting is different than extinction because it is a decline in responding because of the passage of time

10 Extinction of avoidance- CS-alone exposure -Response blocking -Flooding

Extinction of avoidance behavior through response blocking and CS-alone exposure -The most effective extinction procedure for avoidance behavior involves extensive exposure to the CS without the US. However, the participants cannot be permitted to terminate the CS prematurely. -Response blocking—altering the apparatus so that the avoidance response is blocked (in a shuttle box this would be a barrier) Flooding—by blocking avoidance response the participants can be expose to the CS for a long time or "flooded" by the CS longer exposures to the CS without the US lead to more successful extinction of avoidance responding

9 Forms of recovery from extinction

Extinguished behaviors reappear—proof that extinction is not the reverse or elimination of acquisition Forms: Spontaneous recovery, renewal, reinstatement, resurgence

11 Forgetting and sources of memory failure

Forgetting should not be viewed as the absence of remembering -Failures of memory become more likely as time passes after learning an episode but time is not a cause of forgetting -Memory failure is preferable in referring to these instances of poor performance on tests of memory

9 Theories of Partial reinforcement: -Frustration Theory * -Sequential theory

Frustration Theory—based on what individuals learn about the emotional effects of non-reward during partial reinforcement training -Intermittent reinforcement results in learning to make the instrumental response in the face of the expectation of non-reward (ex: gamblers continue to play even though they know the odds are against them) Sequential theory—based on what is learned about the memory of non-reward -in terms of memory concepts rather than expectations of reward or frustration -with intermittent reinforcement, the memory of non-reward becomes a cue for performing the instrumental response The individual learns to respond whenever it remembers not having been reinforced on the preceding trials and creates persistent responding in extinction Frustration and sequential theory vs. partial reinforcement extinction effect point out different ways in which partial reinforcement can promote responding during extinction The emotional learning (frustration theory) is less sensitive to inter-trial intervals and provides a better explanation of the partial-reinforcement extinction effect when widely spaced trials are being used Memory mechanism (sequential theory) may be more in effect when trials are closer together

11 General Versus Specific Rule Learning

General Rule- pick the stimulus that is the same as the rule -Pick the red light -Don't have difficulty when facing novel stimuli -general-rule learning should produce better performance than specific-rule learning. Specific rule- there is a rule to which you should pick -"Select red after exposure to red" and "Select green after exposure to green." -Have difficulty when facing novel stimuli - Because a given sample stimulus is not presented on more than one trial, accurate performance is possible only if the participant learns to respond on the basis of a general same-as rule.

10 Methods of introduction of the Aversive Stimulus -intensity of aversive stimulus

How the aversive stimulus is introduced (initial exposure to punishment) determines how they will respond to punishment subsequently -Initial exposure to mild aversive stimulation that does not disrupt behavior reduces the effects of later intense punishment -Initial exposure to intense aversive stimulation increases the suppressive effect of more mild punishment later

11 Contemporary view

If a memory is in the active state it is assumed to be unstable and potentially modifiable (in the consolidation window) The only way to move info from the active to inactive state is through consolidation -If the memory is in the active state because it was previously learned and is now reactivated, then return to the inactive state will involve the process of reconsolidation -Memories are reconsolidated after each recollection -> there is no permanent record Reconsolidation can serve to strengthen memories and may allow for the priming effect When we recall something, we are not accessing what we originally experienced but how we previously remembered the experience

8 Differential Responding and Stimulus Discrimination

If an organism responds one way in the presence of one stimulus and in a different way in the presence of another stimulus its behavior has come under control of those stimuli An organism is said to exhibit stimulus discrimination if it responds differently to two or more stimuli -Stimulus controls behavior by the context of the stimulus -> If an organism does not discriminate between two stimuli, its behavior is not under the control of those cues

11 Retrograde Amnesia for extinction -consolidation window

If extinction is a form of new learning than extinction should be susceptible to retrograde amnesia Hypothermia disrupted the memory of extinction only if it occurred within 30 minutes or less after the extinction treatment - indicates that the extinction memory was at first malleable and subject to modification but then became immune Memory consolidation—transformation of memory from a flexible and modifiable state to a more stable state Consolidation window—limited period after a memory is activated before it has become stable

10 Fear and instrumental processing

If fear motivates avoidance responding and if fear reduction is reinforcing, then the conditioning of fear and instrumental response should go hand in hand but are only correlated early in training -Avoidance responding may persist without much fear being exhibited when the CS or warning signal occurs

10 Punishment vs positive reinforcement

In most punishment situations, the target response is simultaneously maintained by a schedule of positive reinforcement (ex: violating dress code reinforced by attention) -The effects of punishment always depend on the reinforcer that maintains the target response (ex: drug addiction—effect of punishment by losing family friends etc. depends on the rewarding properties of the drug) Extensively trained rats with cocaine were resistant to effects of punishment -> if resoponse gives greater reward than punishment gives, subject will still do the response

10 Punishment as a signal for the availability of positive reinforcement

Individuals may seek punishment if positive reinforcement is available only when the instrumental response is also punished - punishment becomes a signal or discriminative stimulus for the availability of a positive reinforcement When this occurs, punishment will increase rather than decrease responding Pigeon study: pecked more during times of punishment and reinforcement than safety times -> punishment became a discriminative stimulus for food reinforcement

7 Response Allocation and behavioral Economics

Instead of considering instrumental conditioning in terms of the reinforcement of a response in the presence of a certain stimuli, response allocation is a molar approach that focuses on how instrumental conditioning procedures put limitations on an organism's activities and cause redistributions of behavior among available responses

9 Extinction cue compounded with a conditioned inhibitor during extinction training

Interference rather than extinction process If a safety signal (conditioned inhibitor for freezing) is compounded with a fear stimulus during extinction, the absence of the US will be fully predicted by the safety signal Won't be any error to encourage learning that the fear stimulus no longer ends in shock and this the safety signal will block extinction of the fear stimulus

8 Discrimination training focused on interoceptive cues

Interoceptive cues are things like hunger or jealousy or some stuff that comes from inside It works... been tested with drugs like cocaine, this means the animal can tell when it is feeling the DRUGSSSZZZZ You can increase an animals sensitivity to interoceptive cues by administering this training

11 anthropomorphism

Interpretation of complex behavior in nonhuman animals by projecting our own thoughts, emotions, and intentions on them. cloud understanding because they overemphasize conscious human experience and are often accepted without experimental proof.

8 Effects of discrimination training on stimulus control

Jenkins and Harrison examined how auditory stimuli that differ in pitch can come to control the pecking behavior of pigeons reinforced with food -The control group which did not receive discrimination training responded nearly equally in the presence of all tones (aka tone deaf) -Pigeons who had received discrimination training with S+ tone and S- absence of tone showed an intermediate degree of stimulus control by tonal frequency Shows that discrimination training increases the stimulus control of instrumental behavior and a particular stimulus dimension (like tonal frequency) is most likely to gain control over responding if the S+ and S- differ along that stimulus dimension

12 Evidence for learning of a representation of a serial order -Symbolic distance effect

Latency of the first response when sequence began with B instead of A Latency of the second response to the three-element subset Symbolic distance effect—taking longer to respond to the next item as a function of the number of missing items -Provides further evidence that participants learned a representation of the five-element series

12 Serial Order Learning

Learning to respond to a set of stimuli in a specified order. can be achieved by learning a series of S-R or S-S associations or by forming a mental representation of the order of the stimuli.

6 Complex Choice and Self-Control

Many complex human decisions limit your options once you have made a choice (choice with commitment)—once the selection is made, the other alternatives are no longer available for a while

11 Retrieval cues and memory priming

Memory priming—the retrieval cue is presented at the end of a long retention interval but memory is not tested until the next day Priming with odor improved memory in infants Studies of priming demonstrate that retention can be significantly enhanced by reactivating a memory during the retention interval because it reactivates a previously learned memory without more training Best priming should be conducted in the context of the original training

7 R-O and S(R-O) Relations in instrumental conditioning -Evidence of R-O Associations

Most common technique used to demonstrate the existence of R-O associations involved devaluating the reinforcer after conditioning -Reinforcer devaluation involves making the reinforcer less attractive If the instrumental response occurs because of an R-O association, devaluation of the reinforcer should reduce the rate of the instrumental response If reinforcer devaluation disrupts instrumental behavior, this shows that the memory of the outcome (O) was involved in motivating the instrumental behavior Study with cigarettes and chocolate (devaluating increase opposite key responding) ->Devaluation produced a decline in behavior specific to the response whose reinforcer had been devaluated ->Result of devaluation tests indicate that training established an R-O association by linking each response with its specific reinforcer. Results cannon be explained by S-R associations because they are not influenced by reinforcer devaluation R-O associations are involved in instrumental drug-seeking behavior ->R-O mechanisms predominate in free-operant situations where S-R mechanisms are activated when drug taking is a response to drug-related cues

9 Enhancing extinction

Number and Spacing of extinction trials -The simplest way to increase the impact of extinction is to conduct more extinction trials—the use of larger numbers of extinction trials produces a more profound decrease in conditioned responding Another important variable is the spacing of trials -Although massed extinction trials may produce a rapid immediate decrement in responding, the conditioned behavior is likely to return with a period of rest (spontaneous recovery) or a change in context (renewal)

6 Molecular Maximizing

Organisms always choose whichever response alternative is most likely to be reinforced at a given moment in time thus this model claims that the matching relation is a byproduct of prudent switching behavior that tracks momentary changes in the probability of reinforcement -EX: 4 choices A, B, AB, BA : the relative responses on A and B is presumed to depend on the relative rate of reinforcement for staying on each side versus switching from one side to another

6 Molar Maximizing

Organisms distribute their responses among various alternatives so as to maximize the amount of reinforcement they earn over the long run -Molar theories focus on aggregates of behavior over some period of time, usually the duration of the experiment, rather than on individual choice responses -In concurrent ratio schedules, animals rarely switch back and forth between response alternatives, rather they response exclusively on the ratio component that requires the fewest responses to maximize the reinforcement with the least amount of effort -However, in concurrent VI-VI schedules, the reinforcement could be the same despite the amount of responding -Another challenge is when given a choice between VI and VR - for maximum return on a concurrent VI VR schedule, participants should concentrate their responses on the VR alternatives and respond only occasionally on the VI

7 Hierarchical S(R-O) relations 3 term contingency

Organisms learn to associate an instrumental response with its outcome but R-O associations cannot act alone to produce instrumental behavior ->R-O association is activated by the stimuli (S) that are present when the response is reinforced In addition to activating R directly, S also activates the R-O association ->The subject comes to think of the R-O association when it encounters S and that motivates it to make the instrumental response -Skinner's three-term contingency—S, R and O are connected through a conditional S(R-O) relation S(R-O) associations are learned during the course of instrumental conditioning

10 Avoidance Behavior -Study

Originated with Bechtrev's experiment with a procedure similar to classical conditioning except the participant's behavior could change the delievery of the US (if they lifted their finger off the metal plate, they did not receive a shock) Study with guini pigs: classical conditioning group and avoidance conditioning group -Avoidance group quickly learned to make the instrumental response -Shows that it is very different from classical conditioning

9 Paradoxical Reward Effects

Overtraining extinction effect, Magnitude reinforcement extinction effect, Partial reinforcement extinction effect (PREE)

8 Stimulus Generalization -Pavlov vs Lasley and Wade view

Pavlovs view: occurs because learning about a CS gets transferred to other stimuli on the basis of the physical similarity of those test stimuli to the original CS Lasley and Wade contrasting view: stimulus generalization reflects the absence of learning rather than the transfer of learning -it occurs if organisms have not learned to distinguish differences among the stimuli -Shape of a stimulus gradient is determined by the organism's previous learning experiences rather than by the physical properties of the stimuli tested ->Lashley and Wade were closer to the truth - stimulus control can be dramatically altered by learning experiences

12 Perceptual concept learning

Perceptual concept learning - stimuli that vary in their physical properties can be can be grouped together and associated with a single response through categorization or concept learning -Chain includes all different views: size, distance, feel etc. its still a chair basic level perceptual categorization permits us to recognize our car as a vehicle regardless of the angle of it or how dirty it may be higher level categories are more abstract and there are fewer physical similarities between the members of the category represents a balance between stimulus discrimination and stimulus generalization -generalization within a category or a set of stimuli and discrimination between categories or sets of stimuli -baboons tested for orthographic information: discriminating word vs. non-word

10 Escape from fear procedure (EFF)

Phase 1: condition fear to a CS via pure classical conditioning procedure Phase 2: the participants are exposed to the CS and allowed to perform an instrumental response to turn off the CS -No shocks delivered in the second phase

8 Discrimination Training Focused on compound or configural cues -Positive patterning and negative patterning

Positive is when two behaviors are only reinforced when they occur at the same time -When the stimuli occur individually they are not reinforced -When they occur simultaneously (AB+) they are reinforced Negative pattern training is the reverse of that -When the stimuli occur individually they are reinforced Animal learn this

10 Positive reinforcement theory of avoidance

Positive reinforcement through conditioned inhibition of fear or conditioned safety signals -Safety-signal hypothesis—the safety signals that accompany avoidance responses may provide positive reinforcement for avoidance behavior -The act of making the instrumental response provides spatial, tactile, and proprioceptive stimuli that accompany the avoidance response and these cues become safety signals -Predicts that introducing an explicit feedback stimulus will facilitate the learning of an avoidance response -A response feedback stimulus becomes a conditioned inhibitor of fear and this could be used as a positive reinforcement in avoidance learning Response-associated feedback cues can come to provide positive reinforcement for the free-operant avoidance response

11 Cognitive Maps and other Navigational codes

Possible that animals form a mental map or mental representation of how the maze and food cups are arranged Animals form a cognitive map which guides their spatial navigation has been a prominent hypothesis -Beacon—cues at the location of the goal -Landmarks—distinctive stimulus that is not at the goal location but has a fixed relation to the goal -> To determine if an animal is using a landmark, the landmark must be moved to a new location to see how that influences the animal -Geometric cues—provided by the shape of the arena

10 Predatory imminence continuum -Circa strike response

Predatory imminence continuum—different responses occur depending on the level of danger faced by an animal Different species typical defense responses are assumed to occur at different levels of predatory imminence Circa strike response—when a snake actually touches a rat, the rat is likely to leap into the air but if this does not lead to successful escape, the rat is likely to engage in defensive aggression and if that is successful it will then move on to make recuperative responses -Assumes that defensive behavior initially occurs as unconditioned responding and can come to be elicited by a CS if that CS becomes associated with an aversive event -defensive response occurs to the CS will depend on the temporal relation of the CS to the unconditioned aversive stimulus

9 Presenting Extinction Reminder Cues

Presenting retrieval cues for extinction during tests of recovery of extinguished behavior can overcome the problem of relapse due to the fact that extinction training does not generalize when the learned behavior is tested in a new place -during a period of rest introduce cues to see if they will spontaneously recover Introducing cues that were present during extinction training can reduce spontaneous recovery and enhance extinction performance in taste aversion learning and in appetitive conditioning preparations Extinction cues may similarly reduce the renewal effect by reactivating extinction performance in the renewal context (ex: AA patients carry chips to remind them to stay sober) -> Reminder cues provide reminders of the therapeutic context.

* 9 Compounding Extinction Stimuli

Presenting two stimuli at the same time that are both undergoing extinction can deepen the extinction of those cues This outcome shows that the compound extinction trials deepened the extinction of a stimulus and this deepening of extinction also reduces the reinstatement effect and slows the rate of acquisition of an extinguished stimulus

11 Proactive and Retroactive interference

Proactive interference—interfering information acts forwards to disrupt the memory of a future target event Retroactive interference—interfering stimulus acts backwards to disrupt the memory of a preceding target event

10 Avoidance Theory of Punishment

Punishment as a form of avoidance behavior—follow the two-process theory of avoidance Suppression of behavior is NOT viewed as reflecting the weakening of the punished response rather it is explained in terms of strengthening competing avoidance responses -> understand that by not doing the response they will not be punished

10 Availability of Alternative Reinforced responses

Punishment has dramatically different outcomes depending on whether the individual is able to obtain reinforcement by engaging in some other activity Availability of an alternative response for obtaining positive reinforcement greatly increased the suppressive effects of punishment Reinforcement for alternative behavior increases the effectiveness of mild punishment in suppressing self-injurious behavior in individual with severe developmental disability -seems kind of like a distraction- give them reinforcement for responding to something else so they don't focus on the other response

9 Priming Extinction to Update Memory for Reconsolidation -Reconsolidation window

Reconsolidation window—the period during which an activated memory can be modified (less than 6 hours but long enough for extinction training) Study: Memory of acquisition may be substantially altered by conducting extinction trials during the reconsolidation window -Group that received extinction training outside the reconsolidation window (6 hours after priming CS presentation) showed substantial spontaneous recovery -No spontaneous recovery if extinction training was conducted within the consolidation window Fear memory is especially vulnerable to extinction training if extinction is conducted while the fear memory is in the reconsolidation window

8 Reflexivity, Symmetry, Transitivity -emergent relations

Reflexivity or sameness (A=A) Symmetry (if AB then BA) Transitivity—integration of the two relationships into a third one (if AB and BC then AC) -Rose -> flower and flower -> plant then rose -> plant Emergent relations: The principle that when you train a kid to think of the word rose when they see a picture of a rose, they will also learn to think of a rose when they see the word rose

11 Retention and the problem of rehearsal

Rehearsal—keeping information in an active state, ready for available use -Directed forgetting—accuracy of recall can be modified by cues or instructions indicating that something should or should not be remembered -Memory is disrupted by F-cues (signaling it wont be on the test or no reward will be given) R-cues (remember cue) enhances memory -Shows that memory is an active process that can be brought under stimulus control Anticipation of reward helps to keep recently experienced events in memory and larger rewards produce greater improvements in memory

10 Shock-frequency reduction hypothesis

Reinforcement of Avoidance though Reduction of Shock Frequency -a radical alternative to the two-process theory Views a reduction in frequency of shocks to be critical to the reinforcement of an avoidance behavior -> Found that shock-frequency reduction is not necessary for avoidance learning -> responding decreased frequency, not prevent its delivery- better avoidance learning if response prevented shock delivery

9 Reinstatement of Conditioned Responding

Reinstatement refers to the recovery of conditioned behavior that occurs when the individual encounters the US or reinforcer again Ex: you eat fish and get sick so you avoid it (conditioned aversion) but then eat fish again without getting sick so you eat it again (extinction) then you get sick again and even though it doesn't have anything to do with fish, you begin to avoid fish again (reinstatement) Reinstatement effect can be context specific (different room won't produce the same recovery) Re-exposure to a drug following extinction reverses the therapeutic gains produced by extinction -"reinstatement" often refers to the re-emergence of previously trained behavior as a result of the delivery of a US

* 8 Distinction between excitation and modulation

Related to what is signaled -Conditioned excitatory stimulus signals forthcoming presentation of the US. So if it isn't present then it violates the association made in the organisms brain. ->Conditioning excitatory properties to a stimulus does not make that stimulus function as a modulator -Modulation is different because it signals a relation between the CS and the US. The absence of the US when the modulator is presented alone does not mean that the relation between the target CS and the US has changed. ->A modulator signals a relation between a CS and a US thus the information signaled by a modulator is not invalidated by presenting the modulator by itself during extinction and the ability of the modulatory to promote responding elicited by another CS remains intact during extinction

8 Control by conditional relations -modulator -Pavlov

Relations between two events -Can be controlled by a third event known as a modulator ->Also known as a conditional relation -> the idea that the context can affect the way two events are related/learned/acted upon Ex: discriminative stimuli S+ and S- are modulators that signal the relation between the response and the reinforcer Pavlovian thought: -The third term is known as a facilitator to the relationship between CS and CR -Also known as occasion setting or facilitation

9 Resurgence of Conditioned Behavior

Resurgence involves the reappearance of an extinguished target response when another reinforced response is extinguished Ex: kid squirms and mom extinguished behavior by not rewarding the child with attention.This works, but the parent feels bad and gives the child a toy to play with. Things proceed smoothly for a while until the child gets bored with the toy and thus experiences extinction of the toy-playing response. This extinction of the second response will result in resurgence of the original wiggling and squirming behavior. -"resurgence" typically refers to previously established behavior that re-emerges during extinction.

11 The Generality of Retrieval cues

Retrieval can be enhanced by presenting just one component of the original learning task such as the unconditioned stimulus, reinforced conditioned stimulus (CS+) or a non-reinforced conditioned stimulus (CS-) -Such reminder treatments can be used to reverse many instances of memory loss -Reminder treatments can counteract stimulus-generalization decrements that occur when learned behavior is tested in a new context -Also increase the low levels of conditioned responding that occur in latent inhibition, overshadowing, and blocking

11 Retrospective and prospective coding

Retrospective coding—football players remembering what they learned about the play during practice sessions -Retrospection—thinking about the past Prospective coding—players have to remember what they are supposed to do as they play unfolds -Prospection—imagining the future They use similar brain mechanisms and overlap Mental time travel—mentally moving forward or backward in time Animals showed evidence for planning for the future by "saving room for dessert"

8 intradimensional discrimination * -peak shift

S+ and S- may be related if they are similar except for one feature or attribute (ex: tone discrimination in pigeons) -Learning about S+ and S- are not always independent and they interact especially if they are related in some way -Intradimentsional discrimination is a form of expert performance because it requires detecting a single differentiating feature between S+ and S- -Can produce a counterintuitive phenomemon known as the peak-shift effect The shift of the peak of the generalization gradient away from the original S+ as a result of intradimensional discrimination training (the control group did not show the peak shift effect)

7 The S-R association and the Law of Effect

S-R association—Thorndike association between he contextual stimuli (S) and the instrumental response Law of Effect: A bond forms between S and R and the role of the reinforcer is to strengthen the association (the reinforcer is only used to strengthen or weaken the association, it is not a part of the association) -> S-R mechanisms and habitual behavior, drug addiction, etc... the consequences are not relevant

10 Discriminated Avoidance Procedure -Shuttle box

Shuttle box shuttle avoidance: the animals shuttles back and forth between two sides of the apparatus to avoid the shock Two-way shuttle avoidance- animal moves in both directions on successive trials One-way avoidance: animal is places on same side at start of each trial One way is easier to learn because the two way requires the animal return to the side that they had previously escaped from (only one side gets shock)

10 Response-contingent versus response-independent aversive stimulation

Significantly more suppression of behavior occurs if the aversive stimulus is triggered by the instrumental response Delivering shocks contingent on an instrumental response is more effective in suppressing that response than delivering the aversive stimulus independent of behavior -> Aversive learning is response contingent

11 Acquisition and the Problem of Stimulus Coding

Stimulus coding is a critical feature of the acquisition phase of memory

8 Relative Ease of Conditioning Various Stimuli -Overshadowing

Stimulus control also depends on the presence of other cues in the situation -Overshadowing—how strongly organisms learn about one stimulus depends on how easily other cues in the situations can become overshadowed ->illustrates competition among stimuli for access to the process of learning -Pavlovs studies about stimuli intensity Studies of special navigation: availability of one cue (landmark) vs another

8 Stimulus-element approach vs configurational-cue approach

Stimulus-element approach looks at complex stimuli as multiple discrete stimuli that work independently Configurational-cue approach looks at complex stimuli as an integral of all of the stimuli -Ex: the concept of a configural cue may be illustrated by considering the sound of a symphony orchestra -John Pearce - showed that many phenomena are consistent with this framework -> According to the congfiural cue approach, overshowing reflects different degrees of generalization decrement from training to testing

11 Retrieval

Stored information is recovered and used to guide behavior Retrieval failure—deficit in the recovery of information -Usually memory failure is due to this rather than loss of the information from the memory store Retrieval cues—reminders that trigger retrieval process

6 Measures of Choice Behavior

Technique to calculate the relative rate of responding on each alternative - dividing the rate of responding on the left by the total rate of responding (left key plus right key) (BL and BR) -If the pigeon pecks equally as often on the two response keys the ratio will be .5 -If it is greater on the left than on the right it will be over .5 and if it is less than it will be less than .5 -By responding equally as often, the pigeon will also earn reinforcers equally often on each side (can be calculated in a similar manner with rL and rR) -So on this same schedule, the relative rate of reinforcement for each response alternative will be .5 because the participant earns reinforcers equally often on both sides -Whether a behavior occurs frequently or infrequently dos not depend on its own schedule of reinforcement but also on the rates of reinforcement of other activities the individual may perform (Matching Law)

10 Nondiscriminated (free-operant) avoidance

The aversive stimulus (shock) is scheduled to occur without warning but if it makes the correct avoidance response, it obtains a period of safety during which the shocks do not occur. -Repetition of the avoidance response before the end of the shock-free period serves to start the safe period over again -S-S interval: the time between shocks in the absence of a response -R-S interval: the time of safety between the avoidance response and the next scheduled shock An avoidance response will always reset the R-S interval (thus free-operant avoidance) and can therefore prolong its safety period indefinitely -The safe period between produces by each response (R—S interval) has to be longer than the interval between shocks that would occur without responding (S—S interval)

11 Retrieval cues

The best retention performance for infants was evident in the group that was tested with the same crib liner that had been presented during conditioning Evidence that the cloth liner served as a retrieval cue for the instrumental kicking behavior Contextual cues-- stimuli incidental to a learning task that are present during original acquisition of the task Important for memory retrieval Memory is most successful if the contextual cues at retrieval are the same as the contextual cues that were present during the original learning

7 Elasticity of demand -Factors

The degree to which price influences consumption... -If candy costs more, ppl buy less. If gas costs more, ppl still buy it Influencing factors for elasticity -Availability of alternatives Price range... if it's a low price that then increases 10%... not a big deal. If something costs 50 bucks then 10% increase then it's a bigger deal -Income level If you rich.. then u don't give a hoot. -Link to complementary commodity So if price makes people not buy hot dogs, then they won't buy buns either because they are linked.

9 Compound extinction and the rescorta-wagner model

The fact that compounding two extinction cues deepens the extinction in the individual stimuli suggests that extinction operates at least in part by an error-correction process Compounding two conditioned stimuli increases the resulting error when the trial ends without a reinforcer (surprise) -This induces a larger correction and hence greater reduction of responding -Note: Rescorla-Wagner model- In order for classical conditioning to occur the animal has to be surprised by what follows the CS (the US), if you don't surprise the animal, you don't get conditioning. A strong CS-US association means, essentially, that the CS signals or predicts the US.

8 Stimulus Generalization

The opposite of differential responding/stimulus discrimination -An organism is said to show stimulus generalization if it responds in a similar fashion to two or more stimuli Stimulus generalization gradient - a way to measure stimulus control because it provides precise information about how sensitive the organism's behavior is to systematic variations in a stimulus -A steep generalization gradient indicates strong control of behavior by the stimulus dimension that is tested -A flat generalization gradient indicates weak or nonexistent stimulus control (generalization) -> animal will respond the same to any variation of the stimulus

8 Spence's explanation of peak shift

The peak-shift effect is remarkable because it shows that the S+ or reinforced stimulis is not necessarily the one that produces the highest response rate -Generalization gradients of excitation and inhibition may overlap with the degree of overalap depending on the degree on similarity between S+ and S- Because of the overlap inhibition, S- may generalize to S+ and suppress responding to S+ resulting in a peak shift effect -More inhibition from S- to S+ is expected if S- is closer to S+ and this should result in a greater peak shift -He says this is because when the s- and s+ are so similar, part of the inhibitory effects on s- overlap with the s+ and skew the maximum responding point

6 Melioration

The third major mechanism of choice, melioration, operates on a time scale between molar and molecular mechanisms -Melioration means making something better -> Melioration theory predicts that participants will shift their behavior toward whichever choice alternative provides the higher (or better) local rate of reinforcement -Assumes that the adjustments in the distribution of behavior between choice alternatives will continue until the participant is obtaining the same local rate of reinforcement on each alternative Mechanism of melioration results in matching -Focuses on local rates of responding and reinforcement -Local rates are calculated only over the time period that the participant devotes to a particular choice alternative ->Local rates are always higher than overall rates For example with two options (A and B) the local rate of responding is calculated by dividing the frequency of responses on A by the time the participant spent on side

7 Conditioned emotional states or reward-specific expectancies?

The two process theory assumes that classical conditioning mediates instrumental behavior through the conditioning of positive or negative emotions depending on the valience of the reinforcer. -However, in some cases reward-specific expectancies appear to determine the outcome of Pavlovian instrumental transfer experiments -Expectancies for specific rewards rather than a general positive emotional state can determine the results in a transfer test Ex: reinforcement with sugar increase instrumental behavior reinforced with sugar than with food pellets

7 The Associative structure of Instrumental conditioning - three-term contingency

Thorndike first recognized that instrumental learning involved more than just a response and a reinforcer -Three events to consider - three-term contingency S- stimulus context R- the instrumental response O- the response outcome or reinforcer

10 Punishment and the Negative Law of Effect

Thorndike- Positive reinforcement and punishment involve symmetrically opposite processes and just as positive reinforcement strengthens behavior, punishment weakens it "losing a penny is three times more punishing than earning that same penny is reinforcing"

7 Processes that motivate and direct instrumental behavior -Thorndike/Pavlov vs Skinner

Thorndike/Pavlov: focus on identifying the associative structure of instrumental conditioning (considers molecular mechanisms rather than long-range goal or function of instrumental behavior) Skinnerian: instrumental behavior in the broader context of how organisms distribute or allocate their behavior among various response options -Response-allocation approach considers reinforcement effects to be a consequence of constraints of response options imposed by an instrumental conditioning behavior

7 The Response-Deprivation Hypothesis

Timberlake and Allison abandoned the differential probability principle (premack) altogether and said that even a low-probability response can serve as a reinforcer provided that participants are restricted from making that response -only important thing was that the reinforce was desired because it had been withheld from the organism. Ex: food deprivation food as reinforcer Ex: mentally retarded class- opportunity to trace became an effective reinforcer (although low probability) for filing behavior (higher probability) if the access to tracing was restricted below baseline levels -All instrumental conditioning procedures require withholding the reinforcer until the specified instrumental response has been performed —> the response-deprivation hypothesis states that this defining feature is critical for producing a reinforcement effect

9 Conducting extinction in multiple contexts

To eliminate the renewal effect due to a lack of generalization of extinction performance from the extinction context to other situations. -Conducting extinction in several different contexts may serve to increase the stimulus generalization of extinctions -Whether or not extinction in multiple contexts facilitates the generalization of extinction performance (and reduction of the renewal effect) depends on the number of extinction trials that are conducted -Conducting extinction in multiple contexts has been observed to eliminate the renewal effect

Alternative Theoretical Accounts of Avoidance Behavior -positive reinforcement

Two-process theory says that reinforcement for the avoidance response is provided by reduction of conditioned fear—> negative reinforcement: reinforcement due to the removal of an aversive stimulus -Other theories propose that avoidance procedures also provide positive reinforcement of the avoidance response

10 Avoidance and species-specific defense reactions

What controls an organisms behavior during the early stages of avoidance training ? Species-specific defense reaction (SSDR)—innate responses that are assumed to have evolved because during the course of evolution they were successful in defense against pain and injury Ex: freezing in rats, running, defensive fighting, etc. Boles proposed that the configuration of the environment determines which particular SSDR occurs Predicts that instrumental responses similar to SSDRs will be learned more easily in avoidance experiments than responses unrelated to SSDR

11 Reconsolidation -original vs contemporary view (original learning and memory test)

a reactivated memory is just as unstable and is just as susceptible to disruptions by amnesic agents as a newly learned memory consolidation and reconsolidation fundamentally involve the same process Traditional view -Original learning: STM-> consolidation-> LTM -Memory test: LTM -> STM via memory activation Contemporary view -Original Learning: Active memory consolidation -> inactive memory -Memory Test: Active memory-> reconsolidation -> inactive memory -> memory activation -> active memory

12 Tool use in nonhuman animals

assumed that the construction and use of tools requires complex cognitive processes tool use is learned over a long period of time -reflects advance intelligence -another view is that tool fabrication and use is a form of instrumental behavior reinforced by food items that are obtained by the tool tool use is not a homogenous behavioral category: various instances of tool use may reflect different behavioral and cognitive mechanisms

8 Alternative Accounts of Peak Shift

assumes that organisms learn to respond to a stimulus based on its relation to other cues in the situation Predicts that the shape of a generalization gradient will change as a function of the range of test stimuli that are presented during the generalization test session

12 Scalar Expectancy Theory (SET)

assumes that timing is retrospective providing information about an interval that has been completed (decision to respond is made after the interval has elapsed) this models considers temporally controlled behavior to be the result of three independent processes 1. A clock process: provides info about the duration of time elapsed -number of accumulated pulses by pacemaker is relayed to memory process 2. A memory process: obtains information about the duration of the current stimulus information is stored in working memory 3. A decision process: contents of working memory and reference memory are compared -decide if the durations are similar- if yes-> respond

8 Stimulus Equivalence training

equivalence training encourages generalized responding among a set of stimuli or the establishment of stimulus equivalence -train an animal to generalize stimuli so like a couple diff stimuli will make the animal do the same thing -The critical factor is to associate all of the members of an equivalence class (overarching topic) is a common event (a reinforcer like food or a common stimulus outcome)

8 stimulus discrimination procedure

establishes control by the stimuli that signal when reinforcement is and is not available. Once the S+ and S- have gained control over the individual's behavior, they are called discriminative stimuli S+ is a discriminative stimulus for performing the instrumental response S- is a discriminative stimulus for not performing the response Can also be conducted with S+ and S- at the same time next to each other Pigeons pecking faces and reinforced- easier when both faces are presented at the same time then successive trials Makes discrimination training easier -multiple schedule of reinforcement

9 Partial reinforcement extinction effect (PREE)

extinction is much slower and involves fewer frustration reactions if partial reinforcement rather than continuous reinforcement was in effect before the introduction extinction Continuous reinforcement leads to more rapid extinction The persistence in responding that is created by intermittent reinforcement is evident in many situations Child wants candy and throws tantrum until mom gives in and get its for them - intermittent reinforcement reinforces their behavior and will not cause rapid extinction -> partial reinforcement- even when animal undergoes extinction, still thinks there is a slight possibility of reinforcement and could just be having "bad luck"

7 Antecedents of the Response-Allocation Approach The premack principle

given two responses of different likelihood, H and L, the opportunity to perform the higher P response (H) after the lower P response (L) will result in reinforcement of the response L and the opposite it true if H is before L L-> H reinforces L H-> L does not reinforce H Focuses on the difference in the likelihood of the instrumental and reinforcer responses also called the differential probability principle Ex: eating will reinforce bar pressing because eating is typically more likely than bar pressing ->The power of the premack principle is that potentially any high-probability activity can be an effective reinforcer for a response that the individual is not inclined to perform

12 Language learning in non-human animals

historically thought that language is so complex and specialized that it is uniquely human -according to this view, language depends on innate neural modules only evolved by our species -another view proposes that human beings are able to use language because they are especially intelligent and have experiences that permit the acquisition of language -> suggests that nonhuman organisms may also acquire language skills if they are sufficiently intelligent and encounter the requisite experiences

10 The Avoidance Puzzle -Theories summary

how not getting something can motivate avoidance responses -Two-process theory, conditioned inhibition reinforcement, shock-frequency reduction all provide different views of what happens after an avoidance response to reinforce it -SSDR focuses on unconditioned aspects of defensive behavior which are further elaborated through the concept of predatory imminence -Two process theory explains escape from fear experiment -Safety signal explains free-operant avoidance learning and the role of response-feedback stimuli in avoidance learning -Predatory imminence explains what happens during the early stages of avoidance training -Expectancy theory is useful in dealing with cognitive aspects of human avoidance behavior

7 Pavlovian instrumental transfer experiment

how to test the idea that an S-O association can motivate instrumental behavior Phase 1: standard instrumental conditioning Phase 2: standard pavlovian conditioning Phase 3: participants allowed to perform instrumental lever press response but the Pavlovian CS is presented periodically -If a Pavlovian S-O association motivates instrumental behavior, then the rate of lever pressing should increase when the tone CS is presented -As predicted, the presentation of a Pavlovian CS for food increases the rate of instrumental responding for food Conditioned suppression occurs because the CS+ for shock elicits an emotional state (fear) that is contrary to the positive emotion or expectancy that is established in instrumental conditioning with food

12 Language training procedures -model-rival techniqu

model-rival technique—one research assistant acts as a trainer and the other acts as a rival student who competes with the parrot for the attention of the trainer. The trainer asks a question and a correct response results in praise and a change to manipulate the object and an incorrect response is reprimanded and the object is removed Kanzi learned linguistic skill from mother Matata and his use of language was not explicitly reinforced with food

11 Delayed matching to sample

most frequently used procedure in the study of nonhuman short-term remembering- tests working memory 1. Participant is presented with a sample stimulus that designates which response will be correct at the end of the trial 2. stimulus is then removed for a retention period 3. participant is then given a memory test consisting of two cues, one of which is the original sample -Choice of the sample stimulus during the memory test is the correct response and is reinforced -Tests with delayed matching to sample showed that participants with schizophrenia showed a deficit in performance when trials included a 4- or 8-second delay ->Schizophrenics have deficits in working memory

8 Stimulus discrimination training

most powerful procedure for bringing behavior under control of a stimulus Can be conducted using classical or instrumental conditioning procedures The conditioned responding that develops to A+ generalizes to B- at first, but with further training responding to B- declines and a clear discrimination becomes evident Children with fruit- at first orange and tangerine are the same but with experience they are able to discriminate between them Stimulus discrimination training with instrumental conditioning procedures Child may attempt to cross street during red and green light but eventually can discriminate and not cross during red

11 experimental psychologists -comparative cognition

objective evidence cannot prove/disprove consciousness -comparative cognition does not imply anything about awareness, consciousness, or verbal reasoning comparative cognition refers to theoretical models used to explain aspects of behavior that can't be characterized in terms of simple S-R mechanisms -employs the simplest possible explanations that are consistent with the data

12 Properties of Temporally controlled behavior -Scalar invariance

organisms respond to the relative values of time intervals rather than their absolute value Scalar invariance- A property of the temporal control of behavior that emphasizes that participants respond to time intervals in terms of their relative or proportional durations rather than their absolute durations. Ex) 3 sec vs 9 sec and 9 sec vs 27 sec (both 3 times more) are equivalent -because the longer stimulus was three times the value of the shorter one, peak responding occurred at approximately three times the duration for the longer stimulus for both -variability in response rates is proportional to the value of the interval being timed timing process can be interrupted by a break (blackout) but when the light is turned back on, the timing process continues from where it left off

12 Mechanisms of perceptual concept learning

pigeons used the normal relative position of body parts to judge whether an image was that of a person or not common elements—create a family resemblance among the individual members of a perceptual category learning abstract concepts -same vs different -the degree of variability in the objects that appear determines whether pigeons respond to visual scenes as being the same or different

10 Punishment outside the Laboratory

punishment may be the only source of attention for someone making punishment a discriminative stimulus for positive reinforcement parental punishment associated with increased immediate compliance of the part of the child also related to delinquent and antisocial behavior and increase incidence of aggressive behavior in adulthood

9 Magnitude reinforcement extinction effect

responding declines more rapidly in extinction following reinforcement with a larger reinforcer -Nonreinforcement is apt to be more frustrating if the individual expects a large reward than if the individual expects a small reward

12 Techniques for the Study of Serial Order Learning -Simultaneous chain procedure

simultaneous chain procedure- A procedure that requires a series of responses but all of the response options are always available. Ex: calling someone: all the numbers are available but to call someone you must enter them in the correct order Sequences made of smaller chunks are learned much more easily than sequences that have no structure or sequences in which the internal is more complicated - rats have ability to chunk- I.e. group 123, 234, 345 in threes- rats learned faster if info was chunkable

11 synaptic consolidation systems consolidation

synaptic consolidation—changes in synaptic efficacy that presumably underlie learning -fairly rapid process (minutes) -changes at the cellular and molecular level including synthesis of new proteins systems consolidation—changes at the level of neural circuits and systems -slower process (days to weeks) -changes in how memory is represented in hippocampal or cortical circuits

12 Oscillators Instead of Pacemakers

temporally organized behavior is mediated by oscillators rather than pacemakers ex) 24 hours in a day -Circadian rhythms -periodicity of food-cup entries generated by delivering food at fixed intervals persists after the food is no longer provided. Such results are not predicted by SET and suggest that it is based at least in part on the entrainment of a self-sustaining endogenous oscillator.

9 Overtraining extinction effect

the more training that is provided with continuous reinforcement, the stronger will be the frustration that occurs when extinction is introduced and produces more rapid extinction -Paradoxical because one may think that extensive training should create a stronger response that is more resistant to extinction but it's the opposite -The more extensive the training, the more rapid the extinction