CH 4 Reinforcement and Extinction of Operant Behavior

Operant Level & Continuous Reinforcement

• *Operant level* - the baseline rate of response or the rate of response before any known conditioning. - The rate of responses emitted before any reinforcer is made contingent on a response. • *Continuous reinforcement (CRF)* - A schedule of reinforcement that produces a reinforcer for every individual response

Operant Behavior

• Operant behavior operates on the environment to produce consequences that in turn strengthen the behavior if it is reinforced. • Unlike respondent behavior, which is elicited by a stimulus, operant behavior is selected, shaped, and maintained by the consequences that have followed it in the past. • Also unlike respondent behaviors, whose topography and basic functions are predetermined, operant behaviors can take on an unlimited range of forms and vary in topography (i.e. how many different ways can you open a bag of chips?) • This is known as *operant conditioning*. • Operant conditioning as a process, has evolved over species history and is based on genetic endowment. That is, operant (and respondent) conditioning as a general behavior-change process is based on phylogeny.

The Partial Reinforcement Effect (PRE)

• Resistance to extinction is substantially increased when an *intermittent schedule of reinforcement* has been used to maintain behavior. • *Intermittent schedule of reinforcement* - Only some responses are reinforced. Instead of reinforcing every response, as in CRF, the organism may only be reinforced after every 30 emitted responses. • *The higher the rate of reinforcement, the greater the resistance to change*. • Extinction occurs more rapidly on behavior with a history of *continuous reinforcement (CRF)* relative to behavior with a history of intermittent reinforcement due to discrimination between reinforcement and extinction. - An organism can discriminate between CRF and extinction more easily than between a lean and intermittent schedule and no reinforcement

An Important Note to Remember:

- ReinforceMENT/PunishMENT = The process of increasing the frequency of a behavior/response. - ReinforceR/PunisheR = The stimulus that behavior/response(s) strengthening is contingent upon.

Two Types of Conditioning

1.*Classical Conditioning* - Respondents - reflexive, elicited behavior - CS controls behavior 2. *Operant Conditioning* - Operants - emitted behavior - Outcomes ("consequences") control behavior

Four Basic Contingencies

1.*Positive Reinforcement* - one of the four basic contingencies of operant behavior. Positive reinforcement is when a stimulus follows behavior and, as a result, the rate of that behavior increases. • EX: Immediately after presenting a project you have been working on for your job, your boss says, "excellent job! I am giving you a raise!". Your behavior has been reinforced by both praise and money (the raise). • Positively reinforcing events usually include consequences such as food, praise, and money. These events, however, cannot be said to be positive reinforcers until they have been shown to increase behavior.

Premack's Principle

A higher frequency behavior will function as reinforcement for a lower frequency behavior • Also known as Grandma's law • "Preferred activities" serves as a reinforcer for lowerfrequency behaviors. EXAMPLE: Grandma says in order to have dessert, you must finish all of your vegetables at dinner. • Being able to go out with your friends depends on how much studying you get done

Behavioral Effects of Extinction

Extinction produces several behavioral effects in addition to a decline in a rate of response: 1. *Extinction Burst*: When extinction is started, operant behavior tends to increase in frequency. • An initial increase in rate of response, or extinction burst, occurs when reinforcement is first withdrawn. • EXAMPLE: You stopped answering and returning calls from a girl you went on a few dates with. At first, she blows up your phone, calling constantly, and then it eventually subsides. 2. *Operant variability*: In addition to extinction bursts, operants show increases in operant variability, variations in form or topography as extinction proceeds. • EXAMPLE: The girl from the above example starts texting you, sending messages on facebook, and calling from blocked numbers.

Reinforcement/Punishment

• *Reinforcement*: Occurs when a stimulus change immediately following a response results in the *increase* in the frequency of that behavior. • Reinforcer: (1) A stimulus, event (an environmental event that can be detected by one of the senses), activity, or condition that will increase (strengthen) the future frequency of the responding(behavior) that it has immediately followed. • EX: Smiling is a reinforcer that may generate conversation and will increase the probability of you smiling at others in the future. • *Punishment*: Occurs when a stimulus change immediately following a response results in the *decrease* in the frequency of that behavior. • *Punisher*: (1) A stimulus, event (an environmental event that can be detected by one of the senses), activity, or condition that will decrease (weaken) the future frequency of the responding(behavior) that it has immediately followed. • EX: A popup box asks, "warn when deleting unread messages?" You click on "No", and an important email is lost. In the future you will probably not disregard the popup box. A--B--C

What is shaping?

• *Shaping* - Differential reinforcement of successive approximations to a terminal behavior. - Each behavior that is approximate to the end behavior is reinforced, while behaviors that are farther from the target behavior, are not reinforced (extinction). - *Differential reinforcement* - procedure in which only targeted behaviors are reinforced, while behaviors that do not demonstrate qualities of target behavior are not reinforced. • Shaping is used to teach new behavior not already in the repertoire. - *Repertoire* - All the behavior an organism is capable of emitting on the basis of species and environmental history. • Involves the principles of reinforcement and extinction - Successive approximation are closer and closer responses to the terminal response. - Terminal response is the end response.

Contingencies of Reinforcement

• A *contingency of reinforcement* defines the relationship between the events that set the occasion for behavior, the operant class, and the consequences that follow this behavior. • ABC's = Antecedent -- Behavior --Consequence

Satiation

• A decrease in the frequency of behavior presumed to be the result of the continued contact with or consumption of a reinforcer that has followed the behavior. • Providing copious amounts of a reinforcing stimulus leads to a decrease in the effectiveness of that reinforcer.

Examples of SDs*

• A green traffic light • A professor's question • Telephone ring sets the occasion for picking it up and being reinforced with conversation. • A newly lit up sign on register 7 at a very busy grocery store will set the occasion for you to get to that lane before anyone else. *Please note that the examples above are not SDs under all circumstances. It depends on the person's history of conditioning

Deprivation

• A state of an organism with respect to how much time has elapsed since it consumed or contacted a particular reinforcer. • Withholding access to a particular reinforcer increases its effectiveness - For example, a certain level of food deprivation is necessary for the presentation of food to function as a reinforcer. - Someone who just got back from an all-you-can-eat-buffet, will probably not be reinforced with food. The value of food is increased when you are hungry.

Spontaneous Recovery and Extinction

• After a session of extinction, the response rate may be close to operant level. • When the animal is placed in operant chamber the next day (still on extinction) it will respond above operant level; *Spontaneous Recovery*. • Recall that other confounding variables during initial training can come to regulate behavior. • After repeated sessions of extinction, the level of recovery will decline.

What is a conditioned reinforcer/ punisher?

• An event or stimulus that acquires its reinforcing function over the lifetime of an organism. • These stimulus changes function as reinforcers and punishers only because of their prior pairing with other reinforcers or punishers. • Conditioned reinforcers are different for everyone since they are not related to any biological need. However, there are widely effective conditioned reinforcers that many relate to because people have similar experiences. - EXAMPLE: Social praise, money

Behavioral Effects of Extinction cont..

• Force of Response Reinforcement may be made contingent on the force of response, resulting in *response differentiation* • When extinction occurs, the force of lever pressing becomes more variable. Interestingly, some responses were more forceful than any emitted during reinforcement or during operant level. This increase in response force may be due to behavior generated by extinction procedures. EXAMPLE: That girl whose calls you are no long picking up my start hitting the buttons harder when she is calling you.

Operant Behavior cont..

• From a scientific perspective, operant behavior is lawful and may be analyzed in terms of its relationship to environmental events. • Formally, responses that produce a change in the environment are called *operants*. • The term operant comes from the verb to operate and refers to behavior that operates on the environment to produce a consequence. • A *positive reinforcer* is defined as any consequence that increases the probability of the operant that produced it. • EX: If a child's independent play increases because play is immediately reinforced by a parent's praise and attention, then the parent's praise is considered a positive reinforcer for the child

Examples of S∆'s

• If there is an "out-of-order" sign on the vending machine, you will not try to use it. • If you have a professor that ignores you every time you raise your hand, you will be less likely to ask questions in his class in the future.

Delivery of Reinforcers and Punishers

• No longer than one second after the response occurs should the reinforcer/punisher be delivered • Should be immediate! • Delay of reinforcement/punishment consists of any reinforcers/ punishers being delivered with a latency period of 2 seconds or longer. - Even a one second delay, can reinforce or punish the wrong behavior • If the reinforcer/punisher is not delivered immediately, then there is a strong possibility that the wrong response will be reinforced or punished. • *After returning home from a long day at work, you find a pile of dog poo in the corner of the living room. You've been training Diesel, your dog, not to do this and up until now, he has been pretty successful. You are frustrated and rub his nose in the pile. Is this going to work as a punisher for Diesel's inappropriate behavior of going to the bathroom in the house?*

Discriminative Stimuli

• Stimuli may also precede operant behavior. These events do not force the occurrence of the response that follows but increase the probability of a behavior occurring under some antecedent condition. • An event that precedes an operant and alters its likelihood is said to set the occasion for behavior and is called a *discriminative stimulus*, or SD. • The consequences that follow operant behavior establishes the control exerted by discriminative stimuli. When an SD is followed by an operant that produces positive reinforcement, the operant is more likely to occur the next time the stimulus is present, and unlikely to occur in the absence of that Sᴰ. • When an operant does not produce reinforcement, the stimulus that precedes the response is called an S-delta (S∆.) In the presence of an S-delta, the probability of emitting an operant declines. • An SD (or an S∆) are not defined by their physical parameters but rather by preceding and altering the probability of responses.

Operant Extinction

• The process of withholding reinforcement for a previously reinforced response is called *extinction*. • The procedure of extinction is a contingency of reinforcement. • The contingency is defined as zero probability of reinforcement for the operant response. • Extinction is also a behavioral process and, in this case, refers to a decline in rate of response caused by withholding of reinforcement.

Operant Extinction

• The process of withholding reinforcement for a previously reinforced response is called *extinction*. • The procedure of extinction is a contingency of reinforcement. • The contingency is defined as zero probability of reinforcement for the operant response. • Extinction is also a behavioral process and, in this case, refers to a decline in rate of response caused by withholding of reinforcement

Shaping Cont

• The shaping process begins with reinforcement of responses already in the learners current repertoire. • The responses that are already in the learners current repertoire that are initially reinforced are topographically similar in some way to the terminal behavior. This is referred to as the *Initial Behavior* * What are Successive Approximations?* • The gradually changing criterion for reinforcement during shaping results in a succession of new responses (*Successive Approximations*), each one closer in form to the terminal behavior.

Satiation Examples

• You have just eaten too much food (e.g., Thanksgiving). • You hang out with your friend for three days straight for 24 hours a day. • You just moved in with your girlfriend and enjoy it when she goes to work.

Deprivation Examples

• You haven't drank liquids for 24 hours • You've only slept 5 hours in the last two days • You haven't eaten for 8 hours • Your significant other is out of town for six months • You haven't gotten paid for a month