Quiz 3 SPCE609

Clicker Training in Humans

-Dr. Levy teaching humans to throw frisbees for ultimate frisbee by delivering a click when they are holding the frisbee correctly. -Levy and Karen Pryor developed a clicker training program for teaching students to execute complex surgical techniques -Could be effective for dancers and physical therapy. Page 211-212

Two ways to interpret motivation

-That motivation is nothing more than a concise description of a long-term pattern of behavior. In other words, a motivated student, employee, or parent is someone who has performed well for weeks, months, or years. This concise description allows us to make predictions about human behavior. -Motivation is inherent to the individual. It is a mental essence existing inside the individual, explaining why they behave the way they do (more common but less useful explanation) Page 230

Two factors that influence how quickly behavior decreases to baseline levels under an operant-extinction contingency

-The rate of reinforcement prior to extinction (the higher the rate of reinforcement, the faster extinction will work) -The individual's motivation to acquire the reinforcer (the more the reinforcer is needed, the more persistent behavior will be during extinction). Page 174

Principles of effective shaping

1. Asks us to provide an objective definition of a terminal behavior 2. Asks us to evaluate what the novice player can currently do and how that falls short of the terminal behavior 3. Provides advice for setting the reinforcement contingences - the response approximations should challenge the player, but not so much that the reinforcers cannot be obtained. 4. Differential reinforcement - reinforcing one response and extinguishing other previously reinforced responses. 5. Suggests we ensure the learner has mastered the current response approximation before moving on to the next one. 6. If the individual struggles to obtain reinforcers at the next level, this instructs us to lower the criterion for reinforcement Commonly used for video games. Page 217-218

four other predictable effects of extinction on behavior:

1. Extinction-induced emotional behavior 2. Extinction burst 3. Extinction-induced variability 4. Extinction-induced resurgence Page 179

Five strategies to replace bad habits with good habits

1. Identify the antecedent stimuli that reliably evoke our bad habits. 2. Replace those stimuli that evoke bad habits with stimuli that will evoke good ones. 3. Reinforce the desired behavior hundreds, if not thousands of times (can use shaping and start off by setting the bar low) 4. Make sure the behavior is reinforced 5. Gradually increase the goal

Flow will occur...

1. In the "Goldilocks zone" where the challenge is neither too hard nor too easy. 2. If there are proximal goals (thought of as contingency arranging a conditioned reinforcer along the way) 3. When a skillful or unskillful response occurs, it needs to be known right away. Page 216

Principles of effective Pavlovian conditioning translated to conditioned reinforcement

1. Use an effective backup reinforcer 2. Use a salient conditioned reinforcer 3. Use a conditioned reinforcer that signals a large delay reduction to the backup reinforcer 4. Make sure the conditioned reinforcer is not redundant Page 207-209

Before we can use extinction to positively influence behavior, we need to do two things:

1. We have to determine if the problem behavior is operant behavior (if it is not then extinction is not the right therapeutic tool) 2. If the problem behavior is operant, then we need to identify the reinforcer that maintains it. To implement extinction, we need to turn OFF the reinforcer that maintains the behavior. Page 184

Primary Reinforcers

A consequence that functions as a reinforcer because it is important in sustaining the life of the individual or the continuation of the species. Individuals are more likely to survive if their genes establish life-sustaining consequences as reinforcers. Ex. Food, water, oxygen, sleep, and sexual. Page 201-202

Automatic reinforcer

A consequence that is directly produced by the response - it is not provided by someone else - and which increases the behavior above a no-reinforcer baseline. If automatic reinforcers maintain a problem behavior then extinction is impossible (you cannot turn OFF the automatic stimulation experienced each time face slapping occurs) Page 186

principle 2: use a salient conditioned reinforcer

A noticeable conditioned reinforcer will work better than one that is easily overlooked. Ex. Giving a child a token and having her place it on a token bank is more salient than dropping the token into the bank for her. and A clicker for a dog Page 207

Dictionary definition of habit

A pattern of behavior that is repeated so often that it becomes almost involuntary. Page 248

Differential reinforcement

A procedure in which a previously reinforced behavior is placed on extinction while a second behavior is reinforced. 1. Provides the opportunity to teach an adaptive behavior that will replace the problem behavior. 2. Decreases the frequency of extinction bursts and extinction-induced aggression . Ex. Nasser et al (2016) reported that providing a slow-release medication called burenorphine significantly blocked the reinforcing effects of an opiate drug. Page 187-189

Stimulus Preference Assessments

A rank-ordered list of preferred stimuli is obtained by observing choices between those stimuli Often used on animals by pet companies to improve likeability of their product Page 239-240

The "Liking" Strategy

A second strategy when arranging effective reinforcers is to identify things the individual likes. Behavior analysts use reinforcer surveys and stimulus preference assessments to do this. Page 238

Verbal learning

A second way in which a consequence can come to function as a conditioned reinforcer. During this, information is provided indicating that the conditioned stimulus signals a delay reduction to another reinforcer. That is, the Pavlovian CS--US contingency is verbally described. Ex. If humans are instructed that a red light (CS) will precede an electric shock (US), when that instructed CS is encountered, an involuntary physiological fear response occurs, even when the shock does not. Page 204

Flow

A state in which one feels immersed in a rewarding activity and in which we lose track of time and self Page 215-216

Reinforcer surveys

A structured interview or written survey that asks the individual to identify highly preferred activities. May include items meant to jog someone's memory (help you remember what you like to do) and wish list items (things you want but cannot afford to do). Page 238

Conditioned Reinforcers in zoos

A trainer at the San Diego Zoo is teaching an elephant that the sound of a whistle is a conditioned reinforcer. Principle 1: Uses apple slices as an effective backup reinforcer Principle 2: This whistle in her mouth makes a unique and salient noise Principle 3: Which signals a delay reduction to the next apple slice. Principle 4: The trainer is careful not to present any other stimulus change that could signal delay reduction to food. Page 210

Premack Principle

Access to high-probability behavior will function as a reinforcer when made contingent upon a low-probability behavior. Page 242

B.F. Skinner and The Behavior of Organisms

After each of the rats had pressed the lever and earned food about 100 times, Skinner disconnected the feeder so the lever pressing no longer produced this reinforcer. This introduced an extinction contingency. IF lever press--THEN no food Once the operant extinction contingency was introduced, the rate of lever pressing gradually decreased until the pressing returned to baseline level. Page 170

Motivating Operations (MO)

An environmental and/or biological event that (1) temporarily alters the value of a specific reinforcer and (2) increases/decreases the probability of behaviors yielding that reinforcer. Page 232

Abolishing Operations (AO)

An environmental and/or biological event that (1) temporarily decreases the value of a specific reinforcer and (2) decreases the probability of behaviors yielding that reinforcer. Ex. Sexual climax temporarily decreases the value of sexual stimulation. Many prescription and recreational drugs have AO effects. Page 233

Establishing Operations (EO)

An environmental and/or biological event that (1) temporarily increases the value of a specific reinforcer and (2) increases the probability of behaviors yielding that reinforcer. Ex. Being deprived of oxygen or depriving an individual of an operant behavior. Eating a tub of salty popcorn will increase the efficacy of the reinforcer water. If your car breaks down on side of the road, this would increase efficacy of reinforcer mechanic. Page 232-233

Habits defined by behavioral scientists

An operant behavior that (1) is evoked by antecedent stimuli and (2) persists despite the imposition of an AO. They are less influenced by motivation to acquire the reinforcer and more influenced by the antecedent stimuli that were reliably present when the response was reinforced. Page 248

Purchase task

Because it is often impractical to actually measure breakpoints in clinical or workplace, researchers developed this. In this task, participants indicate how many reinforcers they would hypothetically purchase if they were available at a range of prices. Under this task, the breakpoint is the highest price at which the individual continues to purchase the reinforcer. Page 244

Mentalistic Explanations of Behavior

Behavior analysts reject this. These explanations put the cause of behavior in the individual's vaguely defined mind. Accordingly, high performing employees are assumed to have minds full of motivation, while poor performing employees have motivational deficits. Page 230-231

Two ways to measure reinforcer efficacy

Choice and breakpoint Page 244

Voulgarakis and Forte (2015)

Created a two contingency change with an 8 year old boy with cerebral palsy whose long-lasting food refusals left him undernourished. The first contingency was designed to reduce food refusals: IF food refusal (tantrum)--THEN food is not removed The second contingency change was designed to increase eating: IF food consumed--THEN child can choose to end the meal Page 173

Shaping

Differential reinforcement of successive approximations to a terminal behavior. Page 213

Study regarding Flow

Dr Mihaly had a study in which participants wore pagers that interrupted them at unpredictable times during the day. At each interruption, participants recorded what they were doing and how they felt. Found that flow was related to response-reinforcer contingencies. Page 216

At home Example

Finn is usually reinforced with treats or a scratch on the head for sitting. When they began asking him to sit with no reinforcers, after several times he walked away and did not sit anymore. The next day when Finn was asked to sit, he did (spontaneous recovery) and this time he was reinforced. Page 183

Griffiths et al (1978)

Found that breakpoint for cocaine was about 4x higher than chlorphentermine's. The baboons in this study worked 4x as hard for a cocaine reinfrorcer than chlorphentermine. Page 244

Rate of reinforcement prior to extinction

If behavior has been reinforced every time it occurs, then after extinction starts, behavior will quickly decrease to baseline levels. Conversely, if behavior was infrequently reinforced, then following the contingency change to extinction it will take longer for behavior to decrease to baseline levels. Page 174

Attractive features of token economies

Motivationally robust Non-disruptive Fair compensation Portability Delay-bridging Page 206

Example of Shaping Human Behavior

Plants vs. Zombies game designed to reinforce successive approximations of the terminal behavior. The first approximation taught to novice players is clicking on the seed packet. The next approximation is to click sums as they fall from the sky. Page 215

Alcohol Habit Study (Corbit et al. 2012)

Rats learned to press a lever to produce an alcohol reinforcer. After 1 week the lever pressing was not yet habitual. When AO was imposed it decreased lever pressing by 40%. However, after 8 weeks the same AO no longer decreased the rats' alcohol seeking. Therefore, the behavior became habitual. Page 249

David Premack's experiment to address the circulatory problem

Rats were given free access to water and an exercise wheel for an hour. Found that rats spent more time running than drinking, he classified running as the high-probability behavior and drinking as the low-probability behavior. Next, he predicted what would function as a reinforcer. Made the contingency IF drink--THEN unlock running wheel. As predicted, the rats increased their drinking to access the running wheel. Therefore, the running wheel acted as a reinforcer. Page 242

Contingency

Reinforcers are more effective when delivered contingently and ineffective when provided noncontingently. Few will work for a reinforcer when it may be obtained for free. Page 245

Reinforcer Immediacy

Reinforcers are more effective when they are obtained immediately Reasons: -has to do with stable environment of our evolutionary ancestors. -delayed reinforcers make it more difficult to learn response-reinforcer contingencies. Ex. Immediate food reinforcers maintained the highest rate of key-pecking for pigeons. Page 246-247

Differential reinforcement of incompatible behavior (DRI)

Reinforcing a response that is topographically incompatible with the problem behavior. Zoo Ex. Had monkey that had been throwing feces hold a ring outside the cage instead. Page 190

Extinction-Induced Emotional Behavior demonstration (Sullivan, et al. 1992)

Researchers taught infants to pull a string to produce an audio-visual display in their crib. When the contingency was changed to extinction, their emotions shifted from engaged, interested, and enjoying themselves to anger. Extinction can lead to frustration and even physical aggression later in life. Under long term extinction contingencies such as a job and not being able to find another one, depression is common. Page 179

Differential reinforcement of low-rate behavior (DRL)

Responding quickly is extinguished and responding slowly is reinforced. Ex. If you are in another country and someone is speaking in a foreign language too quickly for you to understand. Page 192

Pavlovian Learning Example

Skinner placed a rat into an empty chamber with a cup attached to the wall. Periodically a mechanical dispenser made a "ka chunk" noise and then a half a second later, a small piece of food fell into the cup. Because of the signaled delay, the ka chunk functioned as a conditioned stimulus. Skinner arranged an operant contingency IF lever press---then ka chunk. The rats learned to press the lever when the only consequence was the ka chunk. However, as they continued to earn ka-chunks, they gradually decreased their rate of lever pressing. Because it no longer signaled food coming (US). Page 202-203

Extinction Burst

Terminating reinforcement contingency sometimes produces a temporary increase in the rate, magnitude, or duration of the previously reinforced response Page 179-180

principle 3: use a conditioned reinforcer that signals a large delay reduction to the backup reinforcer

The bigger the delay reduction to the backup reinforcer, the more effective the conditioned reinforcer will be. Delay reduction= US--US interval/CS--US interval Page 208

Motivation (Perin, 1942)

The hungriest rats pressed the levers during extinction far more often than the ones who had eaten 1-3 hours ago. Page 175

MOs can only change the probability of behaviors that lead to that reinforcer if....

The individual has learned the response-reinforcer contingency Ex. If you are hungry you will not begin engaging in food seeking behaviors if you haven't learned the contingency of IF food-seeking behavior--THEN food reinforcement contingency. Page 234

The Shaping Game

The object of the game is to shape a novel terminal behavior (something the player doesn't normally do), but without telling the player what they're supposed to do. Page 220

Differential reinforcement of alternative behavior (DRA)

The reinforced response could be any adaptive behavior (does not need to be topographically incompatible with the problem behavior). Ex. A teacher who calls on children when they raise their hand appropriately, and ignores children when they yell out the answer. Page 191

Differential reinforcement of high-rate behavior (DRH)

Used if the rate of a behavior is too slow. Low rate-responding is put on extinction and high-rate responding is reinforced. Ex. In the game "5-second rule" the players have 5 seconds to name three things that fit in the category. If they respond quickly they get the reinforcer (points) and if not, no points (extinction) Page 192

Principle 4: Make sure the conditioned reinforcer is not redundant

We want our conditioned reinforcer to be the only stimulus signaling a delay reduction to the backup reinforcer. Ex. You don't want to use verbal praise at the same time as using the clicker for a dog because then the clicker becomes redundant. Page 209-210

Generalized conditioned reinforcer

When a conditioned reinforcer signals a delay reduction to more than one backup reinforcer Page 207

B.F Skinner and E.O Wilson found considerable agreement on...

When it comes to reinforcers, "everything goes back to genes" Our genes have prepared us to to find certain consequences reinforcing. Page 201

Extinction-Induced Resurgence

When one operant behavior is extinguished, other (different) behaviors that were previously reinforced are emitted again. Ex. When an animal trainer, Karen Pryor, stopped reinforcing near the bottom of the tank. the porpoise (Hou) began to emit behaviors that used to be reinforced with fish treats such as corkscrewing, tail-slapping, and jumping in and out of the water. Ex. Bruzek et al. found that when one infant-caregiving behavior (rocking the baby) no longer soothed the crying child (escape extinction), old caregiving behaviors that used to be negatively reinforced (feeding or playing) resurged. Page 182

Davenport and Olson (1968)

When rats pressed a lever it turned off an aversive noise. This is a negative reinforcement contingency. However, in the extinction phase, pressing the lever no longer worked. When this contingency change occurred, the lever pressing decreased to baseline level. Page 171

Shaping animal behavior on the farm (Ferguson and Rosales-Ruiz, 2001)

Worked with quarter horses who were afraid to enter a horse-trailer. They established the sound of a clicker as a conditioned reinforcer (a variety of treats delivered a few seconds after the salient sound of a clicker) The terminal behavior was voluntarily walking into the trailer and the first approximation of this was touching a target (red pot holder) The second approximation was to touch the target regardless of its location (on the ground, fencepost, on a tree) By gradually placing the potholder further inside the trailer, the horse voluntarily entered the trailer without incident. Page 214

preference hierarchy

a list of stimuli rank ordered from most to least preferred Page 239

token economy

a set of rules governing the delivery of response-contingent conditioned reinforcers (tokens, points, etc.) that may be later exchanged for one or more backup reinforcers Ex. The "Good Behavior Game" often used at schools assigning students to teams and points, later exchangeable for goods and privileges. Page 205

Differential reinforcement of variability

a unique contingency in which responses, or patterns of responses, that have either never been emitted before or have not been emitted in quite some time are reinforced, and repetition or recent response topographies are extinguished. Ex. Ross and Neuringer (2002) asked college students to draw squares and rectangles on a computer screen. In one group, reinforcers were obtained if the images were drawn novel in size. For another, reinforcers were obtained if images were drawn on a novel location on the screen. None of the students were informed of these reinforcement contingences but both groups increased their variability along the appropriate dimension. We need not be consciously aware of reinforcement contingences for them to influence our behavior. Page 191-192

Extinction-Induced Variability

an increase in the variety of operant response topographies following extinction Ex. When rats moved the rod suspended from the ceiling two things happened: (1) it obtained a positive reinforcer (food) and (2) Dr. Iverson's camera took a picture of the rat. The rat's rod-deflecting behavior was almost identical each time. After response 140, Iverson changed the contingency to operant extinction and moving the pole did not produce food. Responses 141 and 142 were unchanged but after that, the topography of behavior was more variable. Page 180

Habits are formed when...

an operant response has been repeatedly reinforced, hundreds, if not thousands of times in the presence of the same antecedent stimulus Page 249

Conditioned Reinforcers

consequences that function as reinforcers only after learning occurs Ex. Money (we learn the value of money later in life) Page 202

Four dimensions of effective reinforcers

contingency, size, quality, and immediacy Page 245-246

Partial reinforcement extinction effect (PREE)

direct relation between prior reinforcement rate and how quickly behavior undergoes extinction Page 174

Two types of motivating operations

establishing operations (EO) and abolishing operations (AO) Page 232

Reinforcer size

larger reinforcers are more effective than smaller reinforcers. However, the larger the reinforcer, the sooner satiation will occur. Page 245

Percentile schedule of reinforcement

offers a simple automated training technique incorporating the six principles of effective shaping. Behavior analysts have used this to improve academic performance and social interactions among children with disabilities. Often used to help people smoke less or help obese individuals increase their daily activity levels in a 4-month period. Page 221-222

Functional Communication Training

problematic demands for attention are extinguished while appropriate requests are established and reinforced. Has proven effective in reducing inappropriate requests for social reinforcers in a variety of populations and settings. Page 191

Differential reinforcement of other behavior (DRO)

reinforcement is provided contingent upon abstaining from the problem behavior for a specified interval of time; presumably while "other behavior" is occurring This can be effective but it does not teach a specific activity to replace problem behavior, and increasing intervals between reinforcers can produce resurgence of problem behavior. Page 191

Escape Extinction

responding that meets the negative reinforcement contingency no longer removes or reduces the aversive event; as a result, responding decreases to baseline (no-reinforcer) levels Page 172

Operant Extinction

responding that meets the reinforcement contingency no longer produces the reinforcer and, as a result, it falls to baseline (no-reinforcer) levels. Can be used to eliminate problem behavior Page 169-171

Spontaneous recovery of operant behavior

temporary resumption in operant responding following time away from the extinction setting Page 175

Principle 1: Use an effective backup reinforcer

the better the backup reinforcer, the more effective the conditioned reinforcer will be. One strategy for this is using a token that can be exchanged for a lot of different backup reinforcers. Page 207

Marking

the conditioned reinforcer immediately follows the response, and this helps the individual learn which response produced the backup reinforcer Ex. The trainer presents the "click" of the clicker at the exact moment the dog's backside touches the ground when teaching a dog to sit and helps the dog to learn "IF sit--THEN treat" Page 208

Breakpoint

the maximum amount of behavior the reinforcer will maintain Page 244

Terminal behavior

the performance you ultimately want Page 213

backup reinforcer

the reinforcer provided after the conditioned reinforcer signals the delay reduction to its delivery Page 205

Functional Analysis of Behavior

the scientific method used to 1) determine if a problem behavior is an operant and 2) identify the reinforcer that maintains that operant. A brief experiment in which consequences that might be reinforcers are turned ON and OF, while the effects of these manipulations on problem behavior are recorded. Page 186

Reinforcer quality

the subjective value of a reinforcer, which can vary from one individual to the next. Page 246