Chapter 5- Learning Notecards AP PSYCH
Conditioned reinforcer
A conditioned reinforcer, also called a secondary reinforcer, is one that has acquired reinforcing value by being associating with a primary reinforcer. The classical example of a conditioned reinforcer is money. Money is reinforcing not because those flimsy bits of paper and little pieces of metal have value in and of themselves, but because we've learned that we can use them to acquire primary reinforcers and other conditioned reinforcers.
Primary reinforcer
A primary reinforcer is one that is naturally reinforcing for a given species. That is, even if an individual has not had prior experience with the particular stimulus, the stimulus or event still has reinforcing properties. EX: Food, water, adequate warmth, and sexual contact are primary reinforcers for most animals, including humans.
Operant conditioning
American psychologist B.F. Skinner searched for the "lawful process" that would explain "order in behavior" Skinner acknowledged the existence of what he called "internal factors," such as thoughts, expectations, and perceptions. However, Skinner believed that internal thoughts, beliefs, emotions, or motives could not be used to explain behavior. These fell into the category of "private events" that defy direct scientific observation and should not be included in an objective, scientific explanation of behavior. Skinner(1953) coined the term operant to describe any "active behavior that operates upon the environment to generate consequences." In everyday language, Skinner's principles of operant conditioning explains how we acquire the wide range of voluntary behaviors that we perform in daily life. In a nutshell, Skinner's operant conditioning explains learning as a process in which behavior is shaped and maintained by its consequences. EX: Reinforcement is a consequence of behavior. It is said to occur when a stimulus or an event follows an operant and increases the likelihood of the operant being repeated. Suppose you put your money into a soft drink vending machine and push the button. Nothing happens. You push the button again. Nothing. You try the coin-return lever. Still nothing. Frustrated, you slam the machine with your hand. Yes! Your can of soda rolls down the chute. In the future, if another vending machine swallows your money without giving you what you want, you are likely to hit the machine. In this example, slamming the vending machine with your hand is the operant- the active response you emitted. The soft drink is the reinforcing stimulus, or reinforcer- the stimulus or event that is sought in a particular situation. In everyday language, a reinforcing stimulus is typically desirable, satisfying, or pleasant.
Little Albert Study
At about the same time Pavlov was conducting his systematic studies of classical conditioning in the early 1900s, a young psychologist named John B. Watson was attracting attention in the United States. Watson, like Pavlov, believed that psychology, was following the wrong path by focusing on the study of subjective mental processes. Watson's interest in the role of classical conditioning in emotions set the stage for one of the most famous and controversial experiments in the history of psychology. In 1920, Watson and a graduate student named Rosalie Rayner set out to demonstrate that classical conditioning could be used to deliberately establish a conditioned emotional response in a human subject. Their subject was a baby, whom they called "Albert B," but who is now more popularly known as "Little Albert." Little Albert lived with his mother in the Harriet Lane Hospital in Baltimore, where his mother was employed. Watson and Rayner (1920) first assessed Little Albert when he was only 9 months old. Little Albert was a healthy, unusually calm baby who showed no fear when presented with a tame white rat, a rabbit, a dog, and a monkey. He was also unafraid of cotton, masks, and even burning newspapers. But, as with other infants whom Watson had studied, fear could be triggered in Little Albert by a sudden loud sound-clanging a steel bar behind his head. In this case, the sudden clanging noise is the unconditioned stimulus, and the unconditioned response is fear. Two months after their initial assessment, Watson and Rayner attempted to condition Little Albert to fear the tame white rat (the conditioned stimulus). Watson stood behind Little Albert. Whenever Little Albert reached toward the rat, Watson clanged the steel bar with a hammer. Just as before, of course, the unexpected loud clang (the unconditioned stimulus) startled and scared Little Albert (the unconditioned response). During the first conditioning session, Little Albert experienced two pairings of the white rat with the loud clanging sound. A week later, he experienced five more pairings of the two stimuli. After only these seven pairings of the loud noise and the white rate, the white rat alone triggered the conditioned response-- extreme fear-- in Little Albert. They also found that stimulus generalization had taken place. Along with fearing the rat, Little Albert was now afraid of other furry animals, including a dog and a rabbit. He had even developed a classically conditioned fear response to a variety of fuzzy objects.
Aggression study
Bandura's early observational learning studies showing preschoolers mimicking the movie actions of an adult and pummeling a Bobo doll provided a powerful paradigm to study the effects of "entertainment" violence. Bandura found that observed actions were most likely to be imitated when: the model is attractive, of high status, and/ or a dominant member of the viewer's social group, the model is rewarded for his or her behavior, the model is not punished for his or her actions. Over the past four decades, more than 1,000 studies have investigated the relationship between media depictions of violence and increase in aggressive behavior in the real world. How prevalent is violence on American Television? The amount of violence depicted on American television is staggering. One major research project, the National Television Violence Study, measured depictions of violence in more than 8,000 hours of cable and network programming. They found: 60% of television programs contain violence, 55% have violent interactions did that not include any expression of pain, 80% have violent programs that did not show any long-term negative consequences of the violence, 58% show violent scenes in which aggression is not punished, 40% show violent scenes perpetrated by the "good guys" in their roles as heroes, 10,000 is the number of violent incidents a preschooler would observe by watching two hours of cartoons a day for a year. One large scale mea-analysis pooled the results of more than 200 studies on the effects of violent media. The meta-anaysis found a positive statistically significant overall correlation of +.31 between media violence and aggressive behavior. In other words, the greater the exposure to media violence, the greater the likelehood that someone will behave aggressively. Children who see a lot of violence are more likely to view violence as an effective way of settling conflicts and are more likely to assume that violence is acceptable behavior. In addition, viewing violence can lead to emotional desensitization in real life and can decrease the likelihood that people will help a victim of violence in real life. Are there long term effects? The results show that men who watched the most television violence as children were significantly more likely to have pushed, grabbed, or shoved their spouses and to have shoved another person in response to an insult. They were also three times more likely to have been convicted of a crime.
Biological preparedness
Biological Preparedness is the idea that an organism is innately predisposed to form associations between certain stimuli and responses. If the particular stimulus and response combination is not one that an animal is biologically prepared to associate, then the association may not occur or may occur only with great difficulty. EX: Because of biological preparedness, animals can develop conditioned taste aversions that cause them to avoid harmful foods. Animals can also be biologically prepared to fear stimuli; this fear then helps them avoid or escape stimuli such as dangerous predators. Behaviors that animals are biologically prepared to learn increase their chances of survival and reproduction.
Cognitive map
EX: An enterprising rat had knocked the cover off the maze, climbed over the maze wall and out of the maze, and scampered directly to the food box. To Tolman, such reports indicated that the rats had learned more than simply the sequence of responses required to get to the food. Tolman believed instead that the rats eventually built up, through experience, a cognitive map of the maze-- a mental representation of its layout.
Observational learning & Bobo doll experiment
Classical conditioning and operant conditioning emphasize the role of direct experiences in learning, such as directly experiencing a reinforcing or punishing stimulus following a particular behavior. But much human learning occurs indirectly, by watching what others do, then imitating it. In observational learning, learning takes place through observing the actions of others. Albert Bandura is the psychologist most strongly identified with observational learning. He believed that observational learning is the result of cognitive processes that are "actively judgmental and constructive" not merely "mechanical copying" To illustrate his theory, let's consider his famous experiment involving the imitation of aggressive behaviors. In the experiment, 4-year old children separately watched a short film showing an adult playing aggressively with a Bobo doll- a large, inflated balloon doll that stands upright because the bottom is weighted with sand. All the children saw the adult hit, kick, and punch the Bobo doll in the film. However, there were three different versions of the film, each with a different ending. Some children saw the adult reinforced with soft drinks, candy, and snacks after performing the aggressive actions. other children saw a version in which the aggressive adult was punished for the actions with a scolding and a spanking by another adult. Finally, some children watched a version of the film in which the aggressive adult experienced no consequences. After seeing the film, each child was allowed to play alone in a room with several toys, including a Bobo doll. The playroom was equipped with a one-way window so that the child's behavior could be observed. Bandura found that the consequences the children observed in the film made a difference. Children who watched the film in which the adult was punished were much less likely to imitate the aggressive behaviors than were children who watched either or the other two film endings. Bandura explained the results much as Tolman explained latent learning. Reinforcement is not essential for learning to occur. Rather, the expectation of reinforcement affects the performance of what has been learned. Bandura suggests that four cognitive processes interact to determine whether imitation will occur. First, you must pay attention to the other person's behavior. Second, you must remember the other person's behavior so that you can perform it at a later time. That is, you must form and store a mental representation of the behavior to be imitated. Third, you must be able to transform this mental representation into actions that you are capable of reproducing. Thee three factors-- attention, memory, and motor skills- are necessary for learning to take place through observation.
Learned helplessness
Cognitive factors, particularly the role of expectation, are involved in another learning phenomenon, called learned helplessness. Learned helplessness was discovered by accident. Psychologists were trying to find out if classically conditioned responses would affect the process of operant conditioning in dogs. The dogs were strapped into harnesses and then exposed to a tone paired with an unpleasant but harmless electric shock, which elicited fear. After conditioning, the tone alone elicited the conditioned response of fear. When the classically conditioned dogs were placed in the shuttle box and one side became electrified, the dogs did not try to jump over the barrier. Rather than perform the operant to escape the shock, they just lay down and whined. Why? To Steven F. Maier and Martin Seligman, two young psychology graduate students at the time, the explanation of the dogs' passive behavior seemed obvious. During the tone-shock pairings in the classical conditioning setup, the dogs had learned that shocks were inescapable. No active behavior that they engaged in would allow them to avoid or escape the shock. In other words, the dogs had "learned" to be helpless: They had developed the cognitive expectation that their behavior would have no effect on the environment. To test this idea, Seligman and Maier designed a simple experiment. Dogs were arranged in groups of there. The first dog received shocks that it could escape by pushing a panel with its nose. The second dog was yoked to the first and received the same number of shocks. However, nothing the second dog did could stop the shock- they stopped only if the first dog pushed the panel. The third dog was the control and got no shocks at all. After the initial training, the dogs were transferred to the shuttle box. As Seligman and Maier had predicted, the first and third dogs quickly learned to jump over the barrier when the floor became electrified. But the second dog, the one that had learned that nothing it did would stop the shock, made no effort to jump over the barrier. Because the dog had developed the cognitive expectation it its behavior would have no effect on the environment, it had become passive. The name of this phenomenon is learned helplessness- a phenomenon in which exposure to inescapable and uncontrollable aversive events produces passive behavior.
Conditioning
Conditioning is the process of learning associations between environmental events and behavioral responses. This description may make someone think condition has only a limited application to your life. However conditioning is reflected in most of everyday behavior skills from simply habits to emotional reactions and complex skills. EX: Conditioning has numerous practical applications in everyday training and education. Animal training routinely uses conditioning, with food treats as a form of positive reinforcement for good behavior.
Taste aversion
EX: A few years ago, Sandy made a pot of super spaghetti, with lots of mushrooms, herbs, spices, and some extra-spicy sausage. Being very fond of Sandy's spaghetti, Don ate two platefuls. Several hours later, in the middle of the night, Don came down with a nasty stomach virus. Predictably, Sandy threw up the spaghetti. As a result, Don developed a taste aversion-- he avoided eating spaghetti and felt queasy whenever he smelled spaghetti sauce.
Law of effect
Edward L. Thorndike was the first psychologist to systematically investigate animal learning and how voluntary behaviors are influenced by their consequences. At the time, Thorndike was only in his early twenties and a psychology graduate student. Thorndike's dissertation focused on the issue of whether animals, like humans, use reasoning to solve problems. In an important series of experiments, he put hungry cats in specially constructed cages that he called "puzzle boxes." A cat could escape the cage by a simply act, such as pulling a loop or pressing a lever that would unlatch the cage door. A plate of food was placed just outside the cage, where the cat could see and smell it. Thorndike found that when the cat was first put into the puzzle box, it would engage in many different, seemingly random behaviors to escape. For example, the cat would scratch at the cage door, claw at the ceiling, and try to squeeze through the wooden slats. Eventually, however, the cat would accidentally pull on the loop or step on the lever, opening the door latch and escaping the box. After several trials in the same puzzle box, a cat could get the cage door open very quickly. Thorndike concluded that the cats did not display any humanlike insight or reasoning in unlatching the puzzle box door. Instead, he explained the cats' learning as a process of trial and error. The cats gradually learned to associate certain responses with successfully escaping the box and gaining the food reward. According to Thorndike, these successful behaviors became "stamped in," so that a cat was more likely to repeat these behaviors when placed in the puzzle box again. Unsuccessful behaviors were gradually eliminated. Thorndike's observations led him to formulate the law of effect: Response followed by a "satisfying state of affairs" are "strengthened" and more likely to occur again in the same situation. Conversely, responses followed by an unpleasant or "annoying" state of affairs are "weakened" and less likely to occur again.
Fixed vs. variable-ratio schedule
Fixed Ratio Schedule: Reinforcement occurs after a fixed number of responses. A rat on a 10-to-1 ratio schedule would have to press the bar 10 times in order to receive one food pellet. Fixed-ratio schedules typically produce a high rate of responding that follows a burst-pause-burst pattern. In everyday life, the fixed-ratio schedule is reflected in any activity that requires a precise number of responses in order to obtain reinforcement. Piecework-- work for which you are paid for producing a specific number of items, such as being paid $1 for every 100 envelopes you stuff- is an example of an FR-100 schedule Variable- Ratio Schedule: Reinforcement occurs after an average number of responses, which varies from trial to trial. A rat on a variable-ratio-20 schedule might have to press the bar 25 times on the first trial before being reinforced and only 15 times on the second trial before reinforcement. Although the number of responses required on any specific trial is unpredictable, over repeated trials the ratio of responses to reinforcers works out to the predetermined average. Gambling is the classic example of a variable-ratio schedule in real life. Each spin of the roulette wheel, toss of the dice, or purchase of a lottery ticket could be the big one, and the more often you gamble, the more opportunities you have to win( and lose).
Fixed vs. variable interval- ratio schedule
Fixed-interval schedule: A reinforcer is delivered for the first response emitted after the present time interval has elapsed. A rat on a two-minute fixed interval schedule would receive no food pellets for any bar presses made during the first two minutes. But the first bar press after the two minute interval has elapsed would be reinforced. Fixed-interval schedules typically produce a scallop-shaped pattern of responding in which the number of responses tends to increase as the time for the next reinforcer draws near. For example, if your instructor gives you a test every four weeks, your studying behavior would probably follow the same scallop-shaped pattern of responding as the rat's bar-pressing behavior. Variable-interval schedule: reinforcement occurs for the first response emitted after an average amount of time has elapsed, but the interval varies from trial to trial. Hence, a rat on a VI-30 seconds schedule might be reinforced for the first bar press after only 10 seconds has elapsed on the first trial, for the first bar press after 50 seconds has elapsed n the second trial, and for the first bar press after 30 seconds have elapsed on the third trial. Generally, the unpredictable nature of variable-interval schedules trends to produce moderate but steady rates of responding, especially when the average interval is relatively short. In daily life, we experience variable-interval schedules when we have to wait for events that follow an approximate, rather than a precise, schedule. Trying to connect to the Internet via an old-fashioned dial-up modem during peak usage hours is one example. When you get a busy signal, you periodically try redialing (the operant) because you know that at some point you'll connect and be able to access the Internet (the reinforcer).
Stimulus discrimination
Just as a dog can learn to respond to similar stimuli, so it can learn the opposite- to distinguish between similar stimuli. EX: Pavlov repeatedly gave a dog some food following a high-pitched tone but did not give the dog any food following a low-pitched tone. The dog learned to distinguish between the two tones, salivating to the high-pitched tone but not to the low-pitched tone. This phenomenon, stimulus discrimination, occurs when a particular conditioned response is made to one stimulus but not to other, similar stimuli.
Negative reinforcement
Negative reinforcement involves an operant that is followed by the removal of an aversive stimulus. In negative reinforcement situations, a response is strengthened because something is being subtracted or removed. EX: You take two aspirin(the operant) to remove a headache (the aversive stimulus). Thirty minutes later, the headache is gone. if you are more likely to take aspirin to deal with bodily aches in the future, then negative reinforcement has occurred.
Extinction
Once learned, can conditioned responses be eliminated? In 1927, Pavlov found that conditioned responses could be gradually weakened. If the conditioned stimulus (a ringing bell for example) was repeatedly presented without being paired with the unconditioned stimulus (food, for example) the conditioned response seemed to gradually disappear. Pavlov called this process of decline and eventual disappearance of the conditioned response extinction.
Classical conditioning
One of the major contributors to the study of learning was a Russian physiologist who was awarded a Nobel Prize for his work on digestion. Ivan Pavlov directed several research laboratories. His involvement in psychology began as a result of an observation he made while investigating the role of saliva in digestion, using dogs as his experimental subjects. In order to get a dog to produce saliva, Pavlov (1904) put food on the dog's tongue. After he had worked with the same dog for several days in a row, Pavlov noticed something curious. The dog began salivating before Pavlov put the food on its tongue. In fact, the dog began salivating when Pavlov entered the room or even at the sound of his approaching footsteps. What he discovered is that dogs have learned to anticipate food in association with some signal. The process of conditioning that Pavlov discovered was the first to be extensively studied in psychology. Thus, it's called classical conditioning ( Hilgard & Marquis, 1940). Classical conditioning deals with behaviors that are elicited automatically by some stimulus. Elicit means "draw out" or "bring forth". That is, the stimulus doesn't produce a new behavior, but rather causes an existing behavior to occur. Classical conditioning always involves some kind of reflexive behavior. For example, in Pavlov's original studies of digestion, the dogs salivated reflexively when food was placed on their tongues. But when the dog began salivating in response to the sight of Pavlov or to the sound of his footsteps, a new learned stimulus elicited the salivary response. Thus, in classical conditioning, a new stimulus-response sequence is learned. Essentially, classical conditioning is a process of learning an association between two stimuli. Classical conditioning involves pairing a neutral stimulus with an unlearned, natural stimulus that automatically elicits a reflexive response. If the two stimuli (Pavlov+ food) are repeatedly paired, eventually the neutral stimulus (Pavlov) elicits the same basic reflexive response as the natural stimulus (food)-- even in the absence of the natural stimulus.
Stimulus generalization
Pavlov (1927) noticed that once a dog was conditioned to salivate to a particular stimulus, new stimuli that were similar to the original conditioned stimulus could also elicit the conditioned salivary response. EX: Pavlov conditioned a dog to salivate to a low-pitched tone. When he sounded a slightly higher-pitched tone, the conditioned salivary response would also be elicited. Pavlov called this phenomenon stimulus generalization. Stimulus generalization occurs when stimuli that are similar to the original conditioned stimulus also elicit the conditioned response, even though they have never been paired with the unconditioned stimulus.
Positive reinforcement
Positive reinforcement involves following an operant with the addition of a reinforcing stimulus. In positive reinforcement situations, a response is strengthened because something is added pr presented. Everyday examples of positive reinforcement in action are easy to identify. EX: Your backhand return of the tennis ball (the operant) is low and fast, and your tennis coach yells "Excellent!" (the reinforcing stimulus). In the above example, if the addition of the reinforcing stimulus has the effect of making you more likely to repeat the operant in similar situations in the future, then positive reinforcement has occurred.
Mirror neurons
Psychologists have only recently begun to understand the neural underpinnings of the human ability to imitate behavior. The first clue emerged from an accidental discovery in a lab in Palermo, Italy, in the mid-1990s. Neuroscientist Giacomo Rizzolatti and his colleagues were studying neurons in the premotor cortex of macaque monkeys. He noticed something odd. As one of the wired monkeys watched a lab assistant pick up a peanut, a neuron fired in the monkey's brain- the same neuron that fired when the monkey itself picked up a peanut. At first, the researchers thought that the monkey must be making tiny muscle movements, and that these movements were responsible for the motor neuron activity. But the monkey was sitting perfectly still. The explanation was that the team had discovered a new class of specialized neurons which they dubbed mirror neurons. Mirror neurons are neurons that fire both when an action is performed and when the action is simply perceived. In effect, these neurons imitate or "mirror" the observed action as though they observer were actually carrying out the action. Brain imaging studies have provided indirect evidence of mirror neurons in the human brain. A recent study provided the first direct evidence for the existence of mirror neurons in humans.
Punishment
Punishment is a process in which a behavior is followed by an aversive consequence that decreases the likelihood of the behavior's being repeated. Many people tend to confuse punishment and negative reinforcement, but these two processes produce entirely different effects on behavior. Negative reinforcement always increases the likelehood that an operant will be repeated in the future. Punishment always decreases the future performance of an operant. EX: An employee wears jeans to work (the operant) and is reprimanded by his supervisor for dressing inappropriately (the punishing stimulus).
Shaping
The operant conditioning procedure of selectively reinforcing successively closer approximations of a goal behavior until the goal behavior is displayed . EX: For example, a researcher can use shaping to train a rat to press a lever during an experiment (since rats are not born with the instinct to press a lever in a cage during an experiment). To start, the researcher may reward the rat when it makes any movement at all in the direction of the lever. Then, the rat has to actually take a step toward the lever to get rewarded. Then, it has to go over to the lever to get rewarded (remember, it will not receive any reward for doing the earlier behaviors now...it must make a more advanced move by going over to the lever), and so on until only pressing the lever will produce reward. The rat's behavior was "shaped" to get it to press the lever.
Latent learning
Tolman concluded that reward- or reinforcement- is not necessary for learning to take place. The rats in group 3 had learned the layout of the maze and formed a cognitive map of the maze simply by exploring it for 10 days. However, they had not been motivated to demonstrate that learning until a reward was introduced. Rewards, then, seem to affect the performance of what has been learned rather than learning itself. To describe learning that is not immediately demonstrated in over behavior, Tolman used the term latent learning.