14, 15: DELAYED REINFORCEMENT

Other important factors that determine subjective value- Odum (2011) Money vs. Other commodity

- Area under the curve tend to be larger with higher amounts -Area under curve for money is larger than other commodities

2. Choice: Davison and Baum (2007)

- Compared choice when one alternative produced delayed reinforcers and the other produced immediate reinforcers. - Delay shifts choice - reduces ability of reinforcers to strengthen behaviour? - E.g. 8,0: 8 second delay to left key, 0 second delay to right key - Delays in general show reduced likelihood of responses

Delay, magnitude, and choice

- Delay reduces the impact of a reinforcer on choice - Sometimes this leads to poor decisions - but we can harness it to promote better ones, too. - Our tendency to choose a larger, later reinforcer over a smaller, sooner one depend on both genes and environment. - The fact that environment is a factor means we can shift choice in some situations.

Consequence- Johnson & Bickel (2002) Real and hypothetical money

- Did an experiment where participants were asked hypothetically the smallest possible amount they would receive now instead of waiting "x" delay for a larger amount. - Then participants were asked again what the smallest amount they would receive in the same situation except this time you will get paid that amount. A random amount were to be chosen after all the trials had been run. - The choice itself is real, whether you are choosing in a hypothetical or real situation, a choice is still being made (as all of the choices could be a real or a hypothetical reinforcer since a random trials amount is given). - The choice should still be exactly the same.

Dixon, Jacobs and Sanders (2006)- delayed reinforcement

- Gamblers doing a discounting task with hypothetical money in off-track betting facility, coffee shops, restaurants or a "preferred business of public location". - Compared area under curve for situations associated with risky choice (impulsive) to area associated with no risk. - For all but 3 participants, money is holding a greater extent (less impulsively) in the non-gambling setting compared to the gambling situation. - Can see some scattering which means that there are individual differences (otherwise all points would be along the line of best fit).

Delayed reinforcers are common

- Law of effect: Temporal contiguity is important - Important to understand how delay impacts the effect of the consequence: - Does delay prevent reinforcers from affecting behaviour, or does it simply reduce their effect?

Charlton and Lantino (2008) Same task, different hypothetical commodities matched for dollar value

- Same delay task as Johnson & Bickel, except was repeated with money, food, books, dvd and music. - E.g. $90 worth of music now or $100 worth of music after delay - Money, books, dvd decline at similar rate - Money holds value the most across the set of delays (less steep slope, larger area underneath slope). - Food losing its value most rapidly despite all other things being the same value. - Different commodities = different rates of discount -Individual differences were still maintained across these commodities

Johnson & Bickel (2002) Magnitude difference- delayed reinforcement

- The amount that is dealt with matters. - There is a more rapid decrease in subjective value of money if the amount is smaller. Higher amount = more resistant to devalue

Studying Self control

- Time reduces the value of the reinforcer - Mazur's delay-titration procedure ○ Peck either standard key (smaller delay, smaller reinforcer) or adjusting key (larger delay, larger reinforcer) ○ To find subjective value of delayed reinfrocer, reduce the time of the adjusting key so the choices eventually match ○ Longer adjusting delays= more self control ○ Shorter adjusting delays= more impulsive - The more rapid the decrease in value (steeper slope & more less area below the slope), the more impulsive the choice

Delayed reinforcers on decreasing undesirable options Appelhans (2018)

- Vending machines: ○ Delay on unhealthy snacks in vending machines ○ Discount to healthy snacks ○ Tax on unhealthy snacks - All interventions had an effect, but taxing had most effect

Undesirable choices

- We have a tendency to prefer smaller, more immediate rewards over larger, more delayed ones. - A behavioural definition of self-control might be choosing the larger, later reinforcer over the smaller, sooner one (opposite is impulsiveness) . - Our tendency to choose one of the other depends on two things: ○ Personality (genes, biology) Current environment

1. Acquisition: Wilkinson et al. (1992)

Acquisition - can we learn under conditions of delayed reinforcement? - Magazine trained. - FR1 schedule, but reinforcers preceded by a delay (0, 4, 8, 16 s) - Responses to the lever during the delay added another delay on top of that one. - Control group: No reinforcers - Steeper slope means more responding (faster acquisition) - Reinforcer still affecting behaviour despite delays

Green, Myerson, Lichtman, Rosen & Fry (1996) Income

Upper income individuals shows slower decrease in value of money for both amounts over time (1,000 and 10,000) compared to lower income individuals.