Logical reasoning and rationality experiments

Chater and Oaksford Ten years of the rational analysis of cognition WCST (2)

1) goals: selecting the data that has the greatest expected informativeness about whether a rule is ture or whether the antecedent (p) and consequent (q) are independent 2) Environment: assumption properties are rare 3) Computational limitations: there is a cost of examining data, so as little as possible is examined. Costs determine the number of cards ppt turns over 4) Data: ODS accounts and other paradigms e.g. negation paradigm, therapy experiments, reduced-array selection task, work on so-called fictional outcomes 5) Iteration -tested and observed empirally

Syllogism

A type of problem used in deductive reasoning; there are two statements or premises and a conclusion that may or may not logically follow from the premises e.g. All A are B, all B are C, therefore all C are A. Various biases cause errors in syllogistic reasoning, including belief biases

Chater and Oaksford Ten years of the rational analysis of cognition Anderson

Approach 'processing problem' (Marr) rational analysis = empirical program that attempts to explain the function and purpose of cognitive process Anderson's methodology for deriving ratinal analyses of cognitive processes involve 6 steps 1 specify goals 2 environment - to which organism adapted 3 computational limitations - minimal assumpxs 4. optimization - obtain optimal behaviour function 5. data 6. iteration

Are humans rational? Oaksford and Chater...

As described above, logic deals with certainty. Probability theory, on the other hand, deals with uncertainty—it provides a set of rules for judging likelihood (that a certain event will happen, that a particular statement is true, etc.). Chater & Oaksford (1990) argue that many instances of apparently logical reasoning are actually instances of probabilistic reasoning. For example, if I see Fred walking past at 9 a.m., and know that when he walks by at this time he's on his way to buy a newspaper, I may conclude that he's going to buy a paper, but this conclusion is probabilistic in nature ("he's probably on his way to buy a paper") rather than a logical certainty. They point out that the conclusion is defeasible—i.e., I won't always draw this conclusion (e.g., if it's Christmas Day, he won't be able to buy a paper because the shops will be closed). Logic does not allow this flexibility.

Manktelow and Over

But cheat detection cannot explain all examples of good reasoning in card selection tasks. Manktelow & Over (1991) found that subjects reason well with rules like "if you clear up spilt blood, then you must wear rubber gloves", but it is implausible that people treat clearing up spilt blood as a benefit for which you must pay the associated cost of wearing rubber gloves.

Cheng and Holyoak 92

Cheng & Holyoak (1985): 92% of subjects chose the p and not-q cards when the rule was "if the form says entering, then the other side must say cholera" and they were given the rationale of imagining they were border guards checking whether people entering the country had required immunizations.

Chater and Oaksford Ten years of the rational analysis of cognition Cheng PC theory

Cheng Power PC theory in this model, the main determinant of the causal influence of a factor on some outcome is the difference in probability of the outcome when the causal factor is, or is not, present if all other factors are held constant. this model can be viewed as a rational analysis for the Rescorla-Wagner model of classical conditioning moreover it reveals that apparent biases in causal inference are predicted by a rational model an hence are not really biases at all

Cheng

Cheng et al. (1986) found that 40 hours of formal logic teaching did little to change this pattern of choices.

Cosmides and Tooby's interpretation

Cosmides & Tooby (1994) have suggested that people succeed in reasoning puzzles when they engage evolved reasoning systems that are specialized for particular social dilemmas such as cheat detection—we have evolved mechanisms, says Cosmides, for identifying cheats who take benefits without paying associated costs.

Cosmides

Cosmides (1989) found a dramatic increase in p and not-q choices when card selection tasks were framed in cheat detection terms—e.g., in terms of catching unmarried men who are eating aphrodisiac foods they are not supposed to.

Ermer & Kiehl

Ermer & Kiehl (2010) found that psychopaths are insensitive to cheat detection scenarios, not showing the usual benefit in these versions of card selection tasks.

formal logic as a normative theory

Formal logic is an attractive normative theory. It sets out rules by which conclusion(s) can be validly inferred from a starting set of premises, where 'valid' means that if the premises are true then the conclusion can be drawn with certainty—the rules are truth preserving. One example comes from conditional "if p, then q" rules (where p and q are statements— e.g., if it is a dog, then it is smelly). The valid modus ponens rule specifies that "if p, then q" and "p is true" (it is a dog), then we can conclude with certainty that "q is true" (it is smelly). The valid modus tollens inference specifies that "if p, then q" and "q is false" (it is not smelly), then we can conclude with certainty that "p is false" (it is not a dog). Affirmation of the consequent and denial of the antecedent are logical fallacies—not because the conclusions they lead to are always false, but because their conclusions cannot be guaranteed to be true. Do people reason according to the rules of logic—that is, is logic a good descriptive theory of human reasoning? The answer seems to be a clear "no"

Griggs and Cox 73

Griggs & Cox (1982): 73% of subjects chose the p and not-q cards when the rule to be tested was "if a person is drinking beer, they must be over 18". •

Guth 1.2 dm Hoffman 40$ Cameron 100pk (3m disposable income)

Güth et al. (1982): most people make fair offers (the modal offer is an even split) and responders usually reject offers with very uneven splits. Cameron (1999) and Hoffman et al. (1996) showed that this happens even when relatively large sums of money are at stake (e.g., 3 months disposable income). modal offer 50% mean accepted 30-40%

WCST

Wason's (1966) card selection task gave subjects a conditional rule (if p, then q) and cards corresponding to "p", "not-p" (p is false), "q" and "not-q" to test the rule. Subjects were asked to turn over the minimum number of cards to verify if the rule is true. According to logic, only the p and not-q cards are informative because only for these cards does the rule specify what should be on the other side. Few people choose these two cards (<10%).

Mental models

Johnson Laird argues reasoning involves constructing a mental model - each mental model represents a possibility, capturing what is common to the different ways in which possibility could occur. principle of truth - the notion that we represent assertions by forming mental models concerning what is true but ignoring what is false Legrenzi - lured ppts into error (illusory inferences) by their adherence to the principle of truth - interpreted as consistent (valid) though inconsistent (invalid)

Marcus and Rips

Marcus & Rips (1979): people readily endorse modus ponens inferences, but only about half of the time endorse (valid) modus tollens inferences, and often (~25% of the time) will endorse invalid inferences.

Klauer

Klauer et al. (2000): in a syllogistic reasoning task, subjects often endorse invalid conclusions if they are believable and reject valid conclusions that are inconsistent with their prior knowledge.

Probability theorem and Bayes

Oaksford and Chater (1994) argue that subjects' choices in Wason's card selection task are rational when viewed probabilistically, in that they lead to the highest information gain. They use Bayes theorem to quantify this theory of optimal data selection, but their reasoning can be understood intuitively in their example (not an empirical study, but a thought experiment) about what it is informative to do if you want to test the rule "If a pot falls in the kitchen, then you will hear a clang". According to logic, you should run into the kitchen every single time there is no clang (not-q) to see if by chance a pot has fallen (p), in which case you know the rule is certainly false (but only after an awful lot of running). According to probability theory, in contrast, expected information gain is higher if you run into the kitchen when you do hear a clang (q), because if a pot has fallen (p) you can infer that p may cause q from this coincidence of otherwise rare events.

Overview 1

Overview. This lecture reviews experimental studies of people's logical reasoning ability. Formal logic has been proposed as both a normative and a descriptive theory of human reasoning. However, experimental results suggest that people are rather poor at logical reasoning: For example, they make characteristic mistakes in reasoning about conditional statements, syllogisms and, most famously, in Wason's card selection task. People seem to perform much better with thematic version of this task, a finding that has been explained in terms of the idea that we reason using an evolved cheater detection module.

Ultimatum game behaviour seems irrational

People do not do so in the Ultimatum Game, a simple scenario involving two players, a proposer and a responder. The proposer's role is to offer a certain split of a sum of money (e.g. £10). The responder can either accept the offer, or reject it (in which case both players get nothing). If we suppose that any money is better than no money, the responder should accept any offer (e.g., £1 out of the £10) and, knowing this, the proposer should make the lowest possible offer. However, people deviate from these 'rational' standards:

An interpretation of the results

People seem to produce more 'correct' (according to logic) answers in versions of the card selection task with thematic content, rather than abstract versions of the rule.

Wallace evolutionary basis

Perhaps consistent with this evolutionary interpretation, Wallace et al. (2007) found that monozygotic twins adopted more similar strategies in the game than did dizygotic twins, evidence commonly interpreted to indicate a genetic basis for a trait.

Sanfrey

Sanfey et al. (2003) used fMRI to show increased activity in anterior cingulate cortex and insula when subjects are made unfair offers in the Ultimatum Game—areas associated with negative emotional responses, e.g., to pain, negative feedback, errors.

economic decision making

The final part of the lecture switches focus from logic to economic decision making, again to make the point that apparently irrational behaviour (from one normative perspective) may actually be rational when viewed from a different normative perspective. One normative theory is that we should maximise expected value in making economic decisions.

Oaksford and Chater

The q card is the most popular selection after the p card, as shown for example by Oaksford and Chater's (1994) meta-analysis of 34 different card selection experiments.

Nowak 2000 Fairness versus reason in the Ultimatum Game

The rational solution suggested by game theory is for the proposer ton offer the smallest possible share and for the responder to accept it. However if humans play the game, the most frequent approach to the UG is a fair share. Show that fairness will evolve if the proposer can obtain some information on what deals the responder has accepted in the past. S= expected payoff for a player using strategy: S(p1,q1) strategies given by two parameters p and q p= amount offered when acting as a proposer (as proportion of total amount available to split) q= minimum amount that accept - when acting responder, reject any offer smaller than q q<p r= number of rounds, let's say 50 in computer sim. w= history of previous deals assume amount of money accumulated over life time proportional to offspring and parents' strategy passed down to offspring. Thus evolutionary pressure towards most successful strategy w=0 (no information about previous interactions) then p and q values converge close to rational solution S(0,0) for increasing values of w there is a convergence close to the fair solution S(0.5,0.5) with q being slightly smaller than p

Non-monotonic reasoning update

The same authors (Oaksford & Chater, 1991) go further to argue that logic is in fact a bad normative theory to apply to human reasoning. They argue that we live in an uncertain world, and therefore reasoning needs to be non-monotonic—new information should be able to change old conclusions. This is possible when reasoning probabilistically (Bayes theorem, in particular, tells us how to update likelihood estimates based on new information) but is not possible when reasoning logically.

Implications of cheat detection theory

We might also worry about the implication of this theory that people are fundamentally irrational and incapable of drawing inferences about simple (and ubiquitous) if-then causal relationships.

Overview pt 2.

There is debate about the correct normative standard against which to judge human reasoning. As discussed in Lecture 5, many researchers assume that reasoning should follow the rules of deductive logic. However, others argue that logic is not useful in everyday reasoning, and that probability theory is more applicable. The optimal data selection theory provides a probability-based explanation of results from the card selection task, and suggests that people's choices in this task are actually rational. The Ultimatum Game provides another example of a situation in which apparently irrational behaviour— turning down free money—may actually have a rational basis (in terms of evolutionary pressure towards fairness).

normative and descriptive theory

To evaluate whether people are rational in their thinking, we need to specify a normative theory that defines how people ought to reason. We can then see whether people in practice meet this standard—is there a match between the normative standard and the descriptive theory of how people actually reason?

Deontic rules

WCST involves an indicative rule (if p then there is a q) However also possible to use a deontic rule (if there is a p then you must fo q) deontic rules are concerned with detection of rule violation. They are typically easier for people to understand because the underlying structure of the problem is more explicit - emphasis on disproving rule. Sperber and Girotto much higher rate of correct cards chosen 'Paulo buys things through the interent but is concerned he will be cheated. For each order he fills out a card. On one side, indicates whether he has received the item ordered, and on the other side, he indicates whether he has paid for the items ordered. He places 4 orders and what is visible on the four cards is as follows 'item paid for', 'item not paid for', 'item received', 'item not recieved'.'

Is behaviour on the Ultimatum game irrational? Nowak crucial consideration of history evolutionary game theory

We might conclude that people have an emotional reaction to unfair offers that makes them irrationally reject money. But Nowak et al. (2000) suggest that people's responses may be more rational than this. They use a computer simulation of how strategy might evolve in a population of individuals playing the Ultimatum Game—an evolutionary game theory analysis. The critical variable concerns whether individuals in the population know how other individuals played the game in the past (i.e., whether individuals have 'reputations'). It turns out that if this information is available, the most effective strategy is to reject low offers. Doing so may cost an individual in the short-term, but it is beneficial in the long term because in later interactions others will make higher offers (knowing that the individual will reject low ones).

Deductive reasoning

allows us t draw conclusion that are definitey or certainly valid provided other statements are assumed to be true related to problem solving

Popper

argued there is an important distinction between confirmation and falsification confirmation involves the attempt to obtain evidence confirming the correctness of one's hypothesis in contrast, falsification involves the attempt to falsify hypotheses by experimental tests According to Popper we cannot achieve confirmation via hypothesis testing because future evidence may disprove it. According to Popper falsifiability separates scientific and non-scientific activities Kulkarni and Simon 1988 scients make extensive use of unusualness heuristic, Klahr and Simon scientists usually use 'weak methods' e.g. trial and error, means-ends analysis -also found in everyday problem solving

Chater and Oaksford Ten years of the rational analysis of cognition WCST (1)

as few as 4% choose p, not-q p and q 46% p only 33% the selection task involves optimal data selection there is a hypothesis (the rule) and the problem is to decide which experiments should be conducted (cards should be turned) Popper's falsficationist view of science ' it is never possible to confirm a rule, only to disconfirm it. Because people do not follow this pattern has been taken as casting doubt on human rationality. But saucepan example - environment where most properties and events are rare, looking for apparently confirmatory evidence will be more informative Oaksford and Chater's rational analysis draws on the theory of ODS from Bayesion statistics rather than on Popper's falsificationism

Chater and Oaksford Ten years of the rational analysis of cognition potted comparison of expected information gain with the frequency of card selections in the standard abstract selection task

matched p>q>not-q>not-p

Sanfey The Neural Basis of Economic Decision-Making in the Ultimatum Game

fMRI of UG players to investigate the neural substrates of cognitive and emotional processes involved in economic decision making. Unfair offers elicited activity in brain areas related to both emotion (anterior insula) and cognition (dorsolateral prefrontal cortex) significantly heightened activity in anterior insula for rejected unfair offers suggests an important role for emotions in decision making players partnered and rounds presented randomly, all involving splitting $10 adhered to by a predetermined algorithm that ensured all ppts saw the same set and full range of offers unfair vs. fair, greater activation in bilateral anterior insula, dPFC and ACC magnitude of activation greater for unfair human compared to unfair computer offer and unfair human rejected at significantly higher rate, suggesting stronger emotional reaction to unfair offers from humans activations not solely a function of amount of money - also sensitive to context Further, the more unfair the offer, the greater the activation Anterior insula consistently seen in neuroimaging studies of pain, distress, hunger, thirst, autonomic arousal, anger, disgust - studies mainly focused on physical sensations of taste and odor but suggests that emotion based disgust may be conceptually similar correlation between neural activity and likelihood of rejection Ppts with stronger anterior insula activn rejected a higher proportion of these offers on average Looking across participants, an examination of individual trials also revealed a relation between R anterior insular activity and the decision o accept of reject in contrast dLPFC usually been linked to cognitive processes such as goal maintenance and executive control (Miller review 2001). dLPFC activation may relate to representation and active maintenance of cognitive demands of task - accumulating as much money as possible. An unfair offer is more difficult to accept, as indicated by higher rejection rates, and hence cognitive demands may be placed on the ppt in order to overcome the strong emotional tendency to reject the offer Although dLPFC activated to unfair offers, activation alone did not correlate with acceptance rates - alone, not sufficient to predict competing emotional areas influencing the decision However, balance between activation between anterior insula and dLPFC revealing Unfair offers that are subsequently rejected have greater anterior insula than DLPFC activation, whereas accepted offers exhibit greater DLPFC than anterior insula - contrast significant Consisitent with hypothesis that competition between two areas influences behaviour and outcome may reflect relative engagement of these regions Unfair offers also associated with increased activity in ACC, which has been implicated in the detection of cognitive conflict (Botvinick 1999) implications: human decision making: ethics, social policy, legal practice

Belief bias Klauer Stupple Schmidt and Thompson

in syllogistic reasoning the tendency to accept invalid but believable conclusions and reject valid but unbelievable ones see Klauer 2000 performance on syllogisms with valid conclusions was better when the conclusions were believable, whereas performance on syllogisms with invalid conclusions was worse when the syllogisms were believable. Stupple 2008, 2013 - replicated and found took longer to process too Some issues - difference in language of formal logic and everyday e.g. 'some' means not all in every day life but in logic 'at least one and possible all' Schmidt and Thompson 2008 improved performance on syllogistic reasoning ask when the meaning of the word 'some' in formal logic was made explicit.

Conclusion

not logical but rational optimal data selection - informative fair

WCST revisited

p and not p cards most informative in real word events are rare

Chater and Oaksford Ten years of the rational analysis of cognition Memory degradation

rate of forgetting optimally adapte to the decine in 'need probability' over time decreasing power function of the time since an item was last studies but that increases as a power function of the frequency with which the item has been used. Anderson have shown that by experimentally manipulating need-probability, forgetting functions can be manipulated. Ppts memorized strings of digits and retained them for 1,2,4,8, or 16 seconds. Some trials ended with a recall test, others with no test. Need-probability was manipulated by changing the proportion of tests for each retention interval (e.g. he probability of a test could increase or decrease with length of retention interval). Forgetting curves changed t reflect these need probabilities. Confirmation of the approach which does not emerge from mechanistic accounts of memory. Further evidence from Schooler