Cognitive Psychology - Visual Imagery (ch. 10)

Stroke patients experiment

- 100 right-hemisphere stroke patients were divided into severe/mild neglect and poor/fair IQ - They were asked to draw both a Necker Cube and also a flower (with no time constraints) - Right stroke patients clearly eliminated features from the left side of space - Fair IQ with severe neglect helped to keep the number of correct vertices draw at mild neglect levels - Intelligence seems like a way to support these representations even if you have left side neglect - You somehow "recover" from this sparse information

Spatial Representation

- A representation in which different parts of an image can be described as corresponding to specific locations in space - Kosslyn interpreted the results of his research on imagery as supporting the idea that the mechanism responsible for imagery involves this - Pylyshyn disagreed, saying that just because we experience image as spatial, that doesn't mean that the underlying representation is spatial - we often aren't aware of whats going on in our mind

Food Craving

- An intense desire to eat a specific food - Goes beyond ordinary hunger because of its intensity and specificity - Recurrent cravings have been associated with problems such as overeating, sabotaging attempts at dieting, and binge eating associated with eating disorders, especially in women - It is caused by a number of factors, including nutritional deficiencies, hormonal changes, emotions and proximity to enticing foods - It is also associated with cognitive factors, including imagery - Harvey and coworkers (2005) had female subjects rate their intensity of craving on a 100-point scale and then divided them into two groups - The food imagery group were told to imagine their favourite food; the holiday imagery group were told to imagine their favourite holiday - following this, they again rated their food craving - The results indicate that the food imagery task caused a large increase in craving, but the holiday imagery task had no effect - The effect of imagery was greater in women who were dieting, although the increase in craving occurred in non-dieters as well - Also evidence that nonfood imagery can decrease cravings - visual imagery caused a greater decrease in food craving - The phonological loop, which is responsible for processing visual and auditory information, is involved in creating the auditory images, and the visuospatial sketch pad, which is responsible for visual and spatial information, is involved in creating the visual images

Interaction between Image and Perception

- Another way to demonstrate connections between imagery and perception is to show that they interact with one another - If imagery affects perception, or perception affects imagery, this means that they both have access to the same mechanisms - Cheves Perky (1910) did an experiment. She asked subjects to "project" visual images of common objects onto a screen, and then to describe them - She was back-projecting a very dim image of this object onto the screen (subjects unaware) - So, when subjects were asked to create an image of a banana, a dim banana image was projected - The subjects' descriptions of their images matched the images that were projected - None one of the subjects noticed the dim image on the screen

Pegword Technique

- Associating images with words - Involves imagery, as in the method of loci, but instead of visualizing items in different locations, you associate them with concrete words - Creating list, rhyming - Pair each of the things to be remembered with a pegword by creating a vivid image of your item-to-be-remembered together with the object represented by the word

Watson

- Behaviourism - Behaviourists branded the study of imagery as unproductive because visual images are invisible to everyone except the person experiencing them - He described images as "unproven" and "mythological" and therefore not worthy of study

Propositional Representations

- Can also be used to code spatial relationships without being analogical - Symbolic relationships between words suffice

Patient R.M.

- Case of normal perception but impaired imagery - Suffered damage to his occipital and parietal lobes - Was able to recognize objects and to draw accurate pictures of objects that were placed before him - However, he was unable to draw objects from memory, a task that requires imagery, and also had trouble answering questions that depend on imagery, like answering the question "is a grapefruit larger than an orange"

Modest Problems

- Considering the problem of extra-terrestrial communication highlights the symbolic nature of visual imagery and communication - This is a copy of the Arecibo Broadcast - the most powerful radio signal ever to be intentionally sent out into space - But what to say and how to say it? - Similarly, this is the plaque that was attached to the side of Pioneer 10, in case it happened to be found by some space creatures - "Do you think that they will think his arm is permanently attached in this position... in our country, goodbye just looks like hello..." (Anderson, 1984)

Size

- Despite the alternative hypotheses available for the preservation of space in mental representation, other research has focused on _______ - The idea here is that because of relative sizes (and relative sizes of mental representation) the details of a rabbit are harder to verify when the image is small (versus elephant) than when the image is large (versus fly)

Finke and Pinker

- Did experiment to counter the tacit knowledge explanation of Kosslyn's mental scanning results - Briefly presented a 4 dot display, and then after a 2 second delay, presented an arrow - The subjects task was to indicate whether the arrow was pointing to any of the dots they had just seen - Although the subjects were not told to use imagery or to scan outward from the arrow, they took longer to respond for greater distances - They argue that because their subjects wouldn't have had time to memorize the distances between the arrow and the dot before making their judgment, it is unlikely that they used tacit knowledge about how long it should take to get from one point to another

Spatial Information

- Different ways to represent things - Can be a map in your head, can be by preserving in a non-spatial way, etc.

Brain Imaging

- Early study was carried out by Samuel Le Bihan and coworkers - Demonstrated that both perception and imagery activate the visual cortex - Activity in the striate cortex increased both when a person observed presentations of actual visual stimuli ("perception") and when the person was imagining the stimulus ("imagery") - In another study, asking subjects to think about questions that involved imagery like "is the green of the trees darker than the green of the grass?" generated a greater response in the visual cortex than asking non-imaginary questions, such as "is the intensity of electrical current measured in amperes?" - Kosslyn made us of the way the visual cortex is organized as a topographic map (the fact that specific locations on a visual stimulus cause activity at specific locations in the visual cortex and that points next to each other on the stimulus cause activity at locations next to each other on the cortex) - Looking at a small object causes activity in the back of the visual cortex, and looking at a larger object causes activity to spread towards the front of the visual cortex - Subjects were instructed to create small, medium and large visual images while they were in the brain scanner - The result is that when subjects created small visual images, activity was centred near the back of the brain, but as the size of the mental image increased, activation moved towards the front of the visual cortex, like it does for perception - So, both imagery and perception result in topographically organized brain activities - But there is also some overlap/differences - There are not surprising, since seeing an object is different from imagining it

Unilateral Spatial Neglect

- Focus on one side of space and objects (typically left side neglect) - May be caused by damage to the parietal lobe - The patient ignores objects in one half of the visual field, even to the extent of shaving just one side of his face or eating only the food on one side of her plate - Ex) marking a line slightly more towards the right of a page - In standard line-bisection tasks, Fellini (following right hemisphere stroke) would select a section to the right of middle, indicating that h was neglecting the left side of space - Fellini was also aware of his neglect - In other more complex drawings, evidence of this unilateral spatial neglect was shown either implicitly or explicitly - Bisiach and Luzzatti asked a patient to describe things he saw when imagining himself standing at one end of the Piazza del Duomo. The patient's responses showed that he neglected the left side of his mental image, just as he neglected the left side of his perceptions

Ganis Experiment

- Him and coworkers used fMRI to measure activation under two conditions, perception and imagery - Perception: subjects observed a drawing of an object, such as a tree - Imagery: subjects were told to imagine a picture that they had studied before, when they heard a tone - For both, they were asked "is the object wider than it is tall?" - Results show activation at three different locations in the brain - Perception and imagery both activate the same areas in the frontal lobe; the same result further back in the brain - Activation in the visual cortex, in the occipital lobe at the back of the brain, indicates that perception activates much more of this area of the brain than does imagery - There is almost complete overlap of the activation caused by perception and imagery in the front of the brain, but some difference near the back of the brain

Amedi

- His imaging experiment showed that when subjects were using visual imagery, some areas associated with non-visual stimuli, such as hearing and touch, were deactivated - The reason for this might be that visual images are more fragile than real perception and this deactivation helps quiet down irrelevant activity that might interfere with the mental image

Epiphenomenon

- How Pylyshyn describes the spatial experience of mental images - It is something that accompanies the real mechanism but is not actually part of the mechanism - Ex) Lights flashing as a mainframe computer carries out its calculations. The lights may indicate that something is going on inside the computer, but they don't necessarily tell us what is actually happening - Mental images are similar to this - they indicate that something is happening in the mind, but don't tell us how it is happening

Modest Solutions

- Kekule stumbled across the structure of benzene via a dream in which he visualized a snake swallowing its own tail - This led him to propose that the carbon atoms in benzene were arranged in the form of a ring - Curious visual imagery also led Einstein to develop his theory of relativity - Specifically, he imagined himself travelling besides a beam of light and the consequences this would lead to (including faster grocery shopping)

Transcranial Magnetic Stimulation (TMS)

- Kosslyn and coworkers (1999) - Presented TMS to the visual cortex while subjects were carrying out either a perception task or an imagery task - Perception: subjects briefly viewed a display and were asked to make a judgment about the stripes in two of the quadrants - Imagery: same task, but instead of actually looking at the stripes while answering the question, the subjects closed their eyes and based their judgment on their mental image of the display - Subjects reaction times to make the judgement were measured, both when TMS was being applied to the visual area of the brain and also during a control condition when the stimulation was directed to another part of the brain - Results indicated that stimulation caused subjects to respond more slowly, and that this slowing effect occurred both for perception and for imagery - Concluded that the brain activation that occurs in response to imagery is not an epiphenomenon and that brain activity in the visual cortex plays a casual role in both perception and imagery

Size in the Visual Field

- Kosslyn wondered whether the relationship between viewing distance and the ability to perceive details also occurs for mental images - He asked subjects to imagine two animals, such as an elephant and a rabbit, next to each other and to imagine that they were standing close enough to the larger animal that it filled most of their visual field - He asked questions like "does the rabbit have whiskers?" and asked his subjects to find the part of the animal in their mental image and to answer as quickly as possible - When he repeated this but told subjects to imagine a rabbit and a fly next to each other, the rabbit was much larger - The results show that subjects answered questions about the rabbit more rapidly when it filled more of the visual field - He also asked them to do a *mental walk test*, in which they were to imagine that they were walking toward their mental image of an animal - Their task was to estimate how far away they were from the animal when they began to experience "overflow": image filled the visual field or edges become fuzzy - The result was that subjects had to move closer for small animals than for larger animals, just as they would do if they were walking towards real animals - This provides further evidence for the idea that images are spatial, just like perception

Imagery Neurons

- Kreiman and coworkers - Studied patients who had electrodes implanted in various areas in their medial temporal lobe, which includes the hippocampus and the amygdala - Wanted to determine the source of severe epileptic seizures that could not be controlled by medication - Found that some neurons responded to some objects but not to others - These neurons fired in the same way when the person closed his or her eyes and imagined a baseball (good firing) or a face (no firing) - These are called ___________ ____________ - Demonstrates a possible physiological mechanism for imagery and because these neurons respond in the same way to perceiving an object and to imagining it, thereby supporting the idea of a close relationship between perception and imagery

Galton

- Made the obversation that people who had great difficulty forming visual images were still quite capable of thinking - Supported the idea that imagery was NOT required for thinking

Propositional Networks

- Mimic spatial relationships between portions of an image again without necessarily being analogical - The RT data are essentially the same

Imagery and Perception Debate - conclusion

- Most psychologists have concluded that imagery and perception are closely related and share some (but not all) mechanisms (Pylyshyn does not agree with this at all) - The idea of shared mechanisms follows from all of the parallels and interactions between perception and imagery - The idea that not all mechanisms are shared follows from some of the fMRI results, some of the neuropsychological results, and also from differences between the experience of imagery and perception - Perception is stable - it continues as long as you are observing a stimulus - but imagery is fragile - it can vanish without continued effort - It is harder to manipulate mental images than images that are created perceptually - Subjects who were holding a mental image of a figure were unable to flip from one perception to another (Chalmers and Reisberg) - There is also some evidence to suggest that it is harder to rotate ambiguous mental images in order to create an alternate perception than it is to rotate actual images present in the scene (Mast and Kosslyn) - Imagery and perception have many features in common, but there are also differences between them

Patient M.G.S

- Occipital lobe removed as treatment for severe case of epilepsy - Before the operation, when doing mental walk task, she felt she was about 15 feet from an imaginary horse before its image overflowed - After surgery, the distance increased to 35 feet - This occurred because removing part of the visual cortex reduced the size of her field of view, so the horse filled up the field when she was farther away - Supports the idea that the visual cortex is important for imagery

Brain crossing

- Our brain is crossed in a peculiar way - Information to the right of fixation will be processed by the left side of both retina, and then, information from both eyes will merge in the left hemisphere

Paired-Associate Learning

- Paivio (1963) worked on memory and showed that it is easier to remember concrete nouns, like truck or tree, that can be imaged, than it is to remember abstract nouns, like truth or justice, that are difficult to image - To explain this result, Paivio proposed the *conceptual peg hypothesis*: concrete nouns create images that other words can "hang onto". Ex) If presenting the pair boat-hat creates an image of a boat, then presenting the word boat later will bring back the boat image, which provides a number of places on which subjects can place the hat in their mind - Shepard and Meltzer (1971) inferred cognitive processes by using *mental chronometry*: determining the amount of time needed to carry out various cognitive tasks - They looked at reaction times for same/different 3D abstract shapes - The task was to indicate, as rapidly as possible, whether the two pictures were of the same object or of different objects - When the shapes were the same, reaction time was longer the greater the rotational distance, almost like participants were mentally turning one shape to match the other - Experiment was one of the first to apply quantitative methods to the study of imagery and to suggest that imagery and perception may share the same mechanisms

Double Dissociation between Imagery and Perception

- Patients have been found who collectively show a double dissociation between imagery and perception performance after brain damage - The presence of a double dissociation is usually interpreted to mean that the two functions are served by different mechanisms - However, this conclusion contradicts the other evidence we have presented that shows that imagery and perception share mechanisms - One way to explain this is that the mechanisms overlap only partially, with the mechanism for perception being located at both lower and higher visual centres and the mechanism for imagery being located mainly in higher visual centres - So, visual perception necessarily involves bottom-up processing, which starts when light enters the eye and an image is focused on the retina, then continues as signals are sent along the visual pathways to the visual cortex and then to higher visual centres - Imagery originates as a top-down process, in higher brain areas that are responsible for memory

Wundt

- Proposed that images were one of the three basic elements of consciousness, along with sensations and feelings - Because images accompany thought, studying images was a way of studying thinking - Imageless thought debate: the idea of a link between imagery and thinking -> Aristotle: "Thought is impossible without an image"

Propositional Representation

- Pylyshyn - Proposed that the mechanism underlying imagery is _______ - Representation in which relationships can be represented by abstract symbols, such as an equation, or a statement such as "the cat is under the table" - In addition to suggesting that Kosslyn's results can be explained in terms of propositional representations, Pylyshyn also suggested that one reason that scanning time increases as the distance between two points on an image increases is that subjects are responding to Kosslyn's tasks based on what they know about what usually happens when they are looking at a real scene - "When asked to imagine something, people ask themselves what it would look like to see it, and they then simulate as many aspects of this staged event as they can" - People know that in the real world it takes longer to travel longer distances, so, Pylyshyn suggests that they simulate this event in Kosslyn's experiment - This is called the *tacit knowledge explanation*, because it states that subjects unconsciously use knowledge about the world in making their judgments - Pylyshyn's criticisms stimulated a large number of experiments that have taught us a great deal about the nature of visual imagery

Depictive Representations

- Representations that are like realistic pictures of an object, so that parts of the representation correspond to parts of the object - Ex) a spatial representation involving a spatial layout showing a cat and a table that could be represented in a picture

Visual Imagery

- Seeing in the absence of a visual stimulus - Occurs when a person sees in his or her mind something that isn't physically present - Provides a way of thinking that adds another dimension to the verbal techniques usually associated with thinking - Important example) Friedrich Kekule - he said that the structure of benzene came to him in a dream in which he saw a writhing chain that formed a circle that resembled a snake, with its head swallowing its tail. This gave him the insight that the carbon atoms that make up the benzene molecule are arranged in a ring

Tacit Knowledge

- Should also not be overlooked - We mimic internally the processes that we know happen externally when we really do look at something

Mental Scanning Experiment

- Stephen Kosslyn - Asked subjects to memorize a picture of an object, such as a boat, then create an image of that object in their mind and to focus on one part of the boat, such as the anchor - Then asked to look for another part of the boat, such as the motor, and to press the "true" button when they found this part of the "false" button when they couldn't find it - He reasoned that if imagery, like, perception, is spatial, then it should take longer for subjects to find parts that are located farther from the initial image of the object - and this is indeed what happened - Glen Lea proposed that as subjects scanned, they may have encountered other interesting parts, such as the cabin, and this distraction may have increased their reaction time - To address this, Kossylyn did another scanning experiment, this time asking subjects to scan between two places on a map - Told to imagine an island that contained 7 different locations - By having subjects scan between every possible pair of locations, he determined the relationship between reaction time and distance - Just as in the boat experiment, it took longer to scan between great distances on the image, a result that supports the idea that visual imagery is spatial in nature - Zenon Pylyshyn proposed another explanation, which started what has been called the *imagery debate*: a debate about whether imagery is based on spatial mechanisms, such as those involved in perception, or on mechanisms related to language, called propositional mechanisms

Perception vs. Imagery

- Subjects were asked to image certain letters - They had to decide the two periods and say whether the letter showed up in the period 1 or period 2 - If you had imagined the same letter that faintly appears on the screen, you're more likely to record the perception even though its weak and identical in both cases - The imagination of the letter helps correct perception - Probably has to happen via visual extend - Imagining the image helps you to see the image - The lack of distinction between visual perception and visual imagery is supported by the examination of single neurons in medial temporal lobe - Firing rates are roughly equivalent regardless of whether you see Agent Cooper or just imagine him. The same neuron is not interested in cherry pie (real or imagined) - In the media temporal lobe, have neural ensembles activated somewhat irrespectively as to if you're actually perceiving the object or imagining it - Doesn't care/matter if object is in environment or being imagined

Patient C.K.

- Suffered visual agnosia, the inability to recognize objects - They could recognize parts of objects but couldn't integrate them into a meaningful whole - However, he was able to draw objects from memory, a task that depends on imagery - When shown his own drawings after some time, he was unable to identify the objects we had drawn

Mental Imagery

- The ability to recreate the sensory world in the absence of physical stimuli - Also occurs in senses other than vision - People have the ability to imagine tastes, smells, and tactile experiences

Imagery and Perception

- The idea that they may share the same mechanisms is based on the observation that although mental images differ from perception in that they are not as vivid or as long lasting, imagery shares many properties with perception - Mental and perceptual images both involve spatial representation of the stimulus - the spatial experience for both imagery and perception matches the layout of the actual stimulus - This idea is supported by a number of experiments by Stephen Kosslyn involving a task called *mental scanning*, in which subjects create mental images and then scan them in their minds

Pseudoneglect

- The observation that individuals without neurological damage are also less-than-perfect in their line-bisection - Slight left-ward bias most likely in: younger + use of left hand - Slight ride-ward bias most likely in: older + use of right hand

Imagery and Memory

- The power of imagery to improve memory is tied to its ability to create organized locations at which memories for specific items can be placed - Simonides found that he could remember things by imaging a physical space, and placing his mind items to be remembered in it - he invented what is now called the *method of loci* - Method in which things to be remembered are placed at different locations in a mental image of spatial layout

Representing Numbers

- There are other things in our environment other than spatial relationships that need representing - If we represent numbers in a pure analogue fashion, then distance between numbers should be equal - In a study, participants had to give verbal responses to the target stimuli presented - Some concern was raised about whether single-digit numbers are represented differently from double-digit numbers - Responding to 8 after a 6 led to an estimate of 9 ms per digit, responding to 8 after a 10 led to an estimate of 6 ms per digit - Analogue representation is supported (although other data show saying 2 < 3 is faster than 8 < 9)

TMS

- Use TMS to knockout some of these perceptions - Knocking out the same bit of the brain during both visual and imagery performance led to a slowing of performance (Kosslyn et al., 1999) suggesting that at least some of the same brain regions are implicated in real and imagined perception - Ganis et. al (2004) did point out some important constraints here though. Comparing perceptual versus imagination conditions frontal activity is equivalent although there is more occipital activity during actual perception - Where there is a distinction between imagination and perception is in the occipital lobe (more activity if its real)

Anton Raederscheidt

- Was a German artist who suffered from a right-sided cerebral stroke - His self-portraits after this time strongly depict his spatial neglect in a number of ways - The left hand side of his face tends to be missing and much more detail is to be found on the right - Had a tendency to paint on the right side of the paper, similar to line-bisection tasks - There are also examples in which the art acts as the control stimulus, in cases where the painting (or tapestry) is revisited or completed before and after the time of the neuropsychological deficit

Types of Representation

- When individuals were asked to mentally traverse the map of the island to see whether named landmarks were there: HUT to HIDE OUT, HUT to WELL - RT increased in a linear fashion - The data from Shepard and Meltzer (1971) and Kosslyn et. al (1978) might be telling us the way we mentally perceive is like the way we visually perceive - Representation is both *spatial* and *strict analogical* - Pylyshyn (1973) wasn't very happy with this. Just because the data looks like a spatial effect doesn't mean the representation is spatial. What about the influences of *propositional representation* and *tacit knowledge*? - Visual representations are symbolic

Damage to Right Hemisphere

= Neglect of left visual field

Damage to Left Hemisphere

= Neglect of right visual field

Closure

Ex) In the clock images, the left side of the space is neglected but the whole of the clock face has been drawn

Drawing a Clock Lab

Needed: - Memory to retrieve a clock-face contains the numbers 1-12 - The clock-wise organization of these numbers on the edge of the face - The equal distribution of the numbers 1-12 in space - The assignment of hours to the small hand and minutes to the little hand - The deviation of the hour hand relative to the minutes from-the-hour - A complex interaction between motor skills, spatial planning, application of previous knowledge for an abstract representation of time

Strict Analogical

Spatial information about the outside world is represented in a spatial way which preserves the relative and absolute relationships between positions

Lenient Analogical

Spatial information about the outside world is represented in a spatial way which preserves the relative relationships between positions

Neglect in images

There is spatial (nothing on the left), object (more detail on the right), and colour-based (achromatopsia; monochrome on the left)