psych test 4.5

pointing gestures

kind of a given

summary of ch 1

• Bodily communication is incredibly significant, more significant than we had ever assumed • Bodily communication does not just reveal our emotions and how we feel about another person, it reveals our hidden thoughts. • In the past few years, new research in psychology has made significant progress in understanding what people do when they communicate to one another, and more importantly, exactly how they do it. • This new research challenges many of our long-standing beliefs on this subject. •We are also unaware of the sheer extent of our gestures, and sometimes we are even unaware of whether we are gesturing or not. • According to Katherine Nelson, the movements of the hands in everyday talk represent 'a mode of unconscious meaning unconsciously expressed'. •Gestures and speech are coordinated, and many gestures have a 'preparation phase' to allow this coordination to happen. •People who have been blind from birth still gesture even though they have never actually seen gestures themselves, and they continue to gesture even when conversing with other blind people that they know are blind.

in children's development

•: Iconic gestures develop alongside language when children learn to talk. From Goldin-Meadow (1999): "At a time in their development when children are limited in what they can say, there is another avenue of expression open to them, one that can extend the range of ideas they are able to express. In addition to speaking, the child can also gesture." •Example: A child says "chair" while pointing at a chair. Speech and gesture work together to specify the object: -The word labels and classifies, but doesn't locate the object -The gesture indicates where the object is but not what it is

iconic gestures can be segmented into parts

•A prototypical iconic gesture has 3 phases: (1)A preparation phase, during which the hand rises from its resting place and moves to the front of the body and away from the speaker, in preparation to make the gesture (2)A stroke phrase, the main part of the gesture, during which the gesture exhibits its meaning (3)The retraction phase, during which the hand moves back to its resting position.

ch 3

-"Gesture is really quite a confusing term, because when we think of gestures we often think of things like 'V' sign for 'peace'." This is an "emblem" that: •can substitute for words and has a direct verbal translation •is consciously sent and consciously received -If someone has just used an emblem and is asked to repeat it, they will be able to reproduce the gesture easily

another ex of how gestures convey info differently from speech

•A speaker comments on a spider he sees on a kitchen counter: -In words: "there's a spider running across the counter" while moving his hand, all five fingers wiggling, over the counter •In gesture: a single motion the gesture presents information about: -the spider: it has many legs, indicated by all five fingers moving -the manner of motion: running, as indicated by the wiggling fingers -the path of movement: across, as indicated by the path of the hand -the location: the counter, as indicated by the place where the gesture is produced •*from Goldin-Meadow (2003), Hearing gesture: How our hands help us think

other ex of metaphoric gestures

•A speaker gestures about a moral dilemma: He is asked to judge whether a father has a right to ask his son to give up earned money so the father can go fishing He responds in speech, "I think about opportunities like this where you have two interests that compete with one another. This is the point where people develop the skills of negotiation." At the same time he gestures: (1) Both hands are in front 6-8" apart, thumb and index finger resembling an equal sign (2) Both hands move toward each other and remain in air (3) Wrists pivot in alternating manner so one is forward while the other is back. This indicates two equal points of view negotiating with each other (from Goldin-Meadow)

how our hands can change our minds

•According to Goldin-Meadow, the gestures that we see can change our minds, and the gestures that we make can change our minds •In the TED talk, she provides evidence that children learn better when they are encouraged to gesture: •Children who are told to gesture are much more likely to learn than children who are told not to gesture

cont

•An older child typically gives same response in speech, but a different response in gesture. This child is called a MISMATCHER. Below, the child: •Says that the amount of water has changed "cause this one's (B1) lower than this one (B2)" •But in gesture she indicates the widths of the two containers with her hands. In B1 she holds two C-shaped hands around the wide diameter of short container, and in B2 she holds the left hand around narrower diameter of the tall container

beat gestures

•Beat gestures occur when the hands move along with the rhythmical pulsation of speech •They are typically made with short quick movements in the periphery of gesture space

beats

•Beats are movements that look as if they are beating out musical time •They tend to have the same form regardless of the content of the speech they are accompanying •The typical beat is the simple flick of the hand or fingers up and down, or back and forth; the movement is short and quick and the space in which the gesture is made may be the periphery of the gesture space (the lap, an armrest of the chair etc.)

beats cont

•Beats look like the most insignificant of all gestures but the simplicity of their form belies their real importance. •They accompany the most significant parts of the speech; not necessarily particular words which are important merely because of their content, but the most significant words in the discourse from the speaker's point of view. •Thus, even beats with their regular and simple form may provide a clue as to the inner workings of the mind of the speaker. •They demarcate those parts of the discourse that speakers themselves consider most significant, regardless of what anybody else might think.

beattie argument

•Beattie is arguing that language and gesture are not separate systems: "I want to suggest, following the pioneering work of...David McNeill, that gestures are closely linked to speech and 'yet present meaning in a form fundamentally different from that of speech'" •Through hand movements people "unwittingly display their inner thoughts and ways of understanding events of the world" •For researchers, gestures open up a new way to regard the relationship between thinking and speech • Gestures can be a window on the human mind and allow us to see thoughts and images that would otherwise be invisible.

again

•Beattie: When you consider all of this, it is extraordinary that people have tried to dismiss the movements of the hands and arms which people make when they speak as merely coincidental movements - virtually random flicks and twirls that are merely used for emphasis, merely used to make a point and barely worthy of serious consideration. •But these movements are not insignificant, and they are not merely poor forms of communication about emotion or interpersonal attitudes. They are closely integrated with speech and may provide a unique insight into how speakers are actually thinking.

what kind of spontaneous gestures do people usually produce?

•Because gesture studies is an evolving field, there are no conventional ways to represent gestures, as there are for spoken language (such as letters, words, etc.)

ch 5 analysis cont

•But the gestures that accompany language do not convey meaning in a linear and segmented manner; rather they convey a number of aspects of meaning at the same time •The gesture depicts the table and its size, the movement and its speed and the direction of the movement, all simultaneously, all in a single, multidimensional gesture •The important point is that iconic gestures are free to vary on dimensions of space, time, form, trajectory, and so forth, and can present meaning without undergoing segmentation or linearization •Gestures, unlike speech, convey meaning all at once. •Iconic gestures rely directly on Imagery; speech, in contrast, is mediated by the morphological and grammatical categories of language

beattie's summary of ch 5

•David McNeill argues that 'Speech and gesture often refer to the same event and are partially overlapping, but the pictures they present are different. Jointly, speech and gesture give a more complete insight'. •If we were to focus exclusively on speech, then we would have an incomplete picture of the speaker's mental representation of an event. •A prototypical iconic gesture involves three phases: preparation, 'stroke', and retraction. Some gestures have just two phases and some have just a stroke. •Gestures and speech are not separate in terms of their sequence of development in childhood or in terms of how they break down together with the brain damage that produces aphasia.

embodied cognition

•Embodied cognition is the idea that the mind and language cannot be understood by modeling only internal activity, but rather study must extend outwards to the mind's interactions with the body and environment. -From the perspective of embodied cognition, thought and language depend on action; they are sensorimotor in nature

how our hands reflect whats on our minds

•Examples from Piagetian conservation tasks: https://www.youtube.com/watch?v=gnArvcWaH6I •In a conservation of liquid task, a child is asked if the amount of water has changed when water poured from a tall, skinny glass into a short, wide one •In a conservation of number task, a child is asked if the number of items in a row has changed after the items have been spread further apart

two seperate language?

•Gesture and speech are NOT two separate languages: •The "established orthodoxy" is that humans use two quite separate languages, each with its own function: -A nonverbal channel for managing immediate social interactions, as in animals -A verbal channel for conveying information •Beattie: This is not true - both channels, speech and gesture, are used for both purposes, to negotiating interpersonal attitudes, and to convey information. •Gesture is used with language to communicate complex ideas •While spoken language uses conventional words and syntax for combining words into meaningful sentences, spontaneous gestures, transmit meaning in a more global and spontaneous fashion •These ideas challenge the established orthodoxy in psycholinguistics, and they have enormous theoretical implications for gaining insight into what people are really thinking as they talk.

ex ch 3 summary

•Gestures are a ubiquitous feature of everyday life as a natural accompaniment of speaking. •Body language popularisers who tell us that such movements are separate from language and perform essentially social functions are missing the point. •Some gestures might reflect emotional states, but the vast majority do not. •If you read almost any book with body language in the title, you will be at a complete loss as to what such hand movements actually do. •Gestures are uniquely important in reading another person's underlying thoughts; therefore they are uniquely important.

conclusion

•Gestures increase the capacity of memory; not gesturing decreases it. Gestures "lighten the cognitive load." •The study suggests that gestures actively participate in thought processes. •When participants were not allowed to gesture, mental performance decreased. •Even though gestures were appropriate to the math problem, not to the list of words, the lists of words were remembered better.

goldin-meadows ted talk http://www.youtube.com/watch?v=tPPaJrhluS4

•Goal is to convince the audience that gestures are "not mere hand-waving." Rather: -Gestures reflect what's on our minds -They can also change what's on our minds

"lightening the cognitive load"

•Goldin-Meadow tested the hypothesis that gesturing "lightens the cognitive load" by manipulating the amount of gesturing participants were allowed to engage in •She found that memory improved when participants were allowed to gesture

definition of gesture

•Here is a TED talk given by Susan Goldin-Meadow: http://www.youtube.com/watch?v=tPPaJrhluS4 •The term "gesture" refers to hand movements used to represent information, not to perform actions in the world such as typing or steering a car •Definition: Gestures are hand movements that do not have a direct effect on the world, but rather they affect the world indirectly by representing information about it

more ex

•Here is an example from the TED talk of matchers and mismatchers in a conservation of numbers task. •A young child is asked whether number of checkers in 2 identical rows is the same; and then whether the number of checkers in one row changes when the checkers are spread out •The child responds that the number of checkers in the 2 rows is the same at the beginning, but they are different after they are spread out •When asked to explain, the child says in words that the number of checkers is different "cause you spreaded them out," while at the same time producing a spreading-out motion with her hands •This child is a "matcher"

cont conclusion

•Iconic gestures are not separate from thinking and speech but part of it. •They allow us an enormous insight into the way people think because they offer insight into thinking through a completely different medium from that of language; a medium that is iconic rather than verbal. •Such gestures may indeed offer a window into the human mind and how it represents our thinking about events in the world

iconic gestures

•Iconic gestures depict concrete objects and events •They accompany speech that "tells a story"

ch5 analysis

•In Beattie's analysis: •Speech describes the event in a linear and segmented fashion: -Linear because it unfolds over the dimension of time -Segmented because it breaks the event down into conventional units of meaning (words and phrases) •Speech first identifies what is being raised ('the table'), then describes the action ('can be raised up'), and then describes the direction of the action ('towards the ceiling'). •This "linear-segmented" character of language arises because speech is one-dimensional - it can only vary along the single dimension of time - and because it relies on conventional categories - the categories of morphology and syntax

goldin meadow 2

•In other studies, Goldin-Meadow has shown that the amount of gesturing increases when speaker becomes more difficult: When bilinguals are not equally fluent in their two languages, they gesture more in their nondominant language Children gesture more when children they are asked to count a greater number of objects Speakers gesture more when are asked to describe paintings from memory, rather than when looking at them When kindergartners are asked to reason about objects, such as whether they amount of clay changes when it is reshaped into another form, they use more gestures than when asked simply to describe how the clay looks -Why does gesturing increase when speaking becomes more difficult? Because it save cognitive effort: it "lightens the cognitive load." Speakers produce more gestures in difficult tasks to make the task easier

cont

•In speech we obfuscate and deviate, we avoid the issue, we talk our way around things, we cheat and we lie - and we can do this because our speech is conscious and controlled •But in gesture our movements can give the game away: most gestures are unconsciously produced and contain information that we, the speakers, are unaware is there -When we gesture we are usually unaware of the exact form and trajectory of our gestures, their sheer extent, and sometimes of whether we are moving our hands at all

cont

•It has been argued that nonverbal communication is used for negotiating interpersonal relations while the verbal channel is used primarily for conveying information. •This view is wrong on both counts: •The verbal channel is critical for negotiating interpersonal relations. •The nonverbal channel is critical for conveying semantic information. •Some have argued that "only 7% of communication is verbal." This is a serious misreading of these original studies.

different vehicles of meaning beattie ch 6

•McNeill argues that the method by which gestures convey meaning is fundamentally different to the way speech does this. •Speech acts by segmenting meaning so that an instantaneous thought is divided up into its component parts and strung out through time. •Here is an example from Beattie's research of a narrator telling a cartoon story: the table can be [raised up towards the ceiling] Iconic: Hands are resting on knee; hands move upwards, palms pointing down, forming a large gesture; hands continue moving until the hands reach the area just above shoulder level. •Notice that the same event is described both by speech and gesture

metaphoric gestures

•Metaphoric gestures are similar to iconic gestures in that they are essentially pictorial, but the content depicted here is an abstract idea rather than a concrete object or event. •In the words of David McNeill: 'The gesture presents an image of the invisible - an image of an abstraction.' McNeill (1992: 14) uses the following example to illustrate the concept of a metaphoric gesture: It [was a Sylves]ter and Tweety cartoon Metaphoric: Hands rise up and offer listener an 'object' •The gesture makes the cartoon genre concrete and places it into an act of offering

metaphoric gestures

•Metaphoric gestures embody "abstract" ideas •For example, a narrator says, "it was a Sylvester and Tweety cartoon" while raising his hands as if offering an object to listener

method of this study

•Method Participants were both children and adults. They were asked to: (1)Solve a math problem: •children were asked to solve 4+5+3 = __+3 •adults were given more difficult problems (2)Memorize short and longer random lists of words (3)Explain how they arrived at the solution to the math problem •half of the participants were permitted to gesture* •half were not; they kept their hands still on the table top (4)Recall the lists of words

Here is part of an episode from "Restaurant Makeover: Phil's original barbeque" (from 3:52 to end): https://www.youtube.com/watch?v=OxEjyoand9A

•Notice in the video how the participants in a conversation are part of their environment, and how their hands are often in motion, producing spontaneous gestures. •In the video, you can see that speech and gesture form a single, unified system •This is a new perspective on spoken language, in which language consists not only sound, but rather it is a blend of sound and sight •In this new conception of language, the hand movements we produce as we talk are tightly intertwined with talk in timing and meaning, that is, acts of speaking and gesturing are bound to each other

conclusions about relationship b/w speech and gestures

•Our hands articulate ideas that run parallel to those expressed in speech •In everyday conversation, listeners habitually and effortlessly extract the information from these movements and combine it with the information contained in the speech itself •The meaning expressed is most often complementary to that expressed in speech, quite literally "combining in such a way as to enhance or emphasize each other's qualities"

gestures as a window onto the mind

•Our hands embody our thinking through bodily action, with little or no conscious awareness •Thus gestures provide us with a glimpse of our hidden unarticulated thoughts •They are a window on the human mind; they make thought visible

beatties summary of ch 6

•Speech operates in a linear and segmented fashion; gesture operates in terms of multidimensional images. •The linear-segmented character of speech arises because speech is essentially one-dimensional - it unfolds along the dimension of time. •In speech, various elements of language combine to form larger units - words combine into phrases, clauses into sentences - but different gestures do not combine together to form more complex gestures. Traditional sign languages have the properties of spoken language: a lexicon, standards of well-formedness, and a syntax, or a set of rules for combining signs that includes word order, to form meaningful sentences.

cont ch 6

•Spontaneous gestures have no such lexicon, no such standards of form, and no such syntax. •Images depicted in hand gestures and speech emerge together from the same underlying idea or representation. •A spontaneous gesture is not a translation of a word, concept or sentence, or an independent visual display shown at the same time as the spoken utterance; rather the gesture is a part of the spoken utterance. •Gestures do not arise from some advanced verbal plan of the utterance; rather the two forms of communication arise from some underlying primitive idea. •Analysis of spontaneous, unconscious gestures can allow us a great deal of insight into the nature of that primitive idea.

return to embodied cognition

•The central claim is that language and thought are embodied; that "the mind" is located with respect to: -neurological and biological processes of the brain itself -complex bodily interconnections •Gestures are an example of the embodiment of language -Gestures don't represent what they depict, but rather they embody it; they are action itself -"Gesture and the imagery it embodies are an integral step on the way to a sentence" (David McNeill)

the emerging field of gesture studies

•The field of "gesture studies" is a new and growing field in many language-related disciplines •It forms part of a growing movement away from traditional studies of cognition, toward the new field of "embodied cognition"

beattie ch 5

•The form of iconic gestures displays a close relationship to the meaning of the accompanying speech. •Example: When a narrator describes a scene from a comic book story in which a character bends a tree back to the ground, the speaker appears to grip something and pull it back. The gesture is iconic because it refers to the same act mentioned in the speech; the gesture seems to be connected to the words 'and he bends it way back'.

CEO example

•The speaker does not say what he intends to say and then try to make it clearer with a gestural illustration; rather, he uses both speech and movement simultaneously •Hand movements and words both derive from the same underlying mental representation at exactly the same time •Often, the beginning of the gesture slightly precedes the speech so that the hands can be in exactly the right position to make the critical movement at the right time •The core of a gesture is perfectly timed with speech, and together they form a complete whole - the two systems of gesture and speech are perfectly coordinated •In the words of David McNeill (2012), 'Gestures are components of speech, not accompaniments but actually integral parts of it'

where the action is

•The vast majority of gestures - spontaneous gestures - are not like emblems: -They have no direct verbal translation -They do not substitute for words but rather are produced alongside them -They are produced unconsciously as individuals speak -They are almost impossible to inhibit, as you can see when you watch someone gesturing with their free hand as they speak on the telephone -If you interrupt speakers while they are talking and ask them to reproduce these types of gestures, they find it difficult, even sometimes impossible

cont

•The younger child says in speech that the water level is low or high, and his gestures "say" how low or high •For the older child, gesture allows the child to convey the contrasting dimensions of the containers. The gestures "say" "this one's lower but wide," and "that one's higher but skinny" •In both cases, the children's hands convey substantive information •Goldin-Meadow found that the performance of mismatchers improves after instruction: Mismatchers are more ready to learn than matchers

ch 5 summary cont

•There are three main types of spontaneous gestures that accompany speech - iconic gestures depict concrete objects or events, metaphoric gestures depict abstract ideas, and finally, beats, which are simpler stress-timed movements. (Note: In class we include pointing gestures as well.) •Iconic and metaphoric gestures can reveal unarticulated aspects of thinking. •Even beats may provide a clue as to the inner workings of the mind of the speaker. They demarcate those parts of the discourse that speakers themselves at the moment of production consider most significant.

goldin meadows findings

•Typically, under the age of 7, children respond that the amount of water has changed. The child below responds: in speech, "cause that's (A1) down lower than that one (A2)" in gesture, he points at the relatively low water level in the short, wide container (A1) and then at the higher water level in the tall, skinny container (A2) •G-M calls this child a MATCHER because his speech and gesture "match"

results

•When gestures were not allowed during explanation, recall of long lists was worse -The results held even when speakers were allowed to gesture, but chose not to

cont ch 1 summary

•When speech and gesture refer to the same basic concept but do not match the idea expressed in the hand movement, this can be a much more accurate reflection of the underlying thoughts than the speech itself. •This is because the 'unconscious meaning unconsciously expressed' has not been controlled or edited by the speaker to send a particular type of message. •Movements of the hands and arms can act as a window on the human mind; they make thought visible.

in aphasia...

•communication breaks down with the brain damage that produces different types of aphasia •Speech and gesture break down in strikingly similar ways -In Wernicke's aphasia, patients produce fluent speech that has little appropriate semantic content, and such individuals are also found to use few iconic gestures -In Broca's aphasia, there is appropriate semantic content but little overall structure or fluency, and iconic gestures are preserved

beattie's theory of language

•gesture is as much a part of language as speech -Movements of the arms and hands reflect our thinking, -Speech reflects our thinking as well, but gestures do it in a completely different manner, using a different sort of system of communication with very different properties -We are much less aware of what our hands are communicating than what we are saying verbally -The information in the hands can be incredibly important