Research Methods Chapter 6 Surveys and Observations

Socially desirable response

Giving answers on a survey (or other self-report measure) that make one look better than one really is. Also called faking good.

Faking Good

Giving answers on a survey (or other self-report measure) that make one look better than one really is. Also called socially desirable responding.

Faking Bad

Giving answers on a survey (or other self-report measure) that make one look worse than one really is.

Masked Design

A study design in which the observers are unaware of the experimental conditions to which participants have been assigned. Also called blind design.

Force choice format

A survey question format in which respondents give their opinion by picking the best of two or more options.

Open Ended Question

A survey question format that allows respondents to answer any way they like.

Likert Scale

A survey question format; a rating scale containing multiple response options that are anchored by the terms strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. A scale that does not follow this format exactly is called a Likert-type scale.

double-barreled question

A type of question in a survey or poll that is problematic because it asks two questions in one, thereby weakening its construct validity.

Leading Question

A type of question in a survey or poll that is problematic because its wording encourages only one response, thereby weakening its construct validity.

semantic differential format

A response scale whose numbers are anchored with contrasting adjectives.

Response Set

A shortcut respondents may use to answer items in a long survey, rather than responding to the content of each item. Also called nondifferentiation.

unobtrusive observation

An observation in a study made indirectly, through physical traces of behavior, or made by someone who is hidden or is posing as a bystander.

Professor Meyer gives the students in his class a mid-semester feedback survey asking them how stressed out they are by the assignments in his class. The majority of his class report that they feel "extremely stressed." What might explain this? 1. reactivity 2. socially desirable responding 3. faking bad 4. faking good

Faking Bad

Which of the following is true about self-reports? They are always biased by socially desirable responding. They are always less reliable and valid than observational data. They are superior to observational data because they aren't susceptible to observer bias. They are valuable sources of information when researchers are interested in a person's internal experiences.

valuable sources when interested in personal experiences

Which of the following is true about open-ended questions? They are more efficient than asking forced-choice questions. They completely lack construct validity. They are the most common format for psychologists to ask questions. They provide rich data, but they can be time-consuming for researchers because the responses need to be coded.

provide rich data

The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. When the observations began, the observers noticed that bicyclists slowed down when they neared the observers. How could this reactivity be avoided? The observers could make unobtrusive observations by hiding. The observers could use a blind study design. The observers could use random assignment. The observers could use a masked design.

unobstrusive behaviors

Which of the following is NOT one of the ways that researchers can avoid reactivity in observational studies? use unobtrusive observations wait for the participants to become used to the observer before collecting data use a masked or blind study design measure the traces of behavior

using a blind study

Jenny recently learned of plans to cut down an old beautiful tree on her campus to make way for a new bike path. Jenny is opposed to cutting down the tree, and she decides to survey some students at her university to see if others also oppose cutting down the tree. She plans to share the results of her survey with the school administration to argue to keep the tree. One question on Jenny's survey asks, "Would you be in favor of brutally cutting down this majestic tree to make way for a stupid bike path?" How could this question be changed to improve its construct validity? 1. use more neutral language so that it is not a leading question 2. split the question into two so that it is not double-barreled 3. add another option such as "or would you favor leaving the poor tree alone" to make the question forced-choice 4. change the wording so that it is positively worded, not a double-negative

1.

The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. When the observations began, the observers noticed that bicyclists slowed down when they neared the observers. How could this reactivity be avoided? 1. The observers could make unobtrusive observations by hiding. 2. The observers could use a blind study design. 3. The observers could use a masked design. 4. The observers could use random assignment.

1.

When people are asked why they made a certain choice, they ______________. will tell you why they think they made that choice, but they may not be accurate at identifying the true reason for their choice will lie to you about why they made that choice are always able to tell you why they made that choice will refuse to tell you why they made a certain choice because they don't know

1.

Which of the following is a way of preventing reactivity? 1. waiting for the participants to become used to the observer 2. making sure the observers do not know the study's hypothesis 3. making sure the observer uses a clear code book 4. ensuring the observers have good interrater reliability

1. waiting for the participants to become used to the observer

Professor Law wants to construct a survey to assess involvement with computer games. Which question is a double-barreled question? On a scale of 1 (Strongly agree) to 5 (Strongly disagree) scale, rate the following statement: Computer games are a great pastime. On a scale of 0 (Not at all) to 5 (Very much), rate how much you like and play your favorite game. Which is truer of you? a) I have little interest in computer games or b) I would miss computer games if I couldn't play anymore. What computer games have you played?

2

Which of the following makes it more likely that behavioral observations will have good interrater reliability? 1. a masked study design 2. a clear codebook 3. using naive, untrained coders 4. open-ended questions

2. Clear Codebook

Which of the following is a means of controlling for observer bias? 1. using obtrusive observations 2. waiting for the paricitpants to become used to the observer 3. making sure the observer does not know the study's hypothesis 4. measuring physical traces of behavior rather than observing behavior directly

3. Making sure the observers does not know the study's hypothesis

Professor Law wants to construct a survey to assess involvement with computer games. Which question uses a Likert-type scale? What computer games have you played? On a scale of 0 (Not at all) to 5 (Very much), rate how much you like and play your favorite game. Which is truer of you? a) I have little interest in computer games or b) I would miss computer games if I couldn't play anymore. On a scale of 1 (Strongly agree) to 5 (Strongly disagree) scale, rate the following statement: Computer games are a great pastime.

4

The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. After completing the observational study, Professor Ibrahim sends a survey out to the entire campus about bicycle safety, and asks all bicyclists to respond. He finds a large difference between their reports of how safely they ride and what his observers found. What is the most likely cause of the self-report ratings being inconsistent with the observational data? The bicyclists were probably responding in a socially desirable way on the self-report survey. The bicyclists were probably faking bad on the self-report survey. The bicyclists were probably fence-sitting on the self-report survey. The question order on the self-report survey probably affected the responses on the self-report survey.

4

Judy writes a survey to assess how much people worry. Her survey has 20 questions that people can rate their level of agreement to on a seven-point Likert scale. All of the questions are worded so that higher responses will indicate a higher level of worry. After having 100 people complete her survey, she finds that a lot of respondents often respond to all of the questions by choosing only the "strongly agree" option. How could she improve the construct validity of her survey? 1. She could change the number of response options. 2. She could encourage people to be as honest as possible while filling out the survey. 3. She could use the worry questionnaire on a group of people with anxiety disorders. 4. She could include reverse-worded items.

4.

Observer Bias

A bias that occurs when observers' expectations influence their interpretation of the participants' behaviors or the outcome of the study.

Reactivity

A change in behavior of study participants (such as acting less spontaneously) because they are aware they are being watched.

Observer Effect

A change in behavior of study participants in the direction of an observer's expectation. Also called expectancy effect.

Survey

A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet. Also called poll.

Poll

A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet. Also called survey.

Negative worded association

A question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity.

acquiescence

Answering "yes" or "strongly agree" to every item in a survey or interview. Also called yea-saying.

The following items appear on a survey: Is your cell phone new and does it have all the latest features? What is the biggest problem with this wording? 1. it is a leading question 2. it involves negative wording 3. it is double barreled 4. it is not on a likert scale

Double Barreled

What are three potential problems related to the wording of survey questions? can they be avoided?

Leading questions Double Barreled Negative Wording Yes they can be avoided

What is the difference between observer bias and observer effects? How can such biases be prevented?

Observer Bias: When observers expectations influence their interpretation of the participants behaviors or the outcome of the study. Observer effects: Observer change the behaviors of those they are observing, such that participants behavior changes to match observers expectation. Prevention: Training and Codebooks that provide clear expectations and set guidelines.

Fence Sitting

Playing it safe by answering in the middle of the scale for every question in a survey or interview.

What is reactivity? What three approaches can researchers take to be sure that people do not react to being observed?

Reactivity occurs when people change their behavior in some way when they know when another person is watching. 1. Blend in: Unobtrusive observations: make yourself less noticiable 2. wait it out: Example Jane Goodall and her chimps- had to wait for them to get used to her being there then they would forget that she was there. 3. measure the behavior's results

What are some ways to ensure that survey questions are answered accurately?

Reversed worded items Forced-choice format

For which topics, and in what situations, are people more likely to answer accurately to survey questions?

Self-reports and topics that are about their own experience

Joseph wants to find out what customers at his restraint think about the food and the service. He creates a survey for diners to fill out at the end of their meal with three questions: 1) Please rate the food from 1-10, where 1 is yucky and 10 is delicious; 2) Please rate the service from 1-10 were 1 is unacceptable and 10 is outstanding. What kind of question format is Joseph using? forced choice open-ended Likert scale semantic differential

Semantic Differential

In which of the following situations do people most acccurantley answer survey questions? 1. when their answers are anonymous 2. when they are describing the reasons for their own behavior 3. when they are describing what happened to them, especially after important events 4. when they are describing their subjective experience, how they personally feel

Subjective Experience

When people are using an acquiescent response set there are: 1. trying to give the researcher the responses they think he or she wants to hear 2. misrepresenting their views to appear more socially acceptable 3. giving the same, neutral answer to each question 4. tending to agree with every item, no matter what it says

Tending to agree with every item, no matter what it says

Observational Research

The process of watching people or animals and systematically recording how they behave or what they are doing.

Judy writes a survey to assess how much people worry. Her survey has 20 questions that people can rate their level of agreement to on a seven-point Likert scale. All of the questions are worded so that higher responses will indicate a higher level of worry. After having 100 people complete her survey, she finds that a lot of respondents often respond to all of the questions by choosing only the "strongly agree" option. What is this kind of responding called? fence sitting faking bad acquiescence faking good

acquiescence

Which of the following is a method to control for observer bias? wait for the participants to become used to the observer before collecting data use a masked or blind study design use unobtrusive observations measure the traces of behavior

blind study

Professor Meyer gives the students in his class a mid-semester feedback survey asking them how stressed out they are by the assignments in his class. The majority of his class report that they feel "extremely stressed." What might explain this? faking bad faking good reactivity socially desirable responding

faking bad

Jenny recently learned of plans to cut down an old beautiful tree on her campus to make way for a new bike path. Jenny is opposed to cutting down the tree, and she decides to survey some students at her university to see if others also oppose cutting down the tree. She plans to share the results of her survey with the school administration to argue to keep the tree. One question on Jenny's survey asks, "Would you be in favor of brutally cutting down this majestic tree to make way for a stupid bike path?" A problem with this question is that it is a _______________. negatively worded question forced-choice question leading question double-barreled question

leading question

The following item appears on a survey: "On a five-point scale, where 1 is Strongly disagree, 2 is disagree, 3 is Neither agree nor disagree, 4 is Agree, and 5 is Strongly agree, rate the following statement:"I look forward to coming to class". What type of question format is being used? Likert scale semantic differential forced choice open-ended

likert scale

Jenny recently learned of plans to cut down an old beautiful tree on her campus to make way for a new bike path. Jenny is opposed to cutting down the tree, and she decides to survey some students at her university to see if others also oppose cutting down the tree. She plans to share the results of her survey with the school administration to argue to keep the tree. One question on Jenny's survey asks, "Do you oppose not cutting down this tree?" A problem with this question is that it is a _____________. double-barreled question negatively worded question leading question forced choice question

negative wording

Jenny recently learned of plans to cut down an old beautiful tree on her campus to make way for a new bike path. Jenny is opposed to cutting down the tree, and she decides to survey some students at her university to see if others also oppose cutting down the tree. She plans to share the results of her survey with the school administration to argue to keep the tree. One question on Jenny's survey asks, "Would you be in favor of brutally cutting down this majestic tree to make way for a stupid bike path?" How could this question be changed to improve its construct validity? add another option such as "or would you favor leaving the poor tree alone" to make the question forced-choice split the question into two so that it is not double-barreled use more neutral language so that it is not a leading question change the wording so that it is positively worded, not a double-negative

neutral language

The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. Both observers are very interested in the topic because they have been struck by bicycles. Although the interrater reliability is high, what threat to construct validity should concern Professor Ibrahim? selection bias observer bias observer effects reactivity

observer bias

The campus safety committee has asked Professor Ibrahim to study bicycling on his campus. He trains two observers and has them observe the number of cyclists and their safety at various points around campus. Initially, the observers can't agree about what is dangerous behavior. Professor Ibrahim refines his codebooks to clearly define the rating scales and retrains the observers. What is he addressing by doing this? reactivity the reliability of the measurement acquiescence socially desirable responding

reliability of measurement

Which of the following is NOT a way to control for socially desirable responding? include filler items on the survey that mask the true purpose of a sensitive survey include unrelated items such as "I always help someone in need" that reveal a person to be high on socially desirable responding, and consider excluding their data remove the neutral option from the survey so that a person must choose one side or the other use a special, computer-based technique such as the Implicit Association Test to find implicit opinions

removing the neutral option

Which of the following describes a "fence-sitting" response to a survey? responding to a controversial question on a survey by selecting the response right in the middle responding to the questions on a survey by consistently selecting all the "no" or "strongly disagree" responses responding in a socially desirable way responding to the questions on a survey by consistently selecting all the "yes" or "strongly agree" response

responding to the controversial question

Judy writes a survey to assess how much people worry. Her survey has 20 questions that people can rate their level of agreement to on a seven-point Likert scale. All of the questions are worded so that higher responses will indicate a higher level of worry. After having 100 people complete her survey, she finds that a lot of respondents often respond to all of the questions by choosing only the "strongly agree" option. How could she improve the construct validity of her survey? She could include reverse-worded items. She could use the worry questionnaire on a group of people with anxiety disorders. She could encourage people to be as honest as possible while filling out the survey. She could change the number of response options.

reverse word items

Joseph wants to find out what customers at his restraint think about the food and the service. He creates a survey for diners to fill out at the end of their meal with three questions: 1) Please rate the food from 1-10, where 1 is yucky and 10 is delicious; 2) Please rate the service from 1-10 were 1 is unacceptable and 10 is outstanding. What kind of question format is Joseph using? open-ended forced choice semantic differential Likert scale

semantic differential

The following problematic question appears on a survey: "Please indicate whether the following statement is true or false for you: My cell phone is new and has all the latest features". How could this question be changed to improve its construct validity? use more neutral language so that it is not a leading question split the question into two so that it is not double-barreled change the wording so that it is positively worded, not a double-negative hange the order of "is new" and "has all the latest features" so to address problematic question ordering

split the question in two

Professor Law wants to construct a survey to assess involvement with computer games. Which of the following would be an example of an open-ended question? Which is truer of you? a) I have little interest in computer games or b) I would miss computer games if I couldn't play anymore. On a scale of 0 (Not at all) to 5 (Very much), rate how much you like and play your favorite game. On a scale of 1 (Strongly agree) to 5 (Strongly disagree) scale, rate the following statement: Computer games are a great pastime. What computer games have you played?

what computer games have you played

Professor Law wants to construct a survey to assess involvement with computer games. Which is a forced-choice question? On a scale of 0 (Not at all) to 5 (Very much), rate how much you like and play your favorite game. What computer games have you played? Which is truer of you? a) I have little interest in computer games or b) I would miss computer games if I couldn't play anymore. On a scale of 1 (Strongly agree) to 5 (Strongly disagree) scale, rate the following statement: Computer games are a great pastime.

which is truer of you

Is it ethical for psychological researchers to observe people in a public place? No, because the researchers will have to individually identify the people they observe. Yes, because as long as it is for the sake of science, it is ethical to observe people in public or private places. Yes, because in those settings people can reasonably expect their behavior to be public, not private. No, because informed consent cannot be obtained from those being observed.

yes because its public