Psych 2300 - Chap 6; Exam 2

Leading questions

a question that encourages or implies a particular answer EX. "What did you think of that amazing concert?" *Use neutral phrasing to try to capture a person's 'true" opinion*

Negatively worded questions

*causes confusion, thereby reducing the construct validity of a survey or poll.* -EX. Survey on Holocaust denial - found that 20% of Americans denied the Nazi Holocaust ever happened. They had been asked, " Does it seem possible or impossible to you that the Nazi extermination of the jews never happened?" < How do you answer a question like this?

Every multi-scale item has an order. What should we do?

- produce multiple versions, then compare results. If there is no difference between versions, then we have greater confidence in the results. If there is a difference then we should report both outcomes. - try to identify a set of questions that does not frame the interpretation of later questions. (try to find questions that are not framed inappropriately by earlier questions) -If there is a "most-important" question on your survey, you may wish to ask that question first.

Which of the following might be useful for reducing or eliminating the use of response sets in a long survey? Please consider the response sets listed in each possible answer and determine whether the proposed solution should work.

-Using reverse-worded items to reduce or eliminate yea-saying (or at least some of the effects of yea-saying). -Ensuring that there is no middle option available to reduce or eliminate fence sitting. (e.g., making sure that there are an even number of response options available rather than an odd number of response options in a Likert-type scale).

Response sets and ways to reduce their use (4)

-acquiescence (yea-saying), nay-saying, and fence-sitting - reverse-worded questions

Socially desirable responding (faking good) & faking bad

-giving survey responses that make one appear better (or worse) than is actually the case -guarantee anonymity & ensure that people know that their responses are anonymous -past deception in research may cause problems, if participants don't trust the researcher, they may not believe that their responses really are anonymous.

Problems associated with writing good questions (3)

-leading questions -double-barreled questions -negatively worded questions (double negatives)

Preventing (or diminishing) observer bias and observer effects

-masked (blind) design

Types of Bias (3)

-observer bias -reactivity -observer effects (expectancy effects)

Masked (blind) Design

-observers are unaware of the conditions to which participants have been assigned and are unaware of the purpose of the study The Story of Clever Hans : -William Von Osten tutored his horse (Clever Hans) in mathematics. -He would ask Hans to add 3 and 2, the horse would tap his hooves 5 times and then stop -After 4 years of daily training, Hans would perform math at least as well as an average 5th grader, identify colors, and read German words. -Von Osten allowed many scientists to test Hans abilities and concluded that Hans was truly capable of doing math. -Up until a experimental psychologist (Oskar Pfungst) came up with a more rigorous set of checks. - He suspected that the animal was sensing subtle nonverbal cues from his human questioners. -Pfungsts showed the horse a series of cards printed with number - as suspected Hans was correct only when his questioner saw the card. -Hans was smart at detecting subtle head movements of the questioner.

Types of Survey Questions (4)

-open-ended questions -force-choice questions -Likert & Likert-type scales -Semantic differential scales

Question Order

-preceding questions can frame subsequent interpretations & encourage particular response patterns on later questions. *the most direct way to control for the effect of question order is to prepare different versions of a survey, which the questions in different sequences.* (Wilson, Moore, Mckay, & Avery, 2008) -participants were asked two-forced questions in this order: 1. Do you generally favor or oppose affirmative action programs for women? 2. Do you generally favor or oppose affirmative action for racial minorities? (the other half of respondents were asked the same questions just in a different order) *they recorded that whites reported more support for affirmative action for minorities when they had first been asked about affirmative action for women.*

Good observations are both

-reliable and valid

If, on an anonymous online survey, a person answers a set of thirty questions by consistently giving the response "strongly agree" on a Likert scale, then this person is likely exhibiting which of the following?

Acquiescence

If its response labels are a bit different from the original Likert labels it may be called...

Likert-Type Scales

The story of Clever Hans best demonstrates which of the following ideas?

Subtle behavior from observers can influence how an organism behaves.

Imagine that someone criticizes a question for being "double-barreled." What does that mean?

The question is problematic because it asks two questions instead of one.

Imagine that a researcher asked people only the following question: "What do you think of Senator John Doe's egregious endorsement of Senate Bill X?" (Note: "egregious" means exceptionally bad in this case).

The question is problematic because it is a leading question.

In a study by Langer and Abelson (1974), clinicians were shown a video of a man talking about his feelings and work experiences. Some of the clinicians were told that the man was a job applicant, while other clinicians were told that the man was a patient. The results of this study demonstrate which of the following potential problems for an observational study?

The results of the study demonstrated the problem of observer bias.

Likert & Likert Type Scales

a scale in which each statement is accompanied by 5 response options: strongly disagree, disagree, neither agree nor disagree, agree, strongly agree. *minor variations* - these are very common in psych

Response sets (non differentiation)

a shortcut approach to answering related survey questions. Rather than reporting sincere, thoughtful, & question - specific responses, a person may develop a pattern for responding.

Acquiescence (Yea-saying)

agreeing or answering in a positive manner for all questions *this can threaten construct validity because instead of measuring the construct of true feelings of well-being, the survey could be measuring the tendency to agree, or the lack of motivation to think carefully. *Researchers will try reverse-wording some questions to be able to tell the difference of yea-saying and someone who really does agree with all the items being given. ***The opposite strategy is sometimes called nay-saying****

Open-ended questions

allows respondents to answer any way they like allows for rich information and unanticipated responses requires careful analysis and effortful (time-intensive) coding of information. Ex. What is your favorite style of music? (no response options given)

Double-barreled questions

asking two questions at once. (This is not the same as a single ambiguous question that can be interpreted in different ways) EX. "Wasn't that guitar riff amazing and weren't the song lyrics clever? *if someone says "yes" what does that mean? Are they referring to the guitar solo, the song lyrics, or both? GOAL : Simplicity (one thought at a time)

Reactivity

change in behavior when study participants know another person is watching. They might react by being on their best behavior - or in some case, their worst. What should a researcher to do avoid this? -unobtrusive observations (blend in) -wait for it (allow acclimation to one's presence) -measure the results of a behavior (indirect measures of constructs) rather than the behavior itself. *Observe Ethically*

Recalling Information

flashbulb memories - of major events, vivid but no more accurate confidence - is poorly calibrated with accuracy *we may have poor construct validity as a measure of past behaviors * possible solutions : observe directly, rather than rely on self-reports

Observer effects (expectancy effects)

participants behavior changes to better match the observers expectations *Observers can effect what they see* *This study showed that observers not only see what they expect to see; sometimes they even cause the behavior of those they are observing to conform to their expectations.* -Rosenthal and Fode (1963) demonstrated that rats perform better when their teachers/handlers are led to expect exceptional ability -Rats were labeled as "maze-dull" or "maze-bright" -Participants were asked to train the rats to run a maze over the course of 5 days. -Even though the rats to genetically similar, those that were believed to be maze-bright completed the maze a little faster each day and with fewer mistakes, in contrast the rats considered to be maze-dull did not improve their performance over the testing days.

Forced-choice questions

people give their opinion by picking the best of two or more options. -limits the type of responses; requires selecting from a pre-specified set or responses *often used in political polls* EX. Narcissistic Personality Inventory (Raskin & Terry, 1988) two opposing statements are provided & respondents are asked to choose one. -I really like to be the center of attention -It makes me uncomfortable to be the center of attention

Fence sitting

playing it safe by answering in the middle of the scale (possibly to avoid reporting controversial opinions) *this weakens a survey's construct validity when middle-of-the-road scores suggest that some respondents don't have an opinion, when they actually do.* -removing neutral or middle response options may help.

Observer Bias

rather than rating behaviors objectively, researchers might unknowingly rate behaviors according to their expectations or hypothesis *Observers can see what they expect to see* -In a study, psychoanalytic therapists were shown a video of a 26 yr old man talking to a professor about his feelings and work experiences. (Longer & Abelson, 1974) Some of the therapists were told the young man was a patient, the other therapists were told he was a job applicant. After watching the video, the clinicians were asked for their observations. Although the therapists saw the same person in the video they did not have the same observations. Those who thought he was a job applicant had more positive observations that those who thought he was a patient.

Semantic differential Scales

respondent might be asked to rate a target object using a numeric scale that is anchored with adjectives. EX. Easy 1 2 3 4 5 Hard

Imagine that a researcher is interested in studying the amount of sleep that graduate students typically get at her university. The researcher compiles a list of all graduate students at her particular university and randomly selects 20% of the graduate students. She then sends a survey to those graduate students. In this example, the 20% of graduate students that the researcher surveys is her...

sample

One way to reduce or eliminate the effects of observer bias would be to...

use a masked (or blind) design.

If reliability is low, then

we can't have have good construct validity (are observations better than self-reports?)