EXAM 3 SHORT ANSWER

Name the three types of reliability and the four types of validity.

The student should state the three types of reliability: test-retest reliability, interrater reliability, and internal validity. The student should also state the four types of validity: external validity, statistical validity, internal validity, and construct validity. Note that the students do not need to provide these in any order or provide definitions.

RESEARCH STUDY 5.4 Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below. Statement 1: "I believe in a religion." Statement 2: "Part of who I am stems from my religious beliefs." Statement 3: "I believe that religion is unnecessary." Refer to Research Study 5.4 to answer the following six questions. Provide a suggestion for how Dr. Li could establish the criterion validity of his measure.

Criterion validity could be established by asking people to take his measure and collecting data at the same time about something that would be related to religiosity (examples could include frequency of church attendance, amount donated to religious causes, self-reports of whether they believe in a higher power or not, etc.). The criterion validity could also be established using the known-groups paradigm, which would involve giving the measure to a group of people known to be religious (perhaps church attendees) and those known not to be religious (maybe a group of known atheists) and determine whether the two groups score differently. Students may also state that criterion validity could be established by asking people to take his measure now and collecting data after some time has elapsed. The data collected in the future can be the same type of data collected concurrently.

RESEARCH STUDY 5.4 Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below. Statement 1: "I believe in a religion." Statement 2: "Part of who I am stems from my religious beliefs." Statement 3: "I believe that religion is unnecessary." Refer to Research Study 5.4 to answer the following six questions. Given the above situation, explain the difference between criterion validity, convergent validity, and discriminant validity. Why is each one important?

Criterion validity is a measure's ability to forecast an outcome that is related to the measure. For example, if the measure has criterion validity, it would be correlated with future behaviors (church attendance, amount donated to the church in the next month). Convergent validity is whether a measure is related to similar measures, such as spirituality. Discriminant validity is whether a measure is unrelated to concepts it should be unrelated to, such as health. All three types are important. Criterion validity is important because it indicates the measure's ability to predict things that Dr. Li cares about. Convergent validity is important because Dr. Li needs to make sure that his measure is related to similar concepts. Divergent validity is important because Dr. Li wants to make sure that the measure is not related to everything, but only to those things that make sense.

RESEARCH STUDY 5.3 Lauren, Sarah, and Jennifer are students in Dr. Shaffer's Research Methods class. For a class assignment, they are asked to devise an operational definition for romantic attachment, or love. Lauren decides to ask a group of married couples to report if they are in love with their spouse (1 indicates "Yes, I am in love" and 2 indicates "No, I am not in love"). Sarah decides to watch couples sitting on campus and measure how close they sit together, assuming that people who sit closer together are more in love. Jennifer decides to recruit a group of newlyweds and measure the change in heart rate that occurs between the start of the study (when the person is alone) and the moment they see their spouse walk into the room. Refer to Research Study 5.3 to answer the following two questions. For each student described above, indicate whether the operational definition of her variable is categorical or quantitative. If a student's operational variable is categorical, explain whether it could or could not be made quantitative and how you would do that.

Lauren's operational definition is categorical. To make this quantitative, the student should say that Lauren could ask participants to rate how in love they are, rather than just saying they are in love or are not in love. Sarah's operational definition is quantitative. Jennifer's operational definition is quantitative.

RESEARCH STUDY 5.3 Lauren, Sarah, and Jennifer are students in Dr. Shaffer's Research Methods class. For a class assignment, they are asked to devise an operational definition for romantic attachment, or love. Lauren decides to ask a group of married couples to report if they are in love with their spouse (1 indicates "Yes, I am in love" and 2 indicates "No, I am not in love"). Sarah decides to watch couples sitting on campus and measure how close they sit together, assuming that people who sit closer together are more in love. Jennifer decides to recruit a group of newlyweds and measure the change in heart rate that occurs between the start of the study (when the person is alone) and the moment they see their spouse walk into the room. Refer to Research Study 5.3 to answer the following two questions. For each student described above, indicate whether the operational definition of her variable is self-reported, observational, or physiological.

Lauren's operational definition is self-reported. Sarah's operational definition is observational. Jennifer's operational definition is physiological.

Explain why socially desirable responding happens. Name one way to decrease the likelihood of its occurrence.

Socially desirable responding happens when participants are embarrassed, or shy, or uncomfortable about giving an unpopular answer. Possible means of decreasing socially desirable responding include ensuring anonymity, creating questions designed to assess this tendency, getting others to rate participants (e.g., friends), or using specialized measures to keep participants from knowing exactly what they are studying (e.g., the IAT).

Dr. Leman is interested in using a survey to study people's experiences with natural disasters, like Hurricane Sandy and the tornadoes that hit Joplin, Missouri, and Tuscaloosa, Alabama. Provide one reason why a survey is a good method to study this topic and one reason that a survey may not be a good method.

Students can give a variety of reasons why using a survey may be a good method. These reasons may include that surveys are excellent measures of subjective experiences (e.g., fear, anxiety) or that surveys may be good measures of why people think they responded to the natural disaster the way they did (why they evacuated). Students can give a variety of reasons that is not a good reason, including but not limited to the faulty nature of people's memory, especially for vivid events (like a natural disaster). The answers in both cases should demonstrate that the student realizes that surveys are better for subjective information as opposed to objective information.

There are several ways to decrease socially desirable responding. Choose one of these and explain why it decreases this response tendency.

Students may provide any of the following: a. Ensure anonymity—this may allow participants to feel that others may not judge them or know that their answer is "unpopular." b. Measure participant's tendency to respond in a socially desirable way—this will allow the researcher to determine who these participants might be and thus, they can remove or discount their data. c. Use "other reports" instead of self-report— people may be less concerned about reporting something unpopular or embarrassing about someone other than themselves. d. Using a special measure, such as the IAT—this allows for more indirect or unconscious measurement.

Write three questions designed to measure liking for peanut butter and jelly sandwiches—one forced-choice, one open-ended, and one with a response scale (can be of any type).

Students must create a question for each type. For the response scale question, students may use any type of response scale (semantic differential, Likert-type, etc.). Sample questions include: a. Open-ended: How much do you like peanut butter and jelly sandwiches? b. Forced-choice: Choose between the following two statements: I really like peanut butter and jelly sandwiches OR I prefer not to eat peanut butter and jelly sandwiches. c. Using a 10-point scale in which 1 = hate and 10 = love, rate how much you like peanut butter and jelly sandwiches.

Imagine that you are trying to measure people's stress. Provide an example of measuring stress using a self-report measure, a physiological measure, and a behavioral measure.

Students must provide an actual measure of measuring behavior and not simply provide a conceptual definition. A self-report measure must involve asking a person how stressed they are in some way (the person must report on themselves). A physiological measure must involve collecting some biological information (e.g., heart rate, cortisol levels). A behavioral measure must involve collecting data that can be directly observed (e.g., number of times a person looks at the clock during an exam, how long a student waits to speak to a professor).

Imagine that you are trying to measure people's stress. Provide three quantitative definitions of stress: one that is ordinal, one that is interval, and one that is ratio.

Students should create definitions that are quantitative (numerical), not categorical. They should make an ordinal definition that involves ranking people (e.g., the most stressed people in the class). Their interval definition should be something that has a nonsignificant 0 (e.g., the rating of stress on a 1-10 scale). Their ratio definition should be something that involves a meaningful 0 (e.g., number of stressful days experienced in the past week).

Dr. Sarolli is planning on measuring literacy (defined as the ability to read and write written text in one' s native language). He decides he wants to create a quantitative operational definition. Create an ordinal, interval, and ratio way to measure literacy.

Students should create definitions that are quantitative (numerical), not categorical. They should make an ordinal definition that involves ranking people, so that people with the highest literacy skills are rated highest and people with the lowest literacy skills are rated lowest, for example. Their interval definition should be something that has a nonsignificant 0, such as a 10-point scale on which people's levels of literacy are rated by themselves or by another observer. Their ratio definition should be something that involves a meaningful 0, such as the number of words a person can read in a given amount of time.

RESEARCH STUDY 5.4 Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below. Statement 1: "I believe in a religion." Statement 2: "Part of who I am stems from my religious beliefs." Statement 3: "I believe that religion is unnecessary." Refer to Research Study 5.4 to answer the following six questions. Dr. Li gives his new measure to a group of 100 adults in order to test the internal validity of his measure. Draw two scatterplots: one that demonstrates the likely relationship between Statement 1 and Statement 2 and one that demonstrates the likely relationship between Statement 1 and Statement 3. Explain why you have drawn each scatterplot as you have.

Students should draw two scatterplots. The first scatterplot (S1 and S2) should depict a positive slope/relationship, and the second scatterplot (S1 and S3) should depict a negative slope/relationship. Students may depict moderate or strong relationships but not weak relationships or nonrelationships. Students should explain that a person who says "yes" to S1 will probably say "yes" to S2 (positive relationship) and that a person who says "yes" to S1 will probably say "no" to S3 (negative relationship).

RESEARCH STUDY 5.4 Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below. Statement 1: "I believe in a religion." Statement 2: "Part of who I am stems from my religious beliefs." Statement 3: "I believe that religion is unnecessary." Refer to Research Study 5.4 to answer the following six questions. Name two types of reliability that apply to Dr. Li's measure and how each one could be established, given the above scenario.

Students should name test-retest reliability and internal reliability. If students name interrater reliability that is technically wrong, as it is unlikely that it is necessary given this measurement. The first reliability is test-retest reliability and is established by giving the measure to a group of people at one time and then again after some time has elapsed. The second type of reliability is internal reliability and is established by looking at the relationship between each of the 10 items. Students may also simply say that it can be established by calculating Cronbach's alpha. They may not simply say that it can be calculated with a scatterplot or a correlation coefficient, because those can be used for all types of reliability.

Explain why many people feel that physiological measurements are the best way to operationalize a variable but why they may not, in fact, be the best.

Students should say that many people may feel that physiological measurements are best because they tend to be very accurate and based on biological components. Students should also say that although physiological measurements are good and accurate, just like the other measures they must be validated with other measures.

RESEARCH STUDY 5.4 Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below. Statement 1: "I believe in a religion." Statement 2: "Part of who I am stems from my religious beliefs." Statement 3: "I believe that religion is unnecessary." Refer to Research Study 5.4 to answer the following six questions. Provide a suggestion for how Dr. Li could establish the face validity of his measure and a suggestion for how he could establish the content validity of his measure.

Students should say that the face validity could be established by giving his measure to experts (they can state several examples of experts) and having them look at whether the items make sense (look like they measure religiosity). Students should say that the content validity could be established by giving his measure to experts (again, they can state several examples of experts) and have them look at whether the statements measure the components of religiosity (which, from the definition, may include devotion and piety, although the students do not have to be this specific).

Explain why your textbook argues, "In fact, operationalizations are one place where creativity comes into the research process."

Students should say that this statement stems from the fact that there is no one way to operationalize a concept in psychology. The choice of how to operationalize an abstract concept allows the researchers to think outside the box and think of ways to observe something that is not necessarily easily observed. In addition, students may also state that researchers strive to improve existing operationalizations or create new ones even though an operationalization already exists, thus adding to the continuing creativity of the research field. Students may also say that this adds to the creativity of the research process because different researchers may generate different operationalizations, ensuring that the field has great variety.

Dr. Overup decides to use a hidden camera to videotape the behavior of people on dates at a local restaurant to determine how they express affection in public. State which threat to construct validity Dr. Overup is trying to address. Briefly discuss how she should deal with the ethics of the hidden camera.

Students should state that Dr. Overup is attempting to deal with reactivity. The student could mention several things related to ethics, including getting IRB approval, getting informed consent from people prior to videotaping, or deleting videotapes if participants request that after participation.

Dr. Overup decides to use a hidden camera to videotape the behavior of people on dates at a local restaurant to determine how they express affection in public. She has three research assistants code each video. She calculates an ICC of .94 for their rating of affection. Explain how Dr. Overup should interpret this value and what it tells her in terms of the reliability and validity of her measure.

Students should state that Dr. Overup should be pleased with an ICC of that value as it is very close to one, indicating a high degree of interrater reliability. Students should state that it appears that the research assistants are all rating the same behavior, indicating good reliability but they should state that this says nothing about the validity of her measure (it is possible that the research assistants are all similarly biased).

Dr. Smith is interested in measuring how adults view sexual behavior in teenagers. She asks the following two questions. Explain why Dr. Smith should be worried and how she can correct the problem. i. Is it okay for boys under the age of 18 to be sexually active? ii. Is it okay for girls under the age of 18 to be sexually active?

Students should state that Dr. Smith is likely to see order effects in her questions. They may also state that the order of her questions may affect the validity of her results. For example, people may say that it is okay for girls only because they originally said that it was okay for boys. (Students may actually state that people may be hesitant to reveal having a double standard.) To correct this problem, Dr. Smith should have two versions of the questionnaire—one with the question order of boys then girls and one with the question order of girls then boys—to make sure there are no differences.

What is an advantage of open-ended questions? What is a disadvantage?

Students should state that an advantage is the amount of information they can generate, that the data generated are very rich, or that the data generated can be spontaneous. Students should state that a disadvantage is that the data can be complicated/difficult, that the data still need to be coded/categorized, or that the data are very time-consuming.

Explain why observational methods are typically seen by most psychologists as ethical.

Students should state that in many cases, researchers are observing behavior that is expected to be public, meaning that there is no expectation of privacy. They may also mention that researchers do not identify the identities of the people they observe. Students may also mention that when more secretive measures are used to observe behaviors, researchers often obtain permission beforehand or explain the recording after the study.

Explain the difference between observer bias and observer effects and give an example of each.

Students should state that observer bias occurs when researchers rate or record behavior in a way that is not objective. An example should clearly demonstrate that the researcher is seeing something different based on their expectations (similar to the study described in the book of the psychiatrist judging a person differently based on whether they think they are a patient or not). Observer effects occur when participants' behavior changes to match the expectations of observers. An example should clearly demonstrate that the behavior change is coming from the participant, such that a participant acts smarter because the researcher treats them in a way that makes them behave that way.

State three ways that a researcher could reduce reactivity in a study and why each one is effective.

Students should state that one option is to hide their presence from the person they are observing. This is effective because the person cannot behave differently as a function of being observed if they are not aware they are being observed. A second option involves waiting until the person being observed has become desensitized to being observed and is likely to go back to responding "normally." A third option involves examining some product of people's behavior rather than the people's behavior itself. For example, instead of observing people's eating behavior, you could count the number of chips that are missing from the bowl after the study. This is effective because this method is unobtrusive and people will not be aware that their behavior's results are being observed.

Dr. Katz is interested in studying self-esteem. He is especially curious as to whether people who have higher self-esteem say the words I and me more often than do those who have low self-esteem. He has decided to poll people and ask them how often they say these words during the day. Explain why you would advise him not to use this method and explain what alternative you would propose.

Students should state that people are unlikely to be able to realistically know how many times they say any given word or two in a given day. (Students may be precise and say this is an issue of inability to report.) Thus, he is likely to acquire a lot of very inaccurate data. As an alternative, they should recommend some sort of observational measure. Students may recommend a specific way to collect data, but that is not necessary. If they mention some alternatives, these could include using some sort of recording device (similar to the electronically activated recorder [EAR] described in the book) or even bringing participants into a lab and counting the words in a lab conversation.

Explain the difference between validity and reliability. Explain whether they are related or independent concepts.

Students should state that reliability is the consistency of a measure, or the ability of a researcher to trust a measure. They should state that by contrast, the validity of a measure is belief that a measure is actually a measure of what it is purported to measure. Students should explain that they are fairly independent concepts, meaning that a measure can be reliable but not valid, or neither valid nor reliable. They may also state that reliability is necessary but not sufficient for validity, or that if a measure is valid, it is also reliable.

Explain why each of the following poses problems to a measure's validity: response sets (acquiescence, nay-saying, and fence sitting), socially desirable responding (faking good or faking bad), and inability to report.

Students should state that the response sets pose a problem for validity because it is unclear whether the reader really does agree (or disagrees or has no opinion) with each question/statement, meaning that we are unsure whether the data really does represent what they think (we just do not know). Socially desirable responding is a problem because we cannot tell whether a participant is simply trying to look good/bad and being dishonest or if they really do have those opinions/engage in those behaviors. Inability to report is a problem because participants may not realize that they do not know the exact answer (Why did I buy my new car? How many times did I eat lunch last month?) but may respond as if they do know. Instead of providing no data, they provide some data, but we are unsure of the data's accuracy.

Explain how a masked or blind design deals with both observer bias and observer effects.

Students should state that this design deals with observer bias and observer effects by keeping the observer unaware of the condition they are observing, so there is no way their expectations can affect either their measurements/observations of the participants (observer bias) or cause the participants to change their behavior in response to the expectations of the researcher (observer effects).

RESEARCH STUDY 6.3 Dr. Omiata is working on creating a survey designed to measure people's attitudes toward marijuana use. For each statement, the person is asked to rate their agreement on a 5-point scale. A selection of his statements is below. 1 - 5 Strongly agree - Strongly disagree A. I think that people who illegally ruin their lives by smoking pot are all alike. B. I think that marijuana should be legal in all 50 states and that people should have easy access to it. C. Marijuana consumption should never not be controlled. D. People who smoke marijuana are breaking the law. Refer to Research Study 6.3 to answer the following question. For each of Dr. Omiata's statements, state whether it is well-worded. If it is not, indicate what the problem is and how it could be fixed.

Students should state the following: a. Statement A is leading and is likely to have people all respond in the same way because of its tone. Fixes could involve taking out the phrase "ruin their lives," for example. b. Statement B is a double-barreled statement—it states two different things simultaneously. Students should recommend breaking it into two separate statements. c. Statement C is a double negative and is likely to lead to confusion. Fixes could involve rewording it ("Marijuana consumption should be controlled") or adding another statement ("Controlling marijuana is a good idea"). d. Statement D is well-worded.

RESEARCH STUDY 5.4 Dr. Li is interested in creating a measurement of religiosity. According to the dictionary, religiosity is "the quality of being religious; piety; devoutness." He creates a measure comprising 10 statements. People respond to each statement using the following scale (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree). Some of the statements are below. Statement 1: "I believe in a religion." Statement 2: "Part of who I am stems from my religious beliefs." Statement 3: "I believe that religion is unnecessary." Refer to Research Study 5.4 to answer the following six questions. Dr. Li gives his new measure to a group of 100 adults in order to test the internal validity of his measure. What type of correlation coefficients (r) would Dr. Li expect to find between Statement 1 and Statement 2 and between Statement 1 and Statement 3? What would these correlation coefficients tell Dr. Li about the internal reliability of his measure?

The first correlation coefficient (S1 and S2) should be a positive r. The second correlation coefficient (S1 and S3) should be a negative r. Students may describe moderate or strong relationships (r values of at least 0.5 or higher), but not weak relationships or nonrelationships. Students should explain that a person who says "yes" to S1 will probably say "yes" to S2 (positive relationship) and that a person who says "yes" to S1 will probably say "no" to S3 (negative relationship).