Chapter 7: Survey Research Review

A survey presents respondents with a series of statements and asks them to circle their level of agreement for each statement: "strongly agree; agree; neutral; disagree; strongly disagree" This series of questions uses which of the following?

A Likert scale *A Likert scale is a type of rating scale that captures the respondent's level of agreement or disagreement with a particular statement.*

Which of the following is not a characteristic of a high-quality survey question?

Does not ask "why" *Questions can ask the "why" of things but must do so using open-ended responses rather than closed-ended responses. High-quality survey questions must use clear and simple language, be precise, be cautious with sensitive topics, and must avoid leading questions.*

In recent years, academic and government researchers have increasingly used which type of survey?

Internet-based surveys *Web-based and e-mail-based surveys have long been used in market research, often for very short surveys. In recent years, however, academic and government researchers are increasingly using these platforms. The research organization typically e-mails a unique link to a potential respondent, who then clicks on the link and completes the survey online. The respondent proceeds through a series of questions, either typing in the correct answer, clicking on the appropriate response category, or using a dropdown menu and selecting the preferred response option.*

What is one of the major limitations of mail surveys?

Mail surveys often have very low response rates *Perhaps the most significant limitation of mail surveys is low response rate, often ranging from only 20% to 40%. It is easy to toss a printed questionnaire into the recycling bin, especially when no interviewer is present to urge the respondent to complete the survey. This low response rate is a potential threat to the survey's generalizability because those who complete their mail surveys tend to be significantly different from those who do not. They tend to be literate and well educated, with the free time necessary to fill out the survey, or they may feel a greater sense of social obligation than those who do not complete their surveys. If those who complete and return their surveys differ from those who do not, the results of the survey may be biased accordingly.*

Match each item with the correct definition. 1) Response Set 2) Order Effects 3) Priming Effects A) When the order in which questions appear biases the responses B) A type of order effects in which exposure to a particular image, word, or feeling shapes how respondents think and feel in the immediate aftermath C) The tendency to select the same answer to several sequential questions, perhaps out of boredom or with the desire to quickly finish the survey

1) C 2) A 3) B

Match each type of survey with the correct definition. 1) Longitudinal 2) Panel 3) Repeated Cross-Sectional 4) Cross-Sectional A) A type of longitudinal survey in which data are collected on the same subjects at multiple time points B) A survey in which data are collected at only one time point C) A study in which data are collected at multiple time points D) A type of longitudinal survey in which data are collected at multiple time points but from different subjects at each time point

1) C 2) A 3) D 4) B

A survey based on a high-quality random sample can provide accurate estimates of the characteristics, behaviors, and attitudes of an entire population, even if the survey sample includes only ____ people.

1,000 to 2,000 *When a proper random sample is collected, surveys can have excellent external validity or generalizability.*

Alejandro just gained access to a new nationally representative survey of retired people in the United States. He is unsure whether certain questions were asked in the survey and the raw data is a bit confusing and brief. What should he consult to examine this issue?

A codebook *A codebook lists all of the questions used in a survey along with their response categories. In some cases, the codebook also provides the number and/or proportion of persons in each response category.*

Mary decides to conduct research in which she selects a sample of college students and sends them an email link to answer questions about their background, grades, and sleeping habits. Mary is using which type of research method?

Survey *Surveys provide an exciting opportunity to answer new and unexplored sociological questions. Political pollsters, market research firms, federal statistics organizations, health-care systems, and academic researchers alike use surveys to obtain detailed information on nearly all aspects of social and political life. We can learn about buying habits, political and social attitudes and identities, health behaviors, sexual activity, favorite television shows, and crime victimization, from surveys. Surveys are one of the most widely used research methods in sociology, accounting for roughly one-third of all published studies.*

What is one of the major limitations of telephone surveys?

Telephone surveys are more likely to have sample bias due to who chooses to answer the phone *Many people do not want to be bothered in their homes. The explosion of telemarketing and "robocalls" poses a considerable threat to legitimate survey organizations. Many people see an unfamiliar incoming number and choose not to answer. If those who choose to answer their phones differ significantly from those who do not, sample bias will result. Older adults, those with more spare time, those with a more trusting nature, or those who are lonely are more likely to answer a call from an unfamiliar phone number.*

Match each item with the correct definition. 1) Index 2) Composite Measure 3) Scale 4) Ranking Items A) A type of closed-ended question that asks respondents to rank-order their priorities or preferences B) A measure that combines multiple items, whether as a scale or an index, to create a single value that captures a multifaceted concept C) A composite measure that sums responses to survey items capturing key elements of a particular concept being measured D) A composite measure that averages responses to a series of related items that capture a single concept or trait, such as depressive symptoms or self-esteem

1) C 2) B 3) D 4) A

Which of the following is an example of social desirability bias?

A respondent overreports the amount she has donated to charity *Social desirability bias is when respondents report socially desirable behaviors and attitudes in the survey setting. In this example, the respondent may be trying to impress an interviewer by overreporting the total amount she has donated to charity.*

Justin conducted a panel survey with 3,000 initial respondents two years ago. After designing a follow-up survey and sending it to the initial respondents, he finds that only 2,130 responded the second time. What problem has Justin experienced?

Attrition *The main limitation of panel studies is sample attrition, which occurs when people drop out of the study. Attrition tends not to occur at random, so it may bias study results. Because the people who participate and continue in panel studies tend to have more education and better health than those who do not, panel studies run the risk of providing an overly positive view of their sample members' lives.*

Which of the following is an advantage of panel surveys over cross-sectional surveys?

Causal inference *The main advantage of panel surveys is that researchers can determine cause and effect better than with cross-sectional surveys. In cross-sectional surveys, all measures are obtained at a single point in time, so we can document associations but not causation. However, in panel surveys, measures at one wave can be used to predict outcomes at subsequent waves.*

Before starting data collection with a new survey, the most important step is ____.

The pretest *A pretest is a trial run in which researchers administer the survey to people who are similar to those in the target sample. The pretest identifies problems with the survey, including confusing questions, problematic skip patterns, or questions that might turn off potential respondents. The pretest also provides information on the survey's duration and pacing; it may reveal parts of the questionnaire that feel long and tedious to respondents.*

It is unrealistic and impractical for survey researchers to promise ____ to respondents.

Anonymity *Anonymity means that no identifying information can be linked to respondents or their survey answers. Anonymity may not be desirable from the researcher's perspective: If a respondent's survey record has no identifying information, then researchers cannot conduct follow-up interviews as part of a panel survey or as part of a mixed-mode approach. For these reasons, it is much more realistic for survey researchers to promise their respondents confidentiality rather than anonymity. It is exceedingly rare today that a survey respondent's identity is betrayed, given the many rules and constraints in place.*

If a question has a dichotomous outcome as the possible response, it must be which type of question?

Closed-ended *The most basic closed-ended question provides a dichotomous outcome category, meaning that only two options are available, such as "yes" or "no."*

Which ethical concern should be addressed at the beginning of a survey, often alongside an introduction to the survey?

Informed consent *Regardless of the mode you choose, the initial contact with the respondent should briefly introduce the survey's purpose and convey respect and gratitude to the respondent. The survey researcher will also obtain informed consent from the respondent at this point, informing the respondent of the potential benefits and risks of participation. The researcher also informs respondents that they are free to skip any questions or cease participation if they wish.*

A survey presents respondents with the following question: "Do you think the government should raise income taxes and use it to pay for universal health care costs?" Which problem does this question suffer from?

It is a double-barreled question *A double-barreled question is one that asks about two or more ideas or concepts in a single question. In this question, the respondent might think that the government should raise income taxes for some other reason, or the respondent might think that the government should institute universal health care without raising income taxes.*

Which group of respondents is susceptible to acquiescence bias?

Less educated people *Some survey research experts believe that agree-disagree categories are subject to acquiescence bias, in which some respondents answer "agree" to attitudinal questions. Some researchers have found that respondents who have lower levels of education and who are less informed on a particular topic are particularly susceptible to acquiescence bias and are more likely to agree with any statement read to them. As a result, some researchers encourage the use of forced-choice questions instead of agree-disagree type questions.*

What is one of the major strengths of face-to-face interviews?

The presence of a skilled interviewer helps ensure that respondents understand the survey questions *Face-to-face interviews are considered the "gold standard" of survey research because the presence of a skilled interviewer helps ensure that respondents understand the survey questions and do not skip any questions that are sensitive or troubling. Interviewers can also help to clarify questions and response categories.*

Which of the following is an example of a split-ballot design?

A survey in which half of the respondents answer questions about one topic and half of the respondents answer questions about a different topic *For instance, a randomly chosen subset of 1,500 GSS participants (50% of the 3,000 respondents) may be asked a series of questions about their attitudes toward physician-assisted suicide, while the other 1,500 are asked about their attitudes toward gun control. This design cuts down on total survey time by sparing respondents from answering both modules yet also provides researchers with information on two important and controversial issues.*

Identify the following question type: "What are the goals you would like to accomplish in college?"

Open-ended *An open-ended question is a broad interview question to which subjects are allowed to respond in their own words.*

Generalizability, or high external validity, requires which of the following?

A random sample *Surveys based on random samples have excellent external validity. As you learned in Chapter 6, some samples are randomly selected from well-defined populations, such as all high school students in the United States. Others are convenience samples, such as fellow students in your introduction to sociology class. The results from studying the former could be generalized to characterize the experiences of all U.S. high school students, while the results of studying the latter would likely be representative only of sociology students at your university.*

If a researcher uses U.S. Census data to examine changes in the number of individuals in each racial group across states over time, she is using ____.

A secondary data source *The U.S. Census has already been collected by someone else, thus it is a secondary data source.*

If researchers are interested in the question of "why," which type of question should they use?

Open-ended *If researchers are interested in the question of "why," such as why an individual might be opposed to medical marijuana, they should use an open-ended question. Open-ended questions also are useful if the respondent is unfamiliar with the topic, doesn't have an opinion, or simply hasn't thought about the topic.*

If an interviewer records information about the environment in which an interview is conducted, it is a type of ____.

Paradata *Paradata is information about the process by which the survey data were collected. Data analysts may find these observations useful when trying to understand the ways that people answer survey questions.*

Which of the following is not a strength of survey research?

Response rates *One of the most serious challenges in survey research is achieving a high response rate. Response rates may range from 20% to 80% depending on the survey and mode of administration.*

A survey presents respondents with the following question: "In the past 24 hours, how many text messages have you sent? (circle one) None; 1 to 2 texts; 3 to 5 texts; 6 to 10 texts" Which problem does this question suffer from?

It is not exhaustive *In order for a response set to be exhaustive, respondents must have at least one accurate response available to them. In this case, if you sent more than 10 texts in the past 24 hours, no possible response is accurate.*