Chapter 8 (Survey Research)

Compared to other data collection methods, such as interviews, surveys excel in all but which of the following ways? a) they are cost effective, b) they are versatile, c) results are generalizable, d) data collected is greatly detailed

D. Data collected is greatly detailed

Pros and cons of in-person interviews

Pros: -Preferable when questions are long & complex -Ability to build rapport (increases response rate) **Strongest design when sufficient resources & trained interview staff are available Cons: -Presence of interviewer may make it hard to give honest answers

True or False, It is not necessary to pretest a questionnaire if it will be an electronic survey.

False.

True or false, A contingent question is one that is asked to all survey respondents.

False.

Fence sitter

Survey respondents who see themselves as neutral in their attitudes toward a particular issue. --Remedy: either provide a neutral/undecided answer or provide strict answer choices

Electronic surveys

Survey that is sent and answered by computer, either through e-mail or on the web.

Questionnaire

The survey instrument containing the questions for a self-administered survey.

True or false, Reliability measures help researchers decide whether responses are consistent.

True.

Survey Research

Research involves the collection of information from a sample of individuals through their responses to questions. **Most popular methods for science research

Floaters

Respondents who choose a substantive answer even when they do not know anything about a particular question --Remedy: provide "I don't know" as an option or provide details of the issue you are interested in

Exhaustive responses

Response choices on a survey in which every case can be classified as having one attribute. **May have to uncle an "other, please specify" category

Group-administered surveys

Similar to mailed surveys, except they require the unusual circumstance of having access to the sample in a group setting

Why should questions sometimes be combined into indexes?

Single questions are prone to idiosyncratic variation --When using an index, the idiosyncratic variation in response to single questions will average out, so the main influence on the combined measure will be the concept focused on by the questions. --Additionally, the index can be considered a more complete measure of the concept than can any one of the component questions.

Phone survey

Survey in which interviewers question respondents over the phone & then record their answers.

Interview schedule

Survey instrument containing the questions asked by the interviewer for an in-person interview of phone survey.

Mailed (self-administered) survey

Survey involving a mailed questionnaire to be completed by the respondent.

A floater in a survey is defined as someone who:

Gives a substantive answer to a question when he or she really doesn't know

True or false, Surveys are popular primarily due to their versatility.

True.

Mutually exclusive responses

Response choices on a survey that do not overlap.

Splitt-ball design

Allows the inclusion of more questions w/o increasing the survey's cost; process involves unique questions or other modifications in a survey being administered to randomly selected subsets of the total survey sample

What are the strengths of close-ended questions?

Easy to process & analyze with computers & statistical software

Close-ended (fixed choice) questions

Respondents are provided with explicit responses from which to select

Reference period

A time frame in which a survey question asks respondents to place a particular behavior (e.g., in the last 6 months). **Should not be too short or too long to avoid underreporting

Mixed-model surveys

Surveys that are conducted by more than 1 method, allowing the strengths of one survey design to compensate for the weaknesses of another. e.g., mailed survey may be interviewed in person or over the phone

In-person interviews

Survey in which interviewer questions respondents in person and records their answers.

Why might it be better to use a scale/index vs. single multiple questions?

-Single questions are prone to idiosyncratic variation -Single question is not as good at measuring a variable because respondent may have had trouble with particular word or phrase -When measuring concepts, best option is to devise an index of multiple rather than single questions

What are some methods of improving survey questions & mistakes to avoid when writing questions?

1. Construct clear & meaningful questions that are specific about what is being asked of respondents 2. Avoid confusing phrasing & vagueness -no long or wordy questions -avoid long reference periods 3. Avoid negatives and double negatives 4. Avoid double-barreled questions 5. Avoid making either disagreement or agreement disagreeable 6. Use Likert-type responses 7. Response choices should be mutually exclusive & exhaustive 8. Minimize fence sitting and floating 9. Utilize filter questions for only relevant respondents

What are some survey designs?

1. Mailed, self-administered surveys 2. Group-Administered Surveys 3. Telephone Surveys 4. In-Person Surveys 5. Electronic Surveys 6. Mixed-Model Surveys

What are some things to keep in mind when designing a questionnaire?

1. Maintain focus: well-defined inquiry & target population 2. Built on existing instruments 3. May need to be translated into various languages

What are the 3 attractive features of survey research?

1. Versatility: researchers can ask questions on almost any topic 2. Efficiency: can be collected by many people at low cost relatively quickly 3. Generalizability: develops representative picture of the attitudes & characteristics of a large population

When one or more questions influence how subsequent questions are interpreted, this is known as: a) contextual affect, b) public opinion effect, c) demographic effect, d) question-order effect

A. Contextual effect

Which survey method requires special attention to all of the following problems: need for multiple contact efforts, impersonality of the contact, lack of visual aids, possibility of frequent distractions, and relatively low rapport with the respondent? a) phone survey, b) structured in-person interview, c) unstructured in-person interview, d) mailed questionnaire

A. Phone survey

Which of the following is not a guideline for writing individual questions? a) use idiosyncratic questions, b) use mutually exclusive response categories, c) avoid double barreled questions, d) yes exhaustive response categories

A. Use idiosyncratic questions

Filter questions are useful because: a) they increase rapport b/w respondent and interviewer, b)they can reduce length of interview, c) untrustworthy respondents can be quickly eliminated, d) ambiguous questions can be clarified

B. They can reduce length of interview

A fence-sitter in a survey is defined as someone who: a. Has conflicting opinions on most issues, whether or not they are addressed in the survey b. Gives a substantive answer to a question when he or she really does not know c. Delays giving an answer to controversial questions when interviewed in person d. Gives a neutral response to a question even when they have an opinion

D. Gives a neutral response to a question even when they have an opinion

I want to measure the prevalence of student drinking on campus. I ask them to describe how they would characterize their alcohol consumption using the following response categories: 1) Not at all, 2) Once in a while, 3) Occasionally, 4) Socially, 5) All the time. The problem with this response format is that the categories are not __________________. A) Filtered B) Weighted C) Exhaustive D) Mutually Exclusive

D. Mutually exclusive; what is the difference between "once in a while", "occasionally", "socially"?

Electronic surveys suffer from all of the following issues, except: a) about one quarter of all U.S. households are not connected to the internet, b) low rates of survey completion c) difficult to get representative sample of US population d) difficult to ask about sensitive issues

D. it is difficult to ask about sensitive issues

Open-ended questions

Survey questions to which the respondent replies in his or her own words, either by writing or by talking **Useful for explorative purposes e.g., "In your opinion, what have been the benefits of the community policing program in your neighborhood?"

Suppose I ask the following question on a survey: What is your religious affiliation? -I don't have one -Christian -Jewish -Muslim -Hindu True of False: The response options for this question are exhaustive.

False; there are other religions

Dr. Brown conducts a survey to gather public opinions about the death penalty. His survey consists of many statements with Likert-type responses. Dr. Brown made the middle of his Likert scale a neutral response, "neither agree nor disagree." After collecting his survey responses, he notices that there are a group of individuals who selected the "neither agree nor disagree" responses on a majority of the survey questions. This group who is neutral in their attitudes toward this particular issue is known as:

Fence-sitters

Filter question

Filter questions create skip patterns for people to say no and get directed to other questions. Contingent questions are those after a person says yes.

Cognitive Interviews

Involves asking ppl test questions and then probing them w/ follow-up questions to learn how they understood the questions and what their answers mean. **May be used to evaluate questions

Pros and cons of phone surveys

Pros -interviews can be carefully monitored Cons -limits length & complexity of questionnaire -visual aids cannot be used -ability to build rapport is lower -not reaching proper sampling unit (e.g., teens mostly have cellphones not landlines) & not getting enough complete responses may make results less generalizable (response rates low for ppl 24 and younger, cutting call, etc.) Remedy: -computer-assisted telephone interview, but ppl may be put off by robotic voice

Omnibus Survey. How is it different from regular surveys?

Most surveys are directed at a specific research question. In contrast, this survey covers a range of topics of interest to different social scientists.

If response choices on a survey do not overlap, we can say that the responses are:

Mutually Exclusive

Pros and cons of group-administered surveys

Pro: high response rate Con: assembling a group/captive audience is difficult

Pros and cons of mailed surveys

Pros -Cheap, don't require training to administer, don't have to contract with a survey org. Cons -Nonresponse is an issue -Incomplete responses -least preferred survey design in sampling Remedies -Credible research sponsor -Follow-up mailings

Pros and cons of electronic surveys

Pros -easy, cheap, can reach many people Cons -many people lack internet question or adequate computer capacity for displaying complex web pages -1/4 of U.S. households are not connected to the Internet (tend to be older, poorer, and less educated)

Even though survey research typically poses fewer ethical dilemmas than experimental or field designs. What are some ethical issues in survey research?

Protecting Respondents -Option to decline to participate should be clearly stated in the cover letter (sent w/ mailed questionnaire) or introductory statement -Special care for when questionnaires are asked in a group setting & also for personal questions -Subject confidentiality is a MUST -Can use anonymity, but this precludes follow-up surveys and panel designs

What are the strengths of open-ended questions?

Provide wealth of info, can obtain respondents' interpretations in greater detail & can illuminate flaws in other questions.

Double-negative question

Question of statement that contains 2 negatives, which can muddy the meaning. e.g., Do you disagree that juveniles should not be tried as adults if they commit murder?"

Interpretive questions

Questions included in a questionnaire or interview schedule to help explain answers to other important questions. Especially useful when the questions in a survey have not been pretested.

Demographic questions

Questions on a questionnaire that concern demographic information such as sex, age, race, ethnicity. These questions are important independent variables. **Should be mutually exclusive & exhaustive

A researcher uses the question, "In the last month, how many days did you carry a method of self-protection such as pepper spray?" This is a good, specific question because the researcher gives a specific time frame (i.e., in the last month). This time frame is known as a:

Reference period

True or false, Phone surveys are likely to have low contamination rates and high control over sample selection.

True, thanks to random digit dialing (simple random sampling technique)

Idiosyncratic variation

Variation in responses to questions that is caused by individuals' reactions to particular words of ideas in the question instead of by variation in the concept that the question is intended to measure.

Double-barreled questions

When a question is really asking 2 questions, but only contains one answer. e.g.,, "Do you believe we should stop spending so much money building prisons and put it into building more schools?"

How do you avoid making either disagreement or agreement disagreeable?

When focusing on illegal or socially disapproved behaviors or attitudes, you should write a question and response choices that make agreement seem acceptable or at least not stigmatizing. e.g., "Have you ever taken anything from a store without paying for it?" rather than "Have you ever shoplifted something from a store?" --Also ask about a variety of both acceptable & unacceptable behaviors.

Should questionnaires and interview schedules be pretested?

Yes; they should be pretested on a small sample that is like the sample to be surveyed.

Survey instrument

an existing set of questions; measurement tools have already been established for many concepts like juvenile delinquency, self control, etc.

Reliability measures

special statistics that help researchers decide whether responses are consistent