Chapter 7 - Survey Research

Which type of organization gets the highest response rates to surveys? Who gets second highest?

Highest: government agencies Second highest: universities

Why use a cover letter and give some examples of what to put in it.

A cover letter introduces your project to potential respondents and explains the importance and purpose of the research, which increases response rates. This allows for full disclosure, explains confidentiality/anonymity and adds motivation to respond. Examples of what to include: Study's Sponsor Address/phone/email to contact researcher How respondent was selected Who is the study sample Purpose of research Who will benefit Appeal for cooperation Completion time Payment Anonymity/confidentiality Deadline

True or False: There is no real significant difference between respondents and nonrespondents.

False.

What are some ways to decrease your refusals for an interview?

1) Blanket area with information. Put up fliers with phone number pull tabs for instance. 2) Consider the timing of contact. Don't call early in the morning or late at night, or people's home phones during most people's work hours 3) Communicate how long the interview will take

What are some considerations regarding instructions and question order when creating the survey design?

1) Directions should be precise and straightforward. 2) Early questions should not bias later ones, meaning that you should start out with the "easy" or "soft" questions and work your way up to the "hardball" ones. Questions should be ordered so that participants are interested in completing the measure. They shouldn't be too boring or hard at first.

How should a question be worded? (Five considerations)

1) In the present tense 2) simple and direct 3) consider the reading level of your participants when choosing diction 4) avoid slang. 5) Avoid vague wording such as "bad home life." Make sure your terms are explicit.

What are some strategies for accurately recording an interview?

1) Make an audio or video recording of the interview 2) get good at summarizing "the high points" of what was said 3) Control the interviewer's tone of voice, body language, inflection, word choice, etc. 4) Classify the responses into predetermined categories

What would be a pro of starting with demographic information and what would be a con?

1) Pro - putting it at the beginning provides an easy question that builds motivation. Also you ensure that the participant has enough energy to complete the demographic info because it is important, and that they don't accidentally skip over it in their excitement to finish the survey (if it had been at the end). 2) A con of putting it at the beginning might be that it bores some participants. This is less likely however.

What are three data formats for a focus group?

1) Raw data format 2) Descriptive approach 3) Interpretive model

Name some characteristics of open ended questions. (four)

1) Respondents develop their own response. Not predetermined choices. 2) Responses are not predictable 3) Handling the data is more complex because range of possibility is infinite 4) Respondent may give multiple answers to the same question.

Name three types of commonly used surveys

1) Telephone interviews 2) Online surveys 3) Focus groups

What are some ways to conduct a successful interview?

1) Think of the interview as a social relationship. Make the interviewee feel comfortable and at ease. Rapport is a must. 2) Start with simple questions 3) Use probes to elicit clearer responses but be aware of how your probes are phrased. Don't let the interviewee take probes as a challenge to what they've told you

Name and define three interview structures.

1) Unstandardized - interviewer develops his/her own questions on a general topic. Probes as the interview progresses. 2) Non-scheduled nonstandard - Topic is more narrow than unstandardized. Specific questions are asked of all interviewees. Interviewer can probe and rephrase. 3) Schedule-Standardized - Interviewer has specific instructions. Specific questions are given in a fixed order and there are specific transition phrases. Interviewer cannot rephrase or probe on questions.

Name some characteristics of closed ended questions. (Three)

1) has a fixed set of alternatives 2) all possible theoretically relevant choices determined in advance 3) data is easy to handle because the choices are predictable

Name two ways you can test the wording of your question.

1) use empirical tests to avoid bias 2) use a pretest

What is a filter question and give an example.

A filter question is one by which the respondent's response to it determines which question he/she will then go to on the survey. Example: If you are a woman, you may skip to question #26 from #1 which asks about your sex, to answer questions about ovarian health. If you are a man, you may proceed with #2 and answer questions about prostate health.

What is a focus group?

A group interview with several people at the same time led by one or more moderators who directs the discussion by a previously conceived set of topics.

When would you use an answer box and when would you use a blank?

Answer boxes are usually for closed ended questions and blanks are usually for open ended.

What percentages are considered indicators of good and bad response rates?

Anything above 60% is good, above 70% is excellent, and below 50% is very suspect.

Discuss why you would choose an open or closed ended question.

Closed ended questions are more quantitative and open ended are more qualitative. You have to decide which type would best help you collect the info you desire to gather. If you want to know "Why" they do something you will likely go with open ended. Open ended require better listening and analyzing skills because you have to "read between the lines" a lot. You would also choose open ended when it is impossible to come up with an exhaustive list of all the choices.

What are the questions that follow the filter questions called?

Contingency questions.

Which interview type is the most controlled? The least controlled?

Most: Schedule-standardized Least: Unstandardized

Name one pro and one con of a survey

Pro: Can collect data from large samples of people Con: Can only measure what people SAY they do, not necessarily what people ACTUALLY do (since it is self-report, people can lie, distort, exaggerate, misinterpret, etc)

Name some pros and cons of questionnaire data.

Pros: 1) Faster and cheaper than interviews 2) Able to reach geographically varied sample 3) Effective with sensitive topics because respondents don't have to talk to a live person about it 4) Eliminates interviewer bias Cons: 1) Requires some level of literacy 2) Written instructions - if they don't understand them, there is no way to rephrase or reiterate 3) Can't probe for more info on an answer 4) Can't control whether your target subject is actually the one that completes it 5) Responses might not be independent. Might have someone else in the room when doing it that influences their responses. 6) High response bias 7) possibility of losing data through low response rate.

What are the pros and cons of a focus group?

Pros: 1) very flexible 2) less expensive than single interviews 3) faster than single interviews Cons: 1) Less generalizable 2) Difficult to analyze results 3) Qualitative data rather than quantitative 4) Social desirability effect can be high

What are the pros and cons of an online survey?

Pros: - Can send easily, via email or from a website you visit - very inexpensive - rapid response - anonymity Cons: - non-representative samples (automatically biases toward people who have access to computer and the needed skills to use one) - need to use a compatible format across different computers and operating systems

What are the pros and cons of a telephone interview?

Pros: - inexpensive - quick to complete - reduces response bias (e.g., if done by a machine) - foreign language friendly (can build in an automated language option) Cons: - duration is limited - restricted to voice only communication (can exclude those with handicaps involving voice communication) - non-coverage areas (areas with no phone service or no access to personal phones)

Name the pros and cons of an interview survey technique.

Pros: -Flexible (most styles allow for some variation in technique to increase motivation). -The interviewer can explain questions or directions to the respondents and reply in real time to their questions. -The interviewer adds an element of control not present in questionnaires (for example, WHO you are interviewing is not in doubt). -Interviewer can also collect observational data Cons: - more costly than questionnaires (have to train and pay interviewers, need extra incentive for participants to come to you) - Takes more time - Interviewer bias effect - Meaning can vary between interviewers based on cultural effects, voice intonation, choice of words, etc.

Name two forms of survey research and define them.

Questionnaire - written questions on a form that people respond directly to without the aid of an interviewer. NOTE: Sometimes these can be asked by a person if the subject's cognitive or reading ability is insufficient, but if it was DESIGNED for them to fill out themselves, it is still a questionnaire. Interview - Involves an interviewer who asks the respondent questions and records their answers.

Which is considered more positivist: questionnaire or interview?

Questionnaire because it removes the interaction of the researcher from the subject.

What is a matrix question?

Response options to several questions which are presented in table format. Example: teacher evaluation that uses a likert scale for several items.

Why should we give special consideration to minority populations when conducting surveys? What type of cultural interactions can affect the data?

Since an interview is a social relationship, it is important when working with minority populations that they feel comfortable with the interviewer and that they are trust worthy. Social desirability effect can be activated by gender, racial, or culture differentials. Matching for these factors between interviewer and interviewee is common. The language can also have an impact if using one that is not the respondent's primary language. It is important to consider these factors because minorities may have responses that differ from the majority population which can be crucial to data sets. Your study should attempt to minimize the social distance as much as possible (e.g., by making sure the interviewer's style of dress is appropriate for the population).

Describe a situation in which using slang might be appropriate. Why is it generally discouraged?

The only exception of this is if you are using slang to make it more clear to your participants AND if you have the cultural competence necessary to do so. Do not assume that you have access to a culture until you are invited into it! The pitfalls of using slang is that slang changes over time, and there are multiple slang terms used for the same word depending on the region. If you use an outdated term or one that is not regionally accepted, you might confuse your participants or lose your credibility with them.

Define response rate.

The proportion of the sample that completes and returns the questionnaire or completes the interview.

True or False: A survey is the most widely used data collection method in social research.

True.

True or False: Closed ended questions must include all possible categories relevant to that question.

True. Closed ended must be mutually exclusive and exhaustive. One way to do this without listing all possible choices is by using the "magic word": other.

What is a way you can assure the anonymity of survey data?

Use a separate return of post card and data. The data does not have identifying info on it, while the post card simply has a number. If you get back the same amount of numbered postcards and a corresponding data sets, you know which respondents turned their data in but not who a particular set of data belongs to.

Give an example of how a closed or open ended question asking the same thing can have a different meaning/interpretation?

When asking about marital status, closed ended would ask are you: married, divorced, widowed, single, or separated. An open ended version would be: What is your marital status? This would allow for people to list long term committed relationships they are in that are the equivalence of marriage but does not put them into the category of "single" (such as cohabitation or same sex partners in states where same sex marriage is not allowed). By asking this question in closed ended format, you may make your participant feel that their relationship is not valid or recognized.

Define survey.

a data-collection technique in which information is gathered from respondents by having them respond to questions or statements