Quiz 2: Survey Research

Limitations to Group-administered Surveys

Most populations cannot be sampled in a group setting -Respondents may feel coerced and as a result will be less likely to answer questions honestly

Reliability Measures

Special statistics that help determine whether responses are consistent

Rules for Writing Survey Items: Minimize the Use of Negative Wording

The use of words that negates or indicates the opposite of what was otherwise described

Advantages to Phone Surveys

They can save time and reduce interviewer bias

Rules for Writing Survey Items: Label or Anchor the Rating Scale Points

To clearly indicate what a rating scale means we can use anchors

Questionnaire Development & Assessment: Maintain Focus

A clear conception of the research problem and the population to be sampled is required

Likert Scale

A numeric response scale used to indicate a participant's rating or level of agreement with a question or statement

Restricted Items

A question or statement in a survey that includes a restricted number of answer options to which participants must respond

Versatility

Ability to adapt or be adapted to many different functions or activities -Can be used to enhance understanding of almost any social issue

Anchors

Adjectives given to describe the end points of a rating scale to give the scale greater meaning

Open-ended Items

Allows the respondent to give any response in his or her own words, without restriction -Qualitative research design

Combining Questions into Indices

An index or scale is when several questions are used to measure one concept

Open-ended Items: Researcher Develop Ways to Code Participant Responses

Anticipate and list all possible examples of potential responses in terms of how participants might write or express their responses -Use multiple raters and additional statistical analyses to make sure the coding is accurate

Generalizability

Can be used to develop a representative picture of the attitudes and characteristics of a large population

Group-administered Surveys

Completed by individual respondents assembled in a group

Questionnaire Development & Assessment: Language

Consider translation of surveys to other languages -Differences in language may shape responses

Partially Open-ended Items: Last Open-ended Option

Could be coded further or just analyzed without further coding

Efficiency

Data can be collected from many people at relatively low cost and relatively quickly

Protection of Respondents

Disclose any harmful effect of participation

Rules for Writing Survey Items: Use Neutral or Unbiased Language

Do not use language that is offensive -Responses may be in reaction to the language used

Focus Groups

Guided discussions among potential respondents

Limitations of Electronic Surveys

Hard to get a true representative sample of the U.S. population; households without internet tend to be older, poorer, and less educated than do those that are connected; Much lower rates of survey completion

Advantages to In-person Surveys: Face-to-face Interviews

High response rate and completion rate

Limitation of Mailed/Self-administered Surveys

Higher rates of respondents choosing not to complete and return the survey

Confidentiality

If survey measures attitudes or behaviors that are socially stigmatized, the respondents' rights need to be protected; Only include questions that are important to research; Limit access to information

Phone Surveys

Interviewers questions respondents over the phone and record answers; can in interpersonal or automated -Recommended length of time is between 30-45 minutes

Demographic Questions

Important independent variables necessary for research -Placed at beginning or end of the survey

Partially Open-ended Items

Includes a few restricted answer options and then a last one that allows participants to respond in their own words in case the few restricted options do not fit with the answer they want to give -Researchers can code each answer option as a number

Advantages of Electronic Surveys

Inexpensive and little time commitment from researcher - easy to make online surveys

Reverse Coded Item

Item that is phrased in the semantically opposite direction of most other items in a survey, and is scored by coding or entering responses for the item in reverse order from how they are listed

Rules for Writing Survey Items: Avoid Double-barreled Items

Items that ask participants for one response to two different questions or statements

Rules for Writing Survey Items: Limit the Points on a Rating Scale

Keep the scale between 3 and 10 points

Mailed/Self-administered Surveys

Mailing a questionnaire to respondents who then administer the survey themselves

Maximize Response Rate of Mailed/Self-administered Surveys

Make the questionnaire attractive, make questions clear, minimize open-ended questions, offer incentives, include ways to result the survey, and use follow-up mailings

Questionnaire/Self-report/Survey

Many surveys specifically include questions in which participants report about themselves - their attitudes, opinions, beliefs, activities, emotions, etc.

Pretest the Survey

Obtain feedback on survey by asking others to review it; Administer to small sample; Review responses to each question; Discuss the survey's content with knowledgeable parties; Conduct cognitive interviews

Idiosyncratic Variation

Occurs when individuals' responses vary because of their reactions to particular words or ideas in the question -Single questions prone to this

Anonymity

Only if no identifying information about respondents is obtained is true anonymity provided; Makes follow-up attempts impossible; Allow anonymous responses to a subset of questions

Dichotomous Scales

Only two responses are possible

Response Rate

Percentage of persons surveyed who complete survey

Respondent/Participant

Person who answer questions on a survey

Limitations to Phone Surveys

Reaching sampling units and few people willingly agreeing to complete the survey

Advantage to Mailed/Self-administered Surveys

Relatively inexpensive and requires less time on the part of the researcher

In-person Surveys: Face-to-face Interviews

Researcher can control how long it takes to complete the survey but requires the interviewer to be present for each interview and can be prone to interviewer bias

Cognitive Interviews

Researchers ask test questions, then probe with follow-up questions to learn how questions were interpreted and the meanings of answers

Advantages to Group-administered Surveys

Response rates are typically very high

Bipolar Scales

Scales that have points above and below a zero point -ex: negative 5 to positive 5

Survey

Series of questions or statements, called items, used in a questionnaire or an interview to measure the self-reports or responses or respondents

Interview Schedule

Survey instrument containing the questions asked by the interviewer in an in-person or phone survey

Questionnaire

Survey instrument; The questions for a self-administered survey

Electronic Surveys

Survey is sent and administered by computer, either email or on the Web

Advantage of Restricted Items

Survey responses can be easily entered or coded for statistical analysis

Rules for Writing Survey Items: Avoid the Response Set Pitfalls

Tendency for participants to respond the same way to all items in a survey when the direction of ratings is the same for all items in the survey

Limitations to In-person Surveys: Face-to-face Interviews

Tendency for the demeanor, words, or expressions of a researcher to influence the responses of a participant when the researcher and the participants are in direct contact

Limitation of Restricted Items

The analyses is restricted to the finite number of options provided to participants

Item Response Rate

The percentage of items that are completed on a survey

Cautions of Combining Questions into Indices

The presupposition that each component question is indeed measuring the same concept may be mistaken; Combining responses to specific questions can obscure important differences in meaning among the questions; Questions may cluster together in subsets; Particular questions weighted more than others in the calculation of the index

Questionnaire Development & Assessment: Build on Existing Instruments

Use already designed survey questions to measure key concepts in present study

Rules for Writing Survey Items: Use Rating Scales Consistently

Use only one rating scale at a time -Use only one scale if possible

Rules for Writing Survey Items: Keep it Simple

Use simple words or language in a survey

Use of Survey

Used to quantify, describe or characterize an individual or group

Rules for Writing Survey Items: Minimize item and survey length

Write the survey to be as short and concise as possible, yet still able to convey what is intended to measure -Key goal is to minimize how long it takes, which does not always mean that the number or items in the survey must be small