Social Research Methods - Chapter 9 (Survey Research)

strengths and weaknesses of survey research

strengths: - economic - amount of data collected - useful in describing characteristics of large populations - make large samples feasible (especially self-administered) - you can develop operational definitions from observations (compared to the structure laboratory experiments) - measurement (uniformity in questions and responses) - reliability - standardization of data weaknesses: - the requirement of standardization results in difficulty assessing the complexity of people (attitudes, orientations, circumstances, and experiences) - can appear superficial in covering complex topics - can seldom deal with the context of social life in a natural setting - inflexible initial study design - cannot measure social action (only collect self-reports) - validity

topics appropriate for survey research

- descriptive (especially), exploratory, and explanatory - usually individuals as unit of analysis but can be employed for other units as well - large populations - attitudes and orientations (by asking questions or asking if they agree/disagree with statements expressing different points of view)

ethics and survey research

- maintaining confidentiality - possibility of psychological harm to respondents

guidelines for survey interviewing

1. appearance and demeanor (dress appropriately and act professionally) 2. familiarity with the questionnaire 3. following question wording exactly (could effect quality of data) 4. record responses exactly (no summaries or corrections) 5. probing for responses (when necessary)

guidelines for asking questions

1. choose appropriate question forms for the topic 2. make items clear 3. avoid double-barreled questions (only about one thing) 4. respondents must be competent to answer 5. respondents must be willing to answer 6. questions should be relevant 7. short items are best 8. avoid negative/double negative items 9. avoid biased items and terms 10. avoid ambiguous questions 11. categories should be mutually exclusive and exhaustive

questionnaire construction

1. general questionnaire format: uncluttered layout with clear questions 2. formats for respondents: check boxes, circling answers, etc 3. contingency questions: intended for only some respondents, determined by their response to some previous question 4. matrix questions: several questions with the same set of answers (strongly agree, disagree, etc) 5. ordering items in a questionnaire 6. questionnaire instructions 7. pretesting the questionnaire 8. a composite illustration

types of surveys

1. self-administered questionnaires 2. interview survey 3. telephone survey 4. online survey 5. mixed mode survey

interview surveys

a data collection encounter in which an interviewer asks questions to a respondents (face-to-face or via phone) - interviewers must be neutral in appearance and actions - interviewers must be familiar with the questionnaire (follow wording and order exactly) - can use probing to elicit more elaborate responses *advantages: - more control - respondents are more committed - more detailed responses - fewer misunderstood or incomplete questions - higher response rate (80-85%) - more effective for complicated issues (contingency questions) - greater flexibility in sampling and special observations (compared to self-administered) disadvantages: - role of researcher can affect respondents - respondents are reluctant to report controversial or deviant attitudes/behaviors

computer-assisted telephone interviewing (CATI)

a data collection technique in which a telephone-survey questionnaire is stored in a computer, permitting the interviewer to read the questions from the monitor and enter the answers on the computer keyboard - very promising

questionnaire

a document containing questions and other types of items designed to solicit information appropriate for analysis - the format of a questionnaire can influence the quality of data collected. - a clear format for contingency questions is necessary to ensure that the respondents answer all the questions intended for them. - the matrix question is an efficient format for presenting several items sharing the same response categories. - the order of items in a questionnaire can influence the responses given. - clear instructions are important for getting appropriate responses in a questionnaire. - should be pretested before being administered to the study sample. *mostly used in survey research but also in experiments, field research, and other modes of observation

secondary analysis

a form of research in which the data collected and processed by one researcher are reanalyzed (often for a different purpose by another) - this is especially appropriate in the case of survey data - data archives are repositories (libraries) for the storage and distribution of data for secondary analysis - ex. the General Social Survey (GSS) *advantages: - cheaper and faster than doing original surveys - enhanced the possibility of meta-analysis (researcher brings together a body of past research on a particular topic) - finding answers to questions about social life disadvantages: - validity (if data is appropriate for your particular research interest) - measuring variables *ex. examining the relationship between religion and abortion *ex. early childhood longitudinal study (kindergarten class of 1998-99)

respondent

a person who provides data for analysis by responding to a survey questionnaire

probe

a technique employed in interviewing to solicit a more complete answer to a question - a non direct phrase/question used to encourage a respondent to elaborate on an answer *ex. "anything more?" and "what do you mean by that?"

the role of the survey interviewer

advantages: - serve as a guard against confusion - minimizes "I don't know" answers - observe respondents while completing survey (discomfort, eye contact, etc) disadvantages: - gender, race, age, (etc) can effect results by influencing respondents to feel the need to answer a certain way - bias

online surveys

an increasingly popular method of survey research that involves the use of the internet - similar strengths and weaknesses to mail surveys - online devices (not just computers but smartphones too) - electronic instrument design - use with caution (respondents may not be representative of the intended population) *advantages: - much cheaper (than mail) - high response rate - use of email (to encourage participation) - mass scale - quick - appropriate for certain targeted groups disadvantages: - formatting can be complicated - don't know who is actually responding - difficult getting representative samples

robo-polls

computer-executed phone surveys which involve no human interviewers

mixed-mode surveys

different survey research techniques, combined by a review of existing data and in-depth field observations and interviews - researchers sometimes combine face-to-face, mail, and telephone surveys - online surveys has increased attention to the potential of combining survey techniques > use address-based sampling (basis for mail survey) > invites recipients to respond online if they want

telephone surveys

questionnaires or interviews over the telephone *advantages - cheaper - quicker - can dress however - sometimes respondents are more honest (than face-to-face) - probing is a possibility - allow greater control over data collection - safety/less risks (compared to face-to-face) disadvantages: - limited to people who have phones - interviewers cannot see respondents - interviewers name can have an impact - hampered by the proliferation of bogus "surveys" - people can hang up

open-ended questions

questions for which the respondent is asked to provide his or her own answers - in-depth, qualitative interviewing relies almost exclusively on these

random-digit dialing (RDD)

sampling technique in which random numbers are selected from within the range of numbers assigned to active telephones

close-ended questions

survey questions in which the respondent is asked to select an answer from among a list provided by the researcher - popular in survey research because they provide a greater uniformity of responses and are more easily processed than open-ended questions

survey

survey research is a frequently used mode of observation in the social sciences - a typical survey: the researcher selects a sample of respondents and administers a standardized questionnaire to them - designed to produce data appropriate for quantitative (statistical) analysis *ex. provide data from which to calculate the percentage unemployed in a population, mean incomes, and so forth.

self-administered questionnaires

surveys in which respondents are asked to complete the questionnaires by themselves (usually distributed by mail) - mail distribution and return - monitoring returns - follow-up mailings - response rates: ideal = higher than 70% (the number of participants in a survey / the number selected in the sample) - compensation for respondents: incentive (money, coupons, etc) - a case study *advantages: - cheap - can send to large populations - respondents feel more comfortable being honest (anonymity and privacy) - more effective for sensitive issues - no interview bias (compared to interviews) disadvantages: - response rate - who actually answer the questionnaire - mail surveys typically require a small staff (lots of work) - ethics of compensation

response rate

the number of people participating in a survey divided by the number selected in the sample, in the form of a percentage - also called the completion rate - self-administered surveys: the return rate = the percentage of questionnaires sent out that are returned