survey

advantages of survey

(What is wrong about survey? - easy to show causality) 1. Gather large amount of data i. Describe large populations ii. Good external validity- Generalizability 2. Ask numerous questions i. Opportunity for multivariate analysis ii. Can measure same variable in many ways iii. Best way for getting multiple items for a given concept- Increase reliability 3. Formatted questions/answers i. Encourages standardization and systematic comparison across respondents ii. Relatively easier to administer

interview surveys

1. Advantages i. Interviewer observations ii. Personal contact increases answer rates iii. More difficult for respondent to terminate 2. Disadvantages i. Expensive ii. Time-consuming iii. Interview may "interact" iv. Training is necessary

group-administered surveys

1. Advantages i. Many people take surveys in same location at same time ii. Often used in media (e.g. movie pre-screenings) iii. High response rates 2. Disadvantages i. Respondent interaction ii. Lack of anonymity iii. Potential cost of space

internet surveys

1. Advantages i. Recruitment can happen by mail, phone, or email ii. Survey can be emailed or respondents can be directed to URL iii. Typically, quick and inexpensive 2. Disadvantages i. No way to know recruited ii. Participant is filling out survey iii. Internet-based distractions

Mail surveys

1. Advantages i. Specialized mailing lists- yields targeted information ii. Participant anonymity iii. Eliminates interviewer bias iv. Low cost v. Respondent can answer on their own time 2. Disadvantages i. Must be self-explanatory ii. Low response rates iii. Impossible to know who participates iv. Slow form of data collection

Avoid

1. Avoid double-barreled questions: Two or more questions in the same sentence i. Example→ Do you think students should have more classes in math and history in high school? 2. Avoid biased words/ jargons/acronyms: 3. Example→ Should states regulate PETA and ALF? 4. Avoid leading questions: Suggests a certain response i. Example→ What do you think about the harmful effects of global warming? 5. Avoid sensitive questions, if it is not central to your research i. Example→ annual income, sex life, drug use, religion etc.

Trend

1. Different individuals are asked same question over time 2. Example→ How much did newspaper readership change in the 1980s, 1990s, and 2000s?

Cohort

1. Different individuals belonging to a specific group [sharing some common life event- i.e. Baby boomers, Class of 2015] asked the same question over time. 2. Example→ Examining job placement success of PSU graduates by the year they graduated. 3. Still random sample of people but have to qualify on same level

survey design checklist

1. Easy items at beginning. More difficult in middle. Sensitive at the end. i. Give respondents a chance to become comfortable with the questionnaire ii. Put demographic questions last. 2. Cluster items together that belong together. 3. Logical order of particular items. 4. Easy to administer. 5. Neat and clutter free design 6. Pretest to detect flaws.

disadvantages of survey

1. Hard to show causality- can only establish correlation 2. Hard to manipulate IV 3. Rely on self-report 4. Hard for respondents to accurately recall information 5. Subjects may misreport information 6. Difficult for survey answers to put individuals in "context"- often don't get at "'why" 7. Effective only for topics that are backed by significant theory

Question wording in survey

1. Phrasing of questions is critical i. Make questions clear ii. Use direct and simple language iii. Keep questions short and simple iv. Focus on the issue original researcher wants to study

Panel

1. Same individuals asked same question over time 2. Example→ What was the best music channel followed by a group of people, when they were 20-years, 40-years, 60-years old? 3. Remember, it's the same sample (people), over a 40-year time frame

Criteria for developing answer choices

1. The list of choices should be mutually exclusive i. Ex: What is your age? (both have 35 year old) a. 25yrs to 35yrs b. 35yrs to 45yrs c. 45yrs to 55yrs ii. You're always better off using a ratio level variable: a. What is your age?________ 2. The list of choices should be exhaustive i. Include all possible responses- Include the other option 3. Only one response option per question for each respondent. 4. The list of choices should be as detailed as possible. i. Response categories (e.g., age, income) should be as narrow as possible

response rates

1. The proportion of people who respond: i. # that answered/# you contacted= response rate 2. Example→ If you distribute 50 questionnaires and you get 25 questionnaires back, your response rate is 50%. 3. Why is a high return important? i. It's the only way to know if your results are representative. ii. Decreases the likelihood of biased results iii. Promotes confidence in results.

when to conduct a survey

1. You want to collect information from individuals [vs. a group or collective] 2. You want standardized information from everyone 3. Privacy is important or independent opinions and responses are needed 4. When external validity [i.e. generalizability] is important 5. When more concerned with attitudes than behaviors 6. When causality is less important- when IV is hard to manipulate 7. Conducted for the purpose of description- what's going on 8. One of the most important ways in quantitative research

Open ended questions

Respondent gets feeling that she/he has more control. ii. (+) Researchers can learn nuances of response- more in-depth answers. iii. (−) Answers must be coded in the end. iv. (−) Time-consuming to ask. a. Example→ What's it like to go through the college decision-making process in the 21st century? b. Example→ Think back to yesterday. What topics did you discuss with your children? c. A lot of details and insights into the answer. Know the whys and how's. Time-consuming.

survey

a form of quantitative research does not give you causality 2. A structured way to collect standardized information from individuals using a questionnaire. 3. Survey research is asking questions about: i. Attitudes, Opinions, beliefs, intentions, characteristics

Close ended questions

i. (+) Time efficient- responses easy to code and interpret ii. (+) Standardize the response iii. (−) Respondent can only choose from a list of limited available answers iv. (−) Researcher may miss possible answers, can't further explore them. a. Examples→Did you travel outside United States in the last 21 days? 1. Yes 2. No b. Likert: I believe the IBM workforce is well trained 1. Don't Agree 1 2 3 4 5 6 7 Agree

Longitudinal survey

i. Gather data over a period of time- multiple observations ii. Better for testing causal process-can infer time order. iii. Allow for more in-depth understanding of behavior iv. Three types: Trend, panel and cohort studies

Interview surveys

i. In person administration a. Interviewers ask questions and record respondent answers b. Highly structured (sometimes computer-assisted personal interviewing) ii. Telephone surveys a. Interviewers ask questions by phone b. More control: Questions read verbatim, no commenting

increase response rates

i. Provide incentives ii. Make the topic salient - seem important a. Ensure that respondents see the value of participating. b. Point out personal connection to the topic iii. Appeal to people's helping tendencies - ask them to help. iv. Pre-notification/follow-up letters v. Make the questionnaire interesting-short and easy to complete AND easy to return

Self-administered questionnaires

i. Respondents read and answer surveys independently a. Mail surveys (Nielsen) b. Internet (Qualtrics) c. Group-administered (in-class)

Cross-sectional survey

i. Used to gather information on a population at a single point in time ii. Does not offer time order. iii. Describes current conditions a. Example: U.S. Census, polling, audience readership is iv. You can't infer more than a relationship (correlational data). No causation