Chapter 7: Data Collection for Quantitative Research

Advantages/disadvantages of virtual data collection

- Advantages: Lots of participants can be recruited quickly, higher response rates - Disadvantages: Not everyone has access to the internet, people who feel more strongly are more likely to participate, cannot guarantee identities of respondents

Why isn't a classic experimental design always feasible

- Cannot always randomly assign participants to control and experimental groups - Only non-equivalent groups available - Lack of time - Lack of financial resources

Survey vs Questionnaire

- Survey: Method of data collection - Questionnaire: Actual set of questions

Steps to follow when creating questionnaires

1. Develop list of items/constructs to be measured in the survey 2. Split list into measurement items (easy to measure) and constructs (abstract/difficult to measure) 3. Create variables from measurement items 4. Turn constructs into variables (consult literature/create open-ended questions) 5. Craft good questions: clarity, simplicity, unbiased (don't lead participants to answer a certain way), intelligible (not too technical), avoid social desirability, avoid double-barreled questions (asking about more than one concept), avoid double negatives, contingency questions (have clear progression of questions when one question depends on the answer to an earlier question), avoid dangling alternatives (putting the options before the actual questions), avoid hypothetical questions (ambiguous) 6. Organize well (demographic questions at the end, group questions logically) 7. Create answer scale (eg: likert, agreement, etc) -> needs to measure same concept, consider odd/even no. of options, inclusive of all options 8. Conduct pilot study

Advantages/disadvantages of in-person data collection

Advantages: Body language/other info can be observed/recorded, more likely that participant will fully complete the questionnaire, easier to complete long questionnaire Disadvantages: Time-consuming, cost/time limitations (may have to hire assistants)

Quasi-experimental studies

Allow researchers to conduct some kind of intervention/experiment without satisfying all conditions of classic experimental studies

How is data collected in experimental design

Both a pre-test and a post-test (identical in nature)

Computer-Assisted Telephone Interviewing (CATI)

Computers select participants -> phone calls conducted by interviewers OR interviews conducted by computer - Interviewer must be familiar with study and trained to be neutral while interviewing to avoid leading participants - Interview protocol before (date/time/place/length of interview/confidentiality notification)

Experimental Research

Conducting an experiment on a randomly assigned group while controlling findings for a second group

Experimental designs are qualitative designs. (T/F)

False

Google Forms is not sophisticated enough for standard data collection. (T/F)

False

Surveys and questionnaires are interchangeable entities

False

Standardized survey questionnaire

Has been tested (reliable/valid form of data collection)

Nonstandardized survey questionnaire

Hasn't been tested (we don't know if it is reliable/valid for data colletion)

When do experimenters reject the null hypothesis?

If there is a statistically significant difference between the experimental and control groups

Why are participants more likely to complete the entire questionnaire when in person data collection is used?

In person data collection makes participants feel more accountable to the researcher in the room

Experimental Design

Investigate how variables affected each other in a controlled setting

Petra is collecting data from 50 high school teachers in her area. She would like to compare the job satisfaction rates of the teachers from one high school to the teachers from another high school. Petra will pay $50 to each participating teacher. Which part of Petra's study makes it quasi-experimental?

It is not possible in Petra's case to randomly assign participants to experimental and control groups

In which case would a quasi-experimental study be conducted?

It is not possible to randomly assign participants to experimental and control groups

Why is it important to do a pilot study before launching a questionnaire within a large-scale study?

It provides more data for researchers to analyze and interpret

Why is there little contact between researchers and participants within quantitative studies?

Most of the data within quantitative studies is collected via a computer or assistants

How can quasi-experiments be advantageous for researchers?

Quasi-experiments may be less time- and resource-intensive for a researcher

What can researchers do if they are worried that participants may misreport their true feelings, attitudes, and beliefs within a study?

Statistically control for social desirability bias in the data

Biased questions can sneak into questionnaires without researchers even noticing.

True

Experimenters are responsible for protecting their findings from alternative explanations.

True

Interviewing participants requires extensive preparation. (T/F)

True

Mailing questionnaires to possible participants is not common practice anymore (T/F)

True

Mailing questionnaires to possible participants results in a fairly low response rate.

True

Quasi-experiments may be more cost- and time-effective for a researcher. (T/F)

True

Quasi-experiments may not include pretests. (T/F)

True

The best survey questions are the simplest ones.

True

How should researchers go about making a list of study constructs?

Write down the constructs within the study hypotheses

Variables that have predefined responses from which participants can select are known as ______.

closed

A survey is ______.

data collection method

Variables are different from constructs in that they are ______.

easier to measure

Overall, there are ______ batches of data collected in a classic experimental design.

four

Participants are more likely to complete the entire questionnaire when ______ is used.

in person data collection

Quasi-experiments ______ pretests.

may not include

The control group is the same as the experimental group except ______.

members of the control group do not experience the manipulation

Open variables are best to measure ______.

opinions

In Computer-Assisted Telephone Interviewing (CATI), the first step is for a computer to call ______.

random phone numbers

Experiments include pretests and posttests so researchers can ______.

see how results change after an intervention

When participants try to respond to questionnaires in ways that make them look good, this is known as ______.

social desirability

Questionnaires become standardized after they have been ______.

tested and deemed to be reliable

In experimental designs, researchers hope that the difference between the experimental and control groups at the end of the study is due to ______.

the intervention or manipulation

One benefit of virtual data collection is ______.

the response rate is typically high

In Computer Assisted Telephone Interviewing (CATI), the second step is beginning the interview with ______.

those who have agreed to the interview during step one