Research Methods Ch. 7

Stratified Random Sampling

-A form of probability sampling -A random sampling technique in which the researcher identifies particular demographic categories, or strata, then randomly selects individuals within each category.

Oversampling

-A form of probability sampling -A variation of stratified random sampling in which the researcher intentionally overrepresents one or more groups.

Population

-A larger group from which a sample is drawn -The group a study's conclusions are intended to be applied.

Example of Stratified Random Sampling

-A multistage technique, stratified random sampling, selects particular demographic categories, or strata, then randomly selects individuals within each of the categories, proportionate to their population. -EX: Researchers want to be sure their sample of 1,000 Canadians includes people of South Asian descent in the same proportion as the Canadian population. -They have two categories (strata) in their population: South Asian Canadians and other Canadians. -In a sample of 1,000, make sure to include at least 40 members of the category of interest (South Asian Canadians). All members of both categories selected at random. -Stratified random sampling differs from cluster sampling in two ways. 1) Strata are meaningful categories (ethnic or religious groups), whereas clusters are more arbitrary (any random set of high schools would do). 2) The final sample sizes of the strata reflect their proportion in the population, whereas clusters are not selected with such proportions in mind.

Sampling Only Those Who Volunteer

-A sample biased through self-selection, when a sample is known to only people who volunteer to participate, causing problems for external validity. -When Internet users rate a product on Amazon, quiz on Twitter, a professor on RateMyProfessors.com, they are self-selecting -The people who rate the items are not representative of the population of all people who bought that product, follow that Twitter account, visited that website, or took that class. -Researchers do not know how online "raters" differ from "nonraters," they speculate people who take the time to rate things might have stronger opinions or more willing to share ideas with others. -When members of an Internet survey panel are invited via random selection, then self-selection bias can be ruled out.

Ways a Sample may be Biased

-A sample could be biased in two ways: 1) Studying only those you can contact conveniently 2) Those who volunteer to respond -These threaten external validity because people who are convenient or more willing might have different opinions from those who are less handy and less willing.

Example of Oversampling

-A variation of stratified random sampling is called oversampling, intentionally overrepresents one or more groups. -Researchers sample 1,000 people, sure to include South Asians; population of interest has a low percentage of South Asians of 4%. -Because 40 individuals may not be enough to make precise statistical estimates, researchers decide of the 1,000 people they sample, 100 will be sampled at random from the Canadian South Asian community. -The ethnicities of the participants are still the categories, but researchers are oversampling the South Asian population: The South Asian group will constitute 10% of the sample, even though it represents only 4% of the population. -A survey that includes an oversample adjusts the final results so members in the oversampled group are weighted to their actual proportion in the population. -This is still a probability sample because the 100 South Asians in the final sample were sampled randomly from the population of South Asians.

Combining Techniques

-As long as clusters and individuals are selected at random, the sample will represent the population and have good external validity. -To control for bias, supplement random selection with a statistical technique, weighting. -If they determine that the final sample contains fewer members of a subgroup than it should, adjust the data so responses from members of underrepresented categories count more and overrepresented members count less. -All probability sampling techniques involve randomness, they ensure each individual has an equal and known chance of being selected.

What is the population of interest?

-Before deciding if a sample is biased or unbiased, specify a population to generalize to -A population of interest might be lab mice, undergraduate women, or men with dementia. -At the grocery store, the 50 quiches in the box. -If a sample of people rated a style of shoes on how well they fit, we are interested in generalizing to the population of people who have worn those shoes. -If we are considering the results of an election poll, we care about the population of people who vote in the next election.

Connivence Sampling

-Choosing a sample based on those who are easiest to access and readily available -A biased sampling technique.

Cluster Sampling and Multistage Sampling

-Cluster sampling is an option when people are already divided into arbitrary groups. -Clusters of participants within a population are randomly selected, then all individuals in each selected cluster are used. -EX: A researcher wanted to randomly sample high school students in PA, start with a list of the 952 public high schools (clusters) in PA, randomly select 100 of those high schools (clusters), then include every student from each of those 100 schools in the sample. -In the related technique of multistage sampling, two random samples are selected: 1) A random sample of clusters 2) A random sample of people within those clusters. -In the high school ex, the researcher starts with a list of high schools (clusters) in the PA and selects a random 100 of those schools. -Then, instead of including all students at each school, select a random sample of students from each of the selected schools. -Cluster and multistage sampling are easier than sampling from all PA high schools, and both still produce a representative sample because they involve random selection.

Sampling Only Those Who Are Easy to Contact

-Convenience sampling, using a sample of people who are easy to contact and available to participate. -Psych studies by psych professors find it handy to use college students as participants. -Those easy to reach college students may not be representative of other populations that are less educated, older, or younger. -Psychologists may conduct research through websites such as Prolific Academic or Amazon's Mechanical Turk. -People who want to earn money for participating in research can do so online. -Even though these samples are convenient, those who complete studies on websites sometimes differ slightly from other adult samples in terms of personality traits and political beliefs.

Nonprobability Samples in Research Studies

-EX: Only certain dual-earner families allowed researchers to video their evening activities and record their behavior. -It's possible a family that volunteers for such intrusion has a warmer emotional tone than the full population of dual-earning families. -Without more data on families who do not agree to be videotaped, we cannot know for sure. -EX: The sample from the ACE study was not drawn randomly from a population of Americans, it was given to more than 17,000 people in San Diego. -The study did not attempt to select people at random; we live with uncertainty about if the estimate from this study can generalize.

Nonprobability Samples in the Real World

-EX: Self-selection affects results of an online rating Zappos headline "61% said this shoe felt true to size." -You can be sure people who rated the fit are self-selected and don't represent all the people who own that shoe -The raters have Internet access, whereas some shoe owners might not. -EX: A driver uses an app to report a slowdown on a highway. This driver is not a randomly selected sample of drivers on that road, they are more conscientious and more likely to report problems. -Traffic is the same for everybody, conscientious or not, even though this driver is a nonrandom sample, the traffic report can generalize to the other drivers on that road. -Biased the sample (being conscientious) is not relevant to the variable being measured (being in traffic).

When is a Sample Biased?

-EX: You reached to the bottom of the bag to select your sample pita chip, that sample would be biased, unrepresentative. Broken chips at the bottom are not representative of the population, and choosing a broken chip might cause you to draw the wrong conclusions about the quality of that bag of chips. -EX: Students who rate a professor on a website tend to be the ones who are angry and might not represent the rest of the professor's students well.

Populations and Samples

-EX: You tried a sample of spinach quiche and loved it and assumed all 50 in the box taste the same. -The single bite you tried is the sample. -The box it came from is the population. -A population is the entire set of people or products in which you are interested. -The sample is a smaller set, taken from that population. -If you did taste every chip in the population, you would be conducting a census. -The external validity of a study concerns whether the sample used is adequate to represent the unstudied population. -If the sample can generalize to the population, there is good external validity. -When a sample has good external validity, the sample "is representative of" a population -If the sample is biased in some way, external validity is unknown.

Generalizability: Does the sample represent the population?

-External validity is important for frequency claims, ask: A) "Does the sample of drivers who were asked about road rage adequately represent American drivers?" B) "Can feelings of the Afghan people in the sample generalize to all the people in Afghanistan?" C) "Do the people who reviewed the fit of these shoes represent the population of people who wear them?" -External validity concerns both samples and settings.

Coming from a population VS. Generalizing to that population

-In a biased/unrepresentative sample, members of the population have a higher probability than others of being included in the sample. -In an unbiased/representative sample, all members of the population have an equal chance of being included in the sample. -Only unbiased samples allow us to make inferences about the population

Random Sampling and Random Assignment

-In conversation you hear, "I have a random question..," meaning an unexpected one. -In research, random has a more precise meaning: occurring without any order or pattern. -Each coin flip in a series is random because you cannot predict whether it will come up heads or tails; there's no predictable order. -Random sampling (probability), creates a sample using some random method, drawing names from a hat or random-digit dialer, each member of the population has an equal chance of being in the sample. -Random sampling enhances external validity. -Random assignment is used only in experimental designs; researchers place participants into two groups (treatment and comparison) at random. -Random assignment enhances internal validity by ensuring the comparison and treatment group have the same kinds of people in them, controlling for alternative explanations. -EX: In an experiment testing how exercise affects well being, random assignment makes it likely the people in the treatment and comparison groups are about equally happy at the start.

Example of Quota Sampling

-In quota sampling, similar to stratified random sampling, identifies subsets of the population and sets a target number for each category in the sample (80 Asian Americans, 80 African Americans) -Next, samples are taken from the population nonrandomly until quotas are filled. -Both quota and stratified random sampling specify subcategories and fill targeted percentages or numbers for each subcategory. -In quota sampling participants are selected nonrandomly (through convenience or purposive sampling) -In stratified random sampling, participants selected using a random selection technique.

Example of Systematic Sampling

-In systematic sampling, use a computer or random number table by selecting two random numbers. -If the population is a roomful of students, start with the fourth person in the room then count off, choosing every seventh person until the sample is the desired size. -Mehl used the EAR device to sample conversations every 12.5 minutes. He did not choose this value (12.5 min) at random, the effect is the same as being a random sample of participants' conversations. -External validity involves generalizing to populations of people and settings -Simple random sampling and systematic sampling work well, but difficult and time consuming.

Example of Snowball Sampling

-One variation on purposive sampling that helps researchers find rare individuals is snowball sampling, participants are asked to recommend a few acquaintances for the study. -A study on coping behaviors in people with Crohn's disease, starts with one or two people with the condition, then ask them to recruit people from their support groups. -Each of them might recruit one or two more acquaintances until the sample is large enough. -Snowball sampling is unrepresentative because people are recruited via social networks, which are not random.

Example of Purposive Sampling

-Researchers study certain kinds of people and recruit only those participants, when done in a nonrandom way, called purposive sampling. -Limiting a sample to one type of participant does not make a sample purposive. -Researchers studying the effectiveness of a specific intervention to quit smoking seek only smokers for their sample. -If researchers recruit smokers by contacting members at random, sample not purposive because it is a random sample. -If researchers recruit sample of smokers by posting flyers at a tobacco store, that action makes it a purposive sample, because only smokers will participate and because the smokers are not randomly selected.

Sample

-The group of people, animals, or cases in a study -A subset of the population of interest.

Larger Samples Are Not More Representative

-The idea larger samples are more externally valid than smaller samples is a misconception. -However, when a phenomenon is rare, we NEED a large random sample to locate enough instances to have valid statistical analysis. -EX: A study of religion in American life, Pew Research Center contacted a random sample of 35,071 adults. -The large size enabled them to obtain and analyze large samples of small religious groups, such as Jehovah's Witnesses who make up less than 1% of Americans. -When striving to generalize a sample to a population, size is less important than how that sample was selected; size is a statistical validity issue. -External validity of the sample is how, not how many. -EX: You want to predict the outcome of the election by polling 4,000 people at the Republican Convention. -You have a grand sample, but not about the opinions of the entire country's population because everyone you sampled is a member of one political party. -EX: You read 35% of Canadians support the Liberal Party, plus or minus 3%. Margins of error ("plus or minus 3%"), created so they likely contain the true percentage of Canadians who support the Liberal Party. -Researchers consider 1,000 to be an optimal balance between statistical accuracy and polling effort. -A sample of 1,000 people, as long as it is random, allows them to generalize to the population accurately.

Example of Simple Random Sampling

-The most basic form of probability sampling is simple random sampling. -EX: The name of each member of the population is written on a plastic ball. -The balls are rolled around in a bowl, then a mechanism spits out a number of balls equal to the size of the desired sample, the people whose names are on the selected balls will make up the sample. -EX: Assign a number to each individual in a population then select certain ones using a table of random numbers. -When pollsters need a random sample, they program computers to randomly select numbers or addresses from a database of eligible people.

Summary: GENERALIZABILITY: DOES THE SAMPLE REPRESENT THE POPULATION?

-The quality of a frequency claim depends on the ability to generalize from the sample to the population of interest. -When a sample is externally valid, it is unbiased, generalizable, or representative. -When generalization is the goal, random sampling techniques rather than sample size are vital because they lead to unbiased estimates of a population. -Nonrandom and self-selected samples do not represent the population. -Biased samples obtained when researchers sample only those easy to reach or only those willing to participate. -Probability sampling techniques result in a representative sample; includes simple random sampling, cluster sampling, multistage sampling, stratified random sampling, oversampling, systematic sampling, and combinations of these. -All of them select people or clusters at random, all members of the population of interest are equally likely to be included in the sample. -Nonprobability sampling techniques include convenience sampling, purposive sampling, snowball sampling, and quota sampling, such methods do not allow generalizing from the sample to a population.

Obtaining a Representative Sample: Probability Sampling Techniques

-When external validity is vital and need an unbiased, representative sample, probability sampling is the best option. -All techniques involve an element of random selection. -In probability sampling, random sampling, every member of the population has an equal and known chance of being selected for the sample, regardless of whether they are convenient or motivated to volunteer. -Have excellent external validity and can generalize to the population

Summary: INTERROGATING EXTERNAL VALIDITY: WHAT MATTERS MOST?

-When generalizing a sample to the population, probability sampling (random) is essential. -Random samples are crucial when estimating the frequency of a particular opinion, condition, or behavior in a population. -Nonprobability (nonrandom) samples can occasionally be appropriate when the cause of the bias is not relevant to the survey topic. -For external validity, the sample size is not as important as whether the sample was selected randomly.

Example of Sampling Only Those Who Are Easy to Contact

-You're conducting an exit poll during an election and hired interviewers to ask people who they voted for as they're leaving the polling place. -The sample biased by: 1) You only had enough money to send interviewers to polling places nearby and easy to reach. The resulting sample is biased because the neighboring precincts might be different from the district as a whole. 2) Interviewers approach the population of exiting voters in a biased way. Untrained exit poll workers may feel most comfortable approaching voters who look friendly, similar to themselves, or as if they are not in a hurry. -EX: Younger workers find it easiest to approach younger voters; Younger voters tend to be more liberal, that sample's result leads you to conclude the voters at that location voted for a Democratic candidate more often than they really did. 3) End up with a convenience sample if unable to contact an important subset of people. Might not be able to study those who live far away or who don't show up to a study, result in a biased sample when people the researchers contact are different from the population they want to generalize.

Population of Interests using Biased and Unbiased Sampling Techniques

1) Democrats in Texas -Biased: Recruiting people sitting in the front row at the Texas State Democratic Convention -Unbiased: Obtaining a list of all registered Texas Democrats from public records and calling a sample of them through randomized digit dialing. 2) Drivers -Biased: Asking drivers to complete a survey when they stop to add money to a parking meter. -Unbiased: Obtaining a list of licensed drivers in each state and selecting a sample using a random number generator.

Synonymous Sampling Terms Used

1) Externally Valid: -Unbiased -Probablity -Random -Representative 2) Unknown External Validity: -Biased -Nonprobablity -Nonrandom -Unrepresentative

Quota Sampling

A biased sampling technique in which a researcher identifies subsets of the population, sets a target number for each category in the sample, and nonrandomly selects individuals within each category until the quotas are filled.

Purposive Sampling

A biased sampling technique in which only certain kinds of people are included in a sample.

Nonprobability Sampling

A category name for nonrandom sampling techniques, such as convenience, purposive, and quota sampling, that result in a biased sample.

Probability Sampling (random sampling)

A category name for random sampling techniques, such as simple random, stratified random, and cluster, in which a sample is drawn from a population so each member has an equal chance of being included in the sample.

Self-selection

A form of sampling bias that occurs when a sample contains only people who volunteer to participate.

Cluster Sampling

A probability sampling technique in which clusters of participants within the population are selected at random, followed by data collection from all individuals in each cluster.

Systematic Sampling

A probability sampling technique in which the researcher uses a randomly chosen number N, and counts off every Nth member of a population to achieve a sample.

Multistage Sampling

A probability sampling technique involving at least two stages: a random sample of clusters followed by a random sample of people within the selected clusters.

Census

A set of observations that contains all members of the population of interest.

Snowball Sampling

A variation on purposive sampling, a biased sampling technique in which participants are asked to recommend acquaintances for the study.

Unbiased Sample (representative sample)

All members of the population are equally likely to be included through some random method, and results can generalize to the population

Biased Sample (unrepresentative sample)

Some members of the population are systematically left out, and the results cannot generalize to the population.

Simple Random Sampling

The most basic form of probability sampling, the sample is chosen completely at random from the population (drawing names out of a hat).

Random Assignment

The use of a random method (flipping a coin) to assign participants into different experimental groups.