Research Methods Chapter 7

Census

A census is the set of observations that contains all members of the population of interest. Ex: sampling chips, you do not need to eat the whole bag (whole population) to know whether you like the chips; you only need to test a small sample. If you did taste every chip in the population you would be conducting a census.

Population

A population is the entire set of people or products in which you are interested. Ex: have you ever been offered a free sample in a grocery store say you tried a sample of spinach mini quiche and you loved it you probably assume that all 50 in the box would taste just the same. Maybe you liked one baked pita chip and assumed all the chips in the bag would be good too. The box or bag the sample came from is the population.

Ways a sample may be biased

A sample could be biased and at least two ways, convenience sampling and self-selection

External validity: what matters most?

A sample is either externally valid for a population of interest or it has unknown external validity. Although external validity is crucial for many frequency claims it may not always matter

Oversampling

A variation of stratified random sampling is called oversampling in which the researcher intentionally over represents one or more groups. Perhaps researchers want to sample 1000 people making sure to include South Asians in the sample. Maybe the researchers population of interest has a low percentage of South Asians say 4%. Because 40 individuals may not be enough to make precise statistical estimates the researcher decides that of the 1000 people they sample 100 will be sampled at random from the Canadian South Asian community. In this example the ethnicities of the participants are still the categories but the researchers are over sampling the Asian population.

Stratified random sampling

Another multistage technique is stratified random sampling, in which the researcher purposefully selects particular demographics categories, or strata, And then randomly selects individuals within each of those categories, proportionate to their assumed membership in the population. For example a group of researchers might want to be sure The sample of 1000 Canadians includes people of south Asian dissent in the same proportion as in the Canadian population (which is 4%). The state might have two categories (strata) in their population: south Asian Canadians and other Canadians. In a sample of 1000 they would make sure to include at least 40 members of the category of interest (south Asian Canadians). Importantly however all 1000 members of both categories are selected at random. Strata are meaningful categories and the final sample sizes of the strata reflect their proportion in the population.

Cluster sampling

Cluster sampling is an option when people are already divided into arbitrary groups. Clusters of participants within a population of interest or randomly selected and then all individuals in each selected cluster are used. If a researcher wanted to randomly sample high school students in the state of Pennsylvania he would start with a list of 952 public high schools (clusters) in that state, I randomly select 100 of those high school (clusters) and then include every student from each of those 100 schools in the sample.

Nonprobability samples in the real world

Consider whether self-selection affects the results of an online shopping reading as in the zappos.com headline "61% said that the shoe felt true to size "you can be pretty sure that people who rated the fit of the shoe are self selected and therefore don't represent all the people who own that model. The Raiders obviously have Internet access where us some of the shoe owners may not. The raters probably cared enough to rate The shoes online. Another reason people respond might be that they are conscientious. They like to keep others informed so that they tend to rate everything they buy. In this case the shopping rating sample is self selected to include people who are more helpful than average. When you know a sample is not representative you should think carefully about how much it matters.

Convenience sampling: most common sampling technique

Convenience sampling is a bias sample which uses a sample of people who are easy to contact and readily available to participate. Psychology studies are often conducted by psychology professors and they find it handy to use college students as participants. (The Mehl Study on how much people talk is an example)Another form of convenience sampling is used in online studies. Another form of convenience sampling is used in online studies. People who want to earn money for participating in research can do so online.

When external validity is a lower priority

Even though you need probability sample to support a frequency claim many association and causes can still be accurately detected even in a non-probability sample. Researchers might not have the funds to obtain random samples for their studies and their priorities lie somewhere else. For example random assignment is prioritized over random sampling when conducting an experiment. What about a frequency claim that is not based on a probability sample? It might matter a lot or it might not you will just need to carefully consider whether the reason for the samples bias is relevant to the claim.

Nonprobability samples and research studies

Ex: 30 dual earner families who are allowed the researchers to videotape their evening activities. Only certain kinds of families will let researchers walk around the house and record everyone's behavior. Would this self-selection affect the conclusions of the study? It seems possible that a family that volunteers for such intrusion has a warmer emotional tone than full population of dual earning families. Without more data on families who do not readily agree to be videotaped we cannot know for sure the researchers may have to live with some uncertainty about the generalizability of their data.

Large or samples are not more representative

For external validity is a bigger sample always better? Dancer may surprise you: not really. The idea that larger samples are more externally valid then smaller Staples is perhaps one of the most persistent misconceptions and a research methods course. When I phenomenon is rare we do Need a large random sample in order to locate enough instances of that phenomenon for valid statistical analysis. For example in a study of religion in American life, the pew research Center contacted a random sample of 35,071 adults. The large size enabled them to obtain an ally sufficiently large samples of religious groups such as Jehovah's Witnesses, who make up less than 1% of Americans.

In frequency claims external validity is a priority

Frequency claims or claims about how often something happens in a population. When you read headlines like "8 out of 10 Drivers Experience road rage" "almost 9 out 10 afghans are suffering "or "2/3 of adults have had an adverse childhood experience "it might be obvious that external validity is important. If the driving study used sampling techniques that contain mostly urban residents the road rage estimate might be too high because urban driving might be more stressful. In certain cases the external validity of surveys based on random samples can actually be confirmed.

When is a sample biased?

If you reach all the way to the bottom of the bag to select your sample pita chip that sample would be biased or unrepresentative. Broken chips at the bottom of the bag or not representative of the population and choosing a broken chip might cause you to draw the wrong conclusion about the quality of the bag of chips. Similarly suppose the box of 50 quiches was a variety pack, containing various flavors of quiche. In that case a sample spinach quiche would be unrepresentative too. If other types of quiche are not as tasty as the spinach you would draw and correct conclusions about the varied population. In a consumer survey or an online opinion poll a biased sample could be like getting a handful of from the bottom of the bag where the broken pita chips are more likely to be. In other words a researcher sample might contain too many of the most unusual people

Biased sample

In a biased sample also called an unrepresentative sample some members of the population of interest have a much higher probability than other Members of being included in the sample.

Unbiased sample

In an unbiased sample also called a representative sample all members of the population have an equal chance of being included in the sample. Only unbiased samples allow us to make inferences about the population of interest.

Probability sampling

In probability sampling also called random sampling every member of the population of interest has an equal unknown chance of being selected for the sample regardless of whether they are convenient or motivated to volunteer. Therefore probability samples have excellent external validity and can generalize to the population of interest.

Combining techniques

In reading about studies in the news are an empirical journal articles you will probably come across methods of sampling that combined the techniques described here. Researchers might use a combination of multi stage sampling or oversampling for example. As long as clusters and individuals are selected at random the sample represent the population of interest and will have a good external validity. In addition to control for bias the researchers might supplement random sampling with a statistical technique called weighing.

Systematic sampling

In systematic sampling using a computer or a random number table the researcher starts by selecting two random numbers say "4" and "7". If the population of interest is a room full of students the researcher would start with a fourth person in the room and then count off choosing every seventh person until the sample is the desired size. Mehl and his colleagues use the EAR device to sample conversations every 12.5 minutes. Although they did not choose this value at random the effect is essentially the same as being a random sample of participants conversations. This type of random sampling is time consuming.

What is the population of interest?

Instead of the population as a whole, A research study's intended population is more limited. A population of interest might be laboratory mice, It might be undergraduate women, It might be men with dementia. Add a grocery store the population interest might be the 50 mini quiche in the box or the 200 pita chips in the bag. If a sample of people rated a style of shoe on how well they fit, we might be interested in generalizing to the population of people who have worn those shoes.

Coming from a population versus generalizing to that population

Just because a sample comes from a population does not mean to generalize to the population. Just because a sample consists of American drivers does not mean it represents all American drivers. Just because a sample contains Afghan people doesn't mean that sample can generalize the population of Afghanistan. Samples can be either biased or representative

Non-probability sampling

Non-probability sampling techniques involve non-random sampling and result in a biased sample

How do obtain a representative sample called a probability sampling techniques

One external validity is vital and researchers need an unbiased representative sample from a population probability sampling is the best option there are several techniques for probability sampling but they all involve an element of random selection

Snowball sampling

One variation on purpose of sampling that can help researchers find rare individuals is snowball sampling and which participants are asked to recommend a few acquaintances for a study. For a study on coping behaviors and people who have Crohn's disease for example a researcher might start with one or two people who have the condition and then ask them to recruit people from your support groups. Each of them might in turn recruit one or two more acquaintances until the sample is large enough. Snowball sampling is unrepresentative because people are recruited via social networks which is not random.

Quota sampling

Quota sampling which is similar to stratified random sampling The researcher identify subsets of the population of interest and then set a target number for each category in the sample for example 80 Asian Americans, 80 African-Americans, and 80 Latinos. Next the researcher samples from the population of interest non-randomly until the quotas are filled. As you can see both "a sampling and stratified random sampling specify subcategories an attempt to fill targeted percentages for numbers in each category. I however" a sampling the participants are selected non-randomly perhaps through convenience or purposive sampling and in stratified random sampling they are selected using a random selection technique.

Random assignment

Random assignment is used only an experimental designs. When researchers want to place participants into two different groups such as a treatment group and a comparison group they usually assign them at random. Random assignments enhance internal validity by helping ensure that the comparison group and the treatment group have the same kinds of people in them thereby controlling for alternative explanations. For example in an experiment testing how exercise affects well-being random assignment would make it likely that people in the treatment and comparison groups are about equally happy at the start

Self-selection

Self selection is a bias sample and this is a term used when a sample is known to contain only people who volunteer to participate. Self selection can cause serious problems for external validity. When Internet users choose to write something such as a product on Amazon, and online quiz, On Twitter or BuzzFeed, a professor on ratemyprofessor.com they for our self selecting when doing so.

Simple random sampling

The most basic form of probability sampling is simple random sampling. Visualize this process imagine that the name of each member of the population of interest is written on a plastic balls. The balls are rolled around in a bowl and then A mechanism spits out a number of balls equal to the size of the desired sample. The peoples whose names are on the selected balls will make up the sample. Another way to create a simple random sample is to assign a number to each individual in a population and then select certain ones using a table of random numbers. This type of random sample and can be time consuming

Sample

The sample is a smaller set taken from the population. Ex: have you ever been offered a free sample in a grocery store say you tried a sample of spinach mini quiche and you loved it you probably assume that all 50 in the box would taste just the same. Maybe you liked one baked pita chip and assumed all the chips in the bag would be good too. The single bit you tried is the sample. Researchers usually don't need to study every member of the population instead they study a sample of people assuming that if the sample behaves a certain way the population will do the same. The external validity of a study concerns whether the sample used in the study is adequate to represent the unstudied population.

Multistage sampling

Two random samples are selected: a random sample of clusters and then a random sample of people within those clusters. In the high school example the researchers would start with a list of high schools (clusters) in the state and then select a random 100 of those schools. Then instead of including all the students at each school the researcher would select a random sample of students from each of the selected schools.

How does the sample represent the population?

When interrogating external validity we ask whether the results of a particular study can be generalized to some larger population of interest. External validity is often extremely important for frequency claims. Recall that external validity concerns both samples and settings. A researcher may intend the results of a study to generalize the other members of a certain population. Or a researcher may intend the results to generalize to other settings.

Purposive Sampling

When researchers want to study only certain kinds of people they recruit only particular participants. When this is done in a non-random way it is called purposive sampling. Researchers wishing to study for example the effectiveness of a specific intervention to quit smoking would seek only smoke was further sample. Notice that limiting a sample to only one type of participant does not make a sample purposive. If researchers recruit the sample of smokers by posting flyers at a local tobacco store that action makes it purposive.

Random sampling

With random sampling also known as probability sampling researchers create a sample using some random method such as drawing names from a hat or using a random digit phone dialer so that each and every member of the population has an equal chance of being in the sample. Random sampling enhances external validity

Settling for an unrepresentative sample: non-probability sampling techniques

in cases where external validity is not vital to study skills researchers might be content with a non-probability sampling technique. Depending on the type of study they can choose among a number of techniques for gatherings such a sample