CCJ3701 Exam 2 Notes (Starting w Ch.5 Sampling)

2 things that may reduce random error

-Larger the sample, the more confidence we have in its inferential abilities -The more homogenous the sample, the more confidence we can have in its inferential abilities

validity

-Measure what it is intended to measure -Match between the conceptual and operational definition -Much difficult than reliability

Surveys are popular because:

-Versatility: Can be used to study many domains of human behaviors, attitudes, etc. -Efficient: Many people at low cost -Generalizability: Lends itself well to probability sampling from larger populations

reliability

-We are consistently measuring (although it may not be valid) -Yields consistent scores when the thing being measured is not changing -Don't use reliability as an indication of validity -Not going to run into a measure that is accurate but unreliable

Generalizability

-a key concern in research design

2 tests select interviewees must pass by Rubin & Rubin

-adhering to these guidelines will help insure that a purposive sample adequately represents the setting or issues being studies -completeness -saturation

to assess sample quality, these are some questions to ask

-from what population were the cases selected? -what method was used to select cases from this population? -do the cases that were studied represent, in the aggregate, the population from which they were selected

how to assess sample quality

-from what population were the cases selected? -what method was used to select cases from this population? -do the cases that were studied represent, in the aggregate, the population from which they were selected?

wider intervals

-less confident in the sample estimate -because the confidence we have in statistics obtained from such small samples is so low, they give us very little useful information about the population

average item response?

-possible responses to the 16 questions are between 0 and 2 -when you summate the score is between 32 and 0 -take the summated score and divide it by the number of items on scale

2 key features of measurement (psychometrics)

-reliability -validity

limitations of quota sampling

-researchers can only set quotas for only a small fraction of the characteristics relevant to the study -you must know the characteristics of the entire population to set the right quotas -quota sampling's lack of random

multi-stage cluster sampling

-sample from pre-existing sub-groups of your target population -sub-groups sampled are chosen randomly from all possible sub-groups -then, sample randomly WITHIN each sub-group -requires less prior information -can be useful when a sampling frame is not available, as often is the case for large populations spread across a wide geographic area or among many different organizations

2 aspects of generalizability

-sample generalizability -cross-population generalizability

problems with random samples

-selecting elements from an incomplete list of the total population --failing to obtain adequate response rate

4 most common methods for drawing random samples

-simple random sampling -systematic random sampling -stratified random sampling -cluster sampling

4 types of reliability

-test-retest -alternate forms -internal consistency -inter-rater

problems with snowball sampling

-the initial contacts may shape the entire sample and foreclose access to some members of the population of interest -validating whether individuals claiming to be gang members, so-called wannabes, actually were legitimate membersq

Statistical Power Analysis

Method to determine sample size necessary to test the effect of the independent variable (or relationship between a given set of variables) -professional social science studies typically use a sample size of 1,000-3,000 individuals

systematic sampling error

Overrepresentation or underrepresentation of some population characteristics in a sample due to the method used to select the sample. A sample affected by systematic sampling error is a biased sample

random selection

The fundamental element of probability samples. The essential characteristic of random selection is that every element of the population has a known and independent chance of being selected into the sample.

confidence limits

The lower and upper scores (bounds) defining the confidence interval

representative sample

a sample that "looks like" the population from which it was selected in all respects that are potentially relevant to the study -the distribution of characteristics among the elements of a representative sample is the same as the distribution of those characteristics among the total population -in an unrepresentative sample, some characteristics are overrepresented or underrepresented

probability sampling method (random sample)

a sampling method that relies on a random, or chance, selection method so that the probability of selection of population elements is known -allows us to know in advance how likely it is that any element of the population will be selected for the sample -probability of selection is known and is not zero, so there is some chance of selecting each item -randomly select elements and have no systematic bias -aka random sample -allow a researcher to use the laws of chance, or probability to draw samples from population parameters that can be estimated with a high degree of confidence

respondent-driven sampling

a sampling technique in which a researcher uses a member of the population of interest to actively recruit others, often with some incentive like money for engaging in this recruiting -sophisticated version of snowball sampling -financial incentives called gratuities

non-probability sampling

a sampling technique in which there is no way to calculate the likelihood that a specific element of the population being studied will be chosen -because the chance of any element being selected is unknown, we cannot be certain the selected sample actually represents our population

Single-item scale

a scale format that collects data about only one attribute of an object or construct

Multiple-item scale

a scale format that simultaneously collects data on several attributes of an object or construct -Going from elements that are not of interest to elements that are -ex: census blocks, dwellings, individuals -items can be statements, questions -responsents are given ratings for each item (Likert) --> ordinal

target population

a set of elements larger than or different from the population sampled and to which the researcher would like to generalize study findings

sample

a subset of the population -unnecessary if all of the units in the population are identical

normal distribution curve

a symmetrical, bell-shaped curve that describes the distribution of many types of data; most scores fall near the mean (about 68 percent fall within one standard deviation of it) and fewer and fewer near the extremes. -centered around the population mean -they differ to the extent to which they cluster around the mean

random number table

a table containing lists of numbers that are ordered solely on the basis of chance; it is used for drawing a random sample -simplifies the process considerably

sampling error

any difference between the characteristics of a sample and the characteristics of a population. The larger the sampling error, the less representative the sample is of the population

nonresponse bias

bias introduced to a sample when a large fraction of those sampled fails to respond

population

consists of the set of individuals or other entities to which we wish to be able to generalize our findings

replacement sampling

each element is returned to the sampling frame after it is selected so that it may be sampled again

secondary sampling units

entities selected in the second stage of the sample

simple random sampling

every member of the population has an equal probability of being selected for the sample -requires a procedure that generates numbers or identifies cases strictly on the basis of chance -ex: flipping a coin, rolling a die -an equal probability of selection method -can be done with or without replacement sampling

sampling theory

focuses on the generalizability of descriptive findings to the population from which the sample was drawn -it also consideres whether statements can be generalized from one population to another

goal in research

generalize from the sample to the larger population

Correlation

how a distribution of scores are related in terms of the rank and order when measured at two times. Used to determine whether there is a high degree of stability in scores

power

if there is a relationship in the population, you have a large enough sample to be able to detect it

elements or elementary units

individual members of the sample

Rubin & Rubin 3 guidelines for selecting informants when designing any purposive sampling strategy

informants should be -knowledgable about the cultural arena or situation or experience being studied -willing to talk and -representative of the range of points of view

sampling frame

list of individuals from which a sample is actually selected -list from which the elements of the population are selected -if this list is incomplete, a sample selected randomly from the list will obviously not be a random sample of the population because not everyone in the population was represented on the list -required in most probability sampling methods

statistical tests

mathematical formulation that helps scientists evaluate whether the results of a single experiment demonstrate the effect of treatment -making inference from sample to population

inferential statistics

mathematical tools for estimating how likely it is that a statistical result based on data from a random sample is representative of the population from which the sample was selected

Why dont researchers obtain large random samples all of the time?

money (they are expensive)

cluster sample

obtained by selecting all individuals within a randomly selected collection or group of individuals -often involve multiple stages -to be closer to the true population value, researcher maximizes the number of clusters and minimizes the number of individuals in each cluster -the more homogeneous the clusters, the fewer cases needed per cluster -drawback: sampling error is greater in a cluster sample than in a simple random sample --> error increases as the number of clusters decreases

sampling without replacement

once an element has been included in the sample, it is removed from the population and cannot be selected a second time.

sample size

one determinant of sample quality

2 types of sampling methods

probability and non probability

sample generalizability

refers to the ability to generalize from a sample, or subset, to a larger population to that population itself -exists when a conclusion based on a sample, or subset, of a larger population holds true for that population -depends on sample quality, which is determined by the amount of sampling error

Cross-population generalizability (external validity)s

refers to the ability to generalize from findings about one group, population, or setting to other groups, populations, or settings -aka external validity

disproportionate stratified sampling

sampling in which elements are selected from strata in different proportions from those that appear in the population -the probability of selection of every case is known but unequal between strata

availability sampling

sampling in which elements are selected on the basis of convenience -elements are available or easy to find -aka: haphazard, accidental, or convenience sampling -ex: news reporters often use person-on-the-street interviews -even though they are not generalizable, they are often appropriate in research -ex: John Irwin's "The felon study" & Bourgois, Lettiere, and Quesada

proportionate stratified sampling

sampling method in which elements are selected from strata in exact proportion to their representation in the population

non probability sampling method

sampling methods in which the probability of selection of population elements is unknown -does not let us know the likelihood in advance

Purposive sampling

selecting sample members to study because they possess attributes important to understanding the research topic -each sample element is selected for a purpose, usually because of the unique position of the sample elements -aka judgment sampling --> researchers use their own judgment about whom to select into the sample, rather than drawing sample elements randomly

quotas

set to ensure that the sample represents certain characteristics in proportion to their prevalence in the population

unit of observation

the cases about which measures actually are obtained in a sample

survey research

the collection of information from a sample of individuals through their responses to questions. It is an efficient method for systematic collection of data -Most popular research method in social sciences -Type of research we are doing for course project -Common way to collect data -Young approach to data collection, but has come a long way

random sampling error

the deviation between the sample statistic and the population parameter caused by chance (random error) in selecting a random sample, not systematic sampling error -may or may not result in an unrepresentative sample -the magnitude of sampling error due to change factors can be estimated statistically

sampling distribution

the hypothetical distribution of a statistic across all the random samples that could be drawn from a population -understanding this is the foundation for understanding how researchers can estimate sampling error -usually remains hypothetical or a theoretical distribution -more compact when it is based on larger samples

units of analysis

the level of social life on which a research question is focused, such as individuals, groups, towns, or nations -we obtain samples from many different units -in most social science research, units of analysis are individuals -units of analysis may instead be groups, such as families, schools, prisons, towns, states, or countries -in some studies, groups are the units of analysis, but data are collected from individuals

probability of selection

the likelihood that an element will be selected from the population for inclusion in the sample -that chance

why do we use non-probability sampling if we cannot generalize results to a larger population?

the method is useful for several purposes including those situations in which we do not have a population list, when we are exploring a research question that does not concern a large population, or when we are doing a preliminary or exploratory study

probability theory

the one mean we actually obtain from our sample is unlikely to be far from the true population mean if, in fact, we have drawn a true random sample

primary sampling units

the sampling units in the first stage of a multistage sample

periodicity

the sequence varies in some regular, periodic pattern

sampling units

the target population elements available for selection during the sampling process

sampling interval

the total number of cases in the population divided by the number of cases required for the sample

census

these study the entire population of interest rather than drawing a sample -population studied is generally really small -it is usually too expensive and time consuming to collect data from all the members of some large population -U.S. does this once every 10 years

enumeration units

units that contain one or more elements and that are listed in a sampling frame

completeness test

what you hear provides an overall sense of the meaning of a concept, theme, or process

reductionist fallacy (reductionism)

when data about individuals are used to make inferences about group level processes

saturation test

you gain confidence that you are learning little that is new from subsequent interviews

both size of the sample and the homogeneity (sameness) of the population affect the degree of error due to chance; the proportion of the population that the sample represents do not

-the larger the sample, the more confidence we can have in the sample's representativeness of the population from which it was drawn -the more homogeneous the population, the more confidence we can have in the representativeness of a sample of any particular size -the fraction of the total population that a sample contains does not affect the sample's representativeness, unless that fraction is large -sample size is what makes representativeness more likely, not the proportion of the whole that the sample represents

Hypothesis testing

-type 1 error -type 2 error

lessons about sample quality (ch.5 pg. 132)

-we cannot evaluate the quality of a sample if we do not know what population it is supposed to represent -we cannot evaluate the quality of a sample if we do not know exactly how cases in the sample were selected from the population -sample quality is determined by the sample actually obtained, not just by the sampling method itself -we need to be aware that even researchers who obtain very good samples may talk about the implications of their findings for some group that is larger than or just different from the population they actually sampled

4 nonprobability sampling methods

1) Availability Sampling 2) Quota Sampling 3) Purposive Sampling 4) Snowball Sampling

Steps in Survey Research

1.Establish the goals of the project What do you want to learn? 2.Determine your sample Who will you interview? 3.Choose survey methodology How will you administer the survey? 4.Create your questionnaire What will you ask? 5.Pre-test the questionnaire Test the questions 6.Conduct interviews, enter data Ask the questions 7.Analyze the data Produce the reports

cluster

A naturally occurring, mixed aggregate of elements of the population, with each element appearing in one and only one cluster -drawing a cluster sample is at least a two-stage procedure

population parameter

A statistic computed for the entire population

random digit dialing

A technique used by pollsters to place telephone calls randomly to both listed and unlisted numbers when conducting a survey. -machine dials random numbers within the phone prefixes corresponding to the area in which the survey is conducted -particularly useful when the sampling frame is not available

representative sample

When the characteristics of the sample closely match the population

snowball sampling

You interview some individuals, and then ask them to identify others who will participate in the study, who ask others...etc., etc. -useful for hard-to-reach or hard-to-identify interconnected populations -ex: drug dealers, prostitutes, criminals, gang leaders, and informational organizational leaders

stratified random sampling

a form of probability sampling; a random sampling technique in which the researcher identifies particular demographic categories of interest and then randomly selects individuals within each category -uses information known about the total population prior to sampling to make the sampling process more efficient -all elements in the population are distinguished according to their value on some relevant characteristic, which forms the sampling strata -elements are sampled randomly from within these strata -uses prior information about a population to make sampling more efficient -may be either proportionate or disproportionate

systematic random sampling

a method of sampling in which sample elements are selected from a list or from sequential files, with every nth element being selected after the first element is selected randomly within the first interval -variant of simple random sampling, a little less time consuming -3 steps (ch. 5 sampling pg. 123)

Quota sampling

a nonprobability sampling technique in which researchers divide the population into groups and then arbitrarily choose participants from each group -intended to overcome the most obvious flaw of availability sampling, that the sample will just consist of who or what is available, without any concern for its similarity to the population of interest -generally less rigorous and precise in selection procedures -involves designating the population into proportions of some group that you want to be represented in your sample -proportions may represent the true proportions of the population or predetermined proportions of subsets of people you deliberately want to oversample -not much better than an availability sample

weight

a number you multiply by the value of each case based on the stratum it is in -reduces the influence of the oversampled strata and increases the influence of the under sampled Strata to just what they would have been if purely probability sampling had been used

confidence interval

a range of values so defined that there is a specified probability that the value of a parameter lies within it.

ecological fallacy

an error in reasoning where researcher draws conclusions about individual-level processes from group-level data

sample statistic

an estimated statistic (e.g. proportion, mean) from the one sample we actually selected from a population -an estimate of the population parameter that we want to estimate