Comm Research- exam 2
ex of margin of error
"the margin of error was +- 3 to 95% confidence level" interpreted as: if the poll was conducted 100 times, 95 of those polls would find a result within three point of the results reported here
understanding machine for content analysis
**
designing a content analysis study
- 14,000 tweets at airlines - determine a research question - decide how to measure variables - design codebook - try coding - revise codebook - finish coding
survey methods
- ask some people to tell you things - can be qualitative (open ended) but most are quantitative - used both by practitioners (marketers, brand managers, sales forces, advertisers) and researchers
limitations of content analysis
- can't say why or how content is created - can't say what audience read or affect on audience
types of variables
- categorical or nominal - ordinal - ratio or interval
between-
- compare difference in how groups respond - ask each person question once - random assignment important
within-subject design
- compare different individuals respond at different times - ask each person question twice (at least
purposive
- surveying a particular population because of your interest in them. Highly specific populations where random sample be infeasible - not random - likely biased (but maybe in an acceptable way) - used when random techniques wouldn't make sense
random selection
- surveys and other quantitative methods - external choosing who participates in the survey
benefits of content analysis
- systematic review of information - examines connection between institution and audience - pay attention to what's available
Chi square
- used for two categorical variables - asks " are the observed values different than what we would expect if there was no difference between the two groups?" - null hypothesis: no difference between the variables - where the phrase " on the margins" comes from - can be translated in to p-value; if it is not below .05 that means there is no effect between the two variables
limitations of experiments
- very specific and narrow findings - lacks ecological validity - can only find what its looking for - can't measure what's not measurable - demand effects: the results are reflective that the participants is just trying to please the researcher, say what they want to hear
regression
- when independent and dependent variables are continuous - many independent variables can be included predict what the value of one variable will be when you know the values of another variable
selection bias
- when individual differences are not evenly spread between groups - introduces new variables that can't be accounted for - threatens internal validity - may results in spurious relationships seen as meaningful
factorial design
- when you manipulate more than one thing in an experiment - see how the combination of variable cause different effects - example: 2 by 2 - the manipulations are mixed and matched - a full factorial is all possibilities - each combination is a "cell" - about 50 people per cell - draw each manipulation as a dimension
role of random assignment
-distributes random chance evenly - eliminates need for a perfectly representative sample - looks for differences between groups
You ask the following question on a survey. Please enter the length of your commute in minutes. The answers to the question will be which type of variable?
continuous
manipulation
controlling someone or something to your own advantage, often unfairly or dishonestly
mean
for a data set, also called average, is the central value of a discrete set of numbers; specifically, the sum of the values divided by the number of values
ratio or interval
there is clear and define space between each instant ex: time
margin of error
to what extent could the number be off from reality? calculated based on the size of poll relative to population
inferential statistics
use a random sample of data taken from a population to describe and make inferences about the population
How to do qualtrics
watch youtube video
random chance
we introduce this problem when we are only taking a sample of reality and are unable to not tell if it applies to all reality - wisdom of the crowd
confidence
what percent of polls would find the same results if the poll was repeated? set by the researcher
spurious correlation
when two variables move together (are correlated) but there is no underlying causation
snowball sampling
when you don't know how to reach most of your population, but you know where to start, so you start with one person and then they ask their friends
one tailed test
where you are only interested in one direction. if a mean is x, you might want to know if a set of results is more than x or less than x. more powerful than two tailed
two tailed test
will test both if the mean is significantly greater than x and is the mean is significantly less than x
There are 1,543 M&Ms in the bowl. A class guesses how many M&Ms are in a giant bowl. Guesses range from 3 to 9,500, but the average guess was 1,510. This phenomenon is known as:
wisdom of the crowd
mode
the most common number in a data set
P value
the probability that the findings are attributable to reality and not random chance - has to be below .05
independent sample t test
- compares the results from two groups to see hwo likely the difference is due to random chance - test you use when you have exactly two groups and you want to see if they are different from each other - returns a t-statistic - that t- statistic correlates with a p-value based on the degrees of freedom - if you have more than two groups, use a one way ANOVA - if the p-value is not lower than .05 than that means there is no difference
types of numbers
- counting- ex: numbers - measuring- ex: distance - categorization- ex: racial, ethnicity - scales and indexes
steps to deciding on a sampling technique
- decide your population boundaries - decide your modality - consider the benefits and limitations of each type of sample - consider how much money you want to spend
creating a codebook
- decide your variables of interest - decide how to measure those variables - create instructions on how to measure them - train coders - code 10% of the sample - check back with coders - finish the job - calculate intercode reliability
what to look at to determine the inferential statistical test
- determine independent and dependent variable (s) - determine the type of variable to be analyzed ~ categorical ~ continuous ~ ordinal - determine how many variables need to be included in the model
bad survey questions
- difficult or unclear instructions - too long! participant fatigue - question/answer choices don't match - answer options are not mutually exclusive - answers shouldn't overlap - answers should have a variety of options, covers the entire scope - answer options are not exhaustive - questions have double negative - scale points are meaningless - biased/leading questions - jargon or academic- ese
random
- each person in the population has an equal chance of being selected - used to be gold standard - should be most representative - should give the most accurate - problems with non response cause bias - problems finding whole populations from which to randomly select causes bias
benefits of surveys
- easy to collect data - relatively cost efficient - privacy for participants - low burden for participants - ask anything
content analysis
- examining message production - understanding what content is available - empirically describing how information is framed
Random assignment
- experiments - internal - from the people who participate, randomly assigning them to see a condition - distributes random chance evenly - eliminates need for a perfectly representative sample - looks for difference between groups
field experiments
- experiments conducted outside of controlled conditions in the "real" world - used in marketing, communication, politics, and so much more - A/B testing on websites --> growing field - also used in research - typically no deception used
intercode reliability
- how much agreement between coders about how to categorize content? - 30 statistics ~ Krippendorff's alpha ~ Cohen's Kappa - agreement between coders strengthen validity of results
ANOVA (analysis of variance)
- independent variable is categorical dependent variable is continuous - determines which is greater: variance within a group or variance between groups - especially use in experiments where independent variable is treatment category - if you add control variables, it becomes ANCOVA
good survey questions
- let your participants know whats up (use progress bars) - keep language and length simple - use odd numbered, meaningful scale points (5 to 7) - include a "don't know" or "neither" option where appropriate - ask demographic.psychographic questions that respect all people - "some people... other people" construction - do your best to mutually exclusive and exhaustive - proofread, have someone not connected pre test
descriptive statistics
- mean (average) - median - mode - standard deviation
Corrleation
- measures how two variables move in relation to each other - two continuous variables - it is not causation
experiments
- media effects research - mostly look at audiences - mostly based in cognitive psychology - controlled experiments are done in a lab or online survey - filed experiments ate done "in the wild"
Benefits of experiments
- most closely approximate causation - very specific and narrow findings - difficult to argue away as chance or anecdote - scientifically rigorous - can be generalized to lager population
Why use quantitative methods in social science?
- numbers to describe reality - appearance of being systematic - appearance of less researcher bias - adopts a positivist or post positivist viewpoint that reality can be discovered and described - more generalizable than qualitative
Quantiative
- numbers to describe reality - designed to remove researcher bias - standardized analysis methods - variable studied are specific and narrow - post positivist - generalizability
limitations of surveys
- self reported data isn't always accurate - response rate could bias results ~ people catch on to what your asking about and will reply what you want to hear ~ might want to keep info private even if survey is private - some topics not appropriate - no causation
quota
- survey is taken by enough people from each of the demographic and psychographics of interest to mirror a representative sample -sometimes called stratified sampling - reduced the randomness of the sample - can be difficult to find enough people to fill some quotes - fallen out of favor with most researchers/ pollsters
weighted
- survey is taken by randomly selected participants. the answers of those id underrepresented demographics are give more weight while over presented demos are given less - is more random than quota - bias is multiplied is sample has different opinion than population - is most commonly used method from high quality pollsters and researchers currently
causation
1. relationship between two variables 2. time order (A comes before B) 3. exclude all other variables - where surveys fail, because you can't ask about everything but in a controlled experiments we can- can control all other information except for the one thing we are manipulating
experimental notation
2x3x2 - first manipulation has two conditions - second conditions has three conditions - third manipulations has two conditions - overall, any participant could be randomly assigned to one of 12 combinations of manipulations. multiply all numbers together to get the total number of combinations
experiments & causation
1. most closely can show causation 2. isolate variables of interest 3. control outside variables 4. establish time order 5. randomly assign participants to condition
You read a news story where support for a new law is reported to be at 55%, with a ±5 margin of error at a 90% confidence level. If you were to conduct the poll 100 times, on average how many times would you get a result above 60%?
5
You want to know if there's an interaction between upper/lower classman status and living on/off campus that affects driving to campus. You survey 40 people and get counts of: Upper/lower classman status Live on or off campus Average number of days per week they drive to campus The best statistical test to answer your research question is:
ANOVA
field experiment
An experiment carried out in a 'natural' setting; that is, unlike in the case of laboratory experiments, the setting is not created by the researcher. Field experiments are relatively rare since identifying settings where experimental intervention is both feasible and ethical is difficult.
Write a hypothesis about television watching that would be testable using a survey.
If people are unemployed, then they are more likely to watch more TV than those who have jobs.
survey experiment
Survey refers to a technique of gathering information regarding a variable under study, from the respondents of the population.Experiment implies a scientific procedure wherein the factor under study is isolated to test hypothesis.
You ask the following question on a survey. Of the following, how do you commute to campus most often? By foot Bicycling City bus Taxi/shared ride service A friend's vehicle Personal vehicle The answers to the question will be which type of variable?
categorical
statistically significant
a data set is it when the set is large enough to accurately represent the phenomenon or population sample being studied. it is deemed it when the probability of the phenomenon being random is less than 1/20, resulting in a p-value of 5%
treatment group
a group that receives a treatment in an experiment. The "group" is made up of test subjects (people, animals, plants, cells etc.) and the "treatment" is the variable you are studying. ... The Group that does not receive the treatment is called the control group.
standard deviation
a measure of the amount of variation or dispersion of a set of values
A/B test
a method of comparing two versions of a web page or app against each other to determine which one performs better.
Posttest only design
a research design in which there are at least two groups, one of which does not receive a treatment or intervention, and data are collected on the outcome measure after the treatment or intervention
Solomon's Square
a research method that is sometimes used in social science, psychology and medicine. It can be used if there are concerns that the treatment might be sensitized by the pre-test. The four groups have four different experiences: Pre-test, treatment, post-test.
controlled experiments
a scientific test that is directly manipulated by a scientist, in order to test a single variable at a time. The variable being tested is the independent variable, and is adjusted to see the effects on the system being studied.
descriptive statistics
a summary statistic that quantitatively describes or summarizes features of a collection of information
stimulus
a thing that rouses activity or energy in someone or something; a spur or incentive.
median
about halfway between the two extremes of size of another quality; the middle term
lab experiment
an experiment conducted under highly controlled conditions (not necessarily a laboratory), where accurate measurements are possible. The researcher decides where the experiment will take place, at what time, with which participants, in what circumstances and using a standardized procedure.
pre test/ post test design
an experiment where measurements are taken both before and after a treatment. The design means that you are able to see the effects of some type of treatment on a group. Pretest posttest designs may be quasi-experimental, which means that participants are not assigned randomly.
control group
as the group in an experiment or study that does not receive treatment by the researchers and is then used as a benchmark to measure how the other tested subjects do.
normal curve
bell shaped curve which shows the probability distribution of a continuous random variable. represents a normal distribution. the total area under it logically represents the sum of all probabilities for a random variable
modality
bottom: online next: phone next: in person one place top: in person door to door - as you go up the triangle, the cost goes up but as you go down the data quality gets worse
reaching people online
bottom: people who showed up to a website (a particularly terrible type of convenience sampling that can be be gamed- can be highly bias) next: quota sampling panels next: weighted sampling panels top: nationally representative online panel - as you go up the triangle the data quality gets better
NOVA video
found it very interesting that the idea of p value was created by Fisher from a woman who thought she could distinguish milk from tea or tea from milk. I also found the idea of p-hacking very intriguing because I didn't think that researchers would try and manipulate the p value so that they could get the results they wants and ultimately get the research published. I would think that researchers would always be ethical with conducting research. I think predictions and probabilistic statistic play a somewhat large role in our everyday lives because we are always making predictions, for example predicting the weather or predicting who will win the next sports game, like baseball
confederate
individuals who seem to be participants but in reality are part of the research team. They essentially trick real participants into thinking they are fellow participants. Asch used confederates in an integral way for his research on majority influence.
convenience sampling
interviewing people who are available - good way to get good data, but can not make generalizable about the entire population because they are bias, they just give a portion
Hawthorne effect
just by putting you in a study or experiment, you change what you say/do
categorical or nominal
just randomly assign a category a number, but no rank in order ex: hair color
It's two weeks before a special election for a U.S. Senate seat in New Mexico. Several opinion polls are published. You would expect the survey using which sample to most accurately predict the election?
likely voters
what statistic would be most helpful to analyzing the previous question about parking?
mean
You analyze data from a survey question and find the age of the participant whom half the participants are older than and half the participants are younger than. This statistic is:
median
You believe that you can tell the difference between Dr. Pepper and Mr. Pibb. So you do a run 8 trials of blind taste tests. You answer correctly 5 times. Using a binomial experiment calculator, you determine the p-value for your experiment is p=.054. Can you reject the null hypothesis and accept that you can tell the difference between the two sodas?
no
a limitation to quantitative research is
not everything can be measured with numbers
You want to know if upperclassmen drive to campus more than underclassmen. You survey 40 people who who are students at VT and ask: Upper or lower classman status Average days per week they drive to campus The best statistical test to answer your research question is:
t-test
You want to know which of two marketing messages best gets students to use the bus to get to campus. You find 40 students who currently drive their personal vehicles to campus. You send Group A an email each day for two weeks with Marketing Message 1. You send Group B an email each day for two weeks with Marketing Message 2. At the end of the month you ask each group to report how many days they took the bus to campus. The best statistical test to answer your research question is:
t-test
range
the biggest number subtracted from the smallest number
condition
the circumstances affecting the way in which people live or work, especially with regard to their safety or well-being.
in inferential statistics, a p-value indicates
the degree to which you can rule out random chance
You ask the following question on a survey. How much difficulty do you face when trying to park on campus? Please answer from 1 to 5, where 1 is very difficult and 5 is very easy. The answers to the question will be which type of variable?
ordinal
p value
probability value is the probability of obtaining test results at least as extreme as the results actually observed during the test, assuming that the null hypothesis is correct. numbers range from zero to 1, always looking for number less than .5
You are interested in capturing American public opinion on a current event issue. The best sampling technique to use is:
random
ordinal
ranking to it, the space between the ranks isn't defined and isn't necessarily meaningful
