210-2
approaches to online surveys
-SSI -Knowledge networks gfk -mechanical turk/amazon
content analysis
-a research technique for making replicable and valid inferences from texts to the contexts of their use -summarizing, quantitative analysis of messages that relies on the scientific method and is not limited as to the types of variables that may be measured or the context in which the messages are created or presented -summarizes data -aplicable to all contexts -can be used to analyze all message characteristics -relies on the scientific method -is quantitative
box and whisker plots
-a way of showing data grouped by interquartile range -3 parts: box, whiskers and extreme values/outliers -box: middle 50% of all the values -whiskers: 1.5 times range of box -outliers: individually located with asterix
normal distribution
-allows us to calculate probability of the events occurring in our dataset
response catagories
-avoid open ended questions unless doing exploratory research because they allow for different wordings by respondents and are difficult to code and interpret because respondents interpret them differently -measure the strength of the attitude or opinion opposed to just asking yes/no questions -provide a "dont know option"-can reduce number of missing values in an item bc if people dont know answer they might not answer question -offer an equal number of response categories for each side of the issue so you dont bias respondent to one side -start your scale with the socially undesirable responses-if you dont, respondents will be more likely to choose desirable answer
ways to reduce non response error
-call backs and reminders -incentives -oversampling
interviewing pitfalls
-closed v open ended -leading questions -dichotomous questions -double barreled -beware of the why -interviewer bias
response effects
-errors caused by factors other than problems with sampling or non response, such as misinterperetation of a question or deliberate lying; arise from different parts of the questioning process 3 sources of response error: 1. poor measurement 2. interviewer 3. respondent
qualitative vs quantitive
-focus on meaning rather than quantifiable phenomena -collection of many data on a few cases rather than few data on many cases -study in depth and detail without predetermined categories or directions -sensitivity to context rather than seeking universal generalizations -attn to impact of researchers and other values on the analysis rather than presuming value free inquiry -goal of rich descriptions rather than measurement of specific variables
qualitative methods
-group of method that rely on written or spoken words or observations that do not have a direct numerical interpretation and typically involve exploratory research questions, inductive reasoning, an orientation to social context and human subjectivity, and the meanings attached by participants to events and their lives
focus groups
-groups of individuals interviewed together in which an unstructured group discussion on a particular topic is led by a moderator or facilitator -relatively unstructured but are asked specific questions by a facilitator -can vary in length and size -mimics the natural process of forming and expressing opinions advantages -flexibility, helpful in pilot/exploratory work, promote honest sharing of ideas disadvantages -generalizability, moderator, comments can become extreme
interviewer response effects
-inappropriate prompting/probing -an interviewer may change question wording -may stereotype -provide guidance that is above and beyond instructions -best way to avoid interviewer effects= TRAINING
survey modes
-interviews- phone or face to face -self administered-mail, computer assisted, online
measures of dispersion
-measures of variation/spread-tries to capture the distribution of values around CT in a single estimate 1. range: max-min 2. interquartile range: uses median 3. variance: average squared distance from mean 4. standard deviation: higher means more variability
participant observation
-method of gathering data involving developing a sustained relationship with people while they go about normal activities -natural social processes are studied as they happen and left relatively undisturbed -involves seeing world as the participant sees it -encourages consideration of the context in which the social interaction occurs
web/online surveys
-newer method of admin, research on its effectiveness in reducing error is still being conducted -sampling is problematic due to issues stemming from comp access. sometimes to fix this companies provide equipment -often used to conduct polls, advertising, or getting user info -can create through survey monkey or other developers -send link as you would male or face to face survey advantages -low overhead cost -quick results -automated data entry -efficiency in data reporting disadvantages -maintenance cost -sampling problems can occur depending on resource used -intensive data cleaning -biased representation can occur depending on resource used
descriptive statistics
-numbers that are used to consolidate a large amount of info, like an average -describe basic features of data and characterize data -summarize characteristics of a set of observations or a relationship between 2 variables
6 key attributes of a content analysis that rely on the scientific method
-objectivity: results offer a description/explanation avoiding investigator bias- achieve this through codebook instructions and coder training -a priori design: all decisions on variables made before observations begin -reliability: are measures stable? intercoder reliability when they have simple agreement, agreement controlling for change, and covariation. threats to reliability are poorly executed codebook, inadequate coder training, coder fatique, and rogue coder -validity: are we measuring what we want to measure/ face and content validity and criterion validity that compares to alternative content -generalizability: EV, depends on how sample is selected -replicability: safeguard against over generalizing the findings of one particular research endeavor. involves repeating a study with a different case or in a diff context, checking to see if similar results obtained each time
multi method surveys
-often common to combine survey method. makes it possible to maximize advantages and minimize the disadvantages of different methods
question types
-open ended -unordered close ended-list examples -ordered close ended-scales -too few categories can increase error -a multi item scale is superior to single items for reliability, unidimensionality, and freedom from specific wording bias -mutually exclusive and exhaustive response categories
myths of qualitative research
-open to bias -time consuming -findings are not evidence -only good as adjuncts to more robust and unbiased research
role of researcher in participant observation
-overt observer-complete observer- try to see things as they happen without actively participating in events. awareness of role as researcher. concern of reactive effects. objective,unsympathetic, detached, candid -participant and observer-some active participation in group while informing group of their research interests. fine balance. what is shared and what is not -covert participant/full participant observation-participate in group but not reveal role as researcher. less potential for reactive effects and help gain access to inaccessible groups. challenging and ethical concerns. subjective, sympathetic, involved, deceptive
What surveys do
-permit the development and testing of logical explanations and the clarification of the role of intervening mediating variables in the causal process -allow for generalization and the empirical verification of our scientific theories and ideas
characteristics of the task
-poor measurement types of variables: -methods of administration -open v closed questions -question order -length and wording of questions -memory
question wording
-poor wording can negatively affect respondent answers and also result in misinterpretation -use wording that all respondents can understand -keep wording simple and familiar to all respondents- define terms and avoid jargon -avoid double barreled questions and double negatives -give instructions and definitions
telephone surveys
-sample chosen from phone directory, or using random number techniques -people sampled at time of first phone call or a more convenient time -interviewers record answers on a form best when: -members of pop have phone -questions are straightforward -experienced help is available -quick turn around important advantages: -cheaper than f to f -easy to implement -moderate to high RR -quicker than alernatives disadvantages -limit on types of questions -limits on length -sampling bias -bad for locating rare events
mail surveys
-sample selected from mailing list, respondents fill out questions and send back best when -surveys which an immediate turn around not required -projects that are low on resources like $$ and experienced interviewing staff advantages -usually cheapest to implement if you have a list -perhaps less respondent bias especially with sensitive or threatening questions -no interviewer effects disadvantages -education bias-illiterates less likely to return survey -takes longer, need time to follow up waves and for respondent to mail back -sampling problems if pop is poorly defined-coverage error -non response higher -limits on types of questions asked
considerations in survey research
-sampling -survey type -constructing the survey -response effects
face to face surveys
-select sample from a list and contact each member to conduct in person interview -record responses and transcribe best when: -surveying populations for whom there is no list -collecting info from people not likely to respond by mail or phone -complex questionnaires -experiences interviewers available advantages -more detailed info -can correct procedural mistakes as they occur -respondent can do more complicated tasks-sorting and judgment -high response rate disadvantages -most expensive -respondent and interviewer bias -training costs and overhead -needs detailed specifications to select respondents
Why are surveys so popular despite their limitations for making causal claims?
-sometimes learning about or of associations rather than making causal inferences is sufficient -versatility -efficiency -generalizability
field studies
-studying people in natural setting -ethnography- the study of a culture or cultures that some group of people share using participant observation over an extended period of time. a naturalistic approach to data collection using methods of observation -participant observation
qualitative data collection
-taking notes- detailed field notes of observations, direct quotes and personal thoughts -use of technology- tape recorders
survey types
-telephone -mail -face to face -computer assisted -web/online/internet -multi method
question order
-there should be a logical order to a set of items- ask all questions relevant to one subject before asking questions on another subject in order not to confuse or frustrate the respondent and to keep them focused -start the questionnaire with easy, salient and non threatening questions and then get to the more sensitive ones-helps build rapport and trust between respondent and interviewer -chronological order for recall questions on events -use funneling technique-most general questions to specific-this aids retrieval of info -ask objectionable questions at end of survey -first question is most crucial-determines if survey will be completed or discarded-should apply to everyone and be interesting and easy to understand -demographics and other questions that the respondent may not be willing to disclose related information should go at end of surveys in mail, but beginning of face to face to build rapport
how is content analysis used?
-to measure an ind var and dep var -to inform a policy debate
when to use qualitative methods
-unexplored context -inappropriate context for questionnaires -finding the right questions to ask -theory generation -formative research
decisions before content analysis
-what is hypothesis and research question -what is population of your content? -how are you going to sample from pop and how does it affect generalizability -what is unit of analysis -how are you measuring units?
standard normal distribution
1 SD=68% 2 SD=95.4% 3 SD=99%
5 steps of focus groups
1. assemble he groups 2. prepare study mechanics 3. prepare focus groups session material and questions 4. conduct the session-a skillful moderator is key 5. analyze data and prepare summary report
common uses for focus groups
1. gather preliminary info for research project-formative research 2. help develop questionnaire items for a survey 3. understand reasons behind a particular phenomenon 4. test preliminary ideas or plans
question order effects
1. may effect saliency of particular topics by providing differential contexts within which responses are elicited; saliency effect- attitudes about topics of low salience to the respondent would be more susceptible to order effects 2. redundancy effects: results from overlap in content between sections 3. consistency effects: related to the influence of earlier judgments on later ones; principal type of effect demonstrated in the literature 4. fatigue effects: results from the overall position of the questions in the questionnaire rather than their relation to one another 5. rapport effects: sensitive questions are often put further along in the questionnaire so that the interviewer will have time to establish a good relationship with the respondent -can reduce effects by randomizing the order in which items from particular questions appear
Survey
A quantitative method widely used throughout communication studies and other social science disciplines. A method for collecting information from samples of individuals
ordered close ended
Advantages: -rapid way of asking a question -special form of balance -can measure different forms using same question disadvantages -scale sorts people into XX categories but most analysis puts people into two -interpretable only when asked about the extremes of a continuum -people often agree with a statement just to avoid seeming disagreeable
computer assisted surveys
CASI-computer assisted interviewing ACASI-audio computer assisted interviewing -similar advantages and disadvantages to mail surveys, except audio CASI useful in low literacy pops -prevents errors of administration -good for sensitive questions -expensive
survey sampling international
SSI recruits online respondents from more than 3400 sources using a variety of methods and targeted approaches such as banner ads, keywords, search links, pop up polls, email, online invitations, and co registration. community member ship is double opt in and applicants are carefully screened
key informant
a knowledgeable insider who knows the relevant culture and is willing to share insight and access with the researcher -establishment and maintenance of relationships in research setting in field research is key to being successful
gatekeeper
a person in the field setting who can grant researchers access
in depth interviews
a qualitative method that involves open ended, relatively unstructured questioning in which the interviewer seeks in depth info on the interviewees feelings, experiences and perceptions -similar to survey research but longer and less structured-focus on being thorough -small samples -preplanned outline of topics -look for saturation point-when interview stops yielding new info -get out of the way as much as possible, think about relationship between interviewer and interviewee in critique of research -different form a convo- need consent, on the record, arranged, planned, focus, methodical
unordered close ended
advantages: -providing response categories increases the likelihood that there will be enough people giving any particular answer to be analytically interesting -respondents can more reliably perform the task of answering the question -the researcher can more reliably interpret the meaning of answers disadvantages -response categories may miss something of no interest to pop-need to be mutually exclusive and exhaustive-can be challenging to do
open eded questions
advantages: permits the researcher to obtained unanticipated answers -may more closely describe the views of the respondent -Rs like opportunity to answer some questions in own words -appropriate when a list of possible answers is longer than feasible to present -useful for exploratory purposes disadvantages -many people give rare answers not analytically useful -needs to be coded
When are surveys appropriate?
depends on research problem: -when it lends itself to survey research -when the population is large and you cannot census everybody -when the research questions are clear -when resources are low to moderate, surveys are still quite cost effective compared with other data collection alternatives -when experimental control is low or non existent
respondent effects
frequently an issue of social desirability and need for social approval-tendency of respondents to present themselves as individuals with good social norms and behaviors so they may distort responses because true answers may not be consistent with image the person has of himself -bias toward the dominant belief patters among group to which the respondent feels some identification
histogram
graph plotting values of observation on the horizontal axis with a bar showing how many times each value occurred in data set- a graphical way of presenting a frequency of distribution -important to describe shape of data
surveys and sampling
important to consider response rate
ethical consideration
is participation really voluntary? are the participants truly informed if the identity of researcher is not revealed? confidentiality in reports
kurtosis
measure of how high the peak or hump in distribution is
frequency
number or count of occurrences of a given value or range of values
relative frequency
percent of total cases within a class or group -frequency table -histogram -pie chart -box and whisker plot
qualitative sampling
purposive, snowball, quota-focus is usually on a limited number of cases or settings
qualitative vs quantitative research design
qual- field studies, focus groups, in depth interviews quant-experiments, quasi experiments, survey research, content analysis
mode
relies on frequency so it is only appropriate for nominal data
median
relies on order, only used for ordinal or interval data
ethical issues in surveys
respondents can easily decline to participate and a cover letter explaining survey gives them info to make that decision-little is concealed from respondents in surveys -federal regs protects people -confidentiality is primary concern-many surveys have info that could damage subjects if disclosed-to avoid harm to subjects, researcher must preserve confidentiality
univariate analyses
summarizing data from only one variable -want to know: location along range of values/ central tendency, measure of variability around CT, and shape of distribution
response rate
the percent of respondents from your selected sample who chose to participate- low response can bias your sample
z scores
the translation system which translates any normal distribution to the standard normal distribution
memory effects
two kinds 1. forgetting event entirely 2. telescoping- a compression of time error when the respondent remembers an event occurring more recently than it actually did can be decreased by: -using supplementary devices-length of questions giving added time to think -aided recall- explicitly presenting the possible answers or cues to respondents and asking if they had done or seen any series of things; can result in reduces number of omitted events but does not necessarily reduce telescoping effects -bounded recall- a procedure to eliminate errors in recall due to telescoping-respondents are reminded about their behavior during previous interview and then asked about additional behavior since that time; does not eliminate errors due to omission; the longer the time period being asked about the greater the errors
non response error
two types: -when units of the selected sample do not complete the survey -when survey respondents do not complete the full survey resulting in missing or incomplete data -can introduce systematic bias-is there an underlying reason why they did not participate or fill it out? and result in poorer quality of data
inferential statistics
used to make inference from observations you actually have to the unobserved population from which you sampled -use probability to make inferences -use inferential statistics when we want to draw conclusions -given our observations, what can we say about values not observed, how much error do i expect
mean
uses both order and magnitude, only appropriate for interval and ratio. more sensitive to extreme scores, but preferable to medians
measures of central tendency
value around which cases tend to center, or an estimate of the center of a distribution of values- allows us to describe our data using single estimate -3 ways to describe: mean median mode