COM205 Exam 2 Study Guide

Other Factors Important to an Experiment

1. "Manipulation Checks" a. Assessing the success of our manipulation - do we know if our experimental conditions worked? 2. How we do this depends on the kind of study we do

Understanding Your Data

1. After you have run a study... a. 1st step i. You have to look at your data ii. Examine what the data looks like generally - first glance approach A. What specific information could we glean from looking at the data? B. Example - cleaning your data

Correlations and Values

Diagram in 5/3 lecture

T - Tesy

1. When your hypothesis is focused on differences a. T-tests i. Compare differences between two groups ii. Two different types of t-tests - matched samples and independent samples b. T-tests are widely used i. Simple ii. Easy to understand iii. Easy to explain

Various Statistical Tests

1. Chi Square a. Compare what you expect from your hypothesis to actual responses i. Assess goodness of fit between values you actually observe versus those you expect ii. Used with frequencies iii. Accuracy rates - can tell you more than simply computing averages 2. For when your hypothesis is concerned with associations a. Primarily if you are concerned with how two variables correlate with one another b. Correlation describes the relationship between pairs of variables i. Gives us direction and strength ii. Correlation does not always equal causation

Structure of Questions

1. Closed-Ended vs. Open-Ended a. Closed-ended - answer options are pre-supplied b. Open-ended - answer options are not pre-supplied 2. Closed-ended questions good for questions with simple clear alternatives a. Marital status, political affiliation 3. Open-ended questions good for complex question that defies any readily apparent answer

Quasi-Experiments

1. "Quasi" meaning to be "like" something a. Practical situation - school, consulting 2. Quasi-experiments are like true experiments, but lack random assignment a. Critically different than a true experiment b. Still has some elements i. Examples - control group, some manipulation 3. Result, quasi-experiments are... a. Lower in internal validity - look for threats to internal validity b. Less able to assess cause and effect c. Offer more potential external validity 4. Tightest form of control in terms of establishing cause and effect, then why wouldn't you always do it? - sometimes, it's just not feasible a. Assignment where random assignment isn't possible i. Example - a school says that you can do the research, but they won't let you randomly assign students to attend/not attend an intervention ii. You have to assign one classroom to be a control and another to be the intervention group iii. Perhaps they won't allow you to have a control group at all - just have varied treatment conditions 5. Alternatively, we might get a consulting job to look at data that we didn't collect - we do the best we can a. We can only explore data over time - before a menu change was made and after a menu change was made in a restaurant i. Was it profitable? - we won't always be able to keep things as tight as we would like

Two Types of ANOVA Tests

1. 1 way ANOVA - 1 IV with 2 levels a. You have a group of individuals randomly split into smaller groups and completing different tasks b. Example - you might be studying the effects of tea on weight loss and form 2 groups, green tea and black tea 2. 2 way ANOVA - 2 IVs with multiple levels a. Example - you might want to find out if there is an interaction between income and gender for anxiety level at job interviews b. Gender and Income are the two IVs affecting the DV (anxiety)

Factorial Design Follow-Up

1. Allows us to examine influence of multiple independent variables in one study 2. Allows us to consider if and/or how independent variables may act together to influence the dependent variable(s)

Various Statistical Tests - ANOVA

1. Analysis of Variance (ANOVA) a. Highly used due to its fluidity as a stat test b. Analyzes the differences among group means in a sample - group of psychiatric patients are trying three different therapies (counseling, medication and biofeedback), you want to see if one therapy is better than the others c. Has a similar idea to a t-test

Chi-Square

1. Are my categories distributed evenly or how I expect them to be? 2. Jim collects a sample of 200 people and asks them their political affiliation - republican or democrat a. Jim hypothesizes that there will be more republicans than democrats

Writing the Interview Schedule

1. Are questions... a. Related to the research question? b. Appropriate? c. Clear? d. Directly related to the participant? e. Subject to social desirability? 2. All things to think about in respect to shaping the schedule of questions that you'll be asking... a. Research question - elicit information that can be used to test the research questions/hypotheses b. Appropriate - might use a closed-ended question if a long-winded, in depth response isn't required i. Example - when would you say that you discuss the benefits of donation with a potential donor family, prior to consent, after, or does it just depend on the family? c. Clear - use one idea per question, double-barreled questions i. Ambiguous terms - 'good' 'fair' value-laden judgments d. Personal - controversial or things that may be disapproved by others, including researcher i. Want to remind participants of confidentiality - open with less personal questions and move to these e. Social desirability - be aware of how this might play into your question i. Example - question on racial background & donation 3. Extended Example a. Research Question - "How are the communication strategies used by Organ Procurement Coordinators [OPCs] associated with conversion rates?" i. Interviews with 102 OPCs ii. Mixed-methods approach iii. Structured interview schedule b. Explain basic functioning of OPOs, OPCs, and conversion rates i. Review how coordinators were selected and why interviews were chosen - in-depth information, little prior information on the topic, may vary by organization and geographic location of the country c. Explain why this is mixed methods and why interviews are considered structured

Note About Scales

1. Avoid ambiguity with your statements a. Clear statements to clear response b. Remember that we are trying to gauge all the possibilities of how a person may feel 2. Use clear, formal language that specifies each construct 3. Be careful to not fall into the trap of being too blunt - we want to understand the underlying thought processes that lead to attitudes, behaviors, etc.

Code Unit of Analysis

1. Coding means counting and/or placing our units of analysis into categories 2. We have to train our coders... a. Give them definitions b. Examples c. Pretest d. Test reliability

Assessing Reliability in Content Analysis

1. Coding must be independent 2. Reliability is a function of coder agreement a. High reliability - high agreement b. Low reliability - low agreement 3. Reliability is compromised when... a. Categories are poorly defined i. Not exhaustive or mutually exclusive ii. Too many categories b. Coders are poorly trained 4. Intercoder reliability - extent to which different coders agree what units belong to which categories 5. Intracoder reliability - extent to which a single coder's assignments at Time 1 agree with his/her own assignments at Time 2 6. Reliability should be at least .7

Interviews - Interpersonal Situation

1. Commonplace occurrence, but also critical research tool a. Example - the office 2. Qualitative in nature 3. Could contain open or closed ended questions, depends on topic 4. Interviewer asks the interviewee questions pertaining to RQ/hypothesis 5. Goal is to achieve rapport! 6. Example - Best Interview Ever 7. Interviews are commonplace - see them on television, when applying for colleges/scholarships, etc. 8. Specific method for answering research questions - specific procedure we follow, as we're generally concerned with exploring multiple respondents' answers to our questions of interest

One Method Interviewing

1. Conduct and record structured interviews 2. Transcribe, Verbatim, interview content 3. Two coders use techniques of content analysis to compare and categorize themes 4. Explain how two coders are involved at each step of the process such that a coding schedule doesn't reflect one person's point of view - three people will be involved a. Read transcripts independently b. Compare themes and adjust based on discussion c. Consider new list in comparison to the data - does it work? what needs to be changed? d. Coded for presence/absence of each category - did they do this? did they not do this?

Defining Content Analysis

1. Content analysis is a specific kind of observation with the goal being systematic description of some piece of communication a. Focus - not people but rather the content of something i. Examples - speech, television, show, song, book, conversation, etc.

Types of Nonprobability Sampling

1. Convenience Sample - selected due to accessibility, ease of use a. Example - college students i. COM 101 ii. Most widely used sampling method b. Often appropriate, may or may not represent population at large - you must consider if the convenience sample is worth the risk c. Does not involve random selection - does involve a judgment on behalf of researcher d. In general, probability sampling methods are considered to be more rigorous - nonprobability samples are sometimes more practical e. College students often used because they are easy to gain access to - their use is at times appropriate i. Example - studies of how college students respond to binge drinking campaigns, organ donation, etc. 2. Quota Sampling - sampling with aim of obtaining a specific number of units of a type a. Proportional - represent the proportions of the population by selecting a proportionate amount of units with a particular characteristic i. Example - stand on the street A. 20% of people are under age 18 and 80% are above age 18 in our population B. If we're sampling 100 individuals, we would obtain 20 interviews from those under 18 and 80 from those above age 18 C. If we reach 20 under age 18, we stop and only continue to sample those above age 18 b. Nonproportional - aimed at achieving a minimum number of those with a characteristic, does not have to be proportional i. Example - if we were nonproportionate with our last example A. If we obtained 20 of those under age 18, we might continue to sample B. Ensure that at least 20 are included in the sample, such that their opinions are heard 3. Snowball Sampling - obtain participants with characteristic of interest, ask them to identify others a. This can work for samples that are very difficult to reach b. Special populations such as prison inmates or those with special characteristics c. It is, in fact, one of the only ways to reach difficult populations d. Good for hard to reach samples i. Examples - adolescent bone marrow transplant recipients, transvestite sex workers who are HIV positive

Regressions vs. Correlations

1. Correlation (generally) tells you if a relationship is present a. Do two variables move together? - negatively/positively? 2. Regression describes how an IV is numerically related to the DV a. Estimate one variable on the basis of another - indicates the impact, or change, of one unit (the IV) on another (the DV)

Survey Research

1. Cross-Sectional - describes a sample at a single point in time; "snapshot" a. Most often used in quantitative research 2. Panel - follow same sample over time 3. Cross-Lagged - measure independent and dependent variable at two separate times

Process of Content Analysis

1. Develop a research question 2. Define the population of interest 3. Define the unit of analysis 4. Develop categories for coding 5. Code/Count/Measure units of analysis

Example of a Simple Experiment

1. Door-in-the-Face Effect (DITF) - individuals who reject an initial large request for compliance will be more likely to accept a second smaller request 2. DITF v. Control condition - DV = Donation of $1 3. Two groups, with one post-test observation - Brownstein and Katzev a. People are walking out of the museum and they're randomly assigned to one of two conditions - DITF or small-request only b. DITF is asked - "As you know, these are hard times for public institutions with federal, state and local budget cuts pending, it seems very possible that the Art Museum may lose essential funding. Asking for a donation of $5 - above and beyond the usual request to help out in this time of need." c. Target request gets the first part plus the following - "of this, we are asking for a $1 donation from all those visiting the museum." d. DITF predicts that those who reject the $5 request first will be more likely to give $1, than those who are simply asked to give $1 e. DITF Results i. Chart in 3/29 lecture A. DITF - 86.9% give $1 B. Control - 79.2% give $1 ii. One IV with two levels - type of request and DITF vs. control iii. DV - compliance with $1 donation iv. Manipulated the IV - provided either DITF or control request v. Between-subjects design because each person was assigned to a single category 4. We have random assignment - system is used to randomly assign each person leaving the museum to one group or the other a. Ask the class - is there random selection? NO b. We're simply talking to people who leave a single museum at a specific date and time c. Does not give each unit of our population a random chance of being in the sample - it is a convenience sample

Level of Significance

1. Every study, in every field, typically utilizes a significance level, or what is called alpha a. It is a percent and this percent tells us how certain we are that what our data is showing is meaningful and not random b. Example - 5 percent significance level means we are 95% percent sure that what we are seeing is not random and is meaningful 2. How do we determine our significance, or alpha? a. It is contingent upon the type of study b. Medical studies often need a very small margin of error so they use 1% significant, or 0.01 - this is very low c. Social science typically uses 5% significance or 0.05 in their research i. Not typically life or death so we accept a 5% chance our findings may be incorrect

Factorial Design Examples

1. Example 1 a. "...utilized a 2 (message type) X 2 (message frame) factorial design..." b. 2 Independent Variables i. Each has 2 levels ii. 4 conditions overall c. Chart in 3/29 lecture i. Shows our first IV - message type ii. Two levels - narrative or statistical d. Chart in 3/29 lecture i. Second IV shown here - two levels e. Chart in 3/29 lecture 2. Example 2 a. "...used a 2 (DITF v. control) by 3 (organization v. self v. requester) design to examine the effects of request type and beneficiary on compliance..." i. 2 IVs ii. Request Type = 2 levels iii. Beneficiary = 3 levels iv. 6 conditions in total v. Chart in 3/29 lecture A. If it is a true experiment, individuals will be randomly assigned to condition 3. Example 3 a. Sample quiz/exam question i. "My experiment has a 2 X 3 X 3 design. Tell me A. How many IVs? - 3 B. How many levels for each? - 2,3,3 C. How many conditions overall? - 18 b. If it is a true experiment, individuals will be randomly assigned to condition

Three Ways Interviews May Be Used

1. Exploratory - why do people act the way they do? a. Give example of volunteering study b. We're interested in exploring why volunteers do (or do not) remain with an organization based on integration into the social network c. We have a basic idea i. Examples - team meetings, meetings with management, invitations to social events, etc. ii. We're unsure of what coordinators might actually do to facilitate this process iii. Could be missing something d. We do interviews with a number of coordinators to get ideas about how such processes are handled e. No true direction 2. Main Instrument - solid hypothesis, structured interview with a clear direction a. Know we'll need in-depth information from a group of people b. We use this as the main thrust of our study - talk about Anker and Feeley today 3. Supplement additional methods - do a study and find an unexpected finding, call for interview to discover the reason for the finding a. Example - find that certain strategies used in Anker and Feeley result in less likelihood of consent to donation i. Follow up interviews with families to determine if they've been approached with such a strategy and what their reactions were

Two Roles of Moderator

1. Expressive Role - sees focus group moderator as attending to the emotional needs and dynamics of the group 2. Instrumental Role - sees focus group moderator attending to the goals of the experiment a. More rigid b. Encourages debate and discussion c. Not as concerned about emotional needs of the group

Sampling

1. External Validity - representativeness of the sample, how well does it reflect the population? 2. Greater representativeness = greater confidence in our results 3. If our sample is externally valid, we can have greater assurance that it truly does represent the population at large 4. Example - let's say that we want to know the average number of friends a UB student (who uses Facebook) has on the site a. Parameter vs. Statistic i. Parameter - having to do with population ii. Statistic - having to do with sample (more widely used) iii. Diagram in 4/26 lecture b. Population is unknown - we are unsure of how many UB students use Facebook (although potentially, Facebook could tell us) c. Our population is all UB students - not all students everywhere d. We decide to sit outside the student union and we ask each third student who passes by "Do you use Facebook?" "How many friends to you have on Facebook?" - we do this until we sample 250 students e. Students provide us with responses (200, 400, 751, 255....311) f. We average these responses in order to determine the average number of students - this is a statistic and provides us with an estimate of the value in the population g. If we do this study multiple times, we'll estimate the population parameter, but we'll likely not be exact, as the population parameter is unknown 2. Normal Curve Distribution a. Diagram in 4/26 lecture b. If we could do this an infinite number of times.... c. Let's say that for each study, we graph the frequency of the # of friends i. We have a bar going to 311, to 298, to 305, etc. (show on board) ii. If we could do this an infinite number of times, we end up with the shape of the normal curve

Factorial Designs of Experiments

1. Factorial designs are for experiments with two independent (or more) variables 2. Variables are crossed so that different categories are created 3. We look for main effects - impact of each IV on the DV 4. We look for interactions - impact of the combination of the IVs on the DV

Notating Factorial Designs

1. Factorial designs are notated with numbers indicating how many IVs you have with how many levels each IV has 2. Example - 2 x 2 design has 2 independent variables, each with 2 levels a. Each number represents an IV, and the value of the number indicates the number of levels in the IV 3. Example of a Factorial Design a. You are doing a study assessing the effects of ideology and violent content of television on aggressive behavior b. Independent Variables i. Ideology, 2 levels - republican, democrat ii. Violent content, 2 levels - violent, non-violent c. You have a 2 x 2 factorial design 4. Factorial Design Tricks and Tips a. Count the number of numbers to determine how many IVs were used b. Value of each number tells you how many levels an IV has c. Multiply numbers to determine how many conditions were employed

Field Notes

1. Field notes - written record (or audio/visual) of the researcher's observations and experiences in the field a. Mental notes - not written; memory based recording of an event i. Not just free recall - researcher attempts to make a "freeze frame" of a specific event b. Jotted notes - brief, shorthand notes while in the field that are enhanced later

What Does a Factorial Design Tell Us?

1. Main effect shows us differences between the levels of each independent variable a. Are republicans and democrats different in aggression scores? b. Are viewers of non-violent vs. violent content different in aggression scores? 2. Interaction effect shows us whether there are differences between the different combinations of the two independent variables a. Are republicans and democrats who are both exposed to violent content different on their aggression scores?

Focus Groups

1. Focus Groups - multiple individuals interviewed simultaneously, interviewees relate to one another a. Must be small enough so that all respondents can be heard b. Large enough to be generalizable - externally more valid 2. Generally used to generate future research - useful in understanding advertisement, product placement, people's reactions to a manipulation 3. Led by a moderator a. Keep group on-task b. Follow schedule of questions 4. https://www.youtube.com/watch?v=Sx1J3S6vUJ8 5. Allows us to hear diverse viewpoints on a specific topic a. Must be small enough such that all respondents can be heard, but large enough to achieve diversity in response b. Sometimes an open call for participants, other times, consists of a specific group of individuals i. Example - transplant surgeons 6. Why do this as opposed to interviews? - allows participants to relate to one another, and thus, might uncover something that we haven't heard before a. "That's a good point" "I actually disagree with that." b. Might not have come up in an interview setting c. More exploratory in nature 7. Relatively low cost 8. Increases diversity in response - some may dominate a. Certain individuals opinions may be heard more b. This is up to the moderator to control 9. Not standardized - difficult to compare 10. Low costs as compared to interviews - high cost compared to an online survey a. May be higher in cost if you're paying professionals for their time b. Example - colon cancer paper, transplant surgeon paper 11. Idea is for people to relate to one another, but sometimes, a single individual will dominate over the discussion - skilled moderator should be able to draw others into the conversation, "Ashley, we haven't heard from you." 12. Hard to compare as various issues may come up within different focus groups - focus groups may not be very representative

Experiments

1. High internal validity - helps assess cause and effect a. Why? - more control over extraneous influences 2. Lower external validity - more difficult to make generalizations to population a. Because of control b. Creates an 'artificial' environment 3. Why high internal validity? a. Through random assignment, we assume that our groups are probabilistically equivalent - we know that groups will differ only a specified level of chance i. Using a control group eliminates a lot of the single groups threats to internal validity that we talked about last time A. Examples - history, maturation, testing, mortality, etc. ii. We have a second group present that does not get the treatment - control iii. Such conditions help with establishing cause and effect - recall that the more threats to internal validity we can eliminate, the greater our confidence that something we did caused the associated outcome 4. External Validity - try to keep everything quite controlled in an experiment a. We want to know that our cause resulted in the effect b. Drawback of this is that it can be more difficult to generalize from experiments - if something is observed in a tightly controlled setting, will it still be observed in the field where other variables are at play? 5. Book - have a signal to noise ratio a. Signal is the program or treatment, while the noise is all of the other factors (examples - distractions, mood, etc.) that might influence the DV b. Our job is to amp up the signal (maximize the variance due to the IV), while reducing the noise c. Draw the curves on the board showing variance due to between and within subjects factors

If Your Data Are...

1. IV is nominal and your DV is simple frequency counts... then chi-square 2. IV is nominal and your DV is means... you are trying to determine if two groups scored differently a. Independent samples t-test or matched b. Difference hypotheses 3. IV is continuous and your DV is continuous - a mean/value associated with the various levels a. Regression/correlation b. Continuous hypotheses

Rejecting the Null Hypothesis

1. In most research paradigms, we want to reject the null hypothesis and accept the alternative - how do we do this? 2. We must establish a criterion that, if met, will tell us that we can accept the alternative and reject the null hypothesis 3. We call this threshold our region of rejection - largely determined by our significant level

What statistics do I use?

1. It depends on your data 2. Each statistical test is designed to test a specific null hypothesis a. Results of statistical test give us a probability b. Probability of this statistical value assuming the null is true 3. What level of measure are your data? And how is it structured? - this will tell you what test to use

Non-Experimental Research

1. Lack experimental manipulation and random assignment 2. Common with large scale issues a. Example - poverty, we cannot randomly assign people to be wealthy or poor 3. Big weakness - lack of control, no random assignment 4. Like the quasi experiment, this is a compromise design a. With the experiment, we had random assignment, control group, and manipulation of independent variables b. With the quasi experiment, we had no random assignment, but a control group c. Here, we have no random assignment, no experimental manipulation, and no control group 5. Example - Elbert, 1993 a. Comparison of ADHD with hyperactivity and without hyperactivity on school achievement i. The idea is to see if these children differ in terms of school achievement b. Check groups for equivalence on other variables i. Examples - age, mother's education, IQ A. Even though we don't have random assignment, as with the quasi-experiment, we can consider whether the groups are similar on other variables that may be associated with the DV c. ADD+H perform worse than ADD-H children d. Two features - cannot manipulate the IV (cannot choose who is or is not hyperactive) and cannot randomly assign people to the +/- H groups

Things To Think About

1. Mechanism of delivery (examples - online, paper-pencil) - access, data entry, response rate? 2. Sensitivity of topics - place sensitive topics later to build trust a. Something to consider if you're looking to study relationships b. It's very similar to a first date - you need to work your way to the important, delicate topics 3. Fatigue - are participants fading out? 4. Write clear questions - definitions should be provided a. Example - "do you engage in binge drinking?" i. Define binge drinking 5. Avoid bias in wording a. Example - "many people think parking on campus is a problem - don't you?" 6. Are questions related to your topic? 7. Avoid "double-barreled" questions a. Example - "do you think that parking on campus is a problem and that administration should do something about it?"

Characteristics of Field Research

1. Mostly a qualitative endeavor 2. Aligned more with idiographic causal model 3. Inductive style of research 4. Data collection and analysis - less distinct/separate

Null vs. Alternative

1. Null hypothesis - there is no differences or change in direction a. Example - Asians and Caucasians will not differ in response time to neutral stimuli 2. Alternative a. Our directional or nondirectional hypothesis - essentially the opposite of the null b. Example - Asians will be significantly faster than Caucasians when responding to neutral stimuli

Two Critical Points

1. Number of people - if we are making assumptions about entire populations a. We need enough people to justify those assumptions i. Example - Game Changers and others 2. Who are we sampling? - if our sample does not accurately reflect the population as it should...that's an issue

Field Research

1. Observe subject/participants in their natural setting 2. Assumption - observation in labs are "artificial" while field research is more "natural" 3. WWYD a. Not necessarily a valid assumption - Mook, 1983 4. Reactivity - change in individual or group behavior that is due to a researcher's presence a. https://www.youtube.com/watch?v=EEwCWR5Vkpw b. https://www.dnb.com/ie/perspectives/master-data/how-the-observer-effect-impacts-data-science.html c. "Hawthorne Effect" - what is it? 5. How do we get access to participants in the field? a. Gatekeepers - people who can help or limit researchers' access to the field i. Example - spotlight b. What about when you can't get access? i. Sometimes you can create your own 'field' to study in - SPE 6. Gatekeepers for various participants a. Workers - Boss, Supervisor b. Students - Teachers, school administrators c. Patients - Doctors, hospitals d. Citizens - Community leaders 7. Subjectivity and objectivity in the field a. Diagram in 4/5 lecture

Types of Questions

1. Open-Ended Questions - allows respondents to shape answers in their own words a. Qualitative in Nature 2. Diagram in 3/22 lecture 3. Table in 3/22 Lecture 4. Closed vs. Open-Ended Questions Closed-ended options - even or odd? a. "Clowns are scary" - even i. Strongly agree ii. Agree iii. Disagree iv. Strongly disagree b. "Clowns are scary" - odd i. Strongly agree ii. Agree iii. Slightly agree iv. Neither agree nor disagree v. Slightly disagree vi. Disagree vii. Strongly disagree 5. Dichotomous - only two response options a. Typically demographic questions b. Diagram in 3/22 lecture 6. Nominal/Categorical Response - categories do not indicate more or less of something, arbitrary values a. Diagram in 3/22 lecture 7. Interval Level - size between labels is meaningful, includes likert-type and semantic differential type response scale a. Diagram in 3/22 lecture 8. Filter Questions - used when one can only answer a question based on his/her response to a prior question a. Diagram in 3/22 lecture 9. Single Option Response - respondent can select only one response a. Diagram in 3/22 lecture 10. Multi-Option Response - respondent can select multiple answers to a single question, "check all that apply" a. Diagram in 3/22 lecture

Correlations

1. Output of a correlation typically is given between -1 and 1 a. Positive number would indicate that as one variable increases, or decreases, the other increases, or decreases b. Negative number indicates that as one variable increases the other decreases c. Closer the number is to -1, or 1, the stronger the relationship i. Example - -0.8 is a strong negative relationship ii. Closer the number is to 0, the weaker the relationship

Key Terms

1. Population - every unit of a type that you wish to study; Exact number often unknown a. Example - every commercial sex worker in the United States i. Likely total number is unknown - there is no place that we can go to look this up b. Example - every organ procurement organization in the United States i. We know that there are exactly 54 of these - can be people or objects, often used in communication research when doing content analysis c. Especially with people, it is often difficult to know the size of the population i. We rarely, if ever, do a census - we know that there are 54 OPOs in the US ii. Let's say that on each of these OPOs, we obtain data on whether they are based in a hospital, or independently operated iii. Presume that type of operation influences their consent rate - number consenting to donation based out of the number asked iv. If, however, we couldn't obtain data from all of these organizations, we would have to choose a sample - perhaps we could look at 25 organizations to estimate the effect in the population 2. Census - study of every member of a population 3. Sample - study of a segment of a population, presumed to represent that population

Types of Sampling

1. Probability Sampling - utilizes some form of random selection, how we get our participants 2. Non-Probability Sampling - no random selection, judgment by researcher

Rules of Survey Designs

1. Questions posed should be clear in meaning and free of ambiguity a. Bad - "Do you exercise regularly?" b. Bad - "What is your annual income?" c. Better - "How many days do you exercise per week?" d. Better - "What is your annual net income from your primary occupation?" 2. Questions should use common everyday language a. Avoid jargon b. Avoid specialized language c. Avoid abbreviations d. Bad - "How do you feel about GM food?" e. Better - "How do you feel about genetically modified food?" 3. Questions should use neutral language a. Avoid emotional language and leading questions b. Bad -"What is most offensive about flag burning?" c. Bad - "Why is hitting children wrong?" d. Donald Trump Survey - https://gop.com/mainstream-media-accountability-survey/ 4. Avoid universal language a. Bad - "Do you always eat breakfast?" b. Better - "How many days do you eat breakfast per week?"

Terms to Know

1. Random Assignment - process of assigning participants to two or more subgroups by chance, true experiments must have random assignment 2. Control Group - does not receive treatment/manipulation 3. Experiment - manipulate one variable (IV) to see if changes in another variable (DV) result 4. Random assignment is not random selection a. Random selection allows us to select units from our population by chance b. This has to do with once we have those units, how are they assigned to various conditions? 5. Control Group - group should be similar to the experimental group a. Ideally, due to random assignment 6. Simplest form of experiment shown here - subjects are randomly assigned to two groups a. One group receives a treatment and the other acts as a control b. Control group could also receive the "standard" treatment

Extended Example

1. Recall that qualitative data can be content coded - inter-coder reliability applies here a. Table in 3/29 lecture b. Explain how data is coded in rows and used to predict conversion rates - each OPC assigned his/her organization's conversion rate 2. Purpose - understand students' evaluations of organ donation videos a. Group of undergraduates in focus group b. Reactions to the following videos 3. "Wasted Value;" "Organ Man;" "Rachel" 4. Moderator asks specific questions 5. "What did you like about this video, if anything?" 6. "What did you dislike about this video, if anything?" 7. "Which video did you find most effective?" 8. "Do you have any recommendations for how to create videos that appeal to college students?" 9. Why used - exploratory in terms of understanding tactics that did or did not work, used to shape future videos, rather than to answer a specific research hypothesis

What does it all mean?

1. Remember back to our null and alternative hypotheses? a. Null finding would mean we did not find significant results - our results exceeded 0.05 b. Significant finding means we can reject the null hypothesis and accept the alternative hypothesis - our number was less than 0.05 c. This means that whatever manipulation we introduced was significant and meaningful 2. If your results are greater than 0.05... P > 0.05

Research Methods

1. Research Methods - tools for answering questions 2. Field research is a unique tool of research methodology 3. Field Research - collecting data outside the laboratory; interacting with subject of interest in its/their natural setting a. Most widely used technique by 'non-scientists'

Define the Population of Interest

1. Research Population - total collection of the people, places or things we are studying a. For people i. Students at Canisius ii. Students at UB b. For content analysis - what is the whole of what we want to study? i. All issues of Elle magazine from 2000-2017 ii. All newspapers from 2012-2017

Practice

1. Researcher takes a sample of 100 UB students and determines the mean number of friends they have on Facebook is 100. Standard deviation is 5. Between which two values will 95% of the scores in the sample fall? a. 90-110 - we know that 95% of cases are within two standard deviations, so we move two standard deviations on each side of the mean 2. Ashley asks students in her class to report how many hours they spend watching TV each week. She finds they watch an average of 10 hours (SD = 2). Between which two values will 68% of the students fall? a. 8-12 hours - move one standard deviation on each side of the curve

Things to Know for Surveys

1. Response Rate - percentage of contacted respondents who complete surveys a. Acceptable level? b. As low as 20% or even lower has been shown to be adequate c. Sampling is more important than response rate d. Nonresponse bias - individuals who agree to participate are systematically different than those who choose not to participate e. http://www.pewresearch.org/2017/05/15/what-low-response-rates-mean-for-telephone-surveys/

Measures

1. Scales - measure concepts (example - attitudes), results in a numerical score to represent the concept a. Collection of individual items b. Each item is supposed to measure the same construct/concept c. Scores for items are averaged, or added together, creating an index/composite score 2. Simple definition of a scale - "assignment of objects to numbers according to a rule" 3. Show immigration scale - each statement is assigned a numerical value; In this case, items become more restrictive a. Example - if you agree that you would let immigrants live in your neighborhood, you would also agree that immigrants could live in the country 4. Scaling - results from a process of assigning numbers to objects a. Each item on a scale has a scale value b. Refers to a set of items 5. Response Scale - used to collect the response for the individual item a. Item not associated with a scale value b. Used for a single item

Rosenberg Self-Esteem Scale

1. Self-esteem - overall evaluation of one's worth of value 2. Below is a list of statements dealing with your general feelings about yourself - if you strongly agree, circle SA, if you agree with the statement circle A, if you disagree, circle D, if you strongly disagree, circle SD a. Table in 3/22 lecture b. Table in 3/22 lecture 3. For items, 1, 2, 4, 6, and 7 - SA = 3, A = 2, D = 1, SD = 0 a. Agreement with these items indicates greater self-esteem 4. For items 3, 5, 8, 9, and 10 - SA = 0, A = 1, D = 2, SD = 3 a. These are what is called reversed-scored items - agreement with these items indicates less self-esteem i. Other reason(s) for reverse-scoring? 5. Add or average all items together

Types of Probability Sampling

1. Simple Random Sampling - choose n units out of sampling frame N, such that each has an equal chance of being selected a. Use computer to generate random number for each member of N b. Sort in chronological order based on random number assigned to each unit of n c. Select first n units from chronological list d. Example - study on how satisfied communication students are with their employment after graduation, don't have access to all communication graduates, ever i. Sampling frame consists of the last two year's graduates - number is 850 ii. Want to choose 200 out of 850 - which 200 do we choose? randomly number all 850, choose first 200 iii. Perhaps Excel example? 2. Stratified Random Sampling - take a simple random sample from each of N homogenous subgroups a. Divide N into groups based on characteristic of interest b. Take simple random sample from each to meet set quota c. Example - study on how UB should update their menu for restaurants in the student union i. Simple random sampling of all current undergraduates, we may not get very many vegetarians, or those on a gluten-free diet in our sample ii. Use stratified random sampling when we want to ensure that minority groups are represented - ensures that specific groups will not be missed iii. Often used to make sure that certain racial/ethnic groups are represented in a study 3. Systematic Random Sampling - random determination of where sampling will begin, then sample every Xth unit a. Think back to my example of DITF recruitment at UB b. We're determining a random starting point in our sampling frame and then taking every Xth unit i. Why do this instead of simple random sampling? - best if you have no way of directly going to #x in a sample, as simple random sampling would require c. Example - if you were in registration and records and had filing cabinets full of student records i. They're not numbered, so if you drew #12,514, what would you do? count to get there? d. Example - study of GPA i. We can assume that files are randomly ordered and we know the total number or approximate # of files we have ii. Systematic random sampling would be more appropriate

Don't Fear the Stat

1. Stats can be intimidating for many 2. They provide a real value and should be appreciated 3. Why Love Stats

Activity - Scales

1. Step 1 - determine/carefully phrase or focus of the scale a. What if we were interested in measuring attitudes toward the "Invisible Boyfriend/Girlfriend" Service? b. Learning more i. https://invisibleboyfriend.com/ ii. https://invisiblegirlfriend.com/ c. Our focus - "what are college students' attitudes toward the invisible boyfriend/girlfriend service?" 2. Step 2 - generating items a. Who? - we need to engage the opinions of those who actively date such as college students b. How? - design statements that reflect what we are trying to measure

T - Test, "Beer Stat"

1. T-statistic was created in 1908 by William Sealy Gosset 2. Gosset was a chemist at Guinness brewery 3. Gosset wanted to be able to monitor and compare the quality of the stouts Guinness was producing 4. Guinness prevented its scientists from publishing their research so Gosset published the formulae and paper under the pseudonym "Student"

How Do Interviews Compare to Questionnaires?

1. Table in 3/29 lecture 2. Comparison is between surveys (self administered) and in-depth qualitative interviews a. Book also considers interviews to be situations in which an interviewer asks a series of close-ended questions to an interviewee 3. Talk about costs associated with dissertation research a. Time, money, skills - what if we trained others? b. Manpower - interpret surveys yourself, require another individual for coding associated with a survey

Why do statistics?

1. To test our hypotheses, statistics provide best empirical evidence of effects 2. Mostly objective... a. If you do them according to best practices, highly objective n. If you don't... well

Type of Statisitcs

1. Two main types a. Descriptive i. Describes the data - mean, SD, range, etc. ii. Normal Curve b. Inferential - make inferences or draw concrete conclusions about the data i. More powerful - correlations, regressions, t-tests, etc. c. Nothing is ever proven - strongly support, or conclude with great confidence i. Why?

Two Types of Correlations

1. Types are contingent upon the variable you are using (nominal, ordinal, etc.) - both test strengths of relationship 2. Pearson correlation - variables must be interval or ratio 3. Spearman correlation - used for ordinal variables, concerned with ranking variables only

Extended Example of a Theme

1. What might data look like? - [Querying the Patient's Character] a. 'Was your loved one a caring, giving person?' You know - 'Would they do something for anybody? Give them the shirt off their back? If so, do you not think that they would want to save a life - you know - through organ donation?' (Interview 35) b. 'And once I have a good, solid knowledge of the patient - what kind of person they were or weren't....and I feel like - that - I gain the trust of the family by just talking about their loved one's life...then, I'll mention donation' (Interview 109) c. 'I ask them what kind of family member this person was - you know - were they giving? Were they - um, willing to help out others? You know? Just tell me about them. Tell me - tell me what they were like' (Interview 114) 2. Explain theme and representativeness of quotes

Using Significance to Interpret Results

1. When you calculate results you typically get an output of a number less than 1 (.879) a. To be significant the output MUST be less than our alpha - significance level b. Example - 0.05 alpha and our results are 0.032 i. These are significant results and mean something ii. If the number exceeds 0.05 they are not significant (0.10)

Normal Curve

1. Understanding the normal curve helps you understand where scores in your study should fall a. Where they should not? b. Which scores should be examine more closely? 2. If we want a cross-section of the general population of interest, then we want things (most times) to be on the normal curve 3. Standard Deviation - spread of scores across a sample a. One single sample b. Helps us to understand the differences that would emerge as a result of individuality 4. Diagram in 4/26 lecture a. Sampling distribution is the spread of the scores pulled from the samples b. Let's say that we do our study and we start getting responses - 311, 258, 322, 100.... c. We can calculate a standard deviation for that single study that describes how the scores are distributed around the mean d. Get an average in our study of 311 friends, we would know that the person who had only 100 friends was very far away from the mean and the person who had 258 friends was very close to the mean - this is your standard deviation, because it occurs in the sample e. We take an infinite number of samples - we can't actually do that f. What we do is use our sample to make inferences about our population g. If we did only a single study and we found our average number of friends to be 311, then we would calculate the standard deviation in our sample h. Based on standard deviation and sample size, we can estimate the standard error, which tells us how scores are spread around the sampling distribution - estimating for your population 5. The mean (average) is the mid-point 6. One standard unit away from the mean = 68% of cases 7. Two standard units away from the mean = 95% of cases 8. Three standard units away from the mean = 99% of cases 9. Standard unit can be standard deviation (sample) or standard error (estimates population)

Define Unit of Analysis

1. Unit of analysis - what are you interested in? a. Could be individual people or groups of people b. Could be each page of magazine, each section of magazine, or the magazine as a whole 2. What am I counting or categorizing? - think back to our health campaign examples

Types of Interviews

1. Unstructured - general guiding questions, but much flexibility a. Utilize an 'interview guide' b. Used primarily with exploratory research c. Example - how do women tell their partners they're infected with an STD? i. If no one else has explored this topic, you may need to let the interview be flexible - generally exploring this issue 2. Structured - specific questions, less flexibility, may ask probing questions a. Utilize an 'interview schedule' b. Probing - "Tell me more...", "You mentioned that...", Get more information on a topic, "Did you do anything else?" c. More common and makes comparison easier between participants

Matched Samples (Paired) T-Test

1. Used when you are comparing within groups at two different times a. Pre-test versus post-test - exam 1 vs. exam 2) b. Typically used within subjects design c. Compare participant at time 1 vs. time 2 2. Used a lot when exposing participants to some manipulation and then measuring their responses to that manipulation

Independent Samples T-Test

1. Utilized when comparing two randomly assigned groups to one another 2. Comparing two groups to see if there are significant differences in their average responses a. Example - men vs. women in scores of emotional competency

Regressions

1. Very similar to a correlation but can be used to predict scores if the values are significant 2. Creates a "best-fit" line that allows you to make predictions about future scores you will see 3. Powerful because you can make future predictions about the population you are studying

Why have people content analyzed?

1. Violent content in video games 2. Offensive acts in daytime television 3. Speech patterns of people in conversation 4. Misogynistic presentation in music videos 5. Types of play in close relationships

Why Sampling Matters

1. We use a sample to draw conclusions about the population 2. Let's take a moderately well-known phenomenon - is being Vegan/plant-based the most effective diet out there? a. We would look to the research - what are some immediate considerations/concerns? 3. Check out the gamechangers documentary and the debates on JR podcast

Interview Schedule Excerpt

1. What are the most common reasons that you hear from potential donor families in terms of why they would not want to consent to donation? 2. How, if at all, do you address the benefits of donation with a family? - at what point in the discussion would you do that, before or after obtaining consent, or does it just depend? 3. When communicating with a potential donor family, what is the most important information that you try to provide donor families with in making your request? 4. Structured - all receive the same questions, in the same order, probing questions used as necessary a. Let's consider the characteristics of the schedule - do they relate to the research question? b. Appropriate? - Yes, while it might be personal for families, this is related to their jobs i. Expect that it likely won't be too emotionally trying for them to speak about it - ethical stuff in place if it does become problematic c. Clear questions i. Question #3 could be more clear - use of 'if at all' makes the question less leading, it doesn't assume that the coordinator speaks about benefits with donation with a family, and indeed, some of them do not ii. Question #4 had to be changed based on pilot testing - "What is the most important thing about your interaction with potential donor families?" A. Resulted in information about medical management, which is not what we were interested in, thus, question was re-shaped accordingly d. Social Desirability - not too much of a concern here, although may be if you were working with general members of the population

Develop a Research Question

1. What do you want to study? a. Make the question worthy of study b. Question is generally informed by theory 2. Example - how many supportive vs. critical stories related to government action are present in newspapers?

Rapport

1. What is rapport? - state of mutual understanding between two or more communicators 2. Example - interviewer and interviewee a. Common ground is established b. Sometimes associated with 'liking' c. Can occur nonverbally, verbally, or both d. Is an ENGRAINED human communication skill - https://www.youtube.com/watch?v=apzXGEbZht0 3. Critical for eliciting the most truthful responses a. Great value for researchers - also a slippery slope

Develop Categories for Coding

1. What kind of 'offensive acts' are performed on daytime television? a. Define 'offensive act' - list should be exhaustive and mutually exclusive i. Swearing ii. Violence iii. Sexual behaviors iv. Derogatory statements

Experiment

1. Why do we do experiments? a. To examine causality... b. Can X cause Y? 2. Determining causality - we must isolate and control one variable to observe the effect of this isolation and control on another 3. Experiment - controlled study in which the independent variable is intentionally manipulated so as to observe its impact on the dependent variable a. For a study to be an experiment, it must... i. Have at least one variable that is manipulated by the researcher A. Example - researcher has control over the levels of the variable ii. Have random assignment to the levels of the manipulated variable 4. Experiments Involve a. Random Assignment to experimental and control conditions b. Environments in various conditions that are similar in every respect c. Manipulation of an independent variable d. Control group that receives no treatment e. Measurement of the dependent variable

Surveys

1. Wording of questions can have substantial effects on data acquired 2. Agree-disagree format a. "Best way to achieve peace is through military strength" - 55% agree, 42% disagree 3. Forced choice a. "Best way to achieve peace is through military strength" - 33% b. "Diplomacy is the best way to achieve peace" - 55% 4. Wording of questions or items can have substantial effects on data acquired a. In telephone interviews, the options read last are more likely to be chosen than the options read first (recency effect) i. Recency effect - stronger for older adults as compared to younger adults b. Decoy Effect - https://www.youtube.com/watch?v=33aaQdtD20k 5. There are things we need to be aware of - what are some rules of designing our items and how do we implement them?

Pros and Cons of Field Research

1. Work has the potential to shape our view of the world 2. Unique perspective on difficult to study populations and phenomena a. Emotional Development in Adolescence 3. Real-time, rich data 4. Can ultimately re-define our understanding of scientific principles a. Example - evolutionary hardwired for violence 5. Subjective bias can alter/influence data collection 6. Unethical behavior 7. Not much room to assess causality 8. 'Reactionary effect' 9. 'Rise and fall' of one of the most groundbreaking field studies ever conducted a. Issues b. Too invested?