HDFS 250 Exam 3

Things to Consider

How to recruit a sample? Aim for the best possible If a convenience sample, recruit strategically Laboratory setting vs. natural setting? Test participants individually vs. in groups?

Mean, Median, Mode

How much, on average, scores differ from the mean How spread out are the scores? A "majority" 68% of scores will be within the range of -1SD to +1 SD around the mean. Almost all scores (95%) will be within the range of -2 SD to +2SD around the mean.

Things to consider

How to operationally define the DV? Want high reliability and validity Also, want variation in scores

The Two-Group Experimental Design: Terminology

Reporting two-group mean comparisons (Chp 8) All calculated values in statistics are italicized t# = #.##,p=.##,d=.## t - test symbol # - degrees of freedom (df) #.## t score p value -significance level less than .05 is significant .## Calculated effect size

Administering Surveys

Survey methods In-person, phone, paper survey, online survey, etc. Target population should determine the method Response rate? Need to consider non-response bias Incentives Coercive incentives are unethical

Technology

T tests for independent means (or paired t tests for matched pairs Compares average scores on the DV between the two groups Sees if scores on the DV are different

more complex designs

Two group design (Chp 8) 2 groups Multi group design (Chp 9) 3 or more groups Within subjects design (Chp 10) Same participants are measured repeatedly on the DV Factorial design (Chp 11) Multiple IVs Mixed design (Chp 12) Combination of different designs

Two group vs multigroup

Two groups 1experimental + 1 control 1Treatment + 1 placebo Multiple groups 2experimental + 1control __ Treatment + __ placebo __experimental

Multigroup design advantages

Isolating confounding factors Discovering more complex relationships

Types of validity

*Face validity* How much the items in a scale appear to measure the concept of interest For example, the item "how anxious do you feel?" on an anxiety scale would have high face validity Low face validity may reduce biases Can include distractor items to disguise the purpose of a survey *Content validity* The degree to which a scale measures all relevant material about a concept For example, exams should cover much of what was learned in class *Construct validity * how well a scale actually measures the construct or concept, it is supposed to Convergent validity Divergent validity *Convergent validity*: a scale should correlate with other scales that measure similar things For example, a new scale for measuring depression: Should correlate with other depression scales *Divergent validity*: a scale should not correlate with scales that measure unrelated things For example, a new scale for measuring depression: Should not correlate with a vocabulary test

Types of Reliability

*Inter-Rater Reliability* Agreement between two observers can be examined statistically Cohen's kappa coefficient Ranges from 0 (no agreement) to 1 (agreement) 0.7 is the best *Test-retest reliability* How well a test produces similar measurements when taken at different times Can look at correlations between scores at different times Can be biased due to Practice Maturation *Alternative - form or parallel - form reliability* Participants can complete two different scales that measure the same concept If reliable, scores on the two scales should be highly correlated *Internal consistency reliability* Individual items within a scale should be highly correlated if they are measuring the same thing

The Two-Group Experimental Design (Review)

1 group The group/condition that gets the key treatment in an experiment 1 group Any group/condition that serves as a comparison in an experiment An independent variable IV A stimulus or aspect of the environment that the experimenter A dependent variable DV Response or behavior that is measured

What is wrong with this item

1. Belonging to a Greek organization increases my chances of being successful. _ Wording is not clear what does being successful mean to people 2. Joining a fraternity or sorority will help me achieve my academic goals and meet others with similar goals. -Double - barreled item 3. Rushing a fraternity or sorority will help me make friends. - Not clear what rushing means and what constitutes that 4. Fraternities and sororities are for obtuse students who distinctly lack erudite qualities. - Complex vocabulary and

Running a two group design

1. Choose an IV and operationally define 2. Choose a DV and operationally define 3. Choose a sample 4. Divide the sample into experimental and control groups 6. Present the IV manipulation in the experimental group 7. Measure the DV in both groups 8. Compare and analyze the two sets of DV measurements

What is a survey?

A research strategy for collecting information from a group of individuals Self-report Can contain quantitative or qualitative questions Emphasis is usually quantitative Especially useful in correlational studies

A response set

A response bias where participants give the same answer to most scale items Acquiescent response set Agreeing with most items Can avoid by including reverse-coded items Items that are worded in the opposite direction

Developing a

A standardized set of procedures Everyone gets the same instructions Materials are the same Structured script to follow The researcher should keep notes

Comparison

1. Was spring break fun? (open-ended) 2. Was spring break fun? Strongly agree Agree Neutral Disagree Strongly disagree

Experimental Designs Completely Randomized Design

Each experimental unit is randomloly assigned to a random groupto receive a different treatment Experimental units are usually participants

Two group vs multigroup design

A two group design can tell you whether your IV has an effect If no previous experimental studies have been done on your topic, you should consider a two-group (presence vs. absence) study A multiple group design is appropriate when you have the answer to your basic question, and wish to go further

multigroup design

Also known as a multi group design How many groups? Three or more levels or amounts of an IV Often a control group and two or more experimental groups Control group may be Empty / no treatment group Placebo group

When Writing Questions

Ask: Is this a question that... Will mean the same thing to everyone? People can answer? People will be willing to answer? People will understand? Aim for 8th-grade reading level Wording matters a lot

Sample size want

At least 30 participants per experimental condition / group

The Two-Group Experimental Design: Terminology

Effect size Measure of magnitude of the difference between groups Gives a standardized way to compare across studies

Types of control groups

Empty control group Group receives no manipulation or treatment Is only measured on the DV Shows how participants respond under normal conditions Placebo group Group believes they are getting treatment but are really not Controls for placebo effect

Principals of writing questions part 3

Bad: What was your total income from all sources in 2017? _____________ Total income for 2017 Better: Which category best describes your total income from all sources in 2017? $10,000. or less $10,001. to $20,000. $20,001. to $35,000. $35,001. to $50,000. $50,001. or above Include an I don't know or no opinion option Example: Which of the following increases the chance of having a heart attack? Smoking: [ ] Yes [ ] No [ ] Don't know Being overweight: [ ] Yes [ ] No [ ] Don't know Stress: [ ] Yes [ ] No [ ] Don't know Be careful of evaluative language The wording of questions should not imply liking, approval, disliking, or disapproval Bad: Do you agree with pro-life people about abortion, even though their views are ignorant? Bad: Why do you think people dislike this brand? Make sure questions match the response options Bad: Have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very Often Better: How often have you had pain in the last week? [ ] Never [ ] Seldom [ ] Often [ ] Very Often

Principals of writing questions part 2

Be careful of double-barreled questions Two separate questions rolled into one Bad: How often do you drink coffee or tea do you drink in a day? Better to separate the question into two 1) How often do you drink coffee during a typical day? 2) How often do you drink tea do you drink during a typical day? Avoid hidden assumptions, by making sure to accommodate all possible answers Bad: What brand of computer do you own? (A) IBM PC (B) Apple Better: What brand of computer do you own? (Circle all that apply) Do not own computer IBM PC Apple Other Soften the impact of sensitive questions Bad: "Have you ever shoplifted something from a store?" Better: "Have you ever taken something from a store without paying for it?"

Can Ask Questions aboutq

Behaviors Feelings Knowledge Beliefs and attitudes Ways to ask Oral interviews Mail Online: Qualtrics SurveyMonkey...

Steps in Developing a scale

Brainstorm questions Revise to increase clarity Get feedback on a preliminary draft Pilot testing Create a shorter final draft Collect data on reliability and validity

Summated Ratings Scale, AKA Likert Scale

Commonly used Items usually have 5-9 response options For example, "strongly agree", "agree", "neutral", "disagree", and "strongly disagree" A person's responses can be added up to create a total score

Comparisons of Groups:Post-hoc tests and Planned Comparisonsfor CONTINUOUS data

Comparison of Groups Is Group 1 significantly different from Group 3? Statistics: Post-hoc tests, Planned comparisons Comparison of Groups Is Group 1 significantly different from Group 3? Statistics: Planned comparisons

Experimental Designs: Important Points

Conditions for concluding that the independent variable (IV) causes differences in the dependent variable (DV) 1. Covariation between the IV and DV 2. Temporal precedence of the IV 3. Extraneous (confounding) are ruled out Conditions #1 and #2 are necessary, but not sufficient

Types of variables

Categorical Data has distinct categories A frequency distribution shows how many values fall into each category Continuous Can have an infinitenumber of different values Examples: age, height, weight... Can be described by measures of central tendency Mean, median, and mode

Example of Multiple Group Designs:Bandura's Bobo Doll Studies

Children (n = 66) were divided into 3 groups Each group watched an adult model Then watched each child's behavior IV Degree of adult model's aggressive behavior Levels of IV 3 1. agressive adult model 2. non agressive adult model 3. No adult model DV aggressive behavior in children during free play Hypothesis ℎ_0: ℎ_𝑎: M1=M2 = M3 ha: M1> M2> M3 Extraneous variables?

experimental hypothesis

Clear and specific prediction about how the IV will affect the DV Cell phone example in book: Participants who are restricted from using cell phones in class will have worse test performance than those who are allowed to check their phones (because it is distracting to keep from checking)

Minimizing Confounds and Bias

Confound A variable that the researcher unintentionally varies along with the manipulation Multigroup studies control for confounds by Including more conditions, which exposes the effects of confounds Allowing carefully crafted manipulations Allowing sophisticated relationships to be seen Bias Systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others Multigroup studies can minimize bias by Using manipulations that more closely conform to everyday experience (mundane realism) Increasing experimental realism Hypothesis - guessing When a study participant actively attempts to identify the purpose of the research May have accidentally introduced demand characteristics or may have social desirability concerns Try to disguise the purpose of the study Include distractor questions, questions not related to the topic of interest Assure anonymity and confidentiality Ask during debriefing what they thought the study was about

Statistics

Essential for answering quantitative research questions Descriptive statistics Summarize data in meaningful ways Simplify However, these do not test predictions (inferential statistics do that)

The multigroup Experimental Design

How many _IVs? Will focus on experiments that use one IV Can have multiple IVs (factorial design) Experiments with one IV are simpler, but not inferior to those with multiple IVs A well-designed experiment with one IV is preferable to a sloppy experiment with many variables

internal consistency reliability

Example: Satisfaction With Life Scale (SWLS) In most ways my life is close to my ideal. The conditions of my life are excellent. I am satisfied with my life. So far I have gotten the important things I want in life. If I could live my life over, I would change almost nothing. Participants should answer these items in a *similar way*

Many Questions Can be Addressed with Two-Group Designs

Examples Do certain foods really taste better than others? Do superstitious behaviors (e.g., lucky clothes, rituals) really improve performance at athletic events? Do certain health behaviors (e.g., smoking) really cause cancer? Do stereotypes and discrimination really become self-fulfilling, and affect people's behavior? Two-group experiments can be elegant, and even life-changing

Experimental group One that gets cucumbers Control group One that gets grapes IV: Type of food not being the same DV: Monkey behavior Confounds?

Experimental group Tuesday: Blue eyes on top, Wednesday: Brown eyes on top Control group Tuesday: Brown eyes lesser than, Wednesday: Blue eyes lesser than IV "Group on top" by eye color DV How the children felt Confounds In a classroom/out at recess, not a lab or controlled setting, tumultuous historical period in time (c. 1970)

How to Prove the IV Caused Differences in the DV

Extraneous (or confounding) variables must be ruled out A factor other than the intended treatment that might change the outcome variable

Experimental Designs Randomized Block

First assigns people to a block based on a characteristic that is expected to influence the response of the experimental units to the treatment Then a completely randomized design is performed within each block

Elements of Experimental Designs (Review)

For an experiment to establish causation, there must be Covariation between the IV and DV Temporal precedence of the IV Elimination of extraneous (confounding) variables

Things to consider

How much mundane realism to have? The degree of similarity of the tasks to the real world How much experimental realism to have? Participants' feeling of engagement in the manipulation and is influenced by it

Using Surveys in research

Goal is to collect information that is: Reliable Consistent Valid Accurate Unbiased Discriminating Can distinguish between different people No ceiling or floor effects First, need operational definitions for the variables Then can decide which surveys to use May use a scale that has already been developed Saves times Reliability and validity have been tested May wish to design a new scale Challenging Good questions are difficult to construct Problematic questions are difficult to analyze

Principals of writing questions

Have response options be specific when quantifying time or amount For example, this is vague: How often did you attend religious services last year? Never Rarely Occasionally Regularly This is better: How often did you attend religious services during the past year? Not at all A few times About once a month Two or three times a month About once a week More than once a week Simplify the Vocabulary Use this Instead of this tired.................................exhausted honest..............................candid work.................................employment most important....................top priority free time............................leisure doctor...............................physician

Manipulation check

Helps determine whether the manipulation worked See if participants understood the instructions, and actually perceived the manipulation Cell phone example in book - did the phones vibrate? Did participants actually feel restricted when they couldn't check it?

Measuring Variables

How can we measure variables like feelings, thoughts, and behaviors? Conceptual Definition Defining a variable in theoretical terms Defining a concept Operational Definition How to use or measure the variables in a study Multiple ways to measure the same concept

The Multigroup Experimental Design: Terminology

Hypothesis testing Null hypothesis Hypothesis that there is no difference between the group means trying to statistically reject null hypothesis ℎ_0: 𝑀_1 𝑀_2 𝑀_3 Alternative hypothesis Hypothesis that there is a specific group that will score either higher or lower

The Two-Group Experimental Design: Terminology

Hypothesis testing Null hypothesis for independent means Hypothesis that there is no difference between the group means trying to statistically reject null hypothesis ℎ_0: 𝑀_1 𝑀_2 Alternative hypothesis Hypothesis that there is a specific group that will score either higher or lower ℎ_𝑎: 𝑀_1 (not the same) 𝑀_2 h_a: M1 > M2 h_a:M1 < M2

Multiple Group Examples

IV type of role model Levels of IV (Role models): 4 none peer celebrity parent DV: the likelihood of purchasing a product Hypothesis: ℎ_0: M1 =M2=M3=m4 ℎ_𝑎: M1<M2? M3? M4 Extraneous variables?

Experimental Designs: Design with Matched Pair Design Similar Experimental Units

If certain participant traits are important, can create balanced groups by matching ONLY comparing two treatment groups Each pair is split up and randomly assigned to one of two treatments

Experimental Match pair Designs, Design with the same experimental Units

If certain participant traits are important, can create balanced groups by matcing ONLY comparing two treatment groups Each experimental unit gets both treatments

The Two-Group Experimental Design: The Logic

Importance of Random Assignment Ensures that groups are equivalent at the start of an experiment Should be at least 20 participants per group

The Two-Group Experimental Design:The Logic

In creating two groups, researchers usually: Present some amount or type of IV to one group Withhold that IV from the second group Thus, the presence of the IV is contrasted with the absence of the IV If the two groups differ, may conclude that the IV caused those difference

Multi Group Design

Involves methodological pluralism Using several different methods to examine a research question This provides a detailed picture of how to IV affects the dependent variable DV Maximizes a study's power Study's ability to detect differences between groups, assuming that there is a significant result in the first place The probability that a study will yield significant results

Examples of Multiple Group Designs:Journal Article Activity

Indecent Influence" study IV Levels of IV DV Hypothesis Extraneous variables "Speed and risk-taking" study IV Levels of IV DV Hypothesis Extraneous variables

The Two-Group Experimental Design The Logic

Independent Groups The participants in one group have absolutely to the participants in the other group Created by random assignment Independence The assumption that each participant represents a unique and individual data point Between-Subjects A contrast between independent groups

Testing one's hypothesis

Inference : Drawing a conclusion about a population; applying results from a sample to a larger population Statistical tests allows us to make this inference and gives us a measure of how likely we are to be wrong

Central Tendency

Mean Average score Median Score found in the exact middle Mode Most frequently occurring score

Reverse code Items

Must include reverse coded items in order to avoid response sets Want a mixture of positively - keyed and negatively- keyed items Examples: 1: "I like myself" 1-R: "I dislike myself" 2: "Research methods class is interesting" 2-R: "Research methods class is boring"

Response alternatives

Need to consider the number of response options Forced - choice scale Only two options True or false; yes or no Not much variability Could get ceiling or floor effects *Error of central tendency* The tendency of participants to avoid the most extreme responses Want more than 3 options, but too many is overkill Ideal seems to be 5-9 Such as "strongly disagree", "disagree", "neutral", "agree", and "strongly agree" Agreement Strongly Agree Agree Neither Agree Nor Disagree Disagree Strongly Disagree Likelihood Very Likely Likely Neutral Not Likely Very Unlikely Importance Very Important Important Moderately Important Slightly Important Not Important

How to prove the IV caused the difference

Need to show a relationship( or covariation or correlation) between the IV and DV Covariation: Changes in one variable are associated with changes in another variable Part of determining causation

Developing a protocol

Need to standardize Procedure for informed consent Manipulation check May alternate the manipulation different times of day

Experimental control..

Need to: Randomly assign participants to groups whenever possible Minimize outside influences keep procedure equal for everyone Make sure manipulation was effective

Multigroup Experimental Design

One independent variable (IV) with 3 or more levels Good for any research questions involving more than two conditions to compare How would you improve students' grades? What is the best stress-reduction technique? What is the best treatment for an adolescent with depression?

How to construct

Open-ended questions Participants answer using own words Closed-ended questions Participants answer using response options Scale Usually a series of closed-ended questions

Statistical Comparisons:One-way ANOVA

Overall Difference Hypothesis: The groups will be different on the DV Statistic: One-way ANOVA

Elements of Experimental Designs

Participants are randomly assigned to groups Experimental group(s) receive the manipulation Control group does not receive the manipulation Types of variables Independent variable (IV) Dependent variable (DV) An independent variable (IV) is manipulated or changed so each group experiences it differently IV: Variable that is believed to cause change in other variables (DVs) A dependent variable (DV) is measured within each of the groups DV: variable that is changed by the IV For an experiment to establish causation, there must be Covariation between the IV and DV Temporal precedence of the IV Elimination of extraneous (confounding) variables Internal validity Degree to which one can rule out other explanations for the relationship between the IV and DV

More iffy items

Question: What is wrong with the young people of today and what can we do about it? Problems Double-barreled Implies negativity toward young people Vague Alternative "Do you think people aged 18-25 are have more education than people aged 40-65?" Question: Do you go swimming? __ Never __Rarely __Frequently __Sometimes Problems The question doesn't match answers Language is too vague Alternative "How often do you go swimming in a typical month?" __Less than once a week __1-2 times a week...etc Question: What do you think can be done about global warming? Problems Too vague Assume respondent has an opinion on this issue Alternative "What is your opinion about global warming?" Question: Most medical professionals agree that smoking causes lung disease. Do you: __ Strongly Agree __Agree ___Neither ___Disagree... Problems Leading question: respondent is led into agreeing with medical professionals Alternative "Does smoking cause lung disease?" Have a "don't know" category Question: Do you agree that students should not have to take an exam at the end of their careers? __yes ___no Problems Should include a no opinion or I don't know category Leading question Alternative "Do you think students have to take an exam at the end of their careers?" Question: Has your son ever stolen anything? If so, what and when? Problems Too sensitive, people wouldn't be honest Too many parts to the question Alternative "Has anyone you know ever stolen anything?" ___yes ___no "If so, what did the person steal?" Question: What do you think about the left-wing media's attempt to blackmail the government? Problems Making assumptions Leading question A researcher has own political agenda Alternative Just discard question Question: How much food do you think the average family throws away in a week? Problems No clear definition of average or type of food Alternative "How much fruit do you think a family of 4 throws away in a week?" Question: How much do you earn? Problems Too sensitive Can be answered different ways Alternative "What is your monthly income?" ___ < $1000 ___ $1000-$2000...etc Question: What is your ethnicity? Problems ____________________________________ _________________________________ Alternative Include response options, such as used in the U.S. Census

Basic Research Designs (Review):Nonexperimental, or Correlational, Designs

Questions of what happens Can describe relationships and make predictions However, cannot establish cause and effect

Basic Research Designs (Review)Experimental Designs, or Experiments

Questions of why something happens Can look at causal relationships

Assigning Participants to Conditions

Random assignment is key Can: Have participants draw slips of paper Use three coins Use random number generator

Reliability and Validity

Reliability: Does the tool consistently measure what is measured? Validity: Does the tool measure what it is supposed to measure? Some tools can be reliable but not valid

The Multigroup Experimental Design: Terminology

Reporting 3+ group comparisons (Chp 9) All calculated values in statistics are italicized

More Complex Designs

Researchers may want to go beyond a two-group design and test more complex hypotheses No research design is perfect, each having its own advantages and disadvantages Use methodological pluralism Employ several different methods or strategies to examine a research question

Two Group Design

Simplest experimental design Most basic way to establish cause and effect One experimental group One control group Different participants in both groups

Data Analysis

Statistical software (e.g., SPSS) Descriptive statistics Central tendency Standard deviation Correlation coefficients Statistical hypothesis testing

The Multigroup Experimental Design: Terminology: One-Way Analysis of Variance (One Way Anova)

Statistical test to see if at least one level of the IV is statistically different than the others An omnibus / overall test of differnet, but does not specify where the difference(s) may be Post - hoc tests Statistical tests to find which levels are different after having already finding a statistically significant result Planned Contrasts Pre-planned statistical comparisons between specific levels

How to Prove the IV Caused Differences in the DV

Temporal precedence of the manipulation in the IV must happen before the changes When changes in the suspected cause (treatment) occur before changes in the effect (outcome) A cause must come before an effect

Ethics

The manipulation of the IV should be done as ethically as possible Need to minimize potential harm or embarrassment Consider the effects of not doing well on a task Deception may be necessary Debrief afterward

Extraneous Factors

The previous article about Hepatitis B vaccine and SIDS The vaccine had temporal precedence; it was given before death Still cannot conclude causality Every study has many possible extraneous factors

Want to

Vague, unclear wording Complex vocabulary Jargo Double-barreled items Evaluative language Response options that do not match the question

Extraneous Variables

Variables that have an undesired influence on the DV Confounded experiment An experiment with extraneous variable(s) Lacks internal validity May occur if the groups are not equal before the start of an experiment

Survey / survey scales are

Versatile Very common Well-suited for correlational research; can be used in experimental designs Difficult, but rewarding to design

Experimental Control

Want a high degree of experimental control Ability to keep everything between groups the same except for the IV Control group is key Provides the basis of comparison

Experimental control

keeping everything between groups the same except for the IV

The Two-Group Experimental Design: Terminology (Review)

p value Likelihood of a group difference occurring just by chance Usually, if less than 0.05, or 5%, can conclude the difference is statistically significant

Termionolgy

p-value Likelihood of a group difference occurring just by chance Usually, if less than 0.05 or 5% can conclude the difference is statistically significant