Psych 2300 Research Methods Exam 1

What does reliability tell us?

How consistently we measure a construct

Statistical Validity

How strong is the effect? Is it statistically significant? How well the study minimizes type 1 (false positive) and 2 errors (misses)

Internal Validity

How well the claimed causal statement can be supported against alternate explanations (ie other causes)

Empiricism

Involves using evidence from the senses (sight, hearing, touch) or from instruments that assist the sense as the basis for conclusions

Which of the following statements is true? Choose the best answer.

It is possible to have a measure with high reliability, but low validity.

After two students from his school commit suicide, Marcelino concludes that the most likely cause of death in teenagers is suicide. In fact, suicide is not the most likely cause of death in teens. What happened?

Marcelino may have been influenced by the availability heuristic because the things came to mind

Cons of relying on experience

No comparison group, experience has confounds (don't know cause)

Research Databases for Finding Research

PsychINFO, Web of Science, Scholar.google.com, OSU library

Type 1 error

Rejecting null hypothesis when it is true

3 R's of animal research

Replacement: find alternative if can Refinement: minimize or eliminate animla distress Reduction: fewest animal subjects as possible

What does it mean to say that research is probabilistic?

Research conclusions are meant to explain a certain proportion of possible cases, but may not explain all

Which of the following statements is true?

Research described in chapter 1 of the textbook indicates that the Contact Comfort Theory is better supported by scientific findings, relative to the Cupboard Theory.

Pros of Popular Press Coverage

Research findings can reach people who would never have heard about the findings otherwise

Tuskegee Syphilis Study

Research study conducted by a branch of the U.S. government, lasting for roughly 50 years (ending in the 1970s), in which a sample of African American men diagnosed with syphilis were deliberately left untreated, without their knowledge, to learn about the lifetime course of the disease.

observational measure

a method of measuring a variable by recording observable behaviors or physical traces of behaviors

meta-analysis

a procedure for statistically combining the results of many different research studies

Conceptual variable

abstract concept defined by researcher, aka a construct (ie intelligence)

The Good Story

accepting a conclusion just bc it makes sense or fits familiar narrative (ex. Catharsis, stomach ulcers)

Categorical Measurement

aka nominal scale designates which group or category an entity or even belongs to ex--west=1, south=2, etc

When reading about a single empirical research study, you would be most likely to find a detailed description of the study's methods and results in which of the following sources?

an empirical journal article.

informed consent

an ethical principle that research participants be told enough to enable them to choose whether they wish to participate

Causal Claim

argue that association is causally directed--changes in one variable CAUSE a change in the value of the other variable. one variable manipulated, one measured

random assignment

assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups

Which of the following headlines is a frequency claim? a. Obese kids less sensitive to tastes. b. Exercise: 45% of you shake your booty in Zumba. c. Feeling fat? Maybe Facebook is to blame. d. Daycare and behavior problems are not linked.

b. Exercise: 45% of you shake your booty in Zumba.

Following a study using deception, how does the researcher attempt to restore an honest relationship with the participant? a. by apologizing to the participant and offering monetary compensation for any discomfort or stress b. by debriefing each participant in a structured conversation c. by reassuring the participant that all names and identities will be removed from the data d. giving each participant a written description of the study's goals and hypotheses, along with references for further reading

b. by debriefing each participant in a structured conversation

Reaction time, measured in seconds, is an example of a variable being measured on which type of scale? Choose the best answer. Note: Reaction time is the amount of time that passes between the occurrence of an event and a person's response to that event.

ratio scale

Transitional Research

lessons from basic research to test applications in the real world (connecting Basic and Applied Research)

physiological measures

measures of bodily responses, such as blood pressure or heart rate, used to determine changes in psychological state

deception

misleading participants about the true purpose of a study or the events that will actually transpire

Two researchers listen and take notes recording how often passengers on a bus say kind things, neutral things, or rude things to each other. Which term below best describes the type of measurement that is being used?

observational measure

Which of the following is not a place where psychological scientists publish their research? Scientific journal Online podcast Chapters in edited books Full-length books

online podcast

Ordinal Measurement

rank (1st, 2nd, 3rd) - reports the ranking and ordering of the data without actually establishing the degree of variation between them

Institutional Review Boards (IRB)

"Ethical Review Boards" 5 members: scientist, academic non-science, non-affiliated

Association Claim

(correlation/covariances): argue that one level of a variable is likely to be associated with a particular level of another variable TYPES: +,-, zero, curvilinear

Criterion Validity

(predictive or concurrent validity), you can compute a correlation between your measure and some concrete outcome the extent to which a measure is related to an outcome

Ratio Measurement

Equal intervals and meaningful zero ex) # books you own, time taken to run a footrace

Content Validity

Does it include all the important components of a construct?

Face Validity

Does it seem like a good measure?

Convergent Validity

Does the measure correlate with other measures of the same construct IT SHOULD

Interval Measurement

Equal intervals between units, but no meaningful true zero ex) temp in Celsius, IQ test scale

Which statements below are reasons why experience can be a faulty source of evidence for our beliefs? Select all correct response options.

- Experience usually has confounds, making it difficult (or impossible) to determine what causes an outcome. - Experience does not usually allow for a comparison or control group.

Cons of relying on intuition

- being swayed by a good story -relying on availibility - present/present bias - confirmatory hypothesis testing - the bias blind spot

Which of the following are features of good theories? Answer by selecting all options that correctly complete the following sentence fragment. Good theories are...

- parsimonious - supported by data - falsifiable

probabilistic research

-all behavioral research is probabilistic -aka. its findings are not expected to explain all cases all of the time ex. Honda car reviews are great, but a friend had a bad experience with it

Criteria for establishing causation

1) the association is strong 2) the association is consistent 3) larger values of the explanatory variable are associated with stronger responses 4) all alleged cause precedes the effect in time 5) the alleged cause is plausible

5 General ethical principles from APA

1. Beneficence and nonmaleficence 2. Fidelity and Responsibility 3. Integrity 4. Justice 5. Respect for people's rights and dignity

3 Main Principles of the Belmont Report

1. Respect for Persons 2. Beneficence (do not harm AND maximize possible benefits and minimize possible harms) 3. Justice

3 rules for establishing causal claims

1. covariance/correlation 2. temporal precedence 3. internal validity

Suppose you have decided to work on a study concerning psychiatric conditions in tobacco users. A colleague of yours tells you that there was a potentially relevant study published in the past 13 years (i.e., published since 2006). The lead author's last name was "Morissette." The study was a clinical trial comparing a medication with cognitive-behavioral therapy in "daily smokers", "chippers," and "nonsmokers." The only other information your colleague can remember is that the study had something to do with co-occurring anxiety and alcohol-use disorders. Using the information she gave you, find this article in PsychInfo. To answer this question, please type the year in which the article was originally published (e.g., 2003).

2008

Please consider the fabricated reference listed below. Doe, J. L. (2009). Categorizing research biases. In B. E. Smith, & P. Jones (Eds.), Good Research Methods. (pp. 87-109). Philadelphia PA: Psychology Press. Based on the components of the reference, which of the following types of sources would it be an example of?

A chapter in an edited book.

Imagine that researchers wish to study smoking and the use of tobacco in Ohio undergraduate students. If the following groups were to participate in a research study, which would likely have the greatest external validity? Choose the best answer.

A random sample of undergraduate students from 15 Ohio colleges and universities

Sections of an empirical article

ABSTRACT INTRODUCTION METHOD RESULTS DISCUSSION REFERENCES

Suppose you have decided to work on a study concerning psychiatric conditions in tobacco users. A colleague of yours tells you that there was a potentially relevant study published in the past 13 years (i.e., published since 2006). The lead author's last name was "Morissette." The study was a clinical trial comparing a medication with cognitive-behavioral therapy in "daily smokers", "chippers," and "nonsmokers." The only other information your colleague can remember is that the study had something to do with co-occurring anxiety and alcohol-use disorders. Using the information she gave you, find this article in PsychInfo. To answer this question, please type the name of the journal (not the publisher) in which the article was published.

Addictive Behaviors

Milgram's Obedience Study

An experiment in which Stanley Milgram found that people will usually obey an authority, even if they might think what they are doing is wrong. - 65% of people did the full amount of shock

Which of the following options represent a positive correlation between exam grades and amount of studying? Read all options and then choose the best answer.

B) Lower exam grades paired with lower amounts of studying C) Higher exam grades paired with higher amounts of studying E) Both options "B" and "C" represent positive correlations

To be an empiricist, one should:

Base one's conclusions on direct observations

Which of the following research questions best illustrates an example of basic research?

Can 2 month old human infants discern the difference between four objects and six objects?

Ethical Standard 8 from the APA

Defines several specific research practices/rules: -institutional review boards -informed consent -deception -debriefing -animal research -research misconduct

Destiny concluded that her new white noise machine helped her fall asleep last night. She based this conclusion on personal experience, which might have confounds. In this context, a confound means:

Something else may have helped Destiny fall asleep

Cons of Popular Press Coverage

Sometimes research findings are not reported accurately, or they are reported without additional qualifiers that really should be stated (ex Mozart Effect, etc.)

Properties of good theories

Supported by data, falsifiable, parsimonious, can be replicated

independent variable

The experimental factor that is manipulated; the variable whose effect is being studied.

dependent variable

The outcome factor; the variable that may change in response to manipulations of the independent variable.

James is asked about the best way to study for an exam. He responds that the best way to study is by making flash cards. When answering, he considers all the times he used flash cards and earned "A"s. However, he fails to take into account all the times he earned "A"s and did not use flash cards, as well as all the times that he did not earn a high grade with or without the use of flash cards. His thinking most closely exemplifies which of the following?

The present/present bias.

Reliability and Validity - Relationship

The relationship between the two is that if the study is valid, then it must be reliable. However, a study's reliability does not automatically attest to the validity of the study. Although necessary, reliability is not in itself a condition for validity.

Which of the following best describes an independent variable in an experiment? Choose the best answer.

The variable being manipulated in an experiment testing a causal claim.

Imagine that you are told there is a correlation of r = - 0.94 between two variables. What does this information mean? Choose the best answer.

There is a strong, linear, negative relationship between the two variables.

Which of the following jobs most likely involves producer-of-research skills rather than consumer-of-research skills? Police officer University professor Physician Journalist

University professor

In reading an empirical journal article, what are the two questions you should be asking as you read?

What is the argument? What is the evidence to support the argument?

When is it a good idea to base conclusions on the advice from authorities?

When authorities have conducted the research on which their advice is based, by systematically and objectively comparing different conditions.

data falsification

a form of research misconduct in which a researcher influences a study's results, perhaps by deleting observations from a data set or by influencing participants to act in the hypothesized way

data fabrication

a form of research misconduct in which a researcher invents data that fit the hypothesis

Scatterplot

a graphed cluster of dots, each of which represents the values of two variables

test-retest reliability

a method for determining the reliability of a test by comparing a test taker's scores on the same test taken on separate occasions

known-groups paradigm

a method for establishing criterion validity, in which a researcher tests two or more groups, who are known to differ on the variable of interest, to ensure that they score differently on a measure of that variable

self-report measures

a method of gathering data about people by asking them questions about a sample of their behavior

correlation coefficient

a statistical index of the relationship between two things (from -1 to +1)

correlation coefficient

a statistical measure of the extent to which two factors vary together, and thus of how well either factor predicts the other

Which validity would you be interrogating by asking: How well did the researchers measure sensitivity to tastes in this study? a. Construct validity b. Statistical validity c. External validity d. Internal validity

a. Construct validity

Which of the following headlines is a casual claim? a. Holding a gun may make you think other are, too. b. Younger people can't read emotions on wrinkled faces. c. Strange but true: Babies born in autumn are more likely to live to 100. d. Check the baby! Many new moms show signs of OCD.

a. Holding a gun may make you think other are, too.

In most experiments, trade-offs are made between validities because it is not possible to achieve all four at once. What is the most common trade-off? a. Internal and external validity b. Construct and statistical validity c. Statistical and internal validity d. External and statistical validity

a. Internal and external validity

What type of research misconduct involves representing the ideas of words of others as one's own? a. plagiarism b. obfuscation c. suppression d. data falsification

a. plagiarism

Research that is conducted with goal of enhancing a general body of knowledge is best described by which of the following terms?

basic research

Why is publication an important part of the empirical method?

because publication contributes to making empirical observations independently verifiable

SOURCE: Full-length Books

books that are very long and full of information

Which of the following variables if manipulated, rather than measured? a. Number of pairs of shoes owned, in pairs b. A person's height, in cm c. Amount of aspirin a researcher gives a person to take, either 325 mg or 500 mg d. Degree of happiness, rated on a scale from 1 to 10

c. Amount of aspirin a researcher gives a person to take, either 325 mg or 500 mg

Which validity would you be interrogating by asking: How did the researchers get their sample of people for this survey? a. Construct validity b. Statistical validity c. External validity d. Internal validity

c. External validity

Which of the following headlines is an association claim? a. Chewing gum can improve your mood and focus b. Want to cheer up? Cheer harder. Hard-core sports fans show less depression. c. Swine flue shot tied to narcolepsy, study finds. d. Eating kiwis may help you fall asleep.

c. Swine flue shot tied to narcolepsy, study finds.

Which of the following is not one of the three R's provided by the Guide for the Care and Use of Laboratory Animals? a. reduction b. replacement c. restoration d. refinement

c. restoration

In a study of a new drug for asthma, a researcher finds that the group receiving the drug is doing much better than the control group, whose members are receiving placebo. Which principle of Belmont Report requires the researcher to also give the control group the opportunity the receive the new drug? a. informed consent b. justice c. beneficence d. respect for persons

c. beneficence

Internal reliability

the degree to which a person provides similar ratings or scores for related questions

Continuing from question 8, here's one more part of the story: "The notion that parents are apt to shush their kids by handing them a smartphone or tablet also appears to be false, according to results. To keep their children quietly occupied, moms and dads said they were more apt to turn to toys or activities (88 percent), books (79 percent) or TV (78 percent). Of parents with smartphones or iPads, only 37 percent reported being somewhat or very likely to turn to those devices." Imagine that a reader of this article submitted the following comment: "Parents never tell the truth in situations like this. Either they hate to admit that they quieted their child by putting her in front of a phone, or they don't even realize how much they do it. Look around you in restaurants and just watch how many parents with small children let them sit there playing with a phone. It's not just the parents involved with the study either. It's all parents. There's no way that parents answered the researcher's questions honestly." Based on the comment above, which validity is the reader criticizing? Choose the best answer below.

construct validity

Finding that one measure of depression is both strongly and positively correlated with a different measure of depression would provide evidence for which type of validity? Choose the best answer below.

convergent validity

Which of the following is not one of the three principles of the Belmont Report? a. Respect for Persons b. Justice c. Beneficence d. Fidelity and responsibility

d. Fidelity and responsibility

The example of the "Mozart effect" was used to demonstrate that...

journalists don't always provide an accurate description of research findings.

In order to study a sample of participants from only one ethnic group, researchers must first demonstrate that the problem being studied is especially prevalent in that ethnic group. This is an application of which principle from the Belmont Report? a. respect for persons b. beneficence c. special protection d. justice

d. justice

quantitative measurement

data collected using numbers such as time, mass, distance, volume, temperature

Frequency Claim

describes the rate or level of some variable EX) 58% of yada yada, almost half of yada yada do something

A US News report, "Young Parents Don't Stress Over Kids' Media Use: Survey", (http://health.usnews.com/health-news/news/articles/2013/06/04/young-parents-dont-stress-over-kids-media-use-survey (Links to an external site.)) details the results of polling in which young parents were surveyed about their kids' use of smartphones and tablets. An excerpt from the journalist's story is below. "Surveying more than 2,300 parents of children up to age 8, researchers from Northwestern University found that the vast majority -- 78 percent -- report that their children's media use is not a source of family conflict, and 59 percent said they aren't concerned their kids will become addicted to new media." If you do not believe that this sample of 2300 young parents would generalize to the population of interest, you are questioning what type of validity in this research?

external validity

type 2 error

failing to reject a false null hypothesis

Present/Present Bias

focus on positive instances more than negative ones (ex. only considering when you took medicine AND felt better)

External Validity

how well the claimed effect generalizes beyond the people, stimuli, and circumstances of the particular study

Construct Validity

how well the variables in the study are operationalized, measured, and manipulated

SOURCE: Edited Books

invited authors, overview of journals

SOURCE: Empirical Journal Articles

report original methods and results in detail

Applied Research

research that solves a specific/practical problem--end game

Basic Research

research to enhance the general body of knowledge

Discriminant Validity

scores on the measure are not related to other measures that are theoretically different

SOURCE: Review Journal Articles

summarize a collection or pattern of results from the literature, meta-analysis

Operational Variable

the actual thing you're testing

interrater reliability

the amount of agreement in the observations of different raters who witness the same behavior

Debriefing

the post-experimental explanation of a study, including its purpose and any deceptions, to its participants

Plagiarism

the practice of taking someone else's work or ideas and passing them off as one's own.

A statement, or set of statements, that describes general principles about how variables relate to one another is a(n) __________.

theory

purpose of debriefing

to provide research participants with a full understanding of the research investigation, so that they can understand their role in it, and make sense of any deception that they may have been subject to.