Exam 1 Review
PRACTICE QUESTION: Dave is conducting a survey to see if students in his research methods class preferred cats over dogs. To determine which they prefer, he decides to administer a memory task to the students. This survey has poor face validity because
"At first glance, it doesn't appear to measure what it should" -At first, memory does not seem to be correlated with cat or dog preference in the class
PRACTICE QUESTION: Byron wants to investigate whether running causes more exhaustion before class or after class. Based on Byron's research question, which of the following must be true?
"Bryan's study will have an independent variable" is currently a true statement about this study because at least one variable will need to stay constant (this variable could be time as he's measuring before and after class)
PRACTICE QUESTION: Doug and Lucille are interested in the structure of language during communication between siblings. Doug wants to hold live conversations between siblings and evaluate the language they use. Lucille wants to solicit the last week of text messages sent between the siblings. How are these operationalizations of language similar or different?
"Doug is proposing a behavioral observation measure; Lucille is proposing a behavioral trace measure"
What are Heuristics?
*** Mental Shortcuts (and there are various kinds) - One implication of heuristics is that they may lead to BIASES >> this occurs because Heuristics are based on: -our own experiences -intuition and rapid judgements -authority figures and perceived "expertise"
(Understanding surveys) Explain Question formats, Question Wording, and Response patterns
***Question formats: -Open Ended- written response (gives insight into each participants specific and unique thoughts/ behaviors etc.) -Closed Ended- Multiple choice, fill in the blank, or select all that apply (more quantifiable than open ended because participants must pick a predetermined answer (perhaps not as specific to each participant -Likert (-type): established method of measuring people's attitudes, perceptions, opinions, and behaviors in research methods *measures: Response categories, response options, scores, summation -Semantic differential: "measure the meaning or perception of concepts, objects, or experiences along bipolar dimensions" *measures: Bipolar adjectives, rating scale, summation ***Question Wording: - Double negative items: (NOT GOOD, confusing to the participant and not straightforward ) ex. Did you not, not go to the store? = Did you go to the store? -Leading questions: almost prime the participant or cause them to choose a certain answer based on the wording -Double Barreled questions: "combines two or more distinct issues, topics, or concepts into a single question. This can lead to confusion and inaccurate responses because respondents may find it challenging to address multiple issues simultaneously." ***Response patterns: -Priming: providing material to respondents before results are taken in order to influence their subconscious decision to respond to a question in a certain way -Response Sets: consistent or systematic pattern of responding to a series of questions or items, that may not accurately reflect the respondent's true attitudes, beliefs, or experiences. (BIASES) *including the following: Extreme response bias, social desirability bias, reference group bias, central tendency bias, etc.
Explain the "PROBABILITY SAMPLING APPROACHES" flow chart
***Top tier: Probability Sampling = "Random [selection]" ***2nd tier: -Simple random sampling (selecting blindly) -Stratified random sampling (some of everything) -Cluster sampling (randomly sampling whole groups) ***3rd tier (special cases) : -Simple random sampling (selecting blindly) >>> Systematic Sampling = random # generator -Stratified random sampling (some of everything) >>>Oversampling = compensation for comparison -Cluster sampling (randomly sampling whole groups) >>> Multistage sampling = two random processes
According to the Belmont Report what are the 3 ethical principals? Explain.
- Beneficence: MAXimizing BENEFITS and MINimizing COSTS associated with doing and not doing the research ***Major concern: HARM and Confidentiality -Justice: ensuring that participants involved in the study are representative of the kinds of people who would also benefit from the results *** Major concern: targeting vulnerable groups, providing demonstrably effective treatment to a control group -Autonomy/ Respect for persons: Providing sufficient info to allow individuals to make voluntary choices about their participation *** Major concern: Informed consent, coercion, deception, and debriefing
Discuss Research Misconduct
- Falsification: Untrue information or faulty information (manipulating research material, changing or omitting data ) -Fabrication: Intentionally creating data or evidence for a claim that does not naturally occur with change in levels of a variable in question (Making up data) -What is the response to misconduct? retraction -How to minimize misconduct? valuing null results, encouraging replication, human subjects training, etc.
Discuss longitudinal studies
- Longitudinal studies use the passage of time to est. temporal precedence -Longitudinal studies measure multitudes of variable and multiple points in time -offers hints/ insight on cause and effect -CROSS SECTIONAL CORRELATIONS: 2 different variable at the SAME TIME point -AUTO CORRELATIONS: between the SAME variable at 2 DIFFERENT TIMES -CORSS - LAG CORRELATIONS: between 1 variable EARLIERr and another LATER
What is the difference between a variable and a constant?
- Variables actually change and determine the findings in the study (the change and VARY) -There can be conceptual variables as well, these are CONSTRUCTS -Constants are factors in the research that do not change as the other variables do, it stays the throughout the trials in the research
What is a good hypothesis?
-"Specific predictions that test components of an overarching theory" 1. Supported by data 2. Falsifiable (can be tested) 3. Parsimonious (clear and succinct) 4. Specific (not over explained BUT relatively narrow so it may produce a clear result)
Why does REPRESENTATIVENESS matter?
-A "Representative Sample" is one in which each member of the population has an EQUAL LIKELIHOOD of being part of the sample -A representative sample matters if we want to GENERALIZE what we learn from the sample, to the entire population of interest -LACK of representativeness HURTS EXTERNAL VALIDITY if it systematically affects the outcome of the study -Systematic Error: Lack of representation can introduce systematic error or bias into the study. This bias can affect the outcome in a way that is not representative of the population being studied. For example, if a study on income levels only includes high-income individuals, the findings may not be applicable to the general population's income distribution.
Explain the difference between the following biases: -Availability heuristic -Confirmation bias -Bias Blind Spot -Representativeness heuristic -Better- than-average effect -Focusing Effect -The probability heuristic
-Availability heuristic: things that pop up EASILY in our mind tend to guide our thinking AND using the ease at which something comes into our mind as a sign of its frequency or commonality -Confirmation bias (Confirmatory Hypothesis Testing): tendency to only look at info/ justification, that agrees with what we already believe -Bias Blind spot: Thinking we are less susceptible to biases than others AND thinking biases don't apply to you- meaning "I am LESS bias than others" is a false thought) -Representativeness heuristic: using the prototype as a sign of its frequency or commonality -Better- than-average effect: when more than 1/2 of the group believe they are above average (in some random characteristic) relative to the rest of the group -Focusing Effect: when we ignore peripheral information and fail to perceive the stimuli in our environment -Probability heuristic: mapping from observations directly to subjective probabilities which enables the use of theoretically principled mechanisms
List and define the 4 different types of validity:
-Construct Validity: OPERATIONALIZATION (did we measure what was intended to be measured) -External Validity: GENERALIZATION (do the findings apply to individuals across multiple contexts) -Statistical Validity: DESCRIPTION and INFERENCE (how large and trustworthy are the patterns in the data) -Internal Validity: CONTROL (are the patterns we see in the data happening for the reasons we think they are)
Define the following important terms: -Control Group -Confound -Basic -Casual Factor -Basic Research -Applied Research
-Control Group: condition in which the casual factor is absent; in group that describes the environment or subjects without any change in the variables in question -Confound: other factors that also effect an outcome, leading us to wrongly conclude that one particular factor was responsible -Basic Research: Basic Research tackles THEORETICAL questions; its not typically used to develop a specific solution (more so identifies knowledge) -Applied Research: Applied research tackles SPECIFIC questions; usually used to identify a specific SOLUTION -Casual Factor: major unplanned, unintended contributor to an incident (a negative event or undesirable condition), that if eliminated would have either prevented the occurrence of the incident or reduced its severity or frequency.
Explain the difference between the following types of biases: -Empirical -Probabilistic -Self correcting
-Empirical: uses evidence systematically obtained through observation and measurement -Probabilistic: explains a portion of possible cases (not necessarily "proving" facts, rather providing the likelihood (not guaranteeing any specific thing -Self correcting: and extension of being empirical and probabilistic, SCIENCE UPDATES IDEAS AND APPROACHES
Name and describe the three types of claims:
-Frequency claim: description of a particular rate or degree of a SINGLE VARIABLE -Association claim: argument level of a variable is related to another level of a variable- CORRELATION BETWEEN VARIABLES (positive or negative correlation) -Causal claim: one variable is RESPONSIBLE FOR CHANGING another variable (independent or dependent)
Name and describe the OPERATIONALIZATION- Scales of Measurement
-Nominal: CATEGORY labels (colors, states, universities, treatment condition, etc.) -Ordinal: RANKED categories (gold, silver, bronze; 1st, 2nd, 3rd) -Interval: QUANTITATIVE SCORES w/ consistent differences across the scale (temperature, IQ scores) -Ratio: SCORES w/ CONSISTENT RATIOS across the scale and scale has a TRUE ZERO (zero means "the absence of" etc.) (ex. height, weight, time, price)
(Assessing Operationalization) Explain two concerns of Construct Validity
-RELIABILITY: consistency (how consistent the measures are, are they reliable) -MEASUREMENT: accuracy (is the operationalization measuring what it is supposed to ? is the measurement valid?)
According to IRB and IACUC what are the three R's
-Replacement: avoid using animals -Refinement: minimize unpleasantness to animals -Reduction: use as few animals as possible
What are the possible implications of SAMPLING:
-Sampling can cause implications for EXTERNAL VALIDITY (due to issues with generalization) -Sampling can cause implications for CONSTRUCT VALIDITY (because a measure may not capture its underlying concept equally well for all members of a population) -Sampling can cause implications for STATISTICAL VALIDITY (since most studies/ procedures are designed to allow us to infer characteristics of a population the inferences made from the sample are useless if the sample does not correctly represent the population -Sampling can cause implications for INTERNAL VALIDITY (the more tightly we control the research environment the more difficult it is to generalize from the context of the study to the broader world)
Explain the CONSTRUCT VALIDITY flow chart
-TOP: CONSTRUCT VALIDITY -2nd or middle tier: RELIABILITY vs. MEASUREMENT VALIDITY ****Reliability column: -Test-Retest (consistency ACROSS TIME) -Inter rater (consistency ACROSS RATERS/OBSERVERS; think different people studying the same issue, do they get the same results?) >>> a measure of consistency between two or more independent raters (observers) of the same construct) -Internal validity: (consistency ACROSS COMPONENTS/ RESPONSES) examines whether the study design, conduct, and analysis answer the research questions without bias >> CRONBACH's ALPHA- measurement of validity (measures internal validity, calculations, interpretation, use in research, range in value, limitations, etc.) *** Measurement Validity: -Face: most BASIC form of validity (does it appear to measure what its supposed to at FACE VALUE) -Content: capture EVERY COMPONENT/ aspect of the construct? -Criterion: measurement tool correlated with a related behavioral outcome; performance on the measure should be associated with how they perform on the outcome >>> ***** "how well a measurement or test can predict or correlate with a specific criterion or outcome" >>> (if your compare two different test scores they should correlate strongly to ensure criterion validity) *** EX: "XYZ College Admission Test." You want to assess its criterion validity by comparing the scores of this new test to an established criterion: high school GPA (Grade Point Average). If the correlation between the scores on the "XYZ College Admission Test" and high school GPAs is high or statistically significant, it demonstrates good criterion validity. -convergent: measure of a construct similar to OTHER similar studies measuring the SAME constructs (Does your measure show consistent results to studies that are similar) >>>>EX. with a "self-esteem" questionnaire example, let's say you want to check that your questionnaire doesn't measure something unrelated, like "anxiety." Therefore, you compare the scores from your questionnaire with scores from anxiety questionnaires (the scores should not correlate) -Discriminant: Is your measure DIFFERENT than other measures that are not measuring your construct also (irrelevant constructs/ measures should not correl
Explain the significance of "PATTER and PARSIMONY"
-TRUE EXPERIMENT IS IMPOSSIBLE -"If each study addresses one straightforward aspect of this theory (parsimony),and collectively demonstrate a clear pattern, we gain confidence in the theory"
What is a good research question?
-one that INTERESTS you -one that is TESTABLE
PRACTICE QUESTION: Finish the sentence - The scientific method...
...allows scientists to understand the procedures behind and replicate each other's work REPLICATION is important for validity of the argument
What are the 3 criteria for establishing CAUSALITY
1. COVARIATION: variables related to each other in a SYSTEMATIC way (necessary for testing association claims AND can be captured in correlations and in regression analyses) 2. TEMPORAL PRECEDENCE: One variable MUST undergo CHANGE before another (presumed cause needs to happen earlier in time, longitudinal studies est. this) 3. INTERNAL VALIDITY: No viable alternative explanation for changes in outcome
THE SCIENTIFIC METHOD: Explain the acronyms: A.D.A.P.T.
A-Ask your research question D-Design you A-Analyze the data collected in your study P-Parse your findings (break down the analysis) given your question T- Tell the scientific community
PRACTICE QUESTION Which of the following would result in a biased sample? A. Self-selection B. Purposive sampling C. Convenience sampling D. All of the above
ALL OF THE ABOVE each of the following options create implications and biases within the study. Self selection: "individuals or units voluntarily choose to participate in a research study. In self-selection, participants have the freedom to decide whether or not they want to be part of the research, and they actively opt in or out of the study." Purposeful Sampling: "non-random sampling technique where researchers intentionally select specific individuals or groups to be part of their study." Convenience Sampling: "type of non-probability sampling in which researchers select participants who are most readily available or accessible to them. This method is often used when researchers prioritize convenience and practicality over randomness"
What are Analytical Regression Analyses?
Analytical approaches that simultaneously test the strength and direction of the relationship between multiple predictors and a single outcome variable ***Predictors can be Unstandardized or Standardized UNSTANDARDIZED: STANDARDIZED: -Allows for statistical control for the relationship between predictors -if additional predictors change the strength and/or direction then there may be a THIRD VARIABLE PROBLEM
PRACTICE QUESTION: Alex is a researcher at a company that manufactures prosthetic limbs.Her team wants to investigate what kinds of attitudes people have toward those who wear prostheses. Which is true?
BASIC RESEARCH -the answer is Basic Research because Alex and her team are studying the concept or idea with no intent on finding a solution for a specific problem
PRACTICE QUESTION Dr. Bowen alphabetically places students into 10 breakout rooms with10 students each. He administers a survey to half of the breakoutrooms (chosen at random). This is an example of what kind ofsampling?
CLUSTER SAMPLING Dr. Bowen ordered the original group alphabetically THEN randomly selected groups from those groups (this is not RANDOM sampling but CLUSTER sampling)
Nick has a new survey and wants to check to see how well his survey operationalizes the construct of "clothing preferences". He examines whether scores on his survey correlate with people's clothes shopping habits. What is Nick testing here?
CRITERION VALIDITY: "assessing the relationship between his survey and an external criterion (clothes shopping habits)" why not convergent validity? "testing doesn't involve assessing the relationship between his survey and other measures or surveys that are theoretically related to the same construct"
Explain Construct Validity.
Construct Validity explains how well a study's variables are measured and manipulated. 1st - CONSTRUCT 2nd - OPERATIONALIZATION of construct 3rd- MEASURE or MANIPULATION (operational variable) therefore First you identify a variable in a conceptual sense, then you translate the construct into something measurable, then there are 3 BROAD FORMS (self report of info, Behavioral observation, Biological factors assessed)
PRACTICE QUESTION A researcher has some data from a study on the effects of mood condition (positive vs negative) on task persistence. They predict that those in the positive condition will persist longer than those in the negative condition. However, their spreadsheet of the data was corrupted when they tried to save it and a few responses were accidentally deleted. Not wanting to lose any valuable data, the researcher put numbers back into the missing cells that were a "good approximation" of the missing values. Did research misconduct take place?
DATA FABRICATION - The researcher intentionally "made up data" rather than changing it (falsification) the researcher guessed and input information.
PRACTICE QUESTION: Dr. Chambers hypothesizes that brown rats will run faster than gray rats, provided their maze is circular, less than 10 ft wide, and the rats run the maze before 11 AM. Dr. Chambers finds support for this hypothesis, but Dr. Palmer is critical of the work. Dr. Palmer is most likely concerned that...
Dr. Chambers generated a hypothesis that was not parsimonious (this hypothesis was not clear and concise, there were too many specifications making the overall prediction not clear)
PRACTICE QUESTIONS: Longitudinal studies ... A. Establish internal validity for causal claims B. Establish temporal precedence C. Do not allow for cross-sectional comparisons D. Do not allow for multiple regression analyses
ESTABLISH TEMPORAL PRECEDENCE: longitudinal studies collect data over long periods of time and they demonstrate order of events therefore they can establish temporal precedence by tracking changes over time the answer is not establish internal validity because longitudinal studies cannot validate causal claims on their own (they need to consider more factors that are not usually involved in longitudinal studies)
PRACTICE QUESTION Testing a drug for a rare genetic disease only on a sample of Hopkins students at no risk for the disease is a violation of which of the following ethical principles?
Justice The small group of Hopkins students is not representative of the larger population that would benefit
PRACTICE QUESTION: Tanya wants to examine the relationship between exercise and how energetic people feel. She finds that this relationship depends on whether the exercise is aerobic or anaerobic. Aerobic exercise was associated with more energy, but anaerobic exercise was not. The typeof exercise in this example is a... A. Mediating variable B. Moderating variable C. Control variable
MODERATING VARIABLE: -Not a process (mediation) -Not a control -2 variables -2 different levels/ degrees -CONDITIONAL (people feel this, based on variable)
Explain the difference between Moderation and Mediation
Moderation Vs. Mediation MODERATION: occurs when the relationship between two variables differs depending on the level of the moderating variable (captures conditional association) -Relationship occurs when ONE variable is at a certain level or degree -CONDITIONAL RELATIONSHIP MEDIATION: occurs when 2 variables may be better explained because the predictor is associated with the outcome (captures the possible MECHANISM) -PROCESS
Explain the "NON- PROBABILITY SAMPLING APPROACHES" flow chart
NOTE: (not random BUT not all bad) ***Top tier: Non- Probability Sampling = "biased [selection]" ***2nd tier: -Convenience Sampling: Right in your own backyard -Quota Sampling: Filling requirements -Purposeful sampling: Deliberately recruiting appropriate participants -Snowball Sampling: you tell 2 friends then they tell 2 friends
Explain the MEDIATION flow chart:
PREDICTOR >>> MEDIATOR >>> OUTCOME >>> PREDICTOR (again)
In the terms of "Sampling and choosing the right participants" explain the difference between population and a sample
Population- all the individuals we want to learn about Sample- A subset of your population (the individuals we actually study) NOTE: The SAMPLE is a SMALL PORTION inside of the LARGE POPULATION
PRACTICE QUESTION Mariah is getting together a sample of psychology students at JHU.She knows about 75% of psych majors are women. To recruit her sample, she breaks up the population of psych majors into 2 clusters (women and men). She then goes to several upper division psych classes on campus and asks students if they would be willing to participate. She is careful to ensure that 25% of people who take her survey identify as male. What kind of sampling technique is she using?
QUOTA SAMPLING -Since Mariah is "careful to ensure" certain criteria are met when she recruit is shows she is checking off the boxes and using the quota sampling technique to acquire participants
THE SCIENTIFIC METHOD: Explain the "Theory Data Cycle"
The initial vertical line: -Theory: general consensus based on established knowledge -Research Questions -Research design -Hypothesis: "Specific predictions that test components of an overarching theory," An educated scientific guess/ inference based on the known data and conditions -Data The revisions: -Non-supporting data leading to revised theories in the research design area (occurs in the DATA area, Effects the research design, effects the theory) The support: -Supporting data strengthening the theory
PRACTICE QUESTION: Dave took Research Methods last year and knows that the probability of him winning the lottery is astronomically low. However, he just watched a documentary about lottery winners and goes out and buys a ticket, thinking that he has a good chance of winning. The mental shortcut Dave is using is known as:
The probability heuristic: this is the probability short cut because even though the chance is extremely low overall Dave ignores that fact and gets a mental short cut from the documentary which most likely contains statistics not pertaining to his demographic specifically.
PRACTICE QUESTION: If I want to study the GPAs of first semester freshmen at JHU, what would be a variable in my study?
The students' GPAs -because this is the factor that will change and tell us more about whether our hypothesis is correct
What is operationalization?
To OPERATIONALIZE is to turn a concept into a measured/ manipulated variable
Explain the CONSTRUCT flow chart
Top- CONSTRUCT 2nd tier: - Self- Report - Behavioral - Biological 3rd tier: -Self report: Survey, Interview -Behavioral: Observation, Trace (involves tracing the records of past behavior) , Choice -Biological: Physiological, Neurological
What are two questions you should ask when taking a Behavioral Observation? List and explain 3 techniques of Behavioral observation.
We should ask: -In what CONTEXT are we making our observations? -How do we RECORD the observations 3 techniques: 1. Naturalistic Observation (in nature) 2. Participant observation (subjects choose to participate) 3. Laboratory observation (in lab research) Possible implications within behavioral observations: -Coding scheme (classify behavior in pre-defined categories) -Observer Bias (skew ratings toward your own hypothesis) -Observer (expectancy) effects (observer impacting/ disrupting participant behavior) -Participant reactivity: the participant is aware they are being observed and this disrupts or modifies behavior
PRACTICE QUESTION According to the principle of beneficence, when might it be appropriate to conduct research that is likely to cause some discomfort to participants?
When the potential positive impact of the findings is massive (beneficence is all about balancing the negative and the positive, therefore, if there is discomfort there should be a great positive impact after all)
