TEST #1
Choosing how to measure something
which of the four measurements to use based on the type of item you are measuring weather it be measurable by numbers or physical descriptions.
Demand characteristics (and ways to control)
A feature of the experimental design or procedure that increases the chances that participants will detect the true purpose of the study; Participants can be biased and behave in ways they think they should behave - Good subject role: Tries to act in a way that confirms hypothesis - Bad subject role: Tries to act in a way that disconfirms hypothesis - Social desirability bias: Gives socially desirable answers • Some strategies - Make subjects blind to the condition they are in - Mask the experimental manipulation by using a placebo
Pseudoscience
A field of inquiry that attempts to associate with true science, relies exclusively on selective anecdotal evidence, and is deliberately too vague to be adequately tested.
Directionality problem and third-variable problem
A high correlation indicates that the level of one variable predicts the level of another variable (very useful); Cannot infer cause-effect relationship. Correlation can be "spurious."
Samples and populations
A portion or subset of a population; all of the members of an identifiable group; The entire collection of individuals being considered (the people you are trying to generalize your results to); The subset of the population who actually participate
Stratified random sample
A probability sample that is random, with the restriction that important subgroups are proportionately represented within it; A type of random sample in which subgroups are represented in the same proportions as the population
Blocked randomization
A procedure used in within-subjects design as a counterbalancing procedure to ensure that when participants are tested in each condition more than once, they experience each condition once before experiencing any condition again; Each block of trials involves a random order of each of the levels of the IV - Useful when you have many items/observations per condition
Attrition/Mortality effects
A threat to the internal validity of a study; occurs when participants fail to complete a study, usually but not necessarily a longitudinal study; those finishing the study may not be equivalent to those who started it; Participants may be more likely to complete the experiment in one condition than another • Even if random assignment ensures equivalent groups at the start of an experiment, they may not be equivalent by the end of the experiment
Subject selection effects
A threat to the internal validity of a study; occurs when those participating in a study cannot be assigned randomly to groups; hence the groups are nonequivalent
Counterbalancing
For a within-subjects variable, and procedure designed to control for sequence effects; can be used to control for order effects, sequence effects, and item effects
ratio
Interval with True Zero (e.g., age, response time, errors) ; The amount of time someone waits before finally eating the marshmallow
Continuous variables
Not limited to particular values
2) What describes exactly how a variable is measured in the context of a research study? a. Operational definition b. Convergent validity c. Demand characteristics d. Cohen's d
a
A researcher is interested in the effect of using Snapchat on feelings of self-worth. To explore the relationship between the two variables, the researcher measures the amount of time participants spend using Snapchat per day (in minutes), and the degree to which they report having strong feelings of self-worth on a scale of 0 to 10 (0 indicating low feelings of self-worth; 10 indicating high feelings of self-worth). 10) Which variable is being measured on a ratio scale? a. Snapchat Use b. Self-worth c. Both d. Neither
a
Cohen's d
Effect size used to indicate the standardized difference between two means *The difference between two means divided by the standard deviation. d = (M1 - M2 )/SD
Data
Facts, figures, and other evidence gathered through observations.
Type II errors
Failing to reject the null hypothesis when it is false; failing to find a statically significant effect when the effect truly exists.
between-subject designs
An experimental design in which different groups of participants serve in the different conditions of the study; Each subject experiences only one experimental condition
Within-subject designs
An experimental design in which the same participants serve in each of the conditions of the study: also called a repeated measures design; Each subject experiences all experimental conditions - More powerful design, but requires additional control
confounding variables
An extraneous variable that covaries with the independent variable and could provide an alternative explanation of the results. (threat to researcher); Variables that vary systematically with the IV (large threat to internal validity)
Error bars/Confidence intervals
An inferential statistic in which a range of scores is calculated; with some degree of confidence (e.g., 95%) it is assumed that population values lie within the interval.
Extraneous variables
An uncontrolled factor no of interest to the researcher but that could affect the results; All possible variables other than the IV (can affect the DV, but if controlled, then not a threat to internal validity)
Limits to how we know what we know (e.g., authority, logic, intuition)
Authority - Government, media, professors, doctors, parents. - Often right, sometimes very wrong. Sometimes we just don't know... • Logic and Reason - Assumes assumptions are true... under all conditions... • Intuition and Experience - Initial impression, gut feeling, judgment, etc. - Not objective; can be inaccurate and biased
dependent variables
Behavior measured as the outcome of an experiment; The outcome being measured
Order effects
Can occur in a within-subjects design when the experience of participating in one of the conditions of the study influences performance in subsequent conditions; Subjects may perform differently when a condition is experienced first than when it is experienced second
Correlations and relationships between variables
Cannot infer cause-effect relationship; A representation of the relationship between two variables
nominal
Categorical (e.g., major in college, graduate or not) ; Whether or not someone eats the marshmallow
Importance of the control group
Controls are needed to increase power and prevent confounding variables, but they can also reduce external validity
Basics of the scientific method
Determinism and discoverability • Consider alternative hypotheses and collect data to see which is supported • Objectively verify assumptions through empirical evidence (direct observation and experience) • Remain skeptical and reserve judgment in the face of contradictory information (self-correcting/tentative) • Probabilistic (inferences/assumptions are never 100%)
Variability (range, standard deviation)
Difference between the high and low scores of a sample; Estimate of the average amount by which scores in a sample deviate from the mean
Ensuring equivalent groups through random assignment
Large sample plus random assignment ensures equivalent groups - All extraneous subject factors are controlled - Matched random assignment can also be useful (e.g., small sample size and or when an extraneous variable is likely to co-vary with the DV) • An experiment without random assignment will suffer from subject selection effects and is likely to have low internal validity • Even if a difference is statistically significant (reject the null), you won't be able to interpret - Many possible confounding variables - Low internal validity
Discrete variables
Limited to particular values (often whole numbers)
Problems with single-group pretest-posttest designs
Measure behavior of a single group before and after some treatment/intervention; no control group thus no real interpretation of the results
Disadvantages to using within-subject designs
Need to control for order, sequence, and item effects - Instrumentation and issues with measurement (way in which DV is measured may change across repeated observations) - Study takes longer and subjects may get tired of task/develop strategies (which can threaten external validity)
Experimenter bias (and ways to control)
Occurs when an experimenter's expectations about a study affect its outcome. ways to control include random assignment; Tendency for researcher to (consciously or unconsciously) treat participants differently in the different conditions - Impossible to self-monitor
ordinal
Order (e.g., class standing, birth order) ; The types of food that are most difficult to avoid eating, in rank order
Descriptive statistics
Provide a summary of the main features of a set of data collected from a sample of participants.
Advantages to using within-subject designs
Reduce random error by controlling for individual differences (every participant serves as own control) - Requires fewer participants (more power)
Type I errors
Rejecting the null hypothesis when it is true; finding a statistically significant effect when no true effects exists.
interval
Roughly Equal (e.g., IQ scores, stress rating) ; The extent to which someone believes they would be able to avoid eating the marshmallow (1 = not at all, 5 = maybe, 9 = very likely)
Statistical Power
The chances of finding a significant difference when the null hypothesis is false; depends on alpha, effects size, and sample size; the probability of rejecting the null if the null is false (1 - β)
Definition of internal validity
The extent to which a study is free from methodological flaws, especially confounding factors; Researcher's ability to conclude that the IV caused changes in the DV (IV → DV?)
External validity
The extent to which the findings of a study generalize to other populations, other settings, and other times; Extent to which the results of a study can be generalized to other situations (populations, environments, tasks, etc.)
Independent variables
The factor of interest to the researcher; it can be directly manipulated by the experimenter (e.g., creating different levels of anxiety in subjects), or participants can be selected by virtue of their possessing certain attributes (e.g., selecting two groups that differ in normal anxiety).; The factor being manipulated
item effects
The items in one condition may have a different effect than the items in another condition
Multi-level independent variables
The most common approach is the factorial design, in which each level of one independent variable is combined with each level of the others to create all possible conditions.
Operationally defining an independent variable
The precise meaning of a concept within a study (e.g., how a construct of interest is measured) • Provides a way of measuring something that can't be observed directly (e.g., learning, creativity, anxiety, honesty) • Different operational definitions allow researchers to make different conclusions; explaining how something is being manipulated so it is possible to replicate.
alpha level
The probability of making a Type I error; the significance level; Probability of rejecting the null if the null is true (typically 5%) - In a study we observe a "p-value" for an observed effect and then compare it to alpha - If p-value is less than alpha we "reject the null hypothesis"
Examples of how our intuitions can be biased/inaccurate
We never really/fully know why we do the things we do (or think the way we think) • Cognition is designed to rely on heuristics to make decisions, guide behavior, and inform beliefs (e.g., availability, anchoring) • We are influenced without conscious awareness
hindsight bias
What seems common sense now wasn't common sense before
A researcher is interested in the effect of using Snapchat on feelings of self-worth. To explore the relationship between the two variables, the researcher measures the amount of time participants spend using Snapchat per day (in minutes), and the degree to which they report having strong feelings of self-worth on a scale of 0 to 10 (0 indicating low feelings of self-worth; 10 indicating high feelings of self-worth). 12) After completing the study, participants are given a new question. In this question, they are told to imagine that they are applying for a job and that they will need to decide how much salary to ask for in their negotiations. Individuals who scored high on the selfworth questionnaire ask for more money on average than individuals who scored low on the self-worth questionnaire. What does this observation tell you about the measurement of self-worth? a. It has high convergent validity b. It has high discriminant validity c. It has high systematic error d. It has high operationalizationism
a
Operational definition
a definition of a concept or variable in terms of precisely described operations, measures, or procedures.
Prediction
a goal of psychological science in which statements about the future occurrence of a behavioral event are made, usually with some probability
Random samples
a sample that fairly represents a population because each member has an equal chance of inclusion; Random sample from the population. Ensures a representative sample by giving each member of the population the same chance of being included in the sample. (need to be careful about nonresponse bias)
Theory
a set of statements that summarizes and organizes existing information about phenomenon, provides an explanation for it, and serves as a basis for making predictions to be tested empirically
Scientific theories (useful, parsimonious, falsifiable)
a statement or set of statements that can be used to explain, predict, and understand a given phenomenon or observation - Summarizes and makes tentative conclusions based on existing empirical knowledge - Proposes explanations - Makes predictions
Normal distribution and its role in null hypothesis testing
allows to see which direction the results are from the null
Hypothesis
an educated guess about a relationship between variables that is then tested empirically
Statistical significance
an observed result (e.g., the difference between two means in an experiment) to the distribution of results expected if the null is true
3) Students in section A and B are given a sleep questionnaire in which they report the amount of sleep they got the night before the quiz. Amazingly, the mean number of hours in the two sections was identical (6.8 hours)! The standard deviations, however, were quite different. In section A, the standard deviation was very small. In section B, the standard deviation was very large. Knowing only what you know in this question, which section is most likely to have someone who failed to get any sleep the night before the quiz? a. Section A b. Section B c. They are equally likely
b
4) By definition, a measurement that does not measure what it is supposed to measure has low: a. Reliability b. Construct validity c. Both of the above d. None of the above
b
5) If a theory cannot be proven wrong, then it is: a. A scientific theory b. Not a scientific theory c. An interesting/important idea d. Not an interesting/important idea
b
7) Control procedures such as counterbalancing are designed to maximize internal validity by eliminating ___________. a. Extraneous variables b. Confounding variables c. Both of the above
b
A researcher is interested in the effect of using Snapchat on feelings of self-worth. To explore the relationship between the two variables, the researcher measures the amount of time participants spend using Snapchat per day (in minutes), and the degree to which they report having strong feelings of self-worth on a scale of 0 to 10 (0 indicating low feelings of self-worth; 10 indicating high feelings of self-worth). 11) Participants take two versions of the self-worth questionnaire. Participants who score high on the first questionnaire also tend to score high on the second questionnaire, whereas participants who score low on the first questionnaire also tend to score low on the second questionnaire. Based only on this information, what do you know to be true? a. The questionnaire has high construct validity b. The questionnaire has high reliability c. The questionnaire is worthless
b
Logic of the experimental method
begin with two identical or equivalent groups of individuals and to do something to the members of one group; Manipulate factor of interest while keeping other factors under control (possible to assess cause-effect relationship)
1) Based on what was discussed in lecture, who can be susceptible to the effects of confirmation bias? a. Everyday people when making inferences based on their own experiences b. Well-meaning scientists when deciding what research questions to ask and how to try to answer them c. Both of the above
c
6) Within-subject designs are typically: a. More powerful than between-subject designs b. More susceptible to order and sequence effects than between-subject designs, and thus tend to require more counterbalancing and control c. Both of the above
c
Working independently, two researchers at two different universities examine the benefits of a new mindfulness intervention program on prospective health outcomes. Both researchers find statistically significant benefits and thus reject the null hypothesis that there are no benefits of the mindfulness program. Researcher A's sample consisted of 50 psychology majors currently attending UCSC. Researcher B's sample consisted of 200 individuals sampled more broadly from the entire U.S. population. 8) Who could have possibly committed a Type I error? a. Researcher A b. Researcher B c. Both researchers could have possibly committed a Type I error d. Neither researcher could have possibly committed a Type I error
c
Basic idea of null hypothesis testing
involves deciding whether the data provide enough evidence to "reject" the null hypothesis
Ways of describing/illustrating data
charts, graphs, diagrams
illusory correlations,
correlation appears to exist, but either does not exist or is much weaker than assumed
Working independently, two researchers at two different universities examine the benefits of a new mindfulness intervention program on prospective health outcomes. Both researchers find statistically significant benefits and thus reject the null hypothesis that there are no benefits of the mindfulness program. Researcher A's sample consisted of 50 psychology majors currently attending UCSC. Researcher B's sample consisted of 200 individuals sampled more broadly from the entire U.S. population. 9) Who could have possibly committed a Type II error? a. Researcher A b. Researcher B c. Both researchers could have possibly committed a Type II error d. Neither researcher could have possibly committed a Type II error
d
Sequence (carryover) effects
form of sequence effect in which systematic changes in performance occur as a result of completing one sequence of conditions rather than a different sequence; Experiencing one condition may affect the experiencing of another condition
Criterion validity
form of validity in which a psychological measure is able to predict some future behavior or is meaningfully related to some other measure; Does the measure predict specific outcomes or behaviors associated with the construct?
Construct validity
in measurement, it occurs when the measure being used accurately assesses some hypothetical construct; also refers to whether the construct itself is valid; in research, refers to whether the operational definitions used for independent and dependent variables are valid; The extent to which a measure taps into the construct of interest (does it measure what it is supposed to measure and not something else?)
Systematic error
may not be problematic if you are interested in relative comparisons (between conditions, pre-post, etc.), as opposed to absolute observations; varies between conditions being compared is very problematic
Four scales of measurement (nominal, ordinal, interval, and ratio)
measurement scale in which the numbers have no quantitative value, but rather identify categories into which events can be placed; measurement scale in which assigned number stand for relative standing or ranking; measurement scale in which numbers refer to quantities and intervals assumed to be of equal size- a score of zero is just one of many points on the scale and does not denote the absence of the phenomenon being measured; measurement scale in which numbers refer to quantities and intervals are assumed to be of equal size- a score of zero denotes the absence of the phenomenon being measured
Random error
not problematic if multiple observations lead to a reliable measurement
convergent validity,
occurs when scores on a test designed t measure some construct (e.g., self-esteem) are correlated with scores on other tests theoretically related to the construct; Does the measure relate to scores on other measures that are theoretically related to the construct?
discriminant validity
occurs when scores on a test designed to measure something construct (e.g., self-esteem) are uncorrelated with scores on other tests theoretically unrelated to the construct; Does the measure not relate to scores on other measures that are theoretically unrelated to the construct?
Ceiling/Floor effects
occurs when scores on two or more conditions are at or near the maximum/minimum possible for the scale being used, giving the impression that no differences exist between the conditions; When scores are too high or too low, you can fail to observe an effect even if there is one
What it means to "reject the null hypothesis"
rejecting the null hypothesis means that whatever was believed before the study was proven to be false from the results founded
Multiple methods and converging evidence
research involves more than one method being used in study; refers to the concept that independent scientific findings acquired with different methodological techniques can come together, or converge, to create strong and well-supported conclusions
Basic research
research with the goal of describing, predicting, and explaining fundamental principles of behavior; Describing, predicting, and explaining fundamental principles of behavior and mental processes.
applied research
research with the goal of trying to solve an immediate real-life problem; Direct and immediate relevance to the solution of real-world problems
Ways to achieve internal validity
researchers need to operationally define IV in a way that isolates construct of interest - Simple/subtle manipulations tend to be easiest to interpret - Big manipulations are easily confounded by other factors unrelated to the construct of interest - Researchers often need to operationalize IVs in multiple ways and provide converging evidence
Ecological validity
said to exist when research studies psychological phenomena in everyday situations (e.g. memory for where we put our keys); Extent to which the conclusions of a study can be generalized to natural, real-life settings and situations.
Confirmation bias,
social cognition bias in which events that confirm a strongly held belief are more readily perceived and remembered; disconfirming events are ignored or forgotten; We search for, interpret, favor, and recall information in a way that confirms preexisting beliefs and hypotheses • Ignore/forget contradicting evidence (memory is designed to function this way) • Belief Perseverance • Anecdotal "evidence"
Tentative nature of the scientific method
subject to change in the light of new evidence or new interpretation of existing evidence
Reliability of a measurement
the extent to which measures of the same phenomenon are consistent and repeatable; measures high in reliability contain a minimum of measurement error; The extent to which a measurement gives the same result on different occasions
Central tendency (mode, median, mean)
the most frequently appearing score in a data set; the middle score of a data set- an equal number of scores is both above and below the median; the arithmetic average of a data set found by adding the scores and dividing by the total number of scores in the set.; The most frequently occurring score in a sample; The middle score in a rank-ordered sample; The average of scores in a sample
Psychological science
the scientific method is the backbone of psychology
Skewed distributions; outliers; interpreting the mean
things that can change the interpretation of the results
inferential statistics
used to draw about the broader population on the basis of a study using a sample of that population; Method of drawing conclusions about data to make generalizations
