Methods: Ch.5
Which of the following correlations are the strongest? .25, -.65, -.01, .43
-.65
Name the three common ways in which researchers operationalize their variables.
1. Self-Report 2. Observational 3. Physiological
What relationship must you have to have a good test-retest reliability?
>.05
What relationship must you have to have a good interrater reliability?
>.07
Internal reliability
Consistency in a pattern of answers, no matter how a question is phrased
Frequency
Count of the number of behaviors that occur in a given period of time (# social interactions of kids on playground)
what is an example of Self-report measure?
Diener's 5-point scale on life satisfaction
What are the types of validity?
Face Content Criterion Convergent Discriminant
Physiological measures
Record biological data
why is it important to you to measure things?
To design experiments To interrogate research claims
Interrater reliability
Two or more independent observers should come up with the same (or very similar) findings
cronbach's alpha
a correlation based statistic that measures a scales internal reliability.
strength
a description of an association indicating how closely the data points in a scatterplot cluster along a line of best fit drawn though them
physiological measure
a variable measured by recording biological data
discriminant validity
an empirically supported type of measurement validity that represents the extent to which a measure does not associate strongly with measures of other, theoretically different constructs.
convergent validity
an empirically supported type of measurement validity that represents the extent to which a measure is associated with other measures of a theoretically similar construct.
Which requires stronger correlations for its evidence: convergent validity or discriminant validity?
convergent validity
Nominal Scale
events are assigned to categories
Nominal Scale
events are assigned to categories.
Can kids self-report?
yes
quantitative variable
a variable whose values can be recorded as meaningful numbers
What are some types of behavioral measures?
frequency, latency, and number of errors
internal reliability
in a measuring instrument that contained several items, the consistent in a pattern of answers, no matter how a question is phrased.
What do predictive and concurrent validity have in common? How are they different?
Predictive predicts the correlation between the results and something in the future and concurrent predicts the correlation between the results and something right now.
slope direction
the upward, downward, or neutral slope of the cloud of points in a scatterplot
Can self-report can be replaced (or augmented) by parent or teacher reports?
yes
Criterion Validity
Both address how well a measure relates to a specific outcome How well the measure predicts a future outcome How well the measure correlates with a current outcome
What do face and content validity have in common?
Both involve subject judgment about a measure.
In your own words, describe the difference between categorical and quantitative variables.
Categorical are categories, nominal variables. Examples are sex, female male. They might assign numbers to these categories, like 1=male, 2=female. There is so significance to these numbers. Quantitative variables are coded with meaningful numbers. Examples are height and weight.
What is an example of a self-report measure?
Diener's 5-point scale on life satisfaction
What are types of validity?
Face Content Criterion Convergent Discriminant
what are the three types of behavioral(observational) measures?
Frequency: Count of the number of behaviors that occur in a given period of time (# social interactions of kids on playground) Latency: The amount of time it takes for a behavior to occur (reaction time; how long before kids on playground begin to interact) Number of errors: The number of incorrect responses made (for reaction time, etc)
Observational (Behavioral) measures examples are what?
Happiness: Record how many times a day someone smiles
Measurement reliability versus Measurement validity
Head circumference measures are reliable... ...They are not valid as a measure of intelligence
Face validity
Is this a plausible measure of the variable in question? Life satisfaction as a measure of happiness? Head circumference as a measure of intelligence? Head circumference as a measure of hat size? Rapidity of problem solving as a measure of intelligence?
Self-report measures
Participants report on their own behavior or state of mind
what are the Scales of measurement?
Nominal Ordinal Interval Ratio
What is considered quantitative on the scale of measurement?
Ordinal Interval Ratio
Observational (Behavioral) measures
Record observable behaviors or physical traces of behaviors
How do I know if a study has construct validity?
Reliability: how consistent is the measurement? Validity: is it measuring what it's supposed to measure?
How would you evaluate reliability?
Scatterplots Correlation coefficient, r Measures of reliability Test-retest Interrater Internal
how do you evaluate reliability?
Scatterplots Correlation coefficient, r Measures of reliability Test-retest Interrater Internal
What are the 3 different types of measures?
Self-report measures Observational measures Physiological measures
What are the three types of measures?
Self-report measures Observational measures Physiological measures
For each of the three common times of operationalization's- self-report, observational, and physiological - indicate which types of reliability would be relevant.
Self-report: test-retest and internal may be relevant. Observational: interrater would be relevant; psychological: interrater may be relevant.
Reliability is about consistency. Define the three kinds of reliability, noting what kind of consistency each is designed to show.
Test-Retest: researcher gets consistent results every time they use the measure. Interrater: consistent results are obtained no matter who measures or observes. Internal: a study participant gives a consistent pattern of answers, no matter how the researcher has phrased the question.
What are the three types of reliability?
Test-retest reliability Interrater reliability Internal reliability
Explain why a variable will usually have only one conceptual definition but can have multiple operational definitions.
The conceptual definition, or construct, is the researchers definition of the variable in question at an abstract level (like happiness). The operational definition of a variable represents a researcher's specific decision about how to measure or manipulate the conceptual variable (giving people a scale to say how happy they are).
Number of errors
The number of incorrect responses made (for reaction time, etc)
Why is it important for you to measure things (motivation, emotion, thinking, ect.)?
To design experiments To interrogate research claims
Construct validity
a measurement of how well a variable was measured (or manipulated...but that's later) in a study
self-report measure
a method of measuring a variable in which people answer questions about themselves in a questionnaire or interview
interval scale
a quantitative measurement scale that has no "true zero" and in which the numeral represent equal intervals (distances) between levels (temperature in degrees)
ratio scale
a quantitative scale of measurement in which the numerals have equal intervals and the value of zero truly means "nothing"
ordinal scale
a quantitative scale whose levels represent a ranked order, in which it is unclear whether the distances between levels are equivalent
correlation coefficient
a single number ranging from -1.0 to 1.0 used to indicate the strength and direction of an association.
observational measure
a variable measured by recording observable behaviors or physical traces of behaviors
categorical variable
a variable whose levels are categories
predictive validity
an empirically supported type of measurement validity that represents the extent to which a measure is related to a concrete, future outcome that it should be related to.
concurrent validity
an empirically supported type of measurement validity that represents the extent to which a measure is related to a concrete, simultaneous outcome that it should be related to
What is considered categorical on the scales of measurement?
nominal
Ordinal Scale
permits events or scores to be rank ordered (1st,2nd place)
Ordinal Scale
permits events or scores to be rank ordered.
Ratio Scale
permits rank ordering of events (like ordinal) with the assumption of equal intervals between adjacent events (like interval) and a true zero point.
Ratio scale
permits rank ordering of events (like ordinal) with the assumption of equal intervals between adjacent events (like interval) and a true zero point.
reliability
the consistent of a measure
interrater reliability
the degree to which two or more coders or observers agree in their ratings of a set of targets.
content validity
the extent to which a measure captures all parts of a defined construct
face validity
the extent to which a measure is subjectively considered a plausible operationalization of the conceptual variable in question
Content Validity
Does it capture all parts of a defined construct? Intelligence: Ability to reason Ability to plan Abstract thinking Comprehension of complex ideas Ability to learn quickly Etc.
Describe the difference between ordinal, interval, and ratio scales.
Ordinal - allow us to rank order the levels of the variables being studied; instead of having categories that are simply different as in nominal scale, the categories can be ordered from first to last (Ex. ranking TV shows by popularity, 2- 3- 4- star restaurants) Interval - the difference between the numbers on the scale is meaningful, the intervals between the numbers are equal in size (Ex. intelligence, aptitude test score, temperature) (although these scales might have a zero point, that zero is an arbitrary reference point and does not indicate the absence of that variable) Ratio - has an absolute zero point that indicates the absence of the variable being measured (Ex. reaction time, weight, age, frequencies of behaviors).
Test-retest reliability
Test/inventory is given twice... scores from the two tests are compared Greater the similarity = higher the reliability
Latency
The amount of time it takes for a behavior to occur (reaction time; how long before kids on playground begin to interact)
Measured variables
Variables whose levels are observed and recorded (with no manipulation)
Interval Scale
permits rank ordering of events with the assumption of equal intervals between adjacent events (no absolute zero, like when temperature is zero its still a temperature)
Interval Scale
permits rank ordering of events with the assumption of equal intervals between adjacent events.