Chapter 5
Convergent Validity
An empirical test of the extent to which a measure is associated with other measures of a theoretically similar construct.
Cronbach's Alpha
A correlation-based statistic that measures a scale's internal reliability.
Known-Groups Paradigm
A method for establishing criterion validity, in which a researcher tests two or more groups, who are known to differ on the variable of interest, to ensure that they score differently on a measure of a variable.
Internal Reliability
A participant gives a consistent pattern of answers no matter how how the researcher phrases the question.
Interval Scale
A quantitative measurement scale that has no "true zero," and in which the numerals represent equal intervals between levels (temperature scale, IQ scale (a 0 does not mean no intelligence). A person can get a score of zero but it does not mean "nothing").
Ordinal Scale
A quantitative measurement scale whose levels represent a ranked order. It is unclear whether the differences between levels are equal. (top 10 best selling books, you know the order in which they are most sold, but not how many more book #1 sold than book #2).
Ratio Scale
A quantitative scale of measurement in which the numerals have equal intervals and the value of zero truly means "nothing." (frequencies, 0 answers correct).
Quantitative Variables
A variable whose values can be recorded as meaningful numbers (height, weight IQ score)
Josiane has found an online test that claims to measure IQ. It consists of choosing the correct definitions for a series of words. She is concerned that it doesn't include any tests of other things that are part of IQ, such as problem solving or visual-spatial ability. Which type of validity is she questioning? Question options: A. content validity B. face validity C. criterion validity D. discriminant validity
A. content validity Because: Face Validity and Content Validity: Does It Look Like a Good Measure? — Her concern is that the test does not capture all parts of the construct of intelligence. In her subjective judgment, parts of the construct of intelligence are not included in a vocabulary test.
Lorenzo is studying aggression in children. First, Lorenzo administers a questionnaire to the children that asks them about their feelings of aggression. Then Lorenzo and his lab partner observe the children while they play and record instances of aggression. The results of these two parts of the study are compared. The total number of instances of aggression for each child is used as the measure in the observational part of the study. What type of quantitative variable is this? Question options: A. ratio scale B. interval scale C. ordinal scale D. categorical
A. ratio scale Because: Scales of Measurement — The total instances of aggression for each child is a ratio scale measurement, since a child could have a score of 0 (no aggressive instances) and it would be meaningful to say one child was twice as aggressive as another.
The Department of Motor Vehicles receives a complaint that some of their employees who administer the road test pass a much higher percentage of test-takers than other employees. In this example, what aspect of the road test is being questioned? Question options: A. the interrater reliability of the road test B. the internal reliability of the road test C. the test-retest reliability of the road test D. the measurement validity of the road test
A. the interrater reliability of the road test Because: Introducing Three Types of Reliability — This is a question of whether two observers would have similar findings and the complaint is asserting that they wouldn't, thus it is an interrater reliability question.
Categorical Variable
Also called nominal variables, are variables whose levels are categories (male or female, declared major, etc.)
Criterion Validity
An empirical form of measurement validity that established the extent to which a measure is correlated with a behavior or concrete outcome that it should be related to.
Discriminant Validity
An empirical test of the extent to which a measure does not associate strongly with measures of theoretically different constructs.
Julie has developed an intervention to improve the relationship between parents and pre-school-aged children. In order to evaluate the effectiveness of her intervention, Julie video records the parents interacting with their children at the end of the study. She has two research assistants watch the videos and rate the level of warmth in the interaction. Julie then correlates the ratings of the raters. She finds a high positive correlation (r = .87) between the two raters. What type of reliability is she examining? Question options: A. test-retest B. interrater C. internal D. construct
B. Interrater
Dr. Johnson wants to do a study to investigate whether the physiological measure, heart rate variability, varies over time or whether it is a trait that stays stable within the same person over time. He records participants' heart rate variability once at the beginning of the semester and once at the end of the semester. He finds a high positive correlation (r = .55) between the first and second time points. What would a scatterplot of these results (heart rate variability at the beginning of the semester on the x-axis, heart rate variability at the end of the semester on the y-axis) look like? Question options: A. The cloud of points would have no slope at all. B. The cloud of points would slope upward from left to right. C. The cloud of points would slope downward from left to right. D. There isn't enough information given to determine this.
B. The cloud of points would slope upward from left to right.
Julie has developed an intervention to improve the relationship between parents and pre-school-aged children. In order to evaluate the effectiveness of her intervention, Julie video records the parents interacting with their children at the end of the study. She has two research assistants watch the videos and rate the level of warmth in the interaction. Julie then correlates the ratings of the raters. She finds a high positive correlation (r = .87) between the two raters. What would a scatterplot of these results (ratings by the first research assistant on the x-axis, ratings of the second research assistant on the y-axis) look like? Question options: A. The cloud of points would have no slope at all. B. The cloud of points would slope upward from left to right. C. The cloud of points would slope downward from left to right. D. There isn't enough information given to determine this.
B. The cloud of points would slope upward from left to right. Because: Using the Correlation Coefficient r to Evaluate Reliability — A positive correlation coefficient means that there is an upward (from left to right) slope. The higher ratings of warmth by the first rater correspond to higher ratings of warmth by the second rater.
Some colleges no longer require the SAT I or the ACT tests, instead basing their admissions on other factors, such as high school GPA. A large reason that they have done this is that they have found a low correlation between the scores on the tests and the students' freshman year GPA. In other words, they were concerned that college entrance exams lacked which type of validity? Question options: A. face validity B. criterion validity C. discriminant validity D. content validity
B. criterion validity Because: Criterion Validity: Does It Correlate with Key Behaviors? — The concern is the lack of criterion validity of these tests. The test scores did not predict the freshman year GPA.
Dr. Nolan gives his new anxiety measure to a group of his colleagues who are anxiety experts. They agree that the questions on the measure appear to assess anxiety symptoms. This suggests that Dr. Nolan's measure has which of the following types of measurement validity? Question options: A. content validity B. face validity C. criterion validity D. discriminant validity
B. face validity Because: Face Validity and Content Validity: Does It Look Like a Good Measure? — Face validity means that a measure appears to be a plausible or reasonable measure of the variable.
In a study of aggression in children, a researcher has his undergraduate research assistants watch a group of children on the playground and record the number of instances of physical or verbal attacks. Which category of measured variable is this researcher using? Question options: A. self-report measures B. observational measures C. physiological measures D. neuropsychological measures
B. observational measures Because:
In a study of aggression in children, a researcher has his undergraduate research assistants watch a group of children on the playground and record the number of instances of physical or verbal attacks. Which category of measured variable is this researcher using? Question options: A. self-report measures B. observational measures C. physiological measures D. neuropsychological measures
B. observational measures Because: Three Common Types of Measures — The researcher is recording the observational behaviors of the children by recording acts of aggression.
What information can you learn from a scatterplot that you cannot learn from the correlation coefficient? Question options: A. the direction of the relationship B. the values for each pair of measurements C. the strength of the relationship D. whether the relationship is statistically significant
B. the values for each pair of measurements
Which statistic is used to represent the internal reliability of multiple-item self-report scales? Question options: A. r, the correlation coefficient B. Kappa C. Cronbach's alpha D. s, the standard deviation
C. Cronbach's alpha Because: Using the Correlation Coefficient r to Evaluate Reliability — Cronbach's alpha is a statistic based on the average of inter-item correlations. It is used to assess internal reliability of a scale.
Mendoza et al. (2009) introduced a coin rotation task as a convenient test of motor dexterity. It involves timed completion of twenty 180° rotations of a nickel using the thumb, index, and middle fingers. The results were compared to the results of another widely used test of motor dexterity, the finger-tapping task, in which participants tap their index fingers as many times as possible in 10 seconds. The results indicated that there was a statistically significant relationship between the finger-tapping task and the coin rotation task (r = -.40). What would a scatterplot of these results (coin rotation scores on the x-axis, finger-tapping scores on the y-axis) look like? Question options: A. The cloud of points would have no slope at all. B. The cloud of points would slope upward from left to right. C. The cloud of points would slope downward from left to right. D. There isn't enough information given to determine this.
C. The cloud of points would slope downward from left to right.
Which of the following is an example of a categorical variable? Question options: A. IQ score B. blood pressure reading C. declared major in college D. current age
C. declared major in college Because: Scales of Measurement — Categorical or nominal variables are those that fit into categories. Majors in college, such as psychology, business, or biology, are categorical.
Lorenzo is studying aggression in children. First, Lorenzo administers a questionnaire to the children that asks them about their feelings of aggression. Then Lorenzo and his lab partner observe the children while they play and record instances of aggression. The results of these two parts of the study are compared. Lorenzo runs a statistical test to find how consistent the responses are to different wordings of items on the questionnaire. What type of reliability is he examining? Question options: A. test-retest B. interrater C. internal D. construct
C. internal Because: Using the Correlation Coefficient r to Evaluate Reliability — Lorenzo has more than one question measuring the same construct so he needs to check the internal reliability, or whether their responses are consistent.
Dr. Kamran studies domestic violence and has designed a self-report scale that is meant to assess men's negative attitudes toward women. To validate her scale, she administers it to two groups of recently incarcerated male prisoners: prisoners convicted of domestic violence and prisoners convicted of other crimes. Dr. Kamran finds a statistically significant difference in the mean scores of the two groups. What technique is Dr. Kamran using to validate her scale? Question options: A. test-retest technique B. physiological measurements C. known-groups paradigm D. interrater reliability test
C. known-groups paradigm Because: Known-Groups Evidence for Criterion Validity — She is using two known groups of people, some who committed domestic violence and some who didn't, to test concurrent validity of her scale.
Which of the following is an example of a physiological measure? Question options: A. responses to a questionnaire B. speed in solving a puzzle C. skin conductance D. ratings by an observer
C. skin conductance Because: Three Common Types of Measures — Skin conductance is the only one of these that operationalizes a variable by recording biological data.
Interrater Reliability
Consistent scores are obtained no matter who measures or observes.
Sun Mi is designing a questionnaire on loneliness. She is concerned that some features of loneliness are similar to depression and to low self-esteem. What type of validity does she need to show to demonstrate that her questionnaire assesses loneliness and not depression or low self-esteem? Question options: A. convergent validity B. criterion validity C. face validity D. discriminant validity
D. discriminant validity Because: Convergent Validity and Discriminant Validity: Does the Pattern Make Sense? — Sun Mi wants to show that her results diverge from the results of those that depression or low self-esteem would produce.
Lorenzo is studying aggression in children. First, Lorenzo administers a questionnaire to the children that asks them about their feelings of aggression. Then Lorenzo and his lab partner observe the children while they play and record instances of aggression. What type of measure is the questionnaire? Question options: A. ordinal scale B. observational C. physiological D. self-report
D. self-report Because: Three Common Types of Measures — The questionnaire used to ask children to report their own aggression is a self-report measure.
Professor Morgan questions whether the ratings he receives from his students on "teaching effectiveness" indicate how much the students learn in his class or whether they are just a reflection of how much his students like him. What aspect of the ratings is he questioning? Question options: A. the reliability of the ratings B. the statistical significance of the ratings C. the use of an interval scale D. the measurement validity of the ratings
D. the measurement validity of the ratings Because: Measurement Validity of Abstract Constructs — Professor Morgan is concerned about whether the tool used to assess his teaching effectiveness actually measures that construct or some other construct.
Reliability
How consistent the results of a measure are.
Physiological Measure
Operationalizes a variable by recording biological data such as brain activity, hormone levels, or heart rate.
Self-Report Measure
Operationalizes a variable by recording people's answers to questions about themselves in a questionnaire or interviews.
Observational Measure
Sometimes called a behavioral measure, operationalizes a variable by recording observable behaviors or physical traces of behaviors.
Correlation Coefficient
Symbolized as r. A single number, ranging from -1.0 to 1.0, that indicates the strength and direction of an association between two variables. The closer a correlation coefficient is to 0, the weaker the relation between the two variables.
Validity
The appropriateness of a conclusion or decision.
Content Validity
The extent to which a measure captures all parts of a defined construct.
Face Validity
The extent to which a measure is subjectively considered a plausible operationalization of the conceptual variable in question.
Test-Retest Reliability
The researcher gets consistent scores every time he or she uses the measure.
