Chapter 15: Reliability and Validity

A researcher is measuring the construct "empowerment" in two groups of individuals using an instrument. One group is expected to score extremely high and the other group is expected to score extremely low. The differences in scores are then examined. What is the approach being used and the type of validity being measured? A. Contrasted-groups approach to test construct validity B. Divergent approach to test criterion-related validity C. Multitrait-multimethod approach to test construct validity D. Convergent approach to test face validity

A. In the contrasted-groups approach, the researcher administers an instrument to two groups of individuals expected to score extremely high and low in the characteristic being measured. Construct validity is being measured. The divergent approach uses measurement approaches that differentiate one construct from others that may be similar. Criterion-related ability is not being measured. The multitrait-multimethod approach involves examining the relationship between instruments that should measure the same construct and between those that should measure different constructs. Construct validity is being measured. In the convergent approach, two or more tools that theoretically measure the same construct are identified and administered to the same subjects. Face validity is not being measured.

Which is an example of test-retest reliability? A. Bhandari and Kim's study investigating self-care behaviors of Nepalese adults B. Ganz, Toren, and Fadion's study that examined whether nurses fully implement their scope of practice C. Jessee and Tanner's study to develop a Clinical Coaching Interactions Inventory D. Johansson, Jansson, and Lindmark's study evaluating the oral health of older adults in Sweden

B Ganz, Toren, and Fadion's 2016 study is known to be an example of test-retest reliability.

Which reliability measure is considered to be a test of stability? A. Split-half reliability B. Test-retest reliability C. Parallel, or alternate-form, reliability D. Cronbach's alpha

B Test-retest reliability is considered a test of stability. Split-half reliability is considered a test of homogeneity. Parallel, or alternate-form, reliability is considered a test of equivalence. Cronbach's alpha is considered a test of homogeneity.

Which measurement would a researcher use to test for reliability when the data are in dichotomous ("yes/no") format? A. Chronbach's alpha B. KR-20 coefficient C. Parallel, or alternate-form, reliability D. Split-half reliability

B The KR-20 coefficient is used to estimate the homogeneity of instruments. Cronbach's alpha measures the homogeneity of an instrument with a Likert-type format. Parallel, or alternate-form, reliability is an approach to measuring the stability of an instrument, regardless of its format. Split-half reliability is an approach to test for the reliability of an instrument, regardless of its format.

A Cronbach's alpha coefficient of 0.94 was obtained for an instrument. What does this indicate? A. A low degree of validity B. A high degree of internal consistency C. A low degree of homogeneity D. A high degree of stability

B The higher the coefficient, the higher the degree of internal consistency. Cronbach's alpha tests for reliability, not validity. The higher the coefficient, the higher the degree of homogeneity. Cronbach's alpha does not test for the stability of an instrument.

What does the nurse researcher designing a study recognize, regarding reliability and validity? (Select all that apply.) A. A measure can be considered to be valid while unlikely found to be reliable. B. A measure can be considered to be reliable while unlikely found to be valid. C. A measure can be considered to be neither reliable nor valid. D. A measure can be found to be both reliable and valid.

B, C, D A measure can be reliable without being valid. A measure can be neither reliable nor valid. A measure can be both reliable and valid. A measure cannot be valid if it is not reliable.

All items in a tool should measure the same concept or characteristic. This is known as what? A. Stability B. Homogeneity C. Equivalence D. Precision

B. Homogeneity means that all items in a tool measure the same concept or characteristic. Stability means that the same results are obtained on repeated administration of the instrument. Equivalence is consistency among observers using the same instrument tool. Precision refers to the accuracy of a measurement tool.

A nurse researcher divides a questionnaire in half, with the even-numbered items in one group and the odd-numbered items in a second group, and calculates a reliability coefficient for the two halves. What method of determining reliability is the researcher using? A. Item-to-total correlations B. Split-half reliability C. Cronbach's alpha D. KR-20 coefficient

B. Split-half reliability involves dividing a scale into two halves and making a comparison. Item-to-total correlations measure the relationship between each of the items and the total scale. Cronbach's alpha measures the homogeneity of an instrument with a Likert-type format. The KR-20 coefficient is used to estimate the homogeneity of instruments with a dichotomous response format.

The extent of variability in test scores that is attributable to error rather than a true measure of the dependent variable is known as what? A. Chance error B. Error variance C. Constant error D. Stability

B. The error variance is the difference in scores between the actual measurement of behavioral changes and the extent of variability in test scores. Chance error is the variability of scores due to random confounding variables such as participant anxiety. Constant error is due to relatively stable confounding variables in the study such as incorrect instrument calibration. Stability is a test of reliability.

The nurse researcher finds a Kappa level of 0.90. What does this indicate? A. Good interrater reliability B. Agreeable/substantial interrater reliability C. Tentative conclusions can be drawn D. No interrater reliability

C A Kappa level of 0.90 indicates good interrater reliability.

Which is an example of Kuder-Richardson Formula 20? A. Bhandari and Kim's study investigating self-care behaviors of Nepalese adults B. Ganz, Toren, and Fadion's study that examined whether nurses fully implement their scope of practice C. Jessee and Tanner's study to develop a Clinical Coaching Interactions Inventory D. Johansson, Jansson, and Lindmark's study evaluating the oral health of older adults in Sweden

C Jessee and Tanner's 2016 study is known to be an example of KR-20 usage.

Which reliability measure is considered to be a test of equivalence? A. Split-half reliability B. Test-retest reliability C. Parallel, or alternate-form, reliability D. Cronbach's alpha

C Parallel, or alternate-form, reliability is considered a test of equivalence. Split-half reliability is considered a test of homogeneity. Test-retest reliability is considered a test of stability. Cronbach's alpha is considered a test of homogeneity.

What attribute of an instrument does the KR-20 coefficient measure? A. Internal validity B. Equivalence C. Homogeneity D. Stability

C The Kuder-Richardson (KR-20) coefficient is used to estimate the homogeneity of instruments with a dichotomous response format. The Kuder-Richardson (KR-20) coefficient measures reliability, not validity. Equivalence refers to consistency among observers using the same instrument tool. Stability means that the same results are obtained on repeated administration of the instrument.

The nurse researcher uses an instrument that shows r = 0.85. What does this indicate? A. Internal consistency B. Convergent validity C. Test-retest reliability D. Content validity

C This measurement is indicative of test-retest reliability.

Researchers used an instrument to measure self-esteem in adolescent mothers. To measure the validity of this instrument, they used a second instrument known to measure self-esteem in women. What type of validity were they measuring? A. Predictive validity B. Face validity C. Concurrent validity D. Construct validity

C. Concurrent validity refers to the degree of correlation of two measures of the same concept administered at the same time. Predictive validity refers to the degree of correlation between the measure of the concept and some future measure of the same concept. Face validity refers to expert verification that the instrument measures what it purports to measure. Construct validity refers to the extent to which a test measures a theoretical construct or trait.

When a subject is answering a question in a socially desirable way, the data will contain what? A. Error variance B. Random error C. Constant error D. Unreliability

C. Social desirability is a form of constant error. Error variance is the total of all errors included in a test score, not just the type of error produced by social desirability. Social desirability is not a form of random error. The validity, not the reliability, of the data is impacted in the case of socially desirable responses.

Which are examples of the use of interrater reliability and kappa? A. Bhandari and Kim's study investigating self-care behaviors of Nepalese adults B. Ganz, Toren, and Fadion's study that examined whether nurses fully implement their scope of practice C. Jessee and Tanner's study to develop a Clinical Coaching Interactions Inventory D. Johansson, Jansson, and Lindmark's study evaluating the oral health of older adults in Sweden

D Johansson, Jansson, and Lindmark's 2016 study is known to be an example of interrater reliability and kappa.

Interrater reliability is expressed as what? A. The total number of differences among scorers subtracted from the total number of items B. The total number of differences among scores C. The average score among scorers D. A percentage of agreement between scores

D When two or more investigators are collecting data, interrater reliability is determined by calculating the percentage of agreement between their scores. This number would not measure the consistency among scorers. This number would be meaningless unless the total number of scores was known. An average score would not allow for comparisons among scorers.

Which reliability measures are considered to be tests of stability? (Select all that apply.) A. Item-to-total correlation B. Split-half reliability C. Interrater reliability D. Parallel, or alternate-form, reliability E. Test-retest reliability

D, E Parallel, or alternate-form, reliability is considered a test of both stability and equivalence. Test-retest reliability is considered a test of stability. Item-to-total correlation is considered a test of homogeneity. Split-half reliability is considered a test of homogeneity. Interrater reliability is considered a test of equivalence.

What does a test score or measurement consist of? A. Random errors plus constant errors B. Random errors plus the true score C. Constant errors plus error variance D. Error variance plus the true score

D. A test score includes the inherent bias of the study (error variance) and the true measurement. A test score also contains true measurement. Other types of errors are included in the test score. A test score also contains true measurement.

The nurse researcher decides to administer two tools that theoretically measure the same construct to the same subjects and perform a correlational analysis. This is used in an effort to test what type of validity? A. Predictive validity B. Face validity C. Concurrent validity D. Construct validity

D. Construct validity refers to the extent to which a test measures a theoretical construct or trait. Predictive validity refers to the degree of correlation between the measure of the concept and some future measure of the same concept. Face validity refers to expert verification that the instrument measures what it purports to measure. Concurrent validity refers to the degree of correlation of two measures of the same concept administered at the same time.

The relationship between the error variance, the true variance, and the observed score is known as what? A. Parallel reliability B. Measurement error C. Homogeneity D. The reliability coefficient

D. The reliability coefficient measures the relationship among the error variance, the true variance, and the observed score. Parallel reliability refers to using two comparable forms of an instrument to test the same concept. Measurement error is error that occurs because of problems with calibration of equipment or investigator error in measurement. Homogeneity means that all items in a tool measure the same concept or characteristic.

Which is an example of the use of internal consistency? A. Bhandari and Kim's study investigating self-care behaviors of Nepalese adults B. Ganz, Toren, and Fadion's study that examined whether nurses fully implement their scope of practice C. Jessee and Tanner's study to develop a Clinical Coaching Interactions Inventory D. Johansson, Jansson, and Lindmark's study evaluating the oral health of older adults in Sweden

A Bhandari and Kim's 2016 study is known to be an example of internal consistency.

Which measurement would a researcher use to test for reliability when the data are in Likert-scale response format? A. Chronbach's alpha B. KR-20 coefficient C. Parallel, or alternate-form, reliability D. Split-half reliability

A Cronbach's alpha measures the homogeneity of an instrument with a Likert-type format. The KR-20 coefficient is used to estimate the homogeneity of instruments with a dichotomous response format. Parallel, or alternate-form, reliability is an approach to measuring the stability of an instrument, regardless of its format. Split-half reliability is an approach to test for the reliability of an instrument, regardless of its format.

The administration of the same instrument to the same subjects under similar conditions on two or more occasions is considered to be: A. test-retest reliability. B. parallel or alternate form reliability. C. item to total correlations. D. split-half reliability.

A Test-retest reliability is considered to be the administration of the same instrument to the same subjects under similar conditions on two or more occasions.

A study shows the KR-20 coefficient to be 0.01. What does this indicate? A. No correlation among the error variance, true variance, and observed score B. An almost perfect correlation among the error variance, true variance, and observed score C. Low measurement error D. An equal balance between the error variance and true variance

A The lower the coefficient, the lower the correlation. A coefficient only indicates a degree of correlation, not measurement error. A coefficient only indicates a degree of correlation, not measurement error.

What should be chosen for a nurse researcher who selects tests to estimate the stability of the instrument their study? (Select all that apply.) A. Test-retest reliability B. Kuder-Richardson (KR-20) coefficient C. Parallel reliability D. Cronbach's alpha

A, C Test-retest measures the stability of an instrument. Parallel reliability measures the stability of an instrument. Cronbach's alpha measures the homogeneity of an instrument. The Kuder-Richardson (KR-20) coefficient measures the homogeneity of an instrument.

Which reliability measure is considered to be a test of homogeneity? (Select all that apply.) A. Split-half reliability B. Test-retest reliability C. Parallel, or alternate-form, reliability D. Cronbach's alpha

A, D Split-half reliability is considered a test of homogeneity. Cronbach's alpha is considered a test of homogeneity. Test-retest reliability is considered a test of stability. Parallel, or alternate-form, reliability is considered a test of equivalence.

Researchers administered one tool that measured the concept of hope and another that measured the concept of anxiety to the same group of subjects. The scores on the first instrument were negatively related to the scores on the second instrument. This is a measure of what type of validity? A. Divergent validity B. Convergent validity C. Face validity D. Construct validity

A. Divergent validity uses measurement approaches that differentiate one construct from others that may be similar. Convergent validity refers to a search for other measures of the construct by administering two or more tools to the same subjects. Face validity refers to expert verification that the instrument measures what it purports to measure. Construct validity refers to the extent to which a test measures a theoretical construct or trait.