Module 6.05: Reliability in Research Studies

Reliability is the consistency in which an instrument measures an attribute. Which of the following are aspects of reliability that are important to researchers? (Select all that apply.)

Internal consistency, Test-retest, Stability Rationale: Important aspects of reliability include stability, internal consistency, and test-retest reliability. Stability of an instrument is the degree to which similar results are obtained on a repeated basis. This is evaluated through test-retest reliability procedures. Internal consistency is the extent to which items measure the same trait.

Which reliability coefficient indicates the best reliability?

r .95 Rationale: Researchers compute a reliability coefficient, a numeric index that quantifies an instrument's reliability, to objectively assess small differences attributed to time related influences such as participant fatigue. Results of a test given to a sample population at two different points in time—for example, two weeks apart—would not be identical but should be similar. Researchers can validate the measurement tool's reliability by computing the reliability coefficient. Reliability coefficients (designated as r) range from .00 to 1.00. The higher the value, the more reliable (stable) is the measuring instrument.

A nurse researcher has developed and tested a new instrument. Which of the following coefficient alpha results indicates that the researcher's instrument is highly reliable?

0.82 Rationale: The values for coefficient alpha range from 00 to +1.00. The higher the coefficient, the more accurate or internally consistent the measure. Therefore, an alpha of .82 is closer to +1.00 and is the most accurate.

Which measurement is used when determining the internal consistency of a new instrument?

Cronbach's alpha Rationale: Internal consistency is evaluated by calculating Cronbach's alpha also referred to as coefficient alpha.

What aspects of reliability are important to quantitative researchers? (Select all that apply.)

Stability, Equivalence, Internal consistency Rationale: Three aspects of reliability of interest to quantitative researchers are stability, internal consistency, and equivalence. Stability refers to the ability of a scale or measure tool to obtain similar results on separate occasions. Internal consistency ensures that a scale or measurement tool evaluates the same trait and nothing else. Equivalence is the degree to which two or more observers or coders agree about the scoring on a specific measurement tool.

Internal consistency is evaluated by calculating:

coefficient alpha (Cronbach's alpha). Rationale: Internal consistency is evaluated by calculating coefficient alpha, also referred to as Cronbach's alpha. T-tests assess whether the means of two populations are statistically different from each other. Composite scales are evaluated for internal consistency by using the coefficient alpha. Statistical power refers to the capacity to detect true relationships.

The degree to which two or more observers or coders agree about the scoring in an instrument is considered the:

interrater reliability. Rationale: The degree of error can be assessed through interrater or interobserver reliability procedures, which involve having two or more observers or coders make independent observations. An index of agreement is then calculated with these data to evaluate the strength of the relationship between the ratings. When two independent observers score some phenomenon congruently, the scores are likely to be accurate and reliable. Test-retest, internal consistency, and sample variations are statistical terms not connected to interrater reliability.

The stability of an instrument is described as:

the degree to which similar results are obtained on separate occasions. Rationale: The stability of an instrument is described as the degree to which similar results are obtained on separate occasions. The reliability estimate focuses on the instrument's susceptibility to time-related influences, such as participant fatigue. Stability is assessed through test-retest reliability procedures. Researchers administer the measure to a sample twice and then compare the scores. Researchers compute a reliability coefficient, a numeric index that quantifies an instrument's reliability, to assess objectively how small the differences are. Reliability coefficients (designated as r) range from .00 to 1.00. A wide variation in results from one occasion to the next would indicate that the instrument is not stable.