Validity, Reliability, Precision, Accuracy

Convergent validity

Agreement between instruments that measure same construct Correlate with similar scales availabl

Internal Validity

Occurs when a researcher controls all extraneous variables and the only variable influencing the results of a study is the one being manipulated by the researcher. This means that the variable the researcher intended to study is indeed the one affecting the results and not some other, unwanted variables. (Take a look at the definition for "confound") There are several "threats to internal validity" including: history, maturation of participants, testing, instrument decay, and statistical regression

Criterion related validity

Test's correlation with a similar measure; if the related measure.

Precision

The degree of resemblance among study results, were the study to be repeated under similar circumstances. Simply put: How close the measurements are to each other. Lack of precision is referred to as 'random error'.

Predictive validity

Whether test result accurately predicts some external criterion The relationship between test scores and later performance on a knowledge, skill or ability. SATs are said to have predictive validity; that is, there is a relationship between scores on the SAT and an individual's performance in college. The higher the predictive validity, the more useful the test

Note

Accuracy (validity): Validity and accuracy are used synonymously {AV} Precision (reliability): Precision and reliability are used synonymously {PR}

Split-Half Reliability

A measure of consistency where a test is split in two and the scores for each half of the test is compared with one another. If the test is consistent it leads the experimenter to believe that it is most likely measuring the same thing. This is not to be confused with validity where the experimenter is interested if the test measures what it is supposed to measure. A test that is consistent most likely is measuring something; the experimenter just does not know what that "something" is. This is why it is said that reliability sets the ceiling of validity

Reliability

A measure of how dependably an observation is exactly the same when repeated. It refers to the measuring procedure rather than to the attribute being measured. Simply put: Will one get the same values if the measurements are repeated?

Discriminant validity

Disagreement between two scales measuring different constructs Differentiates from unrelated theoretical concepts

Construct Validity

Extent to which performance on a test fits into a theoretical schema about the attribute the test attempts to measure. Construct Validity - the degree to which inferences are warranted from what is represented by the samples External Validity - validity of inferences about whether the causal relationship holds over variations

Concurrent validity

If a similar measure is administered at the same time, the degree of association is termed concurrent validity

Cronbach's alpha

It is generally used as a measure of internal consistency or reliability of a psychometric instrument. Generally, alpha coefficient ranges in value from 0 to 1 Some professionals insist on a reliability score of 0.70 or higher in order to use a psychometric instrument.

Ecological Validity

It is the degree to which the behaviors observed and recordied in a study reflect the behaviors that actually occur in natural settings. In addition, ecological validity is associated with "generalizability". Essentially this is the extent to which findings (from a study) can be generalized (or extended) to the "real world". In virtually all studies there is a trade-off between experimental control and ecological validity. The more control psychologists exert in a study, typically the less ecological validity and thus, the less they may be able to generalize. For example, when we take people out of their natural environment and study them in the lab, we are exerting some control over them and, as a result, possibly limiting how much we can generalize the findings to all people in natural settings

Experimental validity

Refers to sensitivity to change Show difference in results when an intervention is carried out to modify the measured domain.

Construct validity

Refers to the ability of a measurement tool (e.g., a survey, test, etc) to actually measure the psychological concept being studied. In other words, does it properly measure what it's supposed to measure? For example, if we want to know our height we would use a tape measure and not a bathroom scale because all height measurements are expressed in inches and not in pounds

Reliability

Refers to the accuracy, consistency & stability of test scores across situations. 1. Test-Retest: Cronbach's alpha 2. Inter-Rater: Cohen's kappa

Accuracy

Refers to the correctness of the mean value (How close it is to the true population value) It is comparable to Validity

Content validity (also known as logical validity)

Refers to the extent to which a measure represents all facets of a given social construct.

Precision

The degree to which a calculated central value (eg. mean) varies with repeated sampling. Narrower the variation the more precise the value is said to be. It is comparable to Reliability

Accuracy

The degree to which a measurement represents the true value of something. Simply put: How close a measurement is to the true value

Validity

The extent to which a test measures what it is supposed to measure. All tests are designed to measure something; hopefully something specific. If the test does indeed measure what it is intended to measure, then we can say that the test is --------

Validity

The extent to which the study measures what it is intended to measure. Simply put: Are the values describing what was supposed to be measured? Internal validity: Are the results valid for the study subjects? External validity (Generalizability): Are the results valid for the population from which the sample was drawn? Lack of validity is referred to as 'Bias' or 'systematic error'.

Face Validity

This is a very basic form of validity in which you determine if a measure appears (on the face of it) to measure what it is supposed to measure. In other words, does the measure "appear" to measure what it is supposed to measure? For example, if you were going to measure anxiety, does your measure appear to actually measure anxiety? If so, it has face validity. Obviously this is not a test you should use to determine if a measure should be used, but more of a first step in determining validity. is a property of a test intended to measure something. It is the validity of a test at face value.

External Validity

This refers to the extent to which the results of a study can be generalized or extended to others. For example, if a study on a drug is only conducted on white, middle aged, overweight, women with diabetes, can the results of the study be generalized to the rest of the population? Are the results only valid to the population studied? Researchers go to great lengths to select a group of people for the study (a sample) that is representative enough that the results can be extended to lots of people

Incremental validity

What the test adds to the predictive validity already provided by other measures How better than scales already available

Criterion Validity/ Instrumental Validity

reflects the use of a criterion - a well-established measurement procedure - to create a new measurement procedure to measure the construct you are interested in. The criterion and the new measurement procedure must be theoretically related. Uses Concurrent & Predictive validity.