Intro to ABA Quiz 5

Measurement Artifacts

-*Appears to exist because of the way it is measured* -Data is going up and down even if the behavior remains the same -Can be caused by *discontinuous measurement systems* (measuring some, but not all instances of behavior) -Can also be caused by *poorly scheduled measurement periods*

*IOA Data Obtained by Timing* Mean Duration IOA

-Add each separate duration and divide by the number of duration counts and multiply by 100.

*IOA for Interval and Time Sampling Data* Interval by Interval IOA

-Calculated by *dividing the number of intervals agreed by the total number of intervals* (agreed and disagreed) and multiplying by 100.

Considerations with IOA

-Current practices and recommendations suggest that IOA be *obtained for a minimum of 20% of a study's sessions* and preferably between 25%-33%. -IOA should be *collected across all phases* of your study. -Convention in ABA is to expect *no less than 80% agreement* between observers. -Various factors in a situation may make 80% or even 90% too low or too high for acceptable IOA.

Direct measurement

-Direct measurement systems will always yield more valid results -Occurs when the phenomena that is the focus of study is exactly the same as the phenomena being measured

*IOA Data for Event Recording* Mean Count per Interval IOA

-Divide the *total observation period* into a *series of smaller counting times* -Have the observers *record the number of occurrences of the behavior within each interval* -Calculate the agreement between the two observers' counts within each interval -Use the agreements per interval as the basis for calculating IOA for the total observation period

*IOA Data Obtained by Timing* Total Duration IOA

-Divide the shorter of the two durations by the longer duration and multiply by 100.

*IOA Data for Event Recording* Total Count IOA

-Expressed as a *percentage of agreement between the total number of responses recorded by 2 observers* -Calculated by *dividing the smaller of the counts by the larger count and multiplying by 100* -Use caution - a high degree of agreement provides no assurance that the two observers recorded the same instances of behavior Ex. observer 1 counted 14, observer 2 counted 14, so 14/14 x 100 = 100% agreement *Problem: not likely that you're recording the same 14 counts of the behavior

Interobserver Agreement

-IOA is the most commonly used *indicator of measurement accuracy* in behavior analysis -IOA refers to the degree to which 2 or more independent observers report the same observed values after measuring the same events

Valid IOA

-In order to be valid the 2 observers must: --use same observation code and measurement system --observe and measure the same participants and events --observe and record behavior *independent* of any influence from one another (most important)

Threats to Validity

-Indirect measurements -Measuring the wrong dimension of a behavior (such as taking frequency instead of duration) -When your measurement system produces an artifact of the actual event (something that appears to exist only because the way it is measured)

Threats to Accuracy and Reliability

-Number 1 threat is *human error* -Factors that contribute: --*poorly designed measurement systems* (make it easy, clear, not cumbersome, and practical) --*inadequate observer training*. Observers should practice to a criterion before collecting actual data. Feedback should be provided by a trained observer --Observer expectations: avoid observer subjectivity and observer drift

Indirect measurement

-Occurs when what is measured is in some way different from the target behavior of interest -Indirect measurement provides secondhand or "filtered" information that usually includes inferences and subjective accounts of behavior.

*IOA Data for Event Recording* Exact Count per Interval IOA

-Record the *percentage of total intervals* in which the *two observers recorded the same amount*. -Gives you the most conservative measure of IOA.

Reliability

-The extent to which a measurement procedure yields the same value when an event is measured numerous times -The closer the agreement between your data on 2 separate occasions, the greater the reliability -Doesn't ensure accuracy

Accuracy

-The extent to which the *observed value (your data) matches the true state of the event* as it exists. -True = observed score + error score. The true score should be as close to the observed score as possible -*Avoid measurement bias*: non random error in one direction -For some behaviors, true values are obvious (ie measuring correct math problems) -For other behaviors, true values are more difficult to establish because you must rely on what you were able to see and count. Once it is done it is done (ie number of times student called out in class)

*IOA Data for Event Recording* Trial by Trial IOA

-Used for discrete trial data in which the *only possible scores are 1 (correct) and 0 (incorrect)*. -Calculated by *dividing the number of agreements by the total number of trials and multiplying by 100*.

Can you trust your data?

-Validity -Accuracy -Reliability

Purpose of IOA

1) can be used as a basis for determining competence of new observers 2) can detect observer drift 3) can let you know if the target definition is poor (if it is too unclear or ambiguous) 4) increases confidence that variability in the data is not a function of which observer happened to be collecting data in any given session. Changes in data reflect changes in behavior

Validity

Your data should be relevant to the phenomena being measured - are you measuring what you think you are measuring?