NCRA Study Guide CTR 2020 : Data Quality Assurance

Recoding audit

- Recode samples of submitted abstracts using the text provided - Verify coding guidelines and rules are being correctly applied and identify areas for training

Case Incidence Completeness

1 .Case-finding audits 2. Death Certificate Only 3. Historical case review 4. Independent Case Ascertainment

5 Major Cancer Registry Quality Attributes

1. Accuracy 2. Case incidence completeness 3. Data completeness 4. Coding Consistency 5. Publication Consistency

Unknown values

1. Are monitored through process controls 2. Can be legitimately utilized 3. Should have stable rates over time

Methods to assess data accuracy include the following:

1. Computerized data editing 2. Visual review (Visual editing) 3. Recoding audit 4. Reabstracting studies

5 key components of a Metafile

1. Data Dictionary 2. Record Layouts 3. Edit Sets 4. Standard Setting Agencies 5. User Lookup Tables

2 Methods of Electronic Patient (demographic) Matching

1. Deterministic - Records in 2 data files must have exact match on key fields to be same patient 2. Probabilistic - Relies on match score or "weight" created by comparing key fields in patient records. Computerized matching algorithm runs on incoming data file and creates match score for each records

Name four sources of electronic data files

1. Facility electronic health record 2. Pathology laboratory 3. Physician office/clinic including outpatient oncology 4. Central registry

3 Edit Components

1. Metafile - Instructions to run edits 2. Edit set - Groups of individual edits w/specific purpose 3. Edits - Individual data check

Record consolidation Rules

1. Most frequent 2. Hierarchy 3. Highest Value 4. Time submitted 5. Known over Unknown

Data completeness can include the following:

1. Text documentation - Must be dated, complete, and succinct - Support coded data in abstract 2. Unknown values - Monitor use of unknown or ill-defined codes - Data not available in medical record - Don't understand info. where you can code right

3 Key Components of Data Quality

1. Timeliness - Generate monthly reports to monitor adherence to timeliness standard; degree which stages registration process occur schedule 2. Completeness- Measurement of # of cases missed of of missing data 3. Accuracy (Validity)

Acceptance sampling

Acceptance/Rejection of product based on results of inspection. Errors discovered & corrected Cases that fail central registry edits sent back to facility to be corrected

Timeliness can be monitored by

Accrual method- Evaluation of overall accumulation of cases over months/yrs Process monitoring method - Evaluation of length of time bet. critical events in data acquisition

Probabilistic matching

Algorithm creates a match score based on the mathematical likelihood that identical or similar field values indicate a match

Edit Over-ride flags

Allow the registry staff to indicate that questionable data is actually correct.

Independent case ascertainment

Assess case completeness using estimates from an independent survey

Which organizational methodology(ies) involves comparing your registry's practices and performance metrics to other registries in the hope of improving quality locally?

Benchmarking

Quality control of registry data is a direct responsibility of the

Cancer Committee

Central Cancer Registry LNMOP annually performs case-finding audits at 20% of the reporting facilities in the state. The audits are performed to evaluate

Case incidence completeness

Death clearance only is an example of

Case incidence completeness

What type of study evaluates the registry's completeness of case incidence reporting?

Case-finding audit

An example of a single field (range) edit check is

Checking that only a valid code is used in a particular data field

What needs to be provided to data collectors as part of the registry quality control program to ensure that change is affected?

Communication and feedback

Lag-time reports

Compare date of diagnosis to date received in central registry

Inter-field (Multi-field) edit

Compare related data item codes for correctness; contents of more than one item against another; purpose is to identify errors; Ex. Females cant have CaP. Males cant have uterine cancer

Inter-database edit checks

Compares data from abstract w/ other sources in central registry database (e.g. death certificates)

Historical data review

Comparison of # of cases expected based on previous years or on standard incident rates with # of cases observed

Historical data review involves

Comparison of number of cases expected based on previous years with number of cases observed

Visual review (Visual editing)

Comparison of text in abstract to coded data items to identify errors

Data completeness

Comprehensiveness of data set collected, specifications of code values, and avoidance of omissions

The most cost-effective method for assessing quality?

Computerized edit checks

All case abstractors in facilities that submitted cancer data to the state central registry were required to abstract the same 5 cancer cases as part of a reliability study. This is a way to evaluate what aspect of data quality?

Consistency

What process refers to finding records in a data set that refer to the same entity across different data sources, (i.e., identifies and matches patient data)? Occurs before record consolidation.

Data (record) Linkage

What term refers to the set of information that includes the contents, format, and structure of a database and the relationship between its elements, used to control access to and manipulation of the database?

Data dictionary

Central Registry A matched its data with data from the Department of Motor Vehicles in state A to determine date last alive for patients in the database. This process is:

Data linkage

When using __________ , all key fields used to link records in two files must be exact for the program to declare a match. The result of a ____________ in a situation in which one of the key linking fields between two files did not match would be that no match would be identified.

Deterministic linkage

Which matching method follows this rule: "A match cannot be made if one data file is missing a critical data field"?

Deterministic matching

_______________ requires that records must have an exact match on key fields to be considered the same patient

Deterministic matching

What is the name of the Edits feature that allows a user to note that the data are unusual, but correct?

Edit Over-ride flags

_________________ refers to the evaluation of the results of a computer program that compares the coded data with the rules for abstracting and coding.

Edits review

Designed studies

Estimate aspects of current level of quality & don't change level of quality

Case-finding audits

Evaluate a registry's case completeness Involve review from these sources: ◦ Disease indices ◦ Path reports ◦ Autopsies ◦ Radiation logs

Inter-record (Multi-record) edit

Examine data items between related records; check multiple items in different records for internal consistency Compare data on more than one record Compare sequence numbers for a patient. Ex: Sex and Birthplace would expect to be coded the same for all primaries. One primary lists patients alive; second primary lists patient as dead

Edit set

Group of edits used to edit a data file Group of edits required by a specific standard setter *edits may be grouped for a specific purpose

Computerized edits programs were developed to

Identify possible errors that need manual review

One aspect of record consolidation includes

Identifying multiple records for the same primary cancer.

How can a registrar quickly identify whether unknown or ill-defined codes are used too frequently?

Implement process controls

If primary site code entered is C54.9 (corpus uteri) and sex code is 1 (male), what type of edit will be generated?

Inter-field (item) or multi-field (item) edit

Central Cancer Registry AA has a tumor record for John Smith with a lung primary. A second tumor record is submitted to Central Registry AA for John Smith with a colon primary. The records match, but a 1 year difference in date of birth is identified. This was identified by:

Inter-record (Multi-record) edit

______________ examine data items between related records. A patient may have more than one primary, and so on.

Inter-record (Multi-record) edit

An ____________ is designed to compare data recorded across more than one record and are applied across tumor records for a patient that has more than one primary

Inter-record (multi-record) edit

An ________________would catch the mismatch in the patient demographic data between primaries.

Inter-record (multi-record) edit

Timeliness is critical to a registry quality-control process. The methods for evaluating timeliness are

Lag-time reports

Death certificate only

Monitor percentage of cases in central registry database with cancer diagnosis only from death certificate

Quality assurance

Monitoring processes and methods to ensure quality

Record layout

Named group and organization of fields from the data dictionary into a particular file format, such as the NAACCR data exchange record

Using a deterministic linkage protocol that involves the key fields of first name, last name, social security number and date of birth, what would be the match result if the last name was different between two records in different files?

No match would be identified

Total Quality Management (TQM)

Organization-wide effort to continuously improve products and services Ex. Joint Commission on Accreditation of Healthcare Organizations' (JCAHO's) 10-step Quality Improvement Program Components

Quality control

Planned set of activities to monitor quality; insures completeness and accuracy of case-finding, abstracting, coding, follow-up, reporting activities

What matching method is used if a birth date is different by one day in a patient record in an incoming data file, the match to the patient record in the database can still be made?

Probabilistic matching

Record consolidation

Process of combining data from two or more records for the same patient and same tumor

Benchmarking

Process of improving performance by identifying, understanding, and adapting outside practices and processes - Compare processes and performance to best practices Ex. CoC Cancer Program Practice Profile Reports (CP3R)

Data (record) Linkage

Process of matching data from different sources and is based on identical or similar data items from each source

Edit check

Programmed instruction or subroutine designed to check input data by performing a computerized comparison of data fields

Testing and evaluating data or procedures in a registry is an example of

Quality control

Reabstracting study

Re-abstract data from source document and compare to original (Comparison made bet. data originally abstracted & that of re-abstracted case)

This study assesses discrepancies identified by the reabstractor and the quality of the collected data by having a staff member reabstract a sample of abstracts from the medical records.

Reabstracting study

What method of a designed study would be used to monitor data accuracy?

Reabstracting study

A _________________ provides verification that coding guidelines and rules are correct as applied by the data collectors

Recoding audit

A state central registry received multiple abstracts for the same patient with the same tumor. A new abstract was created using the best information from each of the multiple abstracts. What is this process?

Record consolidation

What term refers to the organization of fields from the data dictionary into a particular file format?

Record layout

Deterministic matching (linkage)

Records must have an exact match on key fields to be considered the same patient

Quality control activities in the cancer registry are performed to evaluate:

Registry data quality Quality of registry operations

Mary is the cancer data manager at an integrated network cancer program. She manages 10 abstractors and wants to evaluate the consistency of their abstracting. What type of study should she perform?

Reliability study

This study uses a set of standardized medical charts by a sample of data collectors and reviews the range of answers and checks on consistency of those answers.

Reliability study

What activity measures data consistency?

Reliability study

______________ involves using standardized executable computer edits on registry data to improve data quality and standardize the way data items are checked for validity.

Running edit checks

Cancer registries in North America use which set of rules for determining multiple primaries?

SEER Multiple Primary and Histology Coding Rules implemented January 1, 2007

An edit that checks for valid values within a data item is considered a

Single Field (Item) edit

Reliability study

Standardized source documents abstracted by a sample of data collectors that measure data consistency Results analyzed to identify differences in coding; Education and training targeted to areas in disagreement

A cancer registrar at Hospital A receives a request from another registry for information that is not needed to complete a cancer abstract. Which of the following should the registrar at Hospital A do?

State that the information cannot be shared.

Process control methods

Statistical measures to monitor the state of a process & determine whether it's in/out of control Identify increase in rejections & notify when intervention needs to take place

What is used to resolve incorrect coding or incongruity between codes?

Text documentation

For the central registry record consolidation process, the consolidated medical information on a single primary neoplasm is the:

Tumor record

Single Field (Item) edit

Verify & validate field values for only one data item at a time. Permits only allowable values to be recorded in a field Ex. Race codes must have only allowable values; Birthplace would verify that only valid codes for birthplace are used in the field

Data edits

Verify data accuracy through the use of a computer software algorithm; logical rules that evaluate true/false/maybe for any value of data item

Computerized data editing

Verify data accuracy using software algorithm Ex. NAACCR Volume IV: Standard Data Edits

What term refers to the review process of identifying incomplete or inaccurate codes and information for individual abstracts?

Visual review (Visual editing)

When a data submitter sends a data file to Central Registry XX, registry staff review every 5th record in the file and compare coded data items to text on the abstract. This is:

Visual review (Visual editing)

When the central cancer registry compares coded information to text documentation, it is called

Visual review (Visual editing)

___________ is the process of manually reviewing data fields on an abstract for consistency and logic. During __________, the reviewer compares coded data fields to supporting text documentation to validate the quality of the data.

Visual review (Visual editing)

________________ involves ensuring that the abstracted text provided validate the codes assigned to the abstract.

Visual review (Visual editing)

The purpose of a reabstracting study is to assess the accuracy of registry data against the

medical record

Linking multiple records for the same primary cancer, linking records from other states, and linking pathology reports are all a part of the

record consolidation

Which of the following information should be included in the text, but is not coded as a discrete data element?

route administration

Examples of Process control methods

◦Edit rejection rates ◦Death certificate only (DCO) percentage ◦Historical data review ◦Use of 'unknown' ◦Lag time in reporting

The Cancer registry quality control program does the following:

◦Monitor activities in all areas of the registry ◦Establish uniform standards of quality ◦Provide communication and feedback ◦Collect and maintain data on data quality ◦Balance costs and benefits

Examples of Designed studies

◦Recoding audit ◦Independent case ascertainment ◦Reabstracting study ◦Reliability study

Examples of acceptance sampling are

◦Visual review (visual editing) ◦Computer edit checks ◦Casefinding audits ◦CoC survey review