Quiz 4

QUOROM statement--recommended checklist for assessing quality of systematic reviews

(quality of reporting of meta-analyses)

Meta-analysis is most often used to assess the ____ ____ of healthcare interventions; it does this by combining data from two or more randomised control trials.

clinical effectiveness

The findings from the individual studies must then be aggregated to produce a 'bottom line' on the clinical effectiveness, feasibility, appropriateness and meaningfulness of the intervention or activity. This aggregation of findings is called...

evidence synthesis

Good meta-analyses aim for complete coverage of all relevant studies, look for the presence of _____, and explore the robustness of the main findings using ____ ____.

heterogeneity; sensitivity analysis

There is potential researcher bias when categorical subgroups are not created a priori and are created later on in the review.

limitation of mixed methods

have more power to detect small but clinically significant effects. . Furthermore, they give more precise estimates of the size of any effects uncovered

meta-analyses

now a hallmark of evidence-based medicine

meta-analyses

statistical technique for combining the findings from independent studies.

meta-analysis

used if homogenous quantitative evidence is assessed for clinical effectiveness

meta-analysis

. The type of evidence synthesis is chosen to fit the types(s) of data within the review. For example, if a systematic review inspects qualitative data, then a ____ is conducted

meta-synthesis

Systematic reviews should be based on a peer-reviewed protocol so that they can be ____ if necessary.

replicated

Of the remaining studies, _____ _____ are extracted onto a data extraction form. Some studies will be excluded even at this late stage. A list of included studies is then created.

reported findings

In the past the most common summary measure of effect size was the odds ratio, but now the ___ ____ can be given.

risk ratio (relative risk)

Meta-analysis requires a comprehensive ____ ____ which interrogates several electronic databases

search strategy

Once a quality score has been assigned, the impact of excluding low quality studies can be assessed by....

sensitivity analysis

Not all published systematic reviews have been produced with meticulous care; therefore, the findings may sometimes mislead. Interrogating published reports by asking a _____ __ ______ can uncover deficiencies.

series of questions

rapid evidence assessment reviews, user involvement, mixed methods

some trends in systematic reviewing

The quality of a systematic review can be assessed by using a...

standard checklist.

They preserve the integrity of the findings of different types of studies by using the appropriate type of analysis that is specific to each type of finding. ● The use of categorical codes as a 'halfway' house to mediate between two forms of data was unproblematic

strengths of mixed methods

developing one of these requires the following steps: 1. Defining an appropriate healthcare question. 2. Searching the literature 3. Assessing the studies. 4. Combining the results 5. Placing the findings in context

systematic review

most needed whenever there is a substantive question, several primary studies - perhaps with disparate findings - and substantial uncertainty

systematic review

Conventionally, these are needed to establish clinical and cost effectiveness of an intervention or drug. Increasingly, however, they are required to establish if an intervention or activity is feasible, if it is appropriate (ethically or culturally) or if it relates to evidence of experiences, values, thoughts or beliefs of clients and their relatives.

systematic reviews

attempt to bring the same level of rigour to reviewing research evidence as should be used in producing that research evidence in the first place.

systematic reviews

why are systematic reviews needed?

the hope is that, when taken together within a systematic review, a clearer (and more consistent) picture will emerge.

The validity of the meta-analysis depends on this (on which it is based.)

the quality of the systematic review

T or F: judgement and balance are not obviated by the rigour of systematic reviews

true-- they are just reduced in impact and made more explicit.

Assessment should ideally be conducted by ___ independent reviewers

two

For an _____ assessment, the search for literature must seek to cover all the literature (not just MEDLINE where, for example, typically less than half of all trials will be found), including non-English sources.

unbiased

The definition of this within the systematic review protocol is recommended; thus, what is expected from a user or user group and at which stages of the review should be clearly defined

user involvement

meta-analysis The main requirement for a worthwhile meta-analysis is a...

well-executed systematic review.

● Is the topic well defined Was the search for papers thorough? ● Were the criteria for inclusion of studies clearly described and fairly applied? Was study quality assessed by blinded or independent reviewers? Was missing information sought from the original study investigators? Do the included studies seem to indicate similar effects? ● Were the overall findings assessed for their robustness Was the play of chance assessed Are the recommendations based firmly on the quality of the evidence presented?

what to ask when critically appraising a systematic review

REA takes __ to __ months to complete and 'is a quick overview of existing research on a constrained topic and a synthesis of the evidence provided by these studies to answer the REA question'

2-6 months

There is a bewildering array of scales for assessing the quality of the individual clinical trials.8 Two scales that are commonly used are those developed by...

Chalmers et al and Jadad et al

These seek to: ● Identify all relevant published and unpublished evidence ● Select studies or reports for inclusion ● Assess the quality of each study or report ● Synthesise the findings from individual studies or reports in an unbiased way ● Interpret the findings and present a balanced and impartial summary of the findings with due consideration of any flaws in the evidence.

High quality systematic reviews

can provide quick summaries of what is already known about a topic or intervention; use systematic review methods to search and evaluate the literature, but the comprehensiveness of the search and other review stages may be limited

Rapid evidence assessments (REAs)

These reviews are complicated and depend largely on what clinical trials are available, how they were carried out (the quality of the trials) and the health outcomes that were measured.

SRs

_____ have increasingly replaced traditional narrative reviews and expert commentaries as a way of summarising research evidence.

Systematic reviews

these are also: ● Needed to propose a future research agenda when the way forward may be unclear or existing agendas have failed to address a clinical problem ● Increasingly required by authors who wish to secure substantial grant funding for primary healthcare research ● Increasingly part of student dissertations or postgraduate theses ● Central to the National Institute for Health and Clinical Excellence health technology assessment process for multiple technology appraisals and single technology appraisals

Systematic reviews

Systematic reviewing techniques are in a period of rapid development. Many systematic reviews still look at clinical effectiveness, but methods now exist to enable reviewers to examine issues of...

appropriateness, feasibility and meaningfulness.

● Inappropriate aggregation of studies that differ in terms of intervention used, patients included or types of data can lead to the...

drowning of important effects.

● Each study needs to be assessed for ____ against inclusion criteria and full text papers are retrieved for those that meet the inclusion criteria.

eligibility

'When there is uncertainty about the effectiveness of a policy or service and there has been some previous research ● 'When a decision is required within months and policy makers/researchers want to make decisions based on the best available evidence within that time ● 'When a map of evidence in a topic area is required to determine whether there is any existing evidence and to direct future research needs.'21

examples of when an REA can be undertaken according to REA toolkit

good meta-analyses will use ___ & ___ criteria for inclusion or rejection of studies on quality grounds

explicit and objective

The need for rigour in the production of systematic reviews has led to the development of a ___ ____ ____ for their conduct

formal scientific process

One simple way of assessing the likely presence of publication bias is to examine a this--displays the studies included in the metaanalysis in a plot of effect size against sample size

funnel plot

material that is not formally published, such as institutional or technical reports, working papers, conference proceedings, or other documents not normally subject to editorial control or peer review

grey literature

● Following a full-text selection stage, the remaining studies are assessed for ____ ____ using a critical appraisal framework. Poor quality studies are excluded but are usually discussed in the review report.

methodological quality

Systematic reviews may examine quantitative or qualitative evidence; put simply, when the two or more types of evidence are examined within one review it is called a ____ systematic review.

mixed-method

used if quantitative data are not homogenous.

narrative summaries

Are the findings from systematic reviews are always in harmony with the findings from large-scale high quality single trials?

no

The precision with which the size of any effect can be estimated depends to a large extent on the ____ of ___ studied.

number of patients

. This will address issues such as the quality and heterogeneity of the included studies, the likely impact of bias, as well as the chance and the applicability of the findings

placing the findings in context step of systematic review process

Meta-analysis of trials provides a ___ ___ of treatment effect, giving due weight to the size of the different studies included.

precise estimate

the way existing studies are found; how the relevant studies are judged in terms of their usefulness in answering the review question; how the results of the separate studies are brought together to give an overall measure of effectiveness (benefits and harms) - statistical techniques used to combine the results are called meta-analysis.

protocol