Research Methods Refresher Notes

What is reflective healthcare?

Action plus thought, observant and aware, knows limits to knowledge, always updating knowledge, confident because knows decisions are informed, can justify what doing, accepts responsibility, versatile and adaptable

Why is action research considered highly practical?

Action research enables clinicians to make improvements in practice at the place where they work.

What does the word "empirical" mean?

Based on real-world observation and experience

What is the difference between basic and applied research?

Basic research investigates fundamental processes that may or may not later lead to practical applications, whereas applied research is about putting knowledge directly to use.

How is quality of research reporting evaluated?

- How well research is reported is different to how well research is done - checklists to evaluate reporting quality - CONSORT statement - clinical trials - STARD statement - diagnostic accuracy studies - PRISMA statement - systematic reviews

Observational studies - researcher only observes and measures, what are the types of studies these could be?

- Longitudinal prospective cohort - Cross-sectional studies - retrospective case control - diagnostic studies - case series - prognostic cohort studies - historical control studies - aetiological studies

Why is identifying causal relationships important for healthcare?

Can help us find what causes healt disorders, determine whether treatments cause improvements, whether there are side effects.

What is Level IV?

Case series with no control group; treatment group only

What does Hume say are a matter of custom?

Causal inferences, the way we think about cause and effect is a habit and custom rather than a logical process.

You are testing the quality of apples in a fruit shop, before you is a display of apples for sale. How might you select a Purposive sample of these apples?

Choose only the apples you want

What is evidence-based recommendation for healthcare practice or policy?

Clinical practice guideline

What is Level III-2?

Comparative study with concurrent control group - control measured concurrently (at the same time) as intervention or cases but without random allocation to groups

What is Level III-3?

Comparative study without a concurrent control group - intervention and control conditions could have occurred at different times

What can we skip in PICO if the terms are not helping?

Comparison and Outcome

What is level III-1 ?

Controlled trials without trully randomised allocation - treatment and control groups but using an approximately random method

What are Hume's three rules for inferring that one event causes another?

1. Contiguity in time and space - the cause and the effect share the same time and place. 2. Causes precede effects - the cause must happen just before its effect. 3. Constant conjunction - the observed relationships between the cause and effect happen consistently over repeated observations.

What are the five evaluation criteria for NHMRC when developing evidence based recommendations or guidelines?

1. Evidence base 2. Consistency of the evidence 3. Clinical importance 4. Generalisability 5. Applicability

What are the grades of recommendations?

A - body of evidence can be trusted to guide practice B - body of evidence can be trusted to guide practice in most situations C - body of evidence provides some support for recommendations but care should be taken in its application D - body of evidence weak; recommendation must be applied with caution

If we decide that a causal relationships exists, what is this called?

A causal inference. An inference is a decision or conclusion that something is true or has happened, based on some evidence or observation.

A randomised clinical trial finds a reliable treatment effect but the treatment effect is larger for people with less severe conditions, so the treatment is less effective for a worse condition. The result is evidence for...

A moderator effect.

You wake up on Monday feeling ill, so you stay home. All Tuesday you're slightly better but not well enough to go out. On Wednesday you're much the same. By Thursday morning you are well enough for work, so you go in. By early afternoon you're feeling so dreadful that you go straight to the doctor, who gives you a prescription that soon makes you feel much better. Without extra information, which of the following is a plausible (reasonable) explanation for your improvement after visiting the doctor?

A placebo effect. Statistic regression to the mean or it could be a genuine treatment effect, the prescription was clinically effective.

Which of the following could be cases in a sample?

Deceased patients. Wheelchairs. Friday nights. All of them

Which of these biases is best addressed by double-blinding in an experimental trial?

Detection bias.

What can target population be?

Disease or treatment group

What is the basis for evidence-based practice and the Western medical tradition over the last several hundred years?

Empirical questions about causes and treatments of disease, answered by real-world observation, by research evidence and not by reasoning alone.

What type of research includes: Looks at risk factors for diseases and health conditions.

Epidemiological studies

What type of research includes: Good for investigating cause and effect.

Experimental design

What type of research includes: Intervention study with at least one treatment and a control group.

Experimental design

Causal inferences can be proved empirically by experiment or repeated observation, true or false?

False, we can be fairly sure but not certain.

What is precision of treatment effect a factor in?

GRADE rating for quality of evidence

What kind of information is Pubmed good for?

Gives access to medline

What is Scopus good for?

Good all-rounder

Why are clinicians encouraged to adopt evidence-based practice?

High-quality evidence can assist clinicians with decisions about patient management.

What can be one advantage of evidence from higher level (i.e., lower number) studies over lower level (higher number) studies in the NHMRC classification?

Higher-level evidence is less likely to be affected by bias.

What is generalisability?

How well people and settings in published research match up with your target population and practice setting Better match between study population and target population - good

What is "consistency of evidence"?

Ideally, results will agree across a range of applicable studies Differing results mean we may be unsure whether results will generalise to clinical settinf or population

Which of these statements about sampling bias is TRUE?

If a sampling method is biased, using the same method to increase the sample size will not reduce the bias.

An implication of J.S. Mill's theory of causation is that...

If the treatment causes the cure, then the cure will happen if and only if the patient receives the treatment.

Why is Research Methods relevant to real-world clinical practice?

If they know how good-quality evidence is found, clinicians are are better able to evaluate the quality of the evidence supporting their practice. Clinicians and researchers ask and want answers to the same types of questions, about treatment effects, diagnostic accuracy, and causes of health conditions. Knowing about research methods helps clinicians to understand reports of the latest findings in their area of interest.

What are the major types of quantitative research?

Intervention studies - researcher sets up what happens to subjects - tries to affect outcomes, this includes experimental designs (randomised controlled trials, pseudo or non-randomised controlled trials sometimes called quasi-experiments, subjects in at least two groups receive treatments, placebo or none, outcomes measured before and after treatments, placebo or none, change in outcome compared for treatments, placebo or none, investigate cause and effect. Observational studies - looks at associations between events or personal characteristics, includes epidemiological studies in general and descriptive studies, useful when intervention study is impractical or unethical, less able to show cause and effect Systematic reviews - studies of studies, combining data

What type of research includes: Researchers test what happens when they deliberately change the way things happen.

Intervention study.

If we agree with David Hume's theory of causality, we shall also agree that...

Just because a treatment benefits all patients in a sample, the treatment's effectiveness with all other patients from the same population has not been logically proven.

What are features of weaker evidence?

Lack of randomisation - Intervention and control groups less likely to be similar Observational - Hard to know or control the influence of other factors linked with events of interest Retrospective - Outcome already known rather than in future - Risk of selection bias Controls not concurrent - Different history - bias No controls at al

What level of evidence is a systematic review of Level 2 evidence?

Level 1 evidence

What level is an intervention study with a control group but without proper randomisation?

Level 3 evidence

What level is an intervention study with a treatment but no control group?

Level 4 evidence

What is the best NHMRC level for single studies of interventions (treatments), diagnostic accuracy, aetiology or prognosis

Level II

What level is a randomised controlled trial?

Level two evidence

What is cochrane good for?

Library of systematic reviews

According to Hume, what are matters of fact?

Matters of fact are about observing and experiencing the physical world. They include statements of cause and effect, and someone does have to do the research and analysing real-world data for that.

What are Mill's two methods of causation?

Method of Agreement - When two or more cases of a given phenomenon have one and only one condition in common then that condition may be regarded as the cause or effect of the phenomenon Method of Difference - If C occurs when observation Z is made, and does not occur when observation Z is not made, then it can be asserted that there is a causal relationship between C and Z

What kind of information is Medline good for?

More for medical

Which of the following is NOT listed as a GRADE criterion for evidence quality when developing clinical practice guidelines?

NHMRC evidence level.

How are relations between ideas proved true or false?

NOT by observation or experiment, but by reasoning and logic.

What are some limits to knowledge?

No one knows everything, there are differences of opinion (even among experts), can't rely on hunches, traditional practice, authority, or biased/ambiguous or false information. Instead aim for reliable knowledge that works.

So do relations between ideas say anything about the cause and effect of the physical world?

No, they are logical truths only, and say nothing about cause and effect in the physical world.

Are relations between ideas empirical?

No, they are only our thoughts, we can only think about them.

Can causal inferences be proved through logical reasoning alone?

No.

What type of research includes: Not so good for demonstrating cause and effect

Observational study

What type of research includes: Researchers monitor or measure events without deliberately changing them.

Observational study

Which of the following is more likely in qualitative than quantitative research?

Patients' individual, personal stories of living with disability.

What does PICO stand for?

Population, Intervention, Comparison, and Outcome

A clinician is looking for a manual task that will estimate future performance on activities of daily living for persons with disabilities. An appropriate task will have high...

Predictive validity.

What type of research includes: Investigates what happens if a condition is left untreated.

Prognostic study

Which of these observational study designs is logically possible?

Prospective, longitudinal cohort study.

What type of research includes: About individual persons' perception of their experiences.

Qualitative research

What type of research includes: Generally non-scientific in its approach.

Qualitative research

What are the major types of qualitative methodologies?

Qualitative research - range of methodologies, investigates human experience, interprets experience in social and political context, acceptance of subjective reality, rejects scientific methods, conclusions specific to time and place: context dependent, it is not scientific research

What do we need to evaluate evidence quality?

Quality of measurement (valid and reliable/unbiased?), research design, analysis and interpretation, need to understand measurement, design and analysis so can be properly evaluated, need to be able to clearly present and explain - quality reporting.

What is evidence base?

Quantity of evidence - more studies, larger samples, the better Quality of evidence - how well the study was done Level of evidence - about the research study desing (the higher the NHMRC level the better) Level of evidence is not identical to evidence quality, high level of evidence is not identical to evidence quality

What is level II of NHMRC?

Randomised controlled trials - treatment and concurrent control groups with subjects randomly assigned

So what is the uncomfortable situation with Hume's fork?

Relations between ideas, which are based on logical reasoning, provable and say nothing about cause and effect in the wider world, and - Matters of fact, which are empirical and do refer to cause and effect but are not necessarily dependable even if they are reliable in practice.

What is "clinical importance"?

Relevance of the evidence to clinical question How long a treatment or policy must be applied to get a beneficial effect How long the benefit lasts Cost-effectiveness of treatment or policy Risks as well as benefit a treatment or policy

What is the opinion that real knowledge is impossible, that we can never be sure of anything called?

Scepticism.

A researcher suspects that the Sesame Street character, Big Bird, in the house has a positive effect on children's reading ability. How might the researcher conduct a case-control study to test this prediction?

See whether children who are good readers are more likely to have Big Bird in the house compared with children who are poor readers.

What are the levels of GRADE recommendation from evidence?

Strong or weak

What's another term for level of evidence?

Study design.

What type of research includes: Combines data from multiple quantitative studies

Systematic Review

What are features of stronger evidence?

Systematic review superior; data combined from carefully selected studies Intervention studies that are experimental rather than passively observational Diagnostic studies using quality reference standard and independent, blinded measures Studies that are prospective and cohort designs rather than retrospective or case-control designs

What is level I of NHMRC?

Systematic reviews - combine results across at least two eligible studies of similar design and topic, unit of analysis is the individual study (study of studies), Level I of studies reviewed are Level II

The PRISMA statement refers to...

Systematic reviews and meta-analyses.

A car-maker deliberately builds a 5 km/h error into its speedometers, to help protect its customers from speeding fines. This type of error is...

Systematic.

If an extraneous factor causes confounding so that experimental results could be invalid, it means that....

The treatment and extraneous factor are related, and the extraneous factor is related to the outcome.

What is the "third variable" problem with testing treatments?

The unknown 3rd factor linked to treatment and outcome affect ing the spurious relationships between treatment and clinical change.

True or false, Mill's theory of causation provides a basis for evidence-based practice via the controlled clinical trial.

True

True or false, all reputable health professions are now evidence based?

True

What is "Hume's Fork"?

Two types of knowledge, each prong refers to a type of knowledge, according to David Hume these are relations between ideas and matters of fact

Does the NHMRC use GRADE system?

Used by NHMRC to evaluate quality of evidence

Summarise Hume's rules for inferring?

We can infer that the first event causes a second event if: - the two events happen in the same time and place, and - the first event happens immediately before the second, and - if the two events always happen together

What is applicability?

Whether evidence base is applicable to Australian health care system - availability of trained staff of equipment, cultural or economic factors, characteristics of work sites

What is a factor affecting GRADE rating for quality of evidence?

Whether future research is likely to change practice recommendations

Can there be a trade-off between generalisability and applicability?

Yes, more specialised study may have greater applicability in relevant setting, but is less generealisable.

Is there a limited ability for causal inferences with non-experimental, observational and qualitative research?

Yes. "Inference" = conclusion or belief based on observation or other available fact, causal inference "one event causes another" = statistical association between treatment and outcome. Non-experimental studies only weak for causal inferences - hard to rule out alternative explanations or other causes, qualitative methodologies less interested in cause and effect

What statement sums up why prospective research is considered scientifically superior to retrospective research?

You don't have to know much about horses to pick the winner after the race is won.

What does empirical mean?

based on experiment, observation and experience, treatments and policy should have prior empirical basis (not just theory, treatment should work but actually doesn't)

What is evidence?

information used to assist decisions (empirical information about the world) just as important to know how to apply evidence is as important as knowing it

What is information?

knowledge, data and facts

What kind of information is CINAHL good for?

nursing and allied health

What do we need to be sure causal relationships are true?

we need good quality and well interpreted empirical evidence.

Can databases have their own search facility?

yes