Research Methods Learning Outcomes
Detail the advantages and disadvantages associated with using questionnaires in research
+ Good for collecting nominal and ordinal data with closed questions such as demographics + easy to complete + suitable for topics with fixed responses + suitable for gathering same info from all respondents + can gather superficial info economically from large number + anonymity easily guaranteed + data from closed questions easy to analyse + can be designed specifically for group/topic - Time consuming to design - often low response rates - participant responses largely constrained to fixed responses - do not allow capacity for qualification - researcher cannot explore topic in depth - little guarantee as to where, when, by whom and in what order questionnaire is completed - data may be skewed by self-selecting bias, those that return questionnaire may have agenda - can be difficult to analyse missing data/ambiguous responses - people who cannot read/write may be excluded Researcher bias - selection bias - wording bias Participant bias - social desirability response - hawthorne effect - recall bias, from asking participants to think back to experience, retrospective design
Discuss the role of the research question within qualitative research
- Give a focus and purpose to the research - Prevent vagueness - Provide something that can be answered - Keep a researcher 'on track'
Describe the difference between research aims and objectives
Aim An aim is the overall intention or goal for the research It is a broad statement about the main associations that one hopes to establish Objectives The objectives are the means by which the aim is achieved They are highly focused and specific statements - each one referring to just one specific aspect of the study Most often starting with a verb eg to measure, to ascertain, to identify There will be more objectives than aims
Demonstrate an understanding of the use of outcome measures in research
Are objective tests, they are the instruments used to gather data They are often used before and after an intervention, as well as at time points in between or longitudinally for follow up Used before an intervention they can provide baseline scores then change can be assessed whenever measure is repeated By using outcome measures close together before and after baseline can give an indication of stability of signs and symptoms over the short term and therefore whether participants were stable or changing anyway irrespective of intervention
Outline the four key principles of research ethics and discuss how they relate to different stages of conducting research
Autonomy Independent free choice Relates to consent at the beginning of the trial and the right to withdraw at any point during the trial Beneficence Respect for others dignity, rights, safety and well-being Relates to doing the best for patients care so affects ability to use a control group in some studies, may not be any overt benefit to being a participant, this is acceptable if participant is aware, acceptable as the research may help others in the future Non-maleficence Above all, do no harm Means taking measures to reduce risk of harm/inconvenience throughout study Large part of what ethics committee will look at Justice Fair to all Relates to giving all equal access and not discriminating Applicable to recruitment phase and throughout process
Discuss the process of consent within research
Best practice is to acquire written informed consent This is to protect both the researcher and participant legally It also ensures that beneficence and autonomy are upheld in showing respect to the participant by fully informing them as to what is involved and giving them the freedom of choice to participate or not It is best done by talking to the participant face to face (if the research involve face to face contact) A grace period should be given between the participants agreeing to participate and the consent form being signed - giving them time to change their mind Participants should be aware that by returning questionnaire provides implied consent
Demonstrate an understanding of why participants might not be randomised but might be allocated into groups
By randomising you may end up by chance with a group that is not representative of the population you are sampling By allocating into groups or stratifying you can ensure equal numbers of subjects with specific characteristics end up in each group, which makes each group representative and also more comparable to each other so reduces the involvement of confounding variables due to unmatched groups
Discuss how the findings from data analysis can be used to prove/disprove a research hypothesis
Calculation of p value if less than 0.05 then can reject null hypothesis and accept research hypothesis
Identify when to use the following statistical tests: a. Chi-squared test b. Wilcoxon test c. Mann-Whitney test d. T-test (one and two-tailed) e. Pearson correlation test f. Spearman test
Chi squared - nominal data - different subject design - 2 different groups Wilcoxon - ordinal/interval/ratio - same subject design - 2 conditions Mann-Whitney - Parametric tests - Student/related t test - Anoya Correlational tests - spearman - non para - pearson - para
Discuss the strengths and limitations of open and closed questions within healthcare research
Closed questions: + easy and quick to answer = increase response rate + contribute to quantitative data + good for gathering factual info eg demographic etc + useful when group sampled do not have much time + useful when group is only likely to share limited info + useful when group do not have high skills of literacy - Do not allow for any wider info to be gathered = limited depth Open questions: + allows greater depth/flexibility of info to be gathered + contributes to qualitative data + useful gathering opinions, attitudes, perceptions + useful when sample group may have a lot to say on subject and have depth to share - offputting and/or only superficial info given - reduce response rates if many open questions
Discuss what is meant by the term 'blinded' within research
Concealment to treatment grp allocation Used to reduce biases that can be introduced from awareness to which intervention has been received by a specific participant
Demonstrate an understanding of how questionnaires are developed, distributed and analysed
DEVELOPMENT Need to be carefully designed - clear questions - appealing layout - sequence easy to follow - interactive style as if talking Stages of developing questionnaire First stage - define target population and research question/objectives Second stage - for each objective list all associated questions Third stage - Consider all info that is required for each question Fourth stage - develop words for each question to obtain info Sampling - random - stratified - convenience DISTRIBUTION post + access to large sample - response rates notoriously low email + cheaper than post + convenient to click link and fill in - people worry about anonymity online - only available to those with internet captive audience + high response rates open audience + allows a wider audience to be approached + face to face questioning with researcher while completing, good for low literacy +/- response rates between postal/captive Need a clear intro invite letter to accompany questionnaire ANALYSIS No manipulation of variables therefore CANNOT demonstrate cause and effect May show associations between different characteristics or phenomena Closed questions lend themselves to descriptive and possibly some inferential stats (looking for relationships between factors reported) Open questions lend themselves to thematic analysis - looking for codes and themes within responses PROMS - patient reported outcome measures - can compare to normative data PREMS - patient reported experience measures - collects attitudes from experiences
Demonstrate the ability to write appropriate documentation
Dear sir/madame, I am a 2nd year physiotherapy student from Hertfordshire University. As part of my studies I am undertaking a research project entitled '...' to increase our understanding of ... As someone with X this email is to invite you to take part If you did wish to take part participation would involve... Please refer to the attached participant information sheet for further details Participation is entirely voluntary The study has been approved by ethics committee If you wish to take part please contact... Thankyou for taking the time to read and consider participating in my research
Discuss the significance of different patterns of distribution curves
Depending on pattern of distribution the type of analysis is different Normal distribution needs to be analysed by parametric tests and shows equal spread around the central tendency Skewed distribution needs to be analysed by non-parametric and shows uneven spread
Discuss the place of both descriptive and inferential statistics in quantitative research
Descriptive statistics give a brief and clear overview of your results, tells you the type of distribution and describes the spread Inferential statistics use more in depth methods to prove or disprove significance in the results
Discuss the difference between experimental and non-experimental/observational research designs
Experimental design - Controlling a set of conditions and modifying some variables (independent variables) to test their outcome on others (dependent variables) - Need to be carried out in a controlled manner and involves some type of intervention (direct action) by the researcher - Often designed to assess for a cause-effect relationship - Experimental studies often designed to introduce the cause (intervention) and see what effect it has (outcome) (eg introducing a stopping smoking intervention to see if it affected rates of lung cancer) - Examples: randomised control trials, non-randomised trials, quasi-experimental trials - Aim to control as many different facets of the research as possible so that any outcomes from an intervention can be genuinely linked to it alone and are not just due to chance - If done well experimental studies are regarded highly because of their rigour and reliability - They can answer the question of cause and effect - Manipulation of intervention must precede the change in outcome variable - Changes to outcome variable must be explained by effect of the intervention and not any other factor - Changes in outcome variable are generally ones of magnitude - quantitative/number data is therefore gathered Observational design - Looks at a set of conditions without modifying or controlling them - uses questionnaires, sampling, surveys - designed to look at a phenomena (effect) and then try to determine what caused it - eg look at rates of lung cancer to see if there are any factors (such as smoking) that affected it - Examples: cohort studies, case control studies, case studies
Describe what is meant by the terms experimental and control group and how the latter may be generated
Experimental group The group that will receive the intervention that is being tested Control group The group that does not receive treatment or receives sham or 'standard' treatment used to compare to the outcome of the experimental group to prove cause and effect Control group generation Same as experiment group randomisation etc need to consider ethical issues to get around this can recruit from those on a waiting list for treatment or offer standard treatment
Generate a research question, aims/objectives and hypothesis/null-hypothesis appropriate to quantitative research
For Kyrdalen et al: Research question For fall-prone older people what is the effect of home training versus group training of the Otago exercise program on reducing risk of falls over 12 weeks and 3 months? Aim To compare the Otago Exercise Program (OEP), originally designed as supervised home training (HT), with the same programme performed as GT, on functional balance and muscle strength, mobility, fall efficacy and self-reported health OR To compare the effect of the OEP run for 12 weeks as supervised GT with HT in home-dwelling older people at high risk of falling Additional aim To assess sustainability of effect of intervention 3 months after cessation of training Objectives To recruit participants that meet the inclusion criteria To use BBS to assess functional balance To use STS to assess functional muscle strength To use etc etc Hypothesis GT will improve functional balance more than the same exercises performed as HT Null hypothesis There will be no difference in the improvement on functional balance of exercises performed as GT versus HT
Discuss the role of the both the hypothesis and null-hypothesis within healthcare research
Hypothesis A research hypothesis H1 proposes a possible explanation for certain behaviours, phenomena or events - it predicts that there is a relationship between the different variables eg A causes B, A is larger than B It is written as a testable, predictive statement and data are collected to explore this testable hypothesis A good hypothesis states clearly the expected relationship (or difference) between variables or factors It should be formulated after reviewing related literature in the topic and not before It should be testable by gathering or examining data and it should be formulated before starting data collection Because it is formulated before starting data collection it is always written in the future tense It is purely quantitative concept and at end of study it should be possible to accept or reject it Null hypothesis Within quantitative research considered best practice to set a null hypothesis H0 alongside hypothesis Null hypothesis predicts absence of a relationship between variables considered If this were the case suggests that any changes were due to chance and not a true/consistent relationship Hypothesis of no difference Note the null hypothesis does not predict that there will be a relationship in the opposite direction to that proposed in hypothesis - it suggests NO RELATIONSHIP
Discuss the role of inclusion and exclusion criteria within research
Inclusion criteria A list of statements that each potential subject must meet to be included Ensures that the sample is homogenous enough to be comparable and is specific to the population you are sampling from Exclusion criteria Additional statements that define conditions that mean the subject cannot take part in the study even if they meet the inclusion criteria Protects participants from entering the study if they have a characteristic which may make participating dangerous for them (contraindications and precautions) Or have a characteristic which may not be appropriate for comparison as could introduce a confounding variable
Discuss the key features that would be included within a letter of invitation and a participant information sheet
Letter of invitation: - Polite address - Introduce self, profession, place of work - State title of study and why you are doing it - State nature of wider population for recruitment and why being actively invited - State nature of the research to provide scientific rationale - State briefly what involvement would be - Invite to read Participant Information Sheet PIS for further info - Provide contact details so can contact if interested - Thank participants for taking time to read PIS: Question answer format Provides logical structure and allows to talk personally Honest - esp benefits if any and risks/inconveniences What is purpose of study? - what participant will have to do Do I have to take part? - voluntary - any benefits Any disadvantages/risks in participating? - honest, if any should include details of strategies to reduce risk Change mind? - free to withdraw at any time Who has reviewed study? - state approval given by ethics committee Will my info be kept confidential? - Make reference to method of safe storage Further questions?
Demonstrate an understanding of different research designs in quantitative research
Meta-analysis - Statistical approach to combine the results from multiple studies in an effort to increase power (over individual studies), improve estimates of the size of the effect and/or to resolve uncertainty when reports disagree Systematic review - Type of literature review that collects and analyses multiple research papers - Follows a clear systematic methodology carefully documented so repeatable by someone else - Aim is to find all available evidence on a topic - Presented in normal structure for research: background, research question, methods, findings, discussion, conclusion - Method section should be clear evidence of how literature was selected and how it was appraised - Strict guidelines to ensure systematic - Clear methodology to answer predefined question - Searches all literature following predefined strategy until all sources exhausted - Inclusion and exclusion criteria to assess which papers need to be reviewed - Critiques all relevant papers against predefined criteria to assess their quality - Combines findings and presents them in systematic way Randomised Control Trial (RCT) - Research methodology used to directly compare two or more conditions and assess for any differences - Often used to establish if one condition is more effective than another - Participants randomly allocated to different treatment arms, allowing results to be compared for any differences in outcomes between groups - Must be ethically appropriate to randomly allocate to grps esp if using a control group - Participants are randomly allocated to one of the study conditions (arms) and both they and the researchers are ideally unaware of which arm they are in - Characteristics of the groups are matched/similar at baseline Non-randomsied control trial - Also an experiment in which people receive different interventions such as treatment, test or training - However the allocation into groups is not random - researcher may choose which group a person should be put in based on: perceived need, their own personal preference, personal details, geographical location
Describe the differences between nominal, ordinal, interval and ratio data
Nominal - discrete - categorical - each category mutually exclusive of other - lowest form of data as gives least info - mode can be calculated bt not mean - use pie charts or bar charts Ordinal - discrete - rank order (arbitrary) - arbitrary zero - median and mean can be calculated - use bar chart Interval - continuous - measured with equal intervals between graduations - arbitrary zero - can calculate mean - histograms, box plots, line chart Ratio - continuous - measured with equal intervals - absolute zero - can calculate mean - histograms, box plots and line charts
Explain the differences between parametric and non-parametric data and discuss how to determine which statistical tests are appropriate
Parametric data follows a normal distribution therefore if your data displays as a bell curve you should use a parametric test such as Anova or related t test Non parametric data follows a scewed distribution therefore if your data has a tail and does not display as a standard bell curve you should use a non-parametric test such as wilcoxon, chi squared, mann whitney u If looking at correlation use spearman for non parametric or pearson for parametric
Demonstrate an understanding of different sampling, invitation and recruitment methods employed within quantitative research designs
Probability sampling any sampling method that utilises random selection means each unit within population has an equal chance of being selected Random sampling all those who might be suitable have an equal chance of being selected + gold standard to reduce bias - but because random group might not be representative, esp if group is small Stratified sampling breaking down sample into subgroups eg by age, gender and then recruiting by convenience or random so that equal numbers of people are recruited from each subgroup + if done right gives best chance of a representative sample - relies on researcher knowing their population well to know which subgroups to stratify Cluster sampling dividing population into groups or clusters (patients attending certain health centres) and recruiting all people from clusters, again either by convenience or randomly + convenient - hard to follow up if spread out - need to make sure clusters are from very different areas/backgrounds or risk getting a very specific sample Convenience sampling recruiting whoever is most easily available + relatively easy way to recruit large group of participants - May have limited representation to wider population Invitation methods Posters Adverts Social media Direct letters Direct phoning
Demonstrate an understanding of the difference between quantitative, qualitative and mixed-methods research
Quantitative research Tests a hypothesis or hypotheses by seeking to establish relationships between the quantifiable variables being studied Relates to testing of ideas and hypotheses using numerical measures - Uses methods of data collection that involve measuring: size, amount, scales, frequency - Aims to be objective - Data analysed using statistical tests and results presented using numbers - The studies tend to be large and involve many participants so that the findings can be applied in other context = generalisability Qualitative research Involves the generation of new ideas and theories based on non-numerical data, without preconceptions - Focus on depth rather than breadth - Researchers seek to understand the whole of an experience and gain insight into a participant's situation - The data collected is not numerical but is collected, often through interview, using words and descriptions given by participants - Researchers do not set out looking for specific ideas, hoping to confirm pre-existing beliefs; instead they code the data according to ideas arising from within it - this process is often referred to as inductive - No statistics; results are descriptive and interpretive - Sample sizes tend to be small - required because in-depth understanding is sought from information-rich participants who take part (sampling tends to be purposive and not random) - Not directly generalizable in statistical sense but results can be used and interpreted by others - transferability Mixed methods research Research where quantitative and qualitative research is intricately linked - Involves a mixture of both quantitative and qualitative elements
Demonstrate an understanding of the use of surveys and questionnaires within healthcare research
Questionnaire - list of questions respondent answers alone Used for observational research - there are no variables and nothing is being manipulated Usually quantitative (closed questions) but can be mixed (closed and open) Used to gather data on: demographics attitudes beliefs behaviours experiences health status/outcomes knowledge Why use questionnaire to hear participants voice - data from large number of people required - data from geographically widely distributed groups of people required - nature of investigation is potentially sensitive and respondent may be happier to share thoughts anonymously (sometimes not always case some sensitive topics may be better explored face to face)
Discuss the key strengths and limitations of the randomised controlled trial
RCT strengths + Can compare two conditions and assess for differences + Most robust/rigorous methodology to reduce/eliminate bias RCT weaknesses - The rigour/inflexibility of the methodology leads some to question how 'real life' the research is and how applicable the findings are to practice
Discuss what is meant by the term randomisation and explain the strengths and weaknesses of some of the methods that might be used to achieve this
Randomisation Randomly assigning participants to an arm of the study therefore giving each an equal chance to be in either Reduces likelihood of bias and therefore any difference in outcome is more likely to be due to the intervention than due to chance Simple + - Block + - Stratified + -
Discuss some of the key psychometric properties that we would look for in an outcome measure, including reliability, validity and sensitivity
Reliability: Ability of tester and tool to measure same each time 2 types: Inter-rater Intra-rater Intra - ability of individual to measure same construct each time Inter - ability of different people to measure same construct each time To enhance reliability: Outcome measures must have high stability and consistency Standardise procedure - protocols/standard operating procedures
Discuss how research differs from audit and service evaluation and why a knowledge of research methods is essential for a physiotherapist
Research: The attempt to derive generalizable new knowledge including studies that aim to generate hypotheses and studies that aim to test them Audit: An inspection against a set standard to see if standard has been met Service evaluation: An arbitrary evaluation of current service quality (no set standard) Research is designed to test a hypothesis (quantitative) or identify/explore themes following established methodology (qualitative), audit is designed to answer the question 'does this service reach a predetermined standard?' and service evaluation is designed to answer the question 'what standard does this service achieve?' Research addresses clearly defined questions, aims and objectives, audit measures against a standard and service evaluation measures current service without reference to a standard Research may involve evaluating or comparing interventions, particularly new ones (quantitative) or involves studying how interventions and relationships are experienced, whereas both audit and service evaluation involve interventions already in use only and the choice of treatment is that of clinician and patient according to guidance, professional standards and patient preference Research involves collecting data that are in addition to those for routine care but may include data collected routinely; may involve treatments, samples or investigations additional to routine care Both audit and service evaluation involves analysis of existing data, but may include administration of simple interview or questionnaire Research study design may involve allocating patients to intervention groups (quantitative) or uses a clearly defined sampling framework underpinned by conceptual or theoretical justification (qualitative) Audit and service evaluation both do not involve allocation to intervention groups, as the healthcare professional and patient have already chosen the intervention before the audit/evaluation Research may involve randomisation whereas audit and service evaluation do not Research usually requires REC review whereas audit and service evaluation do not A physiotherapist is required to stay up to date with current research by the HCPC and has to use evidence-based practice. This means they must be able to evaluate research to use in their own practice and provide the best standard of care for patients. A good knowledge and understanding of research methods is required to do this. They may also have to undertake audits and service evaluations. To enhance one's own practice as an autonomous professional by being able to understand and criticise published articles in journals and then apply the findings Application of evidence informed practice Justification of choices made within practice based on evidence
Discuss the strengths and limitations of researcher generated and patient reported outcome measures
Researcher: + objective, reduces bias - relies on researcher being consistent in use Patient: + able to gather depth of knowledge relating to feelings, reasons etc - subjective, open to bias
Outline the different sections that one would expect to find within both a quantitative and qualitative research article
See workbook
Discuss why it is important for practicing clinicians to be able to critically review research articles
So that we can discern research that is robust so that we can have confidence in safely applying it in a clinical setting for real patients
Briefly outline the process involved in gaining ethical approval, both at the University and using the NHS Integrated Research Application System (IRAS)
Takes a long time Must be approved by committee Sometimes researcher must go to present study and answer questions Helsinki declaration used as a cornerstone
Describe evidence-based practice and its place in health care
The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients Integrating individual clinical experience with the best available external evidence from systematic reviews Used in healthcare to justify choices made by healthcare professionals in providing the best possible care to patients
Demonstrate an understanding of the different structures that could be used to guide the development of a research question
Within quantitative research, the nature of the research question that is set will depend, in part, on whether more or less is already known about the topic - Often within quantitative research, the research question is written as an interrogative question e.g. What/which/does...? - However, in some quantitative studies (and all qualitative studies), the research question may be declarative and written as a statement e.g. An investigation into... • F - Feasible (answerable and realistic) • I - Interesting (clear and unambiguous) • N - Novel • E - Ethical • R - Relevant • P - Patient or population • I - Intervention • C - Comparison intervention • O - Outcome • T - Time
Demonstrate an understanding of statistical significance and probability (p-values)
p value must be less than 0.05 to be significant Relates to 5% or 1 in 20 chance that the effect was due to chance rather than the intervention
