Survey questionnaires (research methods)

Points to consider to assure the questionnaire is clear and unambiguous?

- how many responses the respondent is able to give -do you want them to use a tick, cross etc - if there are sections only relevant to some, is it clear how to navigate through the questions with more surveys now being done online the design of the questionnaire / software should assist with ease of navigation

Why is consent from participants slightly different for questionnaires than face-to-face studies?

- in a face to face study, a particpant can withdraw at any stages and their information given up to that stage not used - for an anonymous questionnaire, once the participant has submitted it, they cannot withdraw their data/info as it would be impossible to identify which was theirs

What are the disadvantages of a questionnaire

- participants responses are largely constrained to fixed responses - which although may be appropriate do not allow capacity for qualification -the researcher cannot explore topics in depth -response rates are often low -there is little guarantee as to where, when , by whom and in what order the questionnaire is completed -data may be skewed by a self-selecting bias - those that return their questionnaire may have an agenda - it can be difficult to analyse missing data / ambiguous responses -people who cannot read and write may well be excluded

Importance of the reliability within questionnaires

- questionnaires need to be constructed in such a way that there is no ambiguity as to what is being asked - consistency of responses between individuals - enhanced reliability

having choosing to use a questionnaire, what must the researcher then determine?

- the nature/size of their sample - the nature of the questionnaire/questions involved - how the sample will be recruited - how the questionnaire will be distributed to this group

the decision on what type of approach will depend on:

- the purpose of the study, research question - philosophical prospective - Extent of prior knowledge

Why/when is a questionnaire a good method to hear the participants voice?

-Data from a large number of people is required -Data from a geographically widely distributed groups of people is required - The nature of the investigation is potentially sensitive the respondent may be happier to share their thoughts in an anonymous way - although this is not always the case and sometimes very sensitive topics are best explore in a face to face interview

Within healthcare surveys and questionnaires are used most often to gather data on:

-demographics -Attitudes -Beliefs -Behaviours -experiences -health status / outcomes -knowledge

What are the advantages of a questionnaire ?

-easy to complete -suitable for topics where there can be fairly fixed responses -suitable for gathering much the same information from all respondents -can gather superficial information economically from a large umber of individuals - anonymity is easily guaranteed -data from closed questions are relatively easy to analyse -they can be designed specifically for the group/topic in mind

when designing a questionnaire it is very important that

-questions are written clearly and easy to follow - has an appealing layout to the respondent - the sequence of questions is easy to follow Normally questionnaires are written in an interactive style, as if someone is talking to the respondent

Define a questionnaire

A written list of questions that the respondent will normally answer alone / without the involvement of the researcher requires the respondent to be able to: - read the questions - interpret them - write down / indicate some sort of response Because the researcher is not involved in the process of completion, the questions need to be understandable without any further explanation.

What type of information are closed questions appropriate for gathering?

Are useful for gathering factual information e.g. demo-graphical information, they are useful when the group being sampled do not have much time, are probably only likely to share limited info. do not have high skills of literacy.

What type of information are open questions appropriate for gathering?

Are useful for gathering opinions, attitudes, perceptions; they are useful when it is anticipated that the sample group might have a lot to say on a subject and have depth that they could share. As well as asking well structured and clear questions, it is important that the guidance for how to complete the questionnaire is also clear and unambiguous.

When determining the possible sample size what is considered?

As with other forms of research design, this needs to be large enough to appropriately capture the depth views of a but not be so large to make data handling impossible For questionnaires with more quantitative/closed questions, the sample size can/should often be larger For questionnaires with more qualitative/open questions, it is often appropriate for the sample size to be smaller as the depth of information gathered will be more extensive and one may not be looking to generalise findings in the same way to the wider population

How do you gain consent via questionnaires

Because questionnaires are often anonymous, explicit signed consent is not appropriate. Instead participants need to be made aware that consent is implied by return of their questionnaire itself

How can bias be reduced within a questionnaire?

Bias, however, can be reduced by using a questionnaire compared to other forms of data collection such as focus groups/interviews Because a questionnaire is most often completed without the researcher being present, they are less able to exert undue influence on how respondent choose to answer the questions or lead respondents in any way

Distributing a questionnaire to a captive audience.. .

Distributing a questionanire to a captive audience (e.g. all those attending a lecture/conference) can be a successful way to distribute a questionnaire It is extremely important that the group are informed that completion is still voluntary and they do not feel coerced to complete it to potentially enhance one's response rate, it may be that there is opportunity for a verbal introduction to the questionnaire / body of work Generally from a captive audience , response rates are high

What can a questionnaire be made up of?

Either all closed questions, all open questions or a mix of both

What four stages can help develop a more robust research tool (questionnaire) ?

First stage: define target population and the research question / objectives Second stage: for each objectives / hypothesis list all the associated questions that one ideally wants answers about Third stage : consider all the information that might be required for each of these associated questions Fourth stage : Develop the words for each questionnaire question to obtain the information required

Outline a range of the key features that would be considered as best practise when constructing questions for a questionnaire

Important that there is no ambiguity and the questions are easy to understand, because the interviewer is not there. Got to be easy to understand so that it does not get mis-interrupted. Short and straight to the point - the right length. Do not have a double negative in a question - otherwise it makes is complicated for it to understand. (like the people that try to send your junk mail). Avoided leading questions that produce bias. Questions must be relevant to the study. Don't repeat questions. Don't want the questionnaire to be to long, will increase the % of comeback. Have all the different elements that you want to have covered. If it's paper based questionnaire chose the colour yellow.. So it doesn't get lost and it makes it stand out. Also consider layout.

It is important to have a clear letter of introduction/invitation to accompany the questionnaire. This needs to ...

Introduce the work and you as the researcher, set the scene as to why the work is being done, explain what involvement would include, explain about consent and that participation is voluntary, provide contact details for questions and return of the questionnaire, finish by thanking them

What is a pilot questionnaire and what is it used for?

It is best practise to pilot a questionnaire (test run) at least once to assess how people interpret the questions and what type of responses are given. - a first pilot may be to friends / colleagues to gather general comments about wording, layout, structure/flow, missing questions etc. - a second pilot may be on a sample of your final population group where the respondents may have greater expertise in relation to the questions being asked

What is respondent fatigue? And why is it important to be accounted for?

It is when you have provided a neutral category, or a mid number in numerical scales, and the respondents select this without giving much thought to their answers. However, if you take out this neutral box, you are removing the option of the respondent not to have an opinion either way. If you phrase all your questions in either a more positive or negative way, again respondents may stop being selective and just indicate a standard / set response

Tips for writing questionnaire questions ..

Keep language and sentence structure simple Avoid using technical words/jargon Keep the meaning of each question simple and avoid ambiguity Do not ask about two points within one question Do not use a double negative within a question Do not ask leading questions Do not ask questions based on presumptions If asking a sensitive question, it often helps to give some background/rationale to this before the question is asked

Compare and contrast the strengths and limitations of open and closed questions within a questionnaire design

Open - strengths = honest opinions/depth of opinion, allows you to listen to their voice, their choice of words - not shaping it in anyway or constraining it in any way, their thoughts and perceptions, depth of answers, Weakness = more subjective way of interpreting it and anylsinsg, people might not be bothered, more time consuming, people who are likely to comment are people that aren't happy, they want to share their moans, here voice more of people who have an axe to grind. 30% might be only those who are unhappy. Closed - weakness = constraining the option that you're giving the participants, might not be able to give the answer that they want. Strengths = quick to answer, you get what you want, increases response rate, don't have to think outside the box and spend to long on it, easier to analyse data, very objective way of analysing it, can send it through statistical packages, If this was an actual short answer question in the exam for 4marks, structure and style = need to see knowledge and understanding. Discursive answer. Something that discussing the compare and contrast our understanding of those strengths and limitations. Need to show the discussion.

Analysis of questionnaire data for open questions ...

Open questions lend themselves to thematic analysis - looking for codes and themes within responses

What is participant bias?

Participants may not actively set out to deceive the researcher in the responses that they give but it is not uncommon for participants to give responses either that they think the researchers want to hear or to paint themselves in the best light (social desirability response) Many questionnaires ask people to think back to experiences - they are retrospective in their design => recall bias

What are PREMS

Patient reported experience measures - A form of questionnaire/survey that we often use is to gather people's attitudes and/or experiences towards a certain topic e.g. A patient satisfaction questionnaire As with other questionnaires, this type of questionnaire can contain both closed and open questions Again, closed questions are easier and quicker to answer but open questions are useful to gather opinions, esp. if you are wanting to develop and improve a service based on responses

what does PROMS mean?

Patient reported outcome measures. This is a way in which questionnaires are used frequently in health care Many standardized tools are available for use - some of which are generic and some are condition / population group specific if using an stabilized measure it is important to check copyright and licensing If using a standardized measure, normative data is often available that one can compares ones own data to

What are the 4 different sampling methods?

Random sampling Stratified sampling Convenience sampling Snowball sampling As with sampling for other types of research, the final sample ideally needs to be homogeneous (common property throughout) enough to draw conclusions but heterogeneous (foreign origin) enough to allow the results to be generalised to others

What are the strengths and weaknesses of open questions

S = allow a greater depth / flexibility of information to be sought W = often off putting and / or only very superficial information given - may reduce response rates if too many open questions as it is time consuming and effortful

What are the strengths and weaknesses of closed questions

S = easy and quick to answer => potentially increasing response rate W = does not allow for any wider information to be sought / gathered => very limited depth / variety

What are the two types of researcher bias?

Selection bias - this may be intentional with the researcher only distributing their questionnaire to people from whom they expect a certain response or unintentional with only a certain group receiving the questionnaire or taking the time to complete/return it It is always worth considering that non responders may have very different views from responders Wording bias of the questions - leading questions can result in skewed data being collected

What determines the quality of the data collated in a questionnaire?

The data is only as good as the questions that are posed in the first place.

What is criterion validity?

The extent to which the measure corresponds / compares to other similar measures on the topic

what is content validity?

The extent to which the measure corresponds / compares to other similar measures on the topic

What is face validity?

The extent to which the questions look as though they are gathering what they are meant to gathered on the topic - do the questions looks credible

How to structure responses for closed questions?

The most common way that respondents can indicate responses are through nominal categories, through ordinal scales, by selecting statements or by ranking information - all of these will offer pre-determined categories of response

What will determine the type of analysis used for a questionnaire

The research objectives, sampling method, data collection methods and nature of data Since there is NO manipulation of variables - the results cannot demonstrate cause and effect The study may however, entail investigations of associations between different characteristics or phenomena

What is questionnaire validity?

The research tool (questionnaire) is doing / gathering what it supposed to do/gather. Questions must be selected and worded base on a sound theoretical underpinning of existing evidence and not just on what the researcher thinks is relevant

What variables does a questionnaire contain

There are no variables and nothing is manipulated. Questionnaires are a form of descriptive / observational research

What are nominal categories useful for in terms of questionnaire responses?

They are useful for gathering demographic information and/or information where a response can only fit into one category. E.g. income bracket

What type of questions is a questionnaire made up of

They are usually quantitative but can be mix of quantitative - closed / structured questions - and qualitative data - generally more open questions.

Distributing a questionnaire by email ..

This method for many studies has either entirely or partly replaced distribution by post. Advantages are that - it is cheaper than providing stamps for postage - making it cost effective participants may find it more convenient to click on a link and fill in information online A potential disadvantage is that some people are worried about anonymity of online responses, even it if is reassured

What will the ratio of open and closed questions depend on?

Will be party pragmatic and party based on what type of information is needed to address the research question / aim

What is needed to enable good face, content and criterion validity ?

You need to explore the literature first, key themes should arise that will form the foundation of questions asked.

Distributing a questionnaire by post is...

a common way to distribute a questionnaire as one potentially has access to a large number of addresses. The disadvantages is that response rates to posted questionnaires are notoriously low - getting 40-50% of responses back would be considered to be good. Getting a small response rate may mean that the nature of responses is not representative of the wider group

Analysis of questionnaire data for closed questions..

closed questions lend themselves to descriptive and possibly some inferential statistics (looking for relationships / associations between factors reported on)

How can you structure ordinal scales response categories

ordinal scales, such as likert scale, can be in the form of: - categories e.g. strongly agree, agree, neither agree nor disagree, disagree, strongly disagree. - numbers - e..g a five/seven/ten point numerical rating scale -visual analogue scale for all the above , the scales can be designed so that they are either two directional or one directional. Participants can also be asked to selected the statement that best describes something, or rank items in a certain order

What needs to be done after the questionnaire has been designed?

the researcher needs to determine how the sample group will be recruited and how the questionnaire will be distributed to this group

Distributing a questionnaire to an open audience in a public place ..

this method allows a more open / wider audience to be approached than a captive audience and is cheaper than posting it Allows for possible option of researcher completing the questionnaire face to face with the respondent - useful for groups with low literacy Response rates for completion are probably somewhere between the postal/captive audience It is important to always have permission both to be in the area of distribution and to hand-out questionnaires