Research Methods Module 3

used for evaluating the sensation of "shortness of breath" or "effort" in patients during exercise is a mechanism of providing a linear output as a number based on a patient's sensation. is scaled in a curvilinear way such that the number scale is expanded on the bottom and compressed near the top of the scale. Is expressed as a power function with the exponent of > 1 (actually 1.2) Correct! All of the above are correct

A Borg Scale is.....

Groups of 8 12 individuals with something in common (i.e. consumers, teachers, nurses, patients) who are led by a facilitator or moderator. The moderator asks initial questions to get it started and the results are recorded for later analysis. Advantages: Working in groups with similar interests helps to overcome barriers. Separating out individuals with different interests (sometimes conflicting interests) allows for more objective evaluations.

. Focus Group Interview

A kind of research where the subjects being studies actively engage in study design, collecting data, and solving problems

ACTION research could be defined by which of the following?

: done in a natural setting; e.g. to study the effect of a new program to encourage people to exercise. Often, if studying people, the subjects are actively involved in the collecting, analyzing and solving problems.

Action research:

1.Identify the group members 2.Questionnaire One 3. Questionnaire Two 4.Questionnaire Three 5.Qustionnaire Four 6.The results of Questionnaire Four are tabulated and presented as the final statement

Actual Steps in the Delphi Method

A process of meeting with the participants all at once and handing out a questionnaire to be given back to them. If you can do this and you have a targeted audience you can get good results.

Administered Questionnaire

Advantages: Structured: Get consistent data that might lend itself to quantification. Important issues TO YOU do not get left out. Unstructured: Feelings or observations that you may not be aware of are revealed. Disadvantages: Structured: May not include or discover information that the investigator is unaware of. Unstructured: Very difficult to quantify and make a consistent message.

Advantages of a STRUCTURED vs. UNSTRUCTURED Interview

Step 1. Decide on which data collection instrument you will use a) If available try to find an instrument previously used for a similar purpose and tested for reliability , validity and objectivity . b) If you have to make up an instrument, incorporate methods to test its reliability , validity and objectivity . Step 2. Decide how you will administer the instrument.

Approaches to Developing Questionnaires and Surveys

Objectives should be clear cut and unambiguous and should not be designed to produce pre determined results. Wording is essential.

Best Practices: AAPOR (American Association of Public Opinion

ex post facto, i.e. performed on data generated before the study was conceived. This is similar to the clinical research design we discussed: i.e. a retrospective cohort" study.

Causal-comparative research:

A highly structured questionnaire with a "laundry list" of items. Check the following that apply: I was very happy with my service at this hotel. My room was very clean The front desk personnel were very helpful Advantage: Get answers that you might desire Disadvantage: Often biased and not objective unless designed correctly

Checklist

1.Length 2.Reputation of the sponsoring agency. Are you more likely to reply to a questionnaire from: a. Red Cross b. University of Florida c. A fraternity d. Some kid in the entrance to Reitz Union 3.Complexity of the questions asked. 4.Importance of the survey to the interests of the respondent Are you more likely to reply to a questionnaire a. Student tuition hikes for next year. b. Quality of the food services on UF campus c. The impact of 16 th street renovations on your quality of life. d. Changing the Social Security benefits from 65 to 67 you. 5.Quality and design of the questionnaire . 6.Time of year , 7.Approac h or survey method (Internet, phone, etc. )

Consider other factors affecting % of returned surveys

The "Delphi Technique" is a group method in which individuals respond to a set of questions to get the "collective consensus" answer. A TYPE OF ACTION RESEARCH " There are specific methods available for use with Delphi technique "The Delphi Technique is a technique in which a group of participants assign ratings to a list of items. Participants rate items again, while aware of the previous ratings of the entire group. The Delphi Technique was originally developed by the Rand Corporation for the Polaris Missile Project, where it was used to forecast war strike scenarios and how to best react to them."

Delphi Technique

Observations of how things change over extended time frames

Developmental Research

Observations of how things change over extended time frames.

Developmental Research

Distributed Questionnaire. Most common method. a. Handed out at specific locations: Mailed back b . Mail questionnaires. c . Internet questionnaires. CONTACT BY EMAIL, Facebook, etc. Advantages: Relatively cheap, can focus on a specific subpopulation depending on where handed out or who it was mailed. Can be easily expanded to a larger population. Disadvantages: Many people will not be willing to take the time to answer. You therefore will end up with a certain subcategory of people who respond.

Distributed Questionnaire. Most common method.

1. Survey Research: (using surveys as a tool) 2. Developmental Research: Observations of how things change over extended time frames. 3. Case studies 4. Correlational Studies. 5. Observational Research: Observations of people, programs, animals in the wild, etc. 6. Action research: done in a natural setting; e.g. to study the effect of a new program to encourage people to exercise. Often, if studying people, the subjects are actively involved in the collecting, analyzing and solving problems. 7. Causal-comparative research: ex post facto, i.e. performed on data generated before the study was conceived. This is similar to the clinical research design we discussed: i.e. a retrospective cohort" study.

General Approaches to Descriptive Research:

You might ask a specific question in a different way in the same questionnaire to test the validity of the answers in the same subjects. You might develop a pilot study in which you give the survey to a group of test subjects on several occasions, separated by several weeks to determine the reproducible of the answers. You might ask well known experts in the field to evaluate the accuracy and appropriateness of the questions being asked. Correct! All of the above

If you were developing a new questionnaire to evaluate attitudes towards gender neutral bathrooms in the general population. How would you test the instrument for validity?

Often a type of SURVEY RESEARCH. e.g. Attempt to measure interests, attitudes, personality, personal problems, likes and dislikes, perceptions, motivations, etc. of a particular population.

Inventory

Another kind of scaling called "bipolar scaling" It is a device that ranks your response on a +/- 4- 9 point scale. The Likert scale ranges from SA= Strongly agree A = Agree U = Undecided D = Disagree SD Strongly disagree. • Note that the output is relatively linear across two quadrants • Order is designed to make people get off the fence . The neutral position is sometimes removed for this purpose. • Scoring systems can be applied to this to get a numerical output

Likert Scale :

Observations of people, programs, animals in the wild, etc.

Observational Research

Advantages: Good for small samples, interviewer can answer questions the respondent might have. Disadvantages: Often a slow, time consuming and expensive process. May have difficulty getting a good cross section sample. For example, only UF students, only subjects who attend church, synagogue, mosque, etc.

Personal Interview:

Advantages: a) Quick, inexpensive b) can survey a large area, c) Can survey a large spectrum of the population. Disadvantages: a) Many people are resistant to do phone interviews b) May result in a certain type of person willing to talk to you, c) Results might be dependent on the personality of the person asking for answers, d) who has a land line anymore?

Phone Interview :

Kind of Case Study - Could be done as an Observational Study Used to measure the general internal feelings and attitudes of an individual in an indirect or ambiguous way. Example: Rorshach Inkblot Test. " Another example "Washington Sentence Completion technique"

Projective Techniques

1. Inventory 2. Sociometric Technique and Analysis 3. Projective Techniques 4. Subject evaluation using Rating Scales 5. Semantic Differentiation Scale 6. Rank Order Scales 7. Likert Scale :

Qualitative Techniques used for Survey Research

subjects asked to rank in the order of importance a set of concepts or descriptors. A Survey tool.

Rank Order Scales

a)Effects of "Reactive Insight" i.e. behavior changed by responding to survey. Examples : repeated surveys on relationship with spouse can change the behavior of the couple. Frequent questionnaires regarding the degree of pain could make pain worse, etc. b)Confidentiality Issues: Examples: Surveys of happiness with workplace, effectiveness of management, drug use, etc . could affect employment status if revealed. Census including undocumented workers could result in deportation.

Reasons why informed consent might be necessary for a research survey study:

Subjects are asked to rank something between two opposing descriptors. Example: You will buy a new car within the next 12 months.: Probable: 7 6 5 4 3 2 1 Improbable

Semantic Differentiation Scale

True

Sensation of physical stimuli always follow a exponential relationship in which relative sensitivity is dampened as the intensity goes up.

a visualization of a social networking. From this, there are numerous advanced methods for quantifying the outcomes and for secondary visual display.

Sociogram

Measuring ways in which people relate to each other with respect to some common theme. A kind of Observational Research Examples: Whom in this room would you choose... 1. for advice on repairing the transmission of your car? 2. to generate creative ideas? 3. for support in taking risks? 4. to relay messages accurately? 5. for help in dealing with a difficult client? 6. to run a business for profit? 7. to get reliable information on top management decisions? 8. to keep a confidence? 9. who gives recognition for a job well done? 10. who has shown the most growth in the past year?

Sociometric Technique and Analysis

Even if you select a previously developed instrument it will often need revision before you use it. Usually you try it on a sample population. 1.Go over the questions carefully, Ask the questions on the next slide. 2. Begin with a pilot study in a small group of subjects to see if the instrument is tolerated well. 3. Include some open ended questions that might help you add or subtract questions from the original instrument. 5. Run it by experts in the field before the final revision

Step 3. Revision of the Instrument

Essentially an oral Questionnaire

Structured Interview

Questions that can be answered with true or false responses. Closed end question • Have you eaten any of the following items in the past four days: Poultry? Yes / No / Don't know Pork? Yes / No / Don't know Beef? Yes / No / Don't know Lamb? Yes / No / Don't know Advantage: Takes less time, more definitive answers. Disadvantage: Limited scope of responses...not graded.

Structured Questionnaire

Subjects rate on a scale, usually numerical but also verbal, that corresponds to their impression of an item of interest. A tool for survey research. Examples Borg Scales: Common in Exercise Science (Example of a numerical Rating Scale) Psychophysical Scaling Verbal anchors: Quasi-logarithmic

Subject evaluation using Rating Scales

False

The Delphi technique is a method of confirming a predetermined concept within a group of diverse stake holders.

It is easier to explore related issues that may not be initially conceived by the investigator

The main advantage of an unstructured interview is which of the following?

1. Decide which type of questioning you will do: Inventory, sociometric (e.g . who do you look to solve problems), projection (e.g. complete a sentence), rating scale, semantic differentiation, rank order scale, Likert scale, structured questionnaire, unstructured questionnaire, checklist , multiple choice, etc. 2. Define all terms and rid the questionnaire of jargon and ambiguity 3 . Re evaluate descriptive adjectives and adverbs that can have different meanings to different people. Examples: frequently , occasionally, and rarely 4. Beware of inadequate alternatives For example: Example: Married? Y or N. (Divorced or separated are also needed. Are you Male or Female? (transgendered individuals?) 5. Avoid "Double Barreled" Questions. Example: Are you happy with the University of Florida and did you think you got a good education here? 6. Try to include a point of reference when asking for a rating. Example: Answer from 1 to 5 regarding how well you liked a movie 1 = The best movie I have seen this year. 5 = Absolutely awful. Will not recommend that my friends see this movie. 7. Avoid Assumptions Example: How have you enjoyed your quality of life in Florida over the past year? (This assumes you lived in Florida over the whole past year and did not visit or move here recently). 8. "Phrase questions so that they are appropriate for all respondents Some concepts and quotes here come from Research Design in Occupational Education Copyright 1997. James P. Key. Oklahoma State University "Example: What is your yearly salary? " Maybe you don't work or are retired. 9. Make the question so that as much information as possible can be gleaned from the answer . Example: Do you read the Alligator? Y/N vs. How much time per day do you spend per week reading the Alligator? 10. "Provide a systematic quantification of responses." Example. What is your favorite radio station? Most won't know the name. Therefore, make it easy for them (see below). Try providing more choices, split the questions. Which of the following types of radio stations do you most frequently listen to? 1. Talk radio 2. Contemporary Rock 3. Country Western 4. Easy Listening 5. Light Jazz, etc.

Tips for developing a new questionnaire.

a)Variance (SD) of the populations, b)acceptable differences between groups, c)Power, d)alpha, e)statistical tests being used. e.g. T test, ANOVA,

Things you need to know to calculate desired sample size (a priori)

Open ended question • List the types of meat that you have eaten in the past 4 days. ____________________________ ____________________________ ____________________________ ____________________________ Advantages: Participants freer to answer in own words Disadvantages: Difficult to interpret and quantify Often used for exploratory information.

Unstructured Questionnaire: "Open ended"

1.Structured Questionnaire 2. Unstructured Questionnaire: "Open ended" 3.Checklist 4. Structured Interview 5.Unstructured Interview: 6. Focus Group Interview 7.Delphi Technique

Types of Questionnaires, Surveys, and Interviews

No set format. • Open ended questions • Usually done to overcome barriers needed to understand the underlying issues. • Often the researchers have preconceived notions as to what is important. This overcomes these biases. • Often questions initiated by the interviewer are repeated as a demonstration of reliability an validity of the outcomes.

Unstructured Interview:

Rank order scale in which residents order from 1-10 the issues that are most important to them, including a blank area for issues that are not listed as options.

You are hired by the Alachua County Commission to seek out information from county residents on what issues are most important for the commission to address in the coming year. Which of the following techniques would you use to acquire that information that would be most quantifiable and accessible to the Commission. Note, CHOOSE THE BEST ANSWER.

Qualitative Research:

a type of research that is performed to collect "descriptive" information and often involves a broad "observational approach" (usually behavioral, social or psychological phenomena).

a process or record of research in which detailed consideration is given to the development of a particular person, group, or situation over a period of time:

case study

Correlational research is a type of non-experimental research in which the researcher measures two variables and assesses the statistical relationship (i.e., the correlation) between them with little or no effort to control extraneous variables.

correlational study

Survey Research

using surveys as a tool