Chapter 7 | Survey Research (Exam 2)

Preparing a Written Research Report

(1) *Abstract* - The abstract is a short (about 100-500 word) summary of the entire paper. It should include: goals and objectives, results, and conclusions. It is usually one of the last parts of the paper to be written. - Generic Overview (2) *Introduction* - The introduction also has three main purposes. First, it provides background and motivation for your topic (usually includes a review of current literature on the topic). Second, it describes the focus and purpose of the paper you are writing. Third, it gives an overview of what is contained in the paper's various sections. (3) *Literature Review* - Sometimes combined with Introduction (4) *Methodology* - This section describes what you did, how you did it, gives strategies, sample calculations, diagrams and circuits, and descriptions of equipment. The goal here is to give the reader sufficient information to be able to repeat your work if desired. (Of course some "standard techniques" can be simply referenced). (5) *Results* - This section is where you prove your point with the data. Give graphs and tables of costs, profits, whatever your data is. Also give some description or guide to help the reader recognize your important points. - No opinion should be listed here. Only the results from the study. Usually the shortest portion of the paper. (6) *Discussion* - Compares / contrasts to what they found in the literature (agreement / departure with literature) - Limitations are outlined - Refers to hypothesis, objectives, or questions (7) *References* - A list of the references you used in the work & writing the paper. (8) *Appendices*

Sampling Techniques

(1) *Probability (Random) Samples*: - Systematic Random (Formula: K=N/n) - Stratified Random - Cluster (2) *Non-Probability (Non-Random) Samples* - Convenience - Snowball - Proportional - Quota *Sample Size*: Larger samples yield more precise results (less subject to sampling error) than smaller samples

Methods of Collection

(1) *Questionnaire* (2) *Interview* (3) *Telephone* (4) *Observation* (5) *Web-Based* - Most commonly used in HC settings - Lowest Response (6) *Face-to-face* - Highest Response (7) *Mail* (8) *Email*

Reliability and Validity

(1) *Reliability* - *Test-Retest*: Used to assess the consistency of a measure from one time to another. - *Inter-Rater*: Used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon. - *Intra-Rater*: Used to asses the degree of agreement among repeated administrations of a diagnostic test performed by a single rater. (2) *Validity*: Instrument measures what it is designed to measure. A scale or test can be highly reliable and be invalid - *Content Validity*: Experts judgment of appropriateness of content after evaluation of an instrument - *Face Validity*: Non-expert judgement

Basic Steps in Survey Research

(1) *What do you want to know* (2) *Formulate purposes and hypotheses* - Not necessary to have a hypothesis because you are testing the null rather than the alternative. (3) *Locate related literature and write a literature review* (4) *Identify population and select sample* (5) *Identify or construct an instrument* - You can use an instrument / assessment previously used or create your own. (6) *Pre-test (pilot) the questionnaire, if practical* - Helps to ensure the questions are understandable to participants (7) *Administer the instrument to participants* (8) *Analyze data using descriptive and inferential (as appropriate) statistics* (9) *Writing up the research report* - Authors Guide: will give you specific instructions on how to format research to submit it to particular journals

Literature Review: Creating a Synthesis

(1) Discuss several studies in groups. Multiple references can be cited when their findings are similar (2) Concise and provide details sparingly. (3) Point out trends in the literature (4) Point out gaps in the literature (5) Discuss studies critically (6) Use appropriate terms to describe research result (7) Write a summary (8) Reference list - *Importance of Literature Review*: Lets us know if something has already been researched. We evaluate to determine previous weaknesses or limitations (found in the discussion section of the study).

Instruments / Measurement Tools

(1) Existing instruments (2) Published instruments

Selecting the Sample

(1) FIRST, define the population (2) Using appropriate sampling technique (or an entire population can be used) (3) Identified persons must be persons who (1) have the desired information and (2) are likely willing to give it. (4) Develop the inclusionary and exclusionary criteria (5) Sample should be representative of the population (6) *Larger the sample; increases generalizability* - *Population (N)* Ex(s) = research class (consisting of OTs, PTs, and P&Os) - *Sample Size (n)* Ex(s) = 33 people out of the research class

Techniques for Increasing Response Rates to Mailed Surveys

(1) Self-addressed stamped envelope (2) Incentive (3) Mail an advance postcard (4) Use professional-looking stationary, such as university stationary (with permission) (5) Keep survey short and straightforward (6) Send follow-up mailings to non-respondents

Survey Research

*A study in which data are collected to describe the opinions, attitudes, demographics, knowledge, outcomes, or perceptions of people about some topic or issue*. - It is NOT an experiment - Not looking at providing intervention (independent / dependent variables) - It is also called a descriptive study

Conducting a Questionnaire Study

- A questionnaire is much more efficient in that it requires less time, is less expensive, and permits collection of data from a larger sample. - The researcher specifies the information that is being used in the questionnaire - Draft the questionnaire - Set up a pilot study for the questionnaire - Redrafting the questionnaire - Administer the questionnaire - May be administered by mail, telephone, email, internet, or in person.

Likert Format

- Allows for indecision Ex(s) = I feel confident in my ability to put on my prosthesis. Strongly agree 5 Agree 4 Neutral 3 Disagree 2 Strongly disagree 1

Constructing the Instrument

- Attractive, brief, and easy to read. - Structured, or closed-ended items should be used if at all possible. Structured items also facilitate data analysis; scoring is very objective and efficient. - The use of scaled items (selection method) (ex = Likert - which tests agreeableness), ranked items, and checklist allows ease of scoring and completion of survey.

Double-Barreled Questions

- Avoid using "or" or "and" to ask about two things within a single questions

Analyzing Results

- Descriptive statistics - Variability - Inferential statistics --> Will be covered more in the next lecture

Forced Choice Format

- Does not allow for a neutral response Ex(s) = The prosthetist always involve me in setting goals for my treatment. Strongly agree 4 Agree 3 Disagree 2 Strongly disagree 1

Probability (Random) Samples

- Everyone has an *EQUAL* chance of being selected (1) *Systematic Random*: Every Kth person (K = Population / Sample) Ex(s): Population = 100 people & you want the sample to = 20 ...you would take every 5th person (2) *Stratified Random*: Sometimes called proportional or quota random sampling, involves dividing your population into homogeneous subgroups and then taking a simple random sample in each subgroup. Even % of the population. (3) *Cluster*: Groups based on geographic location

Administration

- Give survey a title - Brief introduction stating purpose - Amount of time to complete - Conclude with "Thank You' and Contact information

Close Ended Question(s)

- Gives *quantitative* results. - Easier to analyze Ex(s) = How would you rate the care you received in occupational therapy? (Circle your response) Excellent 4 Good 3 Poor 2 Fair 1

Open Ended Question(s)

- Gives a more *qualitative* response. - Lets the participant expound upon their answer. Ex(s) = How would you describe your overall satisfaction with occupational therapy services?

Dealing with Non-Response

- If the total response rate is low (40% or lower), there may be a problem with generalizability of the results. - A major threat to surveys, tend to be low especially when your target population or sample is not well-defined - *Typically a 50% response rate is considered acceptable*

Constructing the Questionnaire

- In an unstructured item format (supply method), participants have complete freedom of response, questions are asked but respondents must construct their own answers. Although it may provide some insight, it may be difficult to analyze and interpret - Avoid *double-barreling*: More than one question within a single question - Avoid leading questions, questions that assume a fact not necessarily in evidence, and questions that do not indicate a point of reference (Do you spend a lot of time studying during the week? Vs. How many hours per week do you spend studying?) - Avoid jargon

Stating the Problem

- Must be of significance to the participant to respond. - Must be defined clearly in terms of the kind of information needed; questions must be formulated and every item on the questionnaire should directly relate to the participant.

Sensitive Questions

- Sensitive or personal issues that can cause some discomfort Ex(s) = sexual practices, mental health status, drug use, compliance with weight loss program - Issue with survey = sometimes people respond the way they think they think they should respond (especially with sensitive questions)

Research Purpose

- Should refer to at least one population and at least one variable. Ex(s): The purpose is to determine older adults (>85 years) with LE amputation attitudes toward wearing a prosthesis. - Others may have one population and two variables. Ex(s): The purpose is to determine adolescents knowledge of STDs and their attitudes towards individuals with an STD

Example of a Survey Used in Healthcare

- Surveys are used in the healthcare industry frequently - Ex(s) = *Follow-up studies*: Conducted to determine the status of a group after some period of times. *Rationale*: FIMs outcomes of an individual or group post discharge at 6 months, 1 year - Ex(s) = *Patient Satisfaction Surveys*: Administered post discharge of clients from healthcare facility to determine the client's current level of satisfaction with services and/or current functional status compared to discharge functional status *Rationale*: Program Evaluation - These both can be used to assess function and satisfaction (effectiveness)

Pre-Testing / Pilot Study

- The questionnaire and cover letter should be tried out in a field test using a few respondents who are similar to those who will respond to the questionnaire. - It yields data concerning the instrument deficiencies as well as suggestions for improvement. Omissions or unclear or irrelevant items should be revised. - A too often overlooked point is to determine the validity of the questionnaire (does it measure what it is intended to measure?) - Individual will *not* be used as participant for the actual study. - Why do a pilot? To correct any kinks in the study (ex = question being unanswerable)

Characteristics of Survey Studies

- Typical survey methods are concerned with assessing attitudes, opinions, preferences, demographics, practices, and procedures - Is useful for investigating a variety of issues and problems. *4 Levels of Measurement (NOIR)* (1) *Nominal*: categorizing / naming | lowest level of measurement | you can only fit into one of these categories within nominal (2) *Ordinal*: intervals that are not equal (ex = 1st, 2nd, 3rd) (3) *Interval* (Scale): order is equal but there is no zero (4) *Ratio* (Scale): has a true zero (ex = age, income, GPA) - *Demographic Information*: Typically the first thing you see on a survey. Describes the person responding to the survey Ex(s) = age, gender, marital status, school program, etc).

Longitudinal Survey

Collects data at two or more time periods to *measure changes over time*. Ex(s) See Picture: - Rationale: There has been a drop in school attendance and decline in academic performance among children experiencing the loss of a parent.

Cross Sectional Survey

Data collected from individuals in *one single period of time*. Ex(s) See Picture: - The aim of the study is to determine graduate students' knowledge of the ADA. - Rationale: Students will complete an ADA assignment in their community-based course. *Always remember to consider why you chose a specific group*

Non-Probability (Non-Random) Samples

Everyone *does not have an equal chance of being selected. Study is less generalized. (1) *Convenience*: Subjects that you have access to are selected. *Easiest, cheapest and most common* (2) *Snowball*: Begin by identifying someone who meets the criteria for inclusion in your study. You then ask them to recommend others who they may know who also meet the criteria. Ex(s) = Researchers recruited 30 subjects but only 25 showed up so they ask those 25 if they know of anyone who could participate (3) *Quota*: You select people non-randomly according to some fixed quota. There are two types of quota sampling: proportional and non proportional. - *Proportional*: Want to represent the major characteristics of the population by sampling a proportional amount of each. Ex(s) You know the population has 40% women and 60% men, and that you want a total sample size of 100, you will continue sampling until you get those percentages and then you will stop. So, if you've already got the 40 women for your sample, but not the sixty men, you will continue to sample men but even if legitimate women respondents come along, you will not sample them because you have already "met your quota." - *Non-Proportional*: A bit less restrictive. You specify the minimum number of sampled units you want in each category. here, you're not concerned with having numbers that match the proportions in the population. Instead, you simply want to have enough to assure that you will be able to talk about even small groups in the population

Purpose and Hypothesis

Ex(s) = *Broad Problem Area*: Autism *Narrower Problem Area*: Satisfaction and access to occupational therapy services *Research Purpose for a Survey*: To determine families of children with ASD knowledge of the availability of occupational therapy services in the state of Alabama. *Research Hypothesis for the Research Purpose*: Families of children with ASD have limited knowledge about occupational therapy services in Alabama.

Use Appropriate Terms to Describe Research Results

Ex(s) = - The results of this pilot study offer some evidence that individuals ..... - Evidence from recent surveys suggests that individuals..... - Based on these findings, there is strong reason to believe that individuals... - Authors of both studies reported preliminary data suggesting that individuals... - These five recent surveys provide strong evidence that individuals....

Rating

Ex(s) = Rate the value to you of each of the following physical therapy continuing education course formats. (Rate all) 1=Preferred; 2= Acceptable; 3 = Not Acceptable ______A. Lecture ______B. Workshop ______C. Computer program

What are the 2 Categories of Surveys?

Longitudinal & Cross Sectional