Data Collection and Management

Scales: Guttman Scale Rank Ordering (Cumulative Scale)

A set of statements is presented that reflects increasing intensity of characteristics being measured There is only one unique combination of responses to achieve a particular score

Creating Surveys

Consider the PICO question A survey is used to obtain information from subjects vs measuring performance Validity concerns Use guiding questions and hypotheses to focus the content Create a questionnaire outlines

Face to Face Interview

Face to face interviews are more effective for establishing rapport with the subject Allow for additional observation of the subjects' response to questions

Ratio Measurements

Highest level of measurement with an absolute zero Has empirical rather than arbitrary meaning All mathematical and statistical operations are permissible with ratio level data (distance, age, time, weight, decibels)

Data Collection: Interview

Interviews are used to gather information in survey research In survey research interview can take place in person or over the phone When completing ethnographic (qualitative) research, interviews are always face to face

Data Collection: Interview Cons

Lack of anonymity Costly to complete (staffing) Time consuming

Written Questionnaires Cons

Limited/poor response Missing or inaccurate data Not helpful for objective observations Respondents may misunderstand questions Researcher may misinterpret responses Recall bias

Nominal Measurements

Lowest level of measurement Classification may be coded by name, number, letter or symbol but has no quantitative value (gender, blood type, diagnosis)

Written Questionnaire Distribution

Mail In person Electronic Distribution Survey Software

Scales: Semantic Differential Scales

Measures attitudes or values based on a continuum that extends between 2 extreme opposites Usually a 7 point scale Only two extremes of the scale are labeled Not based on agree/disagree, but the varying degree of the opposite adjectives

Scales: Visual Analogue Scale

Measures intensity between 2 extreme opposites There are no points on the scale Ratio level data (mark is measured from left anchor in millimeters) Only the two extremes of the scale are labeled Not based on agree/disagree, but the varying degree of the opposite adjectives

Questionnaire Cover Letter

Must include a cover letter to orient the respondents to the survey

Written Questionnaires: Other Category

Need to consider it Should include a fill in the blank so the researcher can account for the difference

Hierarchy of Information: Least to Greatest

Nominal Ordinal Interval Ratio

Quantitative Observations

Observations are structures and formalized

Qualitative Observations

Observations are unstructured and ever-changing according to the context and results of the observations

Data Collection: Observation

Observations do not have to be of human subjects Observations can be made of equipment, objects, or human subjects performing on a video tape

Concept of Measurement: Scales

Ordered system based on a series of questions that provide an overall rating that represents the degree to which a respondent possesses a particular attitude, value, or characteristic

Scales: Summative Scale

Presents a total score with all items contributing equal weight to the total

Constructing Close-Ended Questions

Provide multiple choice response options Choices must be exhaustive and mutually exclusive Avoid double barrel questions (and/or) Easy to code

Questionnaire Cover Letter: Elements

Purpose and importance of the study, include agency info Why respondent was chosen for the survey Assure anonymity and encourage honesty Suggest how long it will take to complete Provides instructions on returning the survey including the deadline date Thank the respondent for their time, and provide an opportunity for them to get a summary of the report/research Sign the letter

Data Collection Methods

Range along a continuum from unstructured observations to highly structured questionnaires Most are used in both qualitative and quantitative research Most are used in conjunction with other data collection techniques

Interval Measurements

Rank ordered characteristics with known and equal distance between the units of measurement No absolute/true zero

Ordinal Measurements

Rank ordered on basis of an operationally defined characteristic or property (manual muscle tests, pain, military rank)

Constructing Open-Ended Questions

Respondents answer in own words Probe for feelings and opinions without bias Used when research does not know all possible responses to a question Difficult to code Easily misunderstood/interpretable

Structured Interview

Standardized set of questions asked in the same order and with the same options for response

Data Collection: Written Questionnaires

Structured surveys that are self administered using pen and paper or electronic formats Can be used in descriptive, exploratory, and experimental research In qualitative studies, surveys and questionnaires are open-ended/unstructured Open and close-ended questionnaires

Scales: Likert Scale

Summative scale to assess attitudes or values Usually a 5 point scale

Data Collection: Surveys

Surveys are a major type of questionnaire used in research Non-experimental instruments designed to measure specific characteristics Time consuming to create (involves multiple stages of revision before creating a useful research tool)

Parametric Statistics

Testing is based on population parameters Includes tests of significance based on interval or ratio scale data

Nonparametric Statistics

Testing not based on population parameters Includes tests of significance based on ordinal and nominal data

Creating Surveys: Consider the PICO Question

The research problem must be considered with reference to the target population

Unstructured Interview

There is no fixed order or agenda for discussing topics of concern Structure is conversational and occurs in a natural setting Popular with qualitative studies More authentic

Creating Surveys: Review Existing Instruments

There may already be an instrument that is applicable or adaptable to the study Use your time wisely; there is no reason to reinvent something Don't plagiarize, but you can get permission to change something on an instrument

Written Questionnaires Pros

Allows for anonymity Subjects can respond on their own timeline Can examine self-observation such as attitudes and values Ability to obtain a large sample at a low cost Can analyze data in a variety of ways (statistical manipulation)

Data Collection: Interview Pros

Allows for in depth analysis of respondents' reaction

Scales: Continuous Scales

Based on interval or ratio measurement Characteristic on a continuum such as age, blood pressure, or years of experience Expressed as a summary score

Scales: Categorical Scales

Based on nominal measurement Respondent assigns self to a classification such as gender, diagnosis, religion, or race Expressed as frequency counts or percentages

Creating Surveys: Questionnaire Outline

Created one guiding questions and hypotheses are determined Includes each item of information that will be needed to answer guiding questions Each item should relate back to at least one of the study objectives Each individual question should add to a larger context that will answer the guiding questions Include demographic questions (to define sample and allow for generalization to the population)

Scales: Cumulative Scale

Demonstrates an accumulated characteristic, with each item representing an increasing amount of the attribute being measured

Correlational Statistics

Determines the relationship between 2 variables

Creating Surveys: Validity Concerns

Does this survey measure what it is intended to measure?

Analysis of Survey Data

Each item on a survey in a data point Each data point must be given a variable name or number Each survey must be reviewed for validity Surveys with missing or incorrectly completed responses must be thrown out All unusable data must be kept to report as a % within the report

Constructing Survey Questions: Sensitive Questions

Word questions to put respondent at ease Avoid value judgments Sensitive topics: compliance, comprehension, etc

Creating Surveys: Designing the Instrument

Write questions to address each behavior, knowledge, skill, or attitude reflected in the guiding questions Questions should be grouped to reflect topics Presentation should be friendly and easy to use Font size should be easy to read (11 point) Complete pilot testing and revisions to refine the survey