CPH 643

o Theoretical definitions

A definition in which the meaning of the concept (or construct) is given by substituting other words or phrases for it Although theoretical definitions are useful in the conceptual phase of a study, they fail to tell us exactly how a concept will be measured in a particular study

Types of questionnaires:

A simple item to choose between two options. A short, written response. A response scales. Multiple choice and true/false items. Checklists.

o Strategies to reduce errors on self-reported questionnaires

A thoughtful and engaging introduction can affect the mental attitude of participants. The presentation of the survey can affect a person's initial impression Tightly spaced questions can cause response errors. You should begin with items relevant to study and have demographics at the end. You should edit and pretest all surveys with a sample that represents target audience.

o Generic random error

Affect a large proportion of the respondents. Survey items that are ambiguous, instructions that are hard to follow, testing situations that are uncomfortable, etc.

o Idiosyncratic random error

Affect a small proportion of the respondents. Individual characteristics such as fatigue, hunger, distraction, etc.

o Conduct an item analysis to evaluate the quality of the items and that of the overall test

After creating a knowledge test, the developers should examine the test to identify poor items The assessment is conducted to evaluate each item and the total test with the goal of creating a test that will adequately reflect the knowledge of the people who answer the items.

o Operational definitions

An operational or empirical definition is the definition of the concept in terms of the manner in which it will be measured. That is, it is stated in terms of the empirical indicators that we will use to measure the concept

how to a void situational errors?

Best way to avoid situational errors is to provide a clear set of rules for each measurement instrument. The measurement process must be standardized so that it is applied in the same way each time in the instrument is administered.

how to fix social desirability?

Create an environment in which the respondent feels comfortable Provide adequate info about the study and explain how the data will be used

o How to avoid satisfying or negative manners?

Developing rapport with participants Carefully explaining the study to participants so that they know their role in responding to items Showing appreciation of the time and effort participants expend in completing the survey Researchers should use short items with easy to understand words, easy to follow directions, and clear formatting to reduce respondent burden

o Some situational factors are:

Environment, concurrent or past events, chance events, administration errors, time factors, investigator errors, and scoring errors.

Some instrument factors that tend to increase error are:

Inadequate instructions, poor formatting, illogical order of items, use of vague or unfamiliar terms, response options that vary, items that fail to correspond to the scale, response options that fail to fit the question, items that favor one group over another, and equipment that is not properly maintained or calibrated.

o Strategies to reduce errors during interviews

Interviewers errors can include, failing to read the items as written, skipping items, using inappropriate probes, providing inconsistent feedback, recording responses inaccurately, leading participants to respond in a specific manner, and failing to maintain an environment conducive to a successful interview. Characteristics (Age, gender, race, etc.) can influence the way participants responds. A way to reduce it is through training and practice.

• Test construction o State the purpose of the test

It is important to state the purpose of the test because the reason for measuring knowledge dictate to some extent the content, specificity, and difficulty level of the items included in the test.

o The six cognitive objectives in ascending order of complexity are:

Knowledge Lowest level of complexity, requiring simple recall or recognition of information. Comprehension • Requires the translation of concepts or principles. Application • The use of concepts or principles in new situations. Analysis • Breaking down information. Synthesis • Combining information to form a new product. Evaluation • Highest level of complexity, involves appraisal of information

Test construction o Write test items

Multiple-choice and true/false items are the two primary types of items used on knowledge tests

o Correlational systematic error

Occur when measures consistently inflate or deflate scores but do so in different ways for different respondents.

o Possible respondent errors include:

Overreporting or underreporting agreement with items, frequency of events, positive or negative attitudes. They can give partial answers, inconsistent answers, or making recording errors.

o Additive systematic error

Refers to a consistent deviation from the true score in the same direction

o The number of response options depends on several considerations, including:

The age of the respondent The experience of the respondent with this type of rating scale The ability of the respondent to discriminate between options the preference of the research The content of the scale

Test construction o Develop a table of test specifications

The item-writing process for tests begins with the development of a table of test specifications (also called a test blueprint) The table of specifications is a matrix in which the rows identify the content to be tested and the columns indicate the dimensions of that content Domain-sampling model • The domain-sampling method requires the development of a pool of items.

Test construction o State the test objectives

The researcher should list specific objectives to guide the development of actual test items Test objectives are synonymous with learning objectives

Test construction o Review content

The researcher uses certain resources to obtain accurate information about the topic. The resources include: • Review of the literature • Discussions with experts/scientists • Contact with agencies or organizations that provide information on the topic.

Idenitify respondents

Who are the respondents? What are their expected levels of language and reading skills? How familiar are they with the topic? How will respondents be recruited for the study? How much time will they have to complete the survey? Will they be willing to respond to the survey?

what is an interval scale?

(meaningful order with equal spacing between attributes)

what is a construct?

A construct is an invention of something that is not otherwise tangible- it serves as an intermediate step between an intervention program and actual behavior change.

What is a nominal scale?

A scale organized by names, and that can't be put in any sort of numerical order

What is an ordinal scale?

A scale organized by order, but that doesn't give any indication of the amount of space between each place (ie. first/second/third place in a race)

interviews

An interview is a situation in which a participant is asked a question or series of questions about a phenomenon.

Biobehavioral Measures

Biobehavioral measures are a variety of physical measures that are used to assess health behavior outcomes.

semi-structured interview format

Combines elements of both unstructured and highly structured interviews. Researchers use semi-structured interviews when they want to explore a range of opinions, perceptions and attitudes about a specific topic. • Focus groups are an example of semi-structured.

Measurement Error Factors

Errors in measurement arise from several sources that are broadly classified as errors due to the respondent, the instrument, or the situation.

A scale and an index can measure the same thing.

False

A scale measures constructs; this is not true of an index.

False

Typically the items contained in an index must be highly correlated.

False

How can scale reliability and validity be established?

Inter-item correlation coefficient

Electronic Measures

Investigators use electronic measures to augment self-reports of behaviors that are difficult or impossible to observe

what does measurement allow us to do?

It allows us to compare similar concepts and to make judgments about things without actually seeing them

what does measurement apply to?

Measurement applies to the attributes of objects rather than to the objects themselves.

what do we do with measurements

Measurements provides the means by which we can test propositions from theory.

Types of health behaviors than can be measured with electronic monitoring devices:

Pedometer. o Electronic event monitor (pill bottles).

Types of biobehavioral measures:

Physiological measures: heart rate, blood pressure, etc. o Anthropometric measures such as height and weight.

The item-writing process consists of the following six steps:

State the Theoretical Definition Identify Dimensions of the Concept Create a Content-Domain by Concept-Dimension Matrix Decide How Many Items to Include for Each Matrix Cell Write Items for Each Content-Domain by Concept-Dimension Cell Write Rules for Scale Scoring and Administration

What is measurement?

Steven (1946): measurement is the assignment of numerals to aspect of objects or events according to rules.

in nominal scales, categories should:

The categories must be mutually exclusive, meaning that a person, case, or event can be assigned to only one of the categories.The categories must also be exhaustive, so that all instances of an event or all of its characteristics can be categorized.

unstructured interview format

The interviewer asks more broad, open-ended questions about a phenomenon, to which respondents are expected to provide detailed responses. • Good for exploratory and qualitative research.

structured interview format

The interviewer reads items verbatim from an interview script and records the respondent's answers. Respondents are supposed to choose from a list of response options.

difference between index and scale?

There are two major differences between an index and a scale. The first is that the response options for items composing an index can vary whereas for a scale, the response options are either the same or similar. The second difference is the relationship among the items. In a scale the items must be positively correlated to one another. In contrast, items composing an index might be positively correlated with each other, but they also might show no correlation with other items on the index. Total scores on an index are found most often my summing responses to individual items

what do we use nominal scales for?

Used to classify variables that we can place in categories based on equivalence.

Response Scales

Used to collect data on opinions, perceptions, and attitudes. o A response scale asks respondents to select on a scale a point that most closely represents their position from low to high or from negative to positive.

what variables do we measure in ordinal scales?

Variables measured on an ordinal scale are those whose categories have a meaningful order that is hierarchical in nature.

what variables do we measure on interval scales?

Variables measured on the interval scale have a meaningful order: higher levels have more of the attribute.

Measurement Error

We refer to the extent to which a measure deviates from the ideal level of reliability and validity as measurement error.

what is Inter-item correlation coefficient

a computer-generated value (ranging from 0 - 1.0) that provides an indication of reliability for the scale. As a rule of thumb, an inter-item correlation coefficient of greater than .70 supports the reliability of the measure. • This method is known as inter-item reliability. • We do this to know whether the items go together.

o Faking bad

a person who believes that portraying themselves in a negative way may be advantageous in a certain situation may answer items in a socially undesirable manner.

what is a scale?

a scale is a selected set of correlated questionnaire items, created to assess a specific construct that has been operationally defined.

what are the levels of measurement of ratio scales

all statistical tests are possible, such as percentage of variation, regression, structured equation modeling, etc.

what is an index?

an index is a set of questionnaire items that independently add to the measure of something that may, in fact, be tangible. Index items are not expected to be correlated with one another. Instead, each item in an index provides. Its own unique contribution to explaining the construct at hand. Index are more objective than scales.

what are the levels of measurement of interval scales

arithmetic mean, standard deviation, product moment correlation and parametric tests such as ANOVAS.

what is an attribute?

attributes are the units of a variable. o Attributes have to be exhaustive and mutually exclusive.

Single-Item versus Multiple-Item Scales

because responses to single item lack precision, tend to change over time, and are limited in scope. • Multiple-item scales overcome these difficulties by expanding the number of items assessing various aspects of the concept and providing more response options o By providing more response options with which to rate each item, the researcher enhances precision

What is reliability?

consistency of measurement: does the instrument produce scores that are internally consistent or stable across time?

what are the levels of measurement of nominal scales?

frequencies and percentages

Visual Analog Scale

is a type of graphic rating scale. Although it is possible to rank scores from low to high, one cannot assume that people with the same score have exactly the same amount of attribute or that there is an equal amount of attribute between each pair of points on the scale.

o Common reasons for making errors include

misunderstanding what information was requested; maturation issues, such as fatigue and boredom; lack of interest or time; carelessness or guessing: copying; and failure to attend to instructions.

what are the levels of measurement of ordinal scales

mode, median, along with percentiles and a rank-order correlation. Also, nonparametric statistics.

What are the scales of measurement?

nominal, ordinal, interval, ratio

Survey Administration • Telephone survey:

o A distinct advantage is the inability of the participant to see the interviewer or the response options o Pauses are more disruptive and pressure to respond quickly increases o This means that the accuracy of responses in which recall, and calculation are required is diminished

• Situation factors

o A situation in which the measurement of the concept is taking place or at the point of data analysis

Survey Administration Paper and pencil survey

o Allows respondents to answer the questions on their own at their own pace o On the negative side, the respondents must figure out for themselves the meanings of terms and how to complete the survey o Surveys administered to large preformed groups suffer from the lack of random sampling, which limits the generalizability of the results

• Difficulty index

o Assesses the difficulty level of each item. o The difficulty index and the discrimination index require the collection of data from a sample of respondents Individuals from the same population for which the knowledge test is being developed. o The p value tells the test developer how each item is performing (difficult/ease) o The norm-referenced achievements tests, the range of p value for correct responses is .6 to .8, that is, between 60% and 80% of the respondents answer each item correctly.

• Discrimination index

o Assessment of how well the items discriminate among groups of participants o An examination of the ability of the items to discriminate between test takers who know the content and those who do not know it well. o For multiple-choice and true/false (or yes/no) items, D is highest when p = .5. When p is near 0 or 1, an item cannot contribute to differentiation among test takers because the total score variance among test takers is low.

Uses for the observation method in health behavior research:

o Conversations and interactions. o Body movements. o Facial expressions and reactions.

• Number of Response Options

o Expanding the response options may offset the tendency of some participants to avoid selecting the extreme (or end) options. o Lack of precision and the loss of information occur when respondents are given an insufficient number of response choices o The most common recommendation is that scales include between five and seven response choices

• Use of Adjectives, Adverbs, or numbers, or Combinations Thereof

o In developing a scale, a researcher must decide whether to use words or numbers or both as response options o We know that adjective and adverb descriptors are generally vague and that respondents interpret the words differently o Using numbers may help people decide on the degree to which they endorse a particular item

• Positive skew

o In positive skew sets, content matter. o The positive skew response set is manifested more with some attitudes and beliefs than with others. One strategy to control it is to examine the item content and modify the wording to make it more difficult for someone to select an extreme value. Another strategy is to include items that assess the highest difficulty level Providing a very extreme response category or providing more gradations.

Questionnaires (Surveys)

o Instruments that consist of items answered directly by the respondent. o Items on a questionnaire are predominantly closed-ended questions.

what are the characteristics of interval scales?

o Interval scales have equal intervals, so that the distances between the successive units of measure are equivalent. o The interval scale does not have a true zero point, a point at which there is nothing left of the attribute.

Journals and Diaries

o Journals and diaries are used to collect data about events, feelings, emotions, and beliefs. o Journaling is an activity in which a respondent writes detailed notes about thoughts and experiences. o The research diary tends to be more structured than the journal.

• The need for a new scale generally arises from one of two situations:

o No scale exists that measures the concept o No scale exists that measures the concept in the population of interest to the researcher

public health uses measures at three levels:

o Self-reported measured of knowledge, attitudes, beliefs, and practices. o Measures from clinical records of biological and physiological functions. o Measures specific to the research study that are assessed based on specimen collection and analysis.

Survey Administration Computer surveys:

o Several advantages include the elimination of the need for a data-entry person to enter responses from paper forms into the computer o Respondents are more likely to admit to sensitive behaviors o The primary advantages of using computers to administer surveys are related to the computer literacy of the staff and participants and to software glitches.

how can Random error affect the reliability of a measure.

o Small idiosyncratic errors are unlikely to affect reliability to a great extent. o Generic random errors may affect the reliability of the measure.

• Meanings of Response option Adjectives or Adverbs

o Some research examines the ways in which people interpret adjectives or adverbs that label options on the response scale o When responding to an item, people will respond with their own interpretations, what the item means to them, at that time

• Test construction (six steps)

o State the purpose of the test o State the test objectives o Review content o Develop a table of test specifications o Write test items o Conduct an item analysis to evaluate the quality of the items and that of the overall test

• Recall

o The ability and motivation of respondents to provide accurate information about behaviors or events that have occurred in the past. Telescoping occurs when respondents include more events within a time period that they should.

Survey Administration In face to face interview:

o The advantages of this approach include the likelihood that all items will be completed or than an explanation will be given for those items that are left unanswered o The disadvantages include a greater possibility of socially desirable responses because the participant must provide the answers to another human being

how to fix recall?

o The cognitive interview is based on five principles of cognition and memory retrieval: Context reinstatement • Ask people to think about environmental conditions and psychological contexts of the question being asked. Focused retrieval • Allow people to think without interruptions or distractions Extensive and varied retrieval • The more attempts one makes and the more varied the attempts, the more successful one will be in recalling. Multiple representations • Images of an event are stored in different forms and asking about these different images may elicit new information.

• Halo

o The halo effect is the tendency for ratings of specific traits to be influenced by a general attitude or set toward a person. o Halo rating occur when the rater allows a general attitude toward the person being evaluated to influence the rating of specific characteristics o Strategies to fix it include training evaluators regarding ways to rate participants, taking steps to ensure consistency across raters, and having evaluators record critical incidences to use as justification for their ratings.

• Odd or Even Number of Categories

o The middle category suggests a neutral point or a don't know response o Some researchers suggest deleting all respondents who choose the middle category or including them in the analyses as a separate o The use of an even number of categories, as shown below, forces respondents to choose an agree or disagree option and avoids the need to make decisions about how data will be treated for analysis and interpretation

What is a ratio scale?

o The ratio scale has all the characteristics of the interval scale except for the arbitrary zero point. o A true zero point permits the meaningful expression of ratios.

• Types of Response Options

o The researcher should select the type of scale (agreement, frequency, evaluative) that corresponds to the statement of the variables o Ideally, the wording of the response options is selected before writing the items

• Acquiescence

o The tendency of individuals to choose true over false, agree over disagree, and yes over no o The opposite exists, naysaying, respondents are more likely to disagree than agree, choose false over true, and say no rather than yes.

• Two terms important in the measurement process are:

o Theoretical definitions o Operational definitions

bias

o This kind of systematic error fails to affect the entire group of respondents but does affect one or more subgroups of respondents. o One group may be more prone, or biased, to answer items in a certain way because of age, gender, education, or cultural background.

• Scoring

o To create a total score for a summated rating scale, responses to individual items are summed o Some researchers use the total score in analyses o Others elect to compute a mean score This converts the total score into the units used to rate the individual items and is sometimes helpful when the researcher compares scores from several scales o When summing responses to individual items, the researcher must reverse-score the negatively worded items so that the total score reflects a greater amount or degree of the concept

• Positively and Negatively Worded Items

o To encourage attention to the task and reduce the tendency for response sets such as yea-saying, Likert suggested that about half the items be presented in a positive way and the other half in a negative way o If a negatively worded item is used, the item should be phrased in such a way that words such as not are avoided.

• End aversion

o Type of response set noted in items for which there are more than two response options. o People who exhibit end aversion tend to avoid selection of extreme values on the rating scale. The best approach to minimize end aversion is to encourage participants to select from the full range of choices

• Item-objective congruency

o Uses content specialists who evaluate the extent to which each item measures the content that the test is intended to measure. o The most important aspect of this task is to determine whether each item is accurate, and the answer correct based on current understanding in the field. o Average congruency percentage. Provides an indication how congruent the overall test is with the content domain. An average congruency percentage of at least .90 is acceptable

• Whether people accurately recall any given event or behavior is a function of two factors:

o how frequently they perform the event and how salient (meaning level of importance in memory) the event or behavior is to them. o High salience equates with better recall; however, this may be the case only when the event or behavior is not frequently repeated.

knowledge tests" can be used for the general purposes of:

o learn how much people know about a given topic/disease/health condition (this would be the case in a community needs assessment) o assess knowledge for the purpose of understanding how it shapes attitudes and behavior (the best example here is when you are using a theory to understand behavior - a good example is the Information-Motivation-Behavioral Skills (IMB) model o evaluate the effects of an HPP

• Social desirability

o the tendency of subjects to attribute to themselves in self-description personality statements with socially desirable scale values and to reject those with socially undesirable scale values. o People who intentionally try to create a favorable impression of themselves (as opposed to having an unconscious tendency to do so) exhibit a response set of faking goodness.

what is necessary to do to an attribute?

quantify it

• The two basic types of measurement error

random and systematic.

o Satisfying

refers to giving the answer that seems most reasonable at the time. Participants pay less attention to what is asked, spend less time considering their answers, and offer a "good enough" response.

what is validity?

refers to the legitimacy of the scores as a measure of the intended attribute. That is, does the instrument measure what it is intended to measure?

o Optimizing

refers to the situation in which people approach self-report tasks with the optimal mental attitude.

Thurstone Scale

respondents are asked to select from a set of items those that they endorse. Total scores are found by adding together the rank values of each item to which the respondent agrees.

how to measure constructs?

scale or index

Likert Scale

the most widely used scale for measuring behaviors, attitudes, and feelings. Likert scale builds on the concept of the one-item response scale. Respondents rate each item individually, and the responses are added to obtain a total score.

• Scaling Methods:

the reason for scaling is to obtain a single score that represents a person's overall attitude, belief, or behavior about a given situation or object.

Semantic Differential Rating Scale:

this scale assesses the meanings of concepts. SD scales generally assess meaning along three dimensions: evaluation, potency, and activity. Respondents rate a selected concept using a set of bipolar adjectives

Guttman Scale

this scale is unique in that the items are constructed in such a way that an affirmative response to one item in a set suggests affirmative responses to previous or late items in that set. Useful fit physical measurements not so good for attitudes/beliefs

types of scaling methods

visual analog scale thurstone scale Likert scale guttman scale semantic differential rating scale

o Forced choice:

yes or no answer that make you answer one way or the other. Should include answers as maybe or I don't know

Concept Selection

• A researcher who accepts the challenge of developing a new measure must first select the concept and the conduct a concept analysis. o The concept analysis allows the researcher to develop a solid understanding of the concept and the ways in which people use and interpret the concept.

Observations

• A systematic observation includes careful identification of the behaviors to observe, definitions of the behaviors to evaluate, examples of what constitutes each behavior and what does not, development of a protocol for the observation, and training of the observers.

Identify Antecedents and Consequences of the Concept

• Antecedents are events, actions, processes, cognitions, or feelings that occur before the expression of the concept

Bloom describes three primary domains of objectives:

• Cognitive (knowledge) • Affective (attitudes or emotions) • Psychomotor (skills)

Concept Analysis

• Concept analysis is the process of developing a thorough understanding of the concept to be measured • The ultimate goal of concept analysis is to clarify the meaning of the concept and its use in theory, research, and everyday life.

The Don'ts of Item Writing

• Don't write items that contain the word "not" • Don't write items that contain jargon or regional expressions • Don't write items that are ambiguous • Don't write items that contain value-laden or biased words

Create a Content-Domain by Concept-Dimension Matrix

• For ease of item development, the researcher creates a matrix (also called blueprint) that visually displays the content to be addressed by the items and the dimensions of the concept that will be measured. • When preparing the matrix, it is important to identify the variable for each cell. o Being explicit about the quantity to measure will assist within writing items.

Write Items for Each Content-Domain by Concept-Dimension Cell

• For the actual phrasing of items, the researcher turns to other sources. These include: o Experts (literature review or personal contacts) The researchers collect examples of phrases from the literature that embodied the concepts. These phrases now become useful in developing items o Qualitative interviews with representatives of the population Information collected from members of the population in which the concept appears, through either focus groups or qualitative interviews, provides another rich source of phrases for the generation of items o Other measuring instruments Researchers can review other instruments to locate useful items during the development of the new scale. They can obtain scaled from other researchers, from written sources such as books and journal articles, from companies that sell the instruments, and from internet sources Researchers may include items as written or they may modify them to fit the scale

Identify Dimensions of the Concept

• If, during the concept analysis, the researcher determines that the concept is composed of more than one dimension (is multidimensional, as opposed to unidimensional), the dimensions can be treated separately.

True/False Items:

• Include only one central significant idea in each statement • Word the statement so precisely that respondents can judge it as unequivocally true or false • Keep the statements short and keep the language structure simple • Use negative statements sparingly • Avoid double negatives • Attribute statements of opinion to some source • Avoid extraneous clues to the answer • Randomly vary the sequence of true statements and false statements (or yes statements and no statements)

State the Variables

• It is important to state a concept in the form of a variable before beginning the item-writing process • A variable is a set of values, only to one of which can an object ever be assigned at any one time • A precisely defined variables is stated in terms of quantities such as "degrees to which", "frequency with which", and "amount to which" • Variables are unidimensional in that they vary on only one attribute

There are three methods of item analysis:

• Item-objective congruency • Difficulty index • Discrimination index

Multiple choice items:

• Measure relevant information • State a single, clear problem in the stem • State the stem in simple, clear language • Put as much of the necessary wording as possible in the stem (don't repeat it in the answers) • State the item in positive form • Emphasize a negative world if it appears in the stem • Make certain that the intended answer is the correct answer or the best answer • Make all answers grammatically consistent with the stem • Avoid verbal cues that lead to the correct answers or help eliminate incorrect answers • Make distracters plausible and appealing • Use "all of the above" and "none of the above" as responses sparingly. • Use the two-layered response option sparingly • Make each item independent • Vary the position of the correct answer in a random fashion • Vary the length of the correct answer to eliminate length as a clue

self-report

• Most common types of measures. • Participants provide a direct report of knowledge, attitudes, intentions, perceptions, or behaviors.

Identify Similar and Different Concepts

• One of the goals of the concept analysis process is to identify concepts that are similar in meaning to the chosen concept and those that are different from it • An evaluation of the attributes that make the concept similar but slightly different advances the researcher's understanding of the concept and helps him or her clarify the meaning of the concept before writing items to measure it • Similar concepts are those that possess some but not all of the attributes of the chosen concepts; different concepts are those that possess none of its attributes

Survey Format

• Order of the items and the manner of presentation • Dillman (2000) recommends that the cover of a survey include a logo or other representation of the topic under consideration • Information about the study and about informed consent be provided in a letter included with the survey • When making decisions about the order of items, the researcher should begin with items that are related to the theme of the survey • Another general principle is to place sensitive items around the mid-point of the survey or toward the end. • People tend to use their earlier answers as evidence for later judgments o People tend to interpret successive questions as related to the same topic unless they are told otherwise o When general question follows a specific question on the same topic, an exclusive interpretation or an inclusive one may result, depending on the circumstances o Conversely, an item that is asked first may establish a perspective from which a respondent then chooses to answer a more general item • Researchers should place instructions where they are needed. o Instructions should occur before every transition in the type of item or the type of information requested • The respondent must be able to see clearly where to put the response for each item o Crowding items tends to increase respondent error

Random Measurement Error

• Random error is due to chance factors that influence the measurement of a variable. • Random error is an unsystematic error

Identify Dimensions of the Concept

• Some concepts are unidimensional (having only one dimension) • Whereas others are multidimensional (including more than one aspect)

Survey Versus Scale

• Survey instruments are used to gather information about health behaviors and about the variety of factors associated with health behaviors • We develop a scale when we are interested in selecting items that measure only one concept

What is a survey?

• Surveys are self-report instruments used by researchers and others to gather information on a variety of issues. o Public health professionals rely on surveys to explore people's knowledge, attitudes, beliefs, and behaviors; to ask about feelings and opinions; and to collect personal information about respondents.

Systematic Measurement Error

• Systematic error is due to factors that systematically increase or decrease true scores of an attribute. • Each time researchers use the instrument, they repeat the systematic error, affecting the extent to which the instrument actually measures the attribute. • Systematic error has its greatest effect on the validity of a measure. • Systematic error causes scores to be inaccurate, and, if extreme, it can limit the usefulness of the measuring instrument.

Identify Definitions and Uses of the Concept

• The analysis usually begins with a literature review that identifies both theory and research that incorporate the concept • It is important to learn how researchers have defined and measured the concept and how they have interpreted the results of the study

Write a Model Case

• The final step in the concept analysis process is to write an exemplar that demonstrates the critical attributes of the concept.

Decide How Many Items to Include for Each Matrix Cell

• The matrix helps the researcher visualize the content domains and the concept dimensions. • It also helps the researcher identify the number and type of items required to adequately measure the concept • Generally, researchers write more items than needed for a scale o In the process of evaluating the scale for reliability and validity, most likely some items will not fare well and will be deleted

Identify Critical Attributed of the Concept

• The review of the uses and definitions of the concept is likely to generate a list of characteristics that define the concept and help differentiate it from others.

• Response sets

• The tendency to provide responses independent of item content. Some response sets are:

Write Rules for Scale Scoring and Administration

• Though scoring rules depend on the type of scale, rules for administration depend on the preference of the scale developer and, to some extent, correspond to the medium used to present the scale and the data collection procedures

State the Theoretical Definition

• To begin the item-writing process, the researcher refers to the theoretical definition and the attributes of the concepts derived from the concept analysis

The Do's of Item Writing

• Write items that are related to the purpose of the survey and that address the objectives • Write each item as a complete sentence with a subject and a verb • Write items that are short and concise when requesting information about neutral topics • Write open-ended or long items when requesting information about sensitive topics o Loading is another way to encourage accurate responses for both open-and-closed-ended questions • Write response choices that are mutually exclusive and exhaustive o The response set should include all possible choices o Choices should be mutually exclusive, so that a respondent cannot select more than one • Spell out acronyms • Define unusual terms o If a vague term is necessary, the researchers should define it, o A better approach is to select a term or phrase that is more descriptive of the concept presented • Write items that contain one idea o If respondent has a different opinion about each idea, he or she might choose a response based on one of the ideas or choose a response that seems to be on the middle ground. • Write items that are specific • Use simple words o Be written at the sixth-through eighth-grade levels o Items are generally short, with simple, common words • Highlight terms that could easily be missed or that are particular important • For recall items, allow a specific time period that is consistent with the behavior o To minimize measurement error, researchers may provide respondents with a reasonable period of time that is consistent with the behavior o When asking about frequently occurring, routine behaviors, consider asking about the last day or the last week o For behaviors that occur less frequently or those that respondents are more likely to remember, the last month or the last year might be most appropriate time frame. • Use ranges rather than precise values for sensitive items • Use response options such as don't know and not applicable sparingly • Make careful decisions about which response options to include o The respondent generally assumes that the researcher wants a response consistent with the domains presented in the item and does not want the respondent to consider other areas o Conversely, providing a list of alternatives may remind participants of responses that might not have occurred to them without prompting