Knowledge Check #1

What is clinical importance?

Clinical importance is the practical relevance of the findings. There is no common agreement about how to evaluate the clinical importance of a finding.

When looking at a measure or questionnaire for validity, we typically descrive this as content validity. Content validity can be broken down into two subtypes. What are they?

Content and predictive validity.

What is probability theory?

It is used to explain the extent of a relationship, the probability that an event will occur in a given situation, or the probability that an event can be accurately predicted.

What is a population?

The population is a particular group of individuals or elements such as a group of people with Type II Diabetes who are the focus of the research.

What are the five possible results you may find as a result of your statistical analysis?

1. significant results that agree with those predicted by the researcher. 2. nonsignificant results. 3. significant and unpredicted results, in which the results are the opposite of those that were predicted by the researcher. 4. mixed results. 5. unexpected or serendipitous results.

What is the typical response rate for a mailed questionnaire?

25 to 40%

What statistical test is used to check a pencil and paper test for homogeneity or internal consistency?

A Cronbach's alpha or a Kuder-Richardson's 20. Both of these are expressed as an "r" value with values above .8 = high internal consistency. Note: internal consistency is checked every time a pencil and paper test is used. If your article for critique used a questionnaire of some sort, then they should report this number for both the questionnaire's development or past use AND the current sample.

What is a Type I Error?

A Type I error occurs when the null hypothesis is rejected when it is true. In plain terms...this means, that the researchers conclude that significant results exist in a study, when really, they don't.

What is a Type II Error?

A Type II error occurs when the null hypothesis is accepted when it is false. In plain terms...this means the researchers conclude that the study results are non-significant when in reality...the results were significant!

What is the purpose of conducting a confirmatory analysis?

A confirmatory analysis is used to confirm expectations regarding data from objectives, questions, or hypotheses. Inferential statistics are used, and investigators can generalize the study findings to the appropriate accessible and target population.

What is systematic variation (or systematic bias)?

A consequence of selecting subjects whose measurement values differ in some specific way from those of the general population. Think: A study about attitudes about health behaviors. What if we only ask nurses? They may view this differently than if we just took a sample of adults from the general population. Because these subjects have something in common, their values tend to be similar to others in the sample, but different from the values that might be achieved in the general population. This can introduce bias into the results.

What is sampling criteria or eligibility criteria?

A list of characteristics essential for eligibility or membership in the target population. Sampling criteria may consist of inclusion and exclusion sampling criteria.

What are the possible results that you could have with a Pearson's product-moment correlation?

A negative, a.k.a. inverse relationship, which the result will be close to -1. A positive relationship, which the result will be close to +1. No relationship, which the result will be close to 0.

A correlational design investigates relationships between or among variables in a single population. What is the inferential statistical test most commonly used with this design?

A pearson product-moment correlation, sometimes this is just called a pearson's r or just an r2. The results range from -1 to +1.

What is a questionnaire (instrument / tool / survey / measurement)?

A printed self-report form designed to elicit information through written or verbal responses of the subject. It tends to have less depth than an interview. It may gather information about facts, beliefs, attitudes, opinions, knowledge, or intentions of the subject or participant. Some questionnaires may include open-ended questions.

What is a rating scale?

A rating scale lists an ordered series of categories of a variable that are assumed to be based on an underlying continuum. Ex. Rate your pain on a 1 - 10 scale with 10 being the worst....

A research design is a ___________ for conducting a study.

A research design is a blueprint for conducting a study.

What is stability?

A type of reliability testing that focuses on the consistency of results when a test is repeated. It is also called test-retest reliability. It is expressed as an "r" value with higher numbers = greater test-retest reliability.

What is homogeneity?

A type of reliability testing that is used with pencil and paper testing. It addresses the correlation or relationship of each question on the test to other questions on the test. It basically asks...are the questions all asking about the same basic constructs? This is also known as internal consistency.

What is equivalence?

A type of reliability which involves the comparison of two versions of the same pencil and paper instrument or two observers who are observing (or grading) the same event. Two versions of the same test = alternate forms reliability. Two judges rating the same person = inter-rater reliability. It is expressed as an "r" value with values above .8 = greater IRR or alternate forms reliability.

What is a visual analog scale (VAS)?

A visual analog scale is typically used to measure the strength, magnitude, or intensity of individuals' subjective feelings, sensations, or attitudes about symptoms or situations. The VAS is usually 100mm long with right angle stops at either end and descriptive anchors for the phenomenon being measured. Subjects are asked to place a mark on the line to describe the intensity of the sensation or feeling.

What is network sampling?

Also known as "snowball, chain, or nominated sampling". Used to locate subjects that might be difficult or impossible to obtain in other ways. Takes advantage of social networks and the fact that friends tend to have characteristics in common. Can be used in both qualitative and quantitative studies.

What is the purpose of conducting an exploratory analysis?

An exploratory analysis is used to examine all of the data descriptively.

What is an interview?

An interview involves verbal communication between the researcher and the subject during which information is provided to the researcher. Interviews may be structured or unstructured.

What are statistical techniques?

Analysis procedures that are used to examine, reduce, and give meaning to the numerical data gathered in a study. For our class, there are two major categories for statistics: descriptive statistics and inferential statistics.

What is quota sampling?

Convenience sampling with a strategy to ensure the inclusion of subject types likely to be underrepresented in the convenience sample, such as females, minority groups, and the elderly. The purpose of quota sampling is to replicate the proportions of subgroups present in the target population. It may help to decrease bias.

What is data collection?

Data collection is the process of acquiring the subjects and collecting the data for the study. The actual steps of collecting the data will be specific to each study and based one the study's design and measurement techniques.

What is decision theory?

Decision theory assumes that all of the groups in a study used to test a particular hypothesis are components of the same population relative to the variables under study. This expectation is expressed as a null hypothesis in which there is no difference between the groups.

Which types of quantitative studies require larger numbers of subjects?

Descriptive and Correlational = larger numbers of subjects because the researchers may examine more than one variable and extraneous variables may affect subjects' responses. Quasi-experimental and experimental = smaller numbers of subjects because there is more control in the design.

What are descriptive statistics?

Descriptive statistics are summary statistics that allow the researcher to organize data in ways that give meaning and facilitate insight. They are calculated to describe the sample (ie. demographic statistics) and key study variables.

What are direct measures?

Direct measures involve determining the value of concrete things, such as height, weight, temperature. accuracy and precision are important.

What is evidence of validity from convergence?

Evidence of validity from convergence is determined when a relatively new instrument is compared with an existing instrument that measures the same construct. They are given to a group of people at the same time and then the scores are compared. Example: two questionnaires that measure depression.

What is evidence of validity from divergence?

Evidence of validity from divergence is determined when a relatively new instrument is compared with an existing instrument that measures the opposite or unrelated construct. They are given to a group of people at the same time and then the scores are compared. Example: a questionnaire that measures hope and despair.

What is a cross-sectional design?

Examining a group of subjects simultaneously in various stages of development, levels of education, severity of illness, or stages of recovery to describe changes in a phenomenon across stages.

What does factor analysis do?

Factor analysis examines interrelationships among a large number of variables and disentangles those relationships to identify clusters of variables that are most closely linked. It can aid in the identification of theoretical concepts or be useful in the development of a new questionnaire.

What is generalization?

Generalization extends the findings from the sample under study to the larger population. The quality of the study and the consistency of the study's findings with the findings from previous research in this area influence the extent of the generalization.

What is validity?

How well does the instrument reflect the abstract concept being measured? Hint: validity exists on a spectrum. It is not an all or nothing phenomenon. We try to determine the degree of validity an instrument or questionnaire has for a specific sample or situation.

What is evidence of validity from contrasting groups?

Identifying groups that are expected (or known) to have contrasting scores on a questionnaire. Example: administering a questionnaire about depression to two groups of people, one group with known depression, one group without, and comparing their scores.

What is the definition of a probability sampling method?

In a probability sampling method, every person or element of the population has an opportunity to be selected for the sample. Probability sampling methods increase the sample's representativeness of the target population.

What is cluster sampling?

In cluster sampling, a researcher develops a sampling frame that includes a list of all the states, cities, institutions, or organizations in which elements of the identified populations can be linked. A random sample of these can then be used in the study.

What are indirect measures?

Indirect measures attempt to measure something that is an abstract idea, a characteristic, or a concept, such as pain, stress, caring, coping, depression, anxiety. Rarely, if ever, can a single strategy measure all aspects of an abstract concept.

What are inferential statistics?

Inferential statistics are designed to address objectives, questions, and hypotheses in studies to allow inference from the study sample to the target population. They help us to: identify relationships. examine predictions. determine differences among groups.

What is the benefit of making a sample as homogenous as possible?

It helps to control for extraneous variables.

How does one ensure an adequate sample size in quantitative studies?

It is evaluated using a power analysis. Power is the ability of the study to detect differences or relationships that actually exist in the population. The minimum acceptable level of power for a study is 80%. This power level results in a 20% chance that the study will fail to detect existing effects, like a Type II error. A power analysis is done BEFORE a study is conducted to know how many subjects you will need.

What does representativeness of a sample mean?

It means that the sample, the accessible population, and the target population are alike in as many ways as possible. In quantitative research, you need to evaluate representativeness in terms of the setting, characteristics of the subjects, and the distribution of values on variables measured in the study.

If the number of variables you are studying goes up, then what probably needs to happen to your sample size?

It will need to go up or become larger because of data analysis on subgroups created by using multiple variables.

What are limitations of a study?

Limitations are problems or restrictions in a study that may decrease the generalizability of the findings.

What is measurement?

Measurement is the process of assigning numbers or values to individuals' health status, objects, events, or situations using a set of rules.

What is non-probability sampling?

Not every element of the population has an opportunity for selection to be used in the sample. This method is more commonly used in nursing studies.

What are observational measurements?

Observational measurements involve an interaction between the study participants and the observer where the observer has the opportunity to watch the participant perform in a specific setting. They may be structured or unstructured observations.

What is a power analysis?

Power is the probability that a statistical test will detect a significant difference that exists. A power analysis can assist you to identify the sample size you will need to avoid a Type II error or to ensure that a significant difference exists. The minimum acceptable power level is .80

What does regression analysis do?

Regression analysis can be used to predict the value of one variable when the value of one or more other variables is already known. Example: can we predict the length of a hospital stay of a patient with a certain diagnosis?

What is reliability?

Reliability is concerned with the consistency of the measurement method.

When a researcher uses a survey or questionnaire, how do they know it is reliable? Hint: What statistic is used, and what is considered to be a "good number"?

Reliability of a survey or a questionnaire is tested using a statistical test called a Cronbach's alpha. A Cronbach alpha value greater than .80 means that the questionnaire is considered to be reliable. Note: Researchers will report Cronbach alpha values for PAST use of the survey or questionnaire (in other studies) and in the CURRENT study.

What is sampling?

Sampling involves selecting a group of people, events, behaviors, or other elements with which to conduct a study.

What is simple random sampling?

Simple random sampling is achieved by randomly selecting elements from the sampling frame. Example: names on a slip of paper and draw from a basket, using a random numbers table, etc.

What is stratified random sampling?

Stratified random sampling is used in situations in which the researcher knows some of the variables for the population that are critical for achieving representativeness. Typical variables used include: age, gender, ethnicity, diagnosis, etc. Stratification ensures that all levels of the identified variables are adequately represented in the sample.

What is convenience sampling?

Subjects are asked to be in the study because they are "in the right place at the right time". It provides little opportunity to control for biases. Convenience samples are inexpensive, accessible, and usually less time consuming to obtain than other types of samples. They are common in healthcare studies.

What is systematic sampling?

Systematic sampling is used when an ordered list of all members of the population is available. The process involves selecting every kth individual on the list, using a starting point selected randomly.

What is the alpha, or "level of statistical significance" for most nursing studies and what does that mean?

The alpha, or level of statistical significance for most nursing studies is 0.05. This is the probability level at which the results of the statistical analysis are judged to indicate a statistically significant difference between the groups. If the p value is less than or equal to 0.05, then the result is considered to be statistically significant. Decision theory requires that this cutoff point be absolute.

What is effect size?

The degree to which the phenomenon is present in the population, or the degree to which the null hypothesis is false.

What is measurement error?

The difference between what you measured...and what the real or "true" measure of the variable is. The amount of error in any measure varies. There could be considerable error in one measurement and very little in the next. Measurement error exists in both direct measures, like blood pressure, and indirect measures, like pain.

What is effect size?

The effect is the presence of a phenomenon examined in a study. The effect size is the extent to which an identifiable effect is present (or that the null hypotheses is false). FYI: effect sizes are expressed as numbers and may be small, medium, or large in size. The statistical test tells you whether the variables are related or there is a difference between the groups. The effect size is then calculated to examine the STRENGTH of the relationship or the SIZE of the difference between the groups. So...do a power analysis before the study starts to be sure you get the right sample size for your study. Then...calculate the descriptive and inferential statistics on your data. Last, calculate an effect size after the statistical tests are completed.

What is random variation?

The expected difference in values that occurs when different subjects from the same sample are examined. The difference is random because some subjects will score higher and others will be lower than the average, a.k.a. mean, population value. Think: IQ scores...we don't all have the same IQ. As the sample size increases, random variation decreases, improving representativeness of the results.

What is content validity?

The extent to which the measurement / questionnaire / scale includes all of the major elements or items relevant to the construct being measured. How do you prove this? 1. do the items in the scale reflect the description from the ROL? 2. how do content experts rate the items on the scale? 3. how might potential subjects respond to items on the scale?

What is an element?

The individual units of the population and sample are called elements. An element can be a person, event, behavior, or any other single unit of study. Remember: when elements are persons, they can be called subjects or participants depending on the research design.

What is the accessible population?

The portion of the target population to which the researcher has reasonable access. The accessible population might include elements within a country, state, city, hospital, nursing unit, or primary care clinic...such as the diabetics in a primary care clinic in Arlington, TX.

If a diagnostic / screening test is very sensitive, what does that mean?

The proportion of patients with the disease who have a positive test result. A highly sensitive test is good at identifying a patient with the disease or disorder and has a low percentage of false negatives.

If a diagnostic / screening test is very specific, what does that mean?

The proportion of patients without the disease who have a negative test result. A highly specific test is good at identifying a patient without the disease or disorder and has a low percentage of false positives.

What is purposeful sampling?

The researcher will consciously select certain participants, elements, events, or incidents to include in the study. The goal is to obtain information-rich cases from which the researchers can obtain in-depth information needed for their studies. Used in qualitative studies.

What is a sample?

The sample defines the selected group of people or elements

What is the research setting?

The site or location used to conduct a study.

What is the target population?

The target population is a group of individuals who meet a particular set of sampling criteria, such as: female, 18 years or older, new diagnosis of type II diabetes, not on insulin.

What is important to ask regarding physiological measures?

Their accuracy, which is validity, precision which is consistency and reliability, and sources of error of measurement.

Identify four types of descriptive statistics.

There are four types of descriptive statistics: frequency distributions like gender, ethnicity, marital status data. measures of central tendency like mean, median, mode - age data. measures of dispersion like range, variance, standard deviation, confidence intervals, standardized scores, and scatterplots.

How does a researcher determine what inferential statistical test to use? (Yikes!!!)

There's no easy answer to this one. Usually it's a combination of the following: the study's purpose. the use of questions, objectives, or hypotheses the study's design (use of groups??). the level of measurement of the variables like nominal, ordinal, and interval.

What are chi square and analysis of variance, a.k.a. ANOVA, tests used for?

To identify differences among groups in a research study.

How does a researcher describe a sample?

Variables relevant to the sample are called demographic variables like age, gender, educational levels. Demographic variables are usually described with descriptive statistics.

How do you know when the number of subjects in a qualitative study is adequate?

When saturation of information, like in redundancy, and verification of information like in further confirmation of hunches, relationships, or theoretical models, occurs.

In which setting does descriptive and correlational studies, and all qualitative studies take place?

a natural or field setting, which is a uncontrolled or real-life situation or environment. Note: the researcher does not manipulate or control or change this environment.

What is a sampling plan or sampling method?

a sampling plan or sampling method outlines the strategies used to obtain samples for studies. There are two major types of sampling plans: probability and non-probability.

What statistics can be used to determine causality?

chi-square, t-tests, ANOVAs, and ANCOVAs

What types of non-probability sampling are more commonly used in quantitative research studies?

convenience and quota sampling

What are the five types of nonprobability sampling?

convenience sampling. quota sampling. purposeful or purposive sampling. network or snowball sampling. theoretical sampling.

What are three common settings for conducting nursing research?

natural or field setting, partially controlled, and highly controlled settings.

What are the four levels of measurement?

nominal data is a category only - like gender, kidney stone. ordinal data is in categories that can be ranked - like pain, dyspnea with ADLs. interval data is in categories with equal numerical distances between them and no absolute zero point - like temperature. ratio data is in categories, has equal numerical distances and has an absolute zero point - like weight.

What types of non-probability sampling are more commonly used in qualitative research studies?

purposive, network, and theoretical sampling

What are the tasks of data collection?

selecting and recruiting subjects. collecting data in a consistent way. maintaining research controls as indicated in the study design.

What are the four types of probability sampling methods?

simple random sampling. stratified random sampling. cluster sampling. systematic sampling.

What are the three aspects of reliability testing?

stability, equivalence, and homogeneity

What is generalization?

the application of information that has been acquired from a specific instance to a general situation. Generalization extends the implications of the findings from the sample studied to a larger population.

What is exclusion sampling criteria?

the characteristics that can cause a person or element to be excluded from the target population.

What is inclusion sampling criteria?

the characteristics that the subject or element must possess to be a part of the target population.

What are the two types of measurement error?

the difference between the measured value and the true value without pattern or direction, which is called random. So the more times you measure the value and average those values together...hopefully the closer you are getting to the person's true value or score unless your random error rate is high. Example: Think about taking the NCLEX over and over...

What are some characteristics of significance in studies / significant studies?

the findings make an important difference in people's lives. you can generalize the results far beyond the study's population and it affects large numbers of people it generates more research. it contributes to theory development and becomes a theory. it has implications for other disciplines besides nursing. it is frequently referred to in the literature.

What is an acceptance rate?

the percentage of subjects who agreed to participate in the study.

What is a refusal rate?

the percentage of subjects who declined to participate in the study.

What is systematic measurement error?

the variation in measurement value from the calculated average is always in the same direction. For example: most of the variation may be higher or lower than the average that was calculated. Example: Why does my scale at home routinely weigh me at 150lbs in the morning....but the scale at the MD's office weigh me at 157lbs for an 0830 appointment?!?!

What is sample attrition?

the withdrawal or loss of subjects from a study that can be expressed as a percentage.

What is a likert scale?

type of rating scale that typically has four, five or seven categories that address agreement, evaluation, or frequency. It is used to determine the opinions or attitudes of study subjects. Ex. strongly agree, agree, uncertain, disagree, strongly disagree.