Research Methods Midterm

Why is variable operationalization important?

It allows others to measure a construct in the same way that you did.

You read the manual for a new assessment, and you see that the reliability of the test is very good. You conclude

Nothing yet, until you learn and think more about validity

A p-value above .05 can be interpreted to mean that

The researchers did not find statistically significant evidence of an association/difference in the variables.

A p-value below .05 can be interpreted to mean that

The researchers found statistically significant evidence of a difference/association.

Rejecting the null hypothesis is functionally equivalent to concluding that

There IS a difference/association between the variables.

True or False: The author argues that there is high-quality external evidence for EVERY clinical decision that a clinician will face.

false

A baseline should include at least __________ data points.

five

The Oxford Hierarchy helps us evaluate

The quality of external evidence

What percent of data in a normal standard deviation falls within 3 standard deviations +/- of the mean?

99.7%

In your reading, what does the author mean by an "index measure"?

A diagnostic procedure being evaluated

Remember that measurement is imprecise; only with perfect reliability will we get the same value twice. A confidence interval captures

A range of values within which we predict the true value to occur, with a specified probability (usually 95%)

The null hypothesis can be described as

A statement that there is no association/difference between variables

What is a nuisance variable?

A variable—OTHER than the one intended by the investigator—that could explain the results

New treatments tend to

Appear more effective in laboratory settings than clinical settings

Please identify what elements are included in the CADE in your reading. Check all that apply. Clear description of procedures Consideration of "confounding variables" Reliability and validity Comparison between index measure and reference standard

Clear description of procedures Consideration of "confounding variables" Reliability and validity Comparison between index measure and reference standard

Which type of research provides the strongest evidence about the existence of cause-and-effect relationships?

Experimental research

What are the three "Es" described by the author?

External evidence, clinical (internal) evidence, and evidence about patient preferences

Research studies will always have only one independent variable, although they will often have many dependent variables.

False

True or False: PICO components must be specified correctly the first time; it is uncommon that they will need to be revised based on an external search.

False

When we say a result is "statistically significant," what it means is

It is really unlikely that we got that result by chance (assuming the null hypothesis is true).

All of the following are common characteristics of experimental research EXCEPT: It uses the deductive scientific method. It can produce important knowledge about cause and effect. It rarely is conducted in a controlled setting or environment. It relies primarily on the collection of numerical data.

It rarely is conducted in a controlled setting or environment.

What two features define a quasi-experimental study? (NOTE: Quasi-experimental research may have one or both of these features.)

Multiple measurements The use of a control group

Which of the following are considered measures of effect size? Odds ratio R^2 p-value Cohen's d

Odds ratio R^2 Cohen's d

What is the defining characteristic of experimental research?

Participants are randomly assigned to groups.

Study compares results from different groups or conditions.

Quasi-experimental

How does a randomized experimental design improve the validity of the study?

Randomization accounts for nuisance/confounding variables. This allows the authors to conclude that the independent variable is truly what caused the change in the dependent variable.

Blinding and placebos are both research features that reduce the potential of

Subjective bias

In your reading, what does the author mean by a "reference standard"?

The best available method for establishing presence or absence of a target condition (i.e., a disorder)

What is "specificity"?

The proportion of people identified by the new test as NOT having the disorder who also do NOT have the disorder according to the reference standard

What is "sensitivity"?

The proportion of people identified by the new test as having the disorder who also have the disorder according to the reference standard

One purpose of a baseline period is to establish the frequency of a behavior. What is the other primary purpose?

To predict future behavior (without intervention)

True or False: Each statistical test is associated with its own distribution. For example, the z-test is associated with a normal distribution.

True

True or false? A narrower confidence interval indicates a more precise measurement.

True

Which of the following are strategies for mitigating the effects of confounding variables? Include your confounding variables in your statistical analyses as "control" variables. Use some other strategy (e.g., "matching") to equate your groups on confounding variables. Test your groups for differences on confounding variables (to see if you need to include confounding variables in your analyses). Use "time series" designs for single subject research. Use random assignment to equate your groups on confounding variables

Use some other strategy (e.g., "matching") to equate your groups on confounding variables. Use random assignment to equate your groups on confounding variables

Because entire populations are often difficult to study, researchers often include a subgroup, called a __________, in their research project.

a sample

If a p. value is >.05, then we ________________ the null hypothesis.

accept

FQs with PICO components will be useful for which of the following? Searches for external literature Evidence about patient preferences Evidence from clinical practice

all of them

What is the name of the hypothesis that states that there IS a difference or association between two study variables?

alternative hypothesis (H1)

What is it called when the effect of one independent variable differs based on another independent variable?

an interaction

A Likert scale is a commonly used measure in research with questionnaires. What kind of variable is a Likert scale?

an ordinal variable

The author quotes Francis Bacon's discussion of

bias

True or False In general, nonexperimental designs are preferred over experimental because they are of higher quality.

false

C in pico stands for

comparison or contrast

Locate and complete this quote: "Seeking evidence . . . to prove what one already believes is __________."

contrary to the fundamental thrust of E3BP

What correlation (r) is considered moderate?

correlation (r) of 0.3 to 0.5

What correlation (r) is considered weak?

correlation (r) of < 0.2

What correlation (r) is considered strong?

correlation (r) of >0.5

The __________ variable is the "output" variable; it is the variable capturing effects.

dependent

Cohen's d is a measure of effect size when looking at the _________________ between two groups (means).

difference

What does the effect size (r^2) tell you?

effect size (r^2) tells you how much of the variability in one variable can be explained by another variable. It is a measure of the association between variables.

Investigator has imposed a manipulation.

experimental

To achieve statistical significance using an alpha level of 0.01, would your test statistic have to be higher or lower than when using an alpha level of 0.05?

higher.

The __________ variable is the "input" variable or causal variable.

independent

In a hypothesis, this is the variable that is thought to produce an effect or difference.

independent variable

Identify the independent and dependent variables in the following hypothesis: "Attention improves more quickly in patients with cognitive communication deficits following TBI when speech therapy begins 2 weeks post-injury compared to 6 weeks post-injury."

independent: speech therapy timing (2 weeks vs. 3 weeks) dependent: improvement in attentions

The following is not a type of reliability: inter-rater infra-rater intra-rater test-retest

infra-rater

This is a variable where the difference between values is meaningful, such as temperature. It does not have a true "0" measurement.

interval

I in pico stands for

intervention

If a data point has a z-score of +3, what does that tell you about the data point. . .

it falls 3 standard deviations above the mean

It is relatively __________ for any given observation to be within one standard deviation of the mean.

likely

In a non-experimental study, what are the two ways of dealing with nuisance/confounding variables?

matching and controlling

This is a word that means "average." It is a measure of central tendency.

mean

Reliability, face validity, and construct validity are all considerations in evaluating the quality of

measurement

Research that is not aimed at answering cause-and-effect questions is considered __________.

nonexperimental

Study reports on a single group or condition.

nonexperimental

What is the name of the hypothesis that states that there is NOT a difference or association between the two study variables.

null hypothesis

O in pico stands for

outcome

P in pico stands for

patient/population/problem

A p. value tells us the _____________ of getting our results if the __________ hypothesis is true.

probability null

In __________, random assignment to groups is not used, but the researcher is interested in cause and effect.

quasi-experimental research

If a p value is <.05, then we ___________________ the null hypothesis.

reject

Which concept can be measured as a percentage: validity or reliability?

reliability

This is a measure that captures the average distance of each data point from the mean.

standard deviation

Effect sizes are very valuable because they tell you __________.

the degree of difference or association observed in your data

Effect size helps you to determine . . .

the magnitude of difference or association between variables

If a study is reliable, that means that. . .

the measures devised for concepts are stable on different occasions/raters.

It is possible to do quasi-experimental research using a single subject.

true

This word is associated with the concept of "validity"

truthfulness

The chi square test (X^2) is used for

two nominal/categorical variables

It is relatively ____________ for any given observation to be more than two standard deviations away from the mean.

unlikely

A single-subject time series employing the AB design is questionably __________.

valid

An invalid measure can never be __________, but an invalid measure can still be ________.

valid reliable

Which two factors related to the study's variables are important to consider when selecting a statistical test?

whether the variables are : categorical(nominal/ordinal) or quantitative (ratio/interval) and whether the researchers are looking for a difference or association.