Research methods test 2

Observer bias

A bias that occurs when observers' expectations influence their interpretation of the participants' behaviors or the outcome of the study.

Quota sampling

A biased sampling technique in which a researcher identifies subsets of the population of interest, sets a target number for each category in the sample, and nonrandomly selects individuals within each category until the quotas are filled.

Purposive sampling

A biased sampling technique in which only certain kinds of people are included in a sample.

Spurious association

A bivariate association that is attributable only to systematic mean differences on subgroups within the sample; the original association is not present within the subgroups.

Reactivity

A change in behavior of study participants (such as acting less spontaneously) because they are aware they are being watched.

Observer effect

A change in behavior of study participants in the direction of an observer's expectation. Also called expectancy effect.

Statistical significance

A conclusion that a result from a sample (such as an association or a difference between groups) is so extreme that the sample is unlikely to have come from a population in which there is no association or no difference.

Cronbach's alpha

A correlation-based statistic that measures a scale's internal reliability. Also called coefficient alpha.

Strength

A description of an association indicating how closely the data points in a scatterplot cluster along a line of best fit drawn through them.

Stratified random sampling

A form of probability sampling; a random sampling technique in which the researcher identifies particular demographic categories of interest and then randomly selects individuals within each category.

Oversampling

A form of probability sampling; a variation of stratified random sampling in which the researcher intentionally overrepresents one or more groups.

Self-selection

A form of sampling bias that occurs when a sample contains only people who volunteer to participate.

Population

A larger group from which a sample is drawn; the group to which a study's conclusions are intended to be applied. Also called population of interest.

Known-groups paradigm

A method for establishing criterion validity, in which a researcher tests two or more groups, who are known to differ on the variable of interest, to ensure that they score differently on a measure of that variable.

Physiological measure

A method of measuring a variable by recording biological data.

Observational measure

A method of measuring a variable by recording observable behaviors or physical traces of behaviors. Also called behavioral measure.

Self-report measure

A method of measuring a variable in which people answer questions about themselves in a questionnaire or interview.

Survey

A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet. Also called poll.

Poll

A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet. Also called survey.

Cluster sampling

A probability sampling technique in which clusters of participants within the population of interest are selected at random, followed by data collection from all individuals in each cluster.

Systematic sampling

A probability sampling technique in which the researcher counts off members of a population to achieve a sample, using a randomly chosen interval (e.g., every nth person, where n is a randomly selected number).

Multistage sampling

A probability sampling technique involving at least two stages: a random sample of clusters followed by a random sample of people within the selected clusters.

Interval scale

A quantitative measurement scale that has no 'true zero,' and in which the numerals represent equal intervals (distances) between levels (e.g., temperature in degrees). See also ordinal scale, ratio scale.

Ordinal scale

A quantitative measurement scale whose levels represent a ranked order, in which it is unclear whether the distances between levels are equal (e.g., a 5-star rating scale). See also interval scale, ratio scale.

Ratio scale

A quantitative scale of measurement in which the numerals have equal intervals and the value of zero truly means 'nothing.' See also interval scale, ordinal scale.

Negatively worded question

A question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity.

Semantic differential format

A response scale whose numbers are anchored with contrasting adjectives.

Representative sample

A sample in which all members of the population of interest are equally likely to be included (usually through some random method), and therefore the results can generalize to the population of interest. Also called unbiased sample.

Biased sample

A sample in which some members of the population of interest are systematically left out, and as a consequence, the results from the sample cannot generalize to the population of interest. Also called unrepresentative sample.

Outlier

A score that stands out as either much higher or much lower than most of the other scores in a sample.

Census

A set of observations that contains all members of the population of interest.

Response set

A shortcut respondents may use to answer items in a long survey, rather than responding to the content of each item. Also called nondifferentiation.

Correlation coefficient r

A single number, ranging from -1.0 to 1.0, that indicates the strength and direction of an association between two variables.

Third-variable problem

A situation in which a plausible alternative explanation exists for the association between two variables. See also internal validity.

Directionality problem

A situation in which it is unclear which variable in an association came first.

Restriction of range

A situation involving a bivariate correlation, in which there is not a full range of possible scores on one of the variables in the association, so the relationship from the sample underestimates the true correlation.

Multiple regression

A statistical technique that computes the relationship between a predictor variable and a criterion variable, controlling for other predictor variables. Also called multivariate regression.

T test

A statistical test used to evaluate the size and significance of the difference between two means.

Masked design

A study design in which the observers are unaware of the experimental conditions to which participants have been assigned. Also called blind design.

Multivariate design

A study designed to test an association involving more than two measured variables.

Longitudinal design

A study in which the same variables are measured in the same people at different points in time.

Forced-choice format

A survey question format in which respondents give their opinion by picking the best of two or more options.

Open-ended question

A survey question format that allows respondents to answer any way they like.

Likert scale

A survey question format; a rating scale containing multiple response options that are anchored by the terms strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. A scale that does not follow this format exactly is called a Likert-type scale.

Moderator

A third variable that, depending on its level, changes the relationship between two other variables.

Double-barreled question

A type of question in a survey or poll that is problematic because it asks two questions in one, thereby weakening its construct validity.

Leading question

A type of question in a survey or poll that is problematic because its wording encourages only one response, thereby weakening its construct validity.

Predictor variable

A variable in multiple-regression analysis that is used to explain variance in the criterion variable. Also called independent variable.

Mediator

A variable that helps explain the relationship between two other variables. Also called mediating variable.

Categorical variable

A variable whose levels are categories (e.g., male/female). Also called nominal variable.

Quantitative variable

A variable whose values can be recorded as meaningful numbers.

Snowball sampling

A variation on purposive sampling, a biased sampling technique in which participants are asked to recommend acquaintances for the study.

Mean

An arithmethic average; a measure of central tendency computed from the sum of all the scores in a set of data, divided by the total number of scores.

Curvilinear association

An association between two variables which is not a straight line; instead, as one variable increases, the level of the other variable increases and then decreases (or vice versa). Also called curvilinear correlation. See also positive association, negative association, zero association.

Bivariate correlation

An association that involves exactly two variables. Also called bivariate association.

Criterion validity

An empirical form of measurement validity that establishes the extent to which a measure is correlated with a behavior or concrete outcome that it should be related to.

Discriminant validity

An empirical test of the extent to which a measure does not associate strongly with measures of other, theoretically different constructs. Also called divergent validity. See also convergent validity.

Convergent validity

An empirical test of the extent to which a measure is associated with other measures of a theoretically similar construct. See also discriminant validity.

Unobtrusive observation

An observation in a study made indirectly, through physical traces of behavior, or made by someone who is hidden or is posing as a bystander.

Acquiescence

Answering 'yes' or 'strongly agree' to every item in a survey or interview. Also called yea-saying.

Convenience sampling

Choosing a sample based on those who are easiest to access and readily available; a biased sampling technique.

Which statistic is used to represent the internal reliability of multiple-item self-report scales?

Cronbach's alpha

Socially desirable responding

Giving answers on a survey (or other self-report measure) that make one look better than one really is. Also called faking good.

Faking good

Giving answers on a survey (or other self-report measure) that make one look better than one really is. Also called socially desirable responding.

Faking bad

Giving answers on a survey (or other self-report measure) that make one look worse than one really is.

Control for

Holding a potential third variable at a constant level while investigating the association between two other variables.

Cross-lag correlation

In a longitudinal design, a correlation between an earlier measure of one variable and a later measure of another variable.

Cross-sectional correlation

In a longitudinal design, a correlation between two variables that are measured at the same time.

Autocorrelation

In a longitudinal design, the correlation of one variable with itself, measured at two different times.

Internal reliability

In a measure that contains several items, the consistency in a pattern of answers, no matter how a question is phrased. Also called internal consistency.

Fence sitting

Playing it safe by answering in the middle of the scale for every question in a survey or interview.

Validity

The appropriateness of a conclusion or decision. See also construct validity, external validity, internal validity, statistical validity.

Test-retest reliability

The consistency in results every time a measure is used.

Reliability

The consistency of the results of a measure.

Parsimony

The degree to which a theory provides the simplest explanation of some phenomenon. In the context of investigating a claim, the simplest explanation of a pattern of data; the best explanation that requires making the fewest exceptions or qualifications.

Interrater reliability

The degree to which two or more coders or observers give consistent ratings of a set of targets.

Content validity

The extent to which a measure captures all parts of a defined construct.

Face validity

The extent to which a measure is subjectively considered a plausible operationalization of the conceptual variable in question.

Sample

The group of people, animals, or cases used in a study; a subset of the population of interest.

Effect size

The magnitude of a relationship between two or more variables.

Simple random sampling

The most basic form of probability sampling, in which the sample is chosen completely at random from the population of interest (e.g., drawing names out of a hat).

Probability sampling

The process of drawing a sample from a population of interest in such a way that each member of the population has an equal chance of being included in the sample, usually via random selection. Also called random sampling.

Observational research

The process of watching people or animals and systematically recording how they behave or what they are doing.

Slope direction

The upward, downward, or neutral slope of the cluster of data points in a scatterplot.

Random assignment

The use of a random method (e.g., flipping a coin) to assign participants into different experimental groups.

Criterion variable

The variable in a multiple-regression analysis that the researchers are most interested in understanding or predicting. Also called dependent variable.

Which of the following would NOT be considered an operational definition of memory?

a cognitive process to retain and restore past information

Josiane has found an online test that claims to measure IQ. It consists of choosing the correct definitions for a series of words. She is concerned that it doesn't include any tests of other things that are part of IQ, such as problem solving or visual-spatial ability. Which type of validity is she questioning?

content validity

Some colleges no longer require the SAT I or the ACT tests, instead basing their admissions on other factors, such as high school GPA. A large reason that they have done this is that they have found a low correlation between the scores on the tests and the students' freshman year GPA. In other words, they were concerned that college entrance exams lacked which type of validity?

criterion validity

Which of the following is an example of a categorical variable?

declared major in college

Sun Mi is designing a questionnaire on loneliness. She is concerned that some features of loneliness are similar to depression and to low self-esteem. What type of validity does she need to show to demonstrate that her questionnaire assesses loneliness and not depression or low self-esteem?

discriminant validity

Dr. Nolan gives his new anxiety measure to a group of his colleagues who are anxiety experts. They agree that the questions on the measure appear to assess anxiety symptoms. This suggests that Dr. Nolan's measure has which of the following types of measurement validity?

face validity

Dr. Kamran studies domestic violence and has designed a self-report scale that is meant to assess men's negative attitudes toward women. To validate her scale, she administers it to two groups of recently incarcerated male prisoners: prisoners convicted of domestic violence and prisoners convicted of other crimes. Dr. Kamran finds a statistically significant difference in the mean scores of the two groups. What technique is Dr. Kamran using to validate her scale?

known-groups paradigm

In a study of aggression in children, a researcher has his undergraduate research assistants watch a group of children on the playground and record the number of instances of physical or verbal attacks. Which category of measured variable is this researcher using?

observational measures

Georgina graduated as valedictorian of her high school class because of her class ranking. What type of scale is used for the quantitative variable of class ranking?

ordinal scale

Which of the following is an example of a physiological measure?

skin conductance

When using a measure to assess a trait that is expected to remain stable over time, a researcher would expect to get consistent results each time the measure is used. This type of reliability is known as which of the following?

test-retest

The Department of Motor Vehicles receives a complaint that some of their employees who administer the road test pass a much higher percentage of test-takers than other employees. In this example, what aspect of the road test is being questioned?

the interrater reliability of the road test

Professor Morgan questions whether the ratings he receives from his students on "teaching effectiveness" indicate how much the students learn in his class or whether they are just a reflection of how much his students like him. What aspect of the ratings is he questioning?

the measurement validity of the ratings

What information can you learn from a scatterplot that you cannot learn from the correlation coefficient?

the values for each pair of measurements