RESEARCH METHODS PARTIAL 2
alternative-form reliability
A correlation between a measure and a similar, but different, measure that assesses the same variable. From the text: If the researchers had two scales assessing self-concept clarity, they could measure the correlation between the scores on the scales. A higher r value would indicate greater reliability.
equivalent-form reliability
A correlation between a measure and different measure that assesses the same variable. From the text: If the researchers had two scales assessing self-concept, they could measure the correlation between the scores on the scales. A higher r value would indicate greater reliability.
multi group design
A design that involves more than two groups of participants. From the text: The experimenters have three temperature conditions, cold, normal temperature and hot.
empty control group
A group of participants who complete the dependent measure without receiving any treatment. From the text: In the climate change study, the participants in the room that is normal temperature serve as an empty control group.
nondirectional hypothesis
A hypothesis that does not predict the nature of the relationship between variables. From the text: There is a relationship between self-concept clarity and attitude toward Greek organizations.
directional hypothesis
A hypothesis that makes a specific prediction about the nature of the relationship between two variables. From the text: There will be a negative correlation between self-concept clarity and attitude toward Greek organizations.
INTRA-observer reliability
A measure of the extent to which an observer codes a variable the same way each time they observe it. example: A researcher has research assistants view and code the same behavioral data multiple times to assess the extent to which they code the same behavior the same way each time. Consistent coding indicates greater reliability.
INTER-observer reliability
A measure of the extent to which different observers code the same behavior the same way. example: A researcher has two different research assistants view and code the same behavioral data and assesses the extent to which they code the same behavior in the same way. Consistent coding indicates greater reliability.
interval recording
A method for assessing the frequency of a behavior that involves splitting a session into equal time periods and noting whether a behavior was observed during each of the time period. example: A research has one hour of observation and breaks it down into 12 intervals of 5 minutes each and indicates whether aggressive behavior was observed in each of the intervals.
reverse-coding
A method for coding scale items that assigns values in a consistent way regardless of how the question was worded. From the text: For the item "Belonging to a fraternity or sorority will help me meet others" the researchers will code responses such that "strongly agree" gets the most points whereas for the item "Greek life is a trivial part of the college experience," researchers will code responses such that "strongly disagree" gets the most points.
statistical hypothesis testing
A method for determining how likely the results were to have occurred by chance given the prediction of the researcher. From the text: The researchers test to see if the correlation between self-concept clarity and attitude toward Greek organization occurred by chance.
scale
A method for quantifying the extent to which an individual possesses the variable of interest. From the text: The researchers use the self-concept clarity scale to determine the self-concept clarity score for each participant in the study.
naturalistic observation
A method of collecting data where the researcher observes and records behaviors in an unobtrusive way. example: A researcher pretends to be reading in a park while they secretly record parents interacting with their children.
blind observation
A method of collecting observational data where the people recording the behavioral data are not aware of the research question. example: Research assistants are trained to record the number of times they observe each of five disciplining behaviors on a playground, and the number of people present on the playground, without knowing that the researcher expects there to be variation in the behavior as a function of the number of people present.
central tendency
A number that represents all the values in the data set. example: The most frequently used discipline measure (the mode) will reflect central tendency.
correlation
A quantification of the linear relationship between two variables represented by a correlation coefficient, r, which can range from -1.0 to 1.0. From the text: The researchers found a positive correlation between self-concept and attitude toward Greek organizations, r = .64.
nonlinear (or functional) relationships
A relationship between two variables that cannot be plotted with a straight line. From the text: It is possible that concern for global warming may increase as a result of being in a hot or normal condition but not when in cold condition, or it is possible that concern for global warming will increase in a hot condition but be similar in a normal or cold condition. Both of these cases would show that temperature and concern for global warming are related, but not linearly related.
correlational study
A research approach which assesses the extent to which one variable changes when another variable changes. From the text: The researchers are interested in seeing the extent to which changes in self-concept clarity ratings are associated with changes in interest in joining a fraternity or sorority.
survey
A research strategy which collects quantifiable data from a sample and then generalizes the results to the larger population. From the text: The researchers administer surveys to a sample of undergraduate students to assess if self-concept clarity and interest in joining a fraternity or sorority are related, and then generalize their findings to the whole population of undergraduates.
frequency-count recording
A researchers counts how many times a given behavior is observed. example: A researcher counts how many times a parent performs a disciplining behavior (e.g., how many threats, how many bribes) in response to a single transgression made by the child.
area probability sampling
A sampling method based on geography, where the experiment identifies geographical boundaries and then randomly samples people from each of the geographical areas. From the text: The researchers will sample in proportion to the size of the different residence halls their population is living in.
Likert scale
A scale that includes questions and a predetermined set of responses that are then summed to create an overall score. From the text: In the self-concept clarity survey participants respond to each question with one of the predetermined responses, and then the researchers sum the responses to determine a self-concept clarity score.
summated ratings score
A scale which includes questions and a predetermined set of responses that are then summed to create an overall score. From the text: In the self-concept survey participants respond to each question with one of the predetermined responses, and then the researchers sum the responses to determine a self-concept score.
effect size
A standardized measure to represent the magnitude of the differences between the groups on the dependent variable. text example: Ignoring texts had a moderate size effect on test performance, as measured by effect size d. example: A researcher finds a large effect size when comparing test scores between those who studied by highlighting and took notes on passages while reading, versus those who simply read.
standard deviation
A statistic that indicates the average variation between a score and the arithmetic mean. From the text: The researchers determine that on average, Greek organization survey scores vary by 8.00 points around the mean of 41.70 points.
Cohen's kappa coefficient
A statistic that reflects the extent to which the scores of two raters are consistent with one another. Scores can range from 0 to 1, higher scores indicate greater agreement. example: A researcher computes the Cohen's kappa coefficient for two raters in their study and finds a score of .88. This score indicates good inter-rater reliability.
Cronbach's alpha
A statistic used to evaluate a scale's consistency. From the text: For the Greek organization scale, researchers found Cronbach's alpha to be .82. This suggests that the scale has good internal reliability.
one-way analysis of variance (one-way ANOVA)
A statistical test that determines whether variation in the dependent variable is significantly different from variation expected by chance. From the text: The researchers conduct a one-way ANOVA to determine whether the groups differed in the self-report measure related to global warming more than would be expected by chance as well as an ANOVA to determine if the group varied in the probability that they would sign the petition more than would be expected by chance.
chi-square test of independence
A statistical test used when all variables are categorical. The test determines whether the group differences exceed what would be expected by chance. From the text: The text example has categorical dependent variables (the self-report data and whether a participant signed the petition) therefore a chi-square test for independence can be conducted.
frequency distribution
A summary of data where the category of the variable is shown on the x-axis and the frequency of a variable is shown on the y-axis. example: A researcher plots five types of discipline on the x-axis and then plots the frequency of each disciplining behavior on the y-axis.
continuous variable
A variable that can have an unlimited number of values between two end points. example: Response time is an example of a continuous variable because it can have an infinite number of values between 0 and infinity.
confound
A variable that varies along with the independent variable. From the text: If the hot group used a tanning bed while the cold group stood in a large air-conditioned room, some confounds would include lying down versus standing up and being in a closed space versus an open space. If an effect of temperature on climate change attitude was found, the researchers could not be sure what caused this: the temperature, how the person was positioned (laying down or standing up), or whether or not they were in a confined space.
categorical variable
A variable which groups data into distinct groups. example: The variable "type of discipline" with five levels (bribe, reasoning, physical punishment, threats, or ignoring the behavior) is a categorical variable.
bar chart
A visual representation of data on a given area of study example: For instance, researchers may collect frequency data from customers entering a fast food restaurant each month and plot the frequency data on a bar chart for visual representation.
mean (M)
AVERAGE A measure of central tendency where the data are summed and divided by the number of data in the sample. example: A researcher takes data reflecting the number of hours slept each night for a week and sums the hours, then divides by seven to get the mean number of hours slept each night for that week.
placebo group
An experimental group whose participants falsely believed they are receiving the experimental treatment. From the text: In the global warming experiment, participants are not aware that the temperature of the room is the treatment and therefore it is not necessary to have the placebo group. If the experimenters were studying the effects of a drug, or other obvious manipulation of the independent variable, then a placebo group would be necessary.
extraneous variable
Any factor other than the independent variable that may be influencing participants' responses in the study. text example: The study will be conducted in a laboratory rather than a classroom setting to minimize other influences on participants' responses. example: In an experiment on whether images accompanying an emotional message can influence subsequent persuasion, all participants listen to the same audio recording of the message. This way the study controls extraneous variables such as voice inflection and rate of speech.
exploratory analysis
Any statistical test that is not planned before the experiment begins. From the text: The researchers do not think about which comparison (hot versus cold, hot versus normal, normal versus cold) is the most meaningful before the experiment, so they conduct an exploratory analysis to see which groups seem to be different.
temporal precedence
Evidence that changes in the proposed causal variable occur before changes in the outcome variable and not vice versa. text example: Need to show that ignoring text messaging leads to distraction, not vice versa. example: Does high self-esteem lead to success or does success lead to high self-esteem? To establish causality, the researcher needs to establish which comes first.
experimental control
Experimental control involves the researcher either holding constant, or accounting for, factors other than the variable that is under investigation. text example: The study used a strict protocol and was conducted in a laboratory setting so that the two groups only vary systematically on the type of instructions they were given. For example, the number and timing of the texts each participant received was pre-planned and consistent for all participants. example: In a study on the effects of random versus predictable bursts of noise on test performance, a researcher holds constant, during the study, the duration of the noise bursts, the volume of the noise bursts, and eliminates any other potential sources of noise in the laboratory environment.
experimental realism
How engaged participants are in the study so that their reactions are natural in response to the manipulation. text example: To simulate a classroom setting, participants will be asked to take a seat at a desk in a room where there are already five other students sitting at desks. example: To examine the effects that an annoying partner can have on task persistence, a participant believes his or her partner is another participant. In actuality, the partner is an accomplice of the experimenter who is instructed to be intentionally annoying.
distractor items
Items included on a scale to help camouflage the research hypothesis. From the text: Including items related to student government, intramural sports and academic clubs, in addition to questions about Greek organizations so that participants do not become aware that the researchers are specifically interested in Greek organizations.
median (Mdn)
MIDDLE A measure of central tendency which indicate the score found at the midpoint of the distribution of scores. example: A researcher arranges their data in ascending order and determines the value of the data point in the midpoint of this arrangement. This provides the median score.
mode (Mo)
MOST COMMON A measure of central tendency that reflects the most commonly observed data point. example: A researcher asks 100 people their favorite color and then looks to the most common response to assess the most popular favorite color.
matched-pair design
Participants are first grouped on the basis of a characteristic believed to be related to the outcome variable. Pairs of participants who are similar on this characteristic are then randomly assigned to the experimental or control group. text example: Each participant could have first been paired with another participant on the basis of some related factor such as how often they typically text. Using random assignment, one member of each pair would be randomly assigned to one group and the other participant to the remaining condition. example: To minimize potential differences in terms of academic ability, a researcher first pairs participants based on their current GPA and major. For each pair, he randomly assigns one participant to the experimental group. The other member of the pair is assigned to the control group.
planned contrasts
Statistical tests that are chosen before the experiment begins. From the text: The researcher could plan ahead of time to conduct a test to see if there is a difference between the hot-temperature group and the cold- temperature group.
post-hoc tests
Statistical tests that are not planned before an experiment is conducted. These tests are more conservative than planned comparisons because knowing an experimental result introduces bias. From the text: The researcher finds a significant effect of temperature of self-report attitude toward global warming and then conducts a post-hoc test to see if there were differences between each pair of groups (cold and normal temperature, normal temperature and hot, hot and cold temperature).
descriptive statistics
Statistics that summarize numerical data. example: A researcher computes the proportion of disciplining events that fall into five predetermined categories as a function of different numbers of spectators in the setting.
predictive validity
The degree to which a measurement is able to predict a future outcome. From the text: The correlation between scores on the Greek organizations scale and the decision to join a Greek organization in the future.
criterion validity
The degree to which a quantitative measure correlates to an observable behavior. From the text: People who score high on the Greek organization attitude scale should be more likely to join a Greek organization than those who score low on the scale.
internal validity
The degree to which one can rule out plausible, rival hypotheses that could also account for the observed results. text example: A strict protocol will be used to keep everything else constant to help eliminate other possible explanations for the study's results. example: In a study examining whether low self-esteem can influence how funny participants find racially-insensitive jokes, a researcher has participants watch a YouTube clip of a White or African American comedian. After the study is over, she realizes that there is a potential threat to internal validity as not only did the race of the comedian vary in the study, but also the quality of their jokes.
construct validity
The degree to which the scale measures the construct. This is assessed by looking at convergent and divergent validity. From the text: The extent to which the scale measures attitudes towards Greek organization.
discriminant validity
The extent to which a scale does not correlate with another scale that measures a different variable. From the text: The researchers are interested in making sure their Greek organization scale does not correlate with unrelated variables, such as interest in attending church.
concurrent validity
The extent to which a scale predicts a current behavioral outcome. From the text: The correlation between scores on the Greek organization scale and how many people from Greek organizations a participant speaks to at an information session held after the scale is administered.
power (P)
The extent to which a study can detect differences in the dependent variable that result from the experimental manipulation. From the text: The experimenters will use extreme differences in temperature in the different conditions to maximize the probability that a difference, if it exists, will be detected.
covariation
The extent to which changes in one variable are associated with changes in another variable. text example: The more students try to ignore their texts, then the more distracted they should become. example: The more knowledgeable one is about exercise, the more physically fit the person is. Knowledge and fitness covary.
variability
The extent to which each measurement of a variable is different from each other measurement of the variable. From the text: The extent to which scores on the Greek organization survey differ from one another.
internal consistency reliability
The extent to which responses to individual items on the scale are correlated with one another. From the text: The extent to which individual items on the self-concept clarity survey are correlated, and the extent to which individual items on the Greek organization survey are correlated.
content validity
The extent to which scale items reflect the material that should be used to measure a variable. From the text: The extent to which the items on the scale reflect different aspects of the Greek organization.
convergent validity
The extent to which scores on a scale correlate with scores from other scales assessing the same measure. From the text: If the Greek organization survey measures attitudes toward extracurricular activities, then scores from this survey should correlate with attitudes toward being involved in campus life in general.
face validity
The extent to which the elements of a scale appear to measure what they are intended to measure. From the text: The extent to which the items on the self-concept clarity scale appear to measure self-concept clarity.
external validity
The extent to which the findings from a research study can be used to predict behavior in other settings. example: The researcher is interested in how people behave in public settings. The behavior observed in a public park is likely to be similar to behavior exhibited in other public settings giving this method high external validity. The behavior observed in a laboratory observation is not as likely to be similar to behavior exhibited in public settings, giving this method lower external validity.
test-retest reliability
The extent to which the same result is obtained when the measurement is repeated. From the text: If the self-concept clarity survey produces the same measure of self-concept clarity for individuals each time they take the survey.
mundane realism
The extent to which the study reflects or resembles everyday situations occurring the "real world." text example: Participants will be asked to watch a video and read a short narrative to simulate the typical types of experiences they would have in a classroom. example: A researcher uses a driving simulator to examine the impact that different types of distraction can have on how well a person drives.
two-group design (also called a simple experiment)
The most basic type of experimental design as it involves only two groups: a treatment group and a comparison group. text example: There are two groups in this study: a "High-Restriction" group (the experimental group) and a "Low-Restriction" group (the control group). example: A researcher wishes to test the effects of time pressure on problem solving. One group is given five minutes to solve 10 anagrams (low time-pressure condition). The other group is given only two minutes (high time-pressure condition).
observation schedule
The paper-and-pencil or electronic data collection sheet that a researcher uses to note the behaviors they have observed. example: The place where the researcher records the particulars of the behaviors they have observed. This may be in paper form or electronic form.
response rate
The percentage of those who were asked to complete the survey who actually complete the survey. From the text: The number of people who are asked to complete the self-concept clarity and Greek organization survey who complete the survey and return it to the experimenters.
coding system
The researcher's method for classifying observed behaviors. example: A researcher decides that all discipline will be classified into one of five types; bribing, reasoning, physical punishment, threats, or ignoring of behavior.
independence
The responses of each participant are not influenced by nor influence other participants' responses. text example: Participants will be run individually rather than in groups so that their behavior is not affected by other participants. example: A researcher decides to run several participants at the same time to be more efficient in her data collection. She realizes that the participants are subtly looking at each other's reactions during the course of the study. Unfortunately, her study is probably flawed, as the participants' responses to the experimental stimuli are no longer independent from other participants.
researcher notes
These are a written record of any unusual or unexpected occurrences that happened while running research participants. text example: The researcher will record anything unusual occurring during the study, such as the number of unplanned texts the participant receives or whether the participant starts texting during the study. example: Researcher notes can be invaluable for evaluating the possibility of hypothesis guessing during the experiment. During debriefing, the researcher will record in his or her notes what the participant thought the purpose of the study was before revealing the true purpose.
experimental group
This group typically receives the treatment of interest in an experimental design. text example: This group is the "High-Restriction/ Forbidden Phone" Group. example: In a study on the impact of encouragement on task persistence, the researcher regularly tells participants in the experimental group that, "they can do it," and "keep up doing a great job!" This is the treatment group in the study.
random assignment
This is a method of assigning participants to experimental conditions in a nonsystematic, nonbiased manner. Each participant has the same likelihood of being assigned to any of the experimental conditions or groups. text example: Participants are randomly assigned either to either the High-Restriction or Low-Restriction conditions using a random numbers table. example: A researcher flips a coin to assign participants to either the experimental or control condition. If the coin comes up "heads," the participant is put into the experimental condition. If it is tails, the participant is in the control condition.
manipulation check
This is a strategy for evaluating how effectively the manipulation was in creating differences between the levels of the independent variable. text example: Participants will be asked how permissible it was for them to use their cell phone during the experiments using a 1-7 scale. example: A researcher has participants fill out a self-esteem measure immediately after they have received positive or negative feedback regarding their test performance.
null hypothesis
This is the hypothesis of no difference and usually the focus of a statistical test. text example: The null hypothesis being used for the statistical analysis is that those in the highly restricted cell phone use group will NOT have worse test performance than those in the group who are allowed to check their phone. example: In a study on whether distraction hinders studying, a researcher is trying to statistically reject the null hypothesis that distraction does not have an effect on test performance.
experimental hypothesis
This is the research hypothesis or prediction being tested in one's study. text example: For this study, the experimental hypothesis was that those in the highly restricted cell phone use group will have worse test performance than those in the group who are allowed to check their phone. This is a directional hypothesis. example: A researcher conducts an experiment testing the hypothesis that restaurant customers will tip more if the server gives them an unexpected treat.
control group
This is typically the comparison group used to evaluate causal claims in an experimental design. In an "all or nothing" approach to experimental design, the control group is not administered the treatment at all. In a "little more versus little less" approach, the comparison group receives a weaker or different version of the treatment being investigated. text example: This is the "Low Restriction/ Phone Checker" Group. example: A researcher wants to know if listening to music while studying improves test performance. The control group would be the group that studies in silence or with no music playing.
t-test for independent means (also called an independent samples t-test)
This statistical test compares the means of two different groups to evaluate the probability that the difference between the two is simply due to chance alone text example: The mean quiz score for the High-Restriction group will be compared to the mean quiz score for the Low-Restriction group to see if they statistically differ. example: A researcher compares the mean number of mistakes participants identify while proofreading an essay while listening to music compared to while proofreading in silence.
error of central tendency
When a participant chooses responses in the middle of the scale, intentionally avoiding extreme responses. From the text: A participant intentionally avoids choosing "strongly disagree" or "strongly agree" when completing the Greek scale.
response set
When a participant chooses the same response to most items on a scale regardless of whether they reflect their actual beliefs or feelings. From the text: A participant chooses "Strongly disagrees" as a response for every survey item.
acquiescent response set
When a participant chooses to agree with most items on a scale, regardless of how they actually feel. From the text: A participant chooses "Strongly agree" to each of the response options.
closed-ended question
When a participant is asked a question and given a choice of possible responses. From the text: A participant is asked a question relating to self-concept clarity and must choose whether they strongly disagree, disagree, neither agree nor disagree, agree, or strongly agree.
evaluation apprehension
When a participant is worried about what the experimenter will think about their responses or behaviors during an experiment. From the text: Participants may determine the purpose of the study and may choose answers on the scale that they think will please the experimenter.
forced choice scale
When a participant must select from two response alternatives. From the text: A true/false question uses a forced choice scale.
reactivity
When a participant's behavior is affected by the fact that they are being observed. example: A parent decides not to use physical punishment to discipline their child because they know their actions are being recorded by an experimenter.
open-ended question
When a question is posed in such a way that the respondent can choose any answer they wish. From the text: "How clear is your self-concept?"
ecological validity
When a research study invokes the same psychological experiences that people have in real world settings. example: A researcher conducting laboratory observation sets up the laboratory to mimic a typical preschool playground, with a small slide, balls, a sandbox, and a climber in order to maximize ecological validity.
participant observation
When a researcher becomes part of a group in order to study their behavior. example: A researcher joins a playgroup with their young child in order to observe how parents discipline their children in front of others.
continuous recording
When a researcher codes all the behavior exhibited by an individual during an observation period. example: A researcher follows one parent-child dyad during a two-hour observation window and records every behavior that the pair engage in during that time.
experimental research
When a researcher collects data that can explain how one variable causes changes in another variable. example: A researcher measures response time in sober participants, gives participants different amounts of alcohol, and then measures response time again to see if alcohol consumption causes a change in response time.
descriptive research
When a researcher collects data to depict a phenomenon example: Conducting observational research to describe how disciplinary measures differ when parents engage in diverse ways in various social settings.
concealed observation
When a researcher collects observational data without informing the participants that they are being observed. example: A researcher wears a hidden camera and records parents and children interacting at a public park.
systematic observational research
When a researcher determines which behaviors they are going to record and then watches individuals noting when they perform the behaviors. example: A researcher decides that they want to observe parents disciplining their children in five different ways. They then observe parents interacting with children and note when each type of discipline is used.
laboratory observation
When a researcher has participants come into a laboratory and engage in an activity which is observed by the researcher. example: A researcher asks parents of three-year-olds to come into their laboratory for a study. Parents and their children then interact with each other for a period of time. The researcher typically records the interactions and then codes the behavior at a later date.
contrived observation
When a researcher orchestrates the presentation of a variable and then observes the outcome. From the text: A researcher has a clown making balloons at the park suddenly stop making balloons in order to spark tantrums and create more opportunities for parents to use discipline.
duration recording
When a researcher records how much time an individual spent on a given behavior. example: A researcher records how long a parent spent disciplining their child.
pilot testing
When a researcher tries out a method to see if it works before collecting experimental data. Sometimes methods are adjusted as a result of pilot testing. example: A researcher conducts a pilot test to see if their coding system for disciplining behaviors covers the full range of disciplinary measures made by parents.
methodological pluralism
When an experimenter gathers both self-report and behavioral data to study a phenomenon. From the text: The researchers will give participants the option to sign the "Cool Earth" petition (which is a behavioral measure) and the researchers will have the participants complete a "Societal Concerns Survey" (which is a self-report measure).
statistically significant
When it is unlikely that the observed result is the result of random error. From the text: The researchers found a correlation between self-concept clarity and attitude toward Greek organizations that was .64. This was deemed statistically significantly because there was less than a .001 chance that this correlation occurred because of random variation.
non concealed observation
When participants are aware that they are being observed by a researcher. example: Parents go into a laboratory where they are told that they will be observed as they interact with their child.
hypothesis-guessing
When the participant tries to figure out the purpose of the research. From the text: The researchers must include questions that are not related to global warming on the questionnaire in order to prevent hypothesis-guessing by the participants. The researcher may ask the participant to guess the hypothesis during debriefing in order to determine if their data may have been affected by their conclusion about the study's hypothesis.
