PSY 302 Research Questions and Designs Module 5
Observational measures
(also known as behavioral measures): Measures in which the behavior of participants is observed.
Four main types of validity used to assess the quality of a research
1.Internal validity 2.Construct validity 3.External validity 4.Statistical validity
Value
A characteristic, a measure, or an amount of a variable.
Correlation
A correspondence or relationship between variables such that a change in one variable coexists with a change in the other variable.
Random assignment
A process that guarantees every participant has an equal chance of being assigned to any of the groups in a study.
Causal relationship
A relation such that a change in one variable brings about a change in another variable.
Case study method
A type of descriptive research that involves a detailed and in-depth study of one or a few individuals.
Temporal precedence
The order in which variables occur in time, such that one variable occurs before the other.
Confounds
Also known as plausible alternative variables; Something else that changes in an experimental study along with the independent variable.
Elimination of alternative explanations
Ascertaining that no other variable is responsible for the outcome found in a study; that is, no variable other than the independent variable produced the change in the dependent variable.
Mill's Criteria
Covariance, Temporal Precedence, and Alternative Explanations
Manipulated variables
Experimental research; Verbs such as "assigned," "instructed," "presented," or "delayed".
Control condition
In an experiment, the condition to which participants are exposed in which no manipulation of a variable takes place.
Experimental condition
In an experiment, the condition to which participants are exposed in which the variable of interest is manipulated; also called the treatment condition.
Self-report measures
Measures in which participants themselves report the value of a variable, typically in a questionnaire or interview
Observational measures (Behavioral measures)
Measures in which the behavior of participants is observed.
Archival Measures
Measures in which the researcher consults official records to measure status variables. relates to both the sample and conditions of the study, specifically the degree to which the sample and conditions represent the population and situation of interest
Physiological measures:
Measures in which the researcher uses instruments to assess variations within the body.
Why would a researcher use quasi-experimental design?
Quasi-experimental designs are useful when it is impossible, impractical, or unethical to conduct an experiment using random assignment to groups and manipulation of the independent variable.
Experimental research
Research design in which an independent variable is manipulated and a dependent variable is measured, while other variables are held constant. Random assignment must be used to ensure all groups begin as equal.
Correlational research
Research design in which the relationships between predictor and outcome variables are observed without any control over the conditions in which these relationships occur.
Descriptive research
Research design that assesses frequencies, averages, and other measures of central tendency of variables.
Quasi-experimental research
Research design that resembles experimental design, except that it is impossible for researchers to randomly assign participants different conditions.
Limitations
Section in a journal article that often discusses validity issues present within the study
Internal validity
The degree to which an experiment is free from flaws in its internal structure; the degree to which its methodology meets the criteria for establishing causality.
Internal Validity
The degree to which an experiment is free from flaws in its internal structure; the degree to which its methodology meets the criteria for establishing causality. Essential. Ask yourself: Does the methodology of this study allow you to answer your research question? For example, if you are interested in how young children approach story problems, but your sample pool includes adolescents, then your study does not have _________ validity. Or, if you allow participants to self-select which group of the independent variable they will join, pre-existing biases may influence your results.
Statistical validity
The degree to which conclusions drawn from statistical tests are accurate and reliable.
Validity
The degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of conclusions drawn from some form of assessment. A ______ ________ is a strong, trustworthy claim.
External validity
The degree to which the results of research can be generalized beyond the sample to other individuals, situations, and time periods.
Covariance
The degree to which two variables vary together (i.e., they are related, associated, or correlated).
Independent variable
The manipulated variable in an experimental design that is believed to influence the dependent variable.
Dependent variable
The measured variable in an experimental research design that is believed to be dependent on, or influenced by, the independent variable.
Measure
To assess the value of the variable using an appropriate method (i.e., observation, self-report, physiological monitoring, or archival research)
Manipulate
To intentionally change (a variable).
Three Basic Research Questions
Value, Causation, and Association
A researcher compares the emotional intelligence scores of a group of kindergarten students attending a large, urban public school with the scores of a group of kindergarten students attending a small, rural public school. The groups were matched for variables such as gender, socioeconomic status, and number of parents living in the household. The scores indicate that the students in the small, rural school have statistically higher levels of emotional intelligence compared to the students in the large, urban school. Can you conclude that attending a small, rural versus a large, urban school caused the difference in emotional intelligence? a. No, this was a quasi-experimental research design and there may be alternative explanations for the difference in scores. b. Yes, because there are two levels of the independent variable. c. Yes, this was an experimental research design and you can make a causal conclusion. d. No, this was a descriptive research design which does not allow you to make a causal conclusion.
a
A researcher wanted to examine the relationship between the number of assignments due per week and anxiety levels in college students. The number of assignments due was assessed through viewing various course syllabi due dates. The measurement for anxiety was a standardized anxiety test that was known to be an accurate measure of anxiety. This study would have high _____ validity. a. construct b. internal c. statistical d. external
a
Elena measured minutes of exercise per day and hours of deep sleep per night for a group of 10 subjects. Because Elena was not sure which hypothesis test to run, she tried several different analyses until one showed significant results. Elena's study has low _____ validity. a. statistical b. internal c. construct d. external
a
If we design a study where only the independent variable differs between the experimental and control groups then we have achieved which of Mill's criteria? a. Eliminate alternative variables b. Temporal precedence c. Large effect size d. Covariance
a
If you ask the question "Do the results of the study apply to the population of interest?" then you are concerned with which type of validity? a. External b. Internal c. Statistical d. Construct
a
In order to eliminate alternative explanations, you should a. conduct an experiment under highly controlled conditions. b. only measure the outcome of one dependent variable. c. use random sampling. d. use an experimental or a quasi-experimental research design.
a
The careful observation and documentation of a behavior or phenomenon is referred to as which type of descriptive research method? a. Observational method b. Quasi-experimental method c. Survey method d. Case study method
a
Which of these claims can be made from a study using a quasi-experimental design? a. A claim about the associative relationship between variables b. A claim about the value of variables c. No claim can be made from this study d. A claim about the causal relationship between variables
a
operational definition
a description of something in terms of the procedures by which it is manipulated or measured.
claim
a statement, assertion, or conclusion about the value of a variable or a relationship between variables produced through careful examination of a research question; it may be used to generalize the results of a study to the larger population and other situations, places, and times.
Associative relationship
a type of relationship such that when one variable changes, the other variable changes
Which of these elements are included in quasi-experimental designs? Select all that apply. a. Measurement of a dependent variable b. An independent variable c. Inclusion of true experimental controls d. Creation of equal groups by random assignment
a, b
Which of these conditions is(are) required for a researcher to make a valid and appropriate claim about an experimental result? Select all that apply. a. There is temporal precedence between the variables. b. There is covariance between the variables. c. Theory and empirical evidence support the conclusions. d. Alternative explanations have been eliminated.
a, b, c, d
Which of Mill's criteria must be met to make a causal claim? Select all that apply. a. Temporal precedence b. Large effect size c. Eliminate alternative variables d. Covariance
a, c, d
Why would a researcher use a quasi-experimental design rather than an experimental design? Select all that apply. a. To collect data in the real world where controls are difficult b. When the independent variable is easily confounded c. When random assignment is not possible d. When manipulation of the independent variable is not possible or ethical
a, c, d
Dr. Prenovost wants to determine whether team cohesion is related to success on the playing field. She works with 20 high school football teams in her region. Ten of the teams are randomly assigned to attend a 3-hour workshop on building team cohesion. The other 10 teams attend a 3-hour skills-based session. She then counts wins and losses over the regular football season. What type of claim can be made from the design of this study? a. A claim about the value of variables b. A claim about the causal relationship between variables c. A claim about the associative relationship between variables d. no claim can be made from this study
b
If we design a study where the measure of the independent variable is related to the measure of the dependent variable, then we have achieved which of Mill's criteria? a. Eliminate alternative variables b. Covariance c. Large effect size d. Temporal precedence
b
To assess mental health in college students, 60 participants completed questionnaires to measure symptoms of anxiety and depression. A strong, positive relationship was found, such that as anxiety symptoms increased, depression symptoms increased as well. Which of Mill's criteria have been met in this study? Select all that apply. a. Eliminate alternative variables b. Covariance c. Large effect size d. Temporal precedence
b
When conducting experimental research, which variable is manipulated? a. Dependent variable b. Independent variable c. Participant variable d. Alternative variable
b
How is a quasi-experiment different from an experiment? Select all that apply. a. A dependent variable is not used in quasi-experimental designs. b. The independent variable is not always manipulated in quasi-experimental designs. c. Groups are not equivalent in quasi-experimental designs. d. An independent variable is not used in quasi-experimental designs.
b,
How are quasi-experimental designs different from correlational designs? Select all that apply. a. Correlational designs do not use random assignment of the variables. b. Quasi-experimental designs may attempt to create equivalent groups. c. Quasi-experimental designs may attempt to manipulate the independent variable. d. Correlational designs cannot be used to make causal claims.
b, c
Which of these research questions is about an associative relationship? Select all that apply. a. What percentage of men in Europe aged 18 to 25 have traveled outside country? b. Does sleeping 8 hours a night predict the level of a person's happiness? c. Does eating a diet high in fiber decrease a person's chance of getting cancer? d. Are high school students' GPAs predictive of their future success in college?
b, d
A researcher operationally defined level of aggression as the number of aggressive acts a child exhibited on the playground in a 15-minute period: 0-1 acts = low aggression; 2-4 acts = medium aggression; and 5 or more or more acts = high aggression. Based on this operational definition, level of aggression is a _____ variable. a. participant b. confounding c. measured d. manipulated
c
A researcher used a convenience sample of 30 undergraduate students from a small liberal arts college to examine the relationship between sleep and memory. Because this is not a random sample of the population of adults in the United States, these results may not generalize to all adults in the United States. This study has low _____ validity Yes, that's correct! a. internal b. construct c. external d. statistical
c
Dr. Morley wanted to know whether using an electronic sleep-promoting device before going to bed has an influence on the amount of time necessary to fall asleep compared to reading a book before bed. Fifty participants were randomly assigned to either the electronic device group or book group and time to fall asleep was measured for both groups. Because only the independent variable was manipulated and there was random assignment to groups, any difference in time to fall asleep should be due to the activity prior to falling asleep. This study has high _____ validity. a. external b. construct c. internal d. statistical
c
Dr. Zhu wonders if anxiety is related to weight change. She recruits participants and measures their weight in September and then again 1 year later. She also counts the number of life stressors (e.g., move, losses) that each has experienced over the year. From these data, Dr. Zhu calculates the change in weight from the beginning to the end of the study. What type of claim can be made from the design of this study? a. A claim about the value of variables b. no claim can be made from this study c. A claim about the associative relationship between variables d. A claim about the causal relationship between variables
c
Dr. Hopkins was interested in finding out if there was a relationship between time spent reading course textbooks per week and living arrangement—on campus versus off campus. An equal number of participants are sampled in the on-campus and off-campus groups and each participant records the number of minutes spent studying each week night. Which of Mill's criteria have been met in this study? Select all that apply. a. Eliminate alternative variables b. Temporal precedence c. Covariance d. Large effect size
c, b
Which of these research questions is about the value of a variable? Select all that apply. a. Does eating a diet high in fiber decrease a person's chance of getting cancer? b. Does sleeping 8 hours a night predict the level of a person's happiness? c. What percentage of men in Europe aged 18 to 25 have traveled outside the continent? d. What proportion of students had a GPA of 3.0 or above?
c, d
survey method
consists of collecting data through the administration of questionnaires or interviews.
Collecting data through the administration of a questionnaire or interview is referred to as which type of descriptive research method? a. Case study method b. Experimental method c. Observational method d. Survey method
d
Dr. Abukhazneh wonders if fidgeting is related to anxiety. He randomly divides his participants into two groups and gives one group a test that has no correct answers, which makes the test seem extremely difficult and provokes high anxiety. The other group is given a test with obvious answers, which makes the test seem very easy and does not provoke anxiety. While the participants are taking the test, Dr. Abukhazneh counts how long they each spend engaged in fidgeting behavior such as tapping their feet and hands or playing with their hair. What type of claim can be made from the design of this study? a. no claim can be made from this study b. A claim about the associative relationship between variables c. A claim about the value of variables d. A claim about the causal relationship between variables
d
Dr. Nissen also wants to determine whether team cohesion is related to success on the playing field. He observes high school football teams during practice and counts the number of times team members congratulate each other during play. He splits the teams into two groups: teams that show high cohesion and teams that show low cohesion. The final groups are matched in terms of geographic location and median income for families within the school district boundaries. Dr. Nissen then counts wins and losses over the regular football season for each group. What type of claim can be made from the design of this study? a. no claim can be made from this study b. A claim about the causal relationship between variables c. A claim about the value of variables d. A claim about the associative relationship between variables
d
If you ask the question "Does the methodology of this study allow you to answer your causal research question?" then you are concerned with which type of validity? a. Statistical b. Construct c. External d. Internal
d
The most appropriate research design is determined by the a. size of the population. b. number and level of measurement for each variable. c. research claim. d. research question.
d
To test the influence of sugar consumption on perceived energy levels, 100 participants were randomly assigned to either consume a sugar snack or a sugar-free alternative. Thirty minutes later, the participants completed a short questionnaire to determine perceived energy level. This is a/n _____ research design, which allows the researcher to make an _____ claim. a. descriptive; values b. correlational; association c. correlational; causal d. experimental; causal
d
Which of these methods is not a type of descriptive research? a. Observational method b. Survey method c. Case study method d. Experimental method
d
Which of these research questions is about a causal relationship? a. What percentage of men in Europe aged 18 to 25 have traveled outside country? b. Are high school students' GPAs predictive of their future success in college? c. Does sleeping 8 hours a night predict the level of a person's happiness? d. Does eating a diet high in fiber decrease a person's chance of getting cancer?
d
Measured variables
descriptive and correlational research; Verbs such as "measured," "surveyed," "tabulated," "recorded," or "assessed".
Types of Research Designs
descriptive, correlational, experimental, quasi-experimental
outcome variable
expresses the degree of association
relates to both the sample and conditions of the study, specifically the degree to which the sample and conditions represent the population and situation of interest; sampling and conditions
external validity
Descriptive research designs...
help researchers to better understand a topic, population, or phenomenon by offering descriptive information about one variable.
participant variables
individual differences among participants like race, age, gender, intelligence, or disposition.
relates to the degree to which an experiment is free from flaws in its internal structure; the degree to which its methodology meets the criteria for establishing causality; manipulating variables
internal validity
Observational method
involves the close observation and documentation of a behavior or phenomenon. This method can be used in the natural world, such as observing and describing interactions that take place on the street between pedestrians and panhandlers; or in the laboratory, such as observing and describing interactions between study participants assigned to roles.
Why would a researcher utilize a descriptive research design?
it may provide a picture of a particular group at a particular time, which may help researchers improve hypotheses and build a foundation for later correlational or experimental research.
Three major types of descriptive research
observational method, case study method, survey method
Construct Validity
the degree to which a test or instrument is capable of measuring a concept, trait, or other theoretical entity. always important because study claims rely on operational definitions that are precise and accurately reflect the variables being studied. Essential.
descriptive research
the researcher gathers data (e.g., attitudes or behaviors) to describe the occurrence and the qualities of a variable; only one variable
experimental research
the researcher observes the relationships between variables under controlled conditions.
predictor variable
used to predict the outcome
Why would a researcher use correlational design?
useful when it is not practical, possible, or ethical to conduct an experiment. For example, if you wanted to examine the relationship between being breastfed as a baby and being healthy as an adult, it is not possible to randomly assign adult participants to have been breastfed or not breastfed.
eliminate alternative explanations
we must be able to say that no other variable is responsible for the relationship we have identified. That is, we have ruled out all other possible explanations and are confident that the effect of the independent variable on the dependent variable has been isolated
quasi-experimental designs
when researchers cannot obtain results using true experimental methods; Specifically, quasi-experimental designs are useful when it is impractical, impossible, or unethical to randomly assign participants to conditions or to manipulate the predictor variable. They might not start with equal groups. They might not manipulate the independent variable.
