Research Methods Test 3
Mixed ANOVA
A __________ is called for when a mixed factorial design is used.
Self-selection Bias
A bias that occurs because people who feel strongly about a subject are more likely to respond to survey questions than people who feel indifferent about it - typically appear in magazines/ newspapers ex: Hite (1987) survey sent only to select group of women's organizations. While only 4.5% of people responded, still claimed more than 90% of married woman felt emotionally abused in relationships
essentially the same, very different in height
A clue you are dealing with kind of interaction where main effects do not matter is a graph where 3 of the bars (or points on line graph) are ________, and a fourth bar is _________. ex: Caffeine/decaf coffee in morning and afternoon on recall in elderly experiment
pilot study
A good use of open‐ended questions is in a __________ as a way of identifying alternatives for a subsequent questionnaire to be composed of closed items. ex: In your judgment, what are the most important problems facing this university today?---> responses become alternatives later
wording
A major problem in survey construction concerns the ________ of the items.
multilevel independent groups designs, multilevel ex post facto designs
A one-way ANOVA for independent groups is used for ___________ and __________.
multilevel matched groups designs, multilevel repeated-measures designs
A one-way ANOVA for repeated measures is used for __________ and ____________.
listserv, social media site
A second form of online survey is one that can be posted on a _______ or __________, collecting data from those who choose to respond.
incentives
A third procedure for online surveying is to follow a probability sampling procedure with _________ for participating in online surveys. ex: some companies create national samples by randomly sampling addresses rather than phone numbers. They then recruit subjects by regular mail to participate in multiple online surveys by providing the incentive of free Internet access
in‐person in the form of an interview, through the mail, on the phone, online
All data collection for survey data begins with some form of written survey that is delivered ________________, __________, _________, or ________.
sequencing
Along with balancing statements, it is important to be concerned about the _________ of items on your survey. ex: if your research requires that you include questions some respondents might find sensitive (e.g., frequency of alcohol or drug use, sexual activities), put them near the end of the survey.
effect size
Along with t-tests, it is also possible to determine the magnitude of the difference by calculating ________________, usually Cohen's d for two-sample tests.
carry-over effects
An important consideration when making a survey is to avoid creating any ________ across items on a survey that might bias survey results. ex: First asked several questions about ranking of US children's test scores in comparison to rest of world. Because respondents might now be thinking about US test scores, could potentially bias response by selecting choice in favor of eliminating recess to have more time for test prep.
mixed mode approach
As with phone surveying, researchers can use a ____________. ex: sending a letter in the mail that appeals to the reader to complete the survey-including in the letter a website address with a password to enter the survey.
memory errors
Asking how often respondents have done certain things in the past requires one to remember past events, and making the interval too long increases the chances of ________. ex: on a survey about drinking behavior (assume the definition of a "drink" has been made clear to the respondent), it is better to ask participants to specify exactly what they have had to drink in the past week than in the past month.
manipulated independent, subject
Assuming proper control, causal conclusions can be drawn with ___________ variables, but with ____________ variables such conclusions cannot be drawn.
within-subjects
Because identifying interactions by examining whether lines are parallel or not is easier with line graphs than with bar graphs, the guideline about line graphs being used only with ______ factors is sometimes ignored by researchers if the key finding is an interaction.
means, standard deviations
Because numbers correspond to particular response options when using the Likert Scale, responses from all of the participants can be summarized with _____ and _____________. ex: If the mean score on the news anchor question was 4.5, the researcher would conclude that respondents are midway between "agree" and "strongly agree" on the item.
nonprobability, probability
Because samples in many studies are not chosen using a random procedure, the selection procedure is referred to as _________ sampling. However, ___________ sampling is most likely to be used in survey research.
difficult to score, time required
Because they can produce such a wide range of responses, however, open‐ended questions are ________ and can add to the ___________ to complete the survey.
cost, logistics, interviewer bias.
Besides sampling issues, other major problems with the interview approach are _____,________, and _________.
placebo controls, wait list controls, and yoked controls. multilevel experimental
Besides the typical control group situation in which a group is untreated, three other special-purpose control groups are: _______, ______, and ______. These types of control groups are most informative when used in the context of ____________ designs.
systematic variance, error variance
Between-Groups Variance: scores differ because they are from different conditions, (i.e.,_________ ). Within-Groups Variance: scores may differ from each other even when participants are in the same condition (i.e.,__________ ).
Environment factor, Person factor, interaction
Causal conclusions can be drawn if a significant main effect occurs for the manipulated ___________, but they cannot be drawn when a main effect occurs for the subject variable or ____________, and they also cannot be drawn if an ___________ occurs.
interval
Closed questions using an ________ scale can help the researcher better ascertain the intensity of one's attitude or opinion about a particular item. - most common type is Likert Scale ex: "strongly agree" to "strongly disagree"
Type 1 Error
Completing multiple t-tests increases the risk of making a _____________- that is, the more -t-tests you calculate, the greater the chances that one of them will accidentally yield significant differences between conditions. ex: 1-(1-alpha)^c For a study with 4 groups (ABCD), you would run 6 tests (A-B,A-C,A-D, B-C, B-D, C-D). Overall chance of this is 1 - (1- .05)6 = 26.5%!
hindsight
Defenders of control groups point out that _______ is usually perfect. - before the fact= not so obvious that a program will be effective - the only way to tell is to do the study ex: PRIOR to nursing home study, one easily could have predicted that the experimental group subjects would be unnecessarily stressed by the added responsibility of caring for themselves.
new treatment, the most favored current treatment
Defenders of control groups point out that in research evaluating a new treatment or program, the comparison is not between the new treatment and no treatment, it is between the _________ and __________________.
cost money
Defenders of control groups point out that treatments __________, and it is certainly worthwhile to spend the bucks on the best treatments (that cannot be determined without well-designed research).
levels
Determining the main effect of 1 factor involves combining all of the data for each of the _______ of that factor.
unclear questions
During an in-person interview, having an interviewer present reduces the problem of _________; the interviewer can clarify information on the spot.
follow‐up questions, probes
During an in-person survey, even though the interviewer typically asks a standard set of questions, the skilled interviewer is able to elicit considerable information through ____________ or________.
standard deviations, standard errors, confidence intervals
Error bars can be in the form of ___________, ____________, or ___________.
between‐subjects, within-subjects
Factorial designs can be completely ________, meaning all the independent variables are between‐subjects factors, or they can be completely ________, in which all the independent variables are within-subjects.
educational research, effectiveness of psychotherapy
Factorial designs that include both subject variables and manipulated variables are popular in_________ and in research on the ____________.
t-tests, ANOVA
For single-factor designs the interval or ratio scales of measurement are used, then _________ or one-way _________ are calculated. - other techniques required when nominal or ordinal scales used
subject welfare, therapies, relatively brief duration
Giving the wait list control group participants an opportunity to benefit from some therapy procedure provides an important protection for ________, but it also creates pressures on the researcher to use this control procedure only for _______ or programs of ________.
specific predictions, particular pairs of levels, planned comparisons
If the ANOVA does not find any significance, subsequent testing is normally not done, unless _________ about ________________ of the independent variable have been made ahead of time. In this latter case, the testing is not post hoc tests, but referred to as ______________ .
mixed P x E factorial
If the P factor is between‐subjects, and the E factor is within‐subjects, then the P × E is a __________.
F ratio
In a factorial design, more than one ______ will be calculated.
F ratio, F
In a factorial design, more than one ______ will be calculated. Specifically, there will be an ___ for each possible main effect and for each possible interaction. ex: in 2 × 2 design, will be calculated to examine the possibility of a main effect for type of training, another for the main effect of presentation rate, and a third for the potential interaction between the two.
equivalent groups, order effects
In a mixed factorial design, the researcher usually gets to deal with both the problems of _________ and the problems of ___________.
cells
In factorial designs, the term conditions equals the number of _________ in a matrix.
main effects, interactions
In factorial studies, two kinds of results occur: __________ and __________.
nonparallel
In general, if the lines on a graph are _______, then an interaction probably exists.
parallel
In general, if the lines on a graph are ________, then no interaction is present.
irrelevant
In mail written surveys, nonresponse also occurs when people have some attribute that makes the survey _________ for them. ex: vegetarians surveyed about meat preferences
between‐subjects, within‐subjects factor
In most cases the P variable is a __________ factor because it is a subject variable. However, it could be a ___________ if the participants are tested over time, as in a developmental design. (The E variable can also be manipulated as either one of these)
pilot study, interpreted
In order to construct a good survey, a _______ should be conducted to test the instrument on others and see if it is __________ in different ways.
define
In order to construct a good survey, you can _______ any terms that you think could be interpreted in more than one way.
probability sampling
In order to obtain a representative sample of the population, scientists use various _____________ procedures. ex: random sampling, stratified sampling, cluster sampling, etc.
biased
In survey research, it is important for the sample to reflect the attributes of the target population as a whole. If the sample is not representative of the population, then the sample is potentially ____________. ex: Literary Digest presidential election of 1936
Closed
Is this an open-ended or closed question: Do you think state lottery money should be used to finance public education?
Open-ended
Is this an open-ended or closed question: How do you think we should finance public education?
Closed
Is this an open-ended or closed question: Should a person anchoring a news show have at least 5 years of experience as a professional journalist? Yes No
normal distributions of data, homogeneity of variance
Like the t-test, the parameters required for a one-way ANOVA are ___________ and _________. If either or both are violated, alternate nonparametric tests should be used.
questionnaire
Most surveys include various questions (in the form of a ____________) that are delivered online, through the mail, or administered in some more direct fashion (e.g., to a class of students).
statements
Often, survey items are phrased in terms of___________ and respondents are asked to indicate their level of agreement.
type, wording
Once an empirical question is framed and its terms operationally defined, the researcher decides on the _____ of items to use and the _______ of those items. ex: question or statement, open-ended or closed question, etc.
elaborate on their responses
One good use of open-ended questions is to give respondents an opportunity to _________________ to closed questions. ex: At the end, the survey might provide space for respondents to comment on their answers to any of the items, especially those for which they gave extreme ratings.
Social Desirability Bias
One important problem that exists with all forms of survey research is a ____________. ex: People are likely to distort their voting record, reporting they vote more often than they actually do
statements
One way to include DK choices, while encouraging respondents to avoid overusing them, is to disguise knowledge questions by prefacing them with certain ________. ex: "Using your best guess..." or "Have you heard or have you read that..."
e‐mail
Online surveys can be sent as Internet url's via _______ to a selected sample of individuals. ex: can be purchased or, following the ethically dubious lead of spammers, obtained by using "search spiders" that search the Internet
eliciting a wide range of responses, self control
Open‐ended questions can be useful for _____________, (including some not even conceived of by the researchers) and they can also increase the respondent's sense of _________.
random sampling, stratified sampling, cluster sampling
Probability sampling procedures (to make a representative sample) include: ________, __________, and __________.
which conditions differ from which, post hoc
Rejecting the null hypothesis after using a one-way ANOVA does not identify ________________________. To determine precisely which condition is significantly different from another requires subsequent testing or ________ (after the fact) analyses. ex: After ANOVA indicated some significant dif. exists, this would analyze each of the 3 pairs of comparisons level 1-level 2 level 1- level 3 level 2- level 3
sample size, differences
Selecting one of the types of post hoc analyses depends on _________ and how conservative the researcher wishes to be when testing _________ between conditions. ex: Tukey's HSD test, with the HSD standing for "honestly significant difference," is one of the most popular of the post hoc choices, but it requires that there are equal numbers of participants in each level of the independent variable
nominal
Simply asking a question that requires a Yes or No response is a ________ scale of measurement.
subject variable, manipulated independent variable
Some between‐subjects factorials include both a _________ and a ______________. ex: Can be called P x E factorial designs
"Other" category
Some survey items can be "partially" open by including a specific checklist, ending it with an ___________ and allowing respondents to write in their responses. ex: Which of the following is an important problem facing this university today? Check all that apply. ____ overall quality of teaching ____ inadequate computer facilities ____ lack of diversity in the student body ____ poor quality of dormitory rooms ____ inadequate career advising ____ other (please state):
empirical question
Sometimes, the _________ will determine the type of demographic information needed. ex: In a survey about attitudes toward animal experimentation, for example, it might be useful for the researcher to know whether or not the respondent is a pet owner.
DK alternative ("don't know")
Survey items that deal with knowledge often include what is called a __________. ex: Some experts discourage this because respondents might overuse them, choosing this even if they have some knowledge of the issue.
memory, know
Surveys sometimes attempt to assess the respondent's _________ or what they _______. ex: Asking respondents how often they have done certain things in the past (e.g. smoking)
closed
Surveys with _________ questions require a specific response, such as a Yes‐No response.
closed
Surveys with __________ questions often use an interval scale for measuring the responses. ex: Likert Scale
experimental tradition
The _________ in the PxE design is concerned with identifying general laws of behavior that apply to some degree to everyone, regardless of individual differences. ex: Main effect for this indicates situational factor powerful enough to influence the behavior of many kinds of persons
interactive effects
The advantage of factorials over single-factor designs lies in their potential to show ___________.
(A)
The best chance for a decent return rate for a mailed written survey is when: (A) Nonresponse triggers follow‐up reminders, a second mailing of the survey, and perhaps even a phone request to fill out the form (B) The entire package is not highly professional in appearance, without a cover letter, in order to appear less intimidating. (C) Return postage is not included in order to make the respondents feel like they have choice in responding or not.
(D) B and C
The best chance for a decent return rate for a mailed written survey is when: (A) The recipient has reason to think the survey is the first step in a sales pitch. (B) Return postage is included. (C) A small gift or token amount of money is included. (D) B and C
E
The best chance for a decent return rate for a mailed written survey is when: (A) The survey is brief and easy to fill out (B) The form starts with demographic information leaves interesting items at the end (C) Before the survey is sent out, participants are notified that a survey is on the way and that their help will be greatly appreciated. (D) B and C only (E) A and C only
are
The concepts "conditions" and "levels of the IV" *(are/are not)* interchangeable in single factor experiments.
subject variable or P factor
The correlational tradition is associated with the study of individual differences, and the ____________ in the P × E design looks specifically at these differences. ex: main effect for this is two different types of individuals perform differently on whatever behavior is being measured as the dependent variable
matched groups, repeated measures designs
The dependent samples t‐test is used with _______ and ___________.
independent groups, ex post facto designs
The independent samples t‐test is used with __________ and __________ .
attitudes, opinions, beliefs
The interview format for surveying individuals about ________, ________,________, and the like has the advantages of being comprehensive and highly detailed.
a large amount of data can be collected in a relatively short time, be completed in less time
The main advantage of online surveying is that_______________ for minimal cost and online surveys can __________ than other forms of surveys. ex: There are costs for researcher time and surveying software, but usually no postage, phone, or employee costs.
behavior, mental processes, environments
The major purpose of interview and survey research is to provide accurate descriptions of _____________ and ____________ and of the interrelationships among individuals and their __________.
mean scores, variability
The points on a line graph and the tops of the bars on a bar graph are the __________; error bars tell you about the amount of _______ in a set of scores.
mean, chance factors
The t-test examines the difference between the __________ scores for the two samples and determines (with some probability) whether this difference is larger than would be expected by ________________ alone.
independent groups, repeated measures, mixed
The type of design dictates whether the one-way ANOVA will be an ANOVA for _____ or a _________ ANOVA. In the same way, a factorial ANOVA will be one of these types or a third type: a _________ ANOVA.
7, 3
There are ____ factorial designs, ____ of which are unique to factorials.
Robert Woodworth's "Columbia Bible"
This book institutionalized the distinction we routinely make today between independent and dependent variables in experimental research and made distinction between experimental and correlational methods......
carry-over effect
This is an example of _______: Donald Trump received one of his highest poll numbers in support of the Republican nominee. The poll was biased. Just before a survey asked choices for Republican nominees, it asked several questions about illegal immigration. May have been thinking about illegal immigration, they may have also been thinking about Trump (most vocal) prior to the questions about selecting a nominee, potentially biasing their response selections in favor of Trump.
interviewer bias
This represents the in-person interview problem of ______: Cross‐race bias may exist, resulting in systematic differences between interviews with members of the interviewer's own race and members of other races.
cost
This represents the in-person interview problem of ______: Interviewers must be hired and trained, and travel expenses can be substantial.
logistics
This represents the in-person interview problem of ______: Interviews might be restricted to a fairly small geographic area because of the logistical problems of sending interviewers long distances.
double-barreled questions
This represents the survey constructing problem of _____: It is wrong for women in bars to curse and to buy drinks for men who are unknown to them.
leading questions
This represents the survey constructing problem of _____: Given the importance to future generations of preserving the environment, do you believe the Clean Air Act should be strengthened, weakened, or left alone?
ambiguity
This represents the survey constructing problem of _____: Visiting relatives can be fun. Do you agree with this statement?
true
True or False: In ALL experimental designs, the term levels refers to the number of levels of any one IV.
False, although Internet use is widespread, the sample tends to be biased
True or false: Internet use is widespread, and therefore the sample tends to not be biased.
True
True or false: When using a Likert Scale, a 7‐point scale yields 5 points even if people avoid the extremes, but adding the extra level of discrimination can increase the time it takes to complete the survey.
True
True or false: When using the Likert Scale, a 5‐point scale normally provides sufficient discrimination among levels of agreement but might become a de facto 3‐point scale in practice, given the tendency for some people to avoid the ends of the scale.
False, in general, it is a good idea to put questions about demographic information at the end of a survey.
True or false: In general, it is a bad idea to put questions about demographic information at the end of a survey because respondents may not provide the information if they are fatigued.
False, interviews can occur in a group format
True or false: In-person interviews can not occur in a group format. Doing so may bias responses
False, there is NO clear advantage to either a 5‐ or a 7‐ (or more) point scale.
True or false: There is a clear advantage to either a 5‐ or a 7‐ (or more) point scale when using a Likert Scale.
False, one way to aid memory is to provide lists.
True or false: When using a survey to asses respondent's memory or what they know, you should not attempt to aid memory because doing so will bias the results.
False, survey items can be phrased as a question OR as a statement.
True or false: Survey items can ONLY be phrased as a question.
False, when items are balanced like this, it is a good idea to emphasize this fact in the instructions.
True or false: When balancing statements favorably and unfavorably you should not emphasize this fact in the instructions because it will bias the responses.
True
True or false: When using surveys, scores from several items designed to measure the same issue could be combined into a single mean score.
telemarketing, cell phones
Two developments created serious difficulties for the phone surveying business: _________ and ____________.
format
What constitutes a good survey can differ slightly depending on the ________. ex: interview, written, etc.
Independent Groups
What single factor design is this? In a study on the effects of motivation on maze-running time, a researcher puts two groups of rats through a training exercise whereby they learn that there is food at the end of an alley-maze. She then has one group deprived of food for 48 hours before running the two groups through the same maze again. She measures the time it takes for the rats to reach the end of the maze. Rats are assigned to groups (motivated or not) such that mean body weight is the same for the two groups
Ex post facto
What single-factor design is this? A researcher is interested in the effects of political orientation (conservative vs. liberal) on tolerance for ambiguity. She divides participants into conservative and liberal groups based on responses to a questionnaire measuring political orientation, then has each group go through an exercise involving difficult ambiguous situations. She then has them fill out a questionnaire measuring their frustration levels
overburden memory, DK ("don't know")
When a survey attempts to assess memory or what respondent's know, don't ______________ and use __________ alternatives sparingly.
linguistic ambiguity, asking two questions in one item, and asking leading items
When constructing a survey, you should avoid these three specific problems: ______, ________, and _______.
effectively answer the empirical question(s), structure, wording
When designing a survey, the researcher must create items that ______________ and must be very careful with the _______ and the _________ of items.
mixing formats
When making a survey, a general rule is to avoid _____________. ex: If you decide to use a 5‐point Likert scale, use it throughout; don't mix 5‐ and 7‐point scales in the same survey.
household, individual
When the only phones in the house were landlines, surveyors would call and the unit of measurement typically would be the ________. With cell phones, however, the unit is the ___________, thereby changing the nature of the population.
representativeness
When there is self-selection bias, the person reporting the survey will try to impress you with the total number of returns rather than the ______________ of the sample.
favorably, unfavorably
When using a Likert scale, it is normally a good idea to word some of the statements ________ and others _________. ex: The instructor was seldom able to hold my attention during class lectures. The instructor wrote exams that fairly tested the material covered. The instructor often appeared unprepared for class. The instructor was available for extra help outside of class.
question being asked
When using a survey to asses respondent's memory or what they know, the proper interval will depend on the __________. ex: If the activity is a relatively infrequent one (e.g., visiting zoos), a short interval will yield too many scores of zero.
sampling
When using an in-person survey, ___________ is sometimes a problem because sizable segments of the population may not be included if they refuse to be interviewed, cannot be located, or live in an area the interviewer would prefer to avoid. ex: the poor and homeless are usually underrepresented in national surveys using the interview format
(A) Do you support or oppose the use of animals in undergraduate laboratory courses? (because you should use balanced items, not those favoring one position or another)
When wording this item on a survey, which choice is better and why: (A) Do you support or oppose the use of animals in undergraduate laboratory courses? (B) Do you support the use of animals in undergraduate laboratory courses?
(B) East Carolina University (you should avoid most abbreviations- some might think ECU is "East Colorado University")
When wording this item on a survey, which choice is better and why: (A) ECU (B) East Carolina University
(A) Should the university have the right to search your dorm room? (Because you should avoid negatively phrased questions; negative statements are more difficult to process than positive ones)
When wording this item on a survey, which choice is better and why: (A) Should the university have the right to search your dorm room? (B) Do you believe the university should not have the right to search your dorm?
(B) When a player shoots the ball and it bounces off the backboard and into the hoop... (because you should avoid jargon- a basketball player might know what "bank shot" means, but other responders may not)
When wording this item on a survey, which choice is better and why: (A) When a player makes a bank shot... (B) When a player shoots the ball and it bounces off the backboard and into the hoop...
(B) what bread type do you like the best? (because you should avoid slang and colloquial expressions- they go out of fashion)
When wording this item on a survey, which choice is better and why: (A) whose your bread bae? (B) what bread type do you like the best?
(B) Doctors should use simple and clear language when explaining a diagnosis to a patient. (simplicity)
When wording this item on a survey, which choice is better and why: (A) Doctors should be intelligible when explaining a diagnosis to a patient. (B) Doctors should use simple and clear language when explaining a diagnosis to a patient.
(B) What is the name of the city where you currently live? (because this is a complete sentence)
When wording this item on a survey, which choice is better and why: (A) Place of residence? (B) What is the name of the city where you currently live?
nonparallel, ANOVA
Whether an interaction exists, (whether the lines are sufficiently _______) is a statistical decision, to be determined by an ________.
self‐selected
With web‐ based surveys, the problem is that the sample is bound to be _________, resulting in bias. ex: it could be that a person out there has little else to do but respond to your survey several hundred times a day for a week or two
how many people actually return a completed survey, differ in some important way
Written surveys sent through the mail may have problems with ___________, and people who do return surveys ____________ from those who don't return them.
cluster
You should ________ items on the same general topic in the same place on the survey. ex: if your survey is about satisfaction with college, group the items about courses together, and separate them from items about student activities.
ANOVAs
________ are usually the analysis of choice for factorial designs.
Surveys
__________ tend to assess attitudes, beliefs, opinions, and projected behaviors.
Psychological tests
________________ can include questionnaires, but such tests are used in more formal assessments of psychological functioning, usually in a clinical setting. - undergone reliability/validity tests ex: Beck Depression Inventory-II
Ensuring survey participants of anonymity
______________________ can help reduce the social desirability bias, but the problem is persistent and it is hard to gauge its extent.
Response Acquiescence
a response set in which a participant tends to agree with statements - avoid this problem by balancing favorable and unfavorable statements
Factorial Matrix
a row and column arrangement that characterizes a factorial design and shows the independent variables, the levels of each independent variable, and the total number of conditions (cells) in the study
Repeated Measures Design
a single factor design - minimum 2 levels of IV - within subjects - manipulated IV - no equivalent groups ex: Last-minute cramming (LC) vs. distributed studying (DS), which is best? Participants study two different texts (equated for difficulty), one after the other. One text is studied by LC, the other by DS. Order of conditions must be counterbalanced to help control for practice/fatigue effects.
Ex post facto Design
a single factor design between subjects with at least two groups of participants that uses a subject variable or that creates nonequivalent groups; subjects placed in groups "after the fact" of their already existing subject characteristics - make the groups as similar with reference to other variables - minimum 2 levels for IV - between subjects - subject IV - matching may increase equivalence ex: Last-minute cramming (LC) vs. distributed studying (DS), which is best? Instead of assigning a study style to people at random, we look at self-described "crammers" vs."distributers". Give subjects text and tell them to use their own preferred strategy. No random assignment, so can't be sure what is causing any differences we see. Test after a week of using own strategy.
Independent Groups Design
a single factor design: - minimum 2 levels for IV - between subjects - manipulated IV - create equivalent groups by random assignment ex: Last-minute cramming (LC) vs. distributed studying (DS), which is best? 60 participants, divided into two groups of 30 by random assignment. Each group studies a text, but one group crams, other does distributed study. Both groups then tested on knowledge of text
Matched Groups Design
a single factor design; subjects are matched on some variable assumed to affect the outcome before being randomly assigned to the groups - minimum 2 levels for IV - between subjects - manipulated IV - create equivalent groups by matching ex: Last-minute cramming (LC) vs. distributed studying (DS), which is best? We can only get 10 participants. Random assignment will not work, so create two groups based on matching via GPA. One group crams, other does distributed study, then both are tested.
ANOVA Source Table
a standardized method for displaying the results of an analysis of variance; includes sources of variance, sums of squares, degrees of freedom, mean squares (variance), F ratios, and probability values
Phone Surveying
a survey method in which the researcher asks questions over the phone ex: Households could be selected through random-digit dialing
Interview Survey
a survey method in which the researcher interviews the participant face to face; allows for more in-depth surveying (e.g. follow-up questions and clarifications) ex: Kinsey Report resulted from this survey type
Closed Question
a type of question found on surveys that can be answered yes or no or by marking a point on a scale
Open-ended Question
a type of question found on surveys that requires a narrative response rather than a yes or no answer
Social Desirability Bias
a type of response bias in survey research; occurs when people respond to a question by trying to put themselves in a favorable light ex: People report they wash their hands after using the bathroom more often than they actually do
Single-factor Design
an experimental design with a single IV; the 4 designs of this result from decisions about IV under investigation - independent groups - matched groups - ex post facto - repeated measures ex: ALL of these designs must have minimum of 2 levels for IV
Single-factor Multilevel Designs
an experimental design with a single independent variable and more than two levels of the IV - enable researcher to discover nonlinear effects - most single-factor studies use this - can test for specific alternative hypothesis and perhaps rule them out while supporting the researcher's hypothesis ex: Bransford/Johnson's recall test had single IV (context of paragraph) 3 levels: no topic, topic before, and topic after
Parametric Test
an inferential statistical analysis based on a set of assumptions about the population ex: Some tests assume your data in level of IV approximates a normal distribution
Nonlinear Effects
an outcome that does not form a straight line when graphed; can only occur when the IV has more than two levels ex: Yerkes-Dodson arousal law testing low arousal, moderate arousal, and high arousal
Factorial Design
any experimental design with more than one independent variable - usually have 2 or 3 factors ex: Type of training (imagery or rote) and Presentation rate (2 sec or 4 sec) on memory recall
Dependent Samples (Paired) t-test
compares the means of two related groups to determine whether there is a statistically significant difference between these means. Use if... - IV is a within-subjects factor - two groups of people are formed that relationship exists between them - matched groups or repeated measures design ex: Measured performance of 10 participants in a spelling test (the dependent variable) before and after they underwent a new form of computerised teaching method to improve spelling. You would like to know if the computer training improved their spelling performance. The first related group consists of the participants at the beginning (prior to) the computerised spell training and the second related group consists of the same participants, but now at the end of the computerised training.
Wait List Control Groups
control group in which participants aren't yet receiving treatment but will eventually; used to ensure that those in the experimental and control groups are similar (e.g. all seeking treatment for the same problem) - common in research assessing effectiveness of program/psychotherapy ex: Evaluated the effectiveness of two forms of therapy nightmares. They wanted to include a no‐ treatment control, but to ensure clients in all three groups were generally equivalent, the control group subjects also had to be nightmare sufferers.
Placebo Control Group
control group in which some participants believe they are receiving the experimental treatment, but they are not. ex: A group given a drink that seems to be alcoholic (and cannot be distinguished in taste from the true alcoholic drink) but is not.
Yoked Control Group
control group in which the treatment given to a member of the control group is matched exactly with the treatment given to a member of the experimental control group - used when each subject in the experimental group participates for varying amounts of time or is subjected to different types of events in the study - keeps time spent participating among groups as a whole constant ex: Control group was yoked in terms of how long the session lasted, so if a subject in the EMDR group took 25 minutes to reach a SUD level of 0-1, a subject in the yoked control group would participate in the control procedures for 25 minutes. The control group did everything the experimental group did, but instead of the eye movements, they focused on a nonmoving red dot.
Demographic Information
data that classifies or identifies individuals ex: gender, age, income, etc.
Simple Effects Analysis
following an ANOVA, a follow-up test to a significant interaction, comparing individual cells ex: 1. For science majors, compare lab emphasis (mean of 80) with lecture emphasis (70) 2. For humanities majors, compare lab emphasis (70) with lecture emphasis (80) 3. For the lab emphasis, compare science (80) with humanities majors (70) 4. For the lecture emphasis, compare science (70) with humanities majors (80)
ATI (Aptitude-Treatment Interaction) Designs
form of PxE factorial design found in educational research, the goal of which is to examine possible interactions between an aptitude variable (person factor) and a treatment variable (environmental factor) - usually when aptitude or person factor is a learning style variable and the treatment or environmental factor is some aspect of instruction
Within Subjects Multilevel Designs
going beyond 2 levels in this design makes all counterbalancing options available - each condition tested once per subject= full/partial counterbalancing available - each condition tested several times per subject= reverse/blocked randomization available ex: Steele et al. questioning "Mozart effect" had all 36 participants (within) tested in each of 3 conditions (Mozart, rainfall, quiet) by randomly assigning with Latin square
Equivalent Groups
groups of participants in a between subjects design that are essentially equal in all ways except levels of the IV
Person (P)
in a P x E Factorial Designs (person by environment), this is defined as some subject variable
Environment (E)
in a P x E Factorial Designs (person by environment), this is defined broadly to include any manipulated IV
Interaction
in a factorial design, occurs when the effect of 1 IV depends on the level of another IV - sometimes render main effects irrelevant ex: Science students did better in lab and humanity students did better in lecture. Effect on 1 variable (course type) depended on level of other variable (major).
Main Effect
in a factorial design, this is used to describe the overall effect of a single IV - difference btw the means of the levels of any one IV ex: In 2x2 factorial, there can be at most 2 of these
Leading question
in a survey, a question asked in such a way that the answer desired by the questioner is clear - occurs frequently in business/ politics ex: Given the fact that installing scrubbers at utility plants could increase electricity bills by 25%, do you believe the Clean Air Act should be strengthened, weakened, or left alone?
Double-barreled question
in a survey, a question that asks or states two different things in a single item ex: It is wrong for a teacher to use corporal punishment and let students eat unhealthy snacks in the classroom.
Placebo
in medicine, an inert substance said to have medicinal effect; in research, a condition in which subjects believe a treatment is effective, but it is not. ex:In drug research, for example, patients will sometimes show improvement when given a placebo but told it is a real drug, simply because they believe the drug will make them better.
Mixed-mode approach
increase the chances of people responding, one technique used by legitimate phone surveyors is to precede the call with a brief letter, post card, or e‐mail alerting the respondent that a phone call is on the way - mixing mail and phone
Inter-rater Reliability
indicates how consistent scores are likely to be if the responses are scored by two or more raters using the same item, scale, or instrument ex: In longhand v. laptop note taking, scoring conceptual questions answered by participants was to some extent subjective, so used this to score reliably.
Nonresponse bias
occurs in survey research when those who return surveys differ systematically from those who don't respond ex: In 1936 election survey, only wealthy Republicans who wanted opinion to be known responded
Homogeneity of Variance
one of the conditions that should be in effect in order to perform parametric inferential tests such as a t test or ANOVA; refers to the fact that variability among all the conditions of a study ought to be similar - if tests for this indicate violation, nonparametric tests can be used ex: Standard deviation for one group is significantly larger than standard deviation for another group, there may be violation of assumption of ___________.
Table
one way to present results of research study where you construct _________ of results. - usually means/standard deviations for each condition are presented here - preferred when data points so numerous graph would be uninterpretable or inform reader of precise values of means/standard deviation
Graph
one way to present results of research study; present data in the form of a ________. - an experimental study DV is always on Y axis and IV on X axis - becomes more complicated when more than one IV used - good if large differences to report, non linear effects, or result is interaction btw 2 factors - bar or line
Sentence
one way to present results of research study; presenting numbers in __________ form. - might be fine when results from studies with 2 or 3 levels - tedious reading as amount of levels increases ex: "The average number of digits correctly recalled was virtually identical for all three conditions: 18.53 (SD = 4.14) for Mozart, 18.50 (SD = 6.07) for the gentle rain, and 18.72 (SD = 5.09) for the control condition."
F ratio
one-way ANOVA yields this; examines the extent to which the obtained mean differences could be due to chance or are the result of some other factor (presumably the IV) ex: typically portrayed in ANOVA source table
Subliminal Self-help
pseudoscientific idea that states these are detected by your unconscious mind and lead you to make changes that will improve your life in some fashion - research shows any positive effects due to what people expect to happen ex: listening to this will help you lose weight
P x E Factorial Designs (person by environment)
some between‐subjects factorials include both a subject variable and a manipulated independent variable, because these designs can yield an interaction between the type of person in the study and the situation or environment created in the study, they can be called this
Online Surveys
survey research conducted over the Internet; can be a survey sent via e-mail or posted on a website or social media site
Independent Samples t-test
test that helps you compare whether two groups have different means. Use if... - random assignment to create equivalent groups - variable is subject variable involving 2 dif. groups - independent groups or ex post facto design ex: whether men and women have different average heights
one-way ANOVA (ANalysis Of VAriance)
the most common inferential statistical tool for analyzing the results of experiments; avoids problem of multiple t-tests - tests for presences of overall significant effect that could exist somewhere between the levels of the IV - yields F score or F ratio - tests null hypothesis Assumptions • Independent random samples • Interval / ratio level data (i.e. parametric) • Normally distributed • Homogeneity of Variance ex: In study with 3 levels, the null hypothesis is "level 1= level 2= level 3"
Likert Scale
the most common type of interval scale used for surveys; typical one has 5-9 distinct points (always an odd #- there must be a "neutral") with each point reflecting score on continuum ex: The person anchoring a news show should have at least 5 years of experience as a professional journalist. 1: strongly disagree 2: somewhat disagree 3: neutral 4: somewhat agree 5: strongly agree
Error Variance
the total variance in participants' responses that remains unaccounted for after systematic variance due to the independent variable being removed - due to random factors or individual differences - inferential statistic decision to reject or not reject H0 depends on this
Bar Graph
use this graph if IV is manipulated between-subjects or is a subject variable - levels of IV represent separate groups of individuals so the data in the graph should best reflect separate groups (separate bars) ex: use this graph to report levels of anxiety between (subject variable)
Line Graph
use this graph if the IV is a within-subjects manipulated/ subject variable - participants are experiencing all levels of that IV so the data should "connect" in more continuous way - points usually represent means of each condition and error bars on each point ex: Use this to graph anxiety levels in experiment where all 36 participants got each condition (within, subject)
Inferential Statistics
used to draw conclusions about the broader population on the basis of a study using a sample of that population; determines whether differences found between conditions of single-factor design are significant or due to chance - depends on systematic variance and error variance - ideal: variability btw conditions is LARGE and variability within each condition is SMALL - parametric or nonparametric variability btw conditions (systematic+error) ----------------------------- variability within each condition (error)
Systematic Variance
variability that can be attributed to an identifiable source, either the systematic variation of the independent variable or the uncontrolled variation of a confound - inferential statistic decision to reject or not reject H0 depends on this - generally measures as the difference between groups (e.g. comparing the means of a set of samples) - due to deliberate experimental actions ex: the 'after' score may be different to the 'before' score, the 'control' score or some segmentation of subjects (eg. male/female).
Mixed Factorial Design
when a mixture of between and within subjects factors exist in the same experiment - at least 1 variable must be tested between subjects - at least 1 variable must be tested within- subjects
Sugging (Selling Under the Guise of a survey)
when telemarketers begin the call by pretending they are conducting a survey, when in fact they are selling a product
