Research Methods Chapters 8-10

Explain what we mean by "error" in statistics?

"Error" in statistics = measure of variability due to unpredictable factors. Basically, how much in your results is likely due to other things in your environment/setting/sample.

What are the 4 types of validity?

- Predictive validity - Construct validity - External validity - Internal validity

Explain how methods of population sampling can threaten measure reliability?

1) Number of observations is important, and we want more observations (rather than less). It's rare to measure behaviour in an entire population, so instead you recruit a subgroup of the population (called sampling). They are likely representative of the population of interest, but smaller and easier to work with. You want to have enough observations for reliable data... so greater number of observations leads to higher confidence that your sample reflects population characteristics overall. 2) Want to avoid biases, so want each member of a population to have an equal chance of being sampled. Previous work shows that unbiased random selection guarantees that sample represents the population (not perfect, but statistically more likely). Hence you want to use random sampling.

What are 2 threats to external validity? Explain.

1) Replication: can observations be reproduced under different circumstances. Ex: does a task applied in college students also ring true for mid-life adults? 2) Issues with the experimental setting, meaning that a task applied may not be representative of "normal" behavior seen outside of a lab setting. Thus, we need to double check by applying same task in several settings to see if the results are upheld. So basically, you are confirming that this is an important psychological process that is independent from setting. Ex: rats can be conditioned in a Skinner box AND in their natural environment.

Explain 2 ways to minimize construct invalidity.

1) Using operational definitions, since it's the recipe for specifying how to produce and measure a construct. Basically, you are carefully specifying concepts and defining how it is to be measured. Thus, you are clear about what you are attempting to measure and someone with a different opinion at least knows how you perceive the topic at hand. 2) Proving a detailed protocol, which is specification of how measurements/procedures taken. This does the same thing as providing operational definitions, but additionally offers clarity as to how things were measured and limitations. OVERALL, both aid by reducing random error!

If I have a 2x2 study design, how many possible combinations of the variables do I have? Explain.

4 possible combinations= 2 IV, and each of them has 2 levels.

What is a mixed-design study and why would we adopt this model?

1+ between-subjects IVs AND 1+ within-subjects IVs. It's the marriage between between and within designs, so you can control for potential confounds in the more comprehensive way possible.

Explain why some items tend to be more stable than others in psychological research? Give examples to support your statements.

1. Some things are intrinsically more stable... Ex: Height. You are likely to easily see potential errors when doing these measurements (ex: someone wearing heels). 2. Other measures are a little trickier and less obviously reveal potential sources of error. Ex: Intelligence. How stable is it? Does it vary throughout life? Does it vary within a couple of days?

Give 2 ways that you can reduce variability in your measures?

1. Taking measurements under same controlled conditions. (Ex: always measure cortisol at 9am) 2. Use sophisticated equipment to get reliable measures. (Ex: computer to record reaction times)

What are 3 ways to test the reliability of an assessment and/or tool?

1. Test-retest reliability. · Give the same test twice in succession over a short time interval. You obtain a coefficient of correlation between the 2 scores from a large sample of individuals. A high, positive correlation indicates high reliability. 2. Parallel forms. · Give alternate forms of the test on 2 testing occasions. If correlations between scores on the 2 versions are high, there is high reliability. This format minimizes practice effects. 3. Split-half reliability. · Give a single test to all participants. You then divide test items into 2 arbitrary groups and correlate scores from both halves. If you have high correlation between the groups of questions/answers, then you have a reliable test. This method also allows to establish equivalence of test items.

What is the Sleeper Effect and describe how a complex study by Pratkanis (1988) helped shed light on this phenomenon (vs. if attempted as a simple study)?

2 X 2 factorial design: 2 IVs each with 2 levels discounting cue presented a) before or b) after the persuasive message and 2. length of delay of a) 0 or b) 6 weeks between the message and opinion rating. 4 combinations of interest: cue before message and no delay, cue after message and no delay, cue before message and delay, cue after message and delay. For multifactorial designs, you will calculate 2 things: A) Main effects indicate each of the IV separately. This mirrors the type of stats results you would have received if you ran a simple study with only 1 IV. Results indicated: No effect of cue presentation order (before or after the message) Significant effect of time, with delayed cues leading to more persuasion of the message B) Interaction indicates the true relationships between the study IVs. Here it shows that that the main effects are INCORRECT. Results indicates that there is no sleeper effect then the cue is presented prior to the message, BUT there is when the cue is presented after message. THEREFORE: If you had run a simple study with only 1 IV investigated at a time, you would have missed the reality that it's a combination of cue presentation order AND time delays that leads to a sleeper effect. So, the interaction study was more accurate.

Explain the volunteer issue in research, outlining the advantages and disadvantages.

Advantage: people who volunteer are easy to obtain, often have intrinsic motivation to follow through the study (less attrition). Disadvantage: They volunteered which leads to a selection bias and this decreases the randomness of the sampling.

What is randomization and how does it increase the equivalency between groups?

Assign participants, at random, to different groups/task conditions. (draw assignment out of a hat, roll a dice for each participant (even # one group, odd # another), computer randomization programs, etc... ). Therefore, each participant has an equal and unbiased opportunity to be in any condition. So, group assignment less likely to be plagued by bias and independent characteristics that could decrease internal validity. (NB: Weiss used randomization of groups to improve on the Brady executive monkey study.)

Why do we have to be careful about the erroneous assumption that a control condition means that the group is not getting any level of an IV of interest? Support your statement with an example.

Being a control means you are likely receiving a baseline that would be expected for your group. There may not be a viable/ethical option that leads you to have none of the IV. Ex: drug trials have control groups on the typically available therapies (and not necessarily on placebo). Ex: atmospheric pressure study has a control group with normal atmospheric pressure (not necessarily that of 0) since that is what most people experience, and thus results are more generalizable. Ex: a control group may engage in a more "benign" task between your experimental conditions, as to minimize practice and/or repetition effects.

Explain the GPS reliability study by Kantowitz (1997) that used a mixed design approach, describing the between and within conditions and levels, participant incentive, control variables, results and discussion.

Between-subjects IV: inaccuracy of traffic information (71 vs 43 %) · ALL participants also complete a 100% correct session. Within-subjects IV: location (familiar city or fictional city) Reward: given $15 for selecting the fastest route and penalized if encounter heavy traffic and delays. Control variables: Both cities are topographically matched. All participants get a 100% accurate session. Results: Interaction between the 2 IVs (GPS accuracy and City familiarity). · Inaccurate information = penalty cost was high regardless of city familiarity. · Accurate information = worse performance in their home city. Discussion: In unfamiliar, follow GPS advice but in familiar, follow own judgment.

What are the 2 typical types of experiment designs? Briefly explain what differentiates them.

Between-subjects design: independent groups (2 or more) receive different levels of the IV. Within-subjects design: all subjects receive all levels of the IV.

What is the main concern that threatens internal validity for within-subjects designs? How can we try to minimize these effects?

Carryover effects are a problem, where the effects of one treatment may carry over to another treatment. Can attempt to minimize by randomizing the task condition order (see Weiss LSD rat study).

How do you choose which variable(s) to match your groups on and how may this still fall short in increasing the internal validity of the study?

Chosen on basis of most likely confounding variables. Issues: Impossible to match for everything AND may not know what characteristics need to be matched.

Why can't we use the same stress tasks in rats and humans?

Classic triggers (ex: noise) work well on lower animals, but not significantly well on upper-order animals. Applying stress tasks in humans is tricky since have "knowledge" that they are protected in labs, so cortisol levels don't spike with same stimulus types. So instead, you must appeal to psyche to induce stress reactivity, like uncontrollability and being judged by others. (Cognitive tasks like arithmetic and public speaking cause statistically significant cortisol release.)

Looking at preliminary data, you see that there's a significant confound variable in your study. Unfortunately, you didn't match your groups on this factor, and you've already started data collection (so you don't want to start over). How can you address the potential effect of this confound variable?

Collect this data and then use these factors as a statistical control mechanism. In other words, apply factors like age, sex, etc... as covariates when run analyses. (Could be referred to as a quasi-independent variable control.)

What is the difference between complete counterbalancing and incomplete counterbalancing. Why would you select one over the other?

Complete is impossible with more study conditions. You are attempting to get every possible permutations considered (123, 132, 231, 213, 231, 312, 321) Incomplete counterbalancing has each treatment occurring equally often in each portion of experiment. It uses a Latin-square design to help with this assignment and there are statistical tests to determine if partial counterbalancing was successful. NOTE: Complete counterbalancing is the safest technique (if possible...), but you select based on your needs and feasibility.

What is a meta-analysis and why are they useful?

Comprehensive literature review on a topic of interest. They are useful for comparing results of controlled experiments and can help uncover discrepancies in the literature body.

What is counterbalancing and how it is particularly useful in within-subject designs?

Counterbalancing is changing the order in which subjects are exposed to the different task conditions. So not everybody starts and ends with the same tasks. Therefore, each treatment has same chance of confound variable influences and you can control the other factors of concern (ex: sessions occur at same time, regardless of which condition is administered).

Which of the different interactions is the most convincing form (highest internal validity)?

Crossover interaction= most convincing form of interaction since it cannot be explained by problems in measuring and scaling of the dependent variable.

What is the gear-switching model?

Cueing techniques with longer time intervals give more time to prepare, thereby better performance. In other words, by giving you more advanced notice, you have more preparation to answer, and therefore provide more accurate answers and show faster reaction times. (NB: with short cue intervals, you see decreased accuracy and slower reaction times)

What is construct validity? Provide a tangible example.

Degree to which independent and dependent variables measure what they are intended to. Refers to whether a scale or test measures the construct (the topic you are probing) adequately, aka if the test/intervention is measuring what you are intending to measure. Ex: A doctor is testing the effectiveness of painkillers on chronic back sufferers. Every day, he asks the test subjects to rate their pain level on a scale of one to ten. We know that pain exists, but we can be concerned that this subjective measure is actually probing other things like discomfort or anxiety. So, you might have participants also take a validated pain questionnaire and the Beck anxiety questionnaire and see if they both align, don't align, etc...

What is a multifactorial experiment?

Design that has more than one IV. You are emphasizing the interaction between IVs in a single study.

What is psychophysics?

Determining psychophysical scales to measure psychological reactions to physical events. Basically, they are determinations that link internal impressions and the outside world. Psychophysicists can measure psychological attributes of physical events (e.g., loudness).

What is a theory that attempts to explain the Sleeper Effect? Explain.

Dissociation hypothesis At first, persuasive message and discounting cue cancel each other out (they are link in immediate memory). Yet, due to a natural memory fall-out with time, there is weakening of the cue information itself. Yet, you remember the persuasive message because it was marked as emotionally salient at encoding when initially paired with that discounting cue.

What is random sampling?

Each member of a population has an equal chance of being in the sample.

What is a within-subject design? Why is this design generally considered more efficient and what does that mean for the required sample size?

Each person is assigned to all conditions of the study design. Generally, more efficient = as performance of each subject is compared across all conditions. Because of decreased group variability, a smaller sample size is acceptable.

Why is it important to follow a developmental process from simple experiments to complex experiments?

Earlier work determines the basic factors that will be essential in crafting a more complex analysis. Hence, need the simple, naturalistic and/or correlational studies to precede complex experiments.

What is a stratified sample? Give an example.

Entirely randomized samples are expensive and time consuming. Method to minimize costs and time investment is use of a stratified sample. This divides the population into smaller units and then apply random sampling in those smaller units. THUS: you sample according to a combination of units comprising all these different regions/facets in a statistically relevant and representative way. Ex: Want to study menstruation duration in American undergraduate students. Impossible to go to all universities and interview all student 1 year. Instead, find relative ratios: 40% of undergraduates in Northeast, 10% in Midwest, 20% in Eastern, and so forth. Then Then go in and stratify the sample by region, according to those representative statistics.

Discuss the major fundamental issue with the Executive Monkeys study (other than the obvious ethical concerns) using conclusions from subsequent research.

Executive monkeys results were believed true for some time BUT subsequent laboratory experiments showed greater number of ulcers in helpless animals. Why did they find this? Selected the "executives" due to higher baseline response rates. So this is a critical design flaw to put high-response monkeys (so higher stressed monkeys) in experimental condition only (executives). Since an individual characteristic influenced the effects of the independent variable, study ≠ valid.

What is external validity? How does size of sampled population fit into this measure?

Extent that one can generalize from research setting and participants to other settings and populations. Large populations usually lead to strong external validity sine you have more observations to build your findings (vs limited observations that offer weak external validity by being increasingly swayed by individual factors in your sample).

What type of validity is typically higher in field studies compared to laboratory experiments?

External validity

Explain how a meta-analysis can help approach problem related to external validity?

External validity is the ability to generalize findings to a larger sample/setting. Research review can synthesize theories and establish relationships among studies. If a given relationship is the same many studies and under many conditions, one can be more certain that observation is real (especially with large effect sizes). Ex: See the same/similar results between lab and natural setting studies.

What is Weber's law?

For a particular sensory modality, the size of the difference threshold relative to the standard stimulus is a constant. (Basically assessing how different must 2 stimuli be to be reliably distinguished?) Difference threshold: 1/2 the average distance in a quality (ex: weight) between stimuli that observer can distinguish.

What is the control condition in experimental research and why is it useful?

Group not receiving levels of interest of the independent variable. So, provides a baseline against which a variable of experiment can be compared. Control group gets the baseline variable, and the experimental group(s) gets a/some level(s) of the IV(s)

What are 2 ways a graph can represent an interaction between 2 IVs in a factorial study?

In these cases, the lines are not parallel: A) lines are not parallel but at an angle (shaped like a V; point interaction (points may touch, but not necessarily). B) the lines cross each other (shaped like an X; called a crossover interaction)

Describe the Brady et al.'s (1958) experiments of the "executive monkeys"; listing the hypothesis, IV, DV, methods and conclusions.

Hypothesis: Stress leads to the development of ulcers, being prominent in high-powered executives. Part 1: Tested monkeys learning curve to press a button to avoid an electric shock. Based on results, fastest responders are assigned as "Executives" in part 2 of study. Part 2: "Executive" and "Co-worker", both placed in stressful situation with periodic shocks over a long period of time. Executive can press button to delay shock BUT Co-worker has no control over shocks. Results: Executives developed severe ulcers and died BUT Co-workers did not. Discussion: Assumed that ulcer development was due to stress of management.

Explain why doing the same study as a between and within design can increase generalizability of the findings?

If you see it presented the same way in both designs, you feel more confident in the findings. BUT, previous work often shows different results when applying different techniques... so it's important to do this in order to better understand the reality of the data and not just the issue with the choice of design.

What is matching when creating equivalent groups? Give an example how we could have approached the Brady study with this technique to increase the internal validity of the results?

Important subject characteristics are matched in various treatment conditions. Brady's study: Match executive and co-worker based on reaction time similarities in 1st part of study. So have 1 executive with a high reaction time in part 1, paired with a co-worker with a similarly high reaction time. This would have eliminated the "ulcer predisposition" and the study would have been valid.

Why do individual factors plague between-subject designs more than within-subject designs? What do we attempt to do to counter this?

In between, there are different people in different groups, who may each have some factors that could influence the results. In the within design, these factors are evenly distributed among all the study conditions, and so don't have a strong influence on the results. To counter this potential effect, you want to take great pains to achieve equivalent groups. This can be done multiple ways: Matching for example.

Why do laboratory experiments typically have more internal validity compared to field studies?

In the lab, one can manipulate the IV and control all other extraneous variables/confounds. Thus, changes in the DV result solely from changes in the IV. So can make valid causal statements.

What type of statistics are used to assess reliability?

Inferential statistics

The cell values on an interaction table are NOT additive. What does that mean?

It means there's an interaction between the 2 study IVs.

Choose the appropriate term to finish the statement: Less contrast validity means "more or less" confounding variables and random error.

Less Construct validity = more confounding variables and random error.

What is a Likert scale and what is a basic assumption about your variable of interest when using such a scale?

Likert scale: Scale that addresses an attitude/judgement/etc.. numerically (ex: scale of 1-7). This is an example of a summated rating scale. When you use such a rating, you are assuming that scores' units mean something. Particular, the ratios between the different items on the continuum are evenly distributed. Ex: Higher score on a negative item, 4, is more negative than a 2 score, and this 2 times more negative. Same distance between 5 and 3 vs 4 and 2, and so forth.

What is a main effect and an interaction effect in a factorial study?

Main effect: effect of single IV in the experiment. Interaction: effects of 1 IV is dependent on the level of another IV.

What is an assumption about "time" when it comes to the reliability of measurements?

Most of the time, measurements are more stable when assessed in a relatively short period of time.

What is a theoretical construct?

Name given to intervening variables due to their complexity. Stems from notion that intervening variables have many more inputs and outputs compared to what we operationally defined (many of which may be unknown or uncovered). So, while we define that A is linked to C through B, there may be other variables like D and/or E that leads to the same results.

Why would you choose a between- or within- design when running a study?

Need to minimize extraneous/uncontrolled variations to increase likelihood that an experiment will produce internally valid results. Between chosen if worried about a carryover effect. Within chosen if worried about individual factors contaminating the results.

Do main effect results indicate anything about main effects results? Explain.

No. Main effects are statistically independent of interactions. Knowing the main effects and their directions cannot predict interactions.

What is a meta-analysis and why is it important for the advancement of a field?

Objective technique for summarizing across many studies dealing with a subject. Useful since there is a limitation of a single experiment where you can't get a definitive picture of how things are through 1 study. But, consideration of many studies can shed light on trends.

What is a mixed research design? How does this type of design address concerns for both between- and within-subject designs?

Often makes sense to treat some independent variables as between-subjects and others as within-subjects. So mixed designs capitalize on having both designs in a single experiment. Allows to deal with the limitations: If a variable is likely to cause carryover effects, it can be made between-subjects. If there are potential concerns of individual characteristics, it can be made within-subjects.

What is subject attrition and why does it particularly plague between-design studies?

One or more subjects don't finish the study and/or show a change for reasons other than the IV(s). Can make any initial matching efforts ineffective, which is a technique often used in between designs. So, if one expects great subject attrition in your study, DO NOT USE MATCHING!

What other concerns related to within-subjects design that can be controls (or at least investigated systematically) when using a randomization of task conditions?

Practice or fatigue may affect results. By using randomization, you can make sure the same last condition completed isn't always the same task condition overall.

What were the results of the Jarrard (1963) LSD study in rats?

Pressing lever slightly enhanced by 2 lowest doses. Rats severely impaired with 2 largest doses.

What is statistical power and how does sample size influence this?

Probability that a test of significance will detect a deviation from the null hypothesis, should such a deviation exist. So basically, higher power occurs when statistical test can detect effects. More observations means less influence of subject characteristic on the data, which leads to higher power (so higher ability to detect).

What does internal validity mean?

Refers to the degree to which we can confidently infer a causal relationship between variables.

11. What is internal validity? Give an example.

Refers to whether one can make causal statements about the relationship between variables. In other words, whether one can make causal statements about variable relationships. So, if an observation is internally valid, the researcher can believe that one variable caused changes in another. Ex: if I have 3 tasks and I'm worried about task fatigue by the time the participants get to the last task. Therefore, instead of participants all getting task 1, then 2 and then 3, I randomize the order. Then, when it comes time to analyze the results, I can see if there are any effects on the last task... the one I was worried about fatigue. At this point, I should have an equal distribution of people who took task 1, 2 and 3 as their final task. If I see that everybody did poorer on their final task, then I have to attribute and effect of fatigue in my protocol, which leads to decreased internal validity. If I do not see any consistent effects on the last task, I can feel more confident that this is a task-dependent effect and not a fatigue effect.

What is predictive validity? Provide a tangible example.

Relating one aspect of behavior to another measure of behavior. So, you are seeing if your item of interest is actually measuring what you are intending by comparing it to another observation or measurement that you already have confidence in. Essentially, you are using a criterion which is a benchmark to validate your own measurement. Ex: A personal trainer might administer a fitness test to NBA rookies and then record the average points per game scored by the players during their next five years in the league. If there is a high degree of correlation between the scores on the fitness test and the average points per game scored by the players, then the personal trainer can say that it's valid to use the test to predict the future points per game of players.

Explain why the haphazard bias also plagued animal research.

So many things are not exactly random in animal research also. For example, you may study the type of mouse/rat is readily available at your institution or via the purchasing company. You may only follow the type of chimps happen to be present in the forest you are examining. Etc...

Explain how some inputs lead to more concrete constructs compared to others?

Some inputs are well defined (soft, medium, loud sound). This leads to a clear judgement (construct) and then a stable reporting/result (quiet, inside voice, really noisy). Other inputs have less specified input (experience, genes, environment). This leads to a combination of experiences that we, as a society, have labeled (construct, ex: the idea of anxiety) and then a less consistent reporting/result (some say they have sweaty palms, others heart palpitations, etc...).

How could human-factors psychologists contribute to product development and usability? (ex: think of a GPS system)

System information is only as efficient as the information source. Reliability decreased when obtained information is delayed in getting to the consumer. BUT perfect reliable system = expensive. Human-factors psychologists can work on system reliability: making sure the product is cost-efficient but reliable enough to use.

What is reliability? Explain the relationship with variability.

The consistency of behavioral measures. We know that there is always some variability when conducting group measures (basically, not everybody is a carbon copy of each other!). High variability = low reliability. Low variability = high reliability.

What is validity?

The truth of explanations, or more specifically, the best available approximation to the truth or falsity of propositions.

What can we conclude if the main effects in a study result to being the same amount? How does that look on a graph?

There is additivity (so no interaction). This leads to parallel lines on the graph.

What are 2 threats to construct validity? Explain.

Threat 1: Measure could be invalid due to extraneous variables (confounding variables). Confounding variables may influence the dependent variables, not the independent variable. Ex: maybe reading something out loud leads to a different response, but that wasn't your intention/study focus. Threat 2: Random error. Extent to which other sources enter observations (confounds, error) determines degree of validity (so something happens that you miss and/or is out of your control.) Ex: the experimenter misreads apparatus and collects incorrect data.

What is a random-groups design? Why is this not always feasible?

Unbiased assignment of subjects to conditions. Usually unable to have total random selection of the subjects. Ex: if studying college students, don't have option to select a pool from every single college student out there.

How is a meta-analysis conducted? Why is this susceptible to error?

Use key words and do a systematic search of all papers within your subject, using multiple databases. You read through all the papers, accepting those within your pre-determined parameters and rejecting the others. You log all your steps and report it in your methods. The final goal is to synthesize findings in a single paper. Observer fallibility can cause greater problems than in direct observation. A researcher is drawing conclusions across many studies, and it is difficult and is subjective. Hence, this work can be susceptible to error and bias when doing a search, deciding on inclusion parameters, and writing up the results.

Explain why a "non-active" control task threatens internal validity of studies?

Waiting around (so not doing something) is very different than asking another group to be engaged. Ex: If wondering if learning 1 list of words influence learning another? You give learn 2 lists of words. Experimental = learn A, then B, and then tested on A again and Control= learns A, does nothing while the other group learns B, and then tested on A again. Results may not the reveal true study results (interference), but act of "doing something" vs. "not doing something". Thus, a proper control will actively engage the control participants in something else during the time that the experimental group (does list B for example).

How did Weiss approach the follow up executive monkeys study in 1971? What were the results?

Weiss (1971) follow-up study suggests that there is higher stress with higher avoidance rates, which leads to a higher occurrence of ulcers. (AKA: helpless animals feel more stress.) They did not pre-select the executive and co-worker group members based on initial reactivity. So, that individual characteristic was evenly distributed throughout both groups.

Describe the Altman (2004) study that studied cue types in relation to the gear-switching model. What was the initial study design, the 3 IVs and the study results? What did they do afterwards to increase the external validity of the findings and what was the result?

Within design · IV#1 (short or long duration cue), IV#2 (rectangle height or width task judgment), IV#3 (category switch or maintenance/continuity of task). · Results: Interaction revealed between cue-stimulus interval and continuity (IV#1 & IV#3). Thus, benefit for longer cue-stimulus interval = greater when task needs to be switched rather than repeated. To increase external validity: · Altman replicated the study as a between-subjects design. · Between design did not reveal the same results as the within design. · So potential carryover effects could have plagued the initial study results.

If the results are not similar among results revealed in a between-subjects study and a within-subjects study, what are two conclusions that could be drawn?

Within-study may be more sensitive. There was a carryover effect you didn't realize that affected the within-subjects results. Maybe the group were non-equivalent in the between design. Maybe there were external circumstances that lead to the between results being different from the within ones (ex: you changed the computer keyboard for one of the studies).

Describe the study by Jarrard in 1963 and the particularities of the research design that yielded internally valid results?

Within-subjects design using rats. Trained rats to press a lever to obtain food. Gave a series of 6 injections (one at a time): Baseline injection was placebo, 5 subsequent injections were increasing LSD doses (0.5 to 0.8 mg per kg). Immediately after injection, rats placed in lever cage that had a lever to press and left there for 2 hours. Several days were allowed between injections. Counterbalancing applied to dose order to avoid rats becoming tolerant or sensitized to LSD by getting the order of "smallest to largest" dose. Therefore, there was a systematic variation of the orders of conditions, so all doses appear equally often at each ordinal position.

Considering how much we have emphasized the need for control in experimentation, why would we engage in studies that are not simple in construction, such as a factorial design?

Yeah, control is super important to increase the internal validity of a study. HOWEVER: major reason to undertake a more complex study (and thus with less control) is that thought and behaviour are complex. Hence, limiting analysis to a "one-at-a-time model" does not have ecological validity (aka doesn't mirror the real world, so there's limited generalization of the results).

Why do self-report methods suffer from many validity problems?

You are not measuring the behaviour directly, but asking for a recount of behaviour, feelings, and such. Therefore, the data is difficult for the observer to corroborate immediately and directly. So, lots of considerations in play: · participant reactivity?! How do you know a person is answer honestly or saying what she or he thinks the researcher wants to hear? · number of variables (short vs. long questionnaire), · how open-ended the questions are, · how did they interpret the question · mood · etc...

What is the necessary tradeoff between sample size and number of observations in a complex within-subjects design study?

You can get away with fewer participants (as you are minimizing the effects of individual characteristics). HOWEVER, you need to increase number of observations per subject in each condition to increase reliability to reduce likelihood of extraneous factors producing results.

Explain why you cannot make inference with predictive validity?

You cannot infer that any other factors are related, unless you systematically study those other factors. This is limited to only the question asked and exact items measured.

Why is it that individual differences influence smaller groups compared to larger groups in research?

You have more datapoints in a large group, so a single data point will carry less weigh on the overall averaging. Hence why larger groups are more representative of the overall population.

If you cannot have total randomization in the selection of your sample from the entire population, how else could you use randomizing to minimize the effects of subject characteristics?

You should then randomly assign participant to the different groups (ex: treatment conditions). Random assignment minimizes differences attributable to individual traits.