Chapter 5 & 6

Mundane Realism

*extent to which experiment is similar to real-life situations*

experimenter expectancy effect

*important source of variation → the experimenters expectations about how participants "should" behave in the experiment*

Prioritizing validities

*usually cannot achieve all 4 validities at once, so need to prioritize*

restriction of range

- *1 or both of variables have limited range in sample relative to population* - not always anticipated or easily avoidable → good practice to examine data for possible restriction of range and to interpret Pearson's r in light of it

Correlation does not imply causation

- *2 reasons:* 1) *directionality problem* 2) *third variable problem*

Directionality problem

- *2 variables, X and Y, can be statistically related because X causes Y or because Y causes X*

Subject pool

*established group of people who have agreed to be contacted about participating in research studies*

Third Variable Problem

- *2 variables, X and Y, can be statistically related not because X causes Y, or because Y causes X, but because some third variable, Z, causes both X and Y* - *correlations as result of third variable often referred to as SPURIOUS CORRELATIONS*

Operationalization

- *Conversion from research question to experiment design*

When to use nonexperimental research

- *Experimental research is appropriate when researcher has specific research question or hypothesis about causal relationship between 2 variables—and is possible, feasible, and ethical to manipulate independent variable* - *Nonexperimental research is appropriate when these conditions are not met* - *nonexperimental research is preferred when:* • *research question or hypothesis can be about single variable rather than statistical relationship between two variables* (e.g., How accurate are people's first impressions?). • *research question can be about noncausal statistical relationship between variables* (e.g., Is there a correlation between verbal intelligence and mathematical intelligence?). • *research question can be about causal relationship, but independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions* (e.g., Does damage to a person's hippocampus impair the formation of long-term memory traces?). • *research question can be broad and exploratory, or can be about what it is like to have particular experience* (e.g., What is it like to be a working mother diagnosed with depression?). - *choice between experimental and nonexperimental approaches dictated by nature of research question* - *3 goals of science: describe, predict, and explain* • if the goal is to explain and research question pertains to causal relationships → experimental approach is typically preferred • if goal is to describe or predict → experimental approach will suffice • the two approaches can also be used to address same research question in complementary ways

Observational Research

- *Goals:* • *describe variable or set of variables* • *obtain snapshot of specific characteristics of individual, group, or setting* - *Nonexperimental because nothing is manipulated or controlled, and cannot arrive at causal conclusions using this approach* - *Data collected often qualitative in nature but may also be quantitative or both (mixed methods)* - *Several different types of observational research designs:*

4th solution to the problem of placebo effects

- *Leave out control condition completely and compare any new treatment with best available alternative treatment* - Does it work? Does it work better than what is already available?

1) No-treatment control condition

- *Participants receive no treatment whatsoever* - *Problem: existence of placebo effects*

Recruiting participants

- *Several approaches to recruiting participants:* • Use *participants from* a formal *subject pool* - task is not recording participants but selecting them - important that this kind of selection can be done according to well defined set of rules that are established before data collection begins and can't be explain clearly afterwards - *point of having well-defined selection rule is to avoid bias in selection of participants* - biases can be entirely unintentional

2) Placebo control condition

- *Solution to placebo effect* - *Participants receive placebo that looks like treatment but lacks active ingredient or element thought to be responsible for treatment's effectiveness* - *principle of informed consent requires participants be told that will be assigned either treatment or placebo control condition—even though they cannot be told which until the experiment ends* • Participants in control condition are offered opportunity to have real treatment

3) Wait-list control condition

- *Solution to placebo effect* - *Participants told they will receive treatment but must wait until participants in treatment condition have already received it* - *disclosure allows researchers to compare participants who have already received treatment with participants who are not currently receiving it but who still expect to improve (eventually)*

Block Randomization

- *all conditions occur once in sequence before any is repeated, then all occur again before any is repeated again* - *within each "blocks," conditions occur in random order* - *sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to next condition in sequence (when procedure is computerized, computer program often handles block randomization)*

Treatment

- *any intervention meant to change people's behavior for the better* • intervention includes psychotherapies in medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on

Double-blind study

- *both participants and experimenters are blind to condition* - *many times when blinding is not possible!!*

Archival research

- another approach considered observational research - *analyzing data that have already been collected for some other purpose* - as with naturalistic observation, *measurement can be more or less straightforward when working with archival data*

Structured observation

- another observational method - *investigators make careful observations of one or more specific behaviors in any particular setting that is more structured than settings used in naturalistic and participant observation* - *often, setting in which the observations are made is not natural setting, rather researcher may observe people in the laboratory environment* - *researcher may observe people in natural setting (like a classroom setting) that they have structured someway* •EX: introducing some specific tasks participants are to engage in or by introducing specific social situation or manipulation - *similar to naturalistic observation and participant observation in that in all cases researchers are observing naturally occurring behavior, however, emphasis in structured observation is on gathering quantitative rather than qualitative data* - *interested in limited set of behaviors* • *allows to quantify behaviors they are observing* - *less global than naturalistic and participant observation because researcher engaged in structured observation is interested in small number of specific behaviors* - *rather than recording everything that happens, researcher only focuses on very specific behaviors of interest* - *Precise specification of sampling process makes data collection manageable for observers, and provides some control over important extraneous variables* - *when observations require judgment on part of observers* → *CODING* - Primary benefit: *more efficient than naturalistic and participant observation* • *reduces time and expense since researchers focused on specific behaviors* • *environment structured to encourage behaviors of interest which means researchers do not have to invest as much time in waiting for behaviors of interest to naturally occur* • *clearly exert greater control over environment, but may make environment less natural which decreases external validity* - *since researchers often not disguised, may be more concerned with reactivity*

manipulate an independent variable

- *change level systematically so that different groups of participants are exposed to different levels (conditions) of that variable, or same group of participants is exposed to different levels (conditions) at different times* - manipulation of independent variable must involve *active intervention of researcher* - comparing groups of people who differ on independent variable before study begins is not same as manipulating variable • distinction important bc *groups that already differ in one way at beginning of study are likely to differ in other ways too* - *active manipulation of independent variable is crucial for eliminating potential alternative explanations for results (third-variable problem)* - many *situations in which independent variable cannot be manipulated for practical or ethical reasons and therefore experiment is not possible* - 2 designs: • *single factor two-level design* • *single factor multi-level design*

Cross-sectional research

- *comparing 2 or more pre-existing groups of people* - *no manipulation of an independent variable in a random assignment of participants to groups* - *used by developmental psychologists who study aging and by researchers interested in sex differences* - *Developmental psychologist compare groups of people w/ different ages* (e.g., young adult spanning from 18 to 25 years of age versus older adults spending 60 to 75 years of age) *on various dependent variables* (e.g., memory, depression, life satisfaction). • limitation: effects of aging and differences between groups other than age may account for differences in dependent variable - *used to study sex differences:* • can't practically or ethically manipulate sex of participants, must rely on cross-section of designs to compare groups of men and women on different outcomes (e.g., verbal ability, substance use, depression). - *unclear what is causing differences:* • *unclear whether differences due to environmental factors like socialization or biological factors like hormones*

Randomized clinical trial

- *controlled medical experiment in which subjects are randomly chosen to receive either experimental treatment or standard treatment (or placebo)* - In research on the effectiveness of psychotherapies in medical treatments

Scatterplots

- *correlations between quantitative variables presented using scatterplots* - *positive relationship*: higher scores on one variable tend to be associated with higher scores on the other - *negative relationship*: higher scores on one variable tend to be associated with lower scores on the other

conditions

- *different levels of independent variable* - often give conditions short descriptive names to make easy to talk and write about them

Within-Subjects Experiment

- *each participant is tested under all conditions* - *primary advantage: provides maximum control of extraneous participant variables* - *possible to use statistical procedure that removes effect of extraneous participant variables on dependent variable and makes data less "noisy" and effect of independent variable easier to detect* - *disadvantage: easier for participants to guess hypothesis* • can result in *order effects*

Between-Subjects Experiment

- *each participant tested in only one condition* - essential that researcher assigns participants to conditions so that different groups are highly similar to each other • Ex: trauma condition neutral condition → include similar proportion of men and women with similar IQs, average levels of motivation, average number of health problems, etc. - *matching is method of controlling extraneous participant variables across conditions so that do not become confounding variables* - *often used to determine whether a treatment works*

Standardizing the procedure

- *easy to introduce extraneous variables during procedure* - to extent of such variables affect participants' behavior, they add noise to data and make effect of independent variable more difficult to detect - if vary systematically across conditions, they become confounding variables and provide alternative explanations for results - important source of variation: the experimenters expectations about how participants "should" behave in experiment(*experimenter expectancy effect*) - *way to minimize unintended variation in procedures: standardize as much as possible so that carried out in same way for all participants regardless of condition they are in* • *Several ways to do this:* 1) *Create written protocol that specifies everything that experimenters are to do and say from time they greet participants to time they dismiss them* 2) *Create standard instructions that participants read themselves or that are read to them word for word by experimenter* 3) *Automate rest of procedure as much as possible by using software packages for this purpose or even simple computer slide shows* 4) *Anticipate participants' questions and either raise and answer them in instructions or develop standard answers for them* 5) *Train multiple experimenters on protocol together and have them practice on each other* 6) *Be sure that each experimenter tests participants in all conditions* - Another good practice: *arrange experiments to be "blind" to research question or to condition in which each participant is tested* → *minimize experimenter expectancy effect by minimizing the experimenters' expectations* (*double-blind study*)

Record Keeping

- *essential to keep good records when conduct experiment* - typical for experimenters to generate written sequence of conditions before study begins and then to test each new participant in next condition in sequence - *good idea to add basic demographic information as tested → date, time, and place of testing; and name of experimenter who did testing* - also good idea to *have place for experimenter to write down comments about unusual occurrences (e.g., a confused or uncooperative participant) or questions that come up* • *information can be useful later if decide to analyze sex differences or effects of different experimenters, or if question arises about particular participant or testing session* - Since participants' identities should be kept confidential (or anonymous), *names and other identifying information should not be included with data* - to identify individual participants, *can be useful to assign identification number to each participant as you test them* (numbering them consecutively beginning with 1 is usually sufficient.) → *number can then be written on any response sheets or questionnaires that participants generate, making it easier to keep them together*

HM case study

- Extremely important for memory researchers bc *suggested disassociation between short-term memory and long-term memory* - *suggested that were two different abilities sub-served by different areas of brain* - *suggested that temporal lobes are particularly important for consolidating new information* (i.e., for transferring information from short-term memory to long-term memory)

Key Takeaways from Chapter 5

- *experiment* is type of *empirical study* that features *manipulation* of *independent variable*, *measurement of dependent variable*, and *control of extraneous variables* - *extraneous variable: any variable other than independent and dependent variables; confound is extraneous variable that varies systematically with independent variable* - *Experiments conducted* using either *between-subjects or within-subjects designs*. Deciding which to use requires *careful consideration of pros and cons of each approach.* - *Random assignment to conditions in in between-subjects* experiments or to *orders of conditions in within-subjects experiments* is fundamental element of experimental research; purpose is to *control extraneous variables so that do not become confounding variables.* - Studies *high in internal validity to extent that way they are conducted supports conclusion that independent variable caused any observed differences in dependent variable*. Experiments generally *high in internal validity because manipulation of independent variable and control of extraneous variables.* - Studies *high in external validity to extent that result can be generalized to people and situations beyond those actually studied*. Although experiments can seem "artificial"—and low in external validity—important to consider whether psychological processes under study are likely to operate in other people and situations. - several effective methods can use to *recruit research participants for experiment:* formal *subject pools, advertisements, and personal appeals*. Field experiments require well-defined participant selection procedures. - *Experimental research on effectiveness of treatment requires both treatment condition and control condition, which can be no-treatment control condition, placebo control condition, or wait-list control condition. Experimental treatments can also be compared with best available alternative.* - important to *standardize experimental procedures to minimize extraneous variables, including experimenter expectancy effects* - important to *conduct 1 or more small-scale pilot tests of experiment to be sure that procedure works as planned*

Internal validity

- *extent to which design of study supports conclusion that changes in independent variable caused observed differences in dependent variable* - *experimental research tends to be highest in internal validity because use of manipulation* (of the independent variable) *and control* (of extraneous variables) *rule out alternative explanations for relationships* - if average score in dependent variable in experiment different across conditions, quite likely that independent variable is responsible for difference - *Non-experimental (correlational) research is lowest in internal validity because designs fail to use manipulation or control* - *Quasi-experimental research is in middle because contains some features of true experiment* → EX: may fail to use random assignment to assign participants to groups or fail to use counterbalancing to control for potential order effects

Psychological Realism

- *extent to which psychological processes triggered in experiment are similar to psychological processes that occur in everyday life* - same mental process used in both laboratory and real world

confounding variable

- *extraneous variable that differs on average across levels of independent variable* (i.e., extraneous variable that varies systematically with independent variable) - difficult to detect effect of independent variable - confound means to confuse → exactly why confounding variables are undesirable - bc *differ systematically across conditions* (just like independent variable), *provide alternative explanation for observed difference in dependent variable* - *avoid confounding variable by:* • *holding extraneous variables constant* • *random assignment to conditions*

Content analysis

- *family of systematic approaches to measurement using complex archival data* - just as structured observation requires specifying behaviors of interest and then noting them as they occur, content analysis *requires specifying keywords, phrases, or ideas and then finding all occurrences of them in data* • *occurrences can be counted, timed (e.g., the amount of time devoted to entertainment topics on the nightly news show), or analyzed in variety of other ways*

Observational research

- *focuses on making observations of behavior in natural or laboratory setting without manipulating anything* - *more qualitative in nature:* • data are usually non-numerical and therefore cannot be analyzed using statistical techniques • has a separate set of analysis tools depending on the research question → EX: thematic analysis would focus on themes that emerge in the data or conversion analysis would focus on the way the words were said in the interview or a focus group.

Case study

- *in-depth examination of individual* - *also completed on social units* (e.g., a cult) *and events* (e.g., a natural disaster) - *provide detailed description and analysis of individual* - *after individual has rare or unusual condition or disorder or has damage to specific region of brain* - *tend to be more qualitative in nature* - *methods involve in-depth and often longitudinal examination of individual* - *depending on focus of case study, individuals may or may not be observed in natural setting* - *if natural setting is not what is of interest, than individual may be brought into therapist's office or researcher's lab for study* - *focus on in-depth description of person rather than on statistical analyses* - *some quantitative data may also be included in the write up of case study* - *variety of different methods and tools can be used to collect information on case* - useful bc provide level of *detailed analysis not found in many other research methods and greater insights may be gained from more detailed analysis* - *researcher may gain sharpened understanding of what might become important to look at more extensively in future more controlled research* - *often only way to study rare conditions bc may be impossible to find larger enough sample to individuals with condition to use quantitative methods* - although at first glance case study of rare individual might seem to tell little about ourselves, often do *provide insights into normal behavior* - can *provide insight into certain areas and variables to study and can be useful in helping develop theories*, but *should never be used as evidence for theories* • can be used as *inspiration to formulate theories in hypothesis*, but those hypothesis and theories then *need to be formally tested using more rigorous quantitative methods* • reason: *suffer from problems with internal and external validity* - *lack proper controls true experiments contain, so suffer from problems with internal validity, so cannot be used to determine causation* - cannot rule out sorts of alternative explanations - *do not permit determination of causation* - because study often of single individual, and typically very abnormal individual, researchers *cannot generalize conclusions to other individuals* - most research designs there is a trade-off between internal and external validity, with case studies there are *problems with both internal validity and external validity* • *limits both ability to determine causation and to generalize results* - *limitation: ample opportunity for theoretical biases of researcher to color or bias case description*

Nonexperimental research

- *lacks manipulation of independent variable* - rather than manipulating independent variable, researchers *measure variables as naturally occur (in the lab or real world)* - *Although experimental research can provide strong evidence that changes in independent variable causes differences in dependent variable, nonexperimental research generally cannot* • inability to make causal conclusions does not mean nonexperimental research is less important than experimental research

Reactivity

- *measure changes participants behavior* - in case of *undisguised naturalistic observation*, concern with reactivity is that *when people know that they are being observed and studied, they may act differently than they normally would* - *disguised observation is less reactive and therefore can have higher validity because people are not aware that their behaviors are being observed and recorded*, however, we know that *people often become used to being observed and with time they begin to behave naturally in the researchers presence (overtime people habituated to being observed)*

Matched Groups

- alternative to simple random assignment of participants to conditions - *participants in various conditions are matched on dependent variable or on extraneous variable(s) prior to manipulation of independent variable → guarantees variables will not be confounded across experimental conditions*

Pearson's Correlation Coefficient (or Pearson's r)

- *measures strength of correlation between quantitative variables* - *ranges from -1.00 (strongest possible negative relationship) to +1.00 (strongest possible positive relationship* - *value of 0 means no relationship between the 2 variables* • points on scatterplot form a shapeless "cloud" • as the value moves toward -1.00 or +1.00, the points come closer and closer to falling on a single straight line - *correlation coefficients near ±0.10 are considered small, values near ±0.30 are considered medium, and values near ±0.50 are considered large* - *sign of Pearson's r is unrelated to strength* - *2 situations in which r can be misleading:* • *good measure only for linear relationships*, in which points best approximated by straight line; *not good measure for nonlinear relationships*, in which the points are better approximated by curved line (*important to make scatterplot and confirm that relationship is approximately linear before using r*) • *1 or both of variables have limited range in sample relative to population* (aka *RESTRICTION OF RANGE*)

Correlational research

- *most common type* of nonexperimental research conducted in psychology - very similar to cross-sectional research (sometimes used interchangeably) • *distinction: rather than comparing two or more pre-existing groups of people as it's done with cross-sectional research, correlation research involves correlating two continuous variables (groups are not formed and compared)*

Qualitative Research

- *most studies* conducted in psychology are *quantitative* in nature • quantitative researchers typically start with a focused research question or hypothesis, collect a small amount of data from each of a large number of individuals, describe the resulting data using statistical techniques, and draw general conclusions about some large population • most common approach to conducting empirical research in psychology - important alternative: *qualitative research* • originated in disciplines of anthropology and sociology but now used to study psychological topics - *begin with less focused research question, collect large amounts of relatively "unfiltered" data from relatively small number of individuals, and describe data using nonstatistical techniques* - *less concerned with drawing general conclusions about human behaviour than with understanding in detail experience of research participants*

Naturalistic observation

- *observational method that involves observing people's behavior in environment in which it typically occurs* - *type of field research* (as opposed to a type of laboratory research) - EX: Observing shoppers in a grocery store, children on a school playground, or psychiatric inpatients in their wards - arguments against ethicality of naturalistic observation of "bathroom behavior" is that people have a reasonable expectation of privacy even in a public restroom and that this expectation was violated

single factor multi-level design

- *one independent variable* manipulated to produce *more than two conditions* - used when greater insights can be gained by adding more conditions to experiment

Single-blind study

- *only participant is blind to condition*

extraneous variables

- *other variables that may influence results of the study* - anything that varies in context of study other than independent and dependent variables - problem bc many likely to have some *effect on dependent variable* • *difficult to separate effect of independent variable from effects of extraneous variables* - difficult to detect effect of the independent variable in two ways • adding variability or "noise" to data

Placebo effect

- *positive effect of placebo treatment* - not well understood; *probably driven primarily by people's expectations that they will improve* • *having expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning* - *Pose serious problem for researchers who want to determine whether treatment works* - *Several solutions:* • *Placebo control condition* • *Wait-list control condition*

4) Statistical Validity

- *proper statistical treatment of data and soundness of researchers' statistical conclusions* - *many different types of inferential statistics (t-tests, ANOVA, regression, correlation)* and statistical validity *concerns the use of the proper type of test to analyze the data* - when considering proper type of test, researchers must *consider scale of measure dependent variable is measured on in design of study* - *many inferential statistics tests carry certain assumptions (the data are normally distributed) and statistical validity is threatened when these assumptions are not met but statistics are used nonetheless* - *common critique of experiments: study did not have enough participants* • *difficult to generalize about population from small sample * → seems critique is about external validity but there are studies where small sizes are not problem → *Small sample sizes is critique of statistical validity. The statistical validity speaks to whether statistics conducted study are sound and support conclusions made.* - *proper statistical analysis should be conducted on data to determine whether difference in relationship that was predicted was found* - *number of conditions and total number of participants will determine overall size of effect* - when designing study, it is best to *think about power analysis so that appropriate number of participants can be recruited and tested* - to design statistically valid experiment, thinking about statistical test at beginning of design will help ensure results can be believed

3) Construct Validity

- *quality of experiments manipulations*

Between-Subjects or Within-Subjects?

- Almost every experiment can be conducted using either between-subjects or within-subjects design → means *researchers must choose between the two approaches based on relative merits for particular situation* - *Between-subjects experiments → conceptually simpler, require less testing time per participant, avoid carryover effects without need for counterbalancing* - *Within-subjects experiments → control extraneous participant variables which reduces noise in data and makes easier to detect relationship between independent and dependent variables* - good rule of thumb: *If possible to conduct within-subjects experiment (with proper counterbalancing) in time that is available per participant (and have no serious concerns about carryover effects) this design is probably best option*. *If within-subjects design would be difficult or impossible to carry out, then should consider between-subjects design instead* - Remember: using one type of design does not preclude using other type in different study; *no reason that researcher could not use both designs to answer same research question* (professional researchers *often take mixed methods approach*)

Types of nonexperimental research

- Cross-sectional research - Correlational research - Observational research

Random Assignment

- *random process to decide which participants are tested in which conditions* - primary way that researchers accomplish control of extraneous variables across conditions - different from random sampling !!!! → method for selecting sample from population, and is rarely used in psychological research - *method for assigning participants in sample to different conditions, and is important element of all experimental research in psychology and other fields* - *should meet 2 criteria:* 1) *each participant has equal chance of being assigned to each condition* (50% chance) 2) *each participants assigned to condition independently of other participants* - ways to assign: • coin flip • for 3 or more conditions, can use computer to generate random integers for each participant - *full sequence of conditions (one for each participants expected to be in experiment) usually created ahead of time, and each new participant is assigned to next condition in sequence tested; when procedure is computerized, computer program often handles random assignment* - *problem with strict procedures* (like coin flip) *for random assignment is they likely to result in unequal sample sizes in different conditions* (not a serious problem), but for fixed number of participants, is *statistically most efficient to divide into equal-sized groups*, so use modified random assignment that *keeps number of participants in each group as similar as possible* • can use *2 approaches:* 1) *block randomization* 2) *matched-groups design* - *not guaranteed to control all extraneous variables across conditions* → always possible just by *chance* participants in one condition turn out to be older, less tired, more motivated, or less depressed on average than participants in another condition • reasons this possibility is *not major concern:* 1) *random assignment works better than expected, especially for large samples* 2) *inferential statistics* used to decide whether difference between groups reflects difference in population takes *"fallibility" of random assignment into account* 3) *even if random assignment result in confounding variable and produces misleading results, confound is likely to be detected when experiment is replicated* 4) *random assignment to condition* (not infallible in terms of controlling extraneous variables) *always considered strength of research design*

Coding

- *requires clearly defined the set of target behaviors* - *observers categorize participants individually in terms of which behavior have engagement and the number of times engaged in each behaviors* - *observers might record duration of each behavior* - *target behaviors must be defined in such a way that different observers code them in same way* - *difficulty with coding is issue of interrater reliability* - *researchers are expected to demonstrate interrater reliability of coding procedure by having multiple raters code same behaviors independently and then show that different observers are in close agreement*

Undisguised participant observation

- *researchers become part of group they are studying and disclose true identity as researchers to group under investigation*

Disguised naturalistic observation

- *researchers make observations as unobtrusively as possible so participants are not aware being studied* - *Ethically, method is considered to be acceptable if participants remain anonymous and behavior occurs in public setting where people would not normally have expectation of privacy*

Disguised participant observation

- *researchers pretend to be members of social group they are observing and conceal true identity as researchers* - *Ethical issues: * • *No informed consent can be obtained* • *Passive deception is being used* → *researcher is passively deceiving participants by intentionally withholding information about motivations for being part of social group they are studying* → *sometimes only way to access protective group* (like a cult) - *less prone to reactivity than undisguised participant observation*

Manipulation Check

- *separate measure of construct researcher is trying to manipulate* - *purpose: confirm that independent variable was successfully manipulated* - *independent variable is construct that can only be manipulated indirectly* - *important when results of experiment turn out null* - *cases where results show no significant effect of manipulation of independent variable on dependent variable, a manipulation check can help experimenter determine whether null result is due to real absence of effect of independent variable on dependent variable or if it is due to problem with manipulation of independent variable* - *usually done at end of procedure to be sure that effect of manipulation lasted throughout entire procedure and to avoid calling unnecessary attention to manipulation (to avoid a demand characteristic)* - *wise to include manipulation check in pilot test of experiment so that avoid spending lot of time and resources on experiment that is doomed to fail and instead spend that time and energy finding better manipulation of independent variable*

Placebo

- *stimulated treatment that lacks any active ingredient or element that should make it effective* - many folk remedies that seem to work (eating chicken soup for a cold or placing soap under the bed sheets to stop nighttime leg cramps) are probably nothing more than placebos

Treatment Condition and Control Condition

- *to determine whether treatment works, participants are randomly assigned to either* *treatment condition*, in which they receive the treatment, *or control condition*, in which they do not receive treatment - *Participants in treatment condition end up better off than participants in the control condition and the researcher can conclude that the treatment works* - *Different types of controlled conditions:* 1) *No-treatment control condition* 2) *Placebo control condition* 3) *Waitlist control condition*

Random Counterbalancing

- *use when number of conditions is large* - *order of conditions randomly determined for each participant - *every possible order of conditions is determined and then one order is randomly selected for each participant* - not as powerful as complete counterbalancing or partial counterbalancing using a Latin squares design - *results in more random error, but if order effects are likely to be small and the number of conditions is large, this is option available to researchers*

Undisguised naturalistic observation

- *used in cases where not ethical or practical to conduct disguised naturalistic observations* - *participants made aware of researcher's presence and of monitoring of their behavior* - *Concern: reactivity*

Simultaneous Within-Subjects Designs

- *used when participants make multiple responses in each condition*

Order Effect

- *when participants' responses in various conditions affected by order of conditions to which exposed* - *order of conditions is confounding variable* - any difference between conditions in terms of dependent variable could be caused by order of conditions and not independent variable itself - *solution: counterbalancing*

Experimenter selected independent variable

- *when researchers use participant characteristic to create groups* (nationality, cannabis use, age, six), *the independent variable is referred to as experimental selected independent variable* (as opposed to the experimental manipulated independent variables used in experimental research). - *unclear whether experiment or a cross-sectional study because unclear whether independent variable is manipulated by researcher or simply selected by researcher* - crucial point is that *what defines a study as experimental cross-sectional* is not variables being studied, nor whether the variables are quantitative or categorical, nor the type of graph or statistics used to analyze the data. It *is how the study is conducted*.

Participant observation

- approach to data collection of observational research - *researchers become active participants in group or situation they are studying* - Very similar to naturalistic observation in that it *involves observing people's behavior in environment in which it typically occurs* - As with naturalistic observation, *data that is collected can include interviews (usually instructed), notes based on observations and interactions, documents, photographs, and other artifacts* - Only difference between naturalistic observation in participant observation is that *researchers engage in participant observation become active members of group or situations they are studying* - *may be important information that is only accessible to or can be interpreted only by someone who is active participant in group or situation* - like naturalistic observation participant observation *can be either disguised or undisguised* - *Primary benefit: researcher is in much better position to understand viewpoint and experiences of people they are studying when are part of social group* - *Primary limitation: mere presence of observer could affect behavior of people being observed* - *may change social dynamic and or influence behavior of people they are studying* - If researcher acts as participant observer, can be *concerns with biases resulting from developing relationships with participants* → *researcher may become less objective resulting in more experimenter bias*

Complete Counterbalancing

- best method of counterbalancing - *equal number of participants complete each possible order of conditions* • EX: ABC, ACB, BAC, BCA, CAB, CBA, 4 → 24, 5 → 120

Data Collection

- correlational research: neither variable is manipulated - *does not matter how or where the variables are measured*

Control of extraneous variables

- essential component of all experiments, and required to prevent extraneous variables from becoming confounding variables and threatening internal validity of study - to make data look more like idealized data which makes effect of independent variable easier to detect - *control by:* • *holding constant* → holding situation or task variables constant by testing all participants in the same location, icing them identical instructions, treating them in the same way, and so on; holding participant variables constant • *limiting participants to 1 very specific category of person* (20 yrs old, hetero, femme, right handed psych major) → downside → *would lower external validity of study* → extent to which results can be generalized beyond people actually studied - advantages of diverse sample (increased external validity) outweigh reduction in noise achieved by homogenous one

Latin Square

- formal system of partial counterbalancing that *ensures each condition in within-groups design appears in each position at least once* - *more efficient way of counterbalancing* • 6 conditions would be 6x6, so 6 orders; complete counterbalancing of 6 conditions would be 720 orders...

Pilot Testing

- good idea to conduct pilot test of experiment - *small-scale study conducted to make sure that new procedure works as planned* - *can recruit participants formally (e.g., from an established participant pool) or can recruit informally from among family, friends, classmates, etc.* - *number of participants can be small, but should be enough to give confidence that procedure works as planned* - *several important questions answered by conducting pilot test:* • Do participants understand the instructions? • What kind of misunderstandings do participants have, what kind of mistakes do they make, and what kind of questions do they ask? • Do participants become bored or frustrated? • Is an indirect manipulation effective? (You will need to include a manipulation check.) • Can participants guess the research question or hypothesis? • How long does the procedure take? • Are computer programs or other automated procedures working properly? • Are data being recorded correctly? - *to answer some of these questions you will need to observe participants carefully during procedure and talk with them afterward* - Participants often hesitant to criticize study in front of researcher, so *be sure they understand that their participation is part of pilot test and are genuinely interested in feedback that will help improve procedure* - If procedure works as planned, then you can proceed with actual study - If there are problems to be solved, you can solve them, pilot test the new procedure, and continue with this process until you are ready to proceed

2) External Validity

- need to manipulate independent variable and control extraneous variables means experiments often conducted under conditions that seem artificial - *empirical study high in external validity if way it was conducted supports generalizing results to people and situations beyond those actually studied* - *studies higher in external validity when participants and situation studied are similar to those that researchers want to generalize to and participants encounter everyday* (*mundane realism*) - experiments are NOT low in external validity: • Experiments need not seem artificial • Experiments are often conducted to learn about psychological processes that are likely to operate in a variety of people and situations

Data Analysis in Quantitative Research

- quantitative and qualitative research generally differ along several important dimensions (specificity of research question, type of data collected) • *method of data analysis distinguishes them more clearly than anything else* - *quantitative-qualitative distinction depends more on what researchers do with data have collected than why or how collected the data* - many ways to analyze data. • *grounded theory* → *in quantitative research, typical for researcher to start with theory, derive hypothesis from that theory, and collect data to test hypothesis* → *in qualitative research using grounded theory, researchers start with data and develop theory or interpretation that is "grounded in" data; do this analysis in stages → first identify ideas repeated throughout data, then organize ideas into smaller number of broader themes, finally write theoretical narrative (interpretation) of data in terms of themes identified* - *theoretical narrative focuses on subjective experience of participants and usually supported by direct quotations from participants themselves*

Fatigue Effect

- type of carryover effect - *participants perform task worse in later conditions bc become tired or bored*

Context Effect (or contrast effect)

- type of carryover effect - *when being tested in one condition changes how participants perceive stimuli or interpret task in later conditions*

Carryover Effect

- type of order effect - *effect of being tested in one condition on participants' behavior in later conditions* - *3 types:* 1) *practice effect* 2) *fatigue effect* 3) *context effect (or contrast effect)* - *problematic when not focus of research*

The Quantitative-Qualitative "Debate"

- quantitative and qualitative research in psychology and related fields do not coexist in complete harmony - quantitative researchers criticize qualitative methods on grounds that they lack objectivity, are difficult to evaluate in terms of reliability and validity, and do not allow generalization to people or situations other than those actually studied - qualitative researchers criticize quantitative methods on grounds that they overlook richness of human behavior and experience and instead answer simple questions about easily quantifiable variables - qualitative researchers are well aware of the issues of objectivity, reliability, validity, and generalizability - developed number of frameworks for addressing these issues - quantitative researchers are well aware of issue of oversimplification → do not believe that all human behavior and experience can be adequately described in terms of small number of variables and statistical relationships among them → Instead, use simplification as strategy for uncovering general principles of human behavior. - *researchers from both quantitative and qualitative camps agree that the two approaches can and should be combined into what has come to be called "mixed-methods research"* - *use qualitative research for hypothesis generation and quantitative research for hypothesis testing* • qualitative study might suggest that families who experience an unexpected suicide have more difficulty resolving the question of why, a well-designed quantitative study could test a hypothesis by measuring these specific variables for a large sample • *triangulation* → *use both quantitative and qualitative methods simultaneously to study same general questions and compare results → if results of quantitative and qualitative methods converge on same general conclusion, reinforce and enrich each other, if results diverge, then suggest interesting new* question: Why do the results diverge and how can they be reconciled? - *using qualitative research can often help clarify quantitative results in triangulation*

Data Collection and Analysis in Qualitative Research

- quite varied and can involve naturalistic observation, participant observation, archival data, artwork, and many other things - *most common approaches: conduct interviews* • can be *unstructured (consisting of a small number of general questions or prompts that allow participants to talk about what is of interest to them) or structured (strict script that the interviewer does not deviate from)* • most *in between the two → semi-structured interviews (researcher has a few consistent questions and can follow up by asking more detailed questions about the topics that come up) → can be lengthy and detailed, but usually conducted with relatively small sample. - *FOCUS GROUPS* → *small groups of people who participate together in interviews focused on particular topic or issue* • *used in qualitative research* • *interaction among participants can bring out more information than can be learned in one-on-one interview* • standard technique in business and industry among those who want to understand consumer tastes and preferences • content is usually recorded and transcribed to facilitate later analyses • we know from social psychology that group dynamics are often at play in any group, including focus groups, and it is useful to be aware of those possibilities

Counterbalancing

- solution to problem of order effects that can be used in many situations - *testing different participants in different orders* - *participants assigned to orders randomly* - 2 ways what counterbalancing accomplishes: 1) *controls order of conditions so that no longer a confounding variable; any difference in dependent variable between 2 conditions cannot have been caused by order of conditions* 2) *makes possible to detect carryover effects*; one can analyze data separately for each order to see whether it had effect

The Purpose of Qualitative Research

- strength: *provide precise answers to specific research questions and draw general conclusion about human behavior* - quantitative research is good at providing precise answers to specific research questions but not nearly as good at *generating novel and interesting research questions* - quantitative research is good at drawing general conclusions about human behavior but not nearly as good at *providing detailed descriptions of behavior of particular groups in particular situations* - quantitative research is not very good at all at *communicating what it is actually like to be member of particular group in particular situation* - *weaknesses of quantitative research are strengths of qualitative research* - *help researchers generate new and interesting research questions and hypotheses* - *provide rich and detailed descriptions of human behavior in real-world contexts in which occurs → "thick description"* - *convey sense of what is actually like to be member of particular group or in particular situation → the "lived experience" of research participants*

1) Internal Validity

- two variables being statistically related does not necessarily mean one causes the other → *"Correlation does not equal causation"* - purpose of experiment is to show that two variables are statistically related and to do so in way that supports conclusion that independent variable caused any observed differences in dependent variable → logic is based on assumption that researcher creates two or more highly similar conditions and then manipulates independent variable to produce one difference between them, then any later difference between conditions must have been caused by independent variable - *empirical study is high in internal validity if way it was conducted supports conclusion that independent variable caused any observed differences in dependent variable*; *experiments high in internal validity because way they are conducted (with manipulation of independent variable and control of extraneous variables) provides strong support for causal conclusions* → *non experimental research designs (e.g., correlational designs), in which variables measured but not manipulated by experimenter, are low in internal validity*

Correlational Research

- type of *non-experimental research* in which *researcher measures two variables and assesses statistical relationship (i.e., the correlation) between them with little or no effort to control extraneous variables* - reasons researchers interested in statistical relationships between variables choose correlational study rather than experiment: • *do not believe statistical relationship is causal one or are not interested in causal relationships* → Recall *2 goals of science: describe and predict* • *allows researchers to achieve both goals; strategy can be used to describe strength and direction of relationship between 2 variables and if there is relationship between variables then researchers can use scores on one variable to predict scores on the other (regression)* • statistical relationship of interest is thought to be causal, but *researcher cannot manipulate independent variable because impossible, impractical, or unethical* - correlation used to *establish reliability and validity of measurements* - *independent variable and dependent variable do not apply* to this kind of research - *higher in external validity than experimental research* - typically a *trade off between internal validity and external validity → as greater controls are added to experiments, internal validity is increased but often at the expense of external validity* - typically have *low internal validity because nothing is manipulated or controlled but often have high external validity* - *results more likely to reflect relationships that exist in real world* - *help to provide converging evidence for theory* - *if theory is supported by true experiment that is high in internal validity as well as by correlational study that is high in external validity, then researchers have more confidence in validity of theory*

Practice Effect

- type of carryover effect - *participants perform task better in later condition because had chance to practice*

Experiment

- used to *determine* whether there is *meaningful relationship between 2 variables* and *whether relationship is causal one that is supported by statistical analysis* - one of most common and useful tools for psychological researcher - designed specifically to *answer question of whether there is causal relationship between 2 variables* - *manipulate independent variable by systematically changing its levels and control other variables by holding them constant* - *2 fundamental features:* • researchers *manipulate*, or systematically vary, level of *independent variable (conditions)* • researcher *controls*, or minimizes variability in, *variables other than the independent and dependent variable (extraneous variables)* - if does not involve random assignment → not an experiment

3 important types of nonexperimental research

1) cross-sectional research 2) correlational research 3) observational research

Four Big Validities

1) internal validity 2) external validity 3) construct validity 4) statistical validity

single factor two-level design

independent variables manipulated to create *two conditions* and experiments involving single independent variable with two conditions

Key takeaways for chapter 6

• *Non-experimental research is research that lacks manipulation of independent variable.* • *Two broad types of non-experimental research. 1) Correlational research: focuses on statistical relationships between variables measured but not manipulated, and 2) observational research: participants observed and behavior is recorded without researcher interfering or manipulating variables.* • *Experimental research high in internal validity, correlational research low in internal validity, and quasi-experimental research is in between.* • *Correlational research: measuring 2 variables and assessing relationship between them, with no manipulation of independent variable.* • Correlation does not imply causation. Statistical relationship between 2 variables, X and Y, does not mean that X causes Y. Also possible that Y causes X, or that third variable, Z, causes both X and Y. • *Correlational research cannot be used to establish causal relationships between variables, but does allow researchers to achieve other important objectives (establishing reliability and validity, providing converging evidence, describing relationships and making predictions)* • *Correlation coefficients can range from -1 to +1. Sign indicates direction of relationship between variables and numerical value indicates strength of relationship.* • *Qualitative research is important alternative to quantitative research in psychology. Involves asking broader research questions, collecting more detailed data (e.g., interviews), and using nonstatistical analyses.* • *Researchers conceptualize quantitative and qualitative research as complementary and advocate combining them. For example, qualitative research can be used to generate hypotheses and quantitative research to test them.* • *Several approaches to observational research including naturalistic observation, participant observation, structured observation, case studies, and archival research.* • *Naturalistic observation used to observe people in natural setting, participant observation involves becoming active member of group being observed, structured observation involves coding small number of behaviors in quantitative manner, case studies typically used to collect in-depth information on single individual, and archival research involves analysing existing data.*